What deleting folders taught me about AI assistants
Chat is a terminal. Sometimes you just want to point.
There are two ways to delete a folder on a Mac.
In Finder, you click the folder, then press Delete. Object first, action second. See how it’s done:
In the terminal, it’s:
Action first, then the object, and you have to produce the object’s name. Or press Tab on Mac.
It’s the same result but using the opposite grammar. Let me reiterate again:
In UI: Object first, action second.
In terminal: Action first, then the object.
The grammatical flip helps explain why talking to AI assistants can feel tiring, even when the AI itself is brilliant.
This isn’t a new idea. The two grammars has been around for quite a while now.
Direct Manipulation
Ben Shneiderman coined the term "direct manipulation" for the UI back in 1983. The objects stay continuously visible. You act on them with clicks and drags instead of syntax. And every action gives immediate, reversible feedback.
Command-language
Command-language interaction is for terminal interaction. The user describes what they want in the system’s language.
Distinction between the two
Hutchins, Hollan and Norman discussed the distinction in their paper. A command language is a conversation about the work: you talk to an intermediary and describe your intent. Direct manipulation is a *model world*: there’s no intermediary, you reach in and touch the thing. Their point wasn’t that conversation is bad; it’s that conversation makes you translate intent into description, and translation costs effort every single time.
What was the cost?
Jef Raskin later boiled it down to grammar in The Humane Interface: noun-verb beats verb-noun.
Here’s why:
Pick the thing first, and it just sits there while you decide what to do with it. Like clicking a folder, then hitting delete.
Pick the action first, and the system waits for you to finish what you started. Like typing
rm -rand then having to remember the folder name.s.
For twenty years, the noun-verb Direct Manipulation / UI side mostly won. Then chatbots arrived.
Chat moved us back to the terminal
What I noticed was a chat assistant has the terminal’s grammar, not the UI’s.
Every message you type is verb-noun.
“Explain...” then the noun.
“Fix...” then the noun.
“Summarise...” then the noun.
And the noun is a description you compose by hand.
Think about what actually happens when something on screen confuses you. The honest question is “what does this mean?”, one word of which is a pointing gesture.
But the assistant can’t see your finger, so the question has to be typed in:
which application you’re in
which page you’re on, which element you mean
what it currently says.
You end up writing a little essay to explain what you want.
Linguists have measured why this feels so heavy. Clark and Brennan’s paper shows that the hard part of communication is getting both parties to focus on the same thing. Pointing at something you can both see is the cheapest way to do this; describing it in words is close to the most expensive
It actually, it already exists in 1980 through the Bolt’s Put-That-There demo:
See the 1:09. I literally can empathize with
the pain when there is a bug on a demo 😅
If that exists, why are we back at this?
Don Norman actually warned us about this in 2010. The ”natural language interfaces” would reimport exactly the problems command languages had.
And is surely did, Luger and Sellen found that people experience conversational agents as being “like having a really bad Personal Assistant.” An ‘smart’ assistant that you have to brief every detail to get things right!
None of this means chat is a mistake. It means chat is a terminal: Great for things you can name faster than you can find and slow for things you can see.
Letting people point at the screen
Can an AI assistant get a noun-verb mode? Could you select the object first, UI-style, and then ask?
I built exactly that into GLAMLI, the guided machine-learning app I’m developing for my Master’s research. The app has an assistant panel docked beside the main workspace, and next to the message box there’s a toggle called click and ask. Switch it on and the whole page becomes selectable. Here’s a demo:
Here’s how did it:
The click has to be intercepted before the page reacts to it, because the most confusing elements are usually interactive themselves; without that, asking about a tile would also open the tile.
Second, the generated question is seeded into the composer rather than sent automatically. The click supplies the noun; you still choose the verb, edit the wording, or delete it entirely. The selection stays visible and editable, which is precisely Raskin’s argument for noun-first grammar.
And this isn’t just my hunch about what would help. At DirectGPT tested the idea. An LLM interface where you select objects directly instead of describing them. Participants were roughly 50% faster and used 50% fewer and 72% shorter prompts than with baseline ChatGPT. The entire saving is the referring expression, the essay that substitutes for a point.
Conclusion
I’m not saying that “pointing beats talking.” But having the option to do chat-only AI forces the terminal’s grammar even in the cases where the UI would be cheaper.
A better approach would be to ask yourself if youar building an AI assistant in your product: is the user naming this thing because language is the right tool, or because you never gave them anything to click?