We've somehow decided that the universal interface for artificial intelligence is an empty text box. Sometimes there's a little sparkle next to it in case the point wasn't obvious enough. You type whatever is on your mind and a very capable model tries to figure out what you meant, what it can do, and whether those two things have anything to do with each other.
This makes sense for ChatGPT. The conversation is the product. It makes a lot less sense when the chat box is bolted onto scheduling software, accounting software, a CRM, or basically anything else where the product is supposed to understand the job already.
A blank chat box looks flexible to the people building it, but that flexibility comes at the customer's expense. The customer has to understand what the AI can do, come up with a prompt, judge the output, and decide what should happen next. We've basically handed them an API and called it a feature.
I've been thinking about this while building Spaarc, a product for medspas. There are plenty of places where AI could be useful in a medspa, but there are also plenty of places where being wrong is incredibly expensive. A model giving a bad answer about somebody's skin is not made better by the answer being fast, cheap, or written in a particularly empathetic tone.
The more I work through these problems the more convinced I am that the most powerful AI product decision is often invisible. The customer shouldn't have to admire the AI. They should experience a product that remembers what matters, prepares the right information, and makes the work easier.
Products are not demos
Broad AI agents make for wonderful demos. You give one an ambitious goal, it calls some tools, and a few minutes later it produces something that would've taken a person an afternoon. I've built these systems. I use them every day. Watching an agent navigate a complicated problem is still kind of magical even after the novelty has worn off.
A demo, however, is a performance of possibility. A product is the tenth time somebody uses the thing. It's the stale context, the strange edge case, the tool call that takes too long, and the action nobody expected it to take. Customers don't experience a benchmark. They experience whether the product did the thing they hired it to do.
This is where the pursuit of maximum capability starts to feel a little backwards. A more capable model can produce a worse product. It can be slower, more expensive, less predictable, and capable of wandering much farther away from the job. Giving that model more tools makes the demo better, but it also makes the number of surprising things it can do much larger.
The conspicuousness of the AI can be a sign that the product team hasn't finished incorporating it. A general-purpose assistant is often a product team's refusal to decide what the product should do.
The principle of least capable system
In software security we have the principle of least privilege. A person or system should have only the permissions needed to do its job. If a service only needs to read appointments then it doesn't also get permission to delete them. This limits the damage from mistakes and makes the system easier to reason about.
AI products need a similar principle: use the least capable system that can reliably complete the job being delegated.
I'm being intentional with the phrase least capable. It doesn't mean picking a model that is bad at the work. It means that capability is not an end by itself. A smaller model may be better for a narrow classification problem. A larger model may be necessary to make sense of messy dictation. Some problems don't need a model at all. Retrieval, a query, or regular old code may be both more accurate and easier to understand.
There are also three kinds of capability that tend to get mashed together in conversations about AI:
- Model capability is what the model can understand or generate.
- System capability is what the context, code, and tools let the feature accomplish.
- Authority is what the system is allowed to change or communicate.
That third one is the most important to trust. A small model with unrestricted database access can do a lot more damage than the largest model in the world confined to drafting text in a box. Likewise, a frontier model can be used inside a tightly constrained product without giving it broad authority.
Least capable is not necessarily about model size. It's about the whole system and the boundary around it.
People delegate jobs, not autonomy
I started this line of thought by asking whether customers want AI with less autonomy. I don't think that's quite the right question. People have wildly different expectations of autonomy depending on the job.
My spam filter can quietly move something out of my inbox. That's a familiar, reversible action and I can inspect what it did. I would not extend the same trust to an email assistant responding to my friends on my behalf. Both systems work with email, but the social consequence of the second action is entirely different.
What customers want is for the system's authority to match the job they believe they delegated.
This ends up looking a lot like role-based access control. A role is made up of permissions and a job is made up of actions. Each action needs access to particular information, tools, and effects. If we can decompose the job then we can decide what the system actually needs to do for each part.
Take a medspa owner who wants help keeping their schedule full. That job could be broken down into something like this:
- Read an unexpected opening and the service requirements attached to it.
- Find clients who are due for a compatible service.
- Rank those clients using rules the practice understands.
- Draft a message in the practice's voice.
- Present the message to the front desk for approval.
None of this gives the system permission to discount a service, message every client in the database, or change an appointment. It also doesn't give the system permission to dream up why a client hasn't returned. Those are different actions and each one requires a separate trust decision.
The same is true when we talk about autonomy. Observing, ranking, drafting, acting after approval, and acting under an explicit rule are different permissions. They should not be bundled together because somebody put the word "agent" on the feature.
Autonomy is a permission, not a capability.
Retrieve facts and generate drafts
The AI industry has an understandable bias toward generation because generation is impressive. You can watch a blank page fill up with something that looks new. Retrieval is comparatively boring. The record was already there and the software simply put it in the right place.
For a lot of products the boring version is better.
One of the things a practitioner needs before an appointment is the record from the last one. For laser and other energy-based services that includes the device settings and how the client's skin responded. I could have a model summarize those settings, but the source information is already structured and available. Showing the actual recorded values is safer and more useful than asking a model to rephrase them.
The same principle applies to post-care. A practice already has instructions for what a client should do after a treatment. If the product needs to answer a question then it should start with the practice's own instructions for the treatment that client actually received. When the answer isn't there, the useful response is to say so and make it easy to contact the provider.
This is not a failure of the system. Correctly recognizing the edge of its knowledge is part of the job.
Generation still has an important place. A practitioner can dictate fragments and have the system draft a clinical note. The front desk can get a drafted message for a client who is overdue. In both cases a person can quickly compare the result to what they intended, make a correction, and sign or send it.
The distinction I've landed on is pretty simple: retrieve facts, generate drafts, and let people make decisions.
Invisible does not mean secret
There is an obvious objection to calling AI invisible. People should know when AI is being used, especially when their information is involved. I agree.
Invisible isn't the same as undisclosed. It means the product interaction is organized around the job instead of around a performance of intelligence. A practice should be able to see what was proposed, what information grounded it, who approved it, and what was ultimately sent or signed. It should be able to turn the feature off. The underlying workflow should continue to work when it does.
If turning off AI makes the rest of the product fall apart then AI was not incorporated into the workflow. It replaced the workflow. That is a much bigger bet than most product teams admit they're making.
Human review is not a magical answer either. A confirmation box on every action eventually trains people to click approve without reading. Review only works when it is faster to verify the output than to do the work and when the person reviewing it has enough context to recognize a mistake. Specific, repetitive, and reversible actions can earn more autonomy over time. Broad autonomy does not have to be the destination.
This matters even more in healthcare-adjacent products. Cost and latency are important, but they come after correctness and appropriate refusal. In the most sensitive workflows the downside of being wrong dominates every other optimization. The system should aim to be correctly useful or correctly silent, not to always produce an answer.
Build the product first
Before adding AI to a product I think teams should be able to explain the job without mentioning AI at all. What is the customer trying to accomplish? What information already exists? Which parts require judgment? What is the smallest action the system can take that would be useful? What would surprise the customer if it happened without approval?
Most importantly: would we build the underlying workflow if the model disappeared tomorrow?
At Spaarc I want a practice to say that cancellations get filled, practitioners finish charting between appointments, and clients feel looked after between visits. None of those sentences need to mention AI. A practice that turns AI off should still have a great waitlist, charting workflow, and post-care experience. The AI should make those systems more attentive and less laborious. It shouldn't be holding them together.
There will continue to be products where a broad conversation is exactly what the customer wants. There will be jobs where a powerful model with a large collection of tools is the right design. I use products like that constantly and have written plenty about how useful they are.
That doesn't make general-purpose autonomy the end state of every product. Often the more mature decision is to take all of that model capability and turn it into something narrow, dependable, and unsurprising. The business works a little better. The customer feels a little more cared for. The machinery recedes into the product.
The best AI products may be the ones whose customers never have to stop and admire the AI.