Automation complacency
The system is right often enough that the reviewer stops reading. Accuracy created the risk; only interaction design can reduce it.
Practice
Designing for systems that decide.
A deterministic interface tells the user what happened. An agentic one has to tell them how sure it is, what it is about to do, and how to stop it — before it does.
Conventional interface design assumes the system knows what it did. An agentic system does not: it estimates, it acts on that estimate, and it is sometimes wrong in ways that look exactly like being right. Everything a user needs in order to work with that safely — how sure it is, what it is about to do, how to stop it, and how to undo it — has to be put there deliberately.
How much autonomy, decided on purpose
“Should it be an agent?” is the wrong question; “how far up this ladder, for which action?” is the right one. This is our framing — there is no ratified standard for it — and we set the rung per action rather than per product, because a system is usually several of these at once.
-
A0
Assistive — the system shows, the person does
Retrieval, summarisation and ranking. The model changes what the user sees and nothing else. The design problem is provenance: every claim on screen traceable to the source it came from.
-
A1
Suggestive — the system proposes, the person disposes
Drafts, recommendations and pre-filled forms, always landing in an editable state. The user must be able to tell at a glance which parts of what they are looking at were written by a machine.
-
A2
Approval-gated — the system acts once a person says yes
The agent assembles a complete, inspectable plan and stops. This is the rung where most consequential enterprise work belongs, and the rung where design earns its money: an approval nobody reads is worse than no approval at all, because it launders the decision.
-
A3
Act-and-report — the system acts, then tells you what it did
Suitable where actions are reversible and the blast radius is bounded by permission rather than by good intentions. Requires a real undo, a notification the user will actually see, and a hard stop control.
-
A4
Delegated — the system runs a standing objective
Continuous operation inside an explicit budget of time, money, calls and scope. Every exit condition is written down before launch, and exceeding any of them pauses the agent rather than asking it to use its judgement.
Confidence, uncertainty and the honest "I do not know"
A probabilistic answer presented with deterministic confidence is a design defect.
Users calibrate their trust in the first week and then stop checking. If the interface looks equally certain when the system is guessing, that calibration settles in the wrong place and every later safeguard is decorative. We make the model's own uncertainty a first-class element of the layout, and we design the refusal — the state where the honest answer is that there is not enough evidence — as carefully as the success state.
What we deliver
- Confidence expressed in the user's terms, not a bare percentage
- Explicit low-evidence and refusal states, designed rather than defaulted
- Source citation inline, at the claim rather than in a footer
- Visual separation of machine-written content from human-written content
- Calibration testing: does displayed confidence predict actual accuracy?
Approval, override and the way back
Human-in-the-loop is a claim about the interface, not about the org chart.
Oversight fails in a specific and predictable way: the review step exists, the reviewer approves everything, and the audit trail records a human decision that never happened. The fix is design work — show the diff not the output, surface the cases that actually warrant attention, make rejection as cheap as approval, and keep a stop control visible while the agent is running rather than only before it starts.
What we deliver
- Review queues ordered by consequence, not by arrival time
- Diff-first review: what changes, not what the finished thing looks like
- Undo and rollback paths for every reversible action
- A stop control that is reachable mid-run, not only pre-run
- Escalation routes for the cases a reviewer should not decide alone
A trail a person — or an auditor — can read back
If you cannot reconstruct why the system did that, you cannot defend it.
An agent that reads mail, files contracts or provisions infrastructure will eventually do something someone has to answer for. Raw traces are built for engineers debugging at 2am; they are not an explanation. We design the readable layer on top — what the agent was asked, what it looked at, what it decided, what it changed, and who approved it — and we make it reachable from the record it affected rather than from a separate observability tool.
What we deliver
- Decision logs written for the affected user, not for the debugger
- Every action linked from the record it changed
- Retention and export that matches the evidence obligations you carry
- Redaction so a trail can be shown without leaking what it touched
- Agent identity: which agent, which version, acting for whom
The six ways agentic products fail their users
None of these are model problems, and none of them are fixed by a better model.
The confidently wrong answer
Fluency reads as competence. Without a visible uncertainty signal the user has nothing to weigh the answer against.
Approval theatre
A confirmation step that shows the output instead of the change. It produces an audit record of a decision nobody actually made.
Unbounded blast radius
Agent permissions inherited from a human account. Bound the agent by scope and budget, not by the assumption it will behave.
The invisible agent
Work happening on the user's behalf with no indication it is running, no progress, and no way to stop it before it finishes.
No way back
An action that cannot be undone should have been gated for approval. If it is neither reversible nor gated, the design has made a choice by omission.
Building something that acts on its own?
Bring us the riskiest action it will take. That is where the design starts.