The Delegation Contract: How Consumers Structure Trust in AI Agents
Consumers trust AI agents conditionally, with 77% wanting human approval before an agent acts and only one in ten comfortable with full independence, including financial decisions. For product teams, platforms and regulators, the implication is clear: adoption will depend on bounded autonomy, verified identity, reversible actions and a reliable route back to human accountability.
That conclusion comes from a Blackbox survey commissioned by Sumsub among 1,050 consumers across mainland China, Hong Kong and Taiwan. Respondents were not simply deciding whether to trust AI agents. They were defining a delegation contract: what an agent may do, where approval is required, what information it may retain and who takes responsibility when something goes wrong.
This distinction matters because AI agents do more than advise. They act. A consumer may trust an agent to compare mortgage options but still refuse to let it choose one, accept a product recommendation but insist on approving the payment, or use an agent daily while expecting a human to remain available when something fails.
Trust in AI agents is therefore less useful as a general sentiment score than as a product and governance challenge. The better question is not simply whether consumers trust AI agents, but what they will allow agents to do, under which safeguards and at what level of risk.
Consumer Trust in AI Agents Depends on Human Approval
Only one in ten consumers is fully comfortable with AI agents acting independently, including on financial decisions. That is the hard edge of the trust question.
Most people are not ready to hand over control completely. More than three-quarters, 77%, prefer human approval before an agent acts. Nearly half, 47%, want to approve every financial transaction manually. Another 42% say an agent remembering their details makes them feel less secure.
Source: Blackbox-Sumsub study. Sample total n=1,050. Skeptics n=24. ▲/▼ = significantly higher/lower than Total at 95% confidence. Skeptics (n=24) shown for directional reference only, not significance-tested.
The same caution appears in giving AI agents access to their personal accounts, such as email, shopping and banking. The preferred setting is not full access or no access, but something in between. Some 44% would confine agents to limited actions in specific apps they choose.
That is an important distinction. Consumers are not rejecting AI agents outright. They are negotiating the terms.
The implied contract is simple: act for me, but within limits I set. Ask for my sign-off where the stakes are higher. Do not assume that convenience gives you permission to remember everything. And when something goes wrong, give me a human I can reach.
That last point also matters for experience design. Seven in ten consumers, 71%, rate a human-only escalation channel as important. In other words, people may be willing to let AI agents take on more responsibility, but they still want a clear route back to human accountability in the event something goes wrong.
For agentic AI, trust is not built by removing friction everywhere. It is built by knowing where friction reassures.
Source: Blackbox-Sumsub study. Sample total n=1,050. Skeptics n=24. Ratings on a 1–10 scale. ▲/▼ = significantly higher/lower than Total at 95% confidence. Skeptics (n=24) shown for directional reference only, not significance-tested.
AI Agent Trust Can Survive Mistakes When Failure Is Contained
The most revealing test of trust is not what people say before something goes wrong. It is what they say after it does.
We asked consumers what they would do after a costly agent mistake, such as an AI agent booking a non-refundable hotel on the wrong date. The result is surprisingly nuanced.
Most people would not abandon the agent altogether. Only 14% say they would stop using it entirely. More than half, 55%, would keep using it, but limit it to non-financial tasks. Nearly a third, 31%, would continue using it for all tasks and treat the mistake as user error. Among Early Adopters, that rises to 44%.
That suggests trust in AI agents is not especially brittle, and those who are experimenting with them do so with a degree of self-awareness about the potential risks. As such, trust in AI agents does not simply collapse after a mistake. It adjusts.
Source: Blackbox-Sumsub study. Sample total n=1,050. Skeptics n=24. Ratings on a 1–10 scale. ▲/▼ = significantly higher/lower than Total at 95% confidence. Skeptics (n=24) shown for directional reference only, not significance-tested.
Our survey results suggest that if something goes wrong, many consumers would not move from trust to outright rejection. They move from higher autonomy to lower autonomy. They pull the agent back from money, purchases and irreversible decisions, but keep it in their lives for lower-risk tasks.
This is where the trust story becomes more useful for brands navigating the use of AI agents in their workflows. Trust in agents is not binary. It is elastic. After a failure, it often snaps back to a lower level of autonomy rather than breaking completely.
Experience does not necessarily destroy trust. Poorly bounded experience does.
The issue is not whether AI agents will make mistakes. They will. The issue is whether the user feels protected when they do. People can forgive an agent that gets something wrong inside a clear system of limits, approvals, reversals and escalation. They are much less likely to forgive one that acts beyond the boundaries they thought were in place.
For agentic AI, trust is not earned by promising perfection. It is earned by making failure survivable.
Unclear Accountability Is Holding Back Trust in AI Agents
The hardest part of agentic AI may not be whether the technology works. It may be what happens when it works badly.
Ask consumers who should be responsible when an AI agent makes a financial mistake and no single answer wins. Some 38% say responsibility should be shared between all parties. Another 28% point to the company that built the AI, 20% to the user, and 14% to the platform where the transaction happened.
That spread matters. It shows that consumers do not yet see a clear owner of the failure case.
The same uncertainty appears in consumer protection. Only 39% trust existing rules to cover them if an agent makes a mistake, falling to 26% among the Late Majority. In other words, many people are not sure the current system knows where responsibility sits.
This is why the approval step matters so much. It is not just a preference for control. It is insurance.
When people are unsure who will protect them after a mistake, they hold on more tightly before the mistake can happen.
Asked what would raise their confidence, consumers do not point to a single magic fix. They want a stack of protections: platform guarantees to cover losses, extra security checks for sensitive actions, stronger regulation, the ability to set limits or switch the agent off, and insurance for AI-related losses.
That tells us something important. Trust infrastructure will not be one feature. It will be a set of reinforcing safeguards.
The clearest single signal is verification. Some 78% say a Verified AI Agent feature would increase their trust, and 77% say agents should identify themselves to platforms.
This is where the consumer view and the industry view start to meet. Sumsub, who originally commissioned the survey, argue that the goal should not be to block automation, but to control it proportionately: let legitimate automation move with minimal friction, while making bad automation traceable and stoppable.
That framing is useful. Automation is not automatically suspicious. Anonymity is the bigger problem.
Businesses want to know that when an agent acts, a real, verified individual is answerable for it. The consumer data suggests customers are asking for the same thing from the other side of the counter.
The Winning AI Agents Will Make Autonomy Visible and Reversible
Consumer trust in AI agents will grow through controlled delegation rather than unrestricted autonomy. The products most likely to reach the Early Majority will allow users to define permissions, approve high-stakes actions, set financial limits, review activity, verify who or what is acting, reverse mistakes and escalate to a human.
For product teams, this means treating trust as part of the product infrastructure rather than a promise made through branding. For platforms, it means making agent identity and activity traceable. For regulators, it means establishing protections that tell consumers who is responsible when delegated actions cause financial or personal harm.
Some friction will therefore remain essential. Approval screens, security checks and escalation routes may add steps, but they also give consumers the confidence to delegate more consequential tasks.
The next phase of AI agent adoption will belong to organisations that make autonomy understandable, controllable and survivable when something goes wrong. The agent can act, but it cannot disappear.
This is the second article in a three-part series on consumer relationships with Agentic AI.
Click here to read part one.
The final piece in this series turns from trust to the marketplace: what agent-mediated buying means for brands, and how they stay chosen when a consumer's instructions — not their habits — decide the purchase.
Ready to Make Agentic AI Work for You?
Reach out to Blackbox for a conversation on how our research solutions can help you build, market or regulate agentic products more effectively.