We Measured User Trust in an AI Feature, the Results Changed the UI

An AI-generated financial summary feature had healthy usage numbers — people opened it, scrolled through it, spent real time on the page. What those numbers didn't show was whether anyone actually trusted the summary enough to act on it without double-checking it elsewhere, so we ran a small research study specifically to measure that.

What we actually found

In follow-up interviews, a striking number of users admitted they opened the AI summary, then separately pulled up the raw transaction data to verify the same numbers manually before trusting the conclusion. The feature was being used as a starting point for their own manual analysis, not as a replacement for it — which meant it was adding a step to their workflow, not removing one.

High engagement with an AI feature can mean people are using it, or it can mean people don't trust it and are reading it twice to be sure. Those look identical in a usage dashboard.

What we changed

• Added explicit source citations inline in the summary — a sentence about 'spending increased 12%' now links directly to the underlying transactions, so verification happens in-place instead of in a separate tab.
• Added a visible confidence indicator on figures derived from incomplete data, since one source of distrust was users who'd once caught the summary being wrong on an edge case and generalized that distrust to everything.
• Reframed the copy from authoritative statements to clearly labeled estimates where the underlying data had any ambiguity.

Follow-up research after the changes showed a meaningful drop in the 'verify elsewhere' behavior. The lesson that stuck with me: usage metrics tell you people are interacting with a feature, not that the feature is doing its job. Trust has to be measured on its own terms, usually by actually asking people, because it doesn't reliably show up as a side effect of anything else you're tracking.