AI Output Control: Silent Errors in Professional Firms

Why AI makes silent mistakes — and why you don't notice them

A human error usually leaves traces: a hesitation, a handwritten correction, a file left open for a long time. An AI system's error does not. The output arrives complete, well written, consistent in tone. It may contain a wrong number, an inaccurate regulatory reference or a conclusion unsupported by the input data — and none of the visual cues we rely on to identify human errors are present.

The phenomenon has been measured with precision by Microsoft's ThinkingBox benchmark, which tested AI agents on 507 real-world workflows, each repeated 20 times. The finding is instructive: a significant share of failed attempts ended without any visible error. The agent completed its operations, responded to the user, and used its tools in an apparently correct manner. Yet the final state in the database was wrong — incorrect fields, missing effects, actions not actually carried out.

For a professional firm, this means an AI agent may appear to have drafted a document, sent a response or updated a deadline — when in fact it has not, or has done so incorrectly. The gap between 'the agent says it has finished' and 'the work is genuinely complete and correct' is precisely what no improvised control system can bridge.

Who is liable if an AI-generated document contains an error?

The short answer: the practitioner who signed it, sent it, or relied on it. AI has no legal personality, no professional liability, and is not registered with any professional body. The Italian Civil Code, professional conduct rules, and — in tax contexts — the risk of penalties all fall entirely on whoever used the tool.

The issue came to the fore in the debate surrounding tax rulings drafted with AI assistance: an error in that context is not merely a typo — it is a public act with legal consequences. The same principle applies to any document a professional firm produces for a client: an expert report, a tax return, a legal opinion, a contract draft. If AI contributed to drafting it, liability does not transfer — it multiplies, because the practitioner must account for both the content and the choice of tool.

This does not mean AI should not be used: it means that using it without a documented review process is a concrete professional risk, not merely a quality concern. We explored this in detail in "Verifying AI output in professional firms: who signs off?"

Important: Using AI to produce a document does not transfer professional liability to the tool. If the output contains an error and the practitioner sent it without checking, the liability is theirs — both towards the client and under professional conduct rules. The review process is not optional: it is an integral part of using AI correctly in a professional firm.

How many resources does it take to check AI without eroding the savings?

This is the question every firm principal asks, and the honest answer is: it depends on what you have decided to check. The problem is not the total time spent on review, but the absence of a clear criterion for deciding which outputs deserve thorough scrutiny and which can be approved quickly.

Without an explicit criterion, review becomes random: you check whatever you see first, or whatever seems most important, or — worse — you trust the output because it looks well written. The result is that time is spent on low-risk outputs while something critical slips through.

The starting point is a classification of tasks by risk level: not every AI output carries the same consequences if it is wrong. A draft courtesy email is a very different matter from a voluntary disclosure calculation or a response to a tax ruling request. Once tasks are classified, you can define which type of review to apply to each — and that is a strategic decision, not a technical one. See also "AI ROI: how to measure it properly" to understand how to quantify the real net gain from automation.

Which firm activities are most exposed to undetected AI errors?

Not all AI outputs are equally likely to contain silent errors, and not all carry the same consequences if an error goes unnoticed. Mapping high-risk activities is the first step towards building a sustainable control process.

The most exposed activities are those where AI must combine numerical data with up-to-date regulation: tax calculations, voluntary disclosures, VAT settlements, pension projections. Here, a decimal error or a reference to a repealed provision can have direct financial consequences for the client. Close behind are activities where AI must interpret an input document to produce an output — for example, extracting data from an invoice to complete a return: if the extraction is wrong, the error propagates downstream without anyone noticing.

The third group is that of multi-step AI agents: as the ThinkingBox data shows, the point of failure is usually not in the reasoning but in the handling of intermediate tools. An agent tasked with retrieving a document, extracting its data, updating the management system and sending a reply to the client can fail silently at any step — and the final result will still appear to have been completed.

High-risk activities for undetected AI errors in professional firms

• ✕
  Tax and pension calculationsAI combines regulation and numerical data: an input error or an outdated regulatory reference is carried directly into the output.
• ✕
  Data extraction from documents (OCR + interpretation)If extraction is partial or incorrect, the error propagates to every subsequent step without any warning.
• ✕
  Automated replies to clients on technical mattersAI may respond confidently on topics where the rules have changed or the client's specific context has not been understood.
• ✕
  Multi-step workflows with AI agentsEach step adds a potential point of silent failure: the agent may appear to have completed the workflow without having done so correctly.
• ✕
  Automatic updates to deadlines and CRMData written in the wrong place — or not written at all — goes unnoticed until the deadline is missed.
• ✕
  Call and meeting summaries converted into actionsIf the transcript is inaccurate or the AI misreads a verbal instruction, the resulting action may be wrong from the outset.

How to build a measurable verification process without grinding the firm to a halt

A verification process need not mean slowdown: it should mean predictability. The difference between a firm that grinds to a halt checking everything and one that works well lies in the design of the process, not in the number of hours dedicated to it.

The fundamental principle is to separate the moment AI produces from the moment a human approves. These two moments need not coincide, but both must be logged: who approved, when, and on which version of the output. Without a log, the review exists but cannot be demonstrated — and in the event of a dispute, 'cannot be demonstrated' is equivalent to 'never happened'.

Another key principle, which also emerges from the ThinkingBox data, is that verifying the final state is more reliable than verifying the process. An AI agent can make all the right calls to its tools and still arrive at a wrong result. Control must fall on the concrete output — the document, the data in the management system, the reply sent — not on the sequence of operations the AI claims to have performed.

The problem isn't the AI: it's the cognitive overload of the people who should be checking

There is a paradox at the heart of this issue: AI is adopted to reduce workload, but the volume of output it generates risks increasing the cognitive load on those who must verify it. If a member of staff has to review fifty documents a day instead of twenty, the average attention per document falls — and the errors that slip through multiply.

This phenomenon — which we might call review saturation — is the hidden side of mass AI adoption without a designed control process. It is not a technology problem: it is an organisational one. And it can only be resolved by deliberately deciding what AI may do autonomously, what requires quick sign-off, and what must be read with full attention.

The Purpose in Action Observatory research shows that many Italian businesses use AI primarily to cut costs. That is a legitimate objective, but pursuing it without measuring output quality is like cutting production costs without monitoring returns: the apparent saving becomes a real cost further down the line. Managing emails with AI-drafted replies ready for approval — as Ovia Inbox does — is a concrete example of how you can reduce manual effort while keeping a structured human approval point before sending.

Important: If your firm has AI producing outputs that are sent or filed without anyone having read them in full at least once, you already have a control problem — even if nothing has surfaced yet. Silent errors do not disappear on their own: they accumulate until someone finds them, and that someone is often the client.