How to check an AI answer in 60 seconds

Image

Julien Gruhier, head of AI at Croner Intelligence explains how to verify AI output and avoid wasting time by re-researching the answers

Every sensible AI policy says a qualified person must review the output but almost none of them say what reviewing actually involves.

Without guidance on the review process, it risks becoming a glance at something that reads well, or full re-research that removes the reason for asking the AI in the first place. A structured check sits between the two but can still take under a minute.

Start with the source

Citations naturally add authority. So start by checking that the cited document supports the claim being made.

One common failure is not an invented reference, but a real one attached to the wrong proposition – right legislation, wrong subsection. It is worth identifying the key sentences, checking the sources that support it, and checking once more that the source supports the position in the AI answer.

Check the date before the detail

Some tax answers age badly. Rates, thresholds and allowances, and complex tax rules are where a confident answer may turn out to be out of date.

It is worth confirming which tax year an answer applies to, including whether the guidance cited is current and has not been superseded. If the answer does not state a year, that omission in itself may be a potential problem.

Confirm the jurisdiction

Devolution has resulted in different rates and taxes. Income tax rates and bands differ in Scotland. Stamp duty land tax is for England, Scotland and Wales have their own land transaction tax frameworks. In the same way there are other devolved taxes with different thresholds, particularly when it comes to income tax rates and thresholds in Scotland.  

An answer that states ‘in the UK’ without qualification may warrant a second look if the client is not based in England, but rather elsewhere in the UK.

Ensure each statement is defensible

Sources tend to attach to factual statements. One example is, ‘so the payment therefore falls within…’. With this kind of statement the vagueness should raise a red flag. So scan for the connecting sentence and ask for the correct link to support the answer.

This routine runs to about a minute. This works for Croner Intelligence because it is built to be checked. The answers are grounded in retrieved material, with citations that state a tax year. That is the point of designing a system such as Croner Intelligence around its sources. The value is not that an answer arrives quickly, but that it can be confirmed quickly.

If verifying an answer takes longer than researching it would have, the tool will not have saved you time. Croner Intelligence does save you time, because everything is there, ready to be checked.

*** Find out more about Croner Intelligence, the game changing AI tool for accountants and tax professionals. Book a demo here ***

Julien Gruhier | Director of search and generative AI, Croner

Julien Gruhier is director of search and generative AI at Croner, and is the driving force behind ...

View profile and articles

0
Be the first to vote

Rate this article

Related Articles
Subscribe