There is an interesting question emerging as AI assurance develops: what does independent actually mean?
Ashley Casovan, Managing Director of the IAPPs AI Governance Center, wrote about this in a piece on AI auditing and independent evaluation. She makes the point that using an external evaluator does not, by itself, make an evaluation credible or independent.
Among the questions she highlights are who controls the methodology, access, funding and publication.
For boards, I think there is another question worth asking. What relationship does the evaluator have with the thing being assessed?
There may be perfectly good reasons to use external expertise to help develop an AI strategy, design governance arrangements or implement controls. The issue comes later, if the same provider is then asked to independently assess whether those arrangements are adequate or working as intended.
That is a familiar assurance problem. The independence of the assessment depends on more than whether the evaluator sits outside the organisation.
As AI audit, evaluation and assurance services become more common, boards will need to understand exactly what they are being asked to rely on. Who did the work, what was assessed, against what criteria, using what evidence, and what prior involvement did the evaluator have?
Calling a review independent doesn’t make it so.
The IAPP piece is worth reading.