Asset managers: Can you prove your AI compliance decisions?

Third-party vendors complicate AI-enabled compliance workflows

AI-assisted compliance workflows can make reconstruction of a decision more challenging because relevant evidence may be spread across multiple systems and providers. Records such as source data, AI-generated outputs, testing and validation activities, controls, investigations and human review may be spread across data sets, systems and teams.

For example, a trade surveillance decision might involve data from a custodian or fund administrator, an alert generated by a surveillance platform and an investigation performed by the asset manager. In an AML or screening workflow, the evidence trail may cross a different combination of firm and vendor systems.

Compliance leaders need to be able to connect those records and explain how the compliance decision was reached.

"Asset managers often have access to vendor documentation and control reports, but the ability to connect data, controls and evidence from both firm and vendor systems can be a challenge,” said Kyle Daddio, Grant Thornton Risk Advisory Partner and AML & Sanctions Practice Leader. “All of those pieces need to be clearly connected to explain how an AI-generated output influenced a specific compliance decision.”

Establish AI-enabled compliance decision ownership

Reconstruction of a decision becomes difficult when responsibility for the supporting evidence is unclear.

Compliance leaders should identify who owns each component of the decision, including:

• The source data
• The AI application or model
• The AI-generated output
• The controls applied to that output
• The investigation and escalation
• Any human override
• The final compliance decision

When a vendor participates in the workflow, the firm should also know which records the vendor maintains, how long those records are retained and whether they can be retrieved for an individual decision.

Five questions to test reconstruction of an AI-enabled decision

Before a regulator, investor or board member requests AI-enabled decision evidence, asset managers should assess their readiness to produce that evidence.

They can start by selecting one closed or representative AI-assisted AML, trade surveillance or other compliance decision. Ideally, this would be a recent decision that involves at least one external provider.

Then, they should attempt to reconstruct it using the following five questions.

1. Can we map the decision from start to finish?

Document how the decision moved through the firm and its vendors, beginning with the source data and ending with the final disposition.

Identify each system, team and provider that contributed data, technology, controls, investigation or review. The objective is to establish a complete decision path that shows how data, AI outputs, controls and human review contributed to the outcome.

A firm should be able to answer:

• Where did the decision begin?
• Which systems and providers influenced it?
• Where did human judgment enter the process?

Who made the final decision?

2. Can we identify and retrieve the AI evidence?

Determine which AI application, model or tool contributed to the outcome and whether the records associated with that use can be retrieved.

Depending on the technology and workflow, relevant evidence may include:

• The model or tool version
• The data or features used
• The output or recommendation generated
• Testing and validation results
• Changes made to the technology
• Records of monitoring or control activity

The evidence should relate directly to the compliance decision under review. General model documentation or a vendor control report may provide context, but it may not explain an individual outcome.

3. Does the testing reflect how AI was used?

Review whether testing reflects the firm's data, workflows and compliance scenarios.

For example, testing should address the types of alerts, exceptions and escalation scenarios the asset manager encounters. It should also show whether the technology performs as intended within the workflow where the firm actually relies on it.

4. Can we show who owned each judgment and control?

Identify who was responsible for reviewing the AI output, applying controls, investigating the result, escalating concerns and making the final decision.

Where vendors participate, document where their responsibilities end and the asset manager's begin. The record should also show whether a human reviewer accepted, challenged or overrode the AI-generated output and the basis for that action.

5. Can we produce a complete and understandable record?

Assemble the evidence as if it had been requested today by an SEC examiner, auditor or board stakeholder.

The record should allow a reviewer to understand:

• Which data informed the outcome
• What the AI generated
• Which controls were performed
• How a human evaluated the output
• Whether the recommendation was accepted or overridden
• Why the firm reached its final decision

The evidence must support a clear explanation of how the decision was reached.

Start with one decision

Completing the exercise helps compliance leaders understand whether they can support and explain an AI-assisted compliance decision with evidence drawn from both firm and vendor systems.

"Asset management compliance leaders do not want to be figuring out how to reconstruct an AI-assisted AML, trade-surveillance or other compliance decision once questions are being asked,” said DJ Rossini, Risk Advisory Partner at Grant Thornton. “A practical first step is testing the evidence, ownership and documentation behind one decision to see whether the firm can clearly explain how it was reached.”