1. What Is the Legal Duty of Explainability?

The legal duty of explainability is the emerging requirement that AI-driven decisions must be understandable to the people they affect, not just to engineers, data scientists, or regulators.

In practice, this duty means an organisation that relies on AI should be able to explain:

• What decision was made about the person.
• Why that decision was made in this specific case.
• Which data and factors mattered most to the outcome.
• What options the person has now (appeal, correction, human review).

This duty connects directly to broader legal principles: transparency, due process, non-discrimination, and the right to challenge harmful or incorrect decisions.

2. Where Does This Duty Come From in Law and Policy?

There is no single global “Explainability Act”. Instead, the duty of explainability grows out of several overlapping legal and regulatory frameworks.

2.1 Data Protection and Automated Decisions

• GDPR / UK GDPR (Article 22 and Recital 71): these rules limit solely automated decisions that produce legal or similarly significant effects, and support rights to meaningful information about the logic involved, especially in credit, employment, and benefits decisions.
• Information rights: data protection law often gives people the right to learn how their data is used, including profiling and automated decision-making logic in important cases.

2.2 AI-Specific Regulation

• The EU AI Act imposes transparency and documentation duties on high-risk AI systems (for example in credit scoring, hiring, and access to essential services). Providers and deployers must be able to explain system characteristics, limitations, and how to interpret outputs.
• Sector regulators (financial supervisors, consumer protection agencies, equality bodies) increasingly expect firms to explain AI-based decisions in their domain, not only to regulators but to affected individuals.

2.3 Regulatory Guidance on Explaining AI

• Guidance from authorities like the UK Information Commissioner’s Office (ICO) and The Alan Turing Institute on “Explaining decisions made with AI” gives practical steps for building explanations into AI systems from the start.
• Policy papers and law reform proposals talk explicitly about a “duty of explainability” as a way to make AI decisions contestable and accountable, not just accurate.

The message is clear: if you use AI to make important decisions about people, you must be ready to show your working in language they can actually use.

3. Why Plain-Language AI Explanations Matter for Real People

For a person on the receiving end, the technical shape of your model is less important than the clarity of your explanation. A decision that cannot be explained in plain language is almost impossible to challenge.

Vague lines like “you did not meet our criteria” or ultra-technical phrases like “the model score fell below threshold 0.61” do nothing to protect rights. They also make it easier for plaintiff-side attorneys to argue that a system was unfair or structurally biased, particularly in fields you already cover such as personal injury and compensation and employment law disputes.

4. Rights Snapshot: What People Expect When AI Decides for Them

The details differ between jurisdictions, but individuals are converging on a very simple set of expectations whenever a machine makes an important decision about them.

When those elements are missing, “we can’t tell you why” quickly becomes a liability theme, similar to trust breakdowns you discuss in client trust in AI-driven legal counsel and the ethics of digital-era attorney–client confidentiality.

5. The Explainability Stack: Three Layers You Must Get Right

Explainability is easier to manage if you think about it as a stack. Each layer supports the next one, from deep technical detail to the final human-facing message.

5.1 Model-Level Explainability

• List the features your model uses and why each was chosen.
• Document training data sources, known limitations, and bias risks.
• Maintain tools for technical explanations: feature importance, example-based reasoning, or local explanation methods that help engineers understand individual decisions.

5.2 Case-Level Explainability

• Store a decision trace: key inputs, model scores, thresholds, and any human overrides.
• Capture which factors mattered most in this particular case (e.g., “recent delinquencies” and “high credit utilisation”).
• Keep logs long enough to reconstruct decisions when a complaint arises months or years later.

5.3 Human-Level Explainability

• Translate internal logic into short, concrete sentences that people can act on.
• Avoid technical labels (“segment 4C, score 0.54”) unless you also explain them in plain language.
• Align tone with broader fairness and accountability goals, similar to how you frame AI and justice in litigation in the age of machines and predictive justice.

6. A Simple Template for Plain-Language AI Explanations

Many organisations struggle because they improvise explanations in every case. A reusable template makes explanations both clearer and more defensible.

Example: Loan Application Declined

Decision: “We’re unable to approve your loan application today.”
Main reasons: “Our system identified several recent late payments and a high balance on your existing credit accounts. These factors suggest a higher risk of missed repayments.”
Data source: “We based this decision on your application form and your credit file from the credit bureau listed below.”
Impact: “This means we cannot offer you this loan at this time.”
Next steps: “You can review your credit report for errors, reduce your existing balances, and apply again in the future. If you believe we made a mistake, you can ask us to review the decision manually using the contact details below.”

This level of clarity gives the individual something concrete they can question, correct, or work on, instead of leaving them with a generic rejection message.

7. Pre-Launch Checklist: Are You Ready to Explain Your AI?

Before a new AI system goes live, treat explainability as a launch gate, not an optional extra. You should be able to answer “yes” to questions like:

• Have we identified which decisions are high-impact for people’s rights or finances?
• Can we generate a clear, user-facing explanation for every high-impact decision?
• Do we have a standard explanation template that non-lawyers can understand?
• Is there a simple way for people to request human review or appeal?
• Do our privacy notices and consent flows accurately describe this AI use?
• Have we tested explanations with real users, not only internal staff?

These questions mirror the broader governance issues you already explore in AI-and-law pieces like reprogramming justice and attorney–AI integration.

8. Evidence Kit: What You Need When Explainability Is Challenged

When someone challenges an AI decision, the conversation quickly moves from “what the model usually does” to “what happened in this specific case.” At that moment, logs and documentation become your first line of defence.

If these elements are missing, “we can’t reconstruct what the system did” becomes part of the legal risk — and part of the narrative in any complaint or lawsuit.

9. Governance: Who Owns Explainability Inside the Organisation?

Explainability often fails not because teams disagree with the idea, but because nobody clearly owns it. Responsibilities are spread across legal, compliance, product, and data science — and gaps appear in between.

A stronger governance model usually:

• Assigns a named owner for AI explainability and automated decision-making policy.
• Creates a cross-functional review group for high-risk use cases (credit, hiring, insurance, benefits, access to essential services).
• Requires UX and product teams to include appeal and explanation flows in every high-impact journey, not after launch.
• Trains frontline staff so that they can talk about AI decisions consistently with written explanations.

This kind of structure makes it far easier to defend AI use in court or in front of regulators, because explainability is not improvised, it is part of the operating model.

10. The Future: Explainability as a Baseline, Not a Bonus

The legal duty of explainability is still evolving, but the trend is clear: opaque AI is becoming harder to justify when it affects people’s rights and livelihoods. Regulators are less interested in being impressed by complex models and more interested in whether ordinary people can understand and challenge important decisions.

Organisations that take explainability seriously today position themselves better for tomorrow’s regulatory environment and litigation landscape. They reduce legal risk, strengthen trust, and make it easier to integrate AI into sensitive domains like finance, employment, and justice.

In short, explainability should not be treated as a favour to users. It is rapidly becoming a baseline legal expectation: if a machine decides something important about you, you deserve to receive reasons you can read, question, and act on.

Disclaimer: This article is for general informational purposes only and does not constitute legal advice. Organisations should consult qualified counsel for advice tailored to their specific AI use and applicable laws.