Editorial note: This article appears in Piya Sadana’s contributor archive because it was included in her submitted packet. The article itself credits Emily Osterweil as its author.
By : Emily Osterweil October 6, 2026
If a computer can be trusted to decide the right thing, then people will want to know why it made that decision. Explainable artificial intelligence explores this very question by analyzing how the AI produces information that is both reliable and intelligible.
The concept of explainable AIs involves the idea of providing evidence to users of the system's output.
Explainable AI, according to the National Institute of Standards and Technology (NIST), involves "four principle desiderata for explainable AIs: providing an explanation; providing an explanation that is useful to the intended audience; providing an accurate explanation; and providing an explanation that acknowledges the limitations of the method."
An explanation, in short, should tell the audience what happened in a way that that they understand, while staying true to what is known about the explainer's knowledge.
This concept of an accurate, yet understandable explanation is important for many reasons. Suppose a school uses some kind of software to decide which students qualify for participating in a program to enrich its students' education. Say there was a boy who did not make it into the program and wanted to know why: it's unlikely the computer could just explain its score to the boy.
If the program did not use the right information to calculate its score, this example shows that an explanation might help the user of the software realize this issue.
The above example also shows how explanations are crucial for accountability. The explanation should allow the user to understand why something happened, which means that the user has a greater ability to demand responsibility from those who programmed the software.
NIST makes a distinction between the accuracy of an explanation and the accuracy of an output. Something could be explained thoroughly, even if the explanation is completely wrong. The explanation framework also states that different people require different levels of detail from an explanation. In the example given earlier, a programmer might need more details about the explanation than the student who used the software.
This means that an organization should value explanations just as much as results. In short, one should not confuse an explanation with a justification of the results.
Explainable artificial intelligence makes sure that an automated system is accountable and transparent.
It encourages people to ensure that the system they are using is working correctly while giving them the tools to understand the AI's decisions. As computers are trusted to do more and more tasks, explanations are useful in debugging the process and realizing when a human should be involved instead.