Microsoft researchers connect AI brain explanations to experiments

Microsoft Research and university collaborators announced work on generative causal testing on June 25. The research overview describes a method for turning explanations of language-related brain responses into new experiments.

Explanations face a follow-up test

The collaboration includes the University of California, Berkeley, the University of California, San Francisco, and Columbia University. The framework summarizes what a brain-prediction model suggests a particular cortical area responds to, then uses generated stories to test that explanation with participants in a scanner.

Microsoft reported that the associated paper had been accepted in Nature Neuroscience. The result is a research method and a set of reported experiments, not a general system for reading a person’s thoughts.

Separate a story from a prediction

For engineering readers, our main takeaway is methodological: a readable explanation becomes more useful when it implies an observable result that could turn out to be wrong. A fluent description of a model’s behavior is not, on its own, a validation of that description.

When building explainability features for another domain, ask what independent test would distinguish two plausible explanations. Specify the expected observation before running the test, and preserve failed predictions rather than rewriting the explanation afterward to fit the outcome.

Avoid a circular evaluation

A useful evaluation design should make clear which model proposed the explanation, which process constructed the test input and which measurement judged the result. If the same generated narrative supplies all three roles without an independent observation, the explanation may be persuasive without being informative.

This research is interesting because it places an experimental measurement after the model-generated hypothesis. Applying that lesson elsewhere requires an appropriate measurement for the domain, rather than copying neuroscience terminology into a software dashboard.