Complex AI models can be explained by decomposing them into subspaces

Evidence

But this is a failure to explain, not a problem of explainability. Lakkaraju and colleagues (2019) have shown that, even with extremely complex models, there are methods of dividing the models into smaller subspaces which can be explained and, in turn, allow for intelligible explanation of overall model workings. — Joseph Avery, Jim Sidanius, Joel Cooper, Gregory Davis, Amanda N., Bias in the Law, 1200

In State v. Loomis (2016), the Wisconsin Supreme Court case discussed above, Justice Abrahamson wrote in a concurrence, “[T]his court’s lack of understanding of COMPAS was a significant problem… . The court needed all the help it could get” (p. 774). But this is a failure to explain, not a problem of explainability. Lakkaraju and colleagues (2019) have shown that, even with extremely complex models, there are methods of dividing the models into smaller subspaces which can be explained and, in turn, allow for intelligible explanation of overall model workings. — Joseph Avery, Jim Sidanius, Joel Cooper, Gregory Davis, Amanda N., Bias in the Law, 1200

Categorization

• QUESTION: What are the actual methods for dividing complex models into explainable subspaces? Worth chasing down the Lakkaraju 2019 paper.
• XREF: Speaks directly to the explainable AI / black box debate in the social-control-of-trust literature — transparency substitutes for trust.
• corroborated: 3 independent passages support this claim — Joseph Avery, Jim Sidanius, Joel Cooper, Gregory Davis, Amanda N., Bias in the Law: A Definitive Look at…, 1200