Can ‘Explainable AI’ provide a way to apply DO-178C for the safety assurance of neural networks in aviation applications? We think not, but there are a few subtle misconceptions to be unwrapped.
However, there does exist a different approach for ensuring the reliable performance of machine-learned (‘AI’) components for a new generation of flight control instruments: demonstrating Machine Learning Generalization and quantifying so-called “Domain Gaps”.
We unwrapped this complicated question earlier in this blog post. This 8-minute video provides a synopsis of the concepts for anyone who has a basic understanding of high school-level math.
00:19 How DO-178 works for the classical software systems in aviation 01:00 What is that 'AI' you speak of 03:44 Certainty and uncertainty 04:56 Traceability: Beware of what you wish for 06:16 How to design a system with a component working with uncertainty 06:56 Will it work 'in vivo' as well as it worked 'in vitro'?