Explainable and Reliable Artificial Intelligence (ERAI)

Recent high-profile successes in Machine Learning have mostly been achieved using deep neural networks, which yield ‘black-box’ input-output mappings that can be challenging to explain to users. Especially in the medical, military and legal fields, black-box machine-learning techniques are unacceptable, since decisions may have a profound impact on peoples’ lives. As reported in recent news, AI may amplify the biases that already pervade our society without strict ethical standards, unbiased data collection, and algorithmic bias considerations.

Research within the ERAI theme Investigates different ways to make intelligent systems better explainable and more reliable. Some of the research foci are:

• Analyzing whether the input-output relations of a system can provide high-level correlation-based explanations of the black-box AI system.
• Logic-based systems can provide explanations and are able to reason with (legal) regulations that should be adhered. Integrating logic-based approaches with machine learning approaches is one possible way to realize explainable artificial intelligence, and is an important challenge for the near future.
• Learning explainable models instead of learning a mapping may improve explainability with minor or no decrease in the quality of predictions/decisions/recommendations.
• Improving the understanding of deep neural networks; especially with respect to the influence of input and intermediate layers on the outputs, information flow through the network, detecting changes that can make the system collapse and determining the tolerance of the system.
• Using machine learning and causal inference to explain and understand the relationship between variables in high-dimensional and complex data.
• Developing model-checking tools for properties of safety-critical engineering systems and medical interventions.
• Making reliable predictions with confidence bounds on the error.