Demystifying Machine-Learning Systems | Understanding Neural Networks with Natural Language Descriptions

Key takeaways

• MILAN describes in plain language what individual neurons in a vision model respond to.
• It finds image regions that activate a neuron and then generates a text label for them.
• It works on vision networks, so do not assume it explains language models.

Neural networks are advanced machine-learning models that can perform tasks like image classification, speech recognition, and natural language processing with remarkable accuracy. However, these models are often called "black boxes" because their internal processes are difficult to understand, even for the researchers who design them. This lack of transparency becomes a concern when neural networks are used in critical applications like medical diagnosis or autonomous vehicles.

To address this issue, researchers from MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL) have developed MILAN (Mutual-Information Guided Linguistic Annotation of Neurons). This new method can automatically generate natural language descriptions of what individual neurons in a neural network do. This innovation allows researchers to analyze, audit, and even edit neural networks more effectively.

Why Is Understanding Neural Networks Important?

When neural networks are used in practical applications, it matters to understand how they make decisions. This understanding can help:

1. Identify Bias: Detect and correct unwanted patterns or discriminatory behavior in the model.

2. Improve Accuracy: Identify and fix neurons responsible for incorrect predictions.

3. Enhance Transparency: Provide clear, understandable explanations of the model’s decisions.

4. Increase Trust: Help users trust AI systems by offering insights into their inner workings.

In fields like healthcare, finance, and autonomous driving, understanding how an AI model works can prevent errors and ensure ethical use.

How MILAN Works

MILAN is designed to describe the functions of individual neurons in neural networks trained for computer vision tasks, such as object recognition and image synthesis. It works in three main steps:

1. Observing Neuron Behavior

MILAN first examines how a particular neuron behaves across thousands of images. It identifies the specific image regions that activate the neuron most strongly.

For example, in a neural network trained to recognize animals, MILAN might find that a particular neuron is highly active when it detects fox ears.

2. Generating Descriptions

MILAN uses a mathematical approach called pointwise mutual information (PMI) to find the most informative and specific descriptions for each neuron. This method ensures that the descriptions are accurate and reflect what the neuron is actually doing.

For instance, rather than simply describing a neuron as detecting "dogs," MILAN can generate a more detailed description like "the left side of ears on German shepherds."

3. Describing All Neurons

Unlike previous methods, which required researchers to provide a predefined list of concepts, MILAN automatically describes every neuron in a network. This is essential for large neural networks that may contain hundreds of thousands of neurons.

What Are the Applications of MILAN?

MILAN is a practical tool with several important applications:

1. Analyzing Neural Networks

MILAN can identify which neurons are most important to a model’s decision-making process. By describing and sorting neurons, researchers can determine how the model understands different features.

For example, in a model trained to recognize household objects, MILAN can reveal whether specific neurons focus on identifying handles of mugs or corners of tables.

2. Auditing for Unexpected Behavior

MILAN can be used to check whether a model has learned unintended patterns or biases. This matters for ensuring fairness and transparency in AI systems.

In one experiment, researchers applied MILAN to audit a neural network trained on blurred human faces. Surprisingly, the model still had neurons that recognized and processed facial features, revealing a potential privacy risk.

3. Editing and Improving Models

MILAN allows researchers to find and deactivate neurons responsible for incorrect or biased outputs, improving the overall performance of the model.

In one case, removing neurons associated with incorrect correlations led to a 5 percent increase in the model’s accuracy on problematic inputs.

Example of MILAN in Action

Consider a neural network trained to classify objects in images. MILAN can describe the specific functions of individual neurons. Examples of these descriptions include:

• "Detects the top boundary of horizontal objects"

• "Recognizes the right side of wheels on bicycles"

• "Identifies fur patterns on fox tails"

If the model makes an incorrect prediction, MILAN helps identify which neuron is responsible, allowing researchers to make targeted adjustments.

What Are the Limitations of MILAN?

While MILAN is a significant advancement, it still has some limitations:

1. Vague Descriptions: For complex neuron behaviors, the generated descriptions may be too general.

2. Incorrect Guesses: When a neuron focuses on a concept that MILAN does not recognize, the description may be inaccurate.

3. Limited Scope: Currently, MILAN is focused on computer vision tasks. It has not yet been adapted to other domains, such as speech recognition or language processing.

Researchers are working to improve the system by making descriptions more accurate and extending its capabilities to other types of neural networks.

The Future of Explainable AI with MILAN

MILAN represents a bottom-up approach to understanding AI systems. By focusing on individual neurons and generating human-readable descriptions, MILAN provides a new level of transparency for machine-learning models.

In the future, researchers aim to:

1. Apply MILAN to Other Models: Extend its use to language models and speech recognition systems.

2. Improve Description Quality: Make explanations more detailed and accurate.

3. Enable Real-World Monitoring: Use MILAN to monitor and improve AI systems deployed in sensitive areas like healthcare and finance.

According to Jacob Andreas, a senior author of the research:

"The ultimate test of any explainable AI technique is whether it helps researchers and users make better decisions about when and how to deploy AI systems."

Conclusion

Understanding the inner workings of neural networks is essential for building AI systems that are transparent, trustworthy, and reliable. MILAN provides an innovative way to describe what individual neurons do using natural language, offering valuable insights for analyzing, auditing, and improving machine-learning models.

As AI continues to play a larger role in everyday life, methods like MILAN will be for ensuring these technologies remain understandable and aligned with human values.

Related reading

• What Is a Vision Transformer (ViT)? Explained Simply for Beginners [2026 Guide]
• [2026] Top 50+ Artificial Intelligence Interview Questions and Answers
• What is Explainable AI (XAI) and how does it help build trust in AI decisions?