Training Computer Vision to Think Like the Brain Enhances Performance and Robustness

Researchers from MIT and IBM have discovered that training artificial neural networks to mimic the processing patterns of the brain’s visual cortex can significantly enhance computer vision systems. By incorporating neural activity patterns from the brain’s inferior temporal (IT) cortex, the team observed improved object recognition capabilities and increased resilience to adversarial attacks. This breakthrough in aligning computer vision with human vision has the potential to advance the field and deepen our understanding of biological neural networks.

Enhancing computer vision with neural similarity

MIT Professor James DiCarlo and his team sought to enhance computer vision models by introducing brain-like features. They built a computer vision model using neural data collected from monkeys’ IT cortex, a vital region for object recognition. By training the artificial neural network to mimic the behavior of primate vision-processing neurons, the researchers aimed to align the model’s internal processes with the brain’s neural mechanisms.

The results were promising, as the biologically informed model exhibited enhanced performance in object recognition tasks. The artificial IT layer of the model closely matched the responses of biological IT neurons, even when presented with distorted images. The model’s interpretations aligned more closely with human perception, demonstrating a more human-like understanding of images. This finding provides valuable insights into how visual information is processed in the brain.

Applying neural alignment to adversarial attacks

Computer vision systems are vulnerable to adversarial attacks, where slight distortions in images can mislead artificial neural networks. The researchers found that the neurally aligned model displayed greater resistance to these attacks compared to conventional models. While stronger attacks could still deceive the model, this phenomenon mirrors human perception. Further exploration is underway to understand the limits of adversarial robustness in humans.

The collaboration between neuroscience and computer science has proven fruitful, with both fields benefiting from shared insights. As computer vision and AI researchers gain new methods to achieve robustness, neuroscientists and cognitive scientists can develop more accurate mechanistic models of human vision. This virtuous cycle fosters progress and innovation in both natural and artificial intelligence.

Future directions

To further enhance computer vision systems, the researchers aim to combine the neural alignment approach with other brain-inspired techniques. By aligning multiple visual processing layers with neural activity patterns, new models can be developed to simulate the brain’s complex visual processing hierarchy. This integration of neuroscience and computer science promises even greater advancements in the field of computer vision.

The integration of artificial neural networks with brain-inspired techniques has demonstrated the potential to improve computer vision systems. By training models to think more like the human brain, researchers have achieved enhanced object recognition and improved resilience to adversarial attacks. The alignment of computer vision with human vision not only leads to practical applications but also deepens our understanding of the intricate workings of the brain. This collaborative synergy between neuroscience and computer science opens up new avenues for innovation and progress in the field of artificial intelligence.