OpenAI has released Microscope, a collection of visualizations of every significant layer and neuron of eight leading computer vision (CV) models which are often studied in interpretability. The tool helps researchers analyze the features and other important attributes which form inside of the neural networks powering these CV models.
The visualizations available for each model include feature visualizations which show neural network understanding of images over many layers, image data points causing neurons to fire strongly, and plots of unit responses to synthetic image families. The models are composed of a graph of nodes which represent neural network layers and they are connected by edges. Each visualization contains hundreds of units, which are analogous to neurons. Microscope gets its name from the metaphorical labels describing the user experience. The user can choose a "location," which represents where the microscope is pointing, and a "technique," which represents the type of lens. Microscope focuses on exploring a small number of models in detail because each model results in hundreds of thousands of neurons, with each model presenting unique challenges in creating the final visualization. OpenAI plans to expand this set over time.
The feedback loop for exploring neurons can be reduced from minutes to seconds with Microscope. This has already helped in discovering unexpected features like high-low frequency detectors. Additionally, Microscope helps to make neural networks interpretability highly accessible, as these important visualizations would take other researchers hundreds of GPU hours to compute.
The primary goal in releasing Microscope is to provide lasting open source artifacts which drive studies comparing these models. One project which can benefit from Microscope is the Circuits Collaboration, which aims to reverse engineer neural networks by analyzing neurons and their connections. Also, for researchers with adjacent experience like neuroscientists, the visualizations provide a novel approach in understanding the internal workings of these vision models.