CNN Explainer: Learning Convolutional Neural Networks with Interactive Visualization

crown jewel figure
With CNN Explainer, learners can visually examine how Convolutional Neural Networks (CNNs) transform input images into classification predictions (e.g., predicting espresso for an image of a coffee cup), and interactively learn about their underlying mathematical operations. In this example, a learner uses CNN Explainer to understand how convolutional layers work through three tightly integrated views, each explaining the convolutional process in increasing levels of detail. (A) The Overview visualizes a CNN architecture where each neuron is encoded as a square with a heatmap representing the neuron’s output, and each edge connects the neuron with its corresponding inputs and outputs. (B) Clicking a neuron reveals how its activations are computed by the previous layer’s neurons, displaying the often-overlooked intermediate computation through animations of sliding kernels. (C) The Convolutional Interactive Formula View allows users to interactively inspect the underlying mathematics of the dot-product operation core to convolution, through hovering the 3×3 kernel over the input, and interactively studying the corresponding output. For clarity, visibility of Overview and annotation text is improved, and the overlay is re-positioned.
Abstract
Deep learning's great success motivates many practitioners and students to learn about this exciting technology. However, it is often challenging for beginners to take their first step due to the complexity of understanding and applying deep learning. We present CNN Explainer, an interactive visualization tool designed for non-experts to learn and examine convolutional neural networks (CNNs), a foundational deep learning model architecture. Our tool addresses key challenges that novices face while learning about CNNs, which we identify from interviews with instructors and a survey with past students. CNN Explainer tightly integrates a model overview that summarizes a CNN's structure, and on-demand, dynamic visual explanation views that help users understand the underlying components of CNNs. Through smooth transitions across levels of abstraction, our tool enables users to inspect the interplay between low-level mathematical operations and high-level model structures. A qualitative user study shows that CNN Explainer helps users more easily understand the inner workings of CNNs, and is engaging and enjoyable to use. We also derive design lessons from our study. Developed using modern web technologies, CNN Explainer runs locally in users' web browsers without the need for installation or specialized hardware, broadening the public's education access to modern deep learning techniques.
Citation
CNN Explainer: Learning Convolutional Neural Networks with Interactive Visualization
@article{wangCNNExplainerLearning2021,
  title = {{{CNN Explainer}}: {{Learning Convolutional Neural Networks}} with {{Interactive Visualization}}},
  shorttitle = {{{CNN Explainer}}},
  author = {Wang, Zijie J. and Turko, Robert and Shaikh, Omar and Park, Haekyu and Das, Nilaksh and Hohman, Fred and Kahng, Minsuk and Chau, Duen Horng},
  journal={IEEE Transactions on Visualization and Computer Graphics (TVCG)},
  publisher={IEEE},
  year={2021},
}