Torchlurk

About

Introduction

In the past recent years, Convolutional Neural Networks (CNN) have shown impressive capabilities in image recognition tasks even outperforming humans. Despite undeniable success in all field of science, one can wonder why do these processing paradigms perform so well. Indeed, complexity increases along with the number of layers and internal computation lack of transparency for the user.

This project aims to develop a visualization tool to ease the interpretation of trained CNN and give insights to the user about what the network is actually learning. The name Torch- refers to the use of the PyTorch framework and -lurk to our willingness to unveil the intrinsic abstraction levels that are 'hidden' in these CNNs.

Motivations

PyTorch is one of the most used deep learing framework and it just keeps getting more and more popular. Nevertheless, we feel like it lacks practical visual tool associated to it to get insights about network structure. We believe that is a great opportunity to develop such a tool as part of this course as it combines utility to visualization.

This visualization made with D3.js allows to dynamically select between 2 features which are indicators of deep learning framework popularity. On one side, we compare scientific publications in 2018 and on the other we compare contributors. TensorFlow remains the most popular framework overall. However, we believe that PyTorch is about to become more and more popular since it has already been closing the gap bewteen its concurrents since several years now.

Related work

ImageNet is as database gathering a large number of images (several millions annotated images) representing thousands of categories. It aims to provide scientists with as many images as possible to meet the growing needs for additional data (the more the best).

Since the ImageNet project was launched in 2010, many researches have been carried out on it, especially in the field of computer vision and deep learning involving the use of neural networks (NN). Due to their increasing success, several works focus on in-depth performance understanding of these NN.

A good way to analyse networks and in particular CNN, is by visualizing them (for e.g. layers or filters). In their Visualizing and Understanding Convolutional Networks paper, Matthew D. Zeiler and Rob Fergus focus on understanding why CNN perform so well. To do so, they propose a visualization technique that reveals the input stimuli exciting each feature map (for each layer). Targeting the same goal, Yosinski et al propose 2 tools: one to visualize activations for each layer and a second to visualize features at each layer. For the backend of our project, we used their toolbox (available on GitHub) as we are interested in those information to visualize the networks internal states.

Visualization and design

From the topic selection to the final product, the graphs and the visual identity of the website have been constantly rethought in order to build an intuitive tool. We developed in particular a logo to give our Torchlurk tool a visual identity. It contains in particular the PyTorch logo with a man, the Lurker, viciously inspecting a newspaper which refers to our willingness to carefully inspect what’s inside a network. The logo is incorporated as the O of a simple and modern font. For the website, we wanted a refined design: the primary advantage for a tool like the one we developed is that it does not drown the user in a flood of unnecessary information. Instead, it offers a general overview of Torchlurk functionalities.

What's next?

Torchlurk library will be maintained even after the project. As we were limited with the computational power, we wish to compute similar results for AlexNet and resNet. We also don’t want to restrict ourselves to explore deep feedforward networks but we also want to incorporate the possibility to visualize Recurrent Neural Networks. Finally, we aim to implement new regularization methods to magnify filter visualization in the same way OpenAI Microscope did with the Lucid library.