First Look at Convolutions

By Max Candocia

|

November 11, 2016

This is a prototype for visualizing the convolutions of a neural network. When you get image predictions, you will also get visualizations of the first few convolutional layers, which is what the network "sees" after processing the image. Green indicates a "positive" response and red indicates "negative". Darker areas do not affect the algorithm as much.

source code for the algorithm on GitHub

Here are the possible predictions:


Animals
Bee
Bird
Butterfly
Cat
Dog
Fish
Horse
Turtle
Shapes
Circle
Cube
Heart
Rectangle
Spiral
Square
Star
Triangle
Plants
Banana
Flower
Mushroom*
Pineapple
Pumpkin
Tree
Flags
American Flag
British Flag
Canadian Flag
Chilean Flag
Chinese Flag
Cuban Flag
French Flag
German Flag
Indian Flag
Israeli Flag
Jamaican Flag
Japanese Flag
Laotian Flag
Mexican Flag
Nigerian Flag
Puerto Rican Flag
Russian Flag
South Korean Flag
Swedish Flag
Tanzanian Flag
Ukrainian Flag
Miscellaneous
Car
Dragon
Ghost
Rainbow
Smiley Face
Snowman
Trebuchet
Characters
Elf
Ninja
Pirate
Santa Claus
Witch
Wizard

* Yes, I know mushrooms are technically fungi

Instructions

You can draw on the canvas while left-clicking with your mouse. The color will be the one from the palette to the upper-left of the canvas. You can erase everything by clicking on the "Everything Eraser" and clicking on the canvas. You can switch to eraser mode by clicking on "Eraser" and treating it like you are drawing. You can switch back to normal drawing mode by clicking on "Marker". You can also download your picture in another window by clicking "download" so you can print it out and hang it on your refrigerator. It won't close your current window.

You may also toggle the background color to change the brightness by clicking on "Toggle Background". There are 17 different shades it cycles through. The background does not affect the image submitted/downloaded, but it may be easier for you to draw something like a snowman if the background is darker.

If you want to submit your drawing, click "Submit Image", and it will be sent to the server. For security reasons, you will not be able to access the images once they are sent to the server and you leave the page. When you submit the image, you grant me (non-exclusive) permission to use, modify, and distribute the image.

The palette chooser was made with Spectrum JS, and the canvas drawing was made with sketch.js. The illustrations below that are made with D3.js.

Brush Size: 5

What is this a picture of? (required)

What is this image, specifically? (optional)

What does the computer think this is?  
Reset Visualizations Below (off=experimental)

RankGuessProbability
1
2
3
4
5

Tags: 

Recommended Articles

Sketching and Guessing with Keras

Draw a picture and see if the computer can guess it. You can also submit images for the algorithm to develop more possible results.

Overlaying Density Heatmaps on Geographic Maps in R

In this example, I use noise complaint data from New York City to demonstrate how you can plot densities of events on a map, as well as how extreme the averages are.