Visualization of neural network learning

Explanation of what this is

I was curious about how neural networks behaved when learning to approximate functions. In order to get a better qualitative feel for their learning behavior, I hacked up this simple visualization. In this application, a simple neural network learns functions from R²->R. While it is learning, the application continually re-draws the surface of the function represented by the current state of the neural network.

The domain for both display and training is restricted to [-1,1] x [-1,1], and the range is restricted to [0,1].

The functions it can learn are as follows

"1 if r < 2 else 0" : f(x,y) is 1 if (x²+y²) < 1, otherwise it is 0
"sin(x)*sin(y)" : This is actually the function 0.5+0.5*sin(5*x)*sin(5*y). The "0.5" elements are to ensure the function to be learned is between 0 and 1. The "5*x" parts are to make the function interesting.
"x*y" : The classic hyperbolic parabaloid. Actually this one is "0.5+0.5*x*y" (for the proper range restriction)
"xor" : A machine learning favorite (or so I'm told) 1 if (x*y) < 0, 0 otherwise

What is seen in this image is a neural network that has been trained on the "xor" function attempting to adjust to the "sin(x)*sin(y)" function. Neural networks do exhibit interesting looking behavior in these scenarios

Note, in order to run this, you need a copy of MFC71.dll and MRC71R.dll.

You can download these files here (for mfc71.dll) and here (for msvcr71.dll). I think you need to put them in the same directory as the exe, but if you are sufficiently familiar with windows to do the "registering dlls" dance, then that might work too.

Download The application. Download someplace alongside the requisite MFC DLLs. Then run it. Please bear in mind that this is a windows executable, while I have not intentionally put malicious code into it, it should be treated with the same degree of caution as any windows executable downloaded from the internet.

If you are interested in this visualization, you might also want to see Training of an obstacle avoiding robot in simulation

Or you might want to look at my templatized neural network code. Though I would reccomend, if you are writing your own neural network classes, not using templates. Any possible speed benefit derived from templatized neural network classes is probably not worth the restriction of having to re-compile in order to change network topology.

mds at es oh ee dot you see es see dot eee dee you