gave user more control over the network’s size (networks can now have multiple hidden layers)

I read about the problem and solution discovered by Marvin Minskey, so I added more layers to the network. The user could choose the number of layers and the number of neurons in the hidden layers. (the number of neurons was the same for each hidden layer) This made the network work better.

made the ANN software’s learning process automated. also sped up processing time significantly

I still didn’t know very much about making a neural network learn. I designed it so that the user would show the network a set of inputs, then showed it the output that it should get, and would then click the Learn Button. This would change the input layer’s input weights pretty much randomly until there were a few on or off, instead of all of them being on or off. It would then make each of the output layers input weights higher or lower depending on whether it would make the output closer to what it should be. This worked, but not very well.

Each neuron was a class module, and it had lots of inputs and outputs. These were also class modules. Each input had a value, and a weight. Each output class kept track of the neuron it was connected to, the input on that neuron that it was connected to, and its value. I made the program a lot faster by using multiple dimension arrays instead of class modules for the inputs and outputs. Also, when one input of a neuron was changed, it would start sending out a new output. This would cause a chain reaction that lasted a long time, and then start over again when the next neuron in the same layer as the first had its input changed. I made it so that when a neurons input was changed, it didn’t give its output until I told it to. I made it so first the input layer changed, then all the neurons in the first hidden layer gave their outputs, then the neurons in the next gave their outputs, etc. This sped up the program a lot. These two things combined made the program over fifty times faster.

started making Artificial Neural Network (ANN) software using Visual Basic

I designed the software so that there are always two layers: the input layer, and the output layer. The neurons in the input layer were connected to "inputs" that were represented by a grid of boxes. The user could change the number of inputs before making a number. When they did make a network, every input was connected to every neuron in the input layer. The inputs could have two values: one or zero. Each neuron in the input layer would look at all of its inputs and decide whether to output a one or a zero. It did this by multiplying each of its inputs by its weight, and then adding up all of the sums. If the total was greater than a certain threshold level, it would output a one, otherwise it would output a zero. To find what the threshold should be for a specific neuron, I took the total number of inputs that that neuron had, and I divided by two.

Every input that a neuron had had its own weight. The weights are what make up a neural network’s memory, and to make a neural network “learn” something, the weights must be changed correct amounts. At first I made it so that the only way to change the weights was to click on the visual representation of one and click the plus or minus button.

There were still some very simple behaviors that were not possible for the network to show. An example was this: I had a network with two inputs and one output, and I was trying to teach it to output a one when either of the inputs was on, but output a zero if neither or both of the inputs was on. Soon I began to think that a neuron shouldn’t only be able to have a positive affect on another neuron; it should also be able to have a negative affect. I made it so the weights could be positive or negative. I also made it so that instead of the neurons outputting zeros and ones, they outputted negative ones and positive ones. I did the same thing for the inputs. This made the network work better, and behaviors like these were now possible.