Decreasing the number of neurons

28Jan10

This is some evidence why 200 neurons maybe too many.
Random Google searches:


Estimation of monthly average daily global solar irradiation using …
The number of neurons in the hidden layer varied between 8 and 18.


Predicting saturates of sour vacuum gas oil using artificial neural networks and genetic algorithms
The number of neurons in the hidden layer, the momentum and the learning rates were determined by using the genetic algorithm.
the optimum number of neurons in the hidden layer for networks was 8


OpenModeller
Number of neurons in the hidden layer
Data type: Integer    Domain: [1, oo)    Typical value: 8


Choose a high number of neurons in the hidden layer (eg. 20). You should see the effect of overfitting. If you don’t, you should add more hidden neurons.


As expected, the worst performance or highest error rates occurs when the number of neurons is either very high (above 13), or very low (1), which is the minim number of neurons that can be used. On the other hand, the best performance was achieved when the number of neurons in the first layer was equal to 12,


Determining the size of the network (the number of neurons) has important consequences for its performance. Too small a network may not reach an acceptable level of accuracy. Too many neurons may result in an inability for the network to generalize (it may rote learn the training patterns).

Let’s run it with nNeurons: 10:
And let’s run it for 260 days:
projected annual gains for the 7 separate tests:
-26.6971%.
-7.0937%
-24.4624%
25.4778%
-23.6797%
The average is -11%.
Disaster. It is worse than random. The 10 neuron is not enough OR there is a bug in the implementation.

Let’s run it with nNeurons: 50:
And let’s run it for 260 days:
projected annual gains for the 7 separate tests:
21.146%.
67.0681%.
-1.9673%.
22.604%.
-33.2258%.
-48.4255%.
The average is 4.6%
I plot the best and the worst charts.

Clearly, we have a lot of work to do.

Advertisements


No Responses Yet to “Decreasing the number of neurons”

  1. Leave a Comment

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s


%d bloggers like this: