Why can't PyBrain Learn Binary
<p>I am attempting to get a network (PyBrain) to learn binary. This my code and it keeps return values around 8, but it should be return 9 when I activate with this target.</p>

<pre><code>from pybrain.tools.shortcuts import buildNetwork
from pybrain.structure import *
from pybrain.datasets import *
from pybrain.supervised.trainers import BackpropTrainer
from matplotlib.pyplot import *


trains = 3000
hiddenLayers = 4
dim = 4
target = (1, 0, 0, 1)

ds = SupervisedDataSet(dim, 1)

ds.addSample((0, 0, 0, 0), (0,))
ds.addSample((0, 0, 0, 1), (1,))
ds.addSample((0, 0, 1, 0), (2,))
ds.addSample((0, 0, 1, 1), (3,))
ds.addSample((0, 1, 0, 0), (4,))
ds.addSample((0, 1, 0, 1), (5,))
ds.addSample((0, 1, 1, 0), (6,))
ds.addSample((0, 1, 1, 1), (7,))
ds.addSample((1, 0, 0, 0), (8,))


net = buildNetwork(dim, hiddenLayers, 1, bias=True, hiddenclass=SigmoidLayer)
trainer = BackpropTrainer(net, ds)

tests = []

for i in range(trains):
    trainer.train()
    tests.append(net.activate(target))


plot(range(len(tests)), tests)


print net.activate(target)
show()
</code></pre>

<p>I have tried adjusting the number hidden Layers, the hiddenclass from TanhLayer to SigmoidLayer and varied the number of trains, but it always converges around 500 times (training the network to the dataset). Should I be using a different trainer than back propagation and if so why?</p>

<p>I would suggest you make the target something in the middle instead of on the fringe.</p>

<p>I tried expanding the training data upwards with 10 and 11, then it produced better results on predicting 9, even with 9 left out of the training data. Also you get a pretty good result if you try to predict 4, even if you do not have 4 in the training data.</p>

<p>From my experience I would not expect a neural net to readily guess numbers that are beyond the borders of the test data.</p>

python neural-network artificial-intelligence pybrain