Adaline Learning Algorithm
<p>I can not seem to debug the following implementation of an Adaline neuron... I'm hoping someone can spot what I cannot. I think the problem lies in the last few lines of my train method?</p>

<pre><code>from numpy import random, array, dot
import numpy as np
import matplotlib.pyplot as plt
from random import choice
import math
import sympy

class adalineANN(object):
    def __init__(self, gamma=.2, trials=500, errors=[], weights=[]):
        self.gamma = gamma
        self.trials = trials
        self.errors = errors
        self.weights = weights

def train(self):
    self.weights = random.rand(3)

    coordinates_class1 = []
    coordinates_class2 = []

    for x in np.random.normal(2, .5, 20):
        for y in np.random.normal(3, .5, 20):
            coordinates_class1.append(([x, y, 1], 1))
            break

    for x in np.random.normal(2, .25, 20):
        for y in np.random.normal(-1, .25, 20):
            coordinates_class2.append(([x, y, 1], -1))
            break

    trainingData = coordinates_class1 + coordinates_class2

    for i in range(self.trials):
        x, target = choice(trainingData)
        y = np.dot(x, self.weights)
        error, errors = [], []
        error = (target - y)
        self.errors.append(error)
        for i in range(0, 3):
            self.weights[i] += self.gamma * x[i] * (target - y) #????* (sympy.cosh(y)**(-1))**2

def plot(self):
    plt.plot(self.errors)
    plt.show()


A = adalineANN()
A.train()
A.plot()
</code></pre>

<p>Do I need the derivative of my threshold function too? See the above note.</p>

<p>There is nothing really wrong, but:</p>

<ol>
<li><p>You are using "i" for iterator in <strong>both</strong> loops, in this code it does not matter since you do not really use it in the outer loop, but it could lead to very nasty bugs in general.</p></li>
<li><p>Your gamma is too high, change it to 0.01 </p></li>
<li><p>Do not plot signed error, adaline is minimizing <strong>squared error</strong> thus you should plot <code>error = (target - y)**2</code>; you can even go for accuracy to see that it discriminates just fine with <code>error = target != np.sign(y)</code>.</p></li>
</ol>

<p><a href="http://i.stack.imgur.com/8xDny.png" rel="nofollow"><img src="http://i.stack.imgur.com/8xDny.png" alt="after changes"></a></p>

python machine-learning neural-network