Why doesn't perceptron's implementation follow the formula in <python machine learning>?
<p>I'm beginner in machine learning. I begin my study with the book <a href="http://rads.stackoverflow.com/amzn/click/1783555130" rel="nofollow">python machine learning</a> and some videos online. </p>

<p>I'm confused by the implementation of Perceptron in "python machine learning". This is the formula:</p>

<p><a href="http://i.stack.imgur.com/LbiHK.png" rel="nofollow"><img src="http://i.stack.imgur.com/LbiHK.png" alt="enter image description here"></a></p>

<p>And this is the python implementation for the formula:
<a href="http://i.stack.imgur.com/ZpuLu.png" rel="nofollow"><img src="http://i.stack.imgur.com/ZpuLu.png" alt="enter image description here"></a></p>

<p>But by the formula, it's W * X, not X * W in the implementation. They're not same for matrix.(For <a href="http://docs.scipy.org/doc/numpy/reference/generated/numpy.dot.html" rel="nofollow">numpy dot</a>, if X is 2-D array, it's matrix multiplication.). Why does the implementation not follow the formula?</p>

<p>The actual values in above python codes from the Iris-example in the book are these:</p>

<pre><code>w[1:]:  [-0.68  1.82]

X: [[ 3.3   0.  ]
    [ 3.32  0.  ]
    [ 3.34  0.  ]
      ..., 
    [ 7.94  6.08]
    [ 7.96  6.08]
    [ 7.98  6.08]]
</code></pre>

<p>Thanks.</p>

<p>A good question here. The answer should be two-fold.</p>

<h2>W dot X =?= X dot W</h2>

<p>You are perfectly right on that exchanging X and W yeilds different result for matrix multiplication. </p>

<p>However, in this situation, w and x are actually vectors, or m*1 matrix. The dot product results in a scalar. So in this situation, 'x dot w' and 'w dot x' are the same.</p>

<p>As you can see that x is interpreted as [x0, x1, x2, ..., xm], which indicates its vector nature.</p>

<p>What this function does is combining the inputs of a neuron with weight being <code>w</code>. These inputs are outputs from neurons from the previous layer. We know a neuron's output is a scalar.</p>

<h2>About the bias term W_[0]</h2>

<p>Actually the implementation <em>is</em> different in that a w_[0] is appended to the end of the polynomial. That is called a bias term, which modifies the result of linear combination of the inputs, or the <code>x</code>. It is a common practice in the implementations to use this bias term. But in math, we usually omit it since it doesn't change the linear nature of the combination. Of course in some situations this bias term is explicitly listed in the math representation.</p>

<h2>====== Update =======</h2>

<p>As the question has been updated, a further explain on the actual situation is added.</p>

<p>In this case, the first dimension is '<a href="https://docs.scipy.org/doc/numpy/user/basics.broadcasting.html" rel="nofollow">broadcast</a>' to the result. </p>

<p>Consider a simpler situation:</p>

<pre><code>w[1:]:  [-0.68  1.82]

X: [ 3.3   0.  ]
</code></pre>

<p>in which as @Peater and I explained, X and w are both vectors.</p>

<p>In the actual setup, X is not a matrix, but rather a vactor variable that changes on sampling, or time. Say X is of dimension [n * 2], that means x is a two-dimensional vector with n samples.</p>

<p>Here sample means different evaluations of a variable in different time frames, or different pixels. This dimension of X is broadcast to the result.</p>

python machine-learning