Implementing softmax regression
<p>I am trying to make a neural network using softmax regression. I am using the following regression formula:</p>

<p><a href="https://i.stack.imgur.com/oaAL7.png" rel="nofollow"><img src="https://i.stack.imgur.com/oaAL7.png" alt="enter image description here"></a></p>

<p>Lets say I have an input of 1000x100. In other words, lets say I have 1000 images each of dimensions 10x10. Now, let's say the images are images of letters from A, B, C, D, E, F, G, H, I, J and I'm trying to predict this. My design is the following: to have 100 inputs (each image) and 10 outputs.</p>

<p>I have the following doubts. Given that n is superscript in x^n, with regards to the numerator, should I perform the dot product of w (w = weights whose dimensions are 10x100 - 10 representing the number of outputs and 100 representing the number of inputs) and a single x(a single image) or all the imagines combined(1000x100)? I am coding in python and so if I do dot product of w and x^T (10x100 dot 100x1000), then I am not sure how I can make that an exponent. I am using numpy. I am having a hard time wrapping my mind around these matrices on how they can be raised as an exponent.</p>

<p>If you are training Neural Networks, it might we worth while to check <a href="http://deeplearning.net/tutorial/mlp.html" rel="nofollow">Theano</a> library. It features various output thresholding functions like <em>tanh</em>, <em>softmax</em>, etc. and allows training of neural networks on GPU.</p>

<p>Also x^n is the output of the last layer in the above formula, not input raised to some exponent. You can't put matrix in exponent.</p>

<p>You should check more about softmax regression. <a href="http://ufldl.stanford.edu/tutorial/supervised/SoftmaxRegression/" rel="nofollow">This</a> might be of help.</p>

python neural-network softmax