memory error while performing matrix multiplication
<p>As part of a project I'm working on, I need to calculate the mean squared error between <code>2m</code> vectors.</p>

<p>Basically I two matrices <code>x</code> and <code>xhat</code>, both are size <code>m</code> by <code>n</code> and the vectors I'm interested in are the rows of these vectors.</p>

<p>I calculate the MSE with this code</p>

<pre><code>def cost(x, xhat): #mean squared error between x the data and xhat the output of the machine
    return (1.0/(2 * m)) * np.trace(np.dot(x-xhat,(x-xhat).T))
</code></pre>

<p>It's working correctly, this formula is correct.</p>

<p>The problem is that in my specific case, my <code>m</code> and <code>n</code> are very large. specifically, <code>m = 60000</code> and <code>n = 785</code>. So when I run my code and it enters this function, I get a memory error.</p>

<p>Is there a better way to calculate the MSE? I'd rather avoid for loops and I lean heavily towards matrix multiplication, but matrix multiplication seems extremely wasteful here. Maybe something in numpy I'm not aware of?</p>

<p>If you are going for performance as well, one alternative approach to compute sum of squared differences could be using <a href="http://docs.scipy.org/doc/numpy/reference/generated/numpy.einsum.html" rel="nofollow"><code>np.einsum</code></a>, like so -</p>

<pre><code>subs = x-xhat
out = np.einsum('ij,ij',subs,subs)
</code></pre>

python numpy math memory matrix