Cosine similarity yields 'nan' values pt.II
<p>This is the other side of this question: <a href="http://stackoverflow.com/questions/33651788/cosine-similarity-yields-nan-values">Cosine similarity yields &#39;nan&#39; values</a> .
In that topic, auther coded the metrics by himself, but iam using scipy's cosine: (<code>ratings</code> is 71869x10000)</p>

<pre><code>A = ratings[:,100]
A = A.reshape(1,A.shape[0])
books_similarity = np.empty([1, ratings.shape[0]])

for book in range(10000):
    books_similarity[0,book] = -cosine(A, ratings[:,book].reshape(1,A.shape[1]))+1
</code></pre>

<p>BUT it outputs: 0, one 1(for for itself) and NaN. So the solution in the topic i mentioned before, is not for me, becuase, iam using scipy. What should i do?</p>

<p>P.S: then i delete "1" from array and do:</p>

<pre><code>m = np.argmax(books_similarity)
books_similarity[0,m]
</code></pre>

<p>It returns "NaN"</p>

<p>P.S.S: First, i had a pickle file, decoded it into CSR, but then used numpy. I think, i should consider everything as np arrays, right?</p>

<p>The cosine distance is not defined if one of the input vectors is all 0.  <a href="http://docs.scipy.org/doc/scipy/reference/generated/scipy.spatial.distance.cosine.html" rel="nofollow"><code>scipy.spatial.distance.cosine</code></a> returns <code>nan</code> in that case:</p>

<pre><code>In [70]: a
Out[70]: array([0, 1, 1, 1, 0, 0, 0, 1, 0, 0])

In [71]: b
Out[71]: array([0, 0, 0, 0, 0, 0, 0, 0, 0, 0])

In [72]: cosine(a, b)
/Users/warren/miniconda3/lib/python3.5/site-packages/scipy/spatial/distance.py:329: RuntimeWarning: invalid value encountered in true_divide
  dist = 1.0 - np.dot(u, v) / (norm(u) * norm(v))
Out[72]: nan
</code></pre>

<p>This may be happening in your code.  Before calling <code>cosine</code>, check that neither input is all 0.</p>

<hr>

<p>P.S. I haven't tried to decipher what you are doing with <code>A</code> and <code>ratings</code>, but I suspect you'll eventually want to use <a href="http://docs.scipy.org/doc/scipy/reference/generated/scipy.spatial.distance.cdist.html" rel="nofollow"><code>scipy.spatial.distance.cdist</code></a> with the argument <code>method='cosine'</code>.</p>

python numpy scipy