how can i fix entropy yielding nan?
<p>I am trying to calculate entropy from array resulted from np.histogram by</p>

<pre><code>mu1, sigma1 = 0, 1
s1 = np.random.normal(mu1, sigma1, 100000)
hist1 = np.histogram(s1, bins=100, range=(-20,20), density=True)
data1 = hist1[0]
ent1 = -(data1*np.log(np.abs(data1))).sum() 
</code></pre>

<p>However, this ent1 would return nan. What is the problem here? </p>

<p>To compute the entropy, you could use <a href="http://docs.scipy.org/doc/scipy/reference/generated/scipy.special.entr.html" rel="nofollow"><code>scipy.special.entr</code></a>.  For example,</p>

<pre><code>In [147]: from scipy.special import entr

In [148]: x = np.array([3, 2, 1, 0, 0.5, 2.5, 5])

In [149]: entr(x).sum()
Out[149]: -14.673474028700136
</code></pre>

<p>To check that result, we can also compute the entropy using <a href="http://docs.scipy.org/doc/scipy/reference/generated/scipy.special.xlogy.html" rel="nofollow"><code>scipy.special.xlogy</code></a>:</p>

<pre><code>In [150]: from scipy.special import xlogy

In [151]: -xlogy(x, x).sum()
Out[151]: -14.673474028700136
</code></pre>

<p>Finally, we can verify that is the same result that you expect:</p>

<pre><code>In [152]: xnz = x[x != 0]

In [153]: -(xnz*np.log(xnz)).sum()
Out[153]: -14.673474028700136
</code></pre>

python numpy entropy