how can i fix entropy yielding nan?
<p>I am trying to calculate entropy from array resulted from np.histogram by</p>

<pre><code>mu1, sigma1 = 0, 1
s1 = np.random.normal(mu1, sigma1, 100000)
hist1 = np.histogram(s1, bins=100, range=(-20,20), density=True)
data1 = hist1[0]
ent1 = -(data1*np.log(np.abs(data1))).sum() 
</code></pre>

<p>However, this ent1 would return nan. What is the problem here? </p>

<p>The problem is that you have zero probabilities in your histogram, which don't make numerical sense when applying Shannon's entropy formula. A solution is to ignore the zero probabilities.</p>

<pre><code>mu1, sigma1 = 0, 1
s1 = np.random.normal(mu1, sigma1, 100000)
hist1 = np.histogram(s1, bins=100, range=(-20,20), density=True)
data1 = hist1[0]
non_zero_data = data1[data1 != 0]
ent1 = -(non_zero_data*np.log(np.abs(non_zero_data))).sum() 
</code></pre>

python numpy entropy