Calculate z-score in data bunch but excluding N.A
<p>So I got this bunch of data with N.A. values in them:</p>

<p><a href="http://i.stack.imgur.com/XVlKo.jpg" rel="nofollow">Data Dump</a></p>

<p>So how do I get the z-score of each column while excluding the N.A. values? Such that the z-score output looks like this?</p>

<p><a href="http://i.stack.imgur.com/S5XiW.jpg" rel="nofollow">Z-Score value output</a></p>

<p>So for this is what I have, which is based on previous questions:</p>

<pre><code>cols = list(df.columns)
df[cols]
for col in cols:
    col_zscore = col + '_zscore'
    df[col_zscore] = (df[col] - df[col].mean())/df[col].std(ddof="N.A.")
</code></pre>

<p>but I got TypeError. </p>

<p>Please help, I am really a beginner at this.</p>

<p>I think you need <a href="http://pandas.pydata.org/pandas-docs/stable/generated/pandas.Series.replace.html" rel="nofollow"><code>replace</code></a> first <code>N.A.</code> to <code>NaN</code> and convert values to <code>float</code>:</p>

<pre><code>df = df.replace({'N.A.': np.nan}).astype(float)

for col in df.columns:
    if col != 'PE Trail':
        col_zscore = col + '_zscore'
        df[col_zscore] = (df[col] - df[col].mean())/df[col].std()

print (df)
   PE Trail  PE fwd   PB  PE fwd_zscore  PB_zscore
0       NaN    1.00  1.0       1.317465   0.707107
1       NaN    0.50  NaN       0.146385        NaN
2       NaN    0.00  0.5      -1.024695  -0.707107
3       NaN    0.25  NaN      -0.439155        NaN
</code></pre>

<p>Also <code>type</code> of value in parameter <code>ddof</code> in <a href="http://pandas.pydata.org/pandas-docs/stable/generated/pandas.Series.std.html" rel="nofollow"><code>std</code></a> is <code>int</code>.</p>

<hr>

<p>If use <a href="http://pandas.pydata.org/pandas-docs/stable/generated/pandas.read_csv.html" rel="nofollow"><code>read_csv</code></a>, parameter <code>na_values</code> causes converting <code>N.A.</code> to <code>NaN</code>:</p>

<pre><code>import pandas as pd
import numpy as np
import io

temp=u"""PE Trail;PE fwd;PB
N.A.;1;1
N.A.;0.5;N.A.
N.A.;0;0.5
N.A.;0.25;N.A."""
#after testing replace io.StringIO(temp) to filename
df = pd.read_csv(io.StringIO(temp), sep=";", na_values='N.A.')
print (df)
   PE Trail  PE fwd   PB
0       NaN    1.00  1.0
1       NaN    0.50  NaN
2       NaN    0.00  0.5
3       NaN    0.25  NaN
</code></pre>

python pandas