Align python arrays with missing data
<p>I have some time series data, say:</p>

<pre><code># [ [time] [ data ] ]
a = [[0,1,2,3,4],['a','b','c','d','e']]
b = [[0,3,4]['f','g','h']]
</code></pre>

<p>and I would like an output with some filler value, lets say None for now:</p>

<pre><code>a_new = [[0,1,2,3,4],['a','b','c','d','e']]
b_new = [[0,1,2,3,4],['f',None,None,'g','h']]
</code></pre>

<p>Is there a built in function in python/numpy to do this (or something like this)? Basically I would like to have all of my time vectors of equal size so I can calculate statistics (np.mean) and deal with the missing data accordingly.</p>

<p>Thanks.</p>

<p>smarx has a fine answer, but <a href="http://pandas.pydata.org/pandas-docs/stable/index.html" rel="nofollow">pandas</a> was made exactly for things like this. </p>

<pre><code># your data
a = [[0,1,2,3,4],['a','b','c','d','e']]
b = [[0,3,4],['f','g','h']]

# make an empty DataFrame (can do this faster but I'm going slow so you see how it works)
df_a = pd.DataFrame()
df_a['time'] = a[0]
df_a['A'] = a[1]
df_a.set_index('time',inplace=True)

# same for b (a faster way this time)
df_b = pd.DataFrame({'B':b[1]}, index=b[0]) 

# now merge the two Series together (the NaNs are in the right place)
df = pd.merge(df_a, df_b, left_index=True, right_index=True, how='outer') 

In [28]: df
Out[28]: 
     A    B
0    a    f
1    b  NaN
2    c  NaN
3    d    g
4    e    h
</code></pre>

<p>Now the fun is just beginning. Within a DataFrame you can </p>

<ul>
<li><p>compute all of your summary statistics (e.g. <code>df.mean()</code>)</p></li>
<li><p>make plots (e.g. <code>df.plot()</code>)</p></li>
<li><p>slice/dice your data basically however you want (e.g <code>df.groupby()</code>)</p></li>
<li><p>Fill in or drop missing data using a specified method (e.g. <code>df.fillna()</code>), </p></li>
<li><p>take quarterly or monthly averages (e.g. <code>df.resample()</code>) and a lot more. </p></li>
</ul>

<p>If you're just getting started (sorry for the infomercial it you aren't), I recommend reading <a href="http://pandas.pydata.org/pandas-docs/stable/10min.html" rel="nofollow">10 minutes to pandas</a> for a quick overview.</p>

python numpy