Pandas Groupby Return Average BUT! exclude NaN
<p>So Im trying to make sense of the pandas groupby function and to reduce a large data frame I have. Here is an example:</p>

<pre><code>                             A        B
2016-09-23 19:36:08+00:00   NaN     34.0
2016-09-23 19:36:11+00:00   NaN     33.0
2016-09-23 19:36:12+00:00   24.1    NaN
2016-09-23 19:36:14+00:00   NaN     34.0
2016-09-23 19:36:17+00:00   NaN     34.0
2016-09-23 19:36:20+00:00   NaN     34.0
2016-09-23 19:36:22+00:00   24.2    NaN
2016-09-23 19:36:23+00:00   NaN     34.0
2016-09-23 19:36:26+00:00   NaN     34.0
2016-09-23 19:36:29+00:00   NaN     34.0
2016-09-23 19:36:32+00:00   24.1    NaN
2016-09-23 19:36:33+00:00   NaN     34.0
2016-09-23 19:37:00+00:00   NaN     34.0
2016-09-23 19:37:02+00:00   24.1    NaN
</code></pre>

<p>So I have 2 data series "A" and "B" that were sampled at different rates with their sampling time as the index of the original data frame. </p>

<p>I would like to now group the rows of the data frame by date/hour/minute and return the average of the data per minute. Here the average should ignore the missing values in the data frame.</p>

<p>So for example, I would return something like this:</p>

<pre><code>                             A        B
2016-09-23 19:36:00+00:00   24      34.0
2016-09-23 19:37:00+00:00   24.1    33.0
</code></pre>

<p>Is it possible to do this with a built in pandas function?</p>

<p>I think you need <a href="http://pandas.pydata.org/pandas-docs/stable/generated/pandas.DataFrame.resample.html" rel="nofollow"><code>resample</code></a> with <a href="http://pandas.pydata.org/pandas-docs/stable/generated/pandas.tseries.resample.Resampler.mean.html" rel="nofollow"><code>Resampler.mean</code></a>, which compute mean of groups, excluding missing values:</p>

<pre><code>print (df.resample('1Min').mean())
                             A          B
2016-09-23 19:36:00  24.133333  33.888889
2016-09-23 19:37:00  24.100000  34.000000
</code></pre>

<p>Another solution with <code>groupby</code>:</p>

<pre><code>print (df.groupby([pd.TimeGrouper('1Min')]).mean())
                             A          B
2016-09-23 19:36:00  24.133333  33.888889
2016-09-23 19:37:00  24.100000  34.000000
</code></pre>

python pandas dataframe