How to average hourly observations stored on a .csv file into daily observations?
<p>I have a .csv file (mydb.csv) with the following entries (+1 million rows). The 7th row of this table contains dates. The dates repeat themselves many times, because this dataset contains hourly records.</p>

<pre><code>QTEwOA==,81881,-7.610773,-72.681333,220,A108,2016-06-11,08,21.4,95,994.3,3.3,0,0,,
QTEwOA==,81881,-7.610773,-72.681333,220,A108,2016-06-11,09,21.3,95,994.1,1.2,0,0,,
QTEwOA==,81881,-7.610773,-72.681333,220,A108,2016-06-11,10,21.2,94,994.5,2.1,0,0,,
QTEwOA==,81881,-7.610773,-72.681333,220,A108,2016-06-11,11,20.9,94,994.7,1.3,0,0,,
QTEwOA==,81881,-7.610773,-72.681333,220,A108,2016-06-11,12,20.9,93,995.6,1.7,0,0,0.0,0.0
</code></pre>

<p>I need to calculate the averages for the day for each recorded observation.</p>

<p>Can I do it in python or should I transform my .csv file into a sqlite file to query?</p>

<p>You can use the <code>pandas</code> library in python to do it very quickly. It would look like that:</p>

<pre><code>import pandas as pd
df = pd.read_csv("initial.csv")
avgd_df = df.groupby('date').mean()
avgd_df.to_csv("averaged.csv")
</code></pre>

python sqlite csv numpy