looping the data using date variable in python
<p>I have the following dataframe with datetime, lon and lat variables. This data is collected for each second which means each date is repeated 60 times</p>

<p>I am doing some calculations using lat, lon values and at the end I need to write 
this data to Postgres table.</p>

<pre><code>2016-07-27 06:43:45    50.62    3.15
2016-07-27 06:43:46    50.67    3.22
2016-07-28 07:23:45    52.32    3.34
2016-07-28 07:24:46    52.67    3.45
</code></pre>

<p>Currently I have 10 million records . It is taking longer time if I use whole dataframe for computing.</p>

<p>How can I loop this for each date, write it to DB and clear the dataframe??</p>

<p>I have converted the datetime variable to date format</p>

<pre><code>df['date'] = df['datetime'].dt.date
df = df.sort(['datetime'])

my computation is
df.loc[(df['lat'] &gt; 50.10) &amp; (df['lat'] &lt;= 50.62), 'var1'] = 1
df.loc[(df['lan'] &gt; 3.00) &amp; (df['lan'] &lt;= 3.20), 'var2'] = 1
</code></pre>

<p>Writing it to DB</p>

<pre><code>df.to_sql('Table1', engine,if_exists = "replace",index = False)
</code></pre>

<p>Have you considered using the <code>groupby()</code> function? You can use it to treat each 'date' as a seperate DataFrame and then run your computations. </p>

<pre><code>for sub_df in df.groupby('date'):
    # your computations
</code></pre>

python loops date pandas