Python Pandas: Write certain rows in file
<p>The csv file is way to big, so I am reading it chunk by chunk.
Therefore, I use read_csv with chunksize.</p>

<p>I want to store all rows, where the last entry has the value 1 in one file and all the other rows where the last entry is 0 in another file.</p>

<p>Suppose it looks like this:</p>

<pre><code>ID   A    B   C
0   0.0  0.1  1
1   0.1  0.2  0
2   0.1  0.0  1
</code></pre>

<p>So, I want to store row with ID 0 and 2 in one file and the row with ID 1 in another file.</p>

<p>How do I do that with pandas?</p>

<p>From <a href="http://stackoverflow.com/questions/19674212/pandas-data-frame-select-rows-and-clear-memory">this</a> post:</p>

<blockquote>
<pre><code>reader = pd.read_csv('big_table.txt', sep='\t', header=0, 
                     index_col=0, usecols=the_columns_i_want_to_use, 
                     chunksize=10000)

df = pd.concat([ chunk.ix[rows_that_I_want_] for chunk in reader ])
</code></pre>
</blockquote>

<p>But instead make 2 data frames:</p>

<pre><code>df0 = pd.concat([ chunk[chunk["C"] == 0] for chunk in reader ])
df1 = pd.concat([ chunk[chunk["C"] == 1] for chunk in reader ])
</code></pre>

<p>Then save each data frame independently</p>

python pandas