python: efficient way to check only a couple of rows in a csvreader iterator?
<p>I have a (very large) CSV file that looks something like this:</p>

<pre><code>header1,header2,header3
name0,rank0,serial0
name1,rank1,serial1
name2,rank2,serial2
</code></pre>

<p>I've written some code that processes the file, and writes it out (using csvwriter) modified as such, with some information I compute appended to the end of each row:</p>

<pre><code>header1,header2,header3,new_hdr4,new_hdr5
name0,rank0,serial0,salary0,base0
name1,rank1,serial1,salary1,base1
name2,rank2,serial2,salary2,base2
</code></pre>

<p>What I'm trying to do is structure the script so that it auto-detects whether or not the CSV file it's reading has <strong><em>already been processed</em></strong>.  If it has been processed, I can skip a lot of expensive calculations later.  I'm trying to understand whether there is a reasonable way of doing this within the reader loop.  I <em>could</em> just open the file once, read in enough to do the detection, and then close and reopen it with a flag set, but this seems hackish.</p>

<p>Is there a way to do this within the same reader?  The logic is something like:</p>

<pre><code>read first N lines   ###(N is small)
   if (some condition)
      already_processed = TRUE
      read_all_csv_without_processing
   else
      read_all_csv_WITH_processing
</code></pre>

<p>I can't just use the iterator that reader gives me, because by the time I've gotten enough lines to do my conditional check, I don't have any good way to go back to the beginning of the CSV.  Is closing and reopening it really the most elegant way to do this?</p>

<p>If you're using the usual python method to read the file (<code>with open("file.csv","r") as f:</code> or equivalent), you can "reset" the file reading by calling <code>f.seek(0)</code>. </p>

<p>Here is a piece of code that should (I guess) look a bit more like the way you're reading your file. It demonstate that reseting <code>csvfile</code> with <code>csvfile.seek(0)</code> will also reset <code>csvreader</code>:</p>

<pre><code>with open('so.txt', 'r') as csvfile:
    csvreader = csv.reader(csvfile, delimiter=',') 
    for row in csvreader:
        print('Checking if processed')
        print(', '.join(row))
        #if condition:
        if True:
            print('File already processed')
            already_processed = True
            print('Reseting the file')
            csvfile.seek(0)
            for row in csvreader:
                print(', '.join(row))
            break
</code></pre>

python csv