Python searching through columns
<p>I have a CSV file that I need to loop through in a specific pattern for specific columns and have the output patterns be stored in new files with the same name + "_pattern" + [1,2,3,etc.] + .csv.</p>

<p>This is the search pattern: Loop through column 1 and find the same # and grab them, then loop through column 2 of the grabbed list and then grab all that have the same date in column 2, then go to column 4 and grab all #s that are NOT the same, and then create a file with the pattern from column 1 and 2 and 4 organized by column time.</p>

<p>Example: </p>

<pre class="lang-none prettyprint-override"><code>1       2           time    4
13.45   9/29/2016   6:00    98765
12.56   9/29/2016   6:05    76548
13.45   9/29/2016   6:07    98764
13.45   9/29/2016   6:21    98766
13.45   9/29/2016   6:20    96765
12.56   9/29/2016   6:06    76553
</code></pre>

<p><a href="https://i.stack.imgur.com/OqxcG.jpg" rel="nofollow">Better view of table</a></p>

<p>The result would be, file_pattern_1.csv would have:</p>

<pre class="lang-none prettyprint-override"><code>1. 13.45    9/29/2016   6:00    98765
2. 13.45    9/29/2016   6:07    98764
3. 13.45    9/29/2016   6:21    98766
</code></pre>

<p>But would not include:</p>

<pre class="lang-none prettyprint-override"><code>4. 13.45    9/29/2016   6:20    96765 
</code></pre>

<p>Because column 4 repeats from a previous entry, file_pattern_2.csv would have:</p>

<pre class="lang-none prettyprint-override"><code>1. 12.56    9/29/2016   6:05    76548
2. 12.56    9/29/2016   6:06    76553
</code></pre>

<p>This is what I have so far but I have become lost on the looping logic:</p>

<pre><code>import os

infile = raw_input("Which file are we working with? ")
assert os.path.exists(infile), "Path is incorrect."
os.chdir(infile)

def createFile(csvFile, fileName):
    with open (fileName, 'wb') as ftext:
        ftext.write(csvFile)

def appendFile(csvFile, fileName):
    with open (fileName, 'a') as ftext:
        ftext.write(csvFile)

def setfilename(tread):
    fileName = tread[0:tread.index('.')] + '_patterns' + str(countItem) + '.csv'
    return fileName

for i in pcolumn:
    if pcolumn == pcolumn:
        return pfile
    for x in date:
        if date == date:
            return date
            for a in acolumn:
                if acolumn != acolumn:
                    createFile(fileName)
else:
    print "Finished."
</code></pre>

<p>As you loop over your files you need to keep a record of which patterns are not eligible for saving. You could use a <code>set</code> for this purpose. To group your entries in each file you could use <code>itertools.groupby</code>. Using your example:</p>

<pre class="lang-python prettyprint-override"><code>import itertools

f = [i.split("   ") for i in """1       2           time    4
13.45   9/29/2016   6:00   98765
12.56   9/29/2016   6:05   76548
13.45   9/29/2016   6:07   98764
13.45   9/29/2016   6:21   98766
13.45   9/29/2016   6:20   96765
12.56   9/29/2016   6:06   76553""".split("\n")[1:]]


seen_patterns = set([('9/29/2016', '96765')])   # You need to add entries to this set which you want to exclude

# Sort and group your entries by the first and second columns
col1 = itertools.groupby(sorted(f, key=lambda x: (x[0], x[1])), key=lambda x: (x[0], x[1]))
for k, v in col1:
    v = list(v)
    # Filter out patterns which are not allowed
    to_save = [" ".join(i) for i in v if (i[1], i[3]) not in seen_patterns]
    for i in to_save:
        print i  # Save this to an appropriate file
    print

&gt;&gt;&gt;
12.56 9/29/2016 6:05 76548
12.56 9/29/2016 6:06 76553

13.45 9/29/2016 6:00 98765
13.45 9/29/2016 6:07 98764
13.45 9/29/2016 6:21 98766
</code></pre>

<p>As a further suggestion, have a look at the <code>glob</code> module for collecting file paths from directories, it is really useful. </p>

python loops csv