Writing intersection data to new CSV
<p>I have 2 CSV files which have a list of unique words. After I complete my intersection on them I get the results, but when I try to write it to a new file it creates a very large sized file of almost 155MB, when it should be well below 2MB.</p>

<p><strong>Code:</strong></p>

<pre><code>alist, blist = [], []

with open("SetA-unique.csv", "r") as fileA:
    reader = csv.reader(fileA, delimiter=',')
    for row in reader:
        alist += row

with open("SetB-unique.csv", "r") as fileB:
    reader = csv.reader(fileB, delimiter=',')
    for row in reader:
        blist += row

first_set = set(alist)
second_set = set(blist)

res = (first_set.intersection(second_set))

writer = csv.writer(open("SetA-SetB.csv", 'w'))

for row in res:
     writer.writerow(res)
</code></pre>

<p>Apart from writing the whole set each iteration you  also don't need to create multiple sets and lists, you can use <em>itertools.chain</em>:</p>

<pre><code>from itertools import chain
with open("SetA-unique.csv") as file_a, open("SetB-unique.csv")  as file_b,open("SetA-SetB.csv", 'w') as inter :
    r1 = csv.reader(file_a)
    r2 = csv.reader(file_b)
    for word in set(chain.from_iterable(r1)).intersection(chain.from_iterable(r2)):
        inter.write(word)+"\n"
</code></pre>

<p>If you are just writing words there is also no need to use <em>csv.writer</em> just use <em>file.write</em> as above.</p>

<p>If you are actually trying do the comparison row wise, you should not be creating a flat iterable of words, you can <em>imap</em> to tuples:</p>

<pre><code>from itertools import imap
with open("SetA-unique.csv") as file_a, open("SetB-unique.csv")  as file_b,open("SetA-SetB.csv", 'w') as inter :
    r1 = csv.reader(file_a)
    r2 = csv.reader(file_b)
    writer = csv.writer(inter)
    for row in set(imap(tuple, r1).intersection(imap(tuple, r2)):
        writer.writerow(row)
</code></pre>

<p>And if you only have one word per line you don't need the csv lib at all.</p>

<pre><code>from itertools import imap
with open("SetA-unique.csv") as file_a, open("SetB-unique.csv") as file_b,open("SetA-SetB.csv", 'w') as inter :
    for word in set(imap(str.strip, file_a)).intersection(imap(str.strip, file_b)):
        inter.write(word) + "\n"
</code></pre>

python csv