Python csv reader // how to ignore enclosing char (because sometimes it's missing)
<p>I am trying to import csv data from files where sometimes the enclosing char " is missing.</p>

<p>So I have rows like this:</p>

<pre><code>"ThinkPad";"2000.00";"EUR"
"MacBookPro";"2200.00;EUR"  
# In the second row the closing " after 2200.00 is missing
# also the closing " before EUR" is missing
</code></pre>

<p>Now I am reading the csv data with this:</p>

<pre><code>csv.reader(
            codecs.open(filename, 'r', encoding='latin-1'),
            delimiter=";",
            dialect=csv.excel_tab)
</code></pre>

<p>And the data I get for the second row is this:</p>

<p>["MacBookPro", "2200.00;EUR"]</p>

<p>Aside from pre-processing my csv files with a unix command like sed and removing all closing chars " and relying on the semicolon to seperate the columns, what else can I do?</p>

<p>This <em>might</em> work:</p>

<pre><code>import csv
import io

file = io.StringIO(u'''
"ThinkPad";"2000.00";"EUR"
"MacBookPro";"2200.00;EUR"
'''.strip())

reader = csv.reader((line.replace('"', '') for line in file), delimiter=';', quotechar='"')
for row in reader:
    print(row)
</code></pre>

<p>The problem is that if there are any legitimate quoted line, e.g.</p>

<pre><code>"MacBookPro;Awesome Edition";"2200.00";"EUR"
</code></pre>

<p>Or, worse:</p>

<pre><code>"MacBookPro:
Description: Awesome Edition";"2200.00";"EUR"
</code></pre>

<p>Your output is going to produce too few/many columns. But if you <em>know</em> that's not a problem then it will work fine. You could pre-screen the file by adding this before the read part, which would give you the malformed line:</p>

<pre><code>for line in file:
    if line.count(';') != 2:
        raise ValueError('No! This file has broken data on line {!r}'.format(line))
file.seek(0)
</code></pre>

<p>Or alternatively you could screen as you're reading:</p>

<pre><code>for row in reader:
if any(';' in _ for _ in row):
    print('Error:')
print(row)
</code></pre>

<p>Ultimately your best option is to fix whatever is producing your garbage csv file.</p>

python csv