Numpy: Read csv, deal with undefined values
<p>What is the best way to read data from a csv file into a numpy array when some of the values are labelled as 'undefined' as follows:</p>

<pre><code>0.231620,0.00001,444.157
0.225370,--undefined--,1914.637
0.237870,0.0003,--undefined--
</code></pre>

<p>I have a lot of these files that I will have to loop over, and it is fine to assume that an undefined value should be zero.</p>

<p>I'd suggest you to try to cast each value you read to float, then catch typecast ValueError exception and assign it to zero in exception handler.</p>

<p>This will be the most pythonic way</p>

<p>Assuming your CSV contains float values, you should end with something like:</p>

<pre><code>with open('data.csv', 'r') as fd:
    # iterate over all lines in csv
    for line in fd:
        # split and iterate over values in line, maintaining item index
        for i, value in enumerate(line.split(',')):
            try:
                value = float(value)
            except ValueError:
                # consider undefined/non-float value equals to 0
                value = 0.0
            # store parsed value wherever you need it
            print('value[%d] = %f' % (i, value))
</code></pre>

<p>Alternatively, of only '--undefined--' strings to be handled as '0.0', you can write it like (the innermost loop)</p>

<pre><code>            value = float(value) if value != '--undefined--' else 0
</code></pre>

python csv numpy