Reading a text file using Pandas where some rows have empty elements?
<p>I have a dataset in a textfile that looks like this.</p>

<pre><code>    0    0CF00400 X       8  66  7D  91  6E  22  03  0F  7D       0.021650 R
    0    18EA0080 X       3  E9  FE  00                           0.022550 R
    0    00000003 X       8  D5  64  22  E1  FF  FF  FF  F0       0.023120 R
</code></pre>

<p>I read this using</p>

<pre><code>file_pandas = pd.read_csv(fileName, delim_whitespace = True, header = None, engine = 'python')
</code></pre>

<p>And got the output</p>

<pre><code>    0   0  0CF00400  X   8  66  7D  91        6E  22    03    0F    7D  0.02165   
    1   0  18EA0080  X   3  E9  FE   0  0.022550   R  None  None  None      NaN   
    2   0  00000003  X   8  D5  64  22        E1  FF    FF    FF    F0  0.02312   
</code></pre>

<p>But I want this read as</p>

<pre><code>    0   0  0CF00400  X   8  66  7D  91        6E  22    03    0F    7D  0.021650   R  
    1   0  18EA0080  X   3  E9  FE  00                                  0.022550   R
    2   0  00000003  X   8  D5  64  22        E1  FF    FF    FF    F0  0.023120   R
</code></pre>

<p>I've tried removing <code>delim_whitespace = True</code> and replacing it with <code>delimiter = "  "</code> but that just combined the first four columns in the output shown above, but it did parse the rest of the data correctly, meaning that the rest of the columns were like the origin txt file (barring the NaN values in whitespaces).</p>

<p>I'm not sure how to proceed from here. </p>

<p>Side note: the <code>00</code> is being parsed as only <code>0</code>. Is there a way to display <code>00</code> instead?</p>

<p>It seems like your data is fixed width columns, you can try <code>pandas.read_fwf()</code>:</p>

<pre><code>from io import StringIO
import pandas as pd

df = pd.read_fwf(StringIO("""0    0CF00400 X       8  66  7D  91  6E  22  03  0F  7D       0.021650 R
0    18EA0080 X       3  E9  FE  00                           0.022550 R
0    00000003 X       8  D5  64  22  E1  FF  FF  FF  F0       0.023120 R"""), 
                 header = None, widths = [1,12,2,8,4,4,4,4,4,4,4,4,16,2])
</code></pre>

<p><a href="https://i.stack.imgur.com/G3bLR.png"><img src="https://i.stack.imgur.com/G3bLR.png" alt="enter image description here"></a></p>

python pandas