Big files containing several columns, export all the columns by naming them with the file name and a value
<p>Thank you in advance for your answer !
I have lot of files wich contain columns.
I want to export all the columns separately in multiple files.
Moreover, I want to use the first value of each column to be the index of the file name.</p>

<p>For example. If I have the file "test_.dat" which contains 3 columns :</p>

<h2>12  54  159</h2>

<h2>2    9   87</h2>

<h2>5   99  201</h2>

<p>...
...</p>

<h2>91   1  777</h2>

<p>I want three files: "test_12.dat", "test_54.dat" &amp; "test_159.dat".
Where "test_12.dat" is :</p>

<hr>

<h2>2</h2>

<h2>5</h2>

<h2>...</h2>

<h2>...</h2>

<h2>91</h2>

<p>I know that I need to consider two loops (one for the initial files) and another one for the reading/export of the columns.
I know only how tu use append, but this is a very time consuming approach.
I would deeply appreciate your support.
Here is my try:</p>

<p>Find all ".dat" from a folder :</p>

<pre><code>for fname in glob.glob(âtest_*.datâ):
    temp=numpy.loadtxt(fname,skiprows=2)
    data.append(temp)



namefiles=glob.glob('test*.dat')
</code></pre>

<p>Append all the columns together (Very long step):</p>

<pre><code>for i in range (len(nomfichier)):
    for k in range (1,nbrechi+1):
        for j in range (points):
            ikj.append(data[i][j][k])
</code></pre>

<p>Define two variables to split the variables (points is the number of rows) </p>

<pre><code>seq2=[ikj[i:i+points] for i in range(0, len(ikj), points)]
chunks = [ikj[points*i:points*(i+1)] for i in range(len(ikj)/points + 1)]
</code></pre>

<p>Export the columns in the specific files:</p>

<pre><code>for j in range(len(nomfichier)):
    for i in range(len(seq2)/len(namefiles)):
            z=z+1
            savetxt(namefiles[j][:-4] + Â«Â _number_Â Â» + str(flattened[i]) + ".dat", zip(firstcolumn,seq2[z]))
            print(namefiles[j][:-4] + Â«Â _number_Â Â» + str(flattened[i]))
            zz.append(z)
</code></pre>

<pre><code>fps_out = []
with open('test_.dat', 'r') as fp_in:
    for line in fp_in:
        if not fps_out:
            for data in line.split():
                fps_out.append(open('test_%s.dat' % data, 'w'))
        else:
            for pos, data in enumerate(line.split()):
                fps_out[pos].write(data + '\n')

for fp in fps_out:
    fp.close()
</code></pre>

python