How to Combine CSV Files with Pandas (And Add Identifying Column)
<p>How do I add multiple CSV files together and an extra column to indicate where each file came from?</p>

<p>So far I have:</p>

<pre><code>import os
import pandas as pd
import glob

os.chdir('C:\...')  # path to folder where all CSVs are stored
for f, i in zip(glob.glob('*.csv'), short_list):
   df = pd.read_csv(f, header = None)
   df.index = i * len(df) 
   dfs.append(df)

all_data = pd.concat(dfs, ignore_index=True)
</code></pre>

<p>It all works well, except for the identifying column. <code>i</code> is a list of <code>strings</code> that I want to put in column A of <code>all_data</code>. One string for every row of each column. Instead it returns a lot of numbers, and gives a <code>TypeError: Index(....) must be called witha  collection of some kind</code>. </p>

<p>Expected output:</p>

<pre><code>str1 file1entry1
str1 file1entry2
str1 file1entry3
str2 file2entry1
str2 file2entry2
str2 file2entry3
</code></pre>

<p>Where <code>short_list = ['str1', 'str2', 'str3']</code>, and <code>file1entery1, file2entry2... etc</code> comes from the CSV files I already have.</p>

<p>Solution:
I wasn't able to get it all in one line like the solution suggested, however it pointed me in the right direction. </p>

<pre><code>for f zip(glob.glob('*csv')):
    df = pd.read_csv(f, header = None)
    df = df.assign(id = os.path.basename(f)) # simpler than pulling from the array. Adds file name to each line. 
    dfs.append(df)

all_data = pd.concat(dfs)
</code></pre>

<p>you can use <a href="http://pandas.pydata.org/pandas-docs/stable/generated/pandas.DataFrame.assign.html" rel="nofollow">.assign(id=i)</a> method, which will add <code>id</code> column to each parsed CSV and will populate it with the <code>i</code> value:</p>

<pre><code>df = pd.concat([pd.read_csv(f, header = None).assign(id=i)
                for f, i in zip(glob.glob('*.csv), short_list)],
               ignore_index=True)
</code></pre>

python csv pandas