Faster way of converting Date column to weekday name in Pandas
<p>Here is my input csv file that i read via pd.read_csv()</p>

<pre><code>ProductCode,Date,Receipt,Total
x1,07/29/15,101790,17.35
x2,07/29/15,103601,8.89
x3,07/29/15,103601,8.58
x4,07/30/15,101425,11.95
x5,07/29/15,101422,1.09
x6,07/29/15,101422,0.99
x7,07/29/15,101422,3
y7,08/05/15,100358,7.29
x8,08/05/15,100358,2.6
z3,08/05/15,100358,2.99


import pandas as pd
df = pd.read_csv('product.csv')

#I have to add some columns to the data:

df['Receipt_Count'] = df.groupby(['Date','Receipt'])['Receipt'].transform('count')
df['Day_of_Week'] = pd.to_datetime(df['Date']).dt.weekday_name
</code></pre>

<p>I have around 800K of lines in my csv file. When I run the line of code for the conversion of date to weekday_name, it takes me around 2 minutes. I know that Im converting my 'Date' column to datetime first because it is treated as a string from the csv then it gets converted to its weekday equivalent.  Is there any way I can shorten the conversion time?</p>

<p>I'm fairly new to Pandas/Python, so I'm not sure if i missed something here. </p>

<p>Specifying the format of your date strings will speed up the conversion considerably:</p>

<pre><code>df['Day_of_Week'] = pd.to_datetime(df['Date'], format='%m/%d/%y').dt.weekday_name
</code></pre>

<p>Here are some benchmarks:</p>

<pre><code>import io
import pandas as pd

data = io.StringIO('''\
ProductCode,Date,Receipt,Total
x1,07/29/15,101790,17.35
x2,07/29/15,103601,8.89
x3,07/29/15,103601,8.58
x4,07/30/15,101425,11.95
x5,07/29/15,101422,1.09
x6,07/29/15,101422,0.99
x7,07/29/15,101422,3
y7,08/05/15,100358,7.29
x8,08/05/15,100358,2.6
z3,08/05/15,100358,2.99
''')

df = pd.read_csv(data)
%timeit pd.to_datetime(df['Date']).dt.weekday_name
# =&gt; 100 loops, best of 3: 2.48 ms per loop
%timeit pd.to_datetime(df['Date'], format='%m/%d/%y').dt.weekday_name
# =&gt; 1000 loops, best of 3: 507 Âµs per loop

large_df = pd.concat([df] * 1000)
%timeit pd.to_datetime(large_df['Date']).dt.weekday_name
# =&gt; 1 loop, best of 3: 1.62 s per loop
%timeit pd.to_datetime(large_df['Date'], format='%m/%d/%y').dt.weekday_name
# =&gt; 10 loops, best of 3: 45.9 ms per loop
</code></pre>

<p>Even for the small sample you provided in the OP, performance improves by a factor of 5 &mdash; for a larger dataframe it gets much, much better.</p>

python pandas