How to sum values of a row of a pandas dataframe efficiently
<p>I have a <code>python dataframe</code> with 1.5 million rows and 8 columns. I want combine few columns and create a new column. I know how to do this but wanted to know which one is faster and efficient. I am reproducing my code here </p>

<pre><code>import pandas as pd
import numpy as np
df=pd.Dataframe(columns=['A','B','C'],data=[[1,2,3],[4,5,6],[7,8,9]])
</code></pre>

<p>Now here is what I want to achieve </p>

<pre><code>df['D']=0.5*df['A']+0.3*df['B']+0.2*df['C']
</code></pre>

<p>The other alternative is to use the <code>apply functionality</code> of pandas</p>

<pre><code>df['D']=df.apply(lambda row: 0.5*row['A']+0.3*row['B']+0.2*row['C'])
</code></pre>

<p>I wanted to know which method takes less time when we have 1.5 millon rows and have to combine 8 columns</p>

<p>Using @jezrael's setup</p>

<pre><code>df=pd.DataFrame(columns=['A','B','C'],data=[[1,2,3],[4,5,6],[7,8,9]])
df = pd.concat([df]*30000).reset_index(drop=True)
</code></pre>

<p>Far more efficient to use a <code>dot</code> product.</p>

<pre><code>np.array([[.5, .3, .2]]).dot(df.values.T).T
</code></pre>

<hr>

<h3>Timing</h3>

<p><a href="http://i.stack.imgur.com/MGbsy.png" rel="nofollow"><img src="http://i.stack.imgur.com/MGbsy.png" alt="enter image description here"></a></p>

python performance pandas numpy vectorization