Why is pandas map slower than list comprehension
<p>Does someone know why pandas/numpy map is slower then list comprehension?
I thought I could optimize my code replacing the list comprehensions by map. Since map doesn't need the list append operation.</p>

<p>Here is one test:</p>

<pre><code>df = pd.DataFrame(range(100000))
</code></pre>

<p><strong>List comprehension:</strong></p>

<pre><code>%timeit -n 10 df["A"] = [x for x in df[0]]

#10 loops, best of 3: 550 ms per loop
</code></pre>

<p><strong>Pandas map</strong>   </p>

<pre><code>%timeit -n 10 df["A"] = df[0].map(lambda x: x)

#10 loops, best of 3: 797 ms per loop
</code></pre>

<p><strong>Update</strong>
based on comment bellow - list comprehension and map calling same function f, list comprehension faster</p>

<pre><code>def f(x):
    return x

%timeit -n 100 df["A"] = df[0].map(f)

#100 loops, best of 3: 475 ms per loop

%timeit -n 100 df["A"] = [f(x) for x in df[0]]

#100 loops, best of 3: 399 ms per loop
</code></pre>

<p>Here are my results:</p>

<p>list comprehension:</p>

<pre><code>In [33]: %timeit df["A"] = [x for x in df[0]]
10 loops, best of 3: 72.6 ms per loop
</code></pre>

<p>simple column assignment:</p>

<pre><code>In [34]: %timeit df["A"] = df[0]
The slowest run took 5.75 times longer than the fastest. This could mean that an intermediate result is being cached.
1000 loops, best of 3: 661 Âµs per loop
</code></pre>

<p>using <code>.map()</code> method:</p>

<pre><code>In [35]: map_df = pd.Series(np.random.randint(0, 10**6, 100000))

In [36]: %timeit df["A"] = df[0].map(map_df)
10 loops, best of 3: 19.8 ms per loop
</code></pre>

python performance pandas list-comprehension