Pandas slow on data frame replace
<p>I have an Excel file (.xlsx) with about 800 rows and 128 columns with pretty dense data in the grid. There are about 9500 cells that I am trying to replace the cell values of using Pandas data frame:</p>

<pre><code>xlsx = pandas.ExcelFile(filename)
frame = xlsx.parse(xlsx.sheet_names[0])
media_frame = frame[media_headers] # just get the cols that need replacing

from_filenames = get_from_filenames() # returns ~9500 filenames to replace in DF
to_filenames = get_to_filenames()

media_frame = media_frame.replace(from_filenames, to_filenames)
frame.update(media_frame)
frame.to_excel(filename)
</code></pre>

<p>The <code>replace()</code> takes 60 seconds. Any way to speed this up? This is not huge data or task, I was expecting pandas to move much faster. FYI I tried doing the same processing with same file in CSV, but the time savings was minimal (about 50 seconds on the <code>replace()</code>)</p>

<p>I got the 60 second task to complete in 10 seconds by removing <code>replace()</code> altogether and using <a href="http://pandas.pydata.org/pandas-docs/stable/generated/pandas.DataFrame.set_value.html" rel="nofollow">set_value()</a> one element at a time.</p>

python excel pandas dataframe