How to replace a list of special characters in a csv in python
<p>I have some csv files that may or may not contain characters like ââÃ  that are undesirable, so I want to write a simple script that will feed in a csv and feed out a csv (or its contents) with those characters replaced with more standard characters, so in the example: </p>

<pre><code>bad_chars = 'ââÃ '
good_chars = '""a'
</code></pre>

<p>The problem so far is that my code seems to produce a csv with perhaps the wrong encoding? Any help would be appreciated in making this simpler and/or making sure my output csv doesn't force an incorrect regex encoding--maybe using pandas? </p>

<p>Attempt:</p>

<pre><code>import csv, string
upload_path = sys.argv[1]
input_file = open('{}'.format(upload_path), 'rb')
upload_csv = open('{}_fixed.csv'.format(upload_path.strip('.csv')), 'wb')
data = csv.reader(input_file)
writer = csv.writer(upload_csv, quoting=csv.QUOTE_ALL)
in_chars = '\xd2\xd3'
out_chars = "''"
replace_list = string.maketrans(in_chars, out_chars)

for line in input_file:
    line = str(line)
    new_line = line.translate(replace_list)
    writer.writerow(new_line.split(','))

input_file.close()
upload_csv.close()
</code></pre>

<p>As you stamped your question with the <code>pandas</code> tag - here is a pandas solution:</p>

<pre><code>import pandas as pd

(pd.read_csv('/path/to/file.csv')
   .replace(r'RegEx_search_for_str', r'RegEx_replace_with_str', regex=True)
   .to_csv('/path/to/fixed.csv', index=False)
)
</code></pre>

python regex csv pandas