Dataframe exported to CSV turns out different with data appearing in different columns then originally was
<p>I'm trying to read a CSV as a dataframe, then sort by column and subsequently output the sorted dataframe into a new CSV. However, the problem is that my output CSV looks nothing like the sorted dataframe with data being moved to wrong columns etc etc. I suspect that the problem lies with the data as some columns are made up of long strings and might have special characters - this is because when I stripped out certain columns, the steps I took below does work. I have tried to export and reimport the dataframe in both dictionary and pickle format and it works perfectly.</p>

<p>First I read in a CSV file and then sort by a column (The csv files I used can be downloaded in the comment below (&lt;100kb in size)</p>

<pre><code>df = pd.read_csv("database.csv",encoding = "ISO-8859-1")
sorteddf = df.sort_values(by="All Comment Score")
</code></pre>

<p><a href="https://i.stack.imgur.com/0qnHB.png" rel="nofollow">This show how the dataframe looks after sorting (What I want)</a></p>

<p>Then I store my dataframe in a new CSV file and read that new CSV as a new dataframe:</p>

<pre><code>sorteddf.to_csv("test.csv")
newdf = pd.read_csv("test.csv",encoding = "ISO-8859-1")
</code></pre>

<p>However, when I read the newly outputed CSV file as a new dataframe, the columns and the data appear to be a mess:
<a href="https://i.stack.imgur.com/kriPo.png" rel="nofollow">This shows how the dataframe imported from the output CSV actually looks like</a></p>

<p>I would really appreciate it if someone could shed some light on my problem and point me in the right direction! </p>

<p>You have decoding/encoding issues. Your encoding is not in "ISO" its in 'latin-1'. Its hard to fix this unless you figure out why you are reading in your data like this. </p>

python csv pandas export-to-excel