Sorting function in Pandas, returns messy data
<p>I am trying to sort data in a CSV file using sort function in Pandas using the following code. I have 229 rows in original file. But the output of sorting is 245 rows, because some of the data in a field were printed in the next row and some of the rows do not have any value.   </p>

<pre><code>sample=pd.read_csv("sample.csv" , encoding='latin-1', skipinitialspace=True)
sample_sorted = sample.sort_values(by = ['rating'])
sample_sorted.to_csv("sample_sorted.csv")
</code></pre>

<p>I think, this problem happened because in some cells  data was entered by generating new lines. For example this is the content of a cell in original file. When I sort the original file, the second line was printed in a new row and 3 rows left empty between first and second line. </p>

<pre><code>"Side effects are way to extreme. 



E-mail me if you have experianced the same things."
</code></pre>

<p>Any suggestion? Thanks !</p>

<p>You could try to remove the newlines in your problem column.</p>

<pre><code>sample=pd.read_csv("sample.csv" , encoding='latin-1', skipinitialspace=True)
sample["problem_column"] = (sample["problem_column"].
                            apply(lambda x: " ".join([word for word in x.split()])
                            )
</code></pre>

<p>and see if that helps at all. It's difficult to see why that's happening without a reproducible sample. </p>

python sorting pandas