Python to CSV is splitting string into two columns when I want one
<p>I am scraping a page with BeautifulSoup, and part of the logic is that sometimes part of the contents of a <code>&lt;td&gt;</code> tag can have a <code>&lt;br&gt;</code> in it.  </p>

<p>So sometimes it looks like this:</p>

<pre class="lang-html prettyprint-override"><code>&lt;td class="xyz"&gt;
    text 1
    &lt;br&gt;
    text 2
&lt;/td&gt;
</code></pre>

<p>and sometimes it looks like this:</p>

<pre class="lang-html prettyprint-override"><code>&lt;td class="xyz"&gt;
    text 1
&lt;/td&gt;
</code></pre>

<p>I am looping through this and adding to an output_row list that I eventually add to a list of lists.  Whether I see the former format or the latter, I want the text to be in one cell.  </p>

<p>I've found a way to determine if I am seeing the <code>&lt;br&gt;</code> tag because the td.string shows up as none and I also know that text 2 always has 'ABC' in it.  So:</p>

<pre class="lang-python prettyprint-override"><code>    elif td.string == None:
        if 'ABC' in td.contents[2]:
            new_string = td.contents[0] + ' ' + td.contents[2]
            output_row.append(new_string)
            print(new_string)
        else:    
            #this is for another situation and it works fine
</code></pre>

<p>As I print this in a Jupyter Notebook, it shows up as "text 1 text 2" as one line.  But when I open up my CSV, it is in two different columns.  So when td.string has contents (meaning no <code>&lt;br&gt;</code> tag), text 1 shows up in one column, but when I get to the pieces that have a <code>&lt;br&gt;</code> tag, all my data gets shifted.  </p>

<p>I'm not sure why it shows up as two different strings (two columns) when I concatenate them before appending them to the list.  </p>

<p>I'm writing to file like this:</p>

<pre class="lang-python prettyprint-override"><code>with open('C:/location/file.csv', 'w',newline='') as csv_file:
    writer=csv.writer(csv_file,delimiter=',')
    #writer.writerow(headers)
    for row in output_rows:
        writer.writerow(row)

csv_file.close
</code></pre>

<p>You can handle both cases using <a href="https://www.crummy.com/software/BeautifulSoup/bs4/doc/#get-text" rel="nofollow"><code>get_text()</code></a> with "strip" and "separator":</p>

<pre><code>from bs4 import BeautifulSoup

dat="""
&lt;table&gt;
    &lt;tr&gt;
        &lt;td class="xyz"&gt;
            text 1
            &lt;br&gt;
            text 2
        &lt;/td&gt;

        &lt;td class="xyz"&gt;
            text 1
        &lt;/td&gt;
    &lt;/tr&gt;
&lt;/table&gt;
"""

soup = BeautifulSoup(dat, 'html.parser')
for td in soup.select("table &gt; tr &gt; td.xyz"):
    print(td.get_text(separator=" ", strip=True))
</code></pre>

<p>Prints:</p>

<pre><code>text 1 text 2
text 1
</code></pre>

python csv beautifulsoup