Python Regex sub doesn't replace line breaks
<p>I have a robot that brings me an html code like this:</p>

<pre><code>&lt;div class="std"&gt;
  &lt;p&gt;CAR:
    &lt;span&gt;Onix&lt;/span&gt;
  &lt;/p&gt;
  &lt;p&gt;MODEL: LTZ&lt;/p&gt;
  &lt;p&gt;
    &lt;span&gt;COLOR:
    &lt;span&gt;Black&lt;/span&gt;
  &lt;/p&gt;
  &lt;p&gt;ACESSORIES:
    &lt;span&gt;ABS&lt;/span&gt;
  &lt;/p&gt;
  &lt;p&gt;
    &lt;span&gt;DESCRIPTION:&lt;/span&gt;
    &lt;span&gt;The Chevrolet Onix is a subcompact car launched by American automaker Chevrolet in Brazil at the 2012 SÃ£o Paulo International Motor Show[1] to succeed some versions of Chevrolet Celta. Offered initially as a five-door hatchback, a four-door sedan was launched in 2013 and called the Chevrolet Prisma.[2] The Onix is currently only sold in some South American countries part of Mercosur, including Brazil, Argentina, Colombia, Paraguay and Uruguay.&lt;/span&gt;
  &lt;/p&gt;
  &lt;p&gt;TECHNICAL DETAIL:
    &lt;span&gt;The Onix is available in three trim levels (LS, LT and LTZ) with two 4-cylinder engines, the 1.0-litre producing 78 PS (57 kW; 77 bhp) (petrol)/ 80 PS (59 kW; 79 bhp) (ethanol) and 1.4-litre 98 PS (72 kW; 97 bhp) (petrol)/106 PS (78 kW; 105 bhp) (ethanol) offering automatic or five-speed manual transmission..&lt;/span&gt;
  &lt;/p&gt;
&lt;/div&gt;
</code></pre>

<p>I applied the code below to remove the HTML tags:</p>

<pre><code>cleanr    = re.compile('&lt;.*?&gt;')
cleantext = re.sub(cleanr,'\n', html_code).strip()
</code></pre>

<p>It returns to me:</p>

<pre><code>CAR: Onix


MODEL: LTZ


COLOR:
Black



ACESSORIES:
ABS



DESCRIPTION:


The Chevrolet Onix is a subcompact car launched by American automaker Chevrolet in Brazil at the 2012 SÃ£o Paulo International Motor Show[1] to succeed some versions of Chevrolet Celta. Offered initially as a five-door hatchback, a four-door sedan was launched in 2013 and called the Chevrolet Prisma.[2] The Onix is currently only sold in some South American countries part of Mercosur, including Brazil, Argentina, Colombia, Paraguay and Uruguay.



TECHNICAL DETAIL:
The Onix is available in three trim levels (LS, LT and LTZ) with two 4-cylinder engines, the 1.0-litre producing 78 PS (57 kW; 77 bhp) (petrol)/ 80 PS (59 kW; 79 bhp) (ethanol) and 1.4-litre 98 PS (72 kW; 97 bhp) (petrol)/106 PS (78 kW; 105 bhp) (ethanol) offering automatic or five-speed manual transmission..
</code></pre>

<p>Now I need to remove the line breaks to have something like this:</p>

<pre><code>CAR: Onix
MODEL: LTZ
COLOR: Black
ACESSORIES: ABS
DESCRIPTION: The Chevrolet Onix is a subcompact car launched by American automaker Chevrolet in Brazil at the 2012 SÃ£o Paulo International Motor Show[1] to succeed some versions of Chevrolet Celta. Offered initially as a five-door hatchback, a four-door sedan was launched in 2013 and called the Chevrolet Prisma.[2] The Onix is currently only sold in some South American countries part of Mercosur, including Brazil, Argentina, Colombia, Paraguay and Uruguay.
TECHNICAL DETAIL: The Onix is available in three trim levels (LS, LT and LTZ) with two 4-cylinder engines, the 1.0-litre producing 78 PS (57 kW; 77 bhp) (petrol)/ 80 PS (59 kW; 79 bhp) (ethanol) and 1.4-litre 98 PS (72 kW; 97 bhp) (petrol)/106 PS (78 kW; 105 bhp) (ethanol) offering automatic or five-speed manual transmission..
</code></pre>

<p>I tried this code below, but it doesn't match the line breaks correctly:</p>

<pre><code>cleantext = re.sub(r':\s*[\r\n]*', ': ', cleantext)
</code></pre>

<p>I tried this another code also:</p>

<pre><code>cleantext = cleantext.replace(': \n', ': ')
</code></pre>

<p>It doesn't work also. How can I manage this?</p>

<p>I think there are two parts to your problem, 
First is to Join the string in two lines like below<br>
<code>COLOR:
Black</code></p>

<p>to<br>
<code>COLOR: black</code></p>

<p>and then remove all empty lines</p>

<p>For first part you could use replace your <code>re.sub</code> with following<br>
<code>cleantext = re.sub(r'(.*):\s*[\r\n](.*)', '\g&lt;1&gt;: \g&lt;2&gt;', cleantext)</code></p>

<p>And for removing empty lines it would be tricky to do it via re.sub so I would suggest to use
<code>cleantext = "\n".join([line for line in cleantext.split('\n') if line.strip() != ''])</code></p>

<p>This would give you answer as expected</p>

python regex line-breaks