&pound; displaying in urllib2 and Beautiful Soup
<p>I'm trying to write a small web scraper in python, and I think I've run into an encoding issue. I'm trying to scrape <a href="http://www.resident-music.com/tickets" rel="nofollow">http://www.resident-music.com/tickets</a> (specifically the table on the page) - a row might look something like this -</p>

<pre><code>    &lt;tr&gt;
        &lt;td style="width:64.9%;height:11px;"&gt;
         &lt;p&gt;&lt;strong&gt;the great escape 2017&amp;nbsp; local early bird tickets, selling fast&lt;/strong&gt;&lt;/p&gt;
        &lt;/td&gt;
        &lt;td style="width:13.1%;height:11px;"&gt;
         &lt;p&gt;&lt;strong&gt;18&lt;sup&gt;th&lt;/sup&gt;&amp;ndash; 20&lt;sup&gt;th&lt;/sup&gt; may&lt;/strong&gt;&lt;/p&gt;
        &lt;/td&gt;
        &lt;td style="width:15.42%;height:11px;"&gt;
         &lt;p&gt;&lt;strong&gt;various&lt;/strong&gt;&lt;/p&gt;
        &lt;/td&gt;
        &lt;td style="width:6.58%;height:11px;"&gt;
         &lt;p&gt;&lt;strong&gt;&amp;pound;55.00&lt;/strong&gt;&lt;/p&gt;
        &lt;/td&gt;
       &lt;/tr&gt;
</code></pre>

<p>I'm essentially trying to replace the <code>&amp;pound;55.00</code> with Â£55, and any other 'non-text' nasties.</p>

<p>I've tried a few different encoding things you can go with beautifulsoup, and urllib2 - to no avail, I think I'm just doing it all wrong.</p>

<p>Thanks</p>

<p>I used <code>requests</code> for this but hopefully you can do that using <code>urllib2</code> also. So here is the code:</p>

<pre><code>#!/usr/bin/env python
# -*- coding: utf-8 -*-

import requests 
from BeautifulSoup import BeautifulSoup

soup = BeautifulSoup(requests.get('your_url').text)
chart = soup.findAll(name='tr') 
print str(chart).replace('&amp;pound;',unichr(163)) #replace '&amp;pound;' with 'Â£'
</code></pre>

<p>Now you should take the expected output!</p>

<p>Sample output:</p>

<pre><code>...
&lt;strong&gt;Â£71.50&lt;/strong&gt;&lt;/p&gt;
...
</code></pre>

<p>Anyway about the parsing you can do it with many ways, what was interesting here is: <code>print str(chart).replace('&amp;pound;',unichr(163))</code> which was quite challenging :)</p>

<blockquote>
  <p>Update</p>
</blockquote>

<p>If you want to escape more than one (or even one) characters (like dashes,pounds etc...), it would be easier/more efficient for you to use a <code>parser</code> as in Padraic's answer. Sometimes as you will also read in the comments they handle and other encoding issues. </p>

python encoding beautifulsoup urllib2