HTMLParser and BeautifulSoup not decoding HTML entities correctly
<p>I am trying to decode <code>HTML entities</code> from a section of <code>HTML</code> source code with both <code>HTMLParser</code> and <code>BeautifulSoup</code> </p>

<p>However neither seems to work completely. Namely they don't decode slashes.</p>

<p>My Python version is <code>2.7.11</code> with <code>BeautifulSoup</code> version <code>3.2.1</code></p>

<pre><code>print 'ORIGINAL STRING: %s \n' % original_url_string

#clean up
try:
    # Python 2.6-2.7
    from HTMLParser import HTMLParser
except ImportError:
    # Python 3
    from html.parser import HTMLParser

h = HTMLParser()
url_string = h.unescape(original_url_string)

print 'CLEANED WITH html.parser: %s \n' % url_string

decoded = BeautifulSoup( original_url_string,convertEntities=BeautifulSoup.HTML_ENTITIES)

print 'CLEANED WITH BeautifulSoup: %s \n' % decoded.contents
</code></pre>

<p>Gives me an output like:</p>

<pre><code>ORIGINAL STRING: api.soundcloud.com%2Ftracks%2F277561480&amp;#038;show_artwork=true&amp;#038;maxwidth=1050&amp;#038;maxheight=1000 

CLEANED WITH html.parser: api.soundcloud.com%2Ftracks%2F277561480&amp;show_artwork=true&amp;maxwidth=1050&amp;maxheight=1000 

CLEANED WITH BeautifulSoup: [u'api.soundcloud.com%2Ftracks%2F277561480&amp;show_artwork=true&amp;maxwidth=1050&amp;maxheight=1000']
</code></pre>

<p>What am I missing here? </p>

<p>Should I try to decode the entire <code>HTML</code> page before pulling out the urls?</p>

<p>Is there a better way to do this with Python?</p>

<p>Are you trying to decode the slashes from the url or the url's html?</p>

<p>If you re trying to decode the slashes, they are not <a href="https://dev.w3.org/html5/html-author/charref" rel="nofollow">HTML entities</a>, but percent-encoded characters.</p>

<p><code>urllib</code> has the method you need:</p>

<pre><code>import urllib
urllib.unquote(original_url_string)
&gt;&gt;&gt; 'api.soundcloud.com/tracks/277561480&amp;#038;show_artwork=true&amp;#038;maxwidth=1050&amp;#038;maxheight=1000'
</code></pre>

<p>If you want to decode the html, you first have to <code>get</code> it with a package like <code>requests</code> or <code>urllib</code></p>

python beautifulsoup html-entities html-parser