Urllib2: get content of html page
<p>I need to parse information from some urls:</p>

<pre><code>http://novosibirsk.baza.drom.ru/personal/actual/bulletins
http://drom.ru
http://novosibirsk.baza.drom.ru
http://moscow.drom.ru/volvo/xc70/21914186.html
http://novosibirsk.baza.drom.ru/personal/actual/bulletins
http://novosibirsk.baza.drom.ru/kolpaki-reno-r15-kubera-30227564.html
</code></pre>

<p>And I try parse from this some information</p>

<pre><code>if 'drom.ru' in url:
    req = urllib2.Request(url)
    response = urllib2.urlopen(req)
    page = response.read()
    soup = BeautifulSoup(page, 'html.parser')
</code></pre>

<p>But it returns to me empty pages.
Where can be a problem?</p>

<p>Using <code>requests</code> will make it more easier. If you dont have <code>requests</code> module installed, try to install it by <code>pip install requests</code></p>

<pre><code>import requests

if 'drom.ru' in url:
    r = requests.get(url)
    soup = BeautifulSoup(r.content, 'html.parser') # lxml works faster than html.parser
</code></pre>

python html urllib2