Urllib2: get content of html page
<p>I need to parse information from some urls:</p>

<pre><code>http://novosibirsk.baza.drom.ru/personal/actual/bulletins
http://drom.ru
http://novosibirsk.baza.drom.ru
http://moscow.drom.ru/volvo/xc70/21914186.html
http://novosibirsk.baza.drom.ru/personal/actual/bulletins
http://novosibirsk.baza.drom.ru/kolpaki-reno-r15-kubera-30227564.html
</code></pre>

<p>And I try parse from this some information</p>

<pre><code>if 'drom.ru' in url:
    req = urllib2.Request(url)
    response = urllib2.urlopen(req)
    page = response.read()
    soup = BeautifulSoup(page, 'html.parser')
</code></pre>

<p>But it returns to me empty pages.
Where can be a problem?</p>

<p>Step 1: can you access site from browser? (if no, goto step 4)</p>

<p>Step 2: can you access site from command-line such as wget, curl, etc.? (if no, goto step 4)</p>

<p>Step 3: Check for proxy issues/try a different library like <a href="http://docs.python-requests.org/en/master/" rel="nofollow">requests</a></p>

<p>Step 4: Get it working in the browser/command-line first, then go back to step 1</p>

python html urllib2