Web scraping using regex
<p>I'm running into a wall why this code does not work, even thought it's the same code as on an online tutorial <a href="http://www.youtube.com/watch?v=5FoSwMZ4uJg&amp;t=6m22s" rel="nofollow">Python Web Scraping Tutorial 5 (Network Requests)</a>. I tried running the code also via online Python interpreter.</p>

<pre><code>import urllib
import re

htmltext = urllib.urlopen("https://www.google.com/finance?q=AAPL")

regex = '&lt;span id="ref_[^.]*_l"&gt;(.+?)&lt;/span&gt;'
pattern = re.compile(regex)
results = re.findall(pattern,htmltext)
results
</code></pre>

<p>I get:</p>

<pre><code>re.pyc in findall(pattern, string, flags)
175 
176     Empty matches are included in the result."""
--&gt; 177     return _compile(pattern, flags).findall(string)
178 
179 if sys.hexversion &gt;= 0x02020000:

TypeError: expected string or buffer 
</code></pre>

<p>Expected result(s):</p>

<pre><code>112.71
</code></pre>

<p>Help appreciated. I tried using "read()" on the url but that didn't work. According to documentation even empty results should be included. Thanks</p>

<p>If you follow the tutorial until the end :) :</p>

<pre><code>% python2                                                                                                     
&gt;&gt;&gt; import urllib
&gt;&gt;&gt; data = urllib.urlopen('https://www.google.com/finance/getprices?q=AAPL&amp;x=NASD&amp;i=10&amp;p=25m&amp;f=c&amp;auto=1').read()
&gt;&gt;&gt; print data.split()[-1]
112.71
</code></pre>

<p>Never use regex to web scrape</p>

<p>I make improvement to fetch last array element simpler</p>

python regex web-scraping typeerror