Web-Scraping with Python on Canopy
<p>I'm having trouble with this line of code in which I want to print the 4 stock prices for the companies listed. My issue is that, while there are no errors when I run it, the code only prints out empty brackets where the stock prices should go. This is the source of my confusion. </p>

<pre><code>import urllib2
import re

symbolslist = ["aapl","spy","goog","nflx"]
i = 0

while i&lt;len(symbolslist):
    url = "http://money.cnn.com/quote/quote.html?symb=' +symbolslist[i] + '"
    htmlfile = urllib2.urlopen(url)
    htmltext = htmlfile.read()
    regex = '&lt;span stream='+symbolslist[i]+' streamformat="ToHundredth" streamfeed="SunGard"&gt;(.+?)&lt;/span&gt;'
    pattern = re.compile(regex)
    price = re.findall(pattern,htmltext)
    print "the price of", symbolslist[i], " is ", price
    i+=1
</code></pre>

<p>Because you don't pass the variable:</p>

<pre><code> url = "http://money.cnn.com/quote/quote.html?symb=' +symbolslist[i] + '"
                                                         ^^^^^
                                                      a string not the list element
</code></pre>

<p>Use <em>str.format</em>:</p>

<pre><code>url = "http://money.cnn.com/quote/quote.html?symb={}".format(symbolslist[i])
</code></pre>

<p>Also you can iterate directly over the list, no need for a while loop, never <a href="http://stackoverflow.com/questions/1732348/regex-match-open-tags-except-xhtml-self-contained-tags/1732454#1732454">parse html with a regex</a>, use a html parse like <a href="https://www.crummy.com/software/BeautifulSoup/bs4/doc/" rel="nofollow">bs4</a> and your regex is also wrong. There is no <code>stream="aapl"</code> etc.. what you want is the span where <code>streamformat="ToHundredth"</code> and <code>streamfeed="SunGard"</code>;</p>

<pre><code>import urllib2
from bs4 import BeautifulSoup

symbolslist = ["aapl","spy","goog","nflx"]


for symbol in symbolslist:
    url = "http://money.cnn.com/quote/quote.html?symb={}".format(symbol)
    htmlfile = urllib2.urlopen(url)
    soup = BeautifulSoup(htmlfile.read())
    price = soup.find("span",streamformat="ToHundredth", streamfeed="SunGard").text
    print "the price of {} is {}".format(symbol,price)
</code></pre>

<p>You can see if we run the code:</p>

<pre><code>In [1]: import urllib2

In [2]: from bs4 import BeautifulSoup

In [3]: symbols_list = ["aapl", "spy", "goog", "nflx"]

In [4]: for symbol in symbols_list:
   ...:         url = "http://money.cnn.com/quote/quote.html?symb={}".format(symbol)
   ...:         htmlfile = urllib2.urlopen(url)
   ...:         soup = BeautifulSoup(htmlfile.read(), "html.parser")
   ...:         price = soup.find("span",streamformat="ToHundredth", streamfeed="SunGard").text
   ...:         print "the price of {} is {}".format(symbol,price)
   ...:     
the price of aapl is 115.57
the price of spy is 215.28
the price of goog is 771.76
the price of nflx is 97.34
</code></pre>

<p>We get what you want.</p>

python web-scraping canopy