Parsing NBA reference with python beautiful soup
<p>So I'm trying to scrape out the miscellaneous stats table from this site <a href="http://www.basketball-reference.com/leagues/NBA_2016.html" rel="nofollow">http://www.basketball-reference.com/leagues/NBA_2016.html</a> using python and beautiful soup.  This is the basic code so far I just want to see if it is even reading the table but when I do print table I just get none.</p>

<pre><code>from bs4 import BeautifulSoup
import requests
import pandas as pd 

url = "http://www.basketball-reference.com/leagues/NBA_2016.html"
r = requests.get(url)
data = r.text
soup = BeautifulSoup(data)

table = soup.find('table', id='misc_stats')
print table
</code></pre>

<p>When I inspect the html on the webpage itself, the table that I want appears with this symbol in front <code>&lt;!--</code> and the html text is green for the portion.  What can I do? </p>

<p><code>&lt;!--</code> is the start of a comment and <code>--&gt;</code> is the end in html so just remove the comments before you parse it:</p>

<pre><code>from bs4 import BeautifulSoup
import requests
comm = re.compile("&lt;!--|--&gt;")

html = requests.get("http://www.basketball-reference.com/leagues/NBA_2016.html").content
cleaned_soup = BeautifulSoup(re.sub("&lt;!--|--&gt;","", html))


tableStats = cleaned_soup.find('table', {'id':'team_stats'})

print(tableStats)
</code></pre>

python html parsing beautifulsoup