parse table using beautifulsoup in python
<p>I want to traverse through each row and capture values of td.text. However problem here is table does not have class. and all the td got same class name. I want to traverse through each row and want following output:</p>

<p>1st row)"AMERICANS SOCCER CLUB","B11EB - AMERICANS-B11EB-WARZALA","Cameron Coya","Player 228004","2016-09-10","player persistently infringes the laws of the game","C" (new line)</p>

<p>2nd row) "AVIATORS SOCCER CLUB","G12DB - AVIATORS-G12DB-REYNGOUDT","Saskia Reyes","Player 224463","2016-09-11","player/sub guilty of unsporting behavior"," C" (new line)</p>

<pre><code>&lt;div style="overflow:auto; border:1px #cccccc solid;"&gt;
&lt;table cellspacing="0" cellpadding="3" align="left" border="0" width="100%"&gt;
    &lt;tbody&gt;
        &lt;tr class="tblHeading"&gt;
            &lt;td colspan="7"&gt;AMERICANS SOCCER CLUB&lt;/td&gt;
        &lt;/tr&gt;
        &lt;tr bgcolor="#CCE4F1"&gt;
            &lt;td colspan="7"&gt;B11EB - AMERICANS-B11EB-WARZALA&lt;/td&gt; 
        &lt;/tr&gt;
        &lt;tr bgcolor="#FFFFFF"&gt;
            &lt;td width="19%" class="tdUnderLine"&gt; &amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp; Cameron Coya                                       &lt;/td&gt;
            &lt;td width="19%" class="tdUnderLine"&gt;
                Rozel, Max
            &lt;/td&gt;
            &lt;td width="06%" class="tdUnderLine"&gt; 
            09-11-2016
            &lt;/td&gt;
            &lt;td width="05%" class="tdUnderLine" align="center"&gt;         
                &lt;a href="http://www.ncsanj.com/gameRefReportPrint.cfm?gid=228004" target="_blank"&gt;228004&lt;/a&gt;    
            &lt;/td&gt;
            &lt;td width="16%" class="tdUnderLine" align="center"&gt; 
                09/10/16 02:15 PM   
            &lt;/td&gt;
            &lt;td width="30%" class="tdUnderLine"&gt;                player persistently infringes the laws of the game   &lt;/td&gt;
            &lt;td class="tdUnderLine"&gt;                Cautioned    &lt;/td&gt;
        &lt;/tr&gt;
        &lt;tr class="tblHeading"&gt;
            &lt;td colspan="7"&gt;AVIATORS SOCCER CLUB&lt;/td&gt;
        &lt;/tr&gt;
        &lt;tr bgcolor="#CCE4F1"&gt;
            &lt;td colspan="7"&gt;G12DB - AVIATORS-G12DB-REYNGOUDT&lt;/td&gt; 
        &lt;/tr&gt;
        &lt;tr bgcolor="#FBFBFB"&gt;
            &lt;td width="19%" class="tdUnderLine"&gt; &amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp; Saskia Reyes                                       &lt;/td&gt;
            &lt;td width="19%" class="tdUnderLine"&gt;
                HollaenderNardelli, Eric
            &lt;/td&gt;
            &lt;td width="06%" class="tdUnderLine"&gt; 
            09-11-2016
            &lt;/td&gt;
            &lt;td width="05%" class="tdUnderLine" align="center"&gt;         

                &lt;a href="http://www.ncsanj.com/gameRefReportPrint.cfm?gid=224463" target="_blank"&gt;224463&lt;/a&gt;    
            &lt;/td&gt;
            &lt;td width="16%" class="tdUnderLine" align="center"&gt; 
                09/11/16 06:45 PM   
            &lt;/td&gt;
            &lt;td width="30%" class="tdUnderLine"&gt;                player/sub guilty of unsporting behavior     &lt;/td&gt;
            &lt;td class="tdUnderLine"&gt;                Cautioned    &lt;/td&gt;
        &lt;/tr&gt;
        &lt;tr class="tblHeading"&gt;
            &lt;td colspan="7"&gt;BERGENFIELD SOCCER CLUB&lt;/td&gt;
        &lt;/tr&gt;
        &lt;tr bgcolor="#CCE4F1"&gt;
            &lt;td colspan="7"&gt;B11CW - BERGENFIELD-B11CW-NARVAEZ&lt;/td&gt; 
        &lt;/tr&gt;
        &lt;tr bgcolor="#FFFFFF"&gt;
            &lt;td width="19%" class="tdUnderLine"&gt; &amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp; Christian Latorre                                  &lt;/td&gt;
            &lt;td width="19%" class="tdUnderLine"&gt;
                Coyle, Kevin
            &lt;/td&gt;
            &lt;td width="06%" class="tdUnderLine"&gt; 
            09-10-2016
            &lt;/td&gt;
            &lt;td width="05%" class="tdUnderLine" align="center"&gt;         

                &lt;a href="http://www.ncsanj.com/gameRefReportPrint.cfm?gid=226294" target="_blank"&gt;226294&lt;/a&gt;    
            &lt;/td&gt;
            &lt;td width="16%" class="tdUnderLine" align="center"&gt; 

                09/10/16 11:00 AM   

            &lt;/td&gt;
            &lt;td width="30%" class="tdUnderLine"&gt;                player persistently infringes the laws of the game   &lt;/td&gt;
            &lt;td class="tdUnderLine"&gt;                Cautioned    &lt;/td&gt;
        &lt;/tr&gt;
</code></pre>

<p>I tried with following code. </p>

<pre><code>import requests
from bs4 import BeautifulSoup
import re
try:
    import urllib.request as urllib2
except ImportError:
    import urllib2

url = r"G:\Freelancer\NC Soccer\Northern Counties Soccer Association Â©.html"
page = open(url, encoding="utf8")
soup = BeautifulSoup(page.read(),"html.parser")

#tableList = soup.findAll("table")

for tr in soup.find_all("tr"):
    for td in tr.find_all("td"):
        print(td.text.strip())
</code></pre>

<p>but it is obvious that it will return text form all td and I will not able to identify particular column name or will not able to determine start of new record. I want to know </p>

<p>1) how to identify each column(because class name is same) and there are headings as well (I will appreciate if you provide code for that) </p>

<p>2) how to identify new record in such structure</p>

<pre><code>count = 0
string = ""
for td in soup.find_all("td"):
string += "\""+td.text.strip()+"\","
count +=1
if(count % 9 ==0):
    print string[:-1] + "\n\n" # string[:-1] to remove the last ","
    string = ""
</code></pre>

<p>as the table is not in the proper required format we shall just go with the td rather than going into each row then going into td in each row which complicates the work. I just used a string you can append the data into a list of lists and get process it for later use.<br>
Hope this solves your problem</p>

python web-scraping beautifulsoup