Scrapping row of info underneath a table header when the html tags for the row are not nested under the header tag
<p>I'm trying to scrape a <a href="https://wd.kyepsb.net/EPSB.WebApps/KECI/view_data.aspx?id=37161" rel="nofollow">table</a> but I've run into a bit of a snag. I want to make sure that the data underneath each header (ex. <code>Cert Issued (30)</code>) is grouped with the corresponding header. </p>

<p>The problem arises when I'm trying to work with the html below.</p>

<pre><code>&lt;tr&gt;
                        &lt;td&gt;                        
                            &lt;table class="EPSBResultGrid" cellspacing="0" rules="cols" border="1" style="border-color:DarkGray;border-collapse:collapse;"&gt;
    &lt;tbody&gt;&lt;tr class="EPSBResultGridHeader"&gt;
        &lt;th scope="col"&gt;Cred&lt;/th&gt;&lt;th scope="col"&gt;Description&lt;/th&gt;&lt;th scope="col"&gt;Effective&lt;/th&gt;&lt;th scope="col"&gt;Expiration&lt;/th&gt;&lt;th scope="col"&gt;Restricted To&lt;/th&gt;
    &lt;/tr&gt;&lt;tr class="EPSBResultGridHeader"&gt;
        &lt;td colspan="9" style="border-width:1px;border-style:solid;font-weight:bold;"&gt;Do Not Print (00)&lt;/td&gt;
    &lt;/tr&gt;&lt;tr class="EPSBResultGridItem"&gt;
        &lt;td&gt;&lt;span id="ctl00_ContentPlaceHolder1_ctl00_ctl03_CRED_CODE"&gt;RANK1&lt;/span&gt;&lt;/td&gt;&lt;td&gt;&lt;span id="ctl00_ContentPlaceHolder1_ctl00_ctl03_CRED_DESC"&gt;Rank I&lt;/span&gt;&lt;/td&gt;&lt;td&gt;&lt;nobr&gt;&lt;span class="" id="ctl00$ContentPlaceHolder1$ctl00$ctl03$EFF_DATE_txtDateMM"&gt;07&lt;input type="hidden" id="ctl00_ContentPlaceHolder1_ctl00_ctl03_EFF_DATE_txtDateMM" name="ctl00_ContentPlaceHolder1_ctl00_ctl03_EFF_DATE_txtDateMM" value="07"&gt;&lt;/span&gt;-&lt;span class="" id="ctl00$ContentPlaceHolder1$ctl00$ctl03$EFF_DATE_txtDateDD"&gt;01&lt;input type="hidden" id="ctl00_ContentPlaceHolder1_ctl00_ctl03_EFF_DATE_txtDateDD" name="ctl00_ContentPlaceHolder1_ctl00_ctl03_EFF_DATE_txtDateDD" value="01"&gt;&lt;/span&gt;-&lt;span class="" id="ctl00$ContentPlaceHolder1$ctl00$ctl03$EFF_DATE_txtDateYYYY"&gt;2000&lt;input type="hidden" id="ctl00_ContentPlaceHolder1_ctl00_ctl03_EFF_DATE_txtDateYYYY" name="ctl00_ContentPlaceHolder1_ctl00_ctl03_EFF_DATE_txtDateYYYY" value="2000"&gt;&lt;/span&gt;&lt;/nobr&gt;&lt;span id="ctl00_ContentPlaceHolder1_ctl00_ctl03_ctl01" style="color:Red;display:none;"&gt;You must enter a day.&lt;/span&gt;&lt;span id="ctl00_ContentPlaceHolder1_ctl00_ctl03_ctl02" style="color:Red;display:none;"&gt;You must enter a month.&lt;/span&gt;&lt;span id="ctl00_ContentPlaceHolder1_ctl00_ctl03_ctl03" style="color:Red;display:none;"&gt;You must enter a year.&lt;/span&gt;&lt;span id="ctl00_ContentPlaceHolder1_ctl00_ctl03_ctl04" evaluationfunction="cb_verifydate_ctl00_ContentPlaceHolder1_ctl00_ctl03_ctl04" style="color:Red;visibility:hidden;"&gt;Invalid Date.&lt;/span&gt;&lt;/td&gt;&lt;td&gt;&lt;nobr&gt;&lt;span class="" id="ctl00$ContentPlaceHolder1$ctl00$ctl03$EXP_DATE_txtDateMM"&gt;06&lt;input type="hidden" id="ctl00_ContentPlaceHolder1_ctl00_ctl03_EXP_DATE_txtDateMM" name="ctl00_ContentPlaceHolder1_ctl00_ctl03_EXP_DATE_txtDateMM" value="06"&gt;&lt;/span&gt;-&lt;span class="" id="ctl00$ContentPlaceHolder1$ctl00$ctl03$EXP_DATE_txtDateDD"&gt;30&lt;input type="hidden" id="ctl00_ContentPlaceHolder1_ctl00_ctl03_EXP_DATE_txtDateDD" name="ctl00_ContentPlaceHolder1_ctl00_ctl03_EXP_DATE_txtDateDD" value="30"&gt;&lt;/span&gt;-&lt;span class="" id="ctl00$ContentPlaceHolder1$ctl00$ctl03$EXP_DATE_txtDateYYYY"&gt;2020&lt;input type="hidden" id="ctl00_ContentPlaceHolder1_ctl00_ctl03_EXP_DATE_txtDateYYYY" name="ctl00_ContentPlaceHolder1_ctl00_ctl03_EXP_DATE_txtDateYYYY" value="2020"&gt;&lt;/span&gt;&lt;/nobr&gt;&lt;span id="ctl00_ContentPlaceHolder1_ctl00_ctl03_ctl06" style="color:Red;display:none;"&gt;You must enter a day.&lt;/span&gt;&lt;span id="ctl00_ContentPlaceHolder1_ctl00_ctl03_ctl07" style="color:Red;display:none;"&gt;You must enter a month.&lt;/span&gt;&lt;span id="ctl00_ContentPlaceHolder1_ctl00_ctl03_ctl08" style="color:Red;display:none;"&gt;You must enter a year.&lt;/span&gt;&lt;span id="ctl00_ContentPlaceHolder1_ctl00_ctl03_ctl09" evaluationfunction="cb_verifydate_ctl00_ContentPlaceHolder1_ctl00_ctl03_ctl09" style="color:Red;visibility:hidden;"&gt;Invalid Date.&lt;/span&gt;&lt;/td&gt;&lt;td&gt;&lt;span id="ctl00_ContentPlaceHolder1_ctl00_ctl03_ORG_NAME"&gt;&lt;/span&gt;&lt;/td&gt;
    &lt;/tr&gt;&lt;tr class="EPSBResultGridHeader"&gt;
        &lt;td colspan="9" style="border-width:1px;border-style:solid;font-weight:bold;"&gt;Cert Issued (30)&lt;/td&gt;
    &lt;/tr&gt;&lt;tr class="EPSBResultGridAlternatingItem"&gt;
        &lt;td&gt;&lt;span id="ctl00_ContentPlaceHolder1_ctl00_ctl05_CRED_CODE"&gt;G20&lt;/span&gt;&lt;/td&gt;&lt;td&gt;&lt;span id="ctl00_ContentPlaceHolder1_ctl00_ctl05_CRED_DESC"&gt;Middle School Teaching Field: Social Studies&lt;/span&gt;&lt;/td&gt;&lt;td&gt;&lt;nobr&gt;&lt;span class="" id="ctl00$ContentPlaceHolder1$ctl00$ctl05$EFF_DATE_txtDateMM"&gt;07&lt;input type="hidden" id="ctl00_ContentPlaceHolder1_ctl00_ctl05_EFF_DATE_txtDateMM" name="ctl00_ContentPlaceHolder1_ctl00_ctl05_EFF_DATE_txtDateMM" value="07"&gt;&lt;/span&gt;-&lt;span class="" id="ctl00$ContentPlaceHolder1$ctl00$ctl05$EFF_DATE_txtDateDD"&gt;01&lt;input type="hidden" id="ctl00_ContentPlaceHolder1_ctl00_ctl05_EFF_DATE_txtDateDD" name="ctl00_ContentPlaceHolder1_ctl00_ctl05_EFF_DATE_txtDateDD" value="01"&gt;&lt;/span&gt;-&lt;span class="" id="ctl00$ContentPlaceHolder1$ctl00$ctl05$EFF_DATE_txtDateYYYY"&gt;1995&lt;input type="hidden" id="ctl00_ContentPlaceHolder1_ctl00_ctl05_EFF_DATE_txtDateYYYY" name="ctl00_ContentPlaceHolder1_ctl00_ctl05_EFF_DATE_txtDateYYYY" value="1995"&gt;&lt;/span&gt;&lt;/nobr&gt;&lt;span id="ctl00_ContentPlaceHolder1_ctl00_ctl05_ctl01" style="color:Red;display:none;"&gt;You must enter a day.&lt;/span&gt;&lt;span id="ctl00_ContentPlaceHolder1_ctl00_ctl05_ctl02" style="color:Red;display:none;"&gt;You must enter a month.&lt;/span&gt;&lt;span id="ctl00_ContentPlaceHolder1_ctl00_ctl05_ctl03" style="color:Red;display:none;"&gt;You must enter a year.&lt;/span&gt;&lt;span id="ctl00_ContentPlaceHolder1_ctl00_ctl05_ctl04" evaluationfunction="cb_verifydate_ctl00_ContentPlaceHolder1_ctl00_ctl05_ctl04" style="color:Red;visibility:hidden;"&gt;Invalid Date.&lt;/span&gt;&lt;/td&gt;&lt;td&gt;&lt;nobr&gt;&lt;span class="" id="ctl00$ContentPlaceHolder1$ctl00$ctl05$EXP_DATE_txtDateMM"&gt;06&lt;input type="hidden" id="ctl00_ContentPlaceHolder1_ctl00_ctl05_EXP_DATE_txtDateMM" name="ctl00_ContentPlaceHolder1_ctl00_ctl05_EXP_DATE_txtDateMM" value="06"&gt;&lt;/span&gt;-&lt;span class="" id="ctl00$ContentPlaceHolder1$ctl00$ctl05$EXP_DATE_txtDateDD"&gt;30&lt;input type="hidden" id="ctl00_ContentPlaceHolder1_ctl00_ctl05_EXP_DATE_txtDateDD" name="ctl00_ContentPlaceHolder1_ctl00_ctl05_EXP_DATE_txtDateDD" value="30"&gt;&lt;/span&gt;-&lt;span class="" id="ctl00$ContentPlaceHolder1$ctl00$ctl05$EXP_DATE_txtDateYYYY"&gt;2020&lt;input type="hidden" id="ctl00_ContentPlaceHolder1_ctl00_ctl05_EXP_DATE_txtDateYYYY" name="ctl00_ContentPlaceHolder1_ctl00_ctl05_EXP_DATE_txtDateYYYY" value="2020"&gt;&lt;/span&gt;&lt;/nobr&gt;&lt;span id="ctl00_ContentPlaceHolder1_ctl00_ctl05_ctl06" style="color:Red;display:none;"&gt;You must enter a day.&lt;/span&gt;&lt;span id="ctl00_ContentPlaceHolder1_ctl00_ctl05_ctl07" style="color:Red;display:none;"&gt;You must enter a month.&lt;/span&gt;&lt;span id="ctl00_ContentPlaceHolder1_ctl00_ctl05_ctl08" style="color:Red;display:none;"&gt;You must enter a year.&lt;/span&gt;&lt;span id="ctl00_ContentPlaceHolder1_ctl00_ctl05_ctl09" evaluationfunction="cb_verifydate_ctl00_ContentPlaceHolder1_ctl00_ctl05_ctl09" style="color:Red;visibility:hidden;"&gt;Invalid Date.&lt;/span&gt;&lt;/td&gt;&lt;td&gt;&lt;span id="ctl00_ContentPlaceHolder1_ctl00_ctl05_ORG_NAME"&gt;&lt;/span&gt;&lt;/td&gt;
    &lt;/tr&gt;&lt;tr class="EPSBResultGridItem"&gt;
        &lt;td&gt;&lt;span id="ctl00_ContentPlaceHolder1_ctl00_ctl06_CRED_CODE"&gt;G71&lt;/span&gt;&lt;/td&gt;&lt;td&gt;&lt;span id="ctl00_ContentPlaceHolder1_ctl00_ctl06_CRED_DESC"&gt;Middle School Teaching Field: Mathematics&lt;/span&gt;&lt;/td&gt;&lt;td&gt;&lt;nobr&gt;&lt;span class="" id="ctl00$ContentPlaceHolder1$ctl00$ctl06$EFF_DATE_txtDateMM"&gt;07&lt;input type="hidden" id="ctl00_ContentPlaceHolder1_ctl00_ctl06_EFF_DATE_txtDateMM" name="ctl00_ContentPlaceHolder1_ctl00_ctl06_EFF_DATE_txtDateMM" value="07"&gt;&lt;/span&gt;-&lt;span class="" id="ctl00$ContentPlaceHolder1$ctl00$ctl06$EFF_DATE_txtDateDD"&gt;01&lt;input type="hidden" id="ctl00_ContentPlaceHolder1_ctl00_ctl06_EFF_DATE_txtDateDD" name="ctl00_ContentPlaceHolder1_ctl00_ctl06_EFF_DATE_txtDateDD" value="01"&gt;&lt;/span&gt;-&lt;span class="" id="ctl00$ContentPlaceHolder1$ctl00$ctl06$EFF_DATE_txtDateYYYY"&gt;1995&lt;input type="hidden" id="ctl00_ContentPlaceHolder1_ctl00_ctl06_EFF_DATE_txtDateYYYY" name="ctl00_ContentPlaceHolder1_ctl00_ctl06_EFF_DATE_txtDateYYYY" value="1995"&gt;&lt;/span&gt;&lt;/nobr&gt;&lt;span id="ctl00_ContentPlaceHolder1_ctl00_ctl06_ctl01" style="color:Red;display:none;"&gt;You must enter a day.&lt;/span&gt;&lt;span id="ctl00_ContentPlaceHolder1_ctl00_ctl06_ctl02" style="color:Red;display:none;"&gt;You must enter a month.&lt;/span&gt;&lt;span id="ctl00_ContentPlaceHolder1_ctl00_ctl06_ctl03" style="color:Red;display:none;"&gt;You must enter a year.&lt;/span&gt;&lt;span id="ctl00_ContentPlaceHolder1_ctl00_ctl06_ctl04" evaluationfunction="cb_verifydate_ctl00_ContentPlaceHolder1_ctl00_ctl06_ctl04" style="color:Red;visibility:hidden;"&gt;Invalid Date.&lt;/span&gt;&lt;/td&gt;&lt;td&gt;&lt;nobr&gt;&lt;span class="" id="ctl00$ContentPlaceHolder1$ctl00$ctl06$EXP_DATE_txtDateMM"&gt;06&lt;input type="hidden" id="ctl00_ContentPlaceHolder1_ctl00_ctl06_EXP_DATE_txtDateMM" name="ctl00_ContentPlaceHolder1_ctl00_ctl06_EXP_DATE_txtDateMM" value="06"&gt;&lt;/span&gt;-&lt;span class="" id="ctl00$ContentPlaceHolder1$ctl00$ctl06$EXP_DATE_txtDateDD"&gt;30&lt;input type="hidden" id="ctl00_ContentPlaceHolder1_ctl00_ctl06_EXP_DATE_txtDateDD" name="ctl00_ContentPlaceHolder1_ctl00_ctl06_EXP_DATE_txtDateDD" value="30"&gt;&lt;/span&gt;-&lt;span class="" id="ctl00$ContentPlaceHolder1$ctl00$ctl06$EXP_DATE_txtDateYYYY"&gt;2020&lt;input type="hidden" id="ctl00_ContentPlaceHolder1_ctl00_ctl06_EXP_DATE_txtDateYYYY" name="ctl00_ContentPlaceHolder1_ctl00_ctl06_EXP_DATE_txtDateYYYY" value="2020"&gt;&lt;/span&gt;&lt;/nobr&gt;&lt;span id="ctl00_ContentPlaceHolder1_ctl00_ctl06_ctl06" style="color:Red;display:none;"&gt;You must enter a day.&lt;/span&gt;&lt;span id="ctl00_ContentPlaceHolder1_ctl00_ctl06_ctl07" style="color:Red;display:none;"&gt;You must enter a month.&lt;/span&gt;&lt;span id="ctl00_ContentPlaceHolder1_ctl00_ctl06_ctl08" style="color:Red;display:none;"&gt;You must enter a year.&lt;/span&gt;&lt;span id="ctl00_ContentPlaceHolder1_ctl00_ctl06_ctl09" evaluationfunction="cb_verifydate_ctl00_ContentPlaceHolder1_ctl00_ctl06_ctl09" style="color:Red;visibility:hidden;"&gt;Invalid Date.&lt;/span&gt;&lt;/td&gt;&lt;td&gt;&lt;span id="ctl00_ContentPlaceHolder1_ctl00_ctl06_ORG_NAME"&gt;&lt;/span&gt;&lt;/td&gt;
    &lt;/tr&gt;&lt;tr class="EPSBResultGridAlternatingItem"&gt;
        &lt;td&gt;&lt;span id="ctl00_ContentPlaceHolder1_ctl00_ctl07_CRED_CODE"&gt;PCS&lt;/span&gt;&lt;/td&gt;&lt;td&gt;&lt;span id="ctl00_ContentPlaceHolder1_ctl00_ctl07_CRED_DESC"&gt;Provisional Certificate For Guidance Counselor, Secondary Grades 5-12&lt;/span&gt;&lt;/td&gt;&lt;td&gt;&lt;nobr&gt;&lt;span class="" id="ctl00$ContentPlaceHolder1$ctl00$ctl07$EFF_DATE_txtDateMM"&gt;01&lt;input type="hidden" id="ctl00_ContentPlaceHolder1_ctl00_ctl07_EFF_DATE_txtDateMM" name="ctl00_ContentPlaceHolder1_ctl00_ctl07_EFF_DATE_txtDateMM" value="01"&gt;&lt;/span&gt;-&lt;span class="" id="ctl00$ContentPlaceHolder1$ctl00$ctl07$EFF_DATE_txtDateDD"&gt;01&lt;input type="hidden" id="ctl00_ContentPlaceHolder1_ctl00_ctl07_EFF_DATE_txtDateDD" name="ctl00_ContentPlaceHolder1_ctl00_ctl07_EFF_DATE_txtDateDD" value="01"&gt;&lt;/span&gt;-&lt;span class="" id="ctl00$ContentPlaceHolder1$ctl00$ctl07$EFF_DATE_txtDateYYYY"&gt;2016&lt;input type="hidden" id="ctl00_ContentPlaceHolder1_ctl00_ctl07_EFF_DATE_txtDateYYYY" name="ctl00_ContentPlaceHolder1_ctl00_ctl07_EFF_DATE_txtDateYYYY" value="2016"&gt;&lt;/span&gt;&lt;/nobr&gt;&lt;span id="ctl00_ContentPlaceHolder1_ctl00_ctl07_ctl01" style="color:Red;display:none;"&gt;You must enter a day.&lt;/span&gt;&lt;span id="ctl00_ContentPlaceHolder1_ctl00_ctl07_ctl02" style="color:Red;display:none;"&gt;You must enter a month.&lt;/span&gt;&lt;span id="ctl00_ContentPlaceHolder1_ctl00_ctl07_ctl03" style="color:Red;display:none;"&gt;You must enter a year.&lt;/span&gt;&lt;span id="ctl00_ContentPlaceHolder1_ctl00_ctl07_ctl04" evaluationfunction="cb_verifydate_ctl00_ContentPlaceHolder1_ctl00_ctl07_ctl04" style="color:Red;visibility:hidden;"&gt;Invalid Date.&lt;/span&gt;&lt;/td&gt;&lt;td&gt;&lt;nobr&gt;&lt;span class="" id="ctl00$ContentPlaceHolder1$ctl00$ctl07$EXP_DATE_txtDateMM"&gt;06&lt;input type="hidden" id="ctl00_ContentPlaceHolder1_ctl00_ctl07_EXP_DATE_txtDateMM" name="ctl00_ContentPlaceHolder1_ctl00_ctl07_EXP_DATE_txtDateMM" value="06"&gt;&lt;/span&gt;-&lt;span class="" id="ctl00$ContentPlaceHolder1$ctl00$ctl07$EXP_DATE_txtDateDD"&gt;30&lt;input type="hidden" id="ctl00_ContentPlaceHolder1_ctl00_ctl07_EXP_DATE_txtDateDD" name="ctl00_ContentPlaceHolder1_ctl00_ctl07_EXP_DATE_txtDateDD" value="30"&gt;&lt;/span&gt;-&lt;span class="" id="ctl00$ContentPlaceHolder1$ctl00$ctl07$EXP_DATE_txtDateYYYY"&gt;2020&lt;input type="hidden" id="ctl00_ContentPlaceHolder1_ctl00_ctl07_EXP_DATE_txtDateYYYY" name="ctl00_ContentPlaceHolder1_ctl00_ctl07_EXP_DATE_txtDateYYYY" value="2020"&gt;&lt;/span&gt;&lt;/nobr&gt;&lt;span id="ctl00_ContentPlaceHolder1_ctl00_ctl07_ctl06" style="color:Red;display:none;"&gt;You must enter a day.&lt;/span&gt;&lt;span id="ctl00_ContentPlaceHolder1_ctl00_ctl07_ctl07" style="color:Red;display:none;"&gt;You must enter a month.&lt;/span&gt;&lt;span id="ctl00_ContentPlaceHolder1_ctl00_ctl07_ctl08" style="color:Red;display:none;"&gt;You must enter a year.&lt;/span&gt;&lt;span id="ctl00_ContentPlaceHolder1_ctl00_ctl07_ctl09" evaluationfunction="cb_verifydate_ctl00_ContentPlaceHolder1_ctl00_ctl07_ctl09" style="color:Red;visibility:hidden;"&gt;Invalid Date.&lt;/span&gt;&lt;/td&gt;&lt;td&gt;&lt;span id="ctl00_ContentPlaceHolder1_ctl00_ctl07_ORG_NAME"&gt;&lt;/span&gt;&lt;/td&gt;
    &lt;/tr&gt;&lt;tr class="EPSBResultGridItem"&gt;
        &lt;td&gt;&lt;span id="ctl00_ContentPlaceHolder1_ctl00_ctl08_CRED_CODE"&gt;PMBF&lt;/span&gt;&lt;/td&gt;&lt;td&gt;&lt;span id="ctl00_ContentPlaceHolder1_ctl00_ctl08_CRED_DESC"&gt;Provisional Certificate For Teaching In The Middle Grades 5-8 (And For Other Assignments As Identified By Kentucky Program Of Studies)&lt;/span&gt;&lt;/td&gt;&lt;td&gt;&lt;nobr&gt;&lt;span class="" id="ctl00$ContentPlaceHolder1$ctl00$ctl08$EFF_DATE_txtDateMM"&gt;07&lt;input type="hidden" id="ctl00_ContentPlaceHolder1_ctl00_ctl08_EFF_DATE_txtDateMM" name="ctl00_ContentPlaceHolder1_ctl00_ctl08_EFF_DATE_txtDateMM" value="07"&gt;&lt;/span&gt;-&lt;span class="" id="ctl00$ContentPlaceHolder1$ctl00$ctl08$EFF_DATE_txtDateDD"&gt;01&lt;input type="hidden" id="ctl00_ContentPlaceHolder1_ctl00_ctl08_EFF_DATE_txtDateDD" name="ctl00_ContentPlaceHolder1_ctl00_ctl08_EFF_DATE_txtDateDD" value="01"&gt;&lt;/span&gt;-&lt;span class="" id="ctl00$ContentPlaceHolder1$ctl00$ctl08$EFF_DATE_txtDateYYYY"&gt;2015&lt;input type="hidden" id="ctl00_ContentPlaceHolder1_ctl00_ctl08_EFF_DATE_txtDateYYYY" name="ctl00_ContentPlaceHolder1_ctl00_ctl08_EFF_DATE_txtDateYYYY" value="2015"&gt;&lt;/span&gt;&lt;/nobr&gt;&lt;span id="ctl00_ContentPlaceHolder1_ctl00_ctl08_ctl01" style="color:Red;display:none;"&gt;You must enter a day.&lt;/span&gt;&lt;span id="ctl00_ContentPlaceHolder1_ctl00_ctl08_ctl02" style="color:Red;display:none;"&gt;You must enter a month.&lt;/span&gt;&lt;span id="ctl00_ContentPlaceHolder1_ctl00_ctl08_ctl03" style="color:Red;display:none;"&gt;You must enter a year.&lt;/span&gt;&lt;span id="ctl00_ContentPlaceHolder1_ctl00_ctl08_ctl04" evaluationfunction="cb_verifydate_ctl00_ContentPlaceHolder1_ctl00_ctl08_ctl04" style="color:Red;visibility:hidden;"&gt;Invalid Date.&lt;/span&gt;&lt;/td&gt;&lt;td&gt;&lt;nobr&gt;&lt;span class="" id="ctl00$ContentPlaceHolder1$ctl00$ctl08$EXP_DATE_txtDateMM"&gt;06&lt;input type="hidden" id="ctl00_ContentPlaceHolder1_ctl00_ctl08_EXP_DATE_txtDateMM" name="ctl00_ContentPlaceHolder1_ctl00_ctl08_EXP_DATE_txtDateMM" value="06"&gt;&lt;/span&gt;-&lt;span class="" id="ctl00$ContentPlaceHolder1$ctl00$ctl08$EXP_DATE_txtDateDD"&gt;30&lt;input type="hidden" id="ctl00_ContentPlaceHolder1_ctl00_ctl08_EXP_DATE_txtDateDD" name="ctl00_ContentPlaceHolder1_ctl00_ctl08_EXP_DATE_txtDateDD" value="30"&gt;&lt;/span&gt;-&lt;span class="" id="ctl00$ContentPlaceHolder1$ctl00$ctl08$EXP_DATE_txtDateYYYY"&gt;2020&lt;input type="hidden" id="ctl00_ContentPlaceHolder1_ctl00_ctl08_EXP_DATE_txtDateYYYY" name="ctl00_ContentPlaceHolder1_ctl00_ctl08_EXP_DATE_txtDateYYYY" value="2020"&gt;&lt;/span&gt;&lt;/nobr&gt;&lt;span id="ctl00_ContentPlaceHolder1_ctl00_ctl08_ctl06" style="color:Red;display:none;"&gt;You must enter a day.&lt;/span&gt;&lt;span id="ctl00_ContentPlaceHolder1_ctl00_ctl08_ctl07" style="color:Red;display:none;"&gt;You must enter a month.&lt;/span&gt;&lt;span id="ctl00_ContentPlaceHolder1_ctl00_ctl08_ctl08" style="color:Red;display:none;"&gt;You must enter a year.&lt;/span&gt;&lt;span id="ctl00_ContentPlaceHolder1_ctl00_ctl08_ctl09" evaluationfunction="cb_verifydate_ctl00_ContentPlaceHolder1_ctl00_ctl08_ctl09" style="color:Red;visibility:hidden;"&gt;Invalid Date.&lt;/span&gt;&lt;/td&gt;&lt;td&gt;&lt;span id="ctl00_ContentPlaceHolder1_ctl00_ctl08_ORG_NAME"&gt;&lt;/span&gt;&lt;/td&gt;
    &lt;/tr&gt;
&lt;/tbody&gt;&lt;/table&gt;&lt;br&gt;&lt;span&gt;&lt;span&gt;Note: Suspended and revoked credentials are shown with red text with a strike through line.&lt;/span&gt;&lt;/span&gt;
                        &lt;/td&gt;
                    &lt;/tr&gt;
</code></pre>

<p>The the info (<code>class="EPSBResultGridItem"</code> &amp; <code>class="EPSBResultGridAlternatingItem"</code>) below the headers (<code>class="EPSBResultHeader"</code>) are not nested underneath them and as a result I've been having trouble finding a way to make sure that the information under each header is grouped with the correct header. </p>

<p>This is my code:</p>

<pre><code>count = 0
header = tree.xpath(
    './/table/tr[@class="EPSBResultGridHeader"]')
difference = 10 - len(header)
for i in range(0, difference):
    header.append('')

for license_row in header:
    count = count + 1

    try:
        header_data = license_row.xpath(".//text()")
        header_data = clean(header_data)

        nested_data = license_row.xpath(".//following-sibling::tr//text()")
        nested_data = clean(nested_data)
        print count, header_data
        print count, nested_data

    except AttributeError:
        header_data = ''

    # Append licensure data
    if count == 1:
        lheader1.append(header_data)
        lheader_info1(nested_data)
    if count == 2:
        lheader2.append(header_data)
        lheader_info2(nested_data)
    if count == 3:
        lheader3.append(header_data)
        lheader_info3(nested_data)
    if count == 4:
        lheader4.append(header_data)
        lheader_info4(nested_data)
    if count == 5:
        lheader5.append(header_data)
        lheader_info5(nested_data)
</code></pre>

<p>My end goal is to have an output like this:</p>

<pre><code>&gt;&gt;&gt;print lheader_info2
['RANK1', 'Rank I', '07-01-2018', '06-30-2021']

&gt;&gt;&gt;print lheader_info3
['G20', 'Middle School Teaching Field: Social Studies----', 'G30', 'Middle School Teaching Field: English And Communications----', 'ILE2', 'Professional Certificate For Instructional Leadership -- Early Elementary School Principal, Grades K-4; Level II', '07-01-2017', '06-30-2021', 'ILM2', 'Professional Certificate For Instructional Leadership--Middle Grade School Principal, Grades 5-8; Level II', '07-01-2017', '06-30-2021', 'ILV2', 'Professional Certificate For Instructional Leadership--Supervisor Of Instruction, Grades K-12; Level II', '07-01-2018', '06-30-2021', 'PMBF', 'Provisional Certificate For Teaching In The Middle Grades 5-8 (And For Other Assignments As Identified By Kentucky Program Of Studies)', '07-01-2016', '06-30-2021']
</code></pre>

<p>I'm using <code>lxml</code> but I've also used <code>BeautifulSoup</code> if that seems like a better way to do it. </p>

<p>I would locate every subheader and iterate over the <a href="https://www.crummy.com/software/BeautifulSoup/bs4/doc/#find-next-siblings-and-find-next-sibling" rel="nofollow">next <code>tr</code> siblings</a> breaking the loop once another header is met, or reached the end of the table:</p>

<pre><code>from collections import defaultdict
from pprint import pprint

import requests
from bs4 import BeautifulSoup

url = "https://wd.kyepsb.net/EPSB.WebApps/KECI/view_data.aspx?id=37161"
response = requests.get(url)
soup = BeautifulSoup(response.content, "html.parser")

data = defaultdict(list)

table = soup.find("table", class_="EPSBResultGrid")
for header in table.select("tr.EPSBResultGridHeader")[1:]:
    header_name = header.get_text(strip=True)
    for row in header.find_next_siblings("tr"):
        if "EPSBResultGridHeader" in row.get("class", []):
            break

        data[header_name].append(row.td.get_text(strip=True))

pprint(dict(data))
</code></pre>

<p>Prints:</p>

<pre><code>{'Cert Issued (30)': ['G20', 'G30', 'ILE2', 'ILM2', 'ILV2', 'PMBF'],
 'Do Not Print (00)': ['RANK1'],
 'History (97)': ['ILE2', 'ILM2', 'ILV2', 'RANK1']}
</code></pre>

<p>The <code>[1:]</code> slice here is to skip the initial top-level table header.</p>

python python-2.7 beautifulsoup lxml