BeautifulSoup HTML Table Parsing for the tags without classes
<p>I have this html table: I need to get specific data from this table and assign it to a variable, I do not need all the information. flag = "United Arab Emirates", home_port="Sharjah" etc. Since there are no 'class' on html elements, how do we extract this data. </p>

<pre><code>        r = requests.get('http://maritime-connector.com/ship/'+str(imo_number),  headers={'User-Agent': 'Mozilla/5.0'})
    soup = BeautifulSoup(r.content, "lxml")
    table = soup.find("table", { "class" : "ship-data-table" })
    for row in table.findAll("tr"):
        tname = row.findAll("th")
        cells = row.findAll("td")


        print (type(tname))
        print (type(cells))
</code></pre>

<p>I am using the python module beautfulSoup.</p>

<pre><code>&lt;table class="ship-data-table" style="margin-bottom:3px"&gt;
                        &lt;thead&gt;
                        &lt;tr&gt;
                            &lt;th&gt;IMO number&lt;/th&gt;
                            &lt;td&gt;9492749&lt;/td&gt;
                        &lt;/tr&gt;
                        &lt;tr&gt;
                            &lt;th&gt;Name of the ship&lt;/th&gt;
                            &lt;td&gt;SHARIEF PILOT&lt;/td&gt;
                        &lt;/tr&gt;
                                                    &lt;tr&gt;
                            &lt;th&gt;Type of ship&lt;/th&gt;
                            &lt;td&gt;ANCHOR HANDLING VESSEL&lt;/td&gt;
                        &lt;/tr&gt;
                                                                                &lt;tr&gt;
                            &lt;th&gt;MMSI&lt;/th&gt;
                            &lt;td&gt;470535000&lt;/td&gt;
                        &lt;/tr&gt;
                                                                                &lt;tr&gt;
                            &lt;th&gt;Gross tonnage&lt;/th&gt;
                            &lt;td&gt;499 tons&lt;/td&gt;
                        &lt;/tr&gt;
                                                                                &lt;tr&gt;
                            &lt;th&gt;DWT&lt;/th&gt;
                            &lt;td&gt;222 tons&lt;/td&gt;
                        &lt;/tr&gt;
                                                                                &lt;tr&gt;
                            &lt;th&gt;Year of build&lt;/th&gt;
                            &lt;td&gt;2008&lt;/td&gt;
                        &lt;/tr&gt;
                                                                                &lt;tr&gt;
                            &lt;th&gt;Builder&lt;/th&gt;
                            &lt;td&gt;NANYANG SHIPBUILDING - JINGJIANG, CHINA&lt;/td&gt;
                        &lt;/tr&gt;
                                                                                &lt;tr&gt;
                            &lt;th&gt;Flag&lt;/th&gt;
                            &lt;td&gt;UNITED ARAB EMIRATES&lt;/td&gt;
                        &lt;/tr&gt;
                                                                                                            &lt;tr&gt;
                            &lt;th&gt;Home port&lt;/th&gt;
                            &lt;td&gt;SHARJAH&lt;/td&gt;
                        &lt;/tr&gt;
                                                                                                            &lt;tr&gt;
                            &lt;th&gt;Manager &amp; owner&lt;/th&gt;
                            &lt;td&gt;GLOBAL MARINE SERVICES - SHARJAH, UNITED ARAB EMIRATES&lt;/td&gt;
                        &lt;/tr&gt;
                                                                                                                                        &lt;tr&gt;
                            &lt;th&gt;Former names&lt;/th&gt;
                            &lt;td&gt;SUPERIOR PILOT until 2008 Sep&lt;/td&gt;
                        &lt;/tr&gt;
                                                    &lt;/thead&gt;
                    &lt;/table&gt;
</code></pre>

<p>Go over all the <code>th</code> elements in the table, get the text and the following <code>td</code> sibling's text:</p>

<pre><code>from pprint import pprint

from bs4 import BeautifulSoup

data = """your HTML here"""

soup = BeautifulSoup(data, "html.parser")

result = {header.get_text(strip=True): header.find_next_sibling("td").get_text(strip=True)
          for header in soup.select("table.ship-data-table tr th")}
pprint(result)
</code></pre>

<p>This would construct a nice dictionary with headers as keys and corresponding <code>td</code> texts as values:</p>

<pre><code>{'Builder': 'NANYANG SHIPBUILDING - JINGJIANG, CHINA',
 'DWT': '222 tons',
 'Flag': 'UNITED ARAB EMIRATES',
 'Former names': 'SUPERIOR PILOT until 2008 Sep',
 'Gross tonnage': '499 tons',
 'Home port': 'SHARJAH',
 'IMO number': '9492749',
 'MMSI': '470535000',
 'Manager &amp; owner': 'GLOBAL MARINE SERVICES - SHARJAH, UNITED ARAB EMIRATES',
 'Name of the ship': 'SHARIEF PILOT',
 'Type of ship': 'ANCHOR HANDLING VESSEL',
 'Year of build': '2008'}
</code></pre>

python html beautifulsoup