Extracting text within tag with BeautifulSoup
<pre><code>    &lt;div&gt;
&lt;p class="tabbed" style="margin-top:2px;"&gt;&lt;span class="tab"&gt;&lt;strong&gt;LANGUAGES&lt;/strong&gt;&lt;/span&gt;Cantonese&lt;/p&gt;
&lt;p class="tabbed" style="margin-top:2px;"&gt;&lt;span class="tab"&gt;&lt;/span&gt;English&lt;/p&gt;
&lt;p class="tabbed" style="margin-top:2px;"&gt;&lt;span class="tab"&gt;&lt;/span&gt;Putonghua&lt;/p&gt;
&lt;p class="tabbed"&gt;&lt;span class="tab"&gt;&lt;strong&gt;GENDER&lt;/strong&gt;&lt;/span&gt;Male&lt;/p&gt;
   &lt;/div&gt;
</code></pre>

<p>I would like to extract the "Male" in the 5th line but I don't know how to do it. Can anyone help? 
I tried " gen = soup.find('span', class_='tab').string" but it doesn't work. </p>

<p>Here's a simpler way for you to understand: You can get your desired output by parsing the "p" tags. </p>

<pre><code>from bs4 import BeautifulSoup

doc = """ 
&lt;div&gt;
&lt;p class="tabbed" style="margin-top:2px;"&gt;&lt;span class="tab"&gt;&lt;strong&gt;LANGUAGES&lt;/strong&gt;&lt;/span&gt;Cantonese&lt;/p&gt;
&lt;p class="tabbed" style="margin-top:2px;"&gt;&lt;span class="tab"&gt;&lt;/span&gt;English&lt;/p&gt;
&lt;p class="tabbed" style="margin-top:2px;"&gt;&lt;span class="tab"&gt;&lt;/span&gt;Putonghua&lt;/p&gt;
&lt;p class="tabbed"&gt;&lt;span class="tab"&gt;&lt;strong&gt;GENDER&lt;/strong&gt;&lt;/span&gt;Male&lt;/p&gt;
&lt;/div&gt;
"""

soup = BeautifulSoup(doc, "lxml")

ptags = soup.find_all("p", attrs={'class':'tabbed'})
for ptag in ptags:
    print ptag.contents[1].string.strip()
</code></pre>

<p>This will give you the output of every "p"tag as follows</p>

<pre><code>Cantonese
English
Putonghua
Male
</code></pre>

<p>Now if you just want the value of the 4th ptag you could replace the above "for loop" with this one line. Assume you are sure you always want the value of 4th ptag, do the folowing</p>

<pre><code>print ptags[3].contents[1].string.strip()
</code></pre>

<p>will give output:</p>

<pre><code>Male
</code></pre>

<hr>

<p>Explanation:</p>

<pre><code>ptags = soup.find_all("p", attrs={'class':'tabbed'})
</code></pre>

<p>This returns a ResultSet - basically a list of ptags. Each ptag in your case has two elements the span tag and string.</p>

<pre><code>print ptags

[&lt;p class="tabbed" style="margin-top:2px;"&gt;&lt;span class="tab"&gt;&lt;strong&gt;LANGUAGES&lt;/strong&gt;&lt;/span&gt;Cantonese&lt;/p&gt;, 
&lt;p class="tabbed" style="margin-top:2px;"&gt;&lt;span class="tab"&gt;&lt;/span&gt;English&lt;/p&gt;, 
&lt;p class="tabbed" style="margin-top:2px;"&gt;&lt;span class="tab"&gt;&lt;/span&gt;Putonghua&lt;/p&gt;, 
&lt;p class="tabbed"&gt;&lt;span class="tab"&gt;&lt;strong&gt;GENDER&lt;/strong&gt;&lt;/span&gt;Male&lt;/p&gt;]
</code></pre>

<p>Now for every ptag if your print its contents "ptag.contents" it returns a list of elements in the tag
eg:</p>

<pre><code>for ptag in ptags:
    print ptag.contents
</code></pre>

<p>will give:</p>

<pre><code>[&lt;span class="tab"&gt;&lt;strong&gt;LANGUAGES&lt;/strong&gt;&lt;/span&gt;, u'Cantonese']
[&lt;span class="tab"&gt;&lt;/span&gt;, u'English']
[&lt;span class="tab"&gt;&lt;/span&gt;, u'Putonghua']
[&lt;span class="tab"&gt;&lt;strong&gt;GENDER&lt;/strong&gt;&lt;/span&gt;, u'Male']
</code></pre>

<p>Now you want the 2nd element in the list, so just get the 2nd element</p>

<pre><code>for ptag in ptags:
    print ptag.contents[1].string.strip()
</code></pre>

<p>output:</p>

<pre><code>Cantonese
English
Putonghua
Male
</code></pre>

<p>To print just the 4th one of the ptags</p>

<pre><code>print ptags[3].contents[1].string.strip()
</code></pre>

<p>output:</p>

<pre><code>Male
</code></pre>

python html text tags beautifulsoup