Using regex to select html content
<p>I have a file with several instances of rows with this structure:</p>

<pre><code>&lt;tr&gt;
                  &lt;td style="width:25%;"&gt;
                     &lt;span class="results_title_text"&gt;DUNS:&lt;/span&gt; &lt;span class="results_body_text"&gt; 012361296&lt;/span&gt;
                  &lt;/td&gt;
                  &lt;td style="width:25%;"&gt;
                  &lt;/td&gt;
                  &lt;!-- label as CAGE when US Territory is listed as Country --&gt;
                  &lt;td style="width:27%;"&gt;
                     &lt;span class="results_title_text"&gt;CAGE Code:&lt;/span&gt; &lt;span class="results_body_text"&gt;HELLO&lt;/span&gt;
                  &lt;/td&gt;
                  &lt;td style="width:15%" rowspan="2"&gt;
                     &lt;input type="button" value="View Details" title="View Details for Rascal X-Press, Inc." class="center" style="height:25px; width:90px; vertical-align:middle; margin:7px 3px 7px 3px;" onClick="viewEntry('4420848', '1472652382619')" /&gt;
                  &lt;/td&gt;
&lt;/tr&gt;
</code></pre>

<p>I want to select only those <code>&lt;span class="results_body_text"&gt;</code> that are preceeded by <code>&lt;span class="results_title_text"&gt;DUNS:&lt;/span&gt;</code> so in this case I would only return the span that contains <code>012361296</code> and not the one that contains <code>HELLO</code></p>

<p>How can I do this using a regular expression or anything else? I have tried the "starts with" regex format, but I am failing to see what string I would be parsing in that case. I eventually want to parse the regex into a <code>re.compile()</code> compile function in python.</p>

<p>Using pyparsing to process HTML allows you to gloss over things like unexpected whitespace, extra/missing attributes, tags in upper or lower case. Assuming you have read your HTML source into a variable <code>html</code>, this pyparsing code will extract the target value:</p>

<pre><code>from pyparsing import makeHTMLTags, SkipTo
span,end_span = makeHTMLTags("span")

patt = span + 'DUNS:' + end_span + span + SkipTo(end_span)("results_body") + end_span

print(patt.searchString(html)[0].results_body)
</code></pre>

<p>prints:</p>

<pre><code>012361296
</code></pre>

python html regex data-cleaning