Building a nested list with XPath-extracted XML document structure
<p>I am trying to get the text (using xpath) of all <code>&lt;h2&gt;</code> tags in:</p>

<pre><code>&lt;div id="static_id"&gt;
 &lt;div&gt;...
 &lt;a ...&gt;
 &lt;div&gt;...
   &lt;h2&gt;Text 1&lt;/h2&gt;
 &lt;a ...&gt;
 &lt;div&gt;...
 &lt;div&gt;...
 &lt;span&gt;...
   &lt;h2&gt;Text 2&lt;/h2&gt;
  &lt;a ...&gt;
  &lt;span&gt;...
    &lt;h2&gt;Text 3&lt;/h2&gt;

&lt;div id="static_id"&gt;
 &lt;div&gt;...
 &lt;span&gt;...
   &lt;h2&gt;Text A&lt;/h2&gt;
 &lt;a ...&gt;
 &lt;div&gt;...
 &lt;p&gt;...
 &lt;div&gt;...
   &lt;h2&gt;Text B&lt;/h2&gt;
 &lt;a ...&gt;
   &lt;h2&gt;Text C&lt;/h2&gt;
 [...]
</code></pre>

<p>In my HTML source code there are <code>&lt;div&gt;'s</code> with the id <code>static_id</code>. Within these div's there is just one <code>&lt;h2&gt;</code> tag, and I want to get its content. In the end, I would like to have a list that looks like this:</p>

<pre><code>lst = [["Text 1", "Text 2", "Text 3"], ["Text A", "Text B", "Text C"]]
</code></pre>

<p>Please notice that it's a list of lists (every h2-text from one <code>&lt;div id="static_id"&gt;</code> should end up in a separate list like in the example above.</p>

<p>Is there an easy way to achieve this?</p>

<p>I thought I count all <code>static_id</code> divs and iterate over all <code>&lt;h2&gt;</code> tags to achive this. My approach:</p>

<pre><code>list_all = []
div_amount = len(tree.xpath('//div[@id="static_id"]')) # 2 elements in this case (works)
for d in range(1, div_amount+1) # 1,2
  h2_count = len(tree.xpath('//div[@class="static_id"]['+str(d)+']//h2')) #count h2
  lst = []
  for i in range(1, h2_count+1) #1,2,3
    h2_text = ''.join(tree.xpath('//div[@id="static_id"]['+str(d)+']//h2['+i+']/text()'))
    lst.append(h2_text)
  list_all.append(lst)
</code></pre>

<p>Line 2: Counts all id="static_id"</p>

<p>Line 3: Loop over all id="static_id"</p>

<p>Line 4: Count all h2 (unfortunately all h2's from the HTML source are counted) </p>

<p>Line 5: Loop over all h2's</p>

<p>Line 6: Get h2'text, and next save in list</p>

<p>Can anyone help me out, please? I feel like this could be done easier, but I don't know how.</p>

<p>Easily made a one-liner:</p>

<pre><code>list_all = [ static_id_div.xpath('.//h2/text()')
             for static_id_div in tree.xpath('//div[@id="static_id"]') ]
</code></pre>

<p>The important difference here is that the inner query is being run <em>against the elements returned by the outer query</em>, rather than making them work starting from the root of the document.</p>

python xpath