lxml XPath - extracting text from multi p nodes
<h2>Please have a look at <a href="http://stackoverflow.com/questions/39242154/lxml-xpath-position-does-not-work">lxml XPath position() does not work</a> first.</h2>

<p>Since XPath does not support to extract text from multi nodes, I decided to write for loop to get 30 stuffs.</p>

<pre><code>for i in range(1,31):
    content = "string(//div[@id='article']/p[" + (print(i)) + "]/.)"
    print(content)
</code></pre>

<p>I imagined it would return like,</p>

<pre><code>"string(//div[@id='article']/p[1]/.)"
"string(//div[@id='article']/p[2]/.)"
"string(//div[@id='article']/p[3]/.)"
....
"string(//div[@id='article']/p[30]/.)"  
</code></pre>

<p>However, obviously it does not work as I expected.. I got error message as following.</p>

<pre><code>TypeError: Can't convert 'NoneType' object to str implicitly
</code></pre>

<p>What should I do? Any other elegant approach to solve this problem?</p>

<p>In Python3, <code>print</code> is a <em>function</em> which prints to the screen <em>and returns</em> <code>None</code>. (In Python2, <code>print</code> is a <em>statement</em> and the code would have raised an error since you can't put a statement in the middle of an expression.) Instead, to build a string use the <code>format</code> method:</p>

<pre><code>content = "string(//div[@id='article']/p[{}]/.)".format(i)
</code></pre>

<hr>

<p>And by the way, you should be able to use <code>position()</code> just fine with lxml. For instance,</p>

<pre><code>import lxml.html as LH
content = '''\
    &lt;bookstore&gt;
      &lt;book&gt;
        &lt;title lang="eng"&gt;Harry Potter&lt;/title&gt;
        &lt;price&gt;29.99&lt;/price&gt;
      &lt;/book&gt;
      &lt;book&gt;
        &lt;title lang="eng"&gt;Learning XML&lt;/title&gt;
        &lt;price&gt;39.95&lt;/price&gt;
      &lt;/book&gt;
      &lt;book&gt;
        &lt;title lang="eng"&gt;Things Fall Apart&lt;/title&gt;
        &lt;price&gt;19.99&lt;/price&gt;
      &lt;/book&gt;
      &lt;book&gt;
        &lt;title lang="eng"&gt;Blood Meridian&lt;/title&gt;
        &lt;price&gt;9.99&lt;/price&gt;
      &lt;/book&gt;
    &lt;/bookstore&gt;'''
root = LH.fromstring(content)

# Compare with http://stackoverflow.com/a/39242701/190597
print(root.xpath('//book[position()&gt;=1 and position()&lt;=last()]/title/text()'))
# ['Harry Potter', 'Learning XML', 'Things Fall Apart', 'Blood Meridian']

# But note that it is equivalent to 
print(root.xpath('//book/title/text()'))
# ['Harry Potter', 'Learning XML', 'Things Fall Apart', 'Blood Meridian']

print(root.xpath('//book[position()&lt;3]'))
</code></pre>

<p>prints</p>

<pre><code>['Harry Potter', 'Learning XML']
</code></pre>

<p>which shows that you can select the first <code>N</code> <code>books</code> without having to loop.</p>

<hr>

<p><a href="http://stackoverflow.com/questions/39242154/lxml-xpath-position-does-not-work#comment65822655_39242701">As Tomalak mentions</a>, the XPath <code>string</code> function only returns the string representation of the first node. For example,</p>

<pre><code>print(root.xpath('string(//book[position()&lt;3]/title/text())'))
</code></pre>

<p>only prints</p>

<pre><code>Harry Potter
</code></pre>

<p>If you want a list of strings, then don't use <code>string</code>.</p>

<p>If, as <a href="http://stackoverflow.com/questions/39256973/lxml-xpath-extracting-text-from-multi-p-nodes/39257015?noredirect=1#comment65851036_39257015">Daniel Haley points out</a>, the desired text is in a mixture of nested nodes and child elements, e.g. <code>&lt;title lang="eng"&gt;Harry &lt;b&gt;Potter&lt;/b&gt;&lt;/title&gt;</code>, then you can extract the desired text using the <code>text_content</code> method:</p>

<pre><code>[title.text_content() for title in root.xpath('//book[position()&lt;3]/title')]
</code></pre>

python python-3.x xpath lxml