How to get an element having its relative XPath?
<p>I have xml file. After parsing it with <code>lxml</code> as an <code>etree</code>, I can get all of its tags as follows:</p>

<pre><code>root = tree.getroot()
for e in root.iter():
    print e.tag
</code></pre>

<p>and the output is something like this:</p>

<pre class="lang-none prettyprint-override"><code>'{http://www.w3.org/1999/xhtml}html'
'{http://www.w3.org/1999/xhtml}head'
'{http://www.w3.org/1999/xhtml}meta'
'{http://www.w3.org/1999/xhtml}link'
'{http://www.w3.org/1999/xhtml}meta'
'{http://www.w3.org/1999/xhtml}meta'
'{http://www.w3.org/1999/xhtml}meta'
'{http://www.w3.org/1999/xhtml}script'
'{http://www.w3.org/1999/xhtml}body'
'{http://www.w3.org/1999/xhtml}section'
'{http://www.w3.org/1999/xhtml}h1'
'{http://www.w3.org/1999/xhtml}p'
'{http://www.w3.org/1999/xhtml}em'
'{http://www.w3.org/1999/xhtml}section'
'{http://www.w3.org/1999/xhtml}h1'
'{http://www.w3.org/1999/xhtml}p'
'{http://www.w3.org/1999/xhtml}a'
'{http://www.w3.org/1999/xhtml}p'
'{http://www.w3.org/1999/xhtml}p'
</code></pre>

<p><strong>I want to get some elements with relative path using python/lxml/bs4.</strong> For example I want first <code>p</code> element in second <code>section</code> and I have following relative path: <code>/section[2]/p[1]</code> .</p>

<p>But I can not even get all sections with following code, which returns <code>None</code>:</p>

<pre><code>xhtml = {http://www.w3.org/1999/xhtml}
section = xhtml + "section"
root.find(section)
</code></pre>

<p>EDIT: Here's part of original file:</p>

<pre class="lang-xml prettyprint-override"><code>&lt;?xml version="1.0" encoding="UTF-8"?&gt;
&lt;?xml-model href="grammar/rash.rng" type="application/xml" schematypens="http://relaxng.org/ns/structure/1.0"?&gt;
&lt;html xmlns="http://www.w3.org/1999/xhtml" prefix="schema: http://schema.org/ prism: http://prismstandard.org/namespaces/basic/2.0/"&gt;
   &lt;head&gt;
      &lt;meta charset="UTF-8"/&gt;
      &lt;meta name="viewport" content="width=device-width, initial-scale=1"/&gt;
      &lt;link rel="stylesheet" href="css/bootstrap.min.css"/&gt;
      &lt;link rel="stylesheet" href="css/rash.css"/&gt;
      &lt;script src="js/jquery.min.js"&gt;&lt;![CDATA[ ]]&gt;&lt;/script&gt;
      &lt;script src="js/bootstrap.min.js"&gt;&lt;![CDATA[ ]]&gt;&lt;/script&gt;
      &lt;script src="js/rash.js"&gt;&lt;![CDATA[ ]]&gt;&lt;/script&gt;
      &lt;title&gt;It ROCS! -- The RASH Online Conversion Service&lt;/title&gt;
      &lt;meta about="#affiliation-1" property="schema:name" content="Department of Computer Science and Engineering, University of Bologna, Italy"/&gt;
      &lt;meta about="#affiliation-2" property="schema:name" content="Oxford e-Research Centre, University of Oxford, UK"/&gt;
      &lt;meta about="#affiliation-3" property="schema:name" content="Knowledge Media Institute, Open University, UK"/&gt;
      &lt;meta property="prism:keyword" content="HTML-based format"/&gt;
      &lt;meta property="prism:keyword" content="Scholarly HTML"/&gt;
      &lt;meta property="prism:keyword" content="RASH"/&gt;
   &lt;script src="https://cdn.mathjax.org/mathjax/latest/MathJax.js?config=TeX-AMS-MML_HTMLorMML"&gt;&lt;![CDATA[ ]]&gt;&lt;/script&gt;&lt;/head&gt;
   &lt;body&gt;
      &lt;section role="doc-abstract"&gt;
         &lt;h1&gt;Abstract&lt;/h1&gt;
         &lt;p&gt;In this poster paper we introduce the &lt;em&gt;RASH Online Conversion Service&lt;/em&gt;, i.e., a Web application that allows the conversion of ODT documents into RASH, a HTML-based markup language for writing scholarly articles, and from RASH into LaTeX. This tool allows authors with no experience in HTML to easily produce HTML-based papers and supports the publishing process by generating also a LaTeX version according to the Springer LNCS and ACM ICPS layouts.&lt;/p&gt;
      &lt;/section&gt;
      &lt;section&gt;
         &lt;h1&gt;Introduction&lt;/h1&gt;
         &lt;p&gt;The use of HTML as format for writing scholarly papers and submitting them to scholarly venues is a very popular, discussed and trendy topic within the scholarly domain. This is demonstrated by the existence of several posts within technical mailing lists of the Web community&lt;a href="#ftn0"&gt; &lt;/a&gt;, by the birth of W3C community groups on such topic&lt;a href="#ftn3"&gt; &lt;/a&gt;, by the development of HTML-based formats for scholarly articles&lt;a href="#ftn4"&gt; &lt;/a&gt;, and by the increasing number of events that are experimenting with HTML-based formats for submissions, such as the SAVE-SD&lt;a href="#ftn5"&gt; &lt;/a&gt; and LDOW&lt;a href="#ftn6"&gt; &lt;/a&gt; workshops at WWW 2016, and the Extended Semantic Web Conference&lt;a href="#ftn7"&gt; &lt;/a&gt;.&lt;/p&gt;
         &lt;p&gt;In order to foster a wider adoption of these formats, frameworks for HTML-based papers should support the needs of all the actors involved in the production, delivery and fruition of scholarly articles, with particular regards to authors and publishers. Hence, this solution calls for a number of requirements that go well beyond those used on the Web. &lt;/p&gt;
         &lt;p&gt;First of all, it is vital to support authors with a variety of tools to provide for an easy transition to the new format. To this end, authors should be allowed to keep using well-known current word processors rather than adopting HTML and/or pure text editors. We thus need to support the conversion from the main word processor formats (e.g., ODT and OOXML) to HTML formats, in particular when authors use only basic features, such as standard styles for paragraphs and tables. In addition, authors should be given the option to focus on the content and let appropriate tools handle the presentation layer after the conversion into the HTML-based format.&lt;/p&gt;
</code></pre>

<p>In this example I want to get <code>&lt;p&gt;</code> element which starts with this sentence: "The use of HTML as format for writing scholarly..."</p>

<p>BeautifulSoup, does not support XPath expressions, but <strong>lxml</strong> you've mentioned, does.  </p>

<p>You can search for elements with XPath like following:</p>

<pre><code>from lxml import etree

htmlparser = etree.HTMLParser()
tree = etree.parse(html_content, htmlparser)
tree.xpath(xpathselector)
</code></pre>

python xml xpath lxml bs4