Order of the link in the page with Scrapy
<p>I have a simple <code>LinkExtractor</code> rule for a given domain. Something like this: <code>Rule(LinkExtractor(allow=('domain\.com/.+/\d+', )), callback='parse_page'),</code></p>

<p>What I would like and I can't figure out, it's to know on which position was the link in the page.</p>

<p>For example, if a given domain has 5 links on the page that match my rule I need to know from the top to the bottom their order in the HTML.</p>

<p>I found many questions about the order of the extraction, but nothing, or I misunderstood something, about the order of the link itself in the HTML</p>

<p>Scrapy uses lxml to for html parsing. <code>LinkExtractor</code> uses <code>root.iter()</code> to iterate through. <a href="https://github.com/scrapy/scrapy/blob/master/scrapy/linkextractors/lxmlhtml.py#L37" rel="nofollow">This line to be more exact.</a></p>

<p><a href="http://lxml.de/tutorial.html#tree-iteration" rel="nofollow">Lxml's docs say:</a></p>

<blockquote>
  <p>Elements provide a tree iterator for this purpose. It yields elements
  in document order, i.e. in the order their tags would appear if you
  serialised the tree to XML:</p>
</blockquote>

<p>so for html source:</p>

<pre><code>&lt;root&gt;
  &lt;child&gt;Child 1&lt;/child&gt;
  &lt;child&gt;Child 2&lt;/child&gt;
  &lt;another&gt;Child 3&lt;/another&gt;
&lt;/root&gt;
</code></pre>

<p>it would yield:</p>

<pre><code>&gt;&gt;&gt; for element in root.iter(tag=etree.Element):
...     print("%s - %s" % (element.tag, element.text))
root - None
child - Child 1
child - Child 2
another - Child 3
</code></pre>

<p>You can replicate the process using the examples provided in the lxml docs link posted above.</p>

python scrapy