How to get the line number of a match with scrapy
<p>Using the following example:</p>

<pre><code>$ scrapy shell http://doc.scrapy.org/en/latest/_static/selectors-sample1.html
</code></pre>

<p>where <em>selectors-sample1-html</em> is:</p>

<pre><code>&lt;html&gt;
 &lt;head&gt;
  &lt;base href='http://example.com/' /&gt;
  &lt;title&gt;Example website&lt;/title&gt;
 &lt;/head&gt;
 &lt;body&gt;
  &lt;div id='images'&gt;
   &lt;a href='image1.html'&gt;Name: My image 1 &lt;br /&gt;&lt;img src='image1_thumb.jpg' /&gt;&lt;/a&gt;
   &lt;a href='image2.html'&gt;Name: My image 2 &lt;br /&gt;&lt;img src='image2_thumb.jpg' /&gt;&lt;/a&gt;
   &lt;a href='image3.html'&gt;Name: My image 3 &lt;br /&gt;&lt;img src='image3_thumb.jpg' /&gt;&lt;/a&gt;
   &lt;a href='image4.html'&gt;Name: My image 4 &lt;br /&gt;&lt;img src='image4_thumb.jpg' /&gt;&lt;/a&gt;
   &lt;a href='image5.html'&gt;Name: My image 5 &lt;br /&gt;&lt;img src='image5_thumb.jpg' /&gt;&lt;/a&gt;
  &lt;/div&gt;
 &lt;/body&gt;
&lt;/html&gt;
</code></pre>

<p>Is it possible to get the line number of a match using Scrapy 1.1.2? For instance, something like:</p>

<pre><code>$ response.selector.xpath('//title/text()').some_magic_to_get_line_number
$ # should output 4
</code></pre>

<p>Thank you!</p>

<p>I don't know of a way to get source line for text nodes, but for element nodes, you can hack into the underlying lxml object of the selector (with <code>.root</code>), and access <code>.sourceline</code> attribute:</p>

<pre><code>$ scrapy shell http://doc.scrapy.org/en/latest/_static/selectors-sample1.html
2016-09-08 18:13:12 [scrapy] INFO: Scrapy 1.1.2 started (bot: scrapybot)
2016-09-08 18:13:12 [scrapy] INFO: Spider opened
2016-09-08 18:13:13 [scrapy] DEBUG: Crawled (200) &lt;GET http://doc.scrapy.org/en/latest/_static/selectors-sample1.html&gt; (referer: None)
&gt;&gt;&gt; response.selector.xpath('//title/text()')
[&lt;Selector xpath='//title/text()' data=u'Example website'&gt;]
&gt;&gt;&gt; s = response.selector.xpath('//title/text()')[0]
&gt;&gt;&gt; type(s)
&lt;class 'scrapy.selector.unified.Selector'&gt;
&gt;&gt;&gt; type(s.root)
&lt;type 'str'&gt;
&gt;&gt;&gt; s = response.selector.xpath('//title')[0]
&gt;&gt;&gt; s.root
&lt;Element title at 0x7fa95d3f1908&gt;
&gt;&gt;&gt; type(s.root)
&lt;type 'lxml.etree._Element'&gt;
&gt;&gt;&gt; dir(s.root)
['__class__', '__contains__', '__copy__', '__deepcopy__', '__delattr__', '__delitem__', '__doc__', '__format__', '__getattribute__', '__getitem__', '__hash__', '__init__', '__iter__', '__len__', '__new__', '__nonzero__', '__reduce__', '__reduce_ex__', '__repr__', '__reversed__', '__setattr__', '__setitem__', '__sizeof__', '__str__', '__subclasshook__', '_init', 'addnext', 'addprevious', 'append', 'attrib', 'base', 'clear', 'cssselect', 'extend', 'find', 'findall', 'findtext', 'get', 'getchildren', 'getiterator', 'getnext', 'getparent', 'getprevious', 'getroottree', 'index', 'insert', 'items', 'iter', 'iterancestors', 'iterchildren', 'iterdescendants', 'iterfind', 'itersiblings', 'itertext', 'keys', 'makeelement', 'nsmap', 'prefix', 'remove', 'replace', 'set', 'sourceline', 'tag', 'tail', 'text', 'values', 'xpath']

&gt;&gt;&gt; s.root.sourceline
4
&gt;&gt;&gt; 
</code></pre>

python scrapy