Creating bytesIO object
<p>I am working on a <a href="https://en.wikipedia.org/wiki/Scrapy" rel="nofollow">Scrapy</a> spider, trying to extract the text from multiple PDF files in a directory, using <a href="https://pypi.python.org/pypi/slate" rel="nofollow">slate</a>. I have no interest in saving the actual PDF to disk, and so I've been advised to look into the io.bytesIO subclass at <a href="https://docs.python.org/2/library/io.html#buffered-streams" rel="nofollow">https://docs.python.org/2/library/io.html#buffered-streams</a>.</p>

<p>However I'm not sure how to pass the PDF body to the bytesIO class and then pass the virtual PDF slate to get the text. So far I have:</p>

<pre><code>class Ove_Spider(BaseSpider):

    name = "ove"


    allowed_domains = ['myurl.com']
    start_urls = ['myurl/hgh/']


    def parse(self, response):
        for a in response.xpath('//a[@href]/@href'):
            link = a.extract()
            if link.endswith('.pdf'):
                link = urlparse.urljoin(base_url, link)
                yield Request(link, callback=self.save_pdf)

    def save_pdf(self, response):

      in_memory_pdf = BytesIO()
      in_memory_pdf.read(response.body) # Trying to read in PDF which is in response body
</code></pre>

<p>I'm getting:</p>

<pre><code>in_memory_pdf.read(response.body)
TypeError: integer argument expected, got 'str'
</code></pre>

<p>How can I get this working?</p>

<p>When you do <code>in_memory_pdf.read(response.body)</code> you are supposed to pass the number of bytes to read. You want to initialize the buffer, not read into it.</p>

<p>In python 2, just initialize <code>BytesIO</code> as:</p>

<pre><code> in_memory_pdf = BytesIO(response.body)
</code></pre>

<p>In Python 3, you cannot use <code>BytesIO</code> with a string because it expects bytes. The error message shows that <code>response.body</code> is of type <code>str</code>: we have to encode it.</p>

<pre><code> in_memory_pdf = BytesIO(bytes(response.body,'ascii'))
</code></pre>

<p>But as a pdf can be binary data, I suppose that <code>response.body</code> would be <code>bytes</code>, not <code>str</code>. In that case, the simple <code>in_memory_pdf = BytesIO(response.body)</code> works.</p>

python pdf