TypeError: takes exactly 1 argument (0 given) - Scrapy
<p>I'm working with scrapy. I want to generate a unique user agent for each request. I have the following:</p>

<pre><code>class ContactSpider(Spider):
    name = "contact"

    def getAgent(self):
        f = open('useragentstrings.txt')
        agents = f.readlines()
        return random.choice(agents).strip()

    headers = {          
        'user-agent': getAgent(),
        'content-type': "application/x-www-form-urlencoded",
        'cache-control': "no-cache"
    }

    def parse(self, response):
        open_in_browser(response)
</code></pre>

<p>getAgent generates an agent from a list of the form:</p>

<pre><code>"Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/52.0.2743.116 Safari/537.36"
</code></pre>

<p>However when I run this I get:</p>

<pre><code>  File "..spiders\contact_spider.py, line 35, in &lt;module&gt;
    class ContactSpider(Spider):
  File "..spiders\contact_spider.py", line 54, in ContactSpider
    'user-agent': getAgent(),
TypeError: getAgent() takes exactly 1 argument (0 given)
</code></pre>

<p><code>getAgent()</code> is an <em>instance method</em> and expects to see the <code>ContactSpider</code> instance as an argument. But, the problem is, you don't need this function to be a member of your spider class - move it to a separate "helpers"/"utils"/"libs" module and import:</p>

<pre><code>from helpers import getAgent

class ContactSpider(Spider):
    name = "contact"

    headers = {          
        'user-agent': getAgent(),
        'content-type': "application/x-www-form-urlencoded",
        'cache-control': "no-cache"
    }

    def parse(self, response):
        open_in_browser(response)
</code></pre>

<p>See also: <a href="http://stackoverflow.com/questions/17134653/difference-between-class-and-instance-methods">Difference between Class and Instance methods</a>.</p>

<hr>

<p>Or, as an alternative approach, there is a <a href="https://github.com/alecxe/scrapy-fake-useragent" rel="nofollow"><code>scrapy-fake-user-agent</code></a> Scrapy middleware that would rotate user agents seamlessly and randomly. User Agent strings are supplied by the <a href="https://pypi.python.org/pypi/fake-useragent" rel="nofollow"><code>fake-useragent</code> module</a>.</p>

python scrapy