POSTing and GETing with grequests
<p>I'm using grequests to scape websites faster. However, I also need to login to the website. </p>

<p>Before (just using requests) I could do: </p>

<p>where <code>headers</code> is my <code>User-Agent</code>. </p>

<pre><code>with requests.Session() as s: 
    s.headers.update(headers)
    s.post(loginURL, files = data)
    s.get(scrapeURL)
</code></pre>

<p>Using <code>grequests</code> I've only been able to pass <code>headers</code> by doing: </p>

<pre><code>rs = (grequests.get(u, headers=header) for u in urls)
response = grequests.map(rs)
</code></pre>

<p>Is there anyway to do a <code>POST</code> at the same time so I can login? The login URL is differnt than the URL(s) I'm scrapping. </p>

<p>You can pass in the <em>Session</em> object exactly the same as the <em>headers</em>:</p>

<pre><code>with requests.Session() as s: 
    s.headers.update(headers)
    s.post(loginURL, files = data)
    s.get(scrapeURL)


    rs = (grequests.get(u, headers=header, session=s) for u in urls)
    response = grequests.map(rs)
</code></pre>

python python-3.x python-requests grequests