Why is Python 3 http.client so much faster than python-requests?
<p>I was testing different Python HTTP libraries today and I realized that <a href="https://docs.python.org/3/library/http.client.html" rel="nofollow"><code>http.client</code></a> library seems to perform much much faster than <a href="http://docs.python-requests.org/en/master/" rel="nofollow"><code>requests</code></a>. </p>

<p>To test it you can run following two code samples.</p>

<pre><code>import http.client

conn = http.client.HTTPConnection("localhost", port=8000)
for i in range(1000):
    conn.request("GET", "/")
    r1 = conn.getresponse()
    body = r1.read()
    print(r1.status)

conn.close()
</code></pre>

<p>and here is code doing same thing with python-requests:</p>

<pre><code>import requests

with requests.Session() as session:
    for i in range(1000):
        r = session.get("http://localhost:8000")
        print(r.status_code)
</code></pre>

<p>If I start SimpleHTTPServer:</p>

<pre><code>&gt; python -m http.server
</code></pre>

<p>and run above code samples (I'm using Python 3.5.2). I get following results:</p>

<p>http.client:</p>

<pre><code>0.35user 0.10system 0:00.71elapsed 64%CPU 
</code></pre>

<p>python-requests:</p>

<pre><code>1.76user 0.10system 0:02.17elapsed 85%CPU 
</code></pre>

<p>Are my measurements and tests correct? Can you reproduce them too? If yes does anyone know what's going on inside <code>http.client</code> that make it so much faster? Why is there such big difference in processing time?</p>

<p>copy-pasting response from @Lukasa posted <a href="https://github.com/kennethreitz/requests/issues/3568#issuecomment-246271506" rel="nofollow">here</a>:</p>

<blockquote>
  <p>The reason Requests is slower is because it does substantially more than httplib. httplib can be thought of as the bottom layer of the stack: it does the low-level wrangling of sockets. Requests is two layers further up, and adds things like cookies, connection pooling, additional settings, and kinds of other fun things. This is necessarily going to slow things down. We simply have to compute a lot more than httplib does.</p>
  
  <p>You can see this by looking at cProfile results for Requests: there's just way more result than there is for httplib. This is always to be expected with high-level libraries: they add more overhead because they have to do a lot more work.</p>
  
  <p>While we can look at targetted performance improvements, the sheer height of the call stack in all cases is going to hurt our performance markedly. That means that the complaint that "requests is slower than httplib" is always going to be true: it's like complaining that "requests is slower than sending carefully crafted raw bytes down sockets." That's true, and it'll always be true: there's nothing we can do about that.</p>
</blockquote>

python http python-requests