calculate avg response time from multiple curl commands using python
<p>I am trying to do a load testing of my web server using curl command.</p>

<p>I am able to run multiple curl commands, but now I also want to calculate the avg response time from all the curl command which were executed </p>

<pre><code>from functools import partial
from multiprocessing.dummy import Pool
from subprocess import call

commands = []
command = "curl -s -w \"Time:%{time_total}\n\" -o /dev/null -k -X GET \"https://google.com\""
for i in range(10):   # run 10 curl commands in total
    commands.append(command)

pool = Pool(5) # Nummber of concurrent commands at a time
for i, returncode in enumerate(pool.imap(partial(call, shell=True), commands)):
    if returncode != 0:
       print("%d command failed: %d" % (i, returncode))
</code></pre>

<p>output</p>

<pre><code>Time:0.654
Time:0.689
Time:0.720
Time:0.725
Time:0.735
Time:0.624
Time:0.635
Time:0.633
Time:0.678
Time:0.708
</code></pre>

<p>How can i capture the <code>Time</code> and calculate the average response time?</p>

<p>Thanks</p>

<p>Instead of relying to <code>call</code> you could create separate function executed by <code>imap</code>. Then you could use <a href="https://docs.python.org/3.5/library/subprocess.html#subprocess.Popen" rel="nofollow"><code>Popen</code></a> that allows you to communicate with the child process. The example below writes only the time to stdout which is then captured and returned to parent process:</p>

<pre><code>from functools import partial
from multiprocessing.dummy import Pool
from subprocess import Popen, PIPE

def child(cmd):
    p = Popen(cmd, stdout=PIPE, shell=True)
    out, err = p.communicate()
    return out, p.returncode

commands = []
command = "curl -s -w \"%{time_total}\" -o /dev/null -k -X GET \"https://google.com\""
for i in range(10):   # run 10 curl commands in total
    commands.append(command)

pool = Pool(5) # Nummber of concurrent commands at a time

times = []
for i, (output, returncode) in enumerate(pool.imap(child, commands)):
    if returncode != 0:
       print("{} command failed: {}".format(i, returncode))
    else:
       print("{} success: {}".format(i, output))
       times.append(float(output))

print 'Average: {}'.format(sum(times) / len(times) if times else 0)
</code></pre>

<p>Output:</p>

<pre><code>0 success: 0.109
1 success: 0.108
2 success: 0.103
3 success: 0.093
4 success: 0.085
5 success: 0.091
6 success: 0.109
7 success: 0.114
8 success: 0.092
9 success: 0.099
Average: 0.1003
</code></pre>

python curl