Python script execution time increases when executed multiple time parallely
<p>I have a python script whose execution time is 1.2 second while it is being executed standalone.</p>

<p>But when I execute it 5-6 time parallely ( Am using postman to ping the url multiple times) the execution time shoots up.</p>

<p>Adding the breakdown of the time taken.</p>

<pre><code>1 run -&gt; ~1.2seconds
2 run -&gt; ~1.8seconds
3 run -&gt; ~2.3seconds
4 run -&gt; ~2.9seconds
5 run -&gt; ~4.0seconds
6 run -&gt; ~4.5seconds
7 run -&gt; ~5.2seconds
8 run -&gt; ~5.2seconds
9 run -&gt; ~6.4seconds
10 run -&gt; ~7.1seconds
</code></pre>

<p>Screenshot of top command(Asked in the comment):
<a href="http://i.stack.imgur.com/Ec3XC.png"><img src="http://i.stack.imgur.com/Ec3XC.png" alt="enter image description here"></a></p>

<p>This is a sample code:</p>

<pre><code>import psutil
import os
import time
start_time = time.time()
import cgitb
cgitb.enable()
import numpy as np
import MySQLdb as mysql
import cv2
import sys
import rpy2.robjects as robj
import rpy2.robjects.numpy2ri
rpy2.robjects.numpy2ri.activate()
from rpy2.robjects.packages import importr
R = robj.r
DTW = importr('dtw')

process= psutil.Process(os.getpid())
print " Memory Consumed after libraries load: "
print process.memory_info()[0]/float(2**20)

st_pt=4
# Generate our data (numpy arrays)
template = np.array([range(84),range(84),range(84)]).transpose()
query = np.array([range(2500000),range(2500000),range(2500000)]).transpose()


#time taken
print(" --- %s seconds ---" % (time.time() - start_time))
</code></pre>

<p>I also checked my memory consumption using <code>watch -n 1 free -m</code> and memory consumption also increases noticeably.</p>

<p>1) How do I make sure that the execution time of script remain constant everytime.</p>

<p>2) Can I load the libraries permanently so that the time taken by the script to load the libraries and the memory consumed can be minimized?</p>

<p>I made an enviroment and tried using </p>

<p><code>#!/home/ec2-user/anaconda/envs/test_python/</code></p>

<p>but it doesn't make any difference whatsoever.</p>

<p><strong>EDIT:</strong></p>

<p>I have AMAZON's EC2 server with 7.5GB RAM.</p>

<p>My php file with which am calling the python script.</p>

<pre><code>&lt;?php

    $response = array("error" =&gt; FALSE); 

    if($_SERVER['REQUEST_METHOD']=='GET'){

        $response["error"] = FALSE;
        $command =escapeshellcmd(shell_exec("sudo /home/ec2-user/anaconda/envs/anubhaw_python/bin/python2.7 /var/www/cgi-bin/dtw_test_code.py"));
        session_write_close();
        $order=array("\n","\\");
        $cleanData=str_replace($order,'',$command);
        $response["message"]=$cleanData;

    } else 
    {
        header('HTTP/1.0 400 Bad Request');
        $response["message"] = "Bad Request.";
    }
    echo json_encode($response);
?&gt;
</code></pre>

<p>Thanks</p>

<p>1)</p>

<h1>More servers equals more availability</h1>

<p>Hearsay tells me that one effective way to ensure consistent request times is to use multiple requests to a cluster. As I heard it the idea goes something like this. </p>

<h3>The chance of a slow request</h3>

<p><sub>(Disclaimer I'm not much of a mathematician or statistician.)</sub></p>

<p>If there is a 1% chance a request is going to take an abnormal amount of time to finish then one-in-a-hundred requests can be expected to be slow. If you as a client/consumer make two requests to a cluster instead of just one, the chance that both of them turn out to be slow would be more like  1/10000, and with three 1/1000000, et cetera. The downside is doubling your incoming requests means needing to provide (and pay for) as much as twice the server power to fulfill your requests with a consistent time, this additional cost scales with how much chance is acceptable for a slow request.</p>

<p>To my knowledge this concept is optimized for consistent fulfillment times.</p>

<h3>The client</h3>

<p>A client interfacing with a service like this has to be able to spawn multiple requests and handle them gracefully, probably including closing the unfulfilled connections as soon as it can.</p>

<h3>The servers</h3>

<p>On the backed there should be a load balancer that can associate multiple incoming client requests to multiple unique cluster workers. If a single client makes multiple requests to an overburdened node, its just going to compound its own request time like you see in your simple example.</p>

<p>In addition to having the client opportunistically close connections it would be best to have a system of sharing job fulfilled status/information so that backlogged request on other other slower-to-process nodes have a chance of aborting an already-fulfilled request. </p>

<hr>

<p>This this a rather informal answer, I do not have direct experience with optimizing a service application in this manner. If someone does I encourage and welcome more detailed edits and expert implementation opinions.</p>

<hr>

<p>2)</p>

<h1>Caching imports</h1>

<p>yes that is a thing, and its awesome!</p>

<p>I would personally recommend setting up django+gunicorn+nginx. Nginx can cache static content and keep a request backlog, gunicorn provides application caching and multiple threads&amp;worker management (not to mention awesome administration and statistic tools), django embeds best practices for database migrations, auth, request routing, as well as off-the-shelf plugins for providing semantic rest endpoints and documentation, all sorts of goodness.</p>

<p>If you really insist on building it from scratch yourself you should study <a href="http://uwsgi-docs.readthedocs.io/en/latest/index.html" rel="nofollow">uWsgi</a>, a great <a href="https://www.python.org/dev/peps/pep-0333/" rel="nofollow">Wsgi implementation</a> that can be interfaced with <a href="http://gunicorn.org/" rel="nofollow">gunicorn</a> to provide application caching. Gunicorn isn't the only option either, Nicholas PiÃ«l has a <a href="http://nichol.as/benchmark-of-python-web-servers" rel="nofollow">Great write up</a> comparing performance of various python web serving apps.</p>

python time