Making a real time application which uses same python code continuously on a server, libraries import on each run and consume memory and time
<p>I am developing a real time navigation application on Android, what it does is collects relevant data from user's smartphone and pings a python code on server with the data. The python code then returns the real time location of the user. Now given that I need to run the same python code again and again, the same exact libraries need to be loaded (I am using libraries like numpy, MySQLdb, sys, rpy2). Now everytime I run the code there's some memory and time which is consumed in loading these libraries. For me as it turns out the main amount of memory and time is getting consumed in this. This is increasing the server costs and execution time unnecessarily as I am utilizing my resources in just loading the same libraries. Is there some way in Python where I can permanently load my libraries onto the server's RAM and cut the cost. Is it even possible using just Python. I thought this would be a common issue but I am finding very disconnected answers every time I google it. Please suggest the best way to do this. I lack a bit on server side coding best practices maybe my problem is a part of that rather than part of Python coding. Thanks!</p>

<p>PS: As I was using AWS I thought AWS Lambda should take care of it automatically, I implemented my code as an AWS Lambda package but I see a degradation in performance actually.</p>

<p>You usually have a web API that responds to the Android apps. These APIs are usually provided by a Python App behind a web server. One way of doing a Python app behind a webserver is by using <a href="https://www.fullstackpython.com/wsgi-servers.html" rel="nofollow">WSGI</a>. All available WSGI implementation do the job of starting the app once (sometimes a few of them) and dispatching requests to the already running app. The job of maintaining running instances etc. is essentially all taken care of.</p>

<p>If you have a Python job spawn upon request and exit after it is done sounds like a badly designed homemade webapp. In this case you should port it over to WSGI.</p>

<p>If on the other hand you have jobs that need to be done asynchronously to your web service (ones with long execution time etc.) you should start a second Python daemon that receives jobs from your webapp through a <a href="https://www.fullstackpython.com/task-queues.html" rel="nofollow">queue</a>. In this case, too, the daemon would be started at boot time and persistently run until you shut down the server.</p>

python amazon-web-services server real-time libraries