Start infinite python script in new thread within Flask
<p>I'd like to start my never ending Python script from within Flask request:</p>

<pre><code>def start_process():
    exec(open("./process/main.py").read(), globals())
    print("Started")
    return
</code></pre>

<p>and with the request:</p>

<pre><code>@app.route("/start")
   def start():
   from threading import Thread
   thread = Thread(target=start_process, args=())
   thread.setDaemon(True)
   thread.start()
   return redirect(url_for('main'))
</code></pre>

<p>The <code>main.py</code> process is a little test server that waits for some messages, but it simply hangs the entire flask script (in fact, through gunicorn, if I send CTRL-C, I can see the output of the subprocess).</p>

<p>How can I make the <code>main.py</code> script start separately?</p>

<p>I've not successfully started long-running thread from within Flask, but gone the other way, starting Flask in thread and providing it a way to communicate with other threads.</p>

<p>The trick is to something along the lines of</p>

<pre><code>def webserver(coordinator):
    app.config['COORDINATOR'] = coordinator
    app.run(use_reloader=False)
    # use_reloader=False is needed to keep Flask happy in a thread

def main():
    coordinator = Coordinator()
    ui = threading.Thread(target=webserver, args=(coordinator,))
    ui.start()
    # start other threads, passing them coordinator
    # start long-running tasks in main thread, passing it coordinator
</code></pre>

<p>Where <code>Coordinator</code> uses <code>threading</code> facilities (e.g., <code>Lock</code>, <code>Queue</code>) to protect access to shared data or resources. You can fairly easily set up the coordinator to support have worker threads block until signaled (from a handler), then start long-running tasks.</p>

python multithreading flask process daemon