How to kill celery worker process for process restart
<p>I have a celery worker process that creates one socket to an external server in <code>celery.signals.worker_process_init</code>. If the connection can't be established and an exception is raised the celery worker fails to start, which is the intended behaviour.</p>

<p>That socket class also sends occasional heartbeat packets using a <code>threading.Timer</code>. Unfortunately, because the Timer thread has no special exception handling, my worker doesn't quit if something happens to the connection.</p>

<p>I started catching exceptions from my socket read and write operations and have tried a bunch of things to gracefully stop the current worker process:</p>

<pre><code>sys.exit(1)  # Doesn't kill the worker process (because it's running in a thread?)
</code></pre>

<p>next attempt, I saw it mentioned somewhere in <code>celery/billiard</code> issues: </p>

<pre><code>raise SystemExit("Socket error")  # Same, I guess
</code></pre>

<p>this works:</p>

<pre><code>os.kill(os.getpid(), signal.SIGHUP)
</code></pre>

<p>BUT the celery worker supervisor process goes into a fast spin creating celery worker processes that exit immediately, instead of giving up and dying after a few attempts.</p>

<p>Is there a nicer way to signal the death of a celery worker process from outside a celery task?</p>

<p>I had similar problem. I ended up creating a separate script to manage workers. </p>

<p>Create an infinite while loop, sleep for required time and then create/kill worker process.</p>

<pre><code>while True:
    time.sleep(5)
    # create workers if possible
    # kill stale workers if not required.
</code></pre>

python multithreading sockets celery python-multiprocessing