Celery kills processes
<p>I use celery 3.1.23. </p>

<p>Task code is very simple:</p>

<pre><code>@app.task(bind=True, max_retries=None, default_retry_delay=settings.CELERY_RECONNECT_TIME)
def analize_text(self, **kwargs):
    print 'test'
    return 1
</code></pre>

<p>I launch celery this command:</p>

<blockquote>
  <p>celery -A tasks worker --loglevel=info --concurrency=4 -n analize_text
  -Q analize.analize_text -Ofair</p>
</blockquote>

<p>So, i use 4 CPU. 
The problem is that celery periodically kills workers and starts new ones instead. New worker has new number in log and new PID.
Here is my log <a href="https://dpaste.de/N1Vk" rel="nofollow">https://dpaste.de/N1Vk</a></p>

<p>Why does it do it? Is this bug?</p>

<p>It depends on CELERY_MAX_TASKS_PER_CHILD i.e. number of tasks a worker shall complete before being recycled, default is no limit.</p>

<p><a href="http://docs.celeryproject.org/en/latest/configuration.html#std:setting-CELERYD_MAX_TASKS_PER_CHILD" rel="nofollow">http://docs.celeryproject.org/en/latest/configuration.html#std:setting-CELERYD_MAX_TASKS_PER_CHILD</a></p>

<p>So,your celery configuration might be having this limit set as some low number, hence your workers are getting recycled.
You can see more information in celery <a href="http://docs.celeryproject.org/en/latest/userguide/workers.html#statistics" rel="nofollow">stats</a>.</p>

python multiprocessing celery