Celery Beat - Worker Consuming Messages, But Never Acks. Them
<p>I've got a simple Django site that I'd like to add some scheduled tasks to via Celery / RabbitMQ. I'm stuck, because, while the beat scheduler pumps out tasks with no issues, the worker fails to consume them. The worker never marks them as acknowledged in RabbitMQ.</p>

<p>Here's my Celery configuration from <code>ego.settings</code>;</p>

<pre><code>from celery.schedules import crontab
...

# Celery configuration
BROKER_URL = os.environ.get('BROKER_URL', 'amqp://guest:guest@localhost:5672')
CELERY_RESULT_BACKEND = os.environ.get('CELERY_RESULT_BACKEND', 'disabled')
CELERY_ACCEPT_CONTENT = ['json']
CELERY_TASK_SERIALIZER = 'json'
CELERY_RESULT_SERIALIZER = 'json'

CELERYBEAT_SCHEDULE = {
    'account-notifications': {
        'task': 'ego.celery.account_alerts',
        'schedule': crontab(),
    },
}
</code></pre>

<p>My celery entrypoint (<code>ego.celery</code>;</p>

<pre><code>from __future__ import absolute_import

import os

from celery import Celery

# set the default Django settings module for the 'celery' program.
os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'ego.settings')

from django.conf import settings

app = Celery('ego')

# Using a string here means the worker will not have to
# pickle the object when using Windows.
app.config_from_object('django.conf:settings')
app.autodiscover_tasks(lambda: settings.INSTALLED_APPS)


@app.task
def account_alerts():
    print("nothing here, yet")
</code></pre>

<p>No, I didn't forget <code>ego.__init__</code> :)</p>

<pre><code>from __future__ import absolute_import

# This will make sure the app is always imported when
# Django starts so that shared_task will use this app.
from .celery import app as celery_app
</code></pre>

<p>The logs from beat look normal, but the worker is very quiet.</p>

<p>Beat instance:</p>

<pre><code>$ celery -A ego beat -s /tmp/celerybeat-schedule`
celery beat v3.1.23 (Cipater) is starting.
__    -    ... __   -        _
Configuration -&gt;
    . broker -&gt; amqp://guest:**@localhost:5672//
    . loader -&gt; celery.loaders.app.AppLoader
    . scheduler -&gt; celery.beat.PersistentScheduler
    . db -&gt; /Users/pnovotnak/Documents/Cyrus/ego/ego/celerybeat-schedule
    . logfile -&gt; [stderr]@%INFO
    . maxinterval -&gt; now (0s)

[2016-09-27 19:14:07,463: INFO/MainProcess] beat: Starting...
[2016-09-27 19:14:07,486: INFO/MainProcess] Scheduler: Sending due task account-notifications (ego.celery.account_alerts)
</code></pre>

<p>Worker instance:</p>

<pre><code>$ celery --autoreload -A ego worker -l debug


[2016-09-27 19:14:59,626: DEBUG/MainProcess] | Worker: Preparing bootsteps.
[2016-09-27 19:14:59,630: DEBUG/MainProcess] | Worker: Building graph...
[2016-09-27 19:14:59,630: DEBUG/MainProcess] | Worker: New boot order: {Beat, Timer, Hub, Queues (intra), Pool, Autoreloader, StateDB, Autoscaler, Consumer}
[2016-09-27 19:14:59,645: DEBUG/MainProcess] | Consumer: Preparing bootsteps.
[2016-09-27 19:14:59,646: DEBUG/MainProcess] | Consumer: Building graph...
[2016-09-27 19:14:59,652: DEBUG/MainProcess] | Consumer: New boot order: {Connection, Events, Heart, Mingle, Gossip, Agent, Tasks, Control, event loop}
[2016-09-27 19:14:59,655: DEBUG/MainProcess] | Worker: Starting Hub
[2016-09-27 19:14:59,655: DEBUG/MainProcess] ^-- substep ok
[2016-09-27 19:14:59,655: DEBUG/MainProcess] | Worker: Starting Pool
[2016-09-27 19:14:59,655: DEBUG/MainProcess] ^-- substep ok
[2016-09-27 19:14:59,655: DEBUG/MainProcess] | Worker: Starting Autoreloader
[2016-09-27 19:14:59,655: DEBUG/MainProcess] ^-- substep ok
[2016-09-27 19:14:59,655: DEBUG/MainProcess] | Worker: Starting Consumer
[2016-09-27 19:14:59,655: DEBUG/MainProcess] | Consumer: Starting Connection
[2016-09-27 19:14:59,663: DEBUG/MainProcess] Start from server, version: 0.9, properties: {'product': 'RabbitMQ', 'platform': 'Erlang/OTP', 'version': '3.6.5', 'copyright': 'Copyright (C) 2007-2016 Pivotal Software, Inc.', 'information': 'Licensed under the MPL.  See http://www.rabbitmq.com/', 'cluster_name': 'rabbit@d78e61a6690d', 'capabilities': {'authentication_failure_close': True, 'exchange_exchange_bindings': True, 'per_consumer_qos': True, 'basic.nack': True, 'direct_reply_to': True, 'connection.blocked': True, 'consumer_priorities': True, 'publisher_confirms': True, 'consumer_cancel_notify': True}}, mechanisms: ['PLAIN', 'AMQPLAIN'], locales: ['en_US']
[2016-09-27 19:14:59,664: DEBUG/MainProcess] Open OK!
[2016-09-27 19:14:59,664: INFO/MainProcess] Connected to amqp://guest:**@127.0.0.1:5672//
[2016-09-27 19:14:59,664: DEBUG/MainProcess] ^-- substep ok
[2016-09-27 19:14:59,664: DEBUG/MainProcess] | Consumer: Starting Events
[2016-09-27 19:14:59,673: DEBUG/MainProcess] Start from server, version: 0.9, properties: {'product': 'RabbitMQ', 'platform': 'Erlang/OTP', 'version': '3.6.5', 'copyright': 'Copyright (C) 2007-2016 Pivotal Software, Inc.', 'information': 'Licensed under the MPL.  See http://www.rabbitmq.com/', 'cluster_name': 'rabbit@d78e61a6690d', 'capabilities': {'authentication_failure_close': True, 'exchange_exchange_bindings': True, 'per_consumer_qos': True, 'basic.nack': True, 'direct_reply_to': True, 'connection.blocked': True, 'consumer_priorities': True, 'publisher_confirms': True, 'consumer_cancel_notify': True}}, mechanisms: ['PLAIN', 'AMQPLAIN'], locales: ['en_US']
[2016-09-27 19:14:59,674: DEBUG/MainProcess] Open OK!
[2016-09-27 19:14:59,674: DEBUG/MainProcess] using channel_id: 1
[2016-09-27 19:14:59,675: DEBUG/MainProcess] Channel open
[2016-09-27 19:14:59,676: DEBUG/MainProcess] ^-- substep ok
[2016-09-27 19:14:59,677: DEBUG/MainProcess] | Consumer: Starting Heart
[2016-09-27 19:14:59,678: DEBUG/MainProcess] ^-- substep ok
[2016-09-27 19:14:59,679: DEBUG/MainProcess] | Consumer: Starting Mingle
[2016-09-27 19:14:59,679: INFO/MainProcess] mingle: searching for neighbors
[2016-09-27 19:14:59,679: DEBUG/MainProcess] using channel_id: 1
[2016-09-27 19:14:59,680: DEBUG/MainProcess] Channel open
[2016-09-27 19:15:00,694: INFO/MainProcess] mingle: all alone
[2016-09-27 19:15:00,695: DEBUG/MainProcess] ^-- substep ok
[2016-09-27 19:15:00,695: DEBUG/MainProcess] | Consumer: Starting Gossip
[2016-09-27 19:15:00,695: DEBUG/MainProcess] using channel_id: 2
[2016-09-27 19:15:00,696: DEBUG/MainProcess] Channel open
[2016-09-27 19:15:00,700: DEBUG/MainProcess] ^-- substep ok
[2016-09-27 19:15:00,701: DEBUG/MainProcess] | Consumer: Starting Tasks
[2016-09-27 19:15:00,704: DEBUG/MainProcess] ^-- substep ok
[2016-09-27 19:15:00,704: DEBUG/MainProcess] | Consumer: Starting Control
[2016-09-27 19:15:00,704: DEBUG/MainProcess] using channel_id: 3
[2016-09-27 19:15:00,705: DEBUG/MainProcess] Channel open
[2016-09-27 19:15:00,708: DEBUG/MainProcess] ^-- substep ok
[2016-09-27 19:15:00,708: DEBUG/MainProcess] | Consumer: Starting event loop
[2016-09-27 19:15:00,709: WARNING/MainProcess] celery@Slug.local ready.
[2016-09-27 19:15:00,709: DEBUG/MainProcess] | Worker: Hub.register Autoreloader...
</code></pre>

<p>Here's the RabbitMQ admin view;</p>

<p><a href="http://i.stack.imgur.com/BldOa.png" rel="nofollow"><img src="http://i.stack.imgur.com/BldOa.png" alt="rabbitmq admin interface screenshot"></a></p>

<p>And that's it. Nothing after that. Obligatory additional software versions;</p>

<ul>
<li>RabbitMQ 3.6.5 (Erlang 19.0.7)</li>
<li>Python 3.5.2</li>
</ul>

<p>And finally, the contents of one of the queued messages;</p>

<pre><code>{
    "args": [],
    "callbacks": null,
    "chord": null,
    "errbacks": null,
    "eta": null,
    "expires": null,
    "id": "c474852a-dd60-4027-959d-5a9436337b17",
    "kwargs": {},
    "retries": 0,
    "task": "ego.celery.account_alerts",
    "taskset": null,
    "timelimit": [
        null,
        null
    ],
    "utc": true
}
</code></pre>

<p>I've found the solution! It's the <code>--autoreload</code> flag. Don't use it.</p>

<p>Lesson #2: Run celery by hand, not via invoke as I was doing. This just makes your life harder by doing bad things to the log output.</p>

<p>It seems the <code>--autoreload</code> flag causes the worker to break for some reason. Here's a comparison of the logs with the flag present / not.</p>

<p>In the following outputs I've changed the task to <code>return "nothing here, yet"</code> rather than <code>print()</code> it.</p>

<p>With <code>--autoreload</code>; </p>

<pre><code>$ celery --autoreload -A ego worker -l debug

[2016-09-27 20:43:15,106: DEBUG/MainProcess] | Worker: Preparing bootsteps.
[2016-09-27 20:43:15,109: DEBUG/MainProcess] | Worker: Building graph...
[2016-09-27 20:43:15,110: DEBUG/MainProcess] | Worker: New boot order: {Timer, Hub, Queues (intra), Pool, Autoscaler, Autoreloader, Beat, StateDB, Consumer}
[2016-09-27 20:43:15,115: DEBUG/MainProcess] | Consumer: Preparing bootsteps.
[2016-09-27 20:43:15,115: DEBUG/MainProcess] | Consumer: Building graph...
[2016-09-27 20:43:15,122: DEBUG/MainProcess] | Consumer: New boot order: {Connection, Events, Mingle, Tasks, Control, Agent, Gossip, Heart, event loop}

 -------------- celery@Slug.local v3.1.23 (Cipater)
---- **** -----
--- * ***  * -- Darwin-16.0.0-x86_64-i386-64bit
-- * - **** ---
- ** ---------- [config]
- ** ---------- .&gt; app:         ego:0x10486f748
- ** ---------- .&gt; transport:   amqp://guest:**@localhost:5672//
- ** ---------- .&gt; results:     disabled://
- *** --- * --- .&gt; concurrency: 8 (prefork)
-- ******* ----
--- ***** ----- [queues]
 -------------- .&gt; celery           exchange=celery(direct) key=celery


[tasks]
  . celery.backend_cleanup
  . celery.chain
  . celery.chord
  . celery.chord_unlock
  . celery.chunks
  . celery.group
  . celery.map
  . celery.starmap
  . ego.celery.account_alerts
  . ledger.tasks.AccountAlerts

[2016-09-27 20:43:15,125: DEBUG/MainProcess] | Worker: Starting Hub
[2016-09-27 20:43:15,125: DEBUG/MainProcess] ^-- substep ok
[2016-09-27 20:43:15,125: DEBUG/MainProcess] | Worker: Starting Pool
[2016-09-27 20:43:15,409: DEBUG/MainProcess] ^-- substep ok
[2016-09-27 20:43:15,410: DEBUG/MainProcess] | Worker: Starting Autoreloader
[2016-09-27 20:43:15,410: DEBUG/MainProcess] ^-- substep ok
[2016-09-27 20:43:15,410: DEBUG/MainProcess] | Worker: Starting Consumer
[2016-09-27 20:43:15,410: DEBUG/MainProcess] | Consumer: Starting Connection
[2016-09-27 20:43:15,420: DEBUG/MainProcess] Start from server, version: 0.9, properties: {'capabilities': {'per_consumer_qos': True, 'direct_reply_to': True, 'basic.nack': True, 'consumer_priorities': True, 'publisher_confirms': True, 'authentication_failure_close': True, 'connection.blocked': True, 'consumer_cancel_notify': True, 'exchange_exchange_bindings': True}, 'information': 'Licensed under the MPL.  See http://www.rabbitmq.com/', 'version': '3.6.5', 'product': 'RabbitMQ', 'cluster_name': 'rabbit@d78e61a6690d', 'copyright': 'Copyright (C) 2007-2016 Pivotal Software, Inc.', 'platform': 'Erlang/OTP'}, mechanisms: ['PLAIN', 'AMQPLAIN'], locales: ['en_US']
[2016-09-27 20:43:15,422: DEBUG/MainProcess] Open OK!
[2016-09-27 20:43:15,422: INFO/MainProcess] Connected to amqp://guest:**@127.0.0.1:5672//
[2016-09-27 20:43:15,422: DEBUG/MainProcess] ^-- substep ok
[2016-09-27 20:43:15,422: DEBUG/MainProcess] | Consumer: Starting Events
[2016-09-27 20:43:15,429: DEBUG/MainProcess] Start from server, version: 0.9, properties: {'capabilities': {'per_consumer_qos': True, 'direct_reply_to': True, 'basic.nack': True, 'consumer_priorities': True, 'publisher_confirms': True, 'authentication_failure_close': True, 'connection.blocked': True, 'consumer_cancel_notify': True, 'exchange_exchange_bindings': True}, 'information': 'Licensed under the MPL.  See http://www.rabbitmq.com/', 'version': '3.6.5', 'product': 'RabbitMQ', 'cluster_name': 'rabbit@d78e61a6690d', 'copyright': 'Copyright (C) 2007-2016 Pivotal Software, Inc.', 'platform': 'Erlang/OTP'}, mechanisms: ['PLAIN', 'AMQPLAIN'], locales: ['en_US']
[2016-09-27 20:43:15,430: DEBUG/MainProcess] Open OK!
[2016-09-27 20:43:15,431: DEBUG/MainProcess] using channel_id: 1
[2016-09-27 20:43:15,433: DEBUG/MainProcess] Channel open
[2016-09-27 20:43:15,433: DEBUG/MainProcess] ^-- substep ok
[2016-09-27 20:43:15,433: DEBUG/MainProcess] | Consumer: Starting Mingle
[2016-09-27 20:43:15,434: INFO/MainProcess] mingle: searching for neighbors
[2016-09-27 20:43:15,434: DEBUG/MainProcess] using channel_id: 1
[2016-09-27 20:43:15,435: DEBUG/MainProcess] Channel open
[2016-09-27 20:43:16,445: INFO/MainProcess] mingle: all alone
[2016-09-27 20:43:16,445: DEBUG/MainProcess] ^-- substep ok
[2016-09-27 20:43:16,445: DEBUG/MainProcess] | Consumer: Starting Tasks
[2016-09-27 20:43:16,451: DEBUG/MainProcess] ^-- substep ok
[2016-09-27 20:43:16,451: DEBUG/MainProcess] | Consumer: Starting Control
[2016-09-27 20:43:16,451: DEBUG/MainProcess] using channel_id: 2
[2016-09-27 20:43:16,452: DEBUG/MainProcess] Channel open
[2016-09-27 20:43:16,455: DEBUG/MainProcess] ^-- substep ok
[2016-09-27 20:43:16,456: DEBUG/MainProcess] | Consumer: Starting Gossip
[2016-09-27 20:43:16,456: DEBUG/MainProcess] using channel_id: 3
[2016-09-27 20:43:16,456: DEBUG/MainProcess] Channel open
[2016-09-27 20:43:16,467: DEBUG/MainProcess] ^-- substep ok
[2016-09-27 20:43:16,467: DEBUG/MainProcess] | Consumer: Starting Heart
[2016-09-27 20:43:16,468: DEBUG/MainProcess] ^-- substep ok
[2016-09-27 20:43:16,469: DEBUG/MainProcess] | Consumer: Starting event loop
[2016-09-27 20:43:16,470: WARNING/MainProcess] celery@Slug.local ready.
[2016-09-27 20:43:16,470: DEBUG/MainProcess] | Worker: Hub.register Autoreloader...



^C
worker: Hitting Ctrl+C again will terminate all running tasks!

worker: Warm shutdown (MainProcess)
[2016-09-27 20:44:33,368: DEBUG/MainProcess] | Worker: Closing Hub...
[2016-09-27 20:44:33,368: DEBUG/MainProcess] | Worker: Closing Pool...
[2016-09-27 20:44:33,368: DEBUG/MainProcess] | Worker: Closing Autoreloader...
[2016-09-27 20:44:33,368: DEBUG/MainProcess] | Worker: Closing Consumer...
[2016-09-27 20:44:33,368: DEBUG/MainProcess] | Worker: Stopping Consumer...
[2016-09-27 20:44:33,368: DEBUG/MainProcess] | Consumer: Closing Connection...
[2016-09-27 20:44:33,368: DEBUG/MainProcess] | Consumer: Closing Events...
[2016-09-27 20:44:33,368: DEBUG/MainProcess] | Consumer: Closing Mingle...
[2016-09-27 20:44:33,368: DEBUG/MainProcess] | Consumer: Closing Tasks...
[2016-09-27 20:44:33,368: DEBUG/MainProcess] | Consumer: Closing Control...
[2016-09-27 20:44:33,368: DEBUG/MainProcess] | Consumer: Closing Gossip...
[2016-09-27 20:44:33,369: DEBUG/MainProcess] | Consumer: Closing Heart...
[2016-09-27 20:44:33,369: DEBUG/MainProcess] | Consumer: Closing event loop...
[2016-09-27 20:44:33,369: DEBUG/MainProcess] | Consumer: Stopping event loop...
[2016-09-27 20:44:33,369: DEBUG/MainProcess] | Consumer: Stopping Heart...
[2016-09-27 20:44:33,369: DEBUG/MainProcess] | Consumer: Stopping Gossip...
[2016-09-27 20:44:33,372: DEBUG/MainProcess] Closed channel #3
[2016-09-27 20:44:33,373: DEBUG/MainProcess] | Consumer: Stopping Control...
[2016-09-27 20:44:33,374: DEBUG/MainProcess] Closed channel #2
[2016-09-27 20:44:33,374: DEBUG/MainProcess] | Consumer: Stopping Tasks...
[2016-09-27 20:44:33,374: DEBUG/MainProcess] Canceling task consumer...
[2016-09-27 20:44:33,375: DEBUG/MainProcess] | Consumer: Stopping Mingle...
[2016-09-27 20:44:33,375: DEBUG/MainProcess] | Consumer: Stopping Events...
[2016-09-27 20:44:33,375: DEBUG/MainProcess] | Consumer: Stopping Connection...
[2016-09-27 20:44:33,375: DEBUG/MainProcess] | Worker: Stopping Autoreloader...
[2016-09-27 20:44:33,375: DEBUG/MainProcess] | Worker: Stopping Pool...
[2016-09-27 20:44:34,388: DEBUG/MainProcess] | Worker: Stopping Hub...
[2016-09-27 20:44:34,388: DEBUG/MainProcess] | Consumer: Shutdown Heart...
[2016-09-27 20:44:34,389: DEBUG/MainProcess] | Consumer: Shutdown Gossip...
[2016-09-27 20:44:34,389: DEBUG/MainProcess] | Consumer: Shutdown Control...
[2016-09-27 20:44:34,389: DEBUG/MainProcess] | Consumer: Shutdown Tasks...
[2016-09-27 20:44:34,389: DEBUG/MainProcess] Canceling task consumer...
[2016-09-27 20:44:34,389: DEBUG/MainProcess] Closing consumer channel...
[2016-09-27 20:44:34,389: DEBUG/MainProcess] | Consumer: Shutdown Events...
[2016-09-27 20:44:34,390: DEBUG/MainProcess] Closed channel #1
[2016-09-27 20:44:34,391: DEBUG/MainProcess] | Consumer: Shutdown Connection...
[2016-09-27 20:44:34,392: DEBUG/MainProcess] Closed channel #1
[2016-09-27 20:44:34,396: DEBUG/MainProcess] removing tasks from inqueue until task handler finished
</code></pre>

<p>Without <code>--autoreload</code> (note "redelivered" status of the first two tasks);</p>

<pre><code>$ celery -A ego worker -l debug

[2016-09-27 20:44:42,075: DEBUG/MainProcess] | Worker: Preparing bootsteps.
[2016-09-27 20:44:42,079: DEBUG/MainProcess] | Worker: Building graph...
[2016-09-27 20:44:42,080: DEBUG/MainProcess] | Worker: New boot order: {Timer, Hub, Queues (intra), Pool, Autoscaler, Autoreloader, Beat, StateDB, Consumer}
[2016-09-27 20:44:42,087: DEBUG/MainProcess] | Consumer: Preparing bootsteps.
[2016-09-27 20:44:42,088: DEBUG/MainProcess] | Consumer: Building graph...
[2016-09-27 20:44:42,101: DEBUG/MainProcess] | Consumer: New boot order: {Connection, Agent, Events, Mingle, Gossip, Heart, Tasks, Control, event loop}

 -------------- celery@Slug.local v3.1.23 (Cipater)
---- **** -----
--- * ***  * -- Darwin-16.0.0-x86_64-i386-64bit
-- * - **** ---
- ** ---------- [config]
- ** ---------- .&gt; app:         ego:0x10516f710
- ** ---------- .&gt; transport:   amqp://guest:**@localhost:5672//
- ** ---------- .&gt; results:     disabled://
- *** --- * --- .&gt; concurrency: 8 (prefork)
-- ******* ----
--- ***** ----- [queues]
 -------------- .&gt; celery           exchange=celery(direct) key=celery


[tasks]
  . celery.backend_cleanup
  . celery.chain
  . celery.chord
  . celery.chord_unlock
  . celery.chunks
  . celery.group
  . celery.map
  . celery.starmap
  . ego.celery.account_alerts
  . ledger.tasks.AccountAlerts

[2016-09-27 20:44:42,105: DEBUG/MainProcess] | Worker: Starting Hub
[2016-09-27 20:44:42,105: DEBUG/MainProcess] ^-- substep ok
[2016-09-27 20:44:42,105: DEBUG/MainProcess] | Worker: Starting Pool
[2016-09-27 20:44:42,387: DEBUG/MainProcess] ^-- substep ok
[2016-09-27 20:44:42,387: DEBUG/MainProcess] | Worker: Starting Consumer
[2016-09-27 20:44:42,388: DEBUG/MainProcess] | Consumer: Starting Connection
[2016-09-27 20:44:42,398: DEBUG/MainProcess] Start from server, version: 0.9, properties: {'copyright': 'Copyright (C) 2007-2016 Pivotal Software, Inc.', 'capabilities': {'publisher_confirms': True, 'authentication_failure_close': True, 'exchange_exchange_bindings': True, 'consumer_priorities': True, 'per_consumer_qos': True, 'consumer_cancel_notify': True, 'direct_reply_to': True, 'basic.nack': True, 'connection.blocked': True}, 'information': 'Licensed under the MPL.  See http://www.rabbitmq.com/', 'platform': 'Erlang/OTP', 'cluster_name': 'rabbit@d78e61a6690d', 'product': 'RabbitMQ', 'version': '3.6.5'}, mechanisms: ['PLAIN', 'AMQPLAIN'], locales: ['en_US']
[2016-09-27 20:44:42,399: DEBUG/MainProcess] Open OK!
[2016-09-27 20:44:42,399: INFO/MainProcess] Connected to amqp://guest:**@127.0.0.1:5672//
[2016-09-27 20:44:42,400: DEBUG/MainProcess] ^-- substep ok
[2016-09-27 20:44:42,400: DEBUG/MainProcess] | Consumer: Starting Events
[2016-09-27 20:44:42,408: DEBUG/MainProcess] Start from server, version: 0.9, properties: {'copyright': 'Copyright (C) 2007-2016 Pivotal Software, Inc.', 'capabilities': {'publisher_confirms': True, 'authentication_failure_close': True, 'exchange_exchange_bindings': True, 'consumer_priorities': True, 'per_consumer_qos': True, 'consumer_cancel_notify': True, 'direct_reply_to': True, 'basic.nack': True, 'connection.blocked': True}, 'information': 'Licensed under the MPL.  See http://www.rabbitmq.com/', 'platform': 'Erlang/OTP', 'cluster_name': 'rabbit@d78e61a6690d', 'product': 'RabbitMQ', 'version': '3.6.5'}, mechanisms: ['PLAIN', 'AMQPLAIN'], locales: ['en_US']
[2016-09-27 20:44:42,409: DEBUG/MainProcess] Open OK!
[2016-09-27 20:44:42,409: DEBUG/MainProcess] using channel_id: 1
[2016-09-27 20:44:42,412: DEBUG/MainProcess] Channel open
[2016-09-27 20:44:42,412: DEBUG/MainProcess] ^-- substep ok
[2016-09-27 20:44:42,412: DEBUG/MainProcess] | Consumer: Starting Mingle
[2016-09-27 20:44:42,412: INFO/MainProcess] mingle: searching for neighbors
[2016-09-27 20:44:42,413: DEBUG/MainProcess] using channel_id: 1
[2016-09-27 20:44:42,426: DEBUG/MainProcess] Channel open
[2016-09-27 20:44:43,445: INFO/MainProcess] mingle: all alone
[2016-09-27 20:44:43,445: DEBUG/MainProcess] ^-- substep ok
[2016-09-27 20:44:43,445: DEBUG/MainProcess] | Consumer: Starting Gossip
[2016-09-27 20:44:43,445: DEBUG/MainProcess] using channel_id: 2
[2016-09-27 20:44:43,446: DEBUG/MainProcess] Channel open
[2016-09-27 20:44:43,450: DEBUG/MainProcess] ^-- substep ok
[2016-09-27 20:44:43,450: DEBUG/MainProcess] | Consumer: Starting Heart
[2016-09-27 20:44:43,451: DEBUG/MainProcess] ^-- substep ok
[2016-09-27 20:44:43,451: DEBUG/MainProcess] | Consumer: Starting Tasks
[2016-09-27 20:44:43,456: DEBUG/MainProcess] ^-- substep ok
[2016-09-27 20:44:43,456: DEBUG/MainProcess] | Consumer: Starting Control
[2016-09-27 20:44:43,456: DEBUG/MainProcess] using channel_id: 3
[2016-09-27 20:44:43,457: DEBUG/MainProcess] Channel open
[2016-09-27 20:44:43,460: DEBUG/MainProcess] ^-- substep ok
[2016-09-27 20:44:43,460: DEBUG/MainProcess] | Consumer: Starting event loop
[2016-09-27 20:44:43,461: WARNING/MainProcess] celery@Slug.local ready.
[2016-09-27 20:44:43,462: DEBUG/MainProcess] | Worker: Hub.register Pool...
[2016-09-27 20:44:43,462: DEBUG/MainProcess] basic.qos: prefetch_count-&gt;32
[2016-09-27 20:44:43,463: INFO/MainProcess] Received task: ego.celery.account_alerts[bc4c5b05-6306-4a57-b1a0-eca1e9282c13]
[2016-09-27 20:44:43,463: DEBUG/MainProcess] TaskPool: Apply &lt;function _fast_trace_task at 0x105263950&gt; (args:('ego.celery.account_alerts', 'bc4c5b05-6306-4a57-b1a0-eca1e9282c13', [], {}, {'is_eager': False, 'delivery_info': {'exchange': 'celery', 'routing_key': 'celery', 'redelivered': True, 'priority': 0}, 'correlation_id': 'bc4c5b05-6306-4a57-b1a0-eca1e9282c13', 'expires': None, 'chord': None, 'task': 'ego.celery.account_alerts', 'timelimit': [None, None], 'callbacks': None, 'headers': {}, 'utc': True, 'errbacks': None, 'eta': None, 'kwargs': {}, 'args': [], 'hostname': 'celery@Slug.local', 'taskset': None, 'group': None, 'retries': 0, 'id': 'bc4c5b05-6306-4a57-b1a0-eca1e9282c13', 'reply_to': '656c174e-c251-3e02-8132-df741c11a046'}) kwargs:{})
[2016-09-27 20:44:43,465: DEBUG/MainProcess] Task accepted: ego.celery.account_alerts[bc4c5b05-6306-4a57-b1a0-eca1e9282c13] pid:63322
[2016-09-27 20:44:43,478: INFO/MainProcess] Task ego.celery.account_alerts[bc4c5b05-6306-4a57-b1a0-eca1e9282c13] succeeded in 0.013773033046163619s: 'nothing here, yet'
[2016-09-27 20:44:45,458: INFO/MainProcess] Received task: ego.celery.account_alerts[b7673a47-dd76-4f1d-b414-03744fcfac22]
[2016-09-27 20:44:45,459: DEBUG/MainProcess] TaskPool: Apply &lt;function _fast_trace_task at 0x105263950&gt; (args:('ego.celery.account_alerts', 'b7673a47-dd76-4f1d-b414-03744fcfac22', [], {}, {'is_eager': False, 'delivery_info': {'exchange': 'celery', 'routing_key': 'celery', 'redelivered': True, 'priority': 0}, 'correlation_id': 'b7673a47-dd76-4f1d-b414-03744fcfac22', 'expires': None, 'chord': None, 'task': 'ego.celery.account_alerts', 'timelimit': [None, None], 'callbacks': None, 'headers': {}, 'utc': True, 'errbacks': None, 'eta': None, 'kwargs': {}, 'args': [], 'hostname': 'celery@Slug.local', 'taskset': None, 'group': None, 'retries': 0, 'id': 'b7673a47-dd76-4f1d-b414-03744fcfac22', 'reply_to': '656c174e-c251-3e02-8132-df741c11a046'}) kwargs:{})
[2016-09-27 20:44:45,461: DEBUG/MainProcess] Task accepted: ego.celery.account_alerts[b7673a47-dd76-4f1d-b414-03744fcfac22] pid:63327
[2016-09-27 20:44:45,462: INFO/MainProcess] Task ego.celery.account_alerts[b7673a47-dd76-4f1d-b414-03744fcfac22] succeeded in 0.0010689240298233926s: 'nothing here, yet'
[2016-09-27 20:45:00,002: INFO/MainProcess] Received task: ego.celery.account_alerts[2d464f3a-36f3-4f45-b303-aa477bc30545]
[2016-09-27 20:45:00,002: DEBUG/MainProcess] TaskPool: Apply &lt;function _fast_trace_task at 0x105263950&gt; (args:('ego.celery.account_alerts', '2d464f3a-36f3-4f45-b303-aa477bc30545', [], {}, {'is_eager': False, 'delivery_info': {'exchange': 'celery', 'routing_key': 'celery', 'redelivered': False, 'priority': 0}, 'correlation_id': '2d464f3a-36f3-4f45-b303-aa477bc30545', 'expires': None, 'chord': None, 'task': 'ego.celery.account_alerts', 'timelimit': [None, None], 'callbacks': None, 'headers': {}, 'utc': True, 'errbacks': None, 'eta': None, 'kwargs': {}, 'args': [], 'hostname': 'celery@Slug.local', 'taskset': None, 'group': None, 'retries': 0, 'id': '2d464f3a-36f3-4f45-b303-aa477bc30545', 'reply_to': '656c174e-c251-3e02-8132-df741c11a046'}) kwargs:{})
[2016-09-27 20:45:00,003: DEBUG/MainProcess] Task accepted: ego.celery.account_alerts[2d464f3a-36f3-4f45-b303-aa477bc30545] pid:63321
[2016-09-27 20:45:00,003: INFO/MainProcess] Task ego.celery.account_alerts[2d464f3a-36f3-4f45-b303-aa477bc30545] succeeded in 0.000625498010776937s: 'nothing here, yet'
[2016-09-27 20:46:00,002: INFO/MainProcess] Received task: ego.celery.account_alerts[f8eba8f2-1c48-48bd-a221-2676c735299e]
[2016-09-27 20:46:00,003: DEBUG/MainProcess] TaskPool: Apply &lt;function _fast_trace_task at 0x105263950&gt; (args:('ego.celery.account_alerts', 'f8eba8f2-1c48-48bd-a221-2676c735299e', [], {}, {'is_eager': False, 'delivery_info': {'exchange': 'celery', 'routing_key': 'celery', 'redelivered': False, 'priority': 0}, 'correlation_id': 'f8eba8f2-1c48-48bd-a221-2676c735299e', 'expires': None, 'chord': None, 'task': 'ego.celery.account_alerts', 'timelimit': [None, None], 'callbacks': None, 'headers': {}, 'utc': True, 'errbacks': None, 'eta': None, 'kwargs': {}, 'args': [], 'hostname': 'celery@Slug.local', 'taskset': None, 'group': None, 'retries': 0, 'id': 'f8eba8f2-1c48-48bd-a221-2676c735299e', 'reply_to': '656c174e-c251-3e02-8132-df741c11a046'}) kwargs:{})
[2016-09-27 20:46:00,004: DEBUG/MainProcess] Task accepted: ego.celery.account_alerts[f8eba8f2-1c48-48bd-a221-2676c735299e] pid:63325
[2016-09-27 20:46:00,005: INFO/MainProcess] Task ego.celery.account_alerts[f8eba8f2-1c48-48bd-a221-2676c735299e] succeeded in 0.0016709549818187952s: 'nothing here, yet'
</code></pre>

python django celery