Import celery task without importing dependencies
<p>I have two modules </p>

<pre><code>alpha.py
beta.py
</code></pre>

<p><code>beta.py</code> can only be run on <code>beta.server</code> because it requires a licensed solver than only exists on <code>beta.server</code>. </p>

<p>Within <code>alpha.py</code>, there's a portion of code that calls: </p>

<pre><code>beta_task.apply_async(kwargs={...})
</code></pre>

<p>As such, it requires</p>

<pre><code>from beta import beta_task
</code></pre>

<p>Which in turn requires the magical proprietary module that is only available on <code>beta.server</code>.</p>

<p>I need to enable <code>alpha_task</code> to run on <code>alpha.server</code>, having the ability to call <code>beta_task</code> without having the <code>beta_task</code> code on the server. </p>

<p>Is this possible?</p>

<h1>UPDATE</h1>

<p>Also, can I prevent <code>beta.task</code> from running on <code>alpha.server</code>?</p>

<p>Since <code>alpha.py</code> import <code>beta.py</code>, the daemon finds <code>beta.task</code> and listens for tasks of this type: </p>

<pre><code>- ** ---------- [config]
- ** ---------- .&gt; app:         app_app
- ** ---------- .&gt; transport:   asdfasdfasd
- ** ---------- .&gt; results:     adfasdfasdf
- *** --- * --- .&gt; concurrency: 12 (prefork)
-- ******* ----
--- ***** ----- [queues]
 -------------- .&gt; celery           exchange=celery(direct) key=celery

 [tasks]
  . alpha.alpha_task
  . beta.beta_task
</code></pre>

<p>I ran into this before but never got it to work "right". I used a hacky workaround instead.</p>

<p>You can put the <code>import proprietary</code> statement in the <code>beta.beta_task</code> def itself.  Your 'alpha' file doesn't actually run the 'beta' def, it just uses celery's task decorator to dispatch a message about it.</p>

<p>While PEP standards dictate a module should be at the top on the outermost scope, it's actually common practice for widely used PyPi modules to place the import within a registration or called function so that uninstalled dependencies for the unused files won't break the package [for example, a caching library will import redis/memcached modules within the backend activation, so the 3rd party modules aren't needed unless that backend is used].</p>

<p>alpha.py</p>

<pre><code>from beta import beta_task

beta_task.apply_async(kwargs={...})
</code></pre>

<p>beta.py</p>

<pre><code>@task
def beta_task(args):
    import proprietary
    proprietary.foo()
</code></pre>

<p>For the Update about running different tasks on each server: that is all covered in the "routing" chapter of the celery docs: <a href="http://docs.celeryproject.org/en/latest/userguide/routing.html" rel="nofollow">http://docs.celeryproject.org/en/latest/userguide/routing.html</a>  </p>

<p>You basically configure different 'queues' (one for alpha, one for beta); start the workers to only handle the queues you specify; and either specify the route in the call to apply_async or configure the celery daemon to match a task to a route (there are several ways to do that, all explained in that chapter with examples.)</p>

python celery