Retrieving result from celery worker constantly
<p>I have an web app in which I am trying to use celery to load background tasks from a database. I am currently loading the database upon request, but would like to load the tasks on an hourly interval and have them work in the background. I am using flask and am coding in python.I have redis running as well. </p>

<p>So far using celery I have gotten the worker to process the task and the beat to send the tasks to the worker on an interval. But I want to retrieve the results[a dataframe or query] from the worker and if the result is not ready then it should load the previous result of the worker. </p>

<p>Any ideas on how to do this?</p>

<p><strong>Edit</strong></p>

<p>I am retrieving the results from a database using sqlalchemy and I am rendering the results in a webpage. I have my homepage which has all the various links which all lead to different graphs which I want to be loaded in the background so the user does not have to wait long loading times. </p>

<p>The Celery <a href="http://docs.celeryproject.org/en/latest/userguide/tasks.html#tasks">Task</a> is being executed by a Worker, and it's Result is being stored in the <a href="http://docs.celeryproject.org/en/latest/getting-started/first-steps-with-celery.html#keeping-results">Celery Backend</a>.</p>

<p>If I get you correctly, then I think you got few options:</p>

<ol>
<li><a href="http://docs.celeryproject.org/en/latest/userguide/tasks.html#ignore-results-you-don-t-want">Ignore the result</a> of the graph-loading-task, store what ever you need, as a side effect of the task, in your database. When needed, query for the most recent result in that database. If the DB is Redis, you may find <a href="http://redis.io/commands/ZADD">ZADD</a> and <a href="http://redis.io/commands/ZRANGE">ZRANGE</a> suitable. This way you'll get the new if available, or the previous if not.</li>
<li><p>You can look for the <a href="http://docs.celeryproject.org/en/latest/reference/celery.result.html#celery.result.AsyncResult">result of a task if you provide it's id</a>. You can do this when you want to find out the status, something like (where <code>celery</code> is the Celery app): <code>result = celery.AsyncResult(&lt;the task id&gt;)</code></p></li>
<li><p>Use <a href="http://docs.celeryproject.org/en/latest/userguide/tasks.html#avoid-launching-synchronous-subtasks">callback</a> to update farther when new result is ready.</p></li>
<li>Let a background thread <a href="http://docs.celeryproject.org/en/latest/reference/celery.result.html#celery.result.AsyncResult.wait">wait</a> for the AsyncResult, or <a href="http://docs.celeryproject.org/en/latest/reference/celery.result.html#celery.result.ResultSet.join_native">native_join</a>, which is supported with Redis, and update accordingly (not recommended)</li>
</ol>

<p>I personally used option #1 in similar cases (using MongoDB) and found it to be very maintainable and flexible. But possibly, due the nature of your UI, option #3 will more suitable for you needs.</p>

python database celery