Python Celery - Permission Issue - Unable to upload files from a download task
<p>I have a Celery task whose job is to download files to a local directory, and then upload to a S3 bucket when download is complete.</p>

<p>My issue is that with a recent update of the workers, I'm getting permission denied errors when accessing the folder to upload.  The code was fundamentally unchanged other than from going to script-method to class based implementation.</p>

<p>I made a task just to create the directories, with a single method call:</p>

<pre><code>os.mkdirs(path, 777)
</code></pre>

<p>and it creates the directory with permissions <code>300</code></p>

<p>This is despite </p>

<pre><code>CELERYD_USERS='ubuntu'
CELERYD_GROUP='ubuntu'
CELERYD_CREATE_DIRS=1
</code></pre>

<p>in the config.</p>

<p>According to the <a href="http://docs.celeryproject.org/en/latest/tutorials/daemonizing.html#example-django-configuration" rel="nofollow">docs</a>, the last option allows celery to create a directory owned byt the user/group specified above.  That much is happening, but nothing is said about <em>permissions</em>.</p>

<p>Is there any way around this?</p>

<p>The CELERY_CREATE_DIR only tells celery create its operational directories:</p>

<blockquote>
  <p>Always create directories (log directory and pid file directory). Default is to only create directories when no custom logfile/pidfile set.</p>
</blockquote>

<p>I believe your problem is with the <code>os.mkdirs</code>. According to the <a href="https://docs.python.org/3/library/os.html#os.mkdir" rel="nofollow">os.mkdir doc</a>, <code>mode</code> may be ignored on your system and you need to use <code>os.chmod</code> to change the mode.</p>

python django celery