Python service - writing filename with timestamp
<p>I wrote a Python script that will run indefinitely. It monitors a directory using <code>PyInotify</code> and uses the <code>Multiprocessing</code> module to run any new files created in those directories through an external script. That all works great. </p>

<p>The problem I am having is writing the output to a file. The filename I chose uses the current date (using <code>datetime.now</code>) and should, theoretically, roll on the hour, every hour.</p>

<pre><code>now = datetime.now()
filename = "/data/db/meta/%s-%s-%s-%s.gz" % (now.year, now.month, now.day, now.hour)
with gzip.open(filename, 'ab') as f:
    f.write(json.dumps(data) + "\n")
    f.close() #Unsure if I need this, here for debug
</code></pre>

<p>Unfortunately, when the hour rolls on -- the output stops and never returns. No exceptions are thrown, it just stops working. </p>

<pre><code>total 2.4M
drwxrwxr-x 2 root root 4.0K Sep  8 08:01 .
drwxrwxr-x 4 root root  12K Aug 29 16:04 ..
-rw-r--r-- 1 root root 446K Aug 29 16:59 2016-8-29-16.gz
-rw-r--r-- 1 root root 533K Aug 30 08:59 2016-8-30-8.gz
-rw-r--r-- 1 root root  38K Sep  7 10:59 2016-9-7-10.gz
-rw-r--r-- 1 root root  95K Sep  7 14:59 2016-9-7-14.gz
-rw-r--r-- 1 root root 292K Sep  7 15:59 2016-9-7-15.gz #Manually run
-rw-r--r-- 1 root root 834K Sep  8 08:59 2016-9-8-8.gz
</code></pre>

<blockquote>
  <blockquote>
    <p>Those files aren't really owned by root, just changed them for public consumption</p>
  </blockquote>
</blockquote>

<p>As you can see, all of the files timestamps end at :59 and the next hour never happens.</p>

<p>Is there something that I should take into consideration when doing this? Is there something that I am missing running a Python script indefinitely? </p>

<hr>

<p>After taking a peek. It seems as if PyInotify was my problem. 
See here (<a href="http://unix.stackexchange.com/questions/164794/why-doesnt-inotifywatch-detect-changes-on-added-files">http://unix.stackexchange.com/questions/164794/why-doesnt-inotifywatch-detect-changes-on-added-files</a>) </p>

<p>I adjusted your code to change the file name each minute, which speeds up debugging quite a bit and yet still tests the hypothesis.</p>

<pre><code>import datetime
import gzip, time
from os.path import expanduser
while True:
    now = datetime.datetime.now()
    filename = expanduser("~")+"/%s-%s-%s-%s-%s.gz" % (now.year, now.month, now.day, now.hour, now.minute)
    with gzip.open(filename, 'a') as f:
        f.write(str(now) + "\n")
        f.write("Data Dump here" + "\n")
    time.sleep(10)
</code></pre>

<p>This seems to run without an issue. Changing the time-zone of my pc was also picked up and dealt with. I would suspect, given the above, that your error may lie elsewhere and some judicious debug printing of values at key points is needed. Try using a more granular file name as above to speed up the debugging.</p>

python multiprocess pyinotify