Python Thread getting items from Queue - how do you determine thread has not called task_done()?
<pre><code>dispQ = Queue.Queue()
stop_thr_event = threading.Event()

def worker (stop_event):
    while not stop_event.wait(0):
        try:
            job = dispQ.get(timeout=1)
            job.do_work()  # does some work and may put more jobs in the dispQ
            dispQ.task_done()
        except Queue.Empty, msg:
            continue


t1 = threading.Thread( target=worker, args=(stop_thr_event,) )
t1.daemon = True
t1.start()

t2 = threading.Thread( target=worker, args=(stop_thr_event,) )
t2.daemon = True
t2.start()

# put some initial jobs in the dispQ here

while True:
    if dispQ.qsize() == 0:
        break
    else:
        # do something important

stop_thr_event.set()
t1.join()
t2.join()
</code></pre>

<p>The Problem:<br>
    dispQ.qsize is 0 when inside worker func, job is creating more items and hence breaking out the loop (after breaking out of the loop, there are more jobs in dispQ)</p>

<p>I need do something like:<br>
if dispQ.qsize() == 0 and there is no item being worked inside the worker func, then break out of the while loop</p>

<p>i.e. task_done() hasnt called yet after get() was called</p>

<p>it would be nice if it had dispQ.join() with a timeout</p>

<p>Here is the source for Queue.join, it looks like you can just use <code>queue.unfinished_tasks &gt; 0</code> to check if any tasks are running or in queue. You may want to acquire (then release) the lock to ensure that the queue doesn't change while you're checking its status. (I believe) You don't need to do this if you can guarantee that no items will be added to the queue after the last task finishes.</p>

<pre><code>def join(self):
    # ...
    self.all_tasks_done.acquire()
    try:
        while self.unfinished_tasks:
            self.all_tasks_done.wait()
    finally:
        self.all_tasks_done.release()
</code></pre>

python multithreading queue