Infinite Loop Exits Automatically while Using multithreading and Queues
<p>I'm trying to run a function that has an infinite loop (to check data after few seconds of delay) using multithreading. Since I read data from a csv file, I'm also using Queues.</p>

<p>My current function fine when I do not use multithreading/queues but when I use them, the function only loops once and then stops.</p>

<p>Here's my function that has a infinite loop. Please note that the first while True loop is for threads (in case I use less number of threads than rows in csv) , the function only requires the second while True loop.</p>

<pre><code>def doWork(q):
    while True:
        #logging.info('Thread Started')
        row=q.get()

        url = row[0]
        target_price = row[1]
        #logging.info('line 79')

        while True:
            delay=randint(5,10)
            headers = {'User-Agent': generate_user_agent()}
            print datetime.datetime.strftime(datetime.datetime.now(), '%Y-%m-%d %H:%M:%S')+': '+'Sleeping for ' + str(delay) + ' seconds'
            #logging.info('line 81')
            eventlet.sleep(delay)
            try:
                #logging.info('line 85')
                with requests.Session() as s:
                    #logging.info('line 87')
                    with eventlet.Timeout(10, False):
                        page = s.get(url,headers=headers,proxies=proxyDict,verify=False)
                    #logging.info('line 89')
                    tree = html.fromstring(page.content)
                    #logging.info('line 91')
                    price = tree.xpath('//div[@class="a-row a-spacing-mini olpOffer"]/div[@class="a-column a-span2 olpPriceColumn"]/span[@class="a-size-large a-color-price olpOfferPrice a-text-bold"]/text()')[0]
                    title = tree.xpath('//h1/text()')[0]
                    #logging.info('line 93')
                    new_price = re.findall("[-+]?\d+[\.]?\d+[eE]?[-+]?\d*", price)[0]
                    #logging.info('line 95')
                    old_price = new_price
                    #logging.info('line 97')
                    #print price
                    print new_price
                    print title + 'Current price:' + new_price
                    if float(new_price)&lt;float(target_price):
                        print 'Lower price found!'
                        mydriver = webdriver.Chrome()
                        send_simple_message()
                        login(mydriver)
                        print 'Old Price: ' + old_price
                        print 'New Price: ' + new_price
                    else:
                        print 'Trying again'
                q.task_done()   
            except Exception as e:
                print e
                print 'Error!'
                q.task_done()
</code></pre>

<p>And here is my thread driver function;</p>

<pre><code>q = Queue(concurrent * 2)

if __name__ == "__main__":

    for i in range(concurrent):
        t = Thread(target=doWork,args=(q,))
        t.daemon = True
        t.start()
    try:
        with open('products.csv','r') as f:
            reader = csv.reader(f.read().splitlines())
            for row in reader:
                q.put((row[0],row[1]))
        q.join()
    except KeyboardInterrupt:
        sys.exit(1) 
</code></pre>

<p>For anyone that is facing the same issue, here's how I solved it.</p>

<p>I removed q.task_done() from the while loop and put it outside the while loop. This is working as intended but I'm not sure if this is the right approach.</p>

python multithreading