Difference between Queue and Sets Python
<p>In case there is multi-threads and one function which adds a value to a list and another function which takes that value. What would the difference be with:</p>

<pre><code>import queue
scrape = queue.Queue()
def scrape():
    scrape.put('example')
def send():
    example = scrape.get()
    print (example)

scrape = set([])
def scrape():
    scrape.add('example')
def send():
    example = scrape.pop()
    print (example)
</code></pre>

<p>Why do people use the queue module which is 170-180 lines with if conditions slowing the process for this situation if they can use sets which also gives them the advantage of duplicates filtering.</p>

<p><code>Queues</code> maintain ordering of possibly non-unique elements. <code>Sets</code>, on the other hand, do not maintain ordering and may not contain duplicates.</p>

<p>In your case you may need to keep a record of each thing scraped and/or the relative order in which it was scraped. In that case, use <code>queues</code>. If you just want a list of the unique things you scraped, and you don't care about the relative order in which you scraped them, use <code>sets</code>. </p>

<p>As <code>@mata</code> points out, a <code>queue</code> should be used if multiple threads are producing and consuming to/from it. <code>Queues</code> implement the blocking functionality  needed to work with producer/consumer <code>threads</code>. <code>Queues</code> are thread-safe, <code>sets</code> are not. </p>

<p>In this example from the docs:</p>

<pre><code>def worker():
    while True:
        item = q.get()
        do_work(item)
        q.task_done()

q = Queue()
for i in range(num_worker_threads):
     t = Thread(target=worker)
     t.daemon = True
     t.start()

for item in source():
    q.put(item)

q.join()   # block until all tasks are done
</code></pre>

<p><code>get</code> in the consumer thread (i.e. <code>worker</code>) blocks until there is something in the <code>queue</code> to get, <code>join</code> in the producer thread blocks until each item that it put into the <code>queue</code> is consumed, and <code>task_done</code> in the consumer thread tells the queue that the item it got has been consumed. </p>

python multithreading set queue