Cannot subclass multiprocessing Queue in Python 3.5
<p>My eventual goal is to redirect the <code>stdout</code> from several subprocesses to some queues, and print those out somewhere (maybe in a little GUI).</p>

<p>The first step is to subclass <code>Queue</code> into an object that behaves much like the <code>stdout</code>. But that is where I got stuck. Subclassing the multiprocessing <code>Queue</code> seems impossible in Python v3.5.</p>

<pre><code># This is a Queue that behaves like stdout
# Unfortunately, doesn't work in Python 3.5   :-(
class StdoutQueue(Queue):
    def __init__(self,*args,**kwargs):
        Queue.__init__(self,*args,**kwargs, ctx='')

    def write(self,msg):
        self.put(msg)

    def flush(self):
        sys.__stdout__.flush()
</code></pre>

<p>I found this snippet in the following post (probably Python 3.5 did not yet exist at that moment): <a href="http://stackoverflow.com/questions/23947281/python-multiprocessing-redirect-stdout-of-a-child-process-to-a-tkinter-text">Python multiprocessing redirect stdout of a child process to a Tkinter Text</a></p>

<p>In Python v3.5 you stumble on strange error messages when subclassing the multiprocessing <code>Queue</code> class. I found two bug reports describing the issue:</p>

<p><a href="https://bugs.python.org/issue21367" rel="nofollow">https://bugs.python.org/issue21367</a></p>

<p><a href="https://bugs.python.org/issue19895" rel="nofollow">https://bugs.python.org/issue19895</a></p>

<p>I have 2 questions:</p>

<ol>
<li>Suppose I want to stick to Python v3.5 - going to a previous version is not really an option. What workaround can I use to subclass the multiprocessing Queue somehow?</li>
<li>Is the bug still around if I upgrade to Python v3.6?</li>
</ol>

<hr>

<p><strong>EDIT :</strong></p>

<p>There is a known issue when you try to subclass the <code>Queue</code> class found in here:</p>

<pre><code>from multiprocessing import Queue    # &lt;- known issue: you cannot subclass
                                     #    this Queue class, because it is
                                     #    not a genuine python class.
</code></pre>

<p>But the following should work:</p>

<pre><code>from multiprocessing.queues import Queue   # &lt;- from this Queue class, you
                                           #    should be able to make a
                                           #    subclass. But Python 3.5
                                           #    refuses :-(
</code></pre>

<p>Sadly, even that doesn't work in Python v3.5. You get the following error:</p>

<pre><code>    C:\Users\..\myFolder &gt; python myTest.py

        Traceback (most recent call last):
            File "myTest.py", line 49, in &lt;module&gt;
              q = StdoutQueue()
            File "myTest.py", line 22, in __init__
              super(StdoutQueue,self).__init__(*args,**kwargs)
        TypeError: __init__() missing 1 required keyword-only argument: 'ctx'
</code></pre>

<hr>

<p><strong>EDIT :</strong></p>

<p>Thank you Darth Kotik for solving the problem! Here is the complete code, updated with his solution. Now it works.</p>

<pre><code>import sys
import time
import multiprocessing as mp
import multiprocessing.queues as mpq
from threading import Thread
from tkinter import *

'''-------------------------------------------------------------------'''
'''                SUBCLASSING THE MULTIPROCESSING QUEUE              '''
'''                                                                   '''
'''         ..and make it behave as a general stdout io               '''
'''-------------------------------------------------------------------'''
# The StdoutQueue is a Queue that behaves like stdout.
# We will subclass the Queue class from the multiprocessing package
# and give it the typical stdout functions.
#
# (1) First issue
# Subclassing multiprocessing.Queue or multiprocessing.SimpleQueue
# will not work, because these classes are not genuine
# python classes.
# Therefore, you need to subclass multiprocessing.queues.Queue or
# multiprocessing.queues.SimpleQueue . This issue is known, and is not
# the reason for asking this question. But I mention it here, for
# completeness.
#
# (2) Second issue
# There is another problem that arises only in Python V5 (and beyond).
# When subclassing multiprocessing.queues.Queue, you have to provide
# a 'multiprocessing context'. Not doing that, leads to an obscure error
# message, which is in fact the main topic of this question. Darth Kotik
# solved it.
# His solution is visible in this code:
class StdoutQueue(mpq.Queue):

    def __init__(self,*args,**kwargs):
        ctx = mp.get_context()
        super(StdoutQueue, self).__init__(*args, **kwargs, ctx=ctx)

    def write(self,msg):
        self.put(msg)

    def flush(self):
        sys.__stdout__.flush()


'''-------------------------------------------------------------------'''
'''                           TEST SETUP                              '''
'''-------------------------------------------------------------------'''

# This function takes the text widget and a queue as inputs.
# It functions by waiting on new data entering the queue, when it
# finds new data it will insert it into the text widget.
def text_catcher(text_widget,queue):
    while True:
        text_widget.insert(END, queue.get())

def test_child(q):
    # This line only redirects stdout inside the current process
    sys.stdout = q
    # or sys.stdout = sys.__stdout__ if you want to print the child to the terminal
    print('child running')

def test_parent(q):
    # Again this only redirects inside the current (main) process
    # commenting this like out will cause only the child to write to the widget
    sys.stdout = q
    print('parent running')
    time.sleep(0.5)
    mp.Process(target=test_child,args=(q,)).start()

if __name__ == '__main__':
    gui_root = Tk()
    gui_txt = Text(gui_root)
    gui_txt.pack()
    q = StdoutQueue()
    gui_btn = Button(gui_root, text='Test', command=lambda:test_parent(q),)
    gui_btn.pack()

    # Instantiate and start the text monitor
    monitor = Thread(target=text_catcher,args=(gui_txt,q))
    monitor.daemon = True
    monitor.start()

    gui_root.mainloop()
</code></pre>

<pre><code>&gt;&gt;&gt; import multiprocessing
&gt;&gt;&gt; type(multiprocessing.Queue)
&lt;class 'method'&gt;
AttributeError: module 'multiprocessing' has no attribute 'queues'
&gt;&gt;&gt; import multiprocessing.queues
&gt;&gt;&gt; type(multiprocessing.queues.Queue)
&lt;class 'type'&gt;
</code></pre>

<p>So as you can see <code>multiprocessing.Queue</code> is just constructor method for <code>multiprocessing.queues.Queue</code> class. If you want to make a child class just do <code>class MyQueue(multiprocessing.queues.Queue)</code></p>

<p>You can see source of this method <a href="https://github.com/python/cpython/blob/master/Lib/multiprocessing/context.py#L99-L102" rel="nofollow">here</a></p>

<p><strong>EDIT</strong>:
Okay. I got your problem now. As you can see  on a link above, <code>multiprocessing.Queue</code> passes <code>ctx</code> argument to Queue. So I managed to get it working by doing it myself in <code>__init__</code> method. I don't completely inderstand where <code>BaseContext</code> object supposed to get <code>_name</code> attribute, so I passed it manually.</p>

<pre><code>def __init__(self,*args,**kwargs):
    from multiprocessing.context import BaseContext
    ctx = BaseContext()
    ctx._name = "Name"
    super(StdoutQueue,self).__init__(*args,**kwargs, ctx=ctx)
</code></pre>

<p><strong>EDIT2</strong>: Turned out docs have some information about context <a href="https://docs.python.org/3/library/multiprocessing.html#contexts-and-start-methods" rel="nofollow">here</a>. So instead of manually creating it like I did you can do </p>

<pre><code>import multiprocessing
ctx = multiprocessing.get_context()
</code></pre>

<p>It will create proper context with <code>_name</code> set (to 'fork' in your particular case) and you can pass it to your queue.</p>

python python-3.x multiprocessing python-multiprocessing