Using Logging with Multiprocessing Module
<p>I need to create a single logger that can be used throughout my python package, but some of the functions implement multiprocessing.  I want all those functions to write to the same log file just like everything else.</p>

<p>I know at Python 3.2+ there is a built in way to do this, but I need to back port to Python 2.7.x as well.</p>

<p>Is there any code out there that works well with both multiprocessing and non-multiprocessing functions for logging?</p>

<p>Normally, I would create a log as such:</p>

<pre><code>module = sys.modules['__main__'].__file__
logging.basicConfig(stream=sys.stderr, level=logging.DEBUG,
                format='%(name)s (%(levelname)s): %(message)s')
log = logging.getLogger(module)
fh = RotatingFileHandler(arguments.o, mode='a', maxBytes=2*1024*1024,
                        backupCount=2, encoding=None, delay=0)
formatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %    (message)s')
fh.setFormatter(formatter)
fh.setLevel(logging.DEBUG)
log.addHandler(fh)
</code></pre>

<p>Then the output would write to a single file.  It works great, but when I implement this, this code will create multiple files, which I do not want.</p>

<p>Any ideas?</p>

<p>Thank you</p>

<p>The functionality to do this in Python >= 3.2 (via the <code>QueueHandler</code> and <code>QueueListener</code> classes, as described <a href="http://plumberjack.blogspot.co.uk/2010/09/using-logging-with-multiprocessing.html" rel="nofollow">here</a>) is available for Python 2.7 through the <a href="https://pypi.python.org/pypi/logutils" rel="nofollow"><code>logutils</code></a> project.</p>

python python-2.7 logging