How to log to different files based on from which python process logging is called?
<p>I am working on a test framework.  Each test is launched as a new python multiprocessing process.
There is one master log file and individual log files corresponding to each test.
There is a master logger created at the launch of framework code and a new logger created in each test process.  Test loggers log to both - it's own log file and master log file.</p>

<p>There are multiple libraries that can be used by any of the test.<br>
Currently there is no logging done in library function. In order to add logging to the library functions, logger object needs to passed as a parameter to this library function. To achieve this, every function signature in library modules and function call will have to be modified, which is not practical.</p>

<p>As I understand, I cannot have module level logger because  module level logger will log to different file for each module and not for each test process. </p>

<p>Can you suggest a solution where I don't have to pass log objects around function and log statements would log to the right file based on which process is calling the function?</p>

<p>The <a href="https://docs.python.org/3/library/threading.html" rel="nofollow">threading</a> module has a <code>get_ident</code> member which could be used to index some logger dictionary, something like;</p>

<pre><code>from threading import get_ident

loggers[get_ident()].logError('blah blah blah')
</code></pre>

<p>However, once you have all of this test logging throughout your libraries, how will that impact your production performance?</p>

python logging python-multiprocessing