Logging for multiple objects: IOError
<p>I have a class called <code>Job</code> which has a logger</p>

<pre><code>class MyFileHandler(logging.FileHandler):
    def __init__(self, filename):
        self.filename = filename
        super(MyFileHandler, self).__init__(filename)

    def emit(self, record):
        log_text = self.format(record)
        try:
            fh = open(self.filename, "a")
            fh.write("%s\n" % log_text)
            fh.close()
            return True

        except:
            return False

log_formatter = logging.Formatter('br: %(message)s')
class Job(object):
    def __init__(self, name):
        self.name = name
        self.logger = logging.getLogger(self.name)
        log_hdlr = MyFileHandler('/tmp/%s' % name)
        log_hdlr.setFormatter(log_formatter)
        self.logger.addHandler(log_hdlr)
        self.logger.setLevel(logging.INFO)

jobs = []
for i in range(100):
    j = Job(str(i))
    job.append(j)
</code></pre>

<p>and jobs go off do something and logs via <code>job.logger.info()</code></p>

<p>but when i have multiple jobs i.e. thousands, it's throwing error</p>

<pre class="lang-none prettyprint-override"><code>IOError: [Errno 24]
    Too many open files:
      '/x/prototype_3885946_1608131132/cm/cm_conv/logs/20160827-195925.log'
</code></pre>

<p>I thought every time I logged something, it would open then close the file as I have overwritten <code>emit()</code></p>

<p>Is there a pattern/ways to have thousands of loggers?</p>

<p>Guess your operating system is running out of file handles.</p>

python python-2.7 logging