How to avoid pickling errors when sharing objects between threads?
<p>I have a program in which I need to store a global variable into a file. I am doing this using the <code>pickle</code> module. </p>

<p>I have another <code>thread</code>(Daemon=<code>False</code>, from <code>threading</code> module) which sometimes changes the value of the global variable. The value is also modified in global scope(the main program). </p>

<p>I am dumping the value of the variable into a <code>.pkl</code> file every 5 seconds (using another <code>thread</code> from <code>threading</code> module).</p>

<p>But I found the following error when <code>dump</code> method was executed:</p>

<pre><code>TypeError: can't pickle _thread.lock objects
</code></pre>

<p>Why is this happening? And what can I do to fix it?</p>

<p>Note: I have found some similar answers with <code>multiprocessing</code> module. But I need an answer for <code>threading</code> module.</p>

<p>Code:</p>

<pre><code>def save_state():
    while True:
        global variable

        lastSession = open('lastSession.pkl', 'wb')

        # error occurs on this line
        pickle.dump(variable, lastSession)

        lastSession.close()          
        time.sleep(5)

state_thread = threading.Thread(target = save_state)
state_thread.setDaemon(False)
state_thread.start()

# variable is changed outside this function and also in another thread(not state_thread).
</code></pre>

<p>As others have mentioned, you cannot pickle "volatile" entities (threads, connections, synchonization primitives etc.) because they don't make sense as persistent data.</p>

<p>Looks like what you're trying to do is be saving session variables for later continuation of said session. For that task, there's nothing you can do to save objects that, well, cannot be saved due to their nature.</p>

<p>The simplest solution is just to ignore them. Replacing them with some "stubs" that would yield an error anytime you do anything with them makes little sense, as a missing variable yields an error anyway (it won't if it was masking a global variable but that is a questionable practice in itself).</p>

<p>Alternatively, you can set up these objects anew when restoring a session. But such a logic is by necessity task-specific.</p>

<p>Finally, e.g. IPython <a href="http://stackoverflow.com/questions/12504951/save-session-in-ipython-like-in-matlab">already has session saving/restoration logic</a>, so you probably don't need to reinvent the wheel at all.</p>

python multithreading pickle locks