Python dynamic function building and closure rule
<p>While interactively executing the following code in <a href="http://pythontutor.com/visualize.html" rel="nofollow">http://pythontutor.com/visualize.html</a>, the frame of each call to <code>build_match_and_apply_functions</code> appears gray in the graphical view:</p>

<p><a href="https://i.stack.imgur.com/sxo9k.png" rel="nofollow"><img src="https://i.stack.imgur.com/sxo9k.png" alt="screenshot from pythontutor"></a></p>

<p>This program is for getting a plural of a word ,  which is quoted from the chapter in DIVE in Python 3</p>

<p>CODE: </p>

<pre><code>import re

def build_match_and_apply_functions(pattern, search, replace):
    def matches_rule(word):                                   
        return re.search(pattern, word)
    def apply_rule(word):                                     
        return re.sub(search, replace, word)
    return (matches_rule, apply_rule)        

patterns =  \
    (
    ('[sxz]$',           '$',  'es'),
    ('[^aeioudgkprt]h$', '$',  'es'),
    ('(qu|[^aeiou])y$',  'y$', 'ies'),
    ('$',                '$',  's')
  )
rules = [build_match_and_apply_functions(pattern, search, replace)  
         for (pattern, search, replace) in patterns]


def plural(noun):
    for matches_rule, apply_rule in rules: 
        if matches_rule(noun):
            return apply_rule(noun)


plural('vacancy')
</code></pre>

<p>question:</p>

<p>1) what does gray frame mean? it is a closure that still takes memory?</p>

<p>2) can I go in memory block. so i can figure out in the objects area , are all matches_rule functions the same or not? if they are the same , f2/f3/f4/f5 should be there to offer pattern/search/replace values.</p>

<p>if not, if all matches_rules functions have already be changed to different functions , f2 3 4 5 could be end of live and dispear. they are useless.</p>

<p>i dont know , that is how dynamic language works and being built.</p>

<hr>

<p>pythontutor.com ANALYZE DIGRAM really surprised me, tutor did the good job  </p>

<p>if u dont expirence it, please copy the link below and paste my code in it.  i bet you are having fun with it.</p>

<p><a href="http://pythontutor.com/visualize.html" rel="nofollow">http://pythontutor.com/visualize.html</a></p>

<p>The gray frames are the execution frames, which contain the closure. They can theoretically be read from memory.</p>

<p>For example, I managed to read the data from the frames with the following code (CPython v3.5):</p>

<p>Some utilities first:</p>

<pre><code>import gc, inspect, ctypes, random

def all_live_ids():
    """Get all valid object IDs"""
    for obj in gc.get_objects():
        yield id(obj)

def getobj(obj_id):
    """Get the object from object ID"""
    if obj_id in all_live_ids():
        return ctypes.cast(obj_id, ctypes.py_object).value
    else:
        return None

frame_ids = []

def print_frames():
    for fid in frame_ids:
        frame = getobj(fid)
        if frame:
            print('frame', hex(fid), 'is alive and has pattern:',
                   frame.f_locals['pattern'])
        else:
            print('frame', hex(fid), 'is dead')
</code></pre>

<p>Then a simplified version of the code from the question:</p>

<pre><code>def build_match_and_apply_functions(pattern, search, replace):
    frame = inspect.currentframe()
    frame_ids.append(id(frame))
    print('executing frame', frame, 'pattern:', pattern)
    def matches_rule(word):                                   
        return re.search(pattern, word)
    return matches_rule

rule1 = build_match_and_apply_functions('[sxz]$', '$', 'es')
rule2 = build_match_and_apply_functions('[^aeioudgkprt]h$', '$', 'es')

print('\nbefore gc.collect():')
print_frames()

gc.collect()

print('\nafter gc.collect():')
print_frames()
</code></pre>

<p>In my case, it printed:</p>

<pre><code>executing frame &lt;frame object at 0x0071EDB0&gt; pattern: [sxz]$
executing frame &lt;frame object at 0x00C0E4B0&gt; pattern: [^aeioudgkprt]h$

before gc.collect():
frame 0x71edb0 is alive and has pattern: [sxz]$
frame 0xc0e4b0 is alive and has pattern: [^aeioudgkprt]h$

after gc.collect():
frame 0x71edb0 is dead
frame 0xc0e4b0 is dead
</code></pre>

<h2>What did I do?</h2>

<p>I remembered IDs of all frames in <code>frame_ids</code>. In CPython, <code>id</code> returns the memory address of the object.</p>

<p>Then I used <a href="http://stackoverflow.com/a/15702647/389289">this solution</a> to convert IDs back to objects.</p>

<p>I had to do that in order to be able to get the frame without ever storing a reference to the frame (Usualy <code>weakref</code> does that, but not with <code>frame</code> objects`)</p>

<h2>There is a catch</h2>

<p>Notice that the frames disappeared after <code>gc.collect</code>. The frames are not needed once the function finishes, but there were some circular references which prevented their immediate deletion. <code>gc.collect</code> is smart enough to detect those and delete the unneeded frames. With different code, that may happen even without calling <code>gc.collect</code> and I believe it should happen in <a href="http://pythontutor.com/" rel="nofollow">http://pythontutor.com/</a> as well, but their internals are in some sort of conflict with the garbace collector.</p>

python functional-programming closures