Hidden references to function arguments causing big memory usage?
<p><strong>Edit:</strong> Never mind, I was just being completely stupid.</p>

<p>I came across code with recursion on smaller and smaller substrings, here's its essence plus my testing stuff:</p>

<pre><code>def f(s):
    if len(s) == 2**20:
        input('check your memory usage again')
    else:
        f(s[1:])
input('check your memory usage, then press enter')
f('a' * (2**20 + 500))
</code></pre>

<p>Before the call, my Python process takes about 9 MB (as checked by Windows task manager). After the 500 levels with ~1MB strings, it's at about 513 MB. No surprise, as each call level is still holding on to its string in its <code>s</code> variable.</p>

<p>But I tried to fix it by <strong>replacing</strong> the reference to the string with a reference to the new string and it still goes up to 513 MB:</p>

<pre><code>def f(s):
    if len(s) == 2**20:
        input('check your memory usage again')
    else:
        s = s[1:]
        f(s)
input('check your memory usage, then press enter')
f('a' * (2**20 + 500))
</code></pre>

<p>Why doesn't that let go off the memory? The strings even only get smaller, so later strings would easily fit into the space of earlier strings. Are there hidden additional references to the strings somewhere or what is going on?</p>

<p>I had expected it to behave like this, which only goes up to 10 MB (a change of 1 MB, as expected because the new string is built while the old string still exists):</p>

<pre><code>input('check your memory usage, then press enter')
s = 'a' * (2**20 + 500)
while len(s) != 2**20:
    s = s[1:]
input('check your memory usage again')
</code></pre>

<p>(Never mind the poor time complexity, btw, I know that, don't bother.)</p>

<blockquote>
  <p>Are there hidden additional references to the strings somewhere or what is going on</p>
</blockquote>

<p>Well, each function has a reference to its string while its on the stack, so the <code>s = s[1:]</code> is still going to keep <code>s[1:]</code> in the next function call.  After 500 recursive calls the fact that you're copying 1 character less each time is insignificant to the roughly 2**20 characters that are being passed in each time.  </p>

python function memory