What is the benefit of a dataframe view or copy
<p>I've seen many questions about the infamous <code>SettingWithCopy</code> warning.  I've even ventured to answer a few of them.  Recently, I was putting together an answer that involved this topic and I wanted to present the benefit of a dataframe view.  I failed to produce a tangible demonstration of why it is a good idea to create a dataframe view or whatever the thing is that generates <code>SettingWithCopy</code></p>

<p>Consider <code>df</code></p>

<pre><code>df = pd.DataFrame([[1, 2], [3, 4]], list('ab'), list('AB'))
df

   A  B
x  1  2
y  3  4
</code></pre>

<p>and <code>dfv</code> which is a copy of <code>df</code></p>

<pre><code>dfv = df[['A']]
</code></pre>

<hr>

<pre><code>print(dfv.is_copy)

&lt;weakref at 0000000010916E08; to 'DataFrame' at 000000000EBF95C0&gt;
</code></pre>

<hr>

<pre><code>print(bool(dfv.is_copy))

True
</code></pre>

<hr>

<p>I can generate the <code>SettingWithCopy</code></p>

<pre><code>dfv.iloc[0, 0] = 0
</code></pre>

<p><a href="http://i.stack.imgur.com/eazEe.png" rel="nofollow"><img src="http://i.stack.imgur.com/eazEe.png" alt="enter image description here"></a></p>

<hr>

<p>However, <code>dfv</code> has changed</p>

<pre><code>print(dfv)

   A
a  0
b  3
</code></pre>

<p><code>df</code> has not</p>

<pre><code>print(df)

   A  B
x  1  2
y  3  4
</code></pre>

<hr>

<p>and <code>dfv</code> is still a copy</p>

<pre><code>print(bool(dfv.is_copy))

True
</code></pre>

<hr>

<p>If I change <code>df</code></p>

<pre><code>df.iloc[0, 0] = 7
print(df)

   A  B
x  7  2
y  3  4
</code></pre>

<hr>

<p>But <code>dfv</code> has not changed.  However, I can reference <code>df</code> from <code>dfv</code></p>

<pre><code>print(dfv.is_copy())

   A  B
x  7  2
y  3  4
</code></pre>

<hr>

<h3>Question</h3>

<p>If <code>dfv</code> maintains it's own data (meaning, it doesn't actually save memory) and it assigns values via assignment operations despite warnings, then why do we bother saving references in the first place and generate <code>SettingWithCopyWarning</code> at all?</p>

<p>What is the tangible benefit?</p>

<p>There is a lot of existing discussion on this, see <a href="https://github.com/pydata/pandas/issues/10954" rel="nofollow">here</a> for instance, including the attempted PRs.  It's also worth noting that true copy-on-write for views is being considered as part of the "pandas 2.0" refactor, see <a href="https://github.com/pydata/pandas-design/issues/10" rel="nofollow">here</a>.</p>

<p>The reason the reference is being maintained in your example is specifically because it's <em>not</em> a view, so that if someone tries this, they'll get a warning.</p>

<pre><code>df[['A']].iloc[0, 0] = 1
</code></pre>

<p>Edit:</p>

<p>In terms of "why use views at all," it's for performance / memory reasons.  Consider, basic indexing (selecting a column), because this operation takes a view, it is almost instantaneous.</p>

<pre><code>df = pd.DataFrame(np.random.randn(1000000, 2), columns=['a','b'])

%timeit df['a']
100000 loops, best of 3: 2.13 Âµs per loop
</code></pre>

<p>Whereas taking a copy has a non-trivial cost.</p>

<pre><code>%timeit df['a'].copy()
100 loops, best of 3: 4.28 ms per loop
</code></pre>

<p>This performance cost would show up in many operations, for instance adding two <code>Series</code> together.</p>

<pre><code>%timeit df['a'] + df['b']
100 loops, best of 3: 4.31 ms per loop

%timeit df['a'].copy() + df['b'].copy()
100 loops, best of 3: 13.3 ms per loop
</code></pre>

python pandas