Confused by python's unicode regex errors
<p>Can someone explain why the middle code excerpt in python 2.7x throws an error?</p>

<pre><code>import re
walden = "WaldenstrÃ¶m"
walden
print(walden)

s1 = "Ã¶"
s2 = "Wal"
s3 = "OOOOO"

out = re.sub(s1, s3, walden)
print(out)

out = re.sub("W", "w", walden)
print(out)

# I need this one to work
out = re.sub('W', u'w', walden)
# ERROR

out = re.sub(u'W', 'w', walden)
print(out)

out = re.sub(s2, s1, walden)
print(out)
</code></pre>

<p>I'm very confused and have tried reading the manual</p>

<p><code>walden</code> is a <code>str</code>:</p>

<pre><code>walden = "WaldenstrÃ¶m"
</code></pre>

<p>This code replaces a character with a <code>unicode</code> string:</p>

<pre><code>re.sub('W', u'w', walden)
</code></pre>

<p>The result of that should be <code>u'w' + "aldenstrÃ¶m"</code>. This is the part that fails.</p>

<p>In order to concatenate <code>str</code> and <code>unicode</code>, both have to be first converted to <code>unicode</code>. The result is <code>unicode</code> as well.</p>

<p>The problem is, the interpreter does not know how to convert <code>'Ã¶'</code> to unicode, because it does not know which encoding to use. The result is ambiguous.</p>

<p>The solution is to convert yourself before doing the replacement:</p>

<pre><code>re.sub('W', u'w', unicode(walden, encoding))
</code></pre>

<p>The <code>encoding</code> should be the one you use to create that file, e.g.</p>

<pre><code>re.sub('W', u'w', unicode(walden, 'utf-8'))
</code></pre>

python unicode