UnicodeEncodeError in python3
<p>Some of my application's libraries are depending on being able to print UTF-8 characters to stdout and stderr. Therefore this must not fail:</p>

<pre><code>print('\u2122')
</code></pre>

<p>On my local machine it works, but on my remote server it raises <code>UnicodeEncodeError: 'ascii' codec can't encode character '\u2122' in position 0: ordinal not in range(128)</code></p>

<p>I tried <code>$ PYTHONIOENCODING=utf8</code> with no apparent effect.</p>

<pre><code>sys.stdout = codecs.getwriter("utf-8")(sys.stdout.detach())
</code></pre>

<p>works for a while, then stalls and finally fails with <code>ValueError: underlying buffer has been detached</code></p>

<p><code>sys.getdefaultencoding()</code> returns <code>'utf-8'</code>, and <code>sys.stdout.encoding</code> returns <code>'ANSI_X3.4-1968'</code></p>

<p>What can I do? I don't want to edit third-party libraries.</p>

<p>I'm guessing you're on a UNIX-like system, and your environment set <code>LANG</code> (or <code>LC_ALL</code> or whatever) to <code>C</code>.</p>

<p>Try editing your default shell's startup file to set <code>LANG</code> to something like <code>en_US.utf-8</code> (or whatever locale makes sense for you)? For example, in bash, edit <code>~/.bash_profile</code> (or <code>~/.profile</code> if you're using that instead for <code>sh</code> compatibility) and add:</p>

<pre><code>export LANG="en_US.utf-8"
</code></pre>

<p>For (t)csh, edit <code>~/.cshrc</code> (or <code>~/.tcshrc</code> if that's what you're using) to add:</p>

<pre><code>setenv LANG "en_US.utf-8"
</code></pre>

<p>Making the changes "live" doesn't work, because your shell is likely hosted in a terminal that has configured itself solely for ASCII display, based on the <code>LANG=C</code> in effect when it was launched (and many terminals do session coalescence, so even if you changed <code>LANG</code> and then launched a new terminal, it would coalesce with the shared terminal process with the out-of-date <code>LANG</code>). So after you change <code>~/.bash_profile</code>, log out and then log back in so your root shell will set <code>LANG</code> correctly for every other process (since they all ultimately <code>fork</code> from the root shell).</p>

python python-3.x unicode utf-8 locale