UTF-8 and colour printing woesâ¦
<p>I have a console program that outputs in wonderful colour. For errors, the following code is used with some trivial examples at the bottom.</p>

<pre><code># coding: utf-8
from __future__ import absolute_import
from __future__ import division
from __future__ import print_function
from __future__ import unicode_literals
from sys import stderr
from colored import fg
from colored import attr
from locale import getpreferredencoding

def format_error(x):
    return '{0}{1}{2}'.format(fg(88), x, attr('reset'))

def print_error(x):
    msg = format_error('â  {0}\n'.format(x))
    stderr.write(msg.encode(getpreferredencoding()))

print_error(str('ook'))
print_error(unicode(b'cafÃ©', 'UTF-8'))
</code></pre>

<p>I have no control over that <code>x</code> is.  It could be anything. Also, some of this script is called from a GUI that captures <code>stdout</code>/<code>stderr</code> via <a href="http://www.pygtk.org/docs/pygobject/glib-functions.html#function-glib--spawn-async" rel="nofollow" title="glib-spawn-async documentation">glib-spawn-async</a>. As such, from time to time, I get <code>UnicodeDecodeError</code> errors. I have read the <a href="https://docs.python.org/2/howto/unicode.html" rel="nofollow" title="Unicode HOWTO">Unicode HOWTo</a> but clearly I am missing something.</p>

<p><em>How can I harden my code such that <code>UnicodeDecodeError</code> are never raised?</em></p>

<p>For example, within a <code>gtk.textview</code>, I get the following whereas on the console, all is fine. Trace has been cut to remove irrelevant data.</p>

<pre><code> File "/home/usr/nifty_logger.py", line 96, in print_success
    sys.stdout.write(msg.encode(getpreferredencoding()))
  File "/home/usr/.virtualenvs/rprs_bootstrap/lib64/python2.7/codecs.py", line 351, in write
    data, consumed = self.encode(object, self.errors)
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 0: ordinal not in range(128)
</code></pre>

<p>The <a href="http://docs.python.org/library/stdtypes.html#str.encode" rel="nofollow">encode()</a> takes an optional argument defining the error handling:</p>

<pre><code>str.encode([encoding[, errors]])
</code></pre>

<p>From the docs:</p>

<blockquote>
  <p>Return an encoded version of the string. Default encoding is the current default string encoding. errors may be given to set a different error handling scheme. The default for errors is 'strict', meaning that encoding errors raise a UnicodeError. Other possible values are 'ignore', 'replace', 'xmlcharrefreplace', 'backslashreplace' and any other name registered via codecs.register_error(), see section Codec Base Classes. For a list of possible encodings, see section Standard Encodings.</p>
</blockquote>

<p>In your case:</p>

<pre><code>msg.encode(getpreferredencoding(), 'backslashreplace')
</code></pre>

python python-2.7 utf-8