Python: convert some encoding to russian alphabet
<p>I have list with unicode</p>

<pre><code>lst = [u'\xd0\xbe', u'/', u'\xd0\xb8', u'\xd1\x81', u'\xd0\xb2', u'\xd0\xba', u'\xd1\x8f', u'\xd1\x83', u'\xd0\xbd\xd0\xb0', u'____', u'|', u'\xd0\xbf\xd0\xbe', u'11', u'search', u'\xd0\xbe\xd1\x82', u'**modis**', u'15', u'\xd0\xa1', u'**avito**', u'\xd0\xbd\xd0\xb5', u'[\xd0\xa1\xd0\xbe\xd1\x85\xd1\x80\xd0\xb0\xd0\xbd\xd1\x91\xd0\xbd\xd0\xbd\xd0\xb0\xd1\x8f', u'\xd0\x92', u'\xd0\xb5\xd1\x89\xd1\x91', u'kid', u'google', u'\xd0\xbb\xd0\xb8', u'13', u'**\xd0\xb0\xd0\xb2\xd0\xb8\xd1\x82\xd0\xbe**', u'[\xd0\x9f\xd0\xbe\xd0\xba\xd0\xb0\xd0\xb7\xd0\xb0\xd1\x82\xd1\x8c', u'\xd0\x9f\xd0\xbe\xd0\xb6\xd0\xb0\xd0\xbb\xd0\xbe\xd0\xb2\xd0\xb0\xd1\x82\xd1\x8c\xd1\x81\xd1\x8f', u'\xd0\x9e', u'&amp;parent-', u'\xd0\xaf\xd0\xbd\xd0\xb4\xd0\xb5\xd0\xba\xd1\x81', u'###', u'**avito**.', u'**kiddieland**', u'\xd0\xbc\xd0\xb0\xd0\xb3\xd0\xb0\xd0\xb7\xd0\xb8\xd0\xbd', u'45', u'click2.yandex.ru/redir', u'72']
</code></pre>

<p>I try to convert some like <code>u'\xd0\xbe'</code> to russian.
I tried to use 2 and 3 python, but I can't do that.
I use IDE Pycharm and in settings I have default utf-8. 
When I print that with</p>

<pre><code>for elem in lst:
    print (elem)
</code></pre>

<p>it returns me <code>ÃÂ¾</code> for the first elem. When I try <code>print (elem.encode('cp1252'))</code>  it returns <code>b'\xd0\xbe'</code>
When I use <code>chardet.detect</code> it returns me, that it's <code>utf-8</code>.
Can anybody explain to me, how can I convert it to russian alphabet and why ways, that I use doesn't fit to get it.</p>

<p>Your data is a <a href="https://en.wikipedia.org/wiki/Mojibake" rel="nofollow">Mojibake</a>, incorrectly decoded from UTF-8 bytes as Latin-1 or CP1252.</p>

<p>You can repair this by manually reverting that process:</p>

<pre><code>repaired = [elem.encode('latin1').decode('utf8') for elem in lst]
</code></pre>

<p>but be careful; if the data actually was decoded as cp1252 the above would fail if there were any bytes in the range 0x80-0x9f in the source data.</p>

<p>You can use the <a href="https://ftfy.readthedocs.io/en/latest/" rel="nofollow"><code>ftfy</code> library</a> instead; it contains specialist codecs that can handle forced decodings too (where bytes are selectively decoded as Latin-1 where a CP1252 mapping is missing):</p>

<pre><code>import ftfy

repaired = [ftfy.fix_text(elem) for elem in lst]
</code></pre>

<p><code>ftfy.fix_text()</code> does a good job at auto-detecting what codec the data was decoded with.</p>

<p>Either method works for the sample data you gave; using <code>ftfy</code> or manually decoding doesn't make a difference for <em>that specific example</em>:</p>

<pre><code>&gt;&gt;&gt; import ftfy
&gt;&gt;&gt; repaired = [ftfy.fix_text(elem) for elem in lst]
&gt;&gt;&gt; repaired
[u'\u043e', u'/', u'\u0438', u'\u0441', u'\u0432', u'\u043a', u'\u044f', u'\u0443', u'\u043d\u0430', u'____', u'|', u'\u043f\u043e', u'11', u'search', u'\u043e\u0442', u'**modis**', u'15', u'\u0421', u'**avito**', u'\u043d\u0435', u'[\u0421\u043e\u0445\u0440\u0430\u043d\u0451\u043d\u043d\u0430\u044f', u'\u0412', u'\u0435\u0449\u0451', u'kid', u'google', u'\u043b\u0438', u'13', u'**\u0430\u0432\u0438\u0442\u043e**', u'[\u041f\u043e\u043a\u0430\u0437\u0430\u0442\u044c', u'\u041f\u043e\u0436\u0430\u043b\u043e\u0432\u0430\u0442\u044c\u0441\u044f', u'\u041e', u'&amp;parent-', u'\u042f\u043d\u0434\u0435\u043a\u0441', u'###', u'**avito**.', u'**kiddieland**', u'\u043c\u0430\u0433\u0430\u0437\u0438\u043d', u'45', u'click2.yandex.ru/redir', u'72']
&gt;&gt;&gt; print repaired[20]
[Ð¡Ð¾ÑÑÐ°Ð½ÑÐ½Ð½Ð°Ñ
</code></pre>

<p>Of course, the <em>better</em> solution is to avoid creating a Mojibake in the first place. If you can correct the source of the error, so much the better.</p>

<p>For example, if you loaded this data using the <code>requests</code> library and assumed that it was safe to use the <code>response.text</code> attribute, then please do read the <a href="http://docs.python-requests.org/en/master/user/advanced/#encodings" rel="nofollow"><em>Encodings</em> section</a> of the <em>Advanced Usage</em> chapter in the library documentation:</p>

<blockquote>
  <p>The only time Requests will not do this is if no explicit charset is present in the HTTP headers and the Content-Type header contains text. In this situation, RFC 2616 specifies that the default charset must be ISO-8859-1. Requests follows the specification in this case. If you require a different encoding, you can manually set the <code>Response.encoding</code> property, or use the raw <code>Response.content</code>.</p>
</blockquote>

<p>so if there is character set defined on the response, <code>response.text</code> will give you Latin-1 decoded text instead. It is better to avoid using <code>response.text</code> and use <code>response.content</code> instead in that case and either manually decode or use a format-appropriate parser to determine the codec used (such as BeatifulSoup for HTML).</p>

python unicode encoding utf-8