How to convert hex string into Japanese and write it to the file in Python 2.7?
<p>I am trying to convert a hex string into Japanese(codec: SHIFT-JIS) and write the Japanese output to a file using Python 2.7. However, all I have got is the original hex string in the file. Can someone tell me where I did wrong? Here is the code I use:</p>

<pre><code>fd = open(path,'w')
temp_str ='\x8d\xc5\x82\xe0\x8d\x82\x8b\x4d\x82\xc8\x89\xa4\x82\xc5\x82\xa0\x82\xc1\x82\xbd\x82\xbc\x81\x76\x80\x01\xff\xff'
fd.write(temp_str.encode('shift-jis'))
fd.close()
</code></pre>

<p>All I have got in the file is "\x8d\xc5\x82\xe0\x8d\x82\x8b\x4d\x82\xc8\x89\xa4\x82\xc5\x82\xa0\x82\xc1\x82\xbd\x82\xbc\x81\x76\x80\x01\xff\xff". </p>

<p>The string seems to be encoded in UTF-16BE:</p>

<pre><code>&gt;&gt;&gt; print temp_str.decode('utf_16_be')
è·è è¶è­èè¦¤èè èè½è¼è¶è
</code></pre>

<p>But it also seems malformed, ie, it was cut halfway. You should first convert the string to Unicode by decoding the bytes:</p>

<pre><code>uni_str = temp_str.decode('utf_16_be')
</code></pre>

<p>And then saving the Unicode string to a file with an encoding that you want:</p>

<pre><code>fd = open(path,'w')
fd.write(uni_str.encode('shift-jis'))
fd.close()
</code></pre>

<p>However, the codec 'shift-jis' doesn't seem to like your string:</p>

<pre><code>&gt;&gt;&gt; print uni_str.encode('shift_jis')
Traceback (most recent call last):
  File "&lt;stdin&gt;", line 1, in &lt;module&gt;
UnicodeEncodeError: 'shift_jis' codec can't encode character u'\u8dc5' in position 0: illegal multibyte sequence
</code></pre>

<p>This is not Japanese, but Chinese:</p>

<pre><code>&gt;&gt;&gt; print uni_str.encode('gb18030')
Úï¿½ï¿½ï¿½fï¿½Gï¿½ï¿½ï¿½Bï¿½iï¿½[ï¿½ï¿½Ñ¿ï¿½dï¿½aï¿½Ï1ï¿½9
</code></pre>

<p>'gb18030' is a Chinese codec, <a href="https://docs.python.org/2.4/lib/standard-encodings.html" rel="nofollow">according to Python docs</a>.</p>

<p>Yes, its jiggebirsh because I don't have a terminal with that codec, but it is the only codec from Python that encodes the string without errors besides UTF8/16/32.</p>

python python-2.7