Python: find a series of Chinese characters within a string and apply a function
<p>I've got a series of text that is mostly English, but contains some phrases with Chinese characters. Here's two examples:</p>

<pre><code>s1 = "You say: ä½ å¥½. I say: åè¦"
s2 = "ç­æ¡, my friend, å¨é¢¨å¨å¹"
</code></pre>

<p>I'm trying to find each block of Chinese, apply a function which will translate the text (I already have a way to do the translation), then replace the translated text in the string. So the output would be something like this:</p>

<pre><code>o1 = "You say: hello. I say: goodbye"
o2 = "The answer, my friend, is blowing in the wind"
</code></pre>

<p>I can find the Chinese characters easily by doing this:</p>

<pre><code>utf_line = s1.decode('utf-8') 
re.findall(ur'[\u4e00-\u9fff]+',utf_line)
</code></pre>

<p>...But I end up with a list of all the Chinese characters and no way of determining where each phrase begins and ends.</p>

<p>You can't get the indexes using <a href="https://docs.python.org/2/library/re.html#re.findall" rel="nofollow">re.findall()</a>. You could use <a href="https://docs.python.org/2/library/re.html#re.finditer" rel="nofollow">re.finditer()</a> instead, and refer to <a href="https://docs.python.org/2/library/re.html#re.MatchObject.group" rel="nofollow">m.group()</a>, <a href="https://docs.python.org/2/library/re.html#re.MatchObject.start" rel="nofollow">m.start()</a> and <a href="https://docs.python.org/2/library/re.html#re.MatchObject.end" rel="nofollow">m.end()</a>.</p>

<p>However, for your particular case, it seems more practical to call a function using <a href="https://docs.python.org/2/library/re.html#re.sub" rel="nofollow">re.sub()</a>.</p>

<blockquote>
  <p>If <em>repl</em> is a function, it is called for every non-overlapping occurrence of <em>pattern</em>. The function takes a single match object argument, and returns the replacement string</p>
</blockquote>

<p><strong>Code:</strong></p>

<pre class="lang-python prettyprint-override"><code>import re

s = "You say: ä½ å¥½. I say: åè¦. ç­æ¡, my friend, å¨é¢¨å¨å¹"
utf_line = s.decode('utf-8')

dict = {"ä½ å¥½" : "hello",
        "åè¦" : "goodbye",
        "ç­æ¡" : "The answer",
        "å¨é¢¨å¨å¹" : "is blowing in the wind",
       }

def translate(m):
    block = m.group().encode('utf-8')
    # Do your translation here

    # this is just an example
    if block in dict:
        return dict[ block ]
    else:
        return "{unknown}"


utf_translated = re.sub(ur'[\u4e00-\u9fff]+', translate, utf_line, re.UNICODE)

print utf_translated.encode('utf-8')
</code></pre>

<p><strong>Output:</strong></p>

<pre class="lang-none prettyprint-override"><code>You say: hello. I say: goodbye. The answer, my friend, is blowing in the wind
</code></pre>

<ul>
<li><a href="http://ideone.com/Dq1muC" rel="nofollow">Ideone demo</a></li>
</ul>

python regex