Python: find a series of Chinese characters within a string and apply a function
<p>I've got a series of text that is mostly English, but contains some phrases with Chinese characters. Here's two examples:</p>

<pre><code>s1 = "You say: ä½ å¥½. I say: åè¦"
s2 = "ç­æ¡, my friend, å¨é¢¨å¨å¹"
</code></pre>

<p>I'm trying to find each block of Chinese, apply a function which will translate the text (I already have a way to do the translation), then replace the translated text in the string. So the output would be something like this:</p>

<pre><code>o1 = "You say: hello. I say: goodbye"
o2 = "The answer, my friend, is blowing in the wind"
</code></pre>

<p>I can find the Chinese characters easily by doing this:</p>

<pre><code>utf_line = s1.decode('utf-8') 
re.findall(ur'[\u4e00-\u9fff]+',utf_line)
</code></pre>

<p>...But I end up with a list of all the Chinese characters and no way of determining where each phrase begins and ends.</p>

<p>Regular expression <code>Match</code> objects give you the start and end indexes of a match. So, instead of <code>findall</code>, do your own search and record the indexes as you go. Then, you can translate each extent and replace in the string based on the known indexes of the phrases.</p>

<pre><code>import re

_scan_chinese_re = re.compile(r'[\u4e00-\u9fff]+')

s1 = "You say: ä½ å¥½. I say: åè¦"
s2 = "ç­æ¡, my friend, å¨é¢¨å¨å¹"

def translator(chinese_text):
    """My no good translator"""
    return ' '.join('??' for _ in chinese_text)

def scanner(text):
    """Scan text string, translate chinese and return copy"""
    print('----&gt; text:', text)

    # list of extents where chinese text is found
    chinese_inserts = [] # [start, end]

    # keep scanning text to end
    index = 0
    while index &lt; len(text):
        m = _scan_chinese_re.search(text[index:])
        if not m:
            break
        # get extent from match object and add to list
        start = index + m.start()
        end = index + m.end()
        print('chinese at index', start, text[start:end])
        chinese_inserts.append([start, end])
        index += end

    # copy string and replace backwards so we don't mess up indexes
    copy = list(text)
    while chinese_inserts:
        start, end = chinese_inserts.pop()
        copy[start:end] = translator(text[start:end])
    text = ''.join(copy)
    print('final', text)
    return text

scanner(s1)
scanner(s2)
</code></pre>

<p>With my questionable translator, the result is</p>

<pre><code>----&gt; text: You say: ä½ å¥½. I say: åè¦
chinese at index 9 ä½ å¥½
chinese at index 20 åè¦
final You say: ?? ??. I say: ?? ??
----&gt; text: ç­æ¡, my friend, å¨é¢¨å¨å¹
chinese at index 0 ç­æ¡
chinese at index 15 å¨é¢¨å¨å¹
final ?? ??, my friend, ?? ?? ?? ??
</code></pre>

python regex