Checking if string contains unicode using standard Python
<p>I have some strings of roughly 100 characters and I need to detect if each string contains an unicode character. The final purpose is to check if some particular emojis are present, but initially I just want a filter that catches all emojis (as well as potentially other special characters). This method should be fast.</p>

<p>I've seen <a href="http://stackoverflow.com/questions/1832893/python-regex-matching-unicode-properties">Python regex matching Unicode properties</a>, but I cannot use any custom packages. I'm using Python 2.7. Thanks!</p>

<p>There is no point is testing 'if a string contains Unicode characters', because <strong>all</strong> characters in a string are Unicode characters. The Unicode standard encompasses all codepoints that Python supports, including the ASCII range (Unicode codepoints U+0000 through to U+007F).</p>

<p>If you want to test for Emoji code, test for specific ranges, as outlined by the <a href="http://unicode.org/Public/emoji/latest/emoji-data.txt" rel="nofollow">Unicode Emoji class specification</a>:</p>

<pre><code>re.compile(
    u'[\u231A-\u231B\u2328\u23CF\23E9-\u23F3...\U0001F9C0]',
    flags=re.UNICODE)
</code></pre>

<p>where you'll have to pick and choose what codepoints <em>you</em> consider to be Emoji. I personally would not include <a href="https://codepoints.net/U+0023" rel="nofollow">U+0023 NUMBER SIGN</a> in that category for example, but apparently the Unicode standard <em>does</em>.</p>

<p>Note: To be explicit, the above expression is not complete. There are 209 separate entries in the Emoji category and I didn't feel like writing them <em>all</em> out.</p>

<p>Another note: the above uses a <code>\Uhhhhhhhh</code> wide Unicode escape sequence; its use is only supported in a regex pattern in Python 3.3 and up, or in a <a href="http://stackoverflow.com/questions/1446347/how-to-find-out-if-python-is-compiled-with-ucs-2-or-ucs-4"><em>wide</em> (UCS-4) build</a> for earlier versions of Python. For a narrow Python build, you'll have to <a href="https://stackoverflow.com/questions/26568722/remove-unicode-emoji-using-re-in-python">match on surrogate pairs</a> for codepoints over U+FFFF.</p>

python regex unicode