Regexp: some questions
<p><strong>Python 3</strong></p>

<pre><code>text = "(CNN)Meaalofa Te'o -- Buemi. Canberra,"

def discard_punctuation(text):
    regex = '\W*^\s^\d*-'
    return re.sub(regex, "", text)

def handle_text(text):
    text_without_punctuation = discard_punctuation(text)
    words_array = text_without_punctuation.split()
    pass // Breakpoint

handle_text(text)
</code></pre>

<p>From an arbitrary text <strong>I want to select words only.</strong> Investigating the problem, I have discovered that sometimes a hyphen is inside a word. Or a number may be as well (9-year-old, canyon-like).</p>

<p>My regex is regex = '\W*^\s^\d*-'. </p>

<p>Take all non-alphanumeric character; exclude all whitespace charactes, which are necessary for split method; exclude all numbers that are not followed by a hyphen.</p>

<p>I should also exclude hyphens that are not in words.</p>

<p>The result is:
: ['(CNN)Meaalofa', "Te'o", '--', 'Buemi.', 'Canberra,']</p>

<p>The documentation: <a href="https://docs.python.org/3/howto/regex.html" rel="nofollow">https://docs.python.org/3/howto/regex.html</a></p>

<pre><code>\W
Matches any non-alphanumeric character; this is equivalent to the class [^a-zA-Z0-9_].
</code></pre>

<p>I thought that points, commas, hyphen, brackets and apostrophe should match \W. </p>

<p>Questions:
<strong>1. I can't understand why: brackets, points and commas, and apostrophe are still present.</strong></p>

<ol start="2">
<li><p>I would say that I excluded the apostrophe. I need it, and it is present in the result, it is Ok. But I can't catch how it occurred there. <strong>Could you help me understand how apostrophe happened to occur in the result.</strong></p></li>
<li><p><strong>Well, "--" is definitely an error here. How to cope?</strong></p></li>
<li><p>Could you, please, suggest me a better regexp.</p></li>
</ol>

<p>With your rather vague definition of a "word", you could come up with:</p>

<pre><code>import re

rx = re.compile(r'\s*(\S+)\s*')

string = """(CNN)Meaalofa Te'o -- Buemi. Canberra,"""
words = rx.findall(string)
print(words)
# ['(CNN)Meaalofa', "Te'o", '--', 'Buemi.', 'Canberra,']
</code></pre>

<p>See <a href="http://ideone.com/WB9QmL" rel="nofollow"><strong>a demo on ideone.com</strong></a> and on <a href="https://regex101.com/r/zH2bQ6/1" rel="nofollow"><strong>regex101.com</strong></a>. You might redefine what a "word" is.</p>

python regex python-3.x