Python - "Undo" text-wrap
<p>I need to take a text and remove the \n character, which I believe I've done. The next task is to remove the hyphen from words where it should not appear but to leave the hyphen in compound words where it should appear. For example, 'encyclo-\npedia to 'encyclopedia' and 'long-\nterm' to 'long-term'. The suggestion is to compare it with an original text.</p>

<pre><code>with open('C:\Users\Paul\Desktop\Comp_Ling_Research_1\BROWN_A1_hypenated.txt', 'rU') as myfile:
data=myfile.read().replace('\n', '')
</code></pre>

<p>I have a general idea of what to do but NLP is quite new to me.</p>

<pre><code>import re


with open('C:\Users\Paul\BROWN_A1.txt', 'rU') as truefile:
    true_corpus = truefile.read()

true_tokens = true_corpus.split(' ')

with open('C:\Users\Paul\Desktop\Comp_Ling_Research_1\BROWN_A1_hypenated.txt', 'rU') as myfile:

my_corpus = myfile.read()

my_tokens = my_corpus.split(' ')
</code></pre>

python nlp nltk