Can sentences be formed AFTER raw text is converted into nltk.Text?
<p>Usual way of converting file data into nltk.Text seems as follows:</p>

<pre><code>f=open('my-file.txt','rU')
raw=f.read()
tokens = nltk.word_tokenize(raw)
text = nltk.Text(tokens)
</code></pre>

<p>Now, 'text' (the nltk.Text object) is just a list of words. 
How can I get a list of sentences from it?
Basically wish to split 'text' into list of sentences. How?</p>

<p>you can use the 'sent_tokenize()' and 'line_tokenize()' methods on the raw text (depends if you want to split lines (\n), sentences (by '.', '?', etc) or both):</p>

<pre><code>f=open('my-file.txt','rU')
raw=f.read()
# tokenize lines first and then by sentence markers
sents = [nltk.tokenize.sent_tokenize(l) for l in nltk.tokenize.line_tokenize(raw)]
tokens = nltk.word_tokenize(raw)
text = nltk.Text(tokens) 
</code></pre>

<p>an alternative would be to use just one of the options:</p>

<pre><code>sents = nltk.tokenize.sent_tokenize(l) # tokenize only by sentence markers
</code></pre>

<p>or </p>

<pre><code>sents = nltk.tokenize.line_tokenize(l) # tokenize only by line markers
</code></pre>

<p>and to answer your original question - you can't really tokenize sentences after converting a raw text to the nltk.Text() structure. Of course you can do some workaround to make it, but it will be far from pretty (and far from useful, assuming you have the original raw text).</p>

python nltk