How to generate bi/tri-grams using spacy/nltk
<p>The input text are always list of dish names where there are 1~3 adjectives  and a noun</p>

<p>Inputs</p>

<pre><code>thai iced tea
spicy fried chicken
sweet chili pork
thai chicken curry
</code></pre>

<p>outputs:</p>

<pre><code>thai tea, iced tea
spicy chicken, fried chicken
sweet pork, chili pork
thai chicken, chicken curry, thai curry
</code></pre>

<p>Basically, I am looking to parse the sentence tree and try to generate bi-grams by pairing an adjective with the noun.</p>

<p>And I would like to achieve this with spacy or nltk</p>

<p>You can achieve this in a few steps with NLTK:</p>

<ol>
<li><p>PoS tag the sequences</p></li>
<li><p>generate the desired n-grams (in your examples there are no trigrams, but skip-grams which can be generated through trigrams and then punching out the middle token)</p></li>
<li><p>discard all n-grams that don't match the pattern <em>JJ NN</em>.</p></li>
</ol>

<p>Example:</p>

<pre><code>def jjnn_pairs(phrase):
    '''
    Iterate over pairs of JJ-NN.
    '''
    tagged = nltk.pos_tag(nltk.word_tokenize(phrase))
    for ngram in ngramise(tagged):
        tokens, tags = zip(*ngram)
        if tags == ('JJ', 'NN'):
            yield tokens

def ngramise(sequence):
    '''
    Iterate over bigrams and 1,2-skip-grams.
    '''
    for bigram in nltk.ngrams(sequence, 2):
        yield bigram
    for trigram in nltk.ngrams(sequence, 3):
        yield trigram[0], trigram[2]
</code></pre>

<p>Extend the pattern <code>('JJ', 'NN')</code> and the desired n-grams to your needs.</p>

<p>I think there is no need for parsing.
The major problem of this approach, however, is that most PoS taggers will probably not tag everything exactly the way you want.
For example, the default PoS tagger of my NLTK installation tagged "chili" as <em>NN</em>, not <em>JJ</em>, and "fried" got <em>VBD</em>.
Parsing won't help you with that, though!</p>

python nlp nltk n-gram spacy