Sorting Bigram by number of occurrence NLTK
<p>I am currently running this code for search for bigram for entire of my text processing.</p>

<p>Variable alltext is really long text (over 1 million words)</p>

<p>I ran this code to extract bigram</p>

<pre><code>from nltk.tokenize import RegexpTokenizer
from nltk.corpus import stopwords
import re



tokenizer = RegexpTokenizer(r'([A-za-z]{2,})')
tokens = tokenizer.tokenize(alltext)
stopwords_list = stopwords.words('english')
tokens = [word for word in tokens if word not in stopwords.words('english')]
finder = BigramCollocationFinder.from_words(tokens, window_size = 2)
bigram_measures = nltk.collocations.BigramAssocMeasures()

for k,v in finder.ngram_fd.items():
    print k,v
</code></pre>

<p>The code above searches for the frequency occurrence for possible bigrams.</p>

<p>The code prints me lots of bigrams and its number of occurrence.</p>

<p>The output is similar to this.</p>

<pre><code>(('upper', 'front'), 1)
(('pad', 'Teething'), 1)
(('shoulder', 'strap'), 1)
(('outer', 'breathable'), 1)
(('memory', 'foam'), 1)
(('shields', 'inner'), 1)
(('The', 'garment'), 2)
......

type(finder.ngram_fd.items()) is a list.
</code></pre>

<p>How can i sort the frequency from highest to lowest number of occurrence. My desire result would be.</p>

<pre><code>(('The', 'garment'), 2)
(('upper', 'front'), 1)
(('pad', 'Teething'), 1)
(('shoulder', 'strap'), 1)
(('outer', 'breathable'), 1)
(('memory', 'foam'), 1)
(('shields', 'inner'), 1)
</code></pre>

<p>Thank you very much, I am quite new to nltk and text processing so my explanation would not be as clear.</p>

<p>It looks like <code>finder.ngram_fd</code> is a dictionary. In that case, in Python 3 the <code>items()</code> method does not return a list, so you'll have to cast it to one.</p>

<p>Once you have a list, you can simply use the <code>key=</code> parameter of the <a href="https://docs.python.org/3/library/stdtypes.html#list.sort" rel="nofollow"><code>sort()</code></a> method, which specifies what we're sorting against:</p>

<pre><code>ngram = list(finder.ngram_fd.items())
ngram.sort(key=lambda item: item[-1], reverse=True)
</code></pre>

<p>You have to add <code>reverse=True</code> because otherwise the results would be in ascending order. Note that this will sort the list <strong>in place</strong>. This is best when you want to avoid copying. If instead you wish to obtain a new list, just use the <code>sorted()</code> built-in function with the same arguments.</p>

<p>Alternatively, you can replace the lambda with <a href="https://docs.python.org/3/library/operator.html#operator.itemgetter" rel="nofollow"><code>operator.itemgetter</code></a>  module, which does the same thing:</p>

<pre><code>ngram.sort(key=operator.itemgetter(-1), reverse=True)
</code></pre>

python nltk