How to check two POS tags are in the same category in NLTK?
<p>Like the title says, how can I check two POS tags are in the same category?</p>

<p>For example,</p>

<pre><code>go -&gt; VB
goes -&gt; VBZ
</code></pre>

<p>These two words are both verbs. Or,</p>

<pre><code>bag -&gt; NN
bags -&gt; NNS
</code></pre>

<p>These two are both nouns.
So my question is that whether there exists any function in NLTK to check if two given tags are in the same category?</p>

<p>Let's take the simple case first: Your corpus is tagged with the Brown tagset (that's what it looks like), and you'd be happy with the simple tags defined in the nltk's <a href="http://www.nltk.org/book/ch05.html#tab-universal-tagset" rel="nofollow">"universal" tagset</a>: <code>., ADJ, ADP, ADV, CONJ, DET, NOUN, NUM, PRON, PRT, VERB, X</code>, where the dot stands for "punctuation". In this case, simply load the nltk's map and use it with your data:</p>

<pre><code>tagmap = nltk.tag.mapping.tagset_mapping("en-brown", "universal")
if tagmap[tag1] == tagmap[tag2]:
    print("The two words have the same part of speech")
</code></pre>

<p>If that's not your use case, you'll need to <em>manually</em> decide on a mapping from each individual tag to the simplified category you want to assign it to. If you are working with the Brown corpus tagset, you can see the tags and their meanings <a href="http://www.scs.leeds.ac.uk/amalgam/tagsets/brown.html" rel="nofollow">here,</a> or from within python like this:</p>

<pre><code>print(nltk.help.brown_tagset())
</code></pre>

<p>Study your tags and define a dictionary that maps each POS tag to your chosen category; people sometimes find it useful to just group Brown corpus tags by their first two letters, putting together "NN", "NN$", "NNS-HL", etc. You could create this particular mapping automatically like this:</p>

<pre><code>from nltk.corpus import brown
alltags = set(t for w, t in brown.tagged_words())
tagmap = dict(t[:2] for t in alltags)
</code></pre>

<p>Then you can customize this map according to your needs; e.g., to put all punctuation tags together in the category ".":</p>

<pre><code>for tag in tagmap:
    if not tag.isalpha():
        tagmap[tag] = "."
</code></pre>

<p>Once your <code>tagmap</code> is to your liking, use it like the one I imported from the <code>nltk</code>.</p>

<p>Finally, you might find it convenient to retag your entire corpus in one go, so that you can simply compare the assigned tags. If <code>corpus</code> is a list of tagged sentences in the format of the nltk's <code>&lt;corpus&gt;.tagged_sents()</code> command (so <em>not</em> a corpus reader object), you can retag everything like this:</p>

<pre><code>newcorpus = []
for sent in corpus:
    newcorpus.append( [ (w, tagmap[t]) for w, t in sent ] )
</code></pre>

python nltk pos-tagging