Siri-like app: calculating similarities between a query and a predefined set of control phrases
<p>I am trying to make a Apple Siri-like application in python in which you give it vocal commands or questions through a microphone, it determines the text version of the inputted audio, and then determines the appropriate action to take based on the meaning of the command/question. I am going to be using the Speech Recognition library to accept microphone input and convert from speech to text (via the IBM Watson Speech to Text API).</p>

<p>The main problem I have with it right now is that when I define an action for the app to execute when the appropriate command is given/question is asked, I don't know how to determine if the said command/question is denoting that action. Let me clarify what I mean by that with an example:</p>

<p>Say we have a action called <code>hello</code>. There are multiple ways for somebody to say "hello" to another person (or in this case, my application), such as:</p>

<ul>
<li>"Hello"</li>
<li>"Hi"</li>
<li>"Howdy"</li>
<li>...Etcetera...</li>
</ul>

<p>Of course, I want all of these ways of saying "hello" to be classified under the action of <code>hello</code>. That is, when someone says "hello", "hi", or "howdy", the response for the action <code>hello</code> should be executed (most likely just the app saying "hello" back in this case).</p>

<p>My first thought on how to solve this was to supply the app with all of or the most common ways to say a certain command/question. So, if I follow the previous example, I would tell the computer that "hello", "hi", and "howdy" all meant the same thing: the <code>hello</code> action. However, this method has a couple flaws. First off, it simply wouldn't understand ways of saying "hello" that weren't hardcoded in, such as "hey". Second off, once the responses for new commands/questions start getting coded in, it would become very tedious entering all the ways to say a certain phrase.</p>

<p>So then, because of the aforementioned problems, I started looking into ways to calculate the similarities between a group of sentences, and a single query. I eventually came across the Gensim library for python. I looked into it and found some very promising information on complex processes such as latent semantic indexing/analysis (LSI/LSA) and Tf-idf. However, it seemed to me like these things were mainly for comparing documents with large word counts as they rely on the frequency of certain terms. Assuming this is true, these processes wouldn't really provide me with accurate results as the commands/questions given to my app will probably be about eight words on average. I could be completely wrong, after all I know very little about these processes.</p>

<p>I also discovered WordNet, and how to work with it in python using the Natural Language Toolkit (NLTK). It looks like it could be useful, but I'm not sure how.</p>

<p>So, finally, I guess my real question here is what would be the best solution to the problem I've mentioned? Should I use one of the methods I've mentioned? Or is there a better way to do what I want that I don't know about?</p>

<p>Any help at all would be greatly appreciated. Thanks in advance.</p>

<p>P.S. Sorry for the wordy explanation; I wanted to be sure I was clear :P</p>

<p>This is a hard problem. It is also the subject of <a href="http://alt.qcri.org/semeval2017/task11" rel="nofollow">Task 11</a> of this year's set of Semantic evaluation challenges (<a href="http://alt.qcri.org/semeval2017/" rel="nofollow">Semeval 2017</a>). So take a look at the <a href="http://alt.qcri.org/semeval2017/task11/index.php?id=task-description" rel="nofollow">task description</a>, which will give you a road map for how this problem can be solved. The task also comes with a suite of training data, which is essential for approaching a problem like this. The challenge is still ongoing, but eventually you'll be able to learn from the solutions as well. </p>

<p>So the short answer to "how do I determine if some command/question is denoting a certain action" is: Use the training data from Semeval2017 (or your own of course), and write a classifier. The <a href="http://nltk.org/book" rel="nofollow">nltk book</a> can help you get up to speed with writing classifiers.</p>

python nlp nltk wordnet gensim