Incorporating Lmtool into PocketSphinx?
<p>I am trying to create a simple way to add new keywords into PocketSphinx. The idea is to have a temporary text file that can be used to (via a script) add a word (or phrase) automatically added to the <code>corpus.txt</code>, <code>dictionary.dic</code> and <code>language_model.lm</code>.</p>

<p>Currently the best way to do this seems to be to use lmtool and then replace the aforementioned files with the updated versions. However this presents three problems:</p>

<ol>
<li>Lmtool is slow for large libraries, so the process will become exponentially slower as more words are added.</li>
<li>Lmtool requires a semi-reliable internet connection to work and I'd like to be able to add commands while offline.</li>
<li>This is not the most efficient way to add commands, and won't work with the setup I'm putting together.</li>
</ol>

<p>What I'd like to be able to do is to (if possible) use/create an offline version of lmtool that takes inputs from a temporary text file (<code>input.txt</code>) processes them and prints the contents into three temporary text files (<code>dic.txt</code>, <code>lm.txt</code>, <code>corp.txt</code>). </p>

<p>The last step would be to run a script that will: </p>

<ol>
<li>Take the output in <code>corp.txt</code> and add it to the end of <code>corpus.txt</code>.</li>
<li>Look through <code>dictionary.dic</code> and add any new words in <code>dic.txt</code>.</li>
<li>Somehow modify <code>language_model.lm</code> to include the new terms in <code>lm.txt</code>.</li>
<li>Erase the contents of the three output files.</li>
</ol>

<p>My question is if it is possible to get an offline version of lmtool that is capable of outputting results into specific text files? I know it is possible to automate lmtool (according to their site), but I would like to be able to run the process offline if possible.</p>

<p>Also, has anyone attempted something like this before that I can use as a guide?</p>

<p>I am running pocketsphinx on a raspberry pi and I am aware that it will likely not be able to run lmtool on it's own. My plan is to have lmtool run on a local server and sync files with the pi via wifi/ethernet. </p>

<p>Any help would be appreciated.</p>

<p>You have few choice, If you want to generate dict and language model locally on Raspberry Pi (Model 2B at least) </p>

<p>For Language Model generation, you can use either</p>

<ol>
<li><p><a href="https://github.com/skerit/cmusphinx/tree/master/cmuclmtk" rel="nofollow">CMUCLMTK</a> or</p></li>
<li><p><a href="http://www.speech.sri.com/projects/srilm/" rel="nofollow">SRILM</a> (SRI Language Modeling Toolkit)</p></li>
</ol>

<p>To compile SRILM on Raspbian you need to tweak some files.
Take a look here <a href="https://github.com/G10DRAS/SRILM-on-RaspberryPi" rel="nofollow">https://github.com/G10DRAS/SRILM-on-RaspberryPi</a></p>

<p>For dictionary generation, you can use either</p>

<ol>
<li><a href="https://github.com/AdolfVonKleist/Phonetisaurus" rel="nofollow">Phonetisaurus</a> with G2P Model available <a href="https://sourceforge.net/projects/cmusphinx/files/G2P%20Models/fst/" rel="nofollow">here</a> (or you can generate FST by yourself using <a href="https://sourceforge.net/projects/cmusphinx/files/G2P%20Models/phonetisaurus-cmudict-split.tar.gz/download" rel="nofollow">phonetisaurus-cmudict-split</a>), or</li>
<li><a href="https://github.com/cmusphinx/g2p-seq2seq" rel="nofollow">g2p-seq2seq</a> (Sequence-to-Sequence G2P toolkit)</li>
</ol>

<p>g2p-seq2seq based on <a href="https://www.tensorflow.org/versions/r0.9/tutorials/seq2seq/index.html" rel="nofollow">TensorFlow</a> which is not officially supported on RaspberryPi. For more details see <a href="https://github.com/samjabrahams/tensorflow-on-raspberry-pi" rel="nofollow">Installing TensorFlow on Raspberry Pi 3</a></p>

<p>For more details (usage, how to compile etc....) please go through the documents of respective toolkit.</p>

python pocketsphinx