Debug slow ANTLR generating and parsing
<p>I'm using ANTLR4 + Python2 target, </p>

<pre><code>time java -jar lib/antlr-4.5-complete.jar -visitor -o build -Dlanguage=Python2 xxx.g4
time ./main.py
</code></pre>

<p>It takes 10s to generate the visitor/lexer/parser file, and another 4s to execute the visitor.</p>

<p>How should I debug its slowness?</p>

<p>I write a lot of antlr4 parsers, and I face speed issues all the time. I have a feeling Python target is usually pretty slow. When a parser is really slow, it usually have reduce/reduce issue meaning there are multiple sub rules having the same terminal rule so the parser get confused.</p>

<p>When a parser is slow, I turn on the trace mode and display trace messages. When the trace message is always delayed at one same rule, it is the rule you start investigating. You will look at the rule's sub rules and see if there are multiple sub rules that could become the top delay rule. You may want to comment out or modify a rule and see if that makes the parser faster. It is hard when the grammar is large like the C++ grammar. </p>

<p>As a rule of thumb of optimizing an antlr parser grammar, one is eliminating left recursion, and the other is eliminating sub rules that has the same starting rule. e.g.</p>

<p><code>
declaration:
       declspecifier* what_follows1
     | declspecifier* what_follows2
     | declspecifier* what_follows3
</code></p>

<p>Empirically I find this kind of grammar rules make the parser slow. Change a bit at a time and check if the speed changes, and make sure reduction step isn't broken. </p>

<p>I hope all this makes sense.</p>

python antlr4