Parsing pyparsing group of mixed character words
<p>I'm trying to parse data fields from a Wikipedia infobox using pyparsing. To start with, the following code works:</p>

<pre><code>from pyparsing import *

test_line = """{{Infobox company | name                = Exxon Mobil Corp | num_employees_year  = 2015 }}"""

data_group = Group(
    Suppress("|") +
    OneOrMore(White()).suppress() +
    Word(alphanums + printables)("key") +
    OneOrMore(White()).suppress() +
    Suppress("=") +
    OneOrMore(White()).suppress() +
    OneOrMore(Word(alphanums))("value") +
    ZeroOrMore(White()).suppress()
)

infobox_parser = (
    Literal("{{").suppress() +
    Word("Infobox") +
    White().suppress() +
    Word("company") +
    OneOrMore(White()).suppress() +
    OneOrMore(data_group)("values") +
    Literal("}}").suppress()
)

print(infobox_parser.parseString(test_line))
</code></pre>

<p>Which produces result:</p>

<pre><code>['Infobox', 'company', ['name', 'Exxon', 'Mobil', 'Corp'], ['num_employees_year', '2015']]
</code></pre>

<p>The problem is when I change the test string to</p>

<pre><code>test_line = """{{Infobox company | name                = Exxon Mobil Corp. | num_employees_year  = 2015 }}"""
</code></pre>

<p>It's failing because I introduced '.' as part of 'Corp.'. I thought I could fix this by changing the Group object to</p>

<pre><code>data_group = Group(
    Suppress("|") +
    OneOrMore(White()).suppress() +
    Word(alphanums + printables)("key") +
    OneOrMore(White()).suppress() +
    Suppress("=") +
    OneOrMore(White()).suppress() +
    OneOrMore(Word(alphanums + printables))("value") +
    ZeroOrMore(White()).suppress()
)
</code></pre>

<p>but I'm getting the following error:</p>

<pre><code>pyparsing.ParseException: Expected "}}" (at char 91), (line:1, col:92)
</code></pre>

<p>What am I missing here? Thanks in advance.</p>

<p>Just a couple of things. Most significantly, pyparsing does not do the same kind of backtracking the way regex does. That is, something like this won't work:</p>

<pre><code>data = '{' + OneOrMore(Word(printables))("data") + '}'
print(data.parseString('{ this is some data }'))
</code></pre>

<p>Why? Because the terminating '}' <em>also</em> matches as a <code>Word(printables)</code>, so the <code>OneOrMore</code> will just keep going until the end, and then fail because there is no terminating '}' to be found after reading data.</p>

<p>Up until recently, the solution was to include a guard inside the <code>OneOrMore</code> expression, a negative lookahed saying in effect "I want Word(printables), but first check if it's a '}' - I don't want that", and that looks like this:</p>

<pre><code>data = '{' + OneOrMore(~Literal('}') + Word(printables))("data") + '}'
</code></pre>

<p>But this was so common, that I recently added an optional <code>stopOn</code> argument to <code>ZeroOrMore</code> and <code>OneOrMore</code>:</p>

<pre><code>data = '{' + OneOrMore(Word(printables), stopOn=Literal('}'))("data") + '}'
</code></pre>

<p>In your case, where each data_group parses a <code>key=value</code> pair, your value was fine when you only parsed <code>OneOrMore(Word(alphanums))</code>. But once you changed it to <code>OneOrMore(Word(alphanums+printables))</code>, your repetition term would greedily match the next '|' or the terminating '}}', and fail just like the example above.</p>

<p>A couple of other items:</p>

<ul>
<li><p>pyparsing will skip whitespace for you. All those White() elements are completely unnecessary.</p></li>
<li><p>In a couple of places you use Word incorrectly, as in <code>Word("Infobox")</code>. In your limited example, this matches ok, but remember that Word is defined with the set of characters that you will want to match as a word group, so <code>Word("Infobox")</code> will match not only "Infobox", but also any other word made up of the letters 'I', 'n', 'f', 'o', 'b', and/or 'x', such as "Inbox", "IbIx", "xoxoxox", etc. In this case, the pyparsing class you want would be <code>Literal</code> or <code>Keyword</code>.</p></li>
<li><p>Stepping back, it looks like your data_groups are <code>key=value</code> pairs, with delimiting '|'s. I would suggest using <code>delimitedList</code> for this.</p></li>
<li><p>Finally, use <code>dump()</code> to output your parsed data, it will help visualize the structure and results names.</p></li>
</ul>

<p>With these changes, your code looks like:</p>

<pre><code>data_group = Group(
    Word(alphas, alphanums+'_')("key") +
    Suppress("=") +
    originalTextFor(OneOrMore(Word(printables), stopOn=Literal('|') | '}}'))("value") 
)

infobox_parser = (
    Literal("{{").suppress() +
    Keyword("Infobox") +
    Keyword("company") + '|' + 
    Group(delimitedList(data_group, '|'))("values") +
    Literal("}}").suppress()
)

print(infobox_parser.parseString(test_line).dump())
</code></pre>

<p>Giving:</p>

<pre><code>['Infobox', 'company', '|', [['name', 'Exxon Mobil Corp.'], ['num_employees_year', '2015']]]
- values: [['name', 'Exxon Mobil Corp.'], ['num_employees_year', '2015']]
  [0]:
    ['name', 'Exxon Mobil Corp.']
    - key: name
    - value: Exxon Mobil Corp.
  [1]:
    ['num_employees_year', '2015']
    - key: num_employees_year
    - value: 2015
</code></pre>

python pyparsing