Parsing key-value log in Python
<p>I have a system log that looks like the following:</p>

<pre><code>{
    a = 1
    b = 2
    c = [
            x:1,
            y:2,
            z:3,
        ]
    d = 4
}
</code></pre>

<p>I want to parse this in Python into a dictionary object with = splitting key-value pairs. At the same time, the array that is enclosed by [] is also preserved. I want to keep this as generic as possible so the parsing can also hold some future variations.</p>

<p>What I tried so far (code will be written): split each line by "=" into key-value pair, determine where [ and ] starts and end and then split the lines in between by ":" into key-value pairs. That seems a little hard-coded.. Any better idea?</p>

<p>This could be pretty easily simplified to YAML. <code>pip install pyyaml</code>, then set up like so:</p>

<pre><code>import string, yaml

data = """
{
    a = 1
    b = 2
    c = [
            x:1,
            y:2,
            z:3,
        ]
    d = 4
}
"""
</code></pre>

<p>With this setup, you can use the following to parse your data:</p>

<pre><code>data2 = data.replace(":", ": ").replace("=", ":").replace("[","{").replace("]","}")

lines = data2.splitlines()
for i, line in enumerate(lines):
    if len(line)&gt;0 and line[-1] in string.digits and not line.endswith(",") or i &lt; len(lines) - 1 and line.endswith("}"):
        lines[i] += ","
data3 = "\n".join(lines)
yaml.load(data3) # {'a': 1, 'b': 2, 'c': {'x': 1, 'y': 2, 'z': 3}, 'd': 4}
</code></pre>

<hr>

<h1>Explanation</h1>

<p>In the first line, we perform some simple substitutions:</p>

<ul>
<li>YAML requires that there is a space after colons in key/value pairs. So with <code>replace(":", ": ")</code>, we can ensure this.</li>
<li>Since YAML key/value pairs are always denoted by a colon and your format sometimes uses equals signs, we replace equal signs with commas using <code>.replace("=", ":")</code></li>
<li>Your format sometimes uses square brackets where curly brackets should be used in YAML. We fix using <code>.replace("[","{").replace("]","}")</code></li>
</ul>

<p>At this point, your data looks like this:</p>

<pre><code>{
    a : 1
    b : 2
    c : {
            x: 1,
            y: 2,
            z: 3,
        }
    d : 4
}
</code></pre>

<p>Next, we have a <code>for</code> loop. This is simply responsible for adding commas after lines where they're missing. The two cases in which for loops are missing are:
- They're absent after a numeric value
- They're absent after a closing bracket</p>

<p>We match the first of these cases using <code>len(line)&gt;0 and line[-1] in string.digits</code> (the last character in the line is a digit)</p>

<p>The second case is matched using <code>i &lt; len(lines) - 1 and line.endswith("}")</code>. This checks if the line ends with <code>}</code>, and also checks that the line is not the last, since YAML won't allow a comma after the last bracket.</p>

<p>After the loop, we have:</p>

<pre><code>{
    a : 1,
    b : 2,
    c : {
            x: 1,
            y: 2,
            z: 3,
        },
    d : 4,
}
</code></pre>

<p>which is valid YAML. All that's left is <code>yaml.load</code>, and you've got yourself a python <code>dict</code>.</p>

<p>If anything isn't clear please leave a comment and I'll happily elaborate.</p>

python