Converting a string to a list of lists
<p>Using Python 2.7, I want to convert a string in the following format to a list of lists:</p>

<pre><code>&gt;&gt;&gt; value = "[[1 0 0 0] [0 1 0 0] [0 0 1 0] [0 0 0 1]]"
&gt;&gt;&gt; ...
&gt;&gt;&gt; result = [[1.0, 0.0, 0.0, 0.0], [0.0, 1.0, 0.0, 0.0], [0.0, 0.0, 1.0, 0.0], [0.0, 0.0, 0.0, 1.0]]
</code></pre>

<p>I did manage to do just that, but it seems sloppy.</p>

<pre><code>&gt;&gt;&gt; value = "[[1 0 0 0] [0 1 0 0] [0 0 1 0] [0 0 0 1]]"
&gt;&gt;&gt; val2 = value.replace('[[', '').replace(']]', '').split('] [')
&gt;&gt;&gt; val2
['1 0 0 0', '0 1 0 0', '0 0 1 0', '0 0 0 1']
&gt;&gt;&gt; val_final = [[float(x) for x in x] for x in [x.replace(' ', '') for x in val2]]
&gt;&gt;&gt; val_final
[[1.0, 0.0, 0.0, 0.0], [0.0, 1.0, 0.0, 0.0], [0.0, 0.0, 1.0, 0.0], [0.0, 0.0, 0.0, 1.0]]
</code></pre>

<p><strong>Is there a better, cleaner way to do this that would be more robust?</strong></p>

<p>Note: I do expect to only have integer or float values in the string (context: it's a 4x4 matrix corresponding to an object position in a 3D environment).</p>

<hr>

<p><strong>Edit:</strong>
Alternative value could contain floats like so:</p>

<pre><code>&gt;&gt;&gt; value = "[[9.231 -0.123 -2 0] [0 1 0 0] [0 0 1 0] [0 0 0 1]]"
&gt;&gt;&gt; ...
&gt;&gt;&gt; result = [[9.231, -0.123, -2.0, 0.0], [0.0, 1.0, 0.0, 0.0], [0.0, 0.0, 1.0, 0.0], [0.0, 0.0, 0.0, 1.0]]
</code></pre>

<p>Well the most flexible and extensible way would be to implement an <a href="https://en.wikipedia.org/wiki/LL_parser" rel="nofollow">LL(1) parser</a>. It requires a bit of work though.
Here is a rough implementation in Python 2.7:</p>

<pre><code>value = "[[1 0 0 0] [0 1 0 0] [0 0 1 0] [0 0 0 1]]"

class ParseError(ValueError):
    pass

def tokenize(s):
    buf = ""

    while s:
        if s[0] == '[':
            yield s[0]
        elif s[0].isalnum():
            buf += s[0]
        elif s[0] == ']':
            if buf:
                yield buf
                buf = ""
            yield ']'
        elif s[0] == ' ':
            if buf:
                yield buf
                buf = ""
        else:
            raise ParseError()
        s = s[1:]

def parse_array(tokens):
    if tokens[0] != '[':
        raise ParseError()
    tokens = tokens[1:]
    elements = []
    while tokens[0] != ']':
        element, tokens = parse(tokens)
        elements.append(element)

        if not tokens:
            raise ParseError()
    return elements, tokens[1:]

def parse_number(tokens):
    return float(tokens[0]), tokens[1:]

def parse(tokens):
    if tokens[0] == '[':
        return parse_array(tokens)
    elif tokens[0].isalnum():
        return parse_number(tokens)
    else:
        raise ParseError()

tokens = list(tokenize(value))
print tokens
# ['[', '[', '1', '0', '0', '0', ']', '[', '0', '1', '0', '0', ']', '[', '0', '0', '1', '0', ']', '[', '0', '0', '0', '1', ']', ']']
parsed, tokens = parse(tokens)
print parsed
# [[1.0, 0.0, 0.0, 0.0], [0.0, 1.0, 0.0, 0.0], [0.0, 0.0, 1.0, 0.0], [0.0, 0.0, 0.0, 1.0]]
</code></pre>

<p>Note : haven't tested all the edge cases, but it should give an idea of how it may look like.</p>

<p>Here I'm assuming that:</p>

<ul>
<li>there can be any number of spaces between the numbers or brackets</li>
<li>brackets don't need to be followed by a space</li>
<li>the format represents <a href="https://en.wikipedia.org/wiki/Jagged_array" rel="nofollow">jagged arrays</a> (they can have rows of different lengths) of any dimensions (they can be arbitrarily nested).</li>
</ul>

<p>This is pretty much the most permissive assumptions I can make about this format, so it's probably overkill if you don't need to handle these use cases. But if this is what you need, then congratulations, you have an LL(1) grammar (in other words, a small language), and you need this kind of solution.</p>

<p>There are two steps:</p>

<ol>
<li>Tokenization: transforms a string to a sequence of symbols (brackets, number, and potentially other things like identifiers).</li>
<li>Parsing: transforms the sequence of symbols in an expression tree. To do that you need to read one symbol ahead before knowing how to interpret the following symbols (thus the 1 in LL1).</li>
</ol>

python python-2.7