Convert a simple command into an AST in python
<p>I would like a function in Python that converts a string command into an AST (Abstract Syntax Tree).</p>

<p>The syntax for the command is as follows:</p>

<pre><code>commandName(3, "hello", 5.0, x::int)
</code></pre>

<p>A command can accept any number of comma separated values that can be either</p>

<ol>
<li>Integers</li>
<li>Strings</li>
<li>Floats</li>
<li>Types</li>
</ol>

<p>Suppose the function is called <code>convert_to_ast</code>, then</p>

<pre><code>convert_to_ast('commandName(3, "hello", 5.0, x::int)')
</code></pre>

<p>Should yield the following AST:</p>

<pre><code>{ 
  'type': 'command',
  'name': 'commandName',
  'args': [{
    'type': 'int',
    'value': 3
  }, {
    'type': 'str',
    'value': 'Hello'
  }, {
    'type': 'float',
    'value': 5.0
  }, {
    'type': 'var',
    'kind': 'int',
    'name': 'x
  }]
</code></pre>

<p>Seems like you could just evaluate the string and then pick off the types from there:</p>

<pre><code>&gt;&gt;&gt; items = ast.literal_eval('(404.5, "Hello", 5)')
&gt;&gt;&gt; [{'type': type(item).__name__, 'value': item} for item in items]
[{'type': 'float', 'value': 404.5}, {'type': 'str', 'value': 'Hello'}, {'type': 'int', 'value': 5}]
</code></pre>

<p>Of course, if you want to do more interesting things, you can access the AST directly:</p>

<pre><code>&gt;&gt;&gt; ast.dump(ast.parse('(404.5, "Hello", 5)'))
"Module(body=[Expr(value=Tuple(elts=[Num(n=404.5), Str(s='Hello'), Num(n=5)], ctx=Load()))])"
&gt;&gt;&gt; ast.parse('(404.5, "Hello", 5)').body[0].value.elts
[&lt;_ast.Num object at 0x107fa1250&gt;, &lt;_ast.Str object at 0x107fa1290&gt;, &lt;_ast.Num object at 0x107fa12d0&gt;]
</code></pre>

<p>For a more general thing than parsing a tuple (as you've added to the question), we still can use python's AST to parse this (as long as your syntax <em>is</em> valid python).  In this case, we'll create an <code>ast.NodeVisitor</code> which will pull out the information that we as it visits each node of the python AST that we care about.  In this case, we care about  <code>Call</code>, <code>Num</code>, <code>Str</code> and <code>Name</code> nodes:</p>

<pre><code>import ast

class Parser(ast.NodeVisitor):

    def __init__(self):
        self.calls = []
        self.current_command = None

    def visit_Call(self, node):
        name = node.func.id
        self.current_command = {
            'type': 'command',
            'name': name,
            'args': []
        }
        self.calls.append(self.current_command)
        for arg in node.args:
            self.visit(arg)
        self.current_command = None

    def visit_Num(self, node):
        if not self.current_command:
            return
        args = self.current_command['args']
        arg = {
            'type': type(node.n).__name__,
            'value': node.n
        }
        args.append(arg)

    def visit_Str(self, node):
        if not self.current_command:
            return
        args = self.current_command['args']
        arg = {
            'type': 'str',
            'value': node.s
        }
        args.append(arg)

    def visit_Name(self, node):
        if not self.current_command:
            return
        args = self.current_command['args']
        arg = {
            'type': 'type',
            'kind': node.id 
        }
        args.append(arg)


S = 'commandName(3, "hello", 5.0, int)'

tree = ast.parse(S)
p = Parser()
p.visit(tree)
print p.calls
</code></pre>

python parsing abstract-syntax-tree pyparsing