Pandas nested dict to dataframe
<p>I have a simple nested list like this:</p>

<pre><code>allFrame= [{'statValues': {'kpi2': 2, 'kpi1': 1}, 'modelName': 'first'},{'statValues': {'kpi2': 4, 'kpi1': 2}, 'modelName': 'second'}, {'statValues': {'kpi2': 3, 'kpi1': 3}, 'modelName': 'third'}]
</code></pre>

<p>a <code>pd.DataFrame(allFrame)</code> or <code>pd.DataFrame.from_dict(allFrame)</code>both do not really work and only return <a href="http://i.stack.imgur.com/UPV9o.jpg" rel="nofollow"><img src="http://i.stack.imgur.com/UPV9o.jpg" alt="json"></a></p>

<h2>How can I instead receive the kpi_X as column-names?</h2>

<p>I found <a href="http://stackoverflow.com/questions/32770359/python-dict-to-dataframe-pandas">Python dict to DataFrame Pandas</a> doing something similar. However, I believe this operation should be simpler</p>

<p>Looks like you need to flatten those dicts first.</p>

<p>Apply a flattening function on the list first:</p>

<pre><code>def flatten_dict(d, prefix='__'):
    def items():
        # A clojure for recursively extracting dict like values
        for key, value in d.items():
            if isinstance(value, dict):
                for sub_key, sub_value in flatten_dict(value).items():
                    # Key name should imply nested origin of the dict,
                    # so we use a default prefix of __ instead of _ or .
                    yield key + prefix + sub_key, sub_value
            else:
                yield key, value
    return dict(items())
</code></pre>

<p>Also note the use of orient=records, meaning each dict in the list is a line in the dataframe.</p>

<p>So:</p>

<pre><code>l = list(map(flatten_dict, allFrame))
df = pd.DataFrame.from_dict(l, orient='records')
</code></pre>

python pandas dictionary nested