Inverting large JSON dictionary
<p>I have a JSON dictionary containing multiple entries (roughly 8 million) of the following form:</p>

<pre><code>{"Some_String": {"Name0": 1, "Name1": 1, "Name42": 2, "Name5": 2, ... }, ...}
</code></pre>

<p>It contains strings that have been used to reference discrete named entities along with counts of how many times that name has been referenced by that string.</p>

<p>I want to invert the mapping so that the Name0 is followed by the strings that have referenced it (maintaining the counts). A name is likely to appear within multiple string entries. </p>

<pre><code>{"Name0": {"Some_String": 1, "Some_other_string": 1,... }, ...}
</code></pre>

<p>My question is: is there some JSON functionality that will allow me to efficiently do this?</p>

<p>My naive approach involves adding each name into a 2D array (adding the strings and counts to that array as they are found). </p>

<p>Initially this ran quite quickly, but as the size of the array increases the running time decreases (linear search). </p>

<pre><code>for string in list(surface.keys()):

    for count, name in zip(surfacs[string].values(),surface[string].keys()):

        if name in pages:
            surface_count_list[pages.index(name)].append([string, count])


        else:
            pages.append(name)
            surface_count_list.append([string, count])
</code></pre>

<p>I realise I could add this data directly into a new dictionary, but I didn't know if this would really increase the efficiency of adding new items as the size of the dictionary increases. </p>

<p>Thanks. </p>

<p>Something like</p>

<pre><code>from collections import defaultdict
result = defaultdict(dict)

for somestring, namesdict in initialdata.items():
    for name, amount in namesdict.items():
        result[name][something] = amount
</code></pre>

<p>Would do it, but with 8 million items it may become time to look at databases.</p>

python arrays json dictionary