Quickly convert numpy arrays with index to dict of numpy arrays keyed on that index
<p>I have a set of numpy arrays. One of these is a list of "keys", and I'd like to rearrange the arrays into a dict of arrays keyed on that key. My current code is:</p>

<pre><code>for key, val1, val2 in itertools.izip(keys, vals1, vals2):
    dict1[key].append(val1)
    dict2[key].append(val2)
</code></pre>

<p>This is pretty slow, since the arrays involved are millions of entries long, and this happens many times. Is it possible to rewrite this in vectorized form? The set of possible keys is known ahead of time, and there are ~10 distinct keys.</p>

<p><strong>Edit:</strong> If there are k distinct keys and the list is n long, the current answers are O(nk) (iterate once for each key) and O(n log n) (sort first). I'm still looking for an O(n) vectorized solution, though. This is hopefully possible; after all, the easiest possible nonvectorized thing (i.e. what I already have) is O(n).</p>

<p><code>defaultdict</code> is intended for building dictionaries like this.  In particular is streamlines the step of creating a new dictionary entry for a new key.</p>

<pre><code>In [19]: keys = np.random.choice(np.arange(10),100)
In [20]: vals=np.arange(100)
In [21]: from collections import defaultdict
In [22]: dd = defaultdict(list)
In [23]: for k,v in zip(keys, vals):
    ...:     dd[k].append(v)
    ...:     
In [24]: dd
Out[24]: 
defaultdict(list,
            {0: [4, 39, 47, 84, 87],
             1: [0, 25, 41, 46, 55, 58, 74, 77, 89, 92, 95],
             2: [3, 9, 15, 24, 44, 54, 63, 66, 71, 80, 81],
             3: [1, 13, 16, 37, 57, 76, 91, 93],
             ...
             8: [51, 52, 56, 60, 68, 82, 88, 97, 99],
             9: [21, 29, 30, 34, 35, 59, 73, 86]})
</code></pre>

<p>But with a small known set of keys you don't need this specialized dictionary, since you can easily create the dictionary key entries ahead of time </p>

<pre><code>dd = {k:[] for k in np.unique(keys)}
</code></pre>

<p>But since you are starting with arrays, array operations to sort and collect like values might well be worth it.</p>

python numpy vectorization