Get unique entries in list of lists by an item
<p>This seems like a fairly straightforward problem but I can't seem to find an efficient way to do it. I have a list of lists like this:</p>

<pre><code>list = [['abc','def','123'],['abc','xyz','123'],['ghi','jqk','456']]
</code></pre>

<p>I want to get a list of unique entries by the third item in each child list (the 'id'), i.e. the end result should be</p>

<pre><code>unique_entries = [['abc','def','123'],['ghi','jqk','456']]
</code></pre>

<p>What is the most efficient way to do this? I know I can use set to get the unique ids, and then loop through the whole list again. However, there are more than 2 million entries in my list and this is taking too long. Appreciate any pointers you can offer! Thanks. </p>

<p>How about this: Create a <code>set</code> that keeps track of ids already seen, and only append sublists where id's where not seen.</p>

<pre><code>l = [['abc','def','123'],['abc','xyz','123'],['ghi','jqk','456']]

seen = set()
new_list = []

for sl in l:
    if sl[2] not in seen:
        new_list.append(sl)
        seen.add(sl[2])

print new_list
</code></pre>

<p>Result:</p>

<pre><code>[['abc', 'def', '123'], ['ghi', 'jqk', '456']]
</code></pre>

python list set