Python large array comparison
<p>I have a large array containing URL (it can contains 100 000 URL strings), and I would like to know if my actual URL is one of the URL from the array. For that, I have to compare the actual URL string with all the URL string in the array. Is there any way to compare with this large array but with less time than I do now ? For now it's :</p>

<pre><code>error = 0
for oldUrl in urlList:
    error = 1 if oldUrl == actualUrl else error
</code></pre>

<p>Don't use a list for this. Lookups in lists have a worst case complexity of O(n).</p>

<p>Use a set (or dictionary if you have other metadata) instead. This has a lookup of roughly O(1). See <a href="http://stackoverflow.com/q/3489071/1048539">here</a> for comparisons between a set, dictionary, and list.</p>

<p>Using a set, the lookup is simple:</p>

<pre><code>urls = set(['url1', 'url2', 'url3'])

print ('url2' in urls)
print ('foobar' in urls)
</code></pre>

<p>Or in your case, convert your list object as a set:</p>

<pre><code>urlListSet = set(urlList)
print(oldUrl in urlListSet)
</code></pre>

<p>You can also add new urls to your set:</p>

<pre><code>urlListSet.add(newurl)
urlListSet.update(listOfNewUrls)
</code></pre>

python arrays performance python-2.7 compare