Limits of hashes comparisons
<p>I'm storing hash codes in a file, one hash per line.</p>

<p>When I have a new hash code, I open the file and check if
the hash code already exists and if it doesn't exist,
I save it to that file.</p>

<pre><code>f = open("hashes.txt", "w")
hashes = f.readlines()
hash_code = "ff071fdf1e060400"
if not hash_code in hashes:
  hashes.append(hash_code)
for h in hashes:
  f.write(h)
f.close()
</code></pre>

<p>This code may be slow, when the number of lines in hashes.txt grows.
Is there some better storage mechanism and check to make it faster
in that case? I need fastest possible check (within a few seconds).</p>

<p>I would agree with Bert on this. If you expect that there will be a lot of hashes it would be better to use a database. If this only happens locally a sqlite database is fine. There is an excellent orm library that works for sqlite; <a href="http://docs.peewee-orm.com/en/latest/" rel="nofollow">Peewee</a>. It got some excellent documentation that will get you started. </p>

python hash