How to filter out elements from list by comparison function, which works with pairs in this list?
<p>I have list of PIL <em>images</em> (more than 2) and function <em>diff (im1, im2)</em>, which return True if percentage difference in images is small - i.e. if it return True, then 2 compared images - duplicate. How do I get PIL list of images without duplicates (i.e. if I have 3 equal images of sun and 2 images of cat, I need only one image of sun and one image of cat)? 
I've wanted use filter, but it filter out elements of list with function one by one, but in my situation I have only comparison function.</p>

<p><strong>upd1:</strong>
I've tried code, suggested by AChampion:</p>

<pre><code>import itertools as it

dupes = {i for i in (i1, i2) for i1, i2 in it.combinations(images_list, 2) if diff(i1, i2)}
set(images_list) - dupes
</code></pre>

<p>but python 3.5 says NameError: name 'i1' is not defined, I quite could not understand it also, </p>

<p><strong>upd2:</strong>
So I've corrected this code as I could and ended up in some sort of nested list, which I "flattened" (and not sure all this stuff works properly), but then I'm getting <strong>TypeError: unhashable type: 'JpegImageFile'</strong> for <strong>string images = set(images) - dupes</strong></p>

<p>example code:</p>

<pre><code>for url in images_urls:
    images.append(Image.open(requests.get(url, stream=True).raw))

dupes = [[i for i in (i1, i2)] for i1, i2 in it.combinations(images, 2) if diff(i1, i2)]
dupes = [item for sublist in dupes for item in sublist]
dupes = set(dupes)
images = set(images) - dupes
</code></pre>

<p><strong>upd3:</strong> as @Padraic Cunningham suggested I've got:</p>

<pre><code>for url in images_urls:
    print('downloading url:', url)
    images.append(Image.open(requests.get(url, stream=True).raw))

dupes = set(itertools.chain(*(t for t in itertools.combinations(images, 2) if diff(*t))))
images = set(images) - dupes
</code></pre>

<p>But still getting <strong>TypeError: unhashable type: 'JpegImageFile'</strong>
maybe there is more easy way to do this without hash/eq approach?
For now I will try to look closely at hash/eq realization approach...</p>

<p>--
<strong>upd4:</strong>
I need {'flower','cat'}, but I'm getting {'flower'}</p>

<pre><code>import itertools
def diff(a, b):
    return a == b
images_list = [
    'flower',
    'cat',
    'flower'
]
some_values = set(itertools.chain(*(t for t in itertools.combinations(images_list, 2) if diff(*t))))
print(some_values)
</code></pre>

<p>Construct a set of all the duplicates and subtract from original set. I've changed the diff to similarity test rather than an equality test to better reflect the problem description:</p>

<pre><code>images_list = [
    'image red contents1',
    'image green contents',
    'image red contents2',
]
import itertools as it
def diff(a, b):
    return a.split()[1] == b.split()[1]

dupes = {i2 for i1, i2 in it.combinations(images_list, 2) if diff(i1, i2)}
set(images_list) - dupes
# {'image green contents', 'image red contents1'}
</code></pre>

python list filter set itertools