python - Remove duplicates and original from list

Question

Given a list of strings I want to remove the duplicates and original word.

For example:

lst = ['a', 'b', 'c', 'c', 'c', 'd', 'e', 'e']

The output should have the duplicates removed, so something like this ['a', 'b', 'd']

I do not need to preserve the order.

Answer 1

Use a collections.Counter() object, then keep only those values with a count of 1:

from collections import counter

[k for k, v in Counter(lst).items() if v == 1]

This is a O(N) algorithm; you just need to loop through the list of N items once, then a second loop over fewer items (< N) to extract those values that appear just once.

If order is important and you are using Python < 3.6, separate the steps:

counts = Counter(lst)
[k for k in lst if counts[k] == 1]

Demo:

>>> from collections import Counter
>>> lst = ['a', 'b', 'c', 'c', 'c', 'd', 'e', 'e']
>>> [k for k, v in Counter(lst).items() if v == 1]
['a', 'b', 'd']
>>> counts = Counter(lst)
>>> [k for k in lst if counts[k] == 1]
['a', 'b', 'd']

That the order is the same for both approaches is a coincidence; for Python versions before Python 3.6, other inputs may result in a different order.

In Python 3.6 the implementation for dictionaries changed and input order is now retained.