Get the number of all keys in a dictionary of dictionaries in Python

Question

I have a dictionary of dictionaries in Python 2.7.

I need to quickly count the number of all keys, including the keys within each of the dictionaries.

So in this example I would need the number of all keys to be 6:

dict_test = {'key2': {'key_in3': 'value', 'key_in4': 'value'}, 'key1': {'key_in2': 'value', 'key_in1': 'value'}}

I know I can iterate through each key with for loops, but I am looking for a quicker way to do this, since I will have thousands/millions of keys and doing this is just ineffective:

count_the_keys = 0

for key in dict_test.keys():
    for key_inner in dict_test[key].keys():
       count_the_keys += 1

# something like this would be more effective
# of course .keys().keys() doesn't work
print len(dict_test.keys()) * len(dict_test.keys().keys())

question from:https://stackoverflow.com/questions/35427814/get-the-number-of-all-keys-in-a-dictionary-of-dictionaries-in-python

Answer 1

Keeping it Simple

If we know all the values are dictionaries, and do not wish to check that any of their values are also dictionaries, then it is as simple as:

len(dict_test) + sum(len(v) for v in dict_test.itervalues())

Refining it a little, to actually check that the values are dictionaries before counting them:

len(dict_test) + sum(len(v) for v in dict_test.itervalues() if isinstance(v, dict))

And finally, if you wish to do an arbitrary depth, something like the following:

def sum_keys(d):
    return (0 if not isinstance(d, dict) 
            else len(d) + sum(sum_keys(v) for v in d.itervalues())

print sum_keys({'key2': {'key_in3': 'value', 'key_in4': 'value'}, 
                'key1': {'key_in2': 'value', 
                         'key_in1': dict(a=2)}})
# => 7

In this last case, we define a function that will be called recursively. Given a value d, we return either:

• 0 if that value is not a dictionary; or
• the number of keys in the dictionary, plus the total of keys in all of our children.

Making it Faster

The above is a succinct and easily understood approach. We can get a little faster using a generator:

def _counter(d):
    # how many keys do we have?
    yield len(d)

    # stream the key counts of our children
    for v in d.itervalues():
        if isinstance(v, dict):
            for x in _counter(v):
                yield x

def count_faster(d):
    return sum(_counter(d))

This gets us a bit more performance:

In [1]: %timeit sum_keys(dict_test)
100000 loops, best of 3: 4.12 μs per loop

In [2]: %timeit count_faster(dict_test)
100000 loops, best of 3: 3.29 μs per loop