Welcome to OGeek Q&A Community for programmer and developer-Open, Learning and Share
Welcome To Ask or Share your Answers For Others

Categories

0 votes
345 views
in Technique[技术] by (71.8m points)

python - setdefault vs defaultdict performance

I am writing code for an application where performance is important. I am wondering why defaultdict seems to be faster then setdefault.

I would like to be able to use setdefault, mostly because i do not like the print output of the nested defaultdict (see implementation below).

In my code, i need to test if element_id is already a key of the dict.

Here are the two functions that i am testing:

def defaultdictfunc(subcases,other_ids,element_ids):
    dict_name= defaultdict(lambda: defaultdict(lambda: defaultdict(dict)))
    for subcase in subcases:
        for other_id in other_ids:
            for element_id in element_ids: 
                if element_id in dict_name[subcase][other_id]:
                    # error duplicate element_id
                    pass
                else:
                    dict_name[subcase][other_id][element_id]=0
    return dict_name

def setdefaultfunc(subcases,other_ids,element_ids):
    dict_name={}
    for subcase in subcases:
        for other_id in other_ids:
            for element_id in element_ids: 
                if element_id in dict_name.setdefault(subcase,{}).setdefault(other_id,{}):
                    # error duplicate element_id
                    pass
                else:
                    dict_name[subcase][other_id][element_id]=0

    return dict_name

IPython input and output:

In [1]: from numpy.random import randint

In [2]: subcases,other_ids,element_ids=(randint(0,100,100),randint(0,100,100),randint(0,100,100))

In [5]: from collections import defaultdict

In [6]: defaultdictfunc(subcases,other_ids,element_ids)==setdefaultfunc(subcases,other_ids,element_ids)
Out[6]: True

In [7]: %timeit defaultdictfunc(subcases,other_ids,element_ids)
10 loops, best of 3: 177 ms per loop

In [8]: % timeit setdefaultfunc(subcases,other_ids,element_ids)
1 loops, best of 3: 351 ms per loop

Why is setdefaultfunc slower. I thought the underlying code would be the same. Is there a way to improve its speed?

Thanks

See Question&Answers more detail:os

与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
Welcome To Ask or Share your Answers For Others

1 Reply

0 votes
by (71.8m points)

According to user aneroid:

It would make sense that defaultdict is faster that dict.setdefault() since the former sets its default for the entire dict at creation time, whereas setdefault() does it per element when it is read. One reason to use setdefault is when the default you assign is based on the key (or something) rather than a generic default for the entire dict.


与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
OGeek|极客中国-欢迎来到极客的世界,一个免费开放的程序员编程交流平台!开放,进步,分享!让技术改变生活,让极客改变未来! Welcome to OGeek Q&A Community for programmer and developer-Open, Learning and Share
Click Here to Ask a Question

...