Welcome to OGeek Q&A Community for programmer and developer-Open, Learning and Share
Welcome To Ask or Share your Answers For Others

Categories

0 votes
362 views
in Technique[技术] by (71.8m points)

mongodb - How can I make my python mongo code run faster?

I am working on a python script that will pull some data from an online db which is named "data". I then am trying to check the entries in "data" with my local mongo repository and updating them if they differ.

Some of the entries in "data" have a "-" which I am choosing to skip by the first if else statement.

The second if statement checks if the id from "data" exists in my local repository. If it does, I assign it to a variable and convert it from a list to a string, otherwise I output that it was not found.

The third if statement checks if the string is not the same as the one in "data" and it is updated. Otherwise I just print that it has already been updated.

I have around 28,000 entries in data that are being checked so it ends up taking about 20 minutes. I am wondering if anyone has any advice of how to make it faster. Thanks for the help!

    if "-" not in data['Unique ID'][index]:
        if(problem.find_one({"_id" : ObjectId(data['Unique ID'][index])})):
            x = problem.find_one({"_id" : ObjectId(data['Unique ID'][index])})
            x = ', '.join(x['tags'])
            if x != (data['Lookup_tags'][index]):
                print("updated index: " + str(index))
                problem.find_one_and_update({"_id" : ObjectId((data['Unique ID'][index]))},{"$set":{"tags": [(data['Lookup_tags'][index])]}})
                print()
            else:
                print("already updated. " + str(index))
        else:
            print("didnt find")
    else:
        print("contains '-'. " + str(index))

与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
Welcome To Ask or Share your Answers For Others

1 Reply

0 votes
by (71.8m points)
等待大神答复

与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
OGeek|极客中国-欢迎来到极客的世界,一个免费开放的程序员编程交流平台!开放,进步,分享!让技术改变生活,让极客改变未来! Welcome to OGeek Q&A Community for programmer and developer-Open, Learning and Share
Click Here to Ask a Question

...