I've recently started testing MongoDB via shell and via PyMongo. I've noticed that returning a cursor and trying to iterate over it seems to bottleneck in the actual iteration. Is there a way to return more than one document during iteration?
Pseudo code:
for line in file:
value = line[a:b]
cursor = collection.find({"field": value})
for entry in cursor:
(deal with single entry each time)
What I'm hoping to do is something like this:
for line in file
value = line[a:b]
cursor = collection.find({"field": value})
for all_entries in cursor:
(deal with all entries at once rather than iterate each time)
I've tried using batch_size() as per this question and changing the value all the way up to 1000000, but it doesn't seem to have any effect (or I'm doing it wrong).
Any help is greatly appreciated. Please be easy on this Mongo newbie!
--- EDIT ---
Thank you Caleb. I think you've pointed out what I was really trying to ask, which is this: is there any way to do a sort-of collection.findAll()
or maybe cursor.fetchAll()
command, as there is with the cx_Oracle module? The problem isn't storing the data, but retrieving it from the Mongo DB as fast as possible.
As far as I can tell, the speed at which the data is returned to me is dictated by my network since Mongo has to single-fetch each record, correct?
See Question&Answers more detail:
os 与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…