Welcome to OGeek Q&A Community for programmer and developer-Open, Learning and Share
Welcome To Ask or Share your Answers For Others

Categories

0 votes
330 views
in Technique[技术] by (71.8m points)

python - Make concurrent.futures.ProcessPoolExecutor skip all errors / exceptions, but continue

here is my simple example where I run a function using ProcessPoolExecutor, passing list o numbers that will be iterated over. Another argument (b) is a constant in each iteration (that's why _helper function).

I put some value in a list, which will cause an exception to occur - 'a'. I want to make the processing to continue, skipping this part. Unfortunately, it is stopping on this value:

How can I make it continue? printing: 2 3 5 6

Thanks

import concurrent.futures as cf

workers = 4

def f(a, b):
    return a + b

def _helper(x):
    return f(x, 1)

my_iterable_collection = [1,2,'a',4,5]

def main():
    with cf.ProcessPoolExecutor(max_workers=workers) as executor:
        try:
            for result in executor.map(_helper, my_iterable_collection):
                print(result)
        except Exception:
            pass


if __name__ == '__main__':
    main()

# 2
# 3

question from:https://stackoverflow.com/questions/65832364/make-concurrent-futures-processpoolexecutor-skip-all-errors-exceptions-but-co

与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
Welcome To Ask or Share your Answers For Others

1 Reply

0 votes
by (71.8m points)

You can try this way:

def f(a, b):
    return a + b

def _helper(x):
    try:
        return f(x, 1)
    except Exception:
        pass

my_iterable_collection = [1,2,'a',4,5]

def main():
    with cf.ProcessPoolExecutor(max_workers=workers) as executor:
        for result in executor.map(_helper, my_iterable_collection):
            if result:
                print(result)

与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
OGeek|极客中国-欢迎来到极客的世界,一个免费开放的程序员编程交流平台!开放,进步,分享!让技术改变生活,让极客改变未来! Welcome to OGeek Q&A Community for programmer and developer-Open, Learning and Share
Click Here to Ask a Question

...