Welcome to OGeek Q&A Community for programmer and developer-Open, Learning and Share
Welcome To Ask or Share your Answers For Others

Categories

0 votes
360 views
in Technique[技术] by (71.8m points)

python - Can't pickle Function

So I'm trying to speed up my computation time by doing a little bit multiprocessing

I'm trying to use the pool workers.

At the top of my code I have

import Singal as s
import multiprocessing as mp
def wrapper(Channel):
    Noise_Frequincies = []
    for i in range(1,125):
        Noise_Frequincies.append(60.0*float(i))
    Noise_Frequincies.append(180.0)
    filter1 = s.Noise_Reduction(Sample_Rate,Noise_Frequincies,Channel)
    return filter1

Then when the time comes I use

Both_Channels = [Chan1, Chan2]
results = mp.Pool(2).map(wrapper,Both_Channels)
filter1 = results[0]
filter2 = results[1]

I get the following error

Exception in thread Thread-2:
Traceback (most recent call last):
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/threading.py", line 808, in __bootstrap_inner
self.run()
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/threading.py", line 761, in run
self.__target(*self.__args, **self.__kwargs)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/multiprocessing/pool.py", line 342, in _handle_tasks
put(task)
PicklingError: Can't pickle <type 'function'>: attribute lookup __builtin__.function failed

Chan1 and Chan2 are arrays of my signal, and I'm trying to filter out some noise out of each. I'm new to multiprocessing so I apologize if this is a dumb error

See Question&Answers more detail:os

与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
Welcome To Ask or Share your Answers For Others

1 Reply

0 votes
by (71.8m points)

I marked this question as dup of that Q/A, and to sum up: you can't pickle functions, and that's what you're trying to do when you pass wrapper() to Pool().map(). To pickle functions, you need to use copy_reg, as shown by this example.

" Classes, functions, and methods cannot be pickled -- if you pickle an object, the object's class is not pickled, just a string that identifies what class it belongs to. " (cf doc)

I don't use custom classes, and that's where his issue is (at least as far as I can tell)

no, his problem that he was trying to pickle an instance method, which is relatively close to a function, as both can't be pickled. And the workaround for that A should work for you too.

Though I did not test it…

HTH


与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
OGeek|极客中国-欢迎来到极客的世界,一个免费开放的程序员编程交流平台!开放,进步,分享!让技术改变生活,让极客改变未来! Welcome to OGeek Q&A Community for programmer and developer-Open, Learning and Share
Click Here to Ask a Question

...