Welcome to OGeek Q&A Community for programmer and developer-Open, Learning and Share
Welcome To Ask or Share your Answers For Others

Categories

0 votes
726 views
in Technique[技术] by (71.8m points)

python - multiprocessing: variable being referenced before assignment in some cases but not others

I found the following example on this website somewhere:

import multiprocessing
import ctypes
import numpy as np

shared_array_base = multiprocessing.Array(ctypes.c_double, 10*10)
shared_array = np.ctypeslib.as_array(shared_array_base.get_obj())
shared_array = shared_array.reshape(10, 10)

# No copy was made
assert shared_array.base.base is shared_array_base.get_obj()

# Parallel processing
def my_func(i, def_param=shared_array):
    shared_array[i,:] = i

if __name__ == '__main__':
    pool = multiprocessing.Pool(processes=4)
    pool.map(my_func, range(10))

    print shared_array

The above code works fine, but if I want to add an array to the shared array, something like shared_array += some_other_array (instead of the above shared_array[i,;] = i) I get

local variable 'shared_array' referenced before assignment

Any ideas why I cannot do that?

See Question&Answers more detail:os

与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
Welcome To Ask or Share your Answers For Others

1 Reply

0 votes
by (71.8m points)

If a variable is assigned to anywhere in a function, it is treated as a local variable. shared_array += some_other_array is equivalent to shared_array = shared_array + some_other_array. Thus shared_array is treated as a local variable, which does not exist at the time you try to use it on the right-hand side of the assignment.

If you want to use the global shared_array variable, you need to explicitly mark it as global by putting a global shared_array in your function.

The reason you don't see the error with shared_array[i,:] = i is that this does not assign to the variable shared_array. Rather, it mutates that object, assigning to a slice of it. In Python, assigning to a bare name (e.g., shared_array = ...) is very different from any other kind of assignment (e.g., shared_array[...] = ...), even though they look similar.

Note, incidentally, that the error has nothing to do with multiprocessing.


与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
OGeek|极客中国-欢迎来到极客的世界,一个免费开放的程序员编程交流平台!开放,进步,分享!让技术改变生活,让极客改变未来! Welcome to OGeek Q&A Community for programmer and developer-Open, Learning and Share
Click Here to Ask a Question

...