scope - globals and locals in python exec()

Question

I'm trying to run a piece of python code using exec.

my_code = """
class A(object):
  pass

print 'locals: %s' % locals()
print 'A: %s' % A

class B(object):
  a_ref = A
"""

global_env = {}
local_env = {}
my_code_AST = compile(my_code, "My Code", "exec")
exec(my_code_AST, global_env, local_env)

print local_env

which results in the following output

locals: {'A': <class 'A'>}
A: <class 'A'>
Traceback (most recent call last):
  File "python_test.py", line 16, in <module>
    exec(my_code_AST, global_env, local_env)
  File "My Code", line 8, in <module>
  File "My Code", line 9, in B
NameError: name 'A' is not defined

However, if I change the code to this -

my_code = """
class A(object):
  pass

print 'locals: %s' % locals()
print 'A: %s' % A

class B(A):
  pass
"""

global_env = {}
local_env = {}
my_code_AST = compile(my_code, "My Code", "exec")
exec(my_code_AST, global_env, local_env)

print local_env

then it works fine - giving the following output -

locals: {'A': <class 'A'>}
A: <class 'A'>
{'A': <class 'A'>, 'B': <class 'B'>}

Clearly A is present and accessible - what's going wrong in the first piece of code? I'm using 2.6.5, cheers,

Colin

* UPDATE 1 *

If I check the locals() inside the class -

my_code = """
class A(object):
  pass

print 'locals: %s' % locals()
print 'A: %s' % A

class B(object):
  print locals()
  a_ref = A
"""

global_env = {}
local_env = {}
my_code_AST = compile(my_code, "My Code", "exec")
exec(my_code_AST, global_env, local_env)

print local_env

Then it becomes clear that locals() is not the same in both places -

locals: {'A': <class 'A'>}
A: <class 'A'>
{'__module__': '__builtin__'}
Traceback (most recent call last):
  File "python_test.py", line 16, in <module>
    exec(my_code_AST, global_env, local_env)
  File "My Code", line 8, in <module>
  File "My Code", line 10, in B
NameError: name 'A' is not defined

However, if I do this, there is no problem -

def f():
  class A(object):
    pass

  class B(object):
    a_ref = A

f()

print 'Finished OK'

* UPDATE 2 *

ok, so the docs here - http://docs.python.org/reference/executionmodel.html

'A class definition is an executable statement that may use and define names. These references follow the normal rules for name resolution. The namespace of the class definition becomes the attribute dictionary of the class. Names defined at the class scope are not visible in methods.'

It seems to me that 'A' should be made available as a free variable within the executable statement that is the definition of B, and this happens when we call f() above, but not when we use exec(). This can be more easily shown with the following -

my_code = """
class A(object):
  pass

print 'locals in body: %s' % locals()
print 'A: %s' % A

def f():
  print 'A in f: %s' % A

f()

class B(object):
  a_ref = A
"""

which outputs

locals in body: {'A': <class 'A'>}
A: <class 'A'>
Traceback (most recent call last):
  File "python_test.py", line 20, in <module>
    exec(my_code_AST, global_env, local_env)
  File "My Code", line 11, in <module>
  File "My Code", line 9, in f
NameError: global name 'A' is not defined

So I guess the new question is - why aren't those locals being exposed as free variables in functions and class definitions - it seems like a pretty standard closure scenario.

Answer 1

Well, I believe it's either an implementation bug or an undocumented design decision. The crux of the issue is that a name-binding operation in the module-scope should bind to a global variable. The way it is achieved is that when in the module level, globals() IS locals() (try that one out in the interpreter), so when you do any name-binding, it assigns it, as usual, to the locals() dictionary, which is also the globals, hence a global variable is created.

When you look up a variable, you first check your current locals, and if the name is not found, you recursively check locals of containing scopes for the variable name until you find the variable or reach the module-scope. If you reach that, you check the globals, which are supposed to be the module scope's locals.

>>> exec(compile("import sys
print sys._getframe().f_code.co_name", "blah", "exec"), {}, {})
<module>
>>> exec("a = 1
class A(object):
print a
", {}, {})
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<string>", line 2, in <module>
  File "<string>", line 3, in A
NameError: name 'a' is not defined
>>> d = {}
>>> exec("a = 1
class A(object):
print a
", d,d)
1

This behavior is why inheritance worked (The name-lookup used code object's scope locals(), which indeed had A in it).

In the end, it's an ugly hack in the CPython implementation, special-casing globals lookup. It also causes some nonsensical artifical situations - e.g.:

>>> def f():
...     global a
...     a = 1
...
>>> f()
>>> 'a' in locals()
True

Please note that this is all my inference based on messing with the interpreter while reading section 4.1 (Naming and binding) of the python language reference. While this isn't definitive (I haven't opened CPython's sources), I'm fairly sure I'm correct about the behavior.