python - Namespace vs regular package

Question

What's the difference between a namespace Python package (no __init__.py) and a regular Python package (has an __init__.py), especially when __init__.py is empty for a regular package?

I am curious because recently I've been forgetting to make __init__.py in packages I make, and I never noticed any problems. In fact, they seem to behave identically to regular packages.

Edit: Namespace packages only supported from Python 3.3 (see PEP 420), so naturally, this question only applies to Python 3.

Answer 1

Namespace packages

As of Python 3.3, we get namespace packages. These are a special kind of package that allows you to unify two packages with the same name at different points on your Python-path. For example, consider path1 and path2 as separate entries on your Python-path:

path1
+--namespace
   +--module1.py
   +--module2.py
path2
+--namespace
   +--module3.py
   +--module4.py

with this arrangement you should be able to do the following:

from namespace import module1, module3

thus you get the unification of two packages with the same name in a single namespace.

If either one of them gain an __init__.py that becomes the package - and you no longer get the unification as the other directory is ignored.

If both of them have an __init__.py, the first one in the PYTHONPATH (sys.path) is the one used.

__init__.py used to be required to make directory a package

Namespace packages are packages without the __init__.py.

For an example of a simple package, if you have a directory:

root
+--package
   +--file1.py
   +--file2.py
   ...

While you could run these files independently in the package directory, e.g. with python2 file1.py, under Python 2 you wouldn't be able to import the files as modules in the root directory, e.g.

import package.file1

would fail, and in order for it to work, you at least need this:

package
  +--__init__.py
  +--file1.py
  +--file2.py
  ...

__init__.py initializes the package so you can have code in the __init__.py that is run when the module is first imported:

run_initial_import_setup()

provide an __all__ list of names to be imported,

__all__ = ['star_import', 'only', 'these', 'names']

if the package is imported with the following:

from module import *

or you can leave the __init__.py completely empty if you only want to be able to import the remaining .py files in the directory.

Namespaces with __init__.py using pkgutil:

You could originally use pkgutil, available since Python 2.3. to accomplish adding namespaces, by adding the following into each separate package's __init__.py:

from pkgutil import extend_path
__path__ = extend_path(__path__, __name__)

Setuptools uses a similar method, again, all __init__.py files should contain the following (with no other code):

import pkg_resources
pkg_resources.declare_namespace(__name__)

Namespaces were more thoroughly addressed in PEP 420

See also more discussion on setuptools and Namespaces here:

http://peak.telecommunity.com/DevCenter/setuptools#namespace-packages