python - Module imports and __init__.py

Question

I am trying to understand what the best practices are with regards to Python's (v2.7) import mechanics. I have a project that has started to grow a bit and lets say my code is organised as follows:

foo/
    __init__.py
    Foo.py
    module1.py
    module2.py
    module3.py

The package name is foo and underneath it I have module Foo.py which contains code for the class Foo. Hence I am using the same name for the package, module and class which might not be very clever to start with.

__init__.py is empty and class Foo needs to import module1, module2 and module3 hence part of my Foo.py file looks like:

# foo/Foo.py

import module1
import module2
import module3

class Foo(object):
    def __init__(self):
....
....
if __name__ == '__main__':
    foo_obj = Foo()

However I later revisited this and I thought it would be better to have all imports in the __init__.py file. Hence my __init__.py now looks like:

# foo/__init__.py

import Foo
import module1
import module2
import module3
....
....

and my Foo.py only needs to import foo:

# foo/Foo.py

import foo

While this looks convenient since it is an one liner, I am a bit worried that it might be creating circular imports. What I mean is that when the script Foo.py is run it will import everything it can and then __init__.py will be called which will import Foo.py again (is that correct?). Additionally using the same name for package, module and class makes things more confusing.

Does it make sense the way I have done it? Or am I asking for trouble?

Answer 1

A couple things you could do to improve your organizaton, if only to adhere to some popular python conventions and standards.

If you search this topic, you will inevitably run across people recommending the PEP8 guidelines. These are the de facto canonical standards for organizing python code.

Modules should have short, all-lowercase names. Underscores can be used in the module name if it improves readability. Python packages should also have short, all-lowercase names, although the use of underscores is discouraged.

Based on these guidelines, your project modules should be named like this:

foo/
    __init__.py
    foo.py
    module1.py
    module2.py
    module3.py

I find it's generally best to avoid importing modules unnecessarily in __init__.py unless you're doing it for namespace reasons. For example, if you want the namespace for your package to look like this

from foo import Foo

instead of

from foo.foo import Foo

Then it makes sense to put

from .foo import Foo

in your __init__.py. As your package gets larger, some users may not want to use all of the sub-packages and modules, so it doesn't make sense to force the user to wait for all those modules to load by implicitly importing them in your __init__.py. Also, you have to consider whether you even want module1, module2, and module3 as part of your external API. Are they only used by Foo and not intended to be for end users? If they're only used internally, then don't include them in the __init__.py

I'd also recommend using absolute or explicit relative imports for importing sub-modules. For example, in foo.py

Absolute

from foo import module1
from foo import module2
from foo import module3

Explicit Relative

from . import module1
from . import module2
from . import module3

This will prevent any possible naming issues with other packages and modules. It will also make it easier if you decide to support Python3, since the implicit relative import syntax you're currently using is not supported in Python3.

Also, files inside your package generally shouldn't contain a

if __name__ == '__main__'

This is because running a file as a script means it won't be considered part of the package that it belongs to, so it won't be able to make relative imports.

The best way to provide executable scripts to users is by using the scripts or console_scripts feature of setuptools. The way you organize your scripts can be different depending on which method you use, but I generally organize mine like this:

foo/
    __init__.py
    foo.py
    ...
scripts/
     foo_script.py
setup.py