python - Avoiding long constructors while inheriting without hiding constructor, optional arguments or functionality

Question

I have a particular problem, but I will make the example more general. I have a Parent class with a mandatory constructor parameter and a few optional ones, each with a default value. Then, I inherit Child from it and add a mandatory parameter, and inherit GrandChild from Child and add another mandatory parameter to the constructor. The result is similar to this:

class Parent():

    def __init__(self, arg1, opt_arg1='opt_arg1_default_val', opt_arg2='opt_arg2_default_val',
                 opt_arg3='opt_arg3_default_val', opt_arg4='opt_arg4_default_val'):
        self.arg1 = arg1
        self.opt_arg1 = opt_arg1
        self.opt_arg2 = opt_arg2
        self.opt_arg3 = opt_arg3
        self.opt_arg4 = opt_arg4


class Child(Parent):
    def __init__(self, arg1, arg2, opt_arg1, opt_arg2, opt_arg3, opt_arg4):
        super().__init__(arg1, opt_arg1, opt_arg2, opt_arg3, opt_arg4)
        self.arg2 = arg2

class GrandChild(Child):
    def __init__(self, arg1, arg2, arg3, opt_arg1, opt_arg2, opt_arg3, opt_arg4):
        super().__init__(arg1, arg2, opt_arg1, opt_arg2, opt_arg3, opt_arg4)
        self.arg3 = arg3

The problem is that this looks rather ugly, especially if I want to inherit more classes from Child, I'd have to copy/paste all the arguments in that new class's constructor.

In search for a solution, I found here that I can solve this problem using **kwargs like so:

class Parent():

    def __init__(self, arg1, opt_arg1='opt_arg1_default_val', opt_arg2='opt_arg2_default_val',
                 opt_arg3='opt_arg3_default_val', opt_arg4='opt_arg4_default_val'):
        self.arg1 = arg1
        self.opt_arg1 = opt_arg1
        self.opt_arg2 = opt_arg2
        self.opt_arg3 = opt_arg3
        self.opt_arg4 = opt_arg4


class Child(Parent):
    def __init__(self, arg1, arg2, **kwargs):
        super().__init__(arg1, **kwargs)
        self.arg2 = arg2

class GrandChild(Child):
    def __init__(self, arg1, arg2, arg3,**kwargs):
        super().__init__(arg1, arg2,**kwargs)
        self.arg3 = arg3

However, I am not sure if this is the right way.

There is also a slight inconvenience while creating objects of these classes. I am using PyCharm to develop, and in this case the IDE has a useful method of displaying a function/class constructor arguments. For instance, in the first example,

This makes it much easier to develop and can help future developers as well since they can see what other arguments the function has. However, in the second example, the optional arguments are not shown anymore:

And I do not think it is a good practice to use **kwargs in this case, since one would have to dig deeper into the code up to the Parent class to check what optional arguments it has.

I've also looked into using the Builder pattern, but then all I do is move the arguments list from my classes to builder classes, and I have the same problem, builders with lots of arguments that when inherited will create even more arguments on top of the already existing ones. Also in Python, as much as I see, Builder doesn't really make much sense considering all class members are public and can be accessed without needing setters and getters.

Any ideas on how to solve this constructor problem?

Answer 1

The basic idea is to write code that generates the __init__ method for you, with all the parameters specified explicitly rather than via *args and/or **kwargs, and without even needing to repeat yourself with all those self.arg1 = arg1 lines.

And, ideally, it can make it easy to add type annotations that PyCharm can use for popup hints and/or static type checking.¹

And, while you're at it, why not build a __repr__ that displays the same values? And maybe even an __eq__, and a __hash__, and maybe lexicographical comparison operators, and conversion to and from a dict whose keys match the attributes for each JSON persistence, and…

Or, even better, use a library that takes care of that for you.

Python 3.7 comes with such a library, dataclasses. Or you can use a third-party library like attrs, that works with Python 3.4 and (with some limitations) 2.7. Or, for simple cases (where your objects are immutable, and you want them to work like a tuple of their attributes in specified order), you can use namedtuple, which works back to 3.0 and 2.6.

Unfortunately, dataclasses doesn't quite work for your use case. If you just write this:

from dataclasses import dataclass

@dataclass
class Parent:
    arg1: str
    opt_arg1: str = 'opt_arg1_default_val'
    opt_arg2: str = 'opt_arg2_default_val'
    opt_arg3: str = 'opt_arg3_default_val'
    opt_arg4: str = 'opt_arg4_default_val'

@dataclass
class Child(Parent):
    arg2: str

… you'll get an error, because it tries to place the mandatory parameter arg2 after the default-values parameters opt_arg1 through opt_arg4.

dataclasses doesn't have any way to reorder parameters (Child(arg1, arg2, opt_arg1=…), or to force them to be keyword-only parameters (Child(*, arg1, opt_arg1=…, arg2)). attrs doesn't have that functionality out of the box, but you can add it.

So, it's not quite as trivial as you'd hope, but it's doable.

---

But if you wanted to write this yourself, how would you create the __init__ function dynamically?

The simplest option is exec.

You've probably heard that exec is dangerous. But it's only dangerous if you're passing in values that came from your user. Here, you're only passing in values that came from your own source code.

It's still ugly—but sometimes it's the best answer anyway. The standard library's namedtuple used to be one giant exec template., and even the current version uses exec for most of the methods, and so does dataclasses.

Also, notice that all of these modules store the set of fields somewhere in a private class attribute, so subclasses can easily read the parent class's fields. If you didn't do that, you could use the inspect module to get the Signature for your base class's (or base classes', for multiple inheritance) initializer and work it out from there. But just using base._fields is obviously a lot simpler (and allows storing extra metadata that doesn't normally go in signatures).

Here's a dead simple implementation that doesn't handle most of the features of attrs or dataclasses, but does order all mandatory parameters before all optionals.

def makeinit(cls):
    fields = ()
    optfields = {}
    for base in cls.mro():
        fields = getattr(base, '_fields', ()) + fields
        optfields = {**getattr(base, '_optfields', {}), **optfields}
    optparams = [f"{name} = {val!r}" for name, val in optfields.items()]
    paramstr = ', '.join(['self', *fields, *optparams])
    assignstr = "
    ".join(f"self.{name} = {name}" for name in [*fields, *optfields])
    exec(f'def __init__({paramstr}):
    {assignstr}
cls.__init__ = __init__')
    return cls

@makeinit
class Parent:
    _fields = ('arg1',)
    _optfields = {'opt_arg1': 'opt_arg1_default_val',
                  'opt_arg2': 'opt_arg2_default_val',
                  'opt_arg3': 'opt_arg3_default_val',
                  'opt_arg4': 'opt_arg4_default_val'}

@makeinit
class Child(Parent):
    _fields = ('arg2',)

Now, you've got exactly the __init__ methods you wanted on Parent and Child, fully inspectable² (including help), and without having to repeat yourself.

---

_{1. I don't use PyCharm, but I know that well before 3.7 came out, their devs were involved in the discussion of @dataclass and were already working on adding explicit support for it to their IDE, so it doesn't even have to evaluate the class definition to get all that information. I don't know if it's available in the current version, but if not, I assume it will be. Meanwhile, @dataclass already just works for me with IPython auto-completion, emacs flycheck, and so on, which is good enough for me. :)}

_{2. … at least at runtime. PyCharm may not be able to figure things out statically well enough to do popup completion.}