python - When should I subclass EnumMeta instead of Enum?

Question

In this article Nick Coghlan talks about some of the design decisions that went in to the PEP 435 Enum type, and how EnumMeta can be subclassed to provide a different Enum experience.

However, the advice I give (and I am the primary stdlib Enum author) about using a metaclass is it should not be done without a really good reason -- such as not being able to accomplish what you need with a class decorator, or a dedicated function to hide any ugliness; and in my own work I've been able to do whatever I needed simply by using __new__, __init__, and/or normal class/instance methods when creating the Enum class:

• Enum with attributes

• Handling missing members

• class constants that are not Enum members

And then there is this cautionary tale of being careful when delving into Enum, with and without metaclass subclassing:

• Is it possible to override __new__ in an enum to parse strings to an instance?

Given all that, when would I need to fiddle with EnumMeta itself?

Answer 1

The best (and only) cases I have seen so far for subclassing EnumMeta comes from these four questions:

• A more pythonic way to define an enum with dynamic members

• Prevent invalid enum attribute assignment

• Create an abstract Enum class

• Invoke a function when an enum member is accessed

We'll examine the dynamic member case further here.

---

First, a look at the code needed when not subclassing EnumMeta:

The stdlib way

from enum import Enum
import json

class BaseCountry(Enum):
    def __new__(cls, record):
        member = object.__new__(cls)
        member.country_name = record['name']
        member.code = int(record['country-code'])
        member.abbr = record['alpha-2']
        member._value_ = member.abbr, member.code, member.country_name
        if not hasattr(cls, '_choices'):
            cls._choices = {}
        cls._choices[member.code] = member.country_name
        cls._choices[member.abbr] = member.country_name
        return member                
    def __str__(self):
        return self.country_name

Country = BaseCountry(
        'Country',
        [(rec['alpha-2'], rec) for rec in json.load(open('slim-2.json'))],
        )

The aenum way ^{1 2}

from aenum import Enum, MultiValue
import json

class Country(Enum, init='abbr code country_name', settings=MultiValue):
    _ignore_ = 'country this'  # do not add these names as members
    # create members
    this = vars()
    for country in json.load(open('slim-2.json')):
        this[country['alpha-2']] = (
                country['alpha-2'],
                int(country['country-code']),
                country['name'],
                )
    # have str() print just the country name
    def __str__(self):
        return self.country_name

---

The above code is fine for a one-off enumeration -- but what if creating Enums from JSON files was common for you? Imagine if you could do this instead:

class Country(JSONEnum):
    _init_ = 'abbr code country_name'  # remove if not using aenum
    _file = 'some_file.json'
    _name = 'alpha-2'
    _value = {
            1: ('alpha-2', None),
            2: ('country-code', lambda c: int(c)),
            3: ('name', None),
            }

As you can see:

• _file is the name of the json file to use
• _name is the path to whatever should be used for the name
• _value is a dictionary mapping paths to values³
• _init_ specifies the attribute names for the different value components (if using aenum)

The JSON data is taken from https://github.com/lukes/ISO-3166-Countries-with-Regional-Codes -- here is a short excerpt:

[{"name":"Afghanistan","alpha-2":"AF","country-code":"004"},

{"name":"?land Islands","alpha-2":"AX","country-code":"248"},

{"name":"Albania","alpha-2":"AL","country-code":"008"},

{"name":"Algeria","alpha-2":"DZ","country-code":"012"}]

Here is the JSONEnumMeta class:

class JSONEnumMeta(EnumMeta):

    @classmethod
    def __prepare__(metacls, cls, bases, **kwds):
        # return a standard dictionary for the initial processing
        return {}

    def __init__(cls, *args , **kwds):
        super(JSONEnumMeta, cls).__init__(*args)

    def __new__(metacls, cls, bases, clsdict, **kwds):
        import json
        members = []
        missing = [
               name
               for name in ('_file', '_name', '_value')
               if name not in clsdict
               ]
        if len(missing) in (1, 2):
            # all three must be present or absent
            raise TypeError('missing required settings: %r' % (missing, ))
        if not missing:
            # process
            name_spec = clsdict.pop('_name')
            if not isinstance(name_spec, (tuple, list)):
                name_spec = (name_spec, )
            value_spec = clsdict.pop('_value')
            file = clsdict.pop('_file')
            with open(file) as f:
                json_data = json.load(f)
            for data in json_data:
                values = []
                name = data[name_spec[0]]
                for piece in name_spec[1:]:
                    name = name[piece]
                for order, (value_path, func) in sorted(value_spec.items()):
                    if not isinstance(value_path, (list, tuple)):
                        value_path = (value_path, )
                    value = data[value_path[0]]
                    for piece in value_path[1:]:
                        value = value[piece]
                    if func is not None:
                        value = func(value)
                    values.append(value)
                values = tuple(values)
                members.append(
                    (name, values)
                    )
        # get the real EnumDict
        enum_dict = super(JSONEnumMeta, metacls).__prepare__(cls, bases, **kwds)
        # transfer the original dict content, _items first
        items = list(clsdict.items())
        items.sort(key=lambda p: (0 if p[0][0] == '_' else 1, p))
        for name, value in items:
            enum_dict[name] = value
        # add the members
        for name, value in members:
            enum_dict[name] = value
        return super(JSONEnumMeta, metacls).__new__(metacls, cls, bases, enum_dict, **kwds)

# for use with both Python 2/3
JSONEnum = JSONEnumMeta('JsonEnum', (Enum, ), {})

A few notes:

• JSONEnumMeta.__prepare__ returns a normal dict

• EnumMeta.__prepare__ is used to get an instance of _EnumDict -- this is the proper way to get one

• keys with a leading underscore are passed to the real _EnumDict first as they may be needed when processing the enum members

• Enum members are in the same order as they were in the file

---

¹ Disclosure: I am the author of the Python stdlib Enum, the enum34 backport, and the Advanced Enumeration (aenum) library.

² This requires aenum 2.0.5+.

³ The keys are numeric to keep multiple values in order should your Enum need more than one.