Python: Accessing YAML values using "dot notation"

Question

I'm using a YAML configuration file. So this is the code to load my config in Python:

import os
import yaml
with open('./config.yml') as file:
    config = yaml.safe_load(file)

This code actually creates a dictionary. Now the problem is that in order to access the values I need to use tons of brackets.

YAML:

mysql:
    user:
        pass: secret

Python:

import os
import yaml
with open('./config.yml') as file:
    config = yaml.safe_load(file)
print(config['mysql']['user']['pass']) # <--

I'd prefer something like that (dot notation):

config('mysql.user.pass')

So, my idea is to utilize the PyStache render() interface.

import os
import yaml
with open('./config.yml') as file:
    config = yaml.safe_load(file)

import pystache
def get_config_value( yml_path, config ):
    return pystache.render('{{' + yml_path + '}}', config)

get_config_value('mysql.user.pass', config)

Would that be a "good" solution? If not, what would be a better alternative?

Additional question [Solved]

I've decided to use Ilja Everil?'s solution. But now I've got an additional question: How would you create a wrapper Config class around DotConf?

The following code doesn't work but I hope you get the idea what I'm trying to do:

class Config( DotDict ):
    def __init__( self ):
        with open('./config.yml') as file:
            DotDict.__init__(yaml.safe_load(file))

config = Config()
print(config.django.admin.user)

Error:

AttributeError: 'super' object has no attribute '__getattr__'

Solution

You just need to pass self to the constructor of the super class.

DotDict.__init__(self, yaml.safe_load(file))

Even better soltution (Ilja Everil?)

super().__init__(yaml.safe_load(file))

Answer 1

The Simple

You could use reduce to extract the value from the config:

In [41]: config = {'asdf': {'asdf': {'qwer': 1}}}

In [42]: from functools import reduce
    ...: 
    ...: def get_config_value(key, cfg):
    ...:     return reduce(lambda c, k: c[k], key.split('.'), cfg)
    ...: 

In [43]: get_config_value('asdf.asdf.qwer', config)
Out[43]: 1

This solution is easy to maintain and has very few new edge cases, if your YAML uses a very limited subset of the language.

The Correct

Use a proper YAML parser and tools, such as in this answer.

---

The Convoluted

On a lighter note (not to be taken too seriously), you could create a wrapper that allows using attribute access:

In [47]: class DotConfig:
    ...:     
    ...:     def __init__(self, cfg):
    ...:         self._cfg = cfg
    ...:     def __getattr__(self, k):
    ...:         v = self._cfg[k]
    ...:         if isinstance(v, dict):
    ...:             return DotConfig(v)
    ...:         return v
    ...:     

In [48]: DotConfig(config).asdf.asdf.qwer
Out[48]: 1

Do note that this fails for keywords, such as "as", "pass", "if" and the like.

Finally, you could get really crazy (read: probably not a good idea) and customize dict to handle dotted string and tuple keys as a special case, with attribute access to items thrown in the mix (with its limitations):

In [58]: class DotDict(dict):
    ...:     
    ...:     # update, __setitem__ etc. omitted, but required if
    ...:     # one tries to set items using dot notation. Essentially
    ...:     # this is a read-only view.
    ...:
    ...:     def __getattr__(self, k):
    ...:         try:
    ...:             v = self[k]
    ...:         except KeyError:
    ...:             return super().__getattr__(k)
    ...:         if isinstance(v, dict):
    ...:             return DotDict(v)
    ...:         return v
    ...:
    ...:     def __getitem__(self, k):
    ...:         if isinstance(k, str) and '.' in k:
    ...:             k = k.split('.')
    ...:         if isinstance(k, (list, tuple)):
    ...:             return reduce(lambda d, kk: d[kk], k, self)
    ...:         return super().__getitem__(k)
    ...:
    ...:     def get(self, k, default=None):
    ...:         if isinstance(k, str) and '.' in k:
    ...:             try:
    ...:                 return self[k]
    ...:             except KeyError:
    ...:                 return default
    ...:         return super().get(k, default=default)
    ...:     

In [59]: dotconf = DotDict(config)

In [60]: dotconf['asdf.asdf.qwer']
Out[60]: 1

In [61]: dotconf['asdf', 'asdf', 'qwer']
Out[61]: 1

In [62]: dotconf.asdf.asdf.qwer
Out[62]: 1

In [63]: dotconf.get('asdf.asdf.qwer')
Out[63]: 1

In [64]: dotconf.get('asdf.asdf.asdf')

In [65]: dotconf.get('asdf.asdf.asdf', 'Nope')
Out[65]: 'Nope'