S3Config

This configuration loads the configuration data from an AWS S3 config file.

Library boto3 is required (pip install boto3)

Central supports JSON, TOML, YAML and INI file formats.

TOML requires the external library toml and you must installed it by yourself (pip install toml)
YAML requires the external library PyYAML and you must installed it by yourself (pip install PyYAML)

The content of the file is determined by the file name extension.

# config.yaml

database:
    host: localhost
    port: 1234

# main.py

import boto3

from central.config.s3 import S3Config

s3 = boto3.resource('s3')

config = S3Config(s3, 'bucket name', 'config.yaml')
config.load()

print(config.get('database.host'))

python main.py
>> localhost

Using a custom reader

By default, S3Config finds the best reader for the file name using the file name extension, Central has readers for JSON, TOML, JSON and INI.

You can also specific your own reader or you can force one specific reader for a file name.

# app.cfg

database:
    host: localhost
    port: 1234

# main.py

from central.config.s3 import S3Config
from central.readers import YamlReader

s3 = boto3.resource('s3')

config = S3Config(s3, 'bucket name', 'app.cfg', reader=YamlReader())
config.load()

print(config.get('database.host'))

python main.py
>> localhost

Loading cascaded configuration files

A S3Config supports multiple configuration files, that is also called cascaded configuration files.

For example, you want Central to load an environment specific configuration file config.prod.json to override the default config.json. To do that, you can define this key at the end of the config.json file:

{
    "@next": "config.${ENV}.json"
}

@next is a special key to define next file to load. ENV is a variable that will expand to value returned from environment variables.

Full example:

# config.json

{
    "database": {
        "host": "localhost",
        "port": 1234
    },
    "@next": "config.${ENV}.json"
}

# config.prod.json

{
    "database": {
        "host": "prod.db"
    }
}

# main.py

import boto3

from central.config.s3 import S3Config

s3 = boto3.resource('s3')

config = S3Config(s3, 'bucket name', 'config.json')
config.load()

print(config.get('database.host'))
print(config.get('database.port'))

export ENV=prod
python main.py
>> prod.db
>> 1234

Reloading configuration

Amazon S3 does not support update notification so if your application needs to be notified when a configuration has been changed it can be archived using polling.

from central.config.s3 import S3Config

s3 = boto3.resource('s3')

config = S3Config(s3, 'bucket name', 'config.json').reload_every(600) # 10 min in seconds
config.load()