S3Config
This configuration loads the configuration data from an AWS S3 config file.
Library boto3 is required (pip install boto3)
Central supports JSON, TOML, YAML and INI file formats.
TOML requires the external library toml and you must installed it by yourself (pip install toml)
YAML requires the external library PyYAML and you must installed it by yourself (pip install PyYAML)
The content of the file is determined by the file name extension.
# config.yaml
database:
host: localhost
port: 1234
# main.py
import boto3
from central.config.s3 import S3Config
s3 = boto3.resource('s3')
config = S3Config(s3, 'bucket name', 'config.yaml')
config.load()
print(config.get('database.host'))
python main.py
>> localhost
Using a custom reader
By default, S3Config
finds the best reader for the file name using the file name extension, Central has readers for JSON, TOML, JSON and INI.
You can also specific your own reader or you can force one specific reader for a file name.
# app.cfg
database:
host: localhost
port: 1234
# main.py
from central.config.s3 import S3Config
from central.readers import YamlReader
s3 = boto3.resource('s3')
config = S3Config(s3, 'bucket name', 'app.cfg', reader=YamlReader())
config.load()
print(config.get('database.host'))
python main.py
>> localhost
Loading cascaded configuration files
A S3Config
supports multiple configuration files, that is also called cascaded configuration files.
For example, you want Central to load an environment specific configuration file config.prod.json to override the default config.json. To do that, you can define this key at the end of the config.json file:
{
"@next": "config.${ENV}.json"
}
@next
is a special key to define next file to load. ENV
is a variable that will expand to value returned from environment variables.
Full example:
# config.json
{
"database": {
"host": "localhost",
"port": 1234
},
"@next": "config.${ENV}.json"
}
# config.prod.json
{
"database": {
"host": "prod.db"
}
}
# main.py
import boto3
from central.config.s3 import S3Config
s3 = boto3.resource('s3')
config = S3Config(s3, 'bucket name', 'config.json')
config.load()
print(config.get('database.host'))
print(config.get('database.port'))
export ENV=prod
python main.py
>> prod.db
>> 1234
Reloading configuration
Amazon S3 does not support update notification so if your application needs to be notified when a configuration has been changed it can be archived using polling.
from central.config.s3 import S3Config
s3 = boto3.resource('s3')
config = S3Config(s3, 'bucket name', 'config.json').reload_every(600) # 10 min in seconds
config.load()