Using Configuration Files

Using Configuration Files#

This is part of the new environment system. This chapter is mostly useful for those who are maintaining or extending DUFT with new features, such as services.

Overview#

The new DUFT configuration system is designed to provide a flexible and robust way to manage configuration settings for both system and user-specific configurations. It leverages JSON files to store configuration data and provides utility functions to read and write these configurations.

Task

Use

Get config diectories

Use directories after importing it from from services.config.service

Get the name of a config file

Use the ConfigFileName enum

Get a specific config file, or the contents of a specific config file

Any of the functions in the config_file module:

get_json_file_as_data get_config_file get_config_item_by_id

Update settings/parameters in the user’s config file (in the user directory)

Use any of the functions in the user_config module

get_user_config_file get_user_parameters_for_data_connection set_user_parameters_for_data_connection get_user_config_settings_for_section set_user_config_settings_for_section

Configuration Files#

The configuration system uses JSON files to store settings. There are two main types of configuration files:

  1. System Configuration Files: These files are predefined and managed by the system. They contain default settings and configurations that apply to all users.

  2. User Configuration Files: These files are user-specific and can be customised by the user. They allow users to override system settings and provide their own configurations.

In most cases, configuration files get merged by the API. For example, there is a navigation.json file for both the system and the user. DUFT merges these two files into a single navigation tree.

Services provided by the Config system#

Directory Structure#

The configuration files are organized into directories to separate system and user configurations:

  • system_config_directory: Contains system configuration files.

  • user_config_directory: Contains user configuration files.

Config directories are accessed through the global directory class:

from services.config.service.config_directories import directories

user_config_file_path = "%s/%s" % (directories.user_config_directory,ConfigFileName.user_config.value,)

# Directories defines the following:

root_path: str
config_directory: str
user_config_directory: str
system_config_directory: str
system_data_tasks_directory: str
user_data_tasks_directory: str
system_dashboards_directory: str
system_sql_directory: str
user_dashboards_directory: str
data_directory: str
upload_directory: str
data_tasks_temp_directory: str
assets_directory: str

It is important to understand what sets the “root” directories:

def get_directories(
  working_directory: str  
  config_directory_name: str = os.getenv("DUFT_CONFIG", "duft-config"),
  data_directory_name: str = os.getenv("DUFT_DATA", "data"),
  upload_directory_name: str = os.getenv("DUFT_UPLOADS", "uploads"),
  data_tasks_temp_directory_name: str = os.getenv("DUFT_DATATASKS_TEMP_DIR", "tmp-data-tasks")
  ) -> EnvironmentDirectories:

    config_directory = "%s/%s" % (working_directory, config_directory_name)
    data_directory = "%s/%s" % (working_directory, data_directory_name)
    upload_directory = "%s/%s" % (working_directory, upload_directory_name)
    data_tasks_temp_directory = "%s/%s" % (
        working_directory,
        data_tasks_temp_directory_name,
    )

Key Components#

ConfigFileNames Enum#

The ConfigFileNames enum defines the various configuration files used in the system. Each enum value represents a specific configuration file, such as data_connections for data connection settings or user_config for user-specific settings.

class ConfigFileNames(Enum):
    ai_models = "ai_models.json"
    proxy_apis = "proxy_apis.json"
    data_connections = "data_connections.json"
    data_tasks = "data_tasks.json"
    user_config = "config.json"
    test = "test.json"
    navigation = "navigation.json"
    theme = "theme.json"
    settings = "settings.json"
    test_config = "test_config.json"

Config File Helpers#

To access config files, use several utility helpers that are defined in config_file.py. They allow you to extract relevant sections out of config files. For example:

from services.config.service.user_config import get_user_config_file

data_connection = config_file.get_config_item_by_id(
  ConfigFileNames.data_connections, 
  ConfigFileNames.data_connections, 
  "dataConnections", 
  "ANA"

)

This extracts an item called ANA out of the data_connections.json file’s dataConnections section. It searches both the system and the user directories, and returns the first one found. The file is structured as follows:

{
 "dataConnections": 
 [
  {
   "id":"ANA",
   ...

For this to work, it is important to keep files structures the same way (section, id).

Example Usage#

Retrieving a Data Connection#

To retrieve a data connection with resolved parameters, you can use the following code:

from services.config.service.data_connections import get_data_connection

data_connection = get_data_connection(by_id="my_connection_id", with_resolved_parameters=True)
print(data_connection)

This code retrieves the data connection with the specified ID and includes any resolved parameters from the user’s configuration file.

Setting User-Specific Parameters#

To set user-specific parameters for a data connection, you can use the following code:

from services.config.service.user_config import set_user_parameters_for_data_connection

parameters = {"username": "user", "password": "pass"}
set_user_parameters_for_data_connection("my_connection_id", parameters)

This code sets the specified parameters for the data connection with the given ID in the user’s configuration file.

Contents