Configuring DUFT Server

Configuring DUFT Server#

DUFT Server has several configuration options, controlling its behaviour. Most are set via environment variables, or through any of the config files.

Note

DUFT Server is written in Django, which has its own settings.py file. This file should not be used for configuring DUFT Server, as it is used internally to control the different packages DUFT Server uses, but it has no options for controlling DUFT Server itself.

Features vs User Settings#

Features control which part of DUFT are enabled, and how the should behave, for example, the ability to upload files, where the files should be stored, or the title that DUFT should display in the user interface. These settings are not configurable or meant to be configured by end users, but rather the developers that put a DUFT Package together.
User Settings control parts that are controlled by end users, for example, connections to analysis databases.

Examples of Features#

Features can control:

Whether DUFT Server can run data tasks or not
Whether DUFT Server should use authentication or not
Whether DUFT Server accepts file uploads
Whether DUFT Server should run scheduled tasks
The title of DUFT UI, the navigation bar titles, and footer
The location of data and upload directories
The name of the duft-config directory
More

Examples of User Settings#

User Settings can control:

The connection settings for data connections
Which data tasks should be run on a schedule

Configurable Features#

DUFT_CONFIG (environment variable): The name of the config directory#

The config directory is the directory containing the dashboards, queries, data connections and other assets that define the user experience of DUFT. It is the part of DUFT that is customisable by developers for end users.

The config directory contains two sub directories:

system which contains assets defined by the team preparing DUFT.
user which contains settings for an individual user, and may be customised for an individual user.

By default, the name of the duft-config directory is duft-config. If you wish to change this, you can set the DUFT_CONFIG environment variable to the name of the duft-config directory. In most cases, you do not want to change this name. Regardless of the name, the config dirctory must be at the same level as duft-server.

Feature Flags: controlling specific features#

Feature Flags control behaviour of DUFT Server. Features can be turned on or off. They must all be set as environment variables, or through a .env file.

The following feature flags are supported:

FEATURE_DATA_TASKS=True|False
FEATURE_USER_AUTHENTICATION=True|False
FEATURE_SERVER_UPLOADS=True|False
FEATURE_TASK_SCHEDULER=True|False
DUFT_UPLOADS=uploads
DUFT_DATA=data
LOG_LEVEL=DEBUG|INFO|WARNING

Feature	Description
`FEATURE_DATA_TASKS`	Controls whether data tasks are supported for this instance or not. If data tasks are not supported, APIs controlling data tasks will not be available. However, data tasks still run on a schedule, if the scheduler is enabled. Can be `True` or `False`.
`FEATURE_USER_AUTHENTICATION`	Controls whether user authentication is enabled for this instance or not. If user authentication is enabled, most DUFT features require a security token, which must be obtained by logging in as a user. Some features will also require additional persmissions. This is recommended for scenarios where DUFT Server is used by more than one user. If UA is not enabled, no authentication checks will be performed, and the user can do anything. This is recommended for stand-alone scenarios. Can be `True` or `False`.
`FEATURE_SERVER_UPLOADS`	Controls whether the server accepts file uploads, which is useful for settings where facilities upload files to a central repository. When this flag is enabled, you must also enable user authentication, otherwise this flag won’t have any effect. Can be `True` or `False`.
`DUFT_UPLOADS`	Controls the name of the directory where the files are to be uploaded to by setting `DUFT_UPLOADS`, this directory will always be at the same level as `duft-server`. The default (when not set) is `uploads`.
`FEATURE_TASK_SCHEDULER`	Controls whether the task scheduler is enabled or not. The task scheduler allows you to run data tasks at certain intervals, for example, every day at 22:00. If this feature is enabled, DUFT Server will schedule the data tasks. Tasks should be scheduled in `duft-config/user/config.json` Can be `True` or `False`.
`DUFT_DATA`	Sets the name of the data directory, which is used by the logger. The directory will be created at the same level as `duft-server`. The default (when not set) is `data`.
`LOG_LEVEL`	DUFT maintains a log file, containing info, error, warning and debug messages. You can specify the level of messages you would like to see. Can be `INFO`, `WARNING`, `ERROR` or `DEBUG`. In most cases, `INFO` is sufficient. If you are debugging queries or data tasks, the `DEBUG` level may be helpful as it produces more frequent output messages. The log file is `app.log` and is located in `duft-server`.

The Task Scheduler#

Tasks should be scheduled in duft-config/user/config.json, for example as follows:

 "scheduledDataTasks": {
    "SAMPLE": {
      "description": "Scheduled sample task",
      "schedule": {
        "cron": {
          "second": "00",
          "minute": "00",
          "hour": "22",
          "day_of_month": "*",
          "month": "*",
          "day_of_week": "*"
        }
      }
    }
  }

In this case, SAMPLE refers to the ID of the data task as configured in duft-config/system/data_tasks.json.

Authentication#

If you enable authentication, you must enable some other settings too. See User Authentication in DUFT.

How to configure features#

Features are set in .env files. There are two places where these files can be set.

Place	Description
`workspace`	This is the directory above `duft-server`. DUFT Server looks for an .env file here, and when it finds it, applies the settings.
`duft-config`	If a .env file exists in duft-config, it is also loaded, and settings are applied onto the file found in the `workspace`. So settings defined in the `workspace` but not explicitely overriden in `duft-config` remain active.

Note

You cannot set DUFT_CONFIG from within an .env file in the duft-config directory, as DUFT Server has already resolved the config directory at that time.

How it works#

The .env files are processed by duft-server/app/__init__.py.

import logging
import os
from pathlib import Path

from dotenv import load_dotenv


logger = logging.getLogger(__name__)
logger.info(f"Current working directory: {os.getcwd()}")

# Load from local directories first
env_path = Path(".") / ".env"
  
if not env_path.exists():
  env_path = Path("..") / ".env"

if env_path.exists():
  load_dotenv(dotenv_path=env_path, override=True)
  logger.info(f".env loaded from: {env_path.resolve()}")
 

# Load from config directory last (takes precedence)
config_env_path = Path(Environment().directories.config_directory) / ".env"
if config_env_path.exists():
  load_dotenv(dotenv_path=config_env_path, override=True)
  logger.info(f".env loaded from: {config_env_path}")

Sample .env file#

You can use the following file to get started, .env files are typically not included in DUFT repos as they may be listed in .gitignore. It is recommend to not place .env files in config repos in .gitignore files though.

FEATURE_DATA_TASKS=True
FEATURE_USER_AUTHENTICATION=False
FEATURE_SERVER_UPLOADS=False
FEATURE_TASK_SCHEDULER=False
DUFT_UPLOADS=uploads
DUFT_DATA=data
DUFT_CONFIG=duft-config