DUFT Config: The Control Layer

DUFT Config: The Control Layer#

DUFT Config defines the system’s behaviour through a structured set of plain files, making it an independent and version-controlled configuration layer. Since DUFT Server does not store dashboards, queries, or navigation structures internally, everything that appears in DUFT UI is dictated by DUFT Config. This approach allows users to modify dashboards, update queries, and define new data processing tasks without modifying the backend.

Engineering Decisions: DUFT Config

  • Configuration over Code by not embedding dashboards, queries, or workflows directly in the server or frontend, but using configuration files instead. This ensures that system behaviour can be modified dynamically without requiring code changes or redeployment, reducing maintenance overhead and increasing flexibility.

  • Plain Text, Git-Compatible Formats as all configurations are stored as simple, human-readable text files (JSON, SQL, Python, Jupyter notebooks, and 3DL). This allows DUFT Config to be version-controlled using Git, making updates trackable, reversible, and auditable—just like software code.

  • Declarative UI through 3DL, a declarative syntax resembling XML or JSX. This improves readability, enables hierarchical structuring of dashboards and UI elements, and keeps the syntax intuitive for developers familiar with front-end technologies. It also reduces data mistakes by ensuring structured, predictable configurations.

  • Decoupling from DUFT Server by making DUFT Config completely independent of DUFT Server so that updates can be pushed to production without restarting or redeploying the backend. This design ensures seamless evolution of dashboards, queries, and workflows while keeping the core server lightweight.

  • Support for Modular Apps to handle complex configurations at scale. DUFT Config introduces the concept of “Apps”—self-contained modules that group related dashboards, queries, and ETL tasks. This structure prevents configuration bloat, improves maintainability, and allows users to switch between different data contexts easily.

  • Separation of Concerns Between Data Processing and UI whereby Queries, ETL tasks, and data transformation logic are defined in DUFT Config rather than being hardcoded in the UI or backend. This keeps DUFT Server focused on execution while ensuring that the UI dynamically adapts to configuration changes without requiring manual updates.

DUFT Config is Git-able

DUFT Config consists entirely of plain-text files, including JSON, SQL, Python scripts, Jupyter notebooks, and 3DL (DUFT Dashboard Definition Language). 3DL, a core part of DUFT, provides a declarative, human-readable syntax resembling XML or JSX, making it easier to compose UI elements and dashboards hierarchically.

Since DUFT Config is stored as files, it can be versioned and managed using Git or any other version control system. This allows teams to track changes, collaborate efficiently, and ensure safe rollbacks when needed. Every update—whether adding a new dashboard, refining a query, or modifying an ETL script—can be committed and reviewed just like software code.

Because DUFT Config is fully decoupled from DUFT Server, updates can be applied dynamically without redeploying the backend. New dashboards, reports, and workflows can be introduced simply by distributing updated configuration files. This approach enables rapid iteration, reduces deployment overhead, and ensures system evolution remains flexible and responsive to user needs.

What DUFT Config Defines

DUFT Config serves as the system’s control layer, defining:

  • Dashboards – Specified using 3DL, a declarative format for UI components.

  • Queries – SQL scripts that power dashboard visualisations.

  • Data Tasks – Python or Jupyter-based ETL scripts for data transformation.

  • Navigation – The menu structure linking dashboards, data tasks, and other system components.

  • Role-based Access (Future Feature) – Custom roles and permissions integrated with Django Auth.

Organising DUFT Config into Apps

To improve scalability and organisation, DUFT structures configurations into apps—self-contained modules that group related dashboards, queries, and ETL tasks. This approach allows for logical separation of different reporting and analysis areas. For example:

  • CQI Dashboards App – Dashboards and queries specific to Continuous Quality Improvement (CQI).

  • Monthly Reports App – Handles automated report generation and scheduled data pulls.