DUFT Architecture#
DUFT follows a modular, configuration-driven architecture, designed to keep the backend flexible while allowing full customisation through external configuration files. The system consists of three key components:
DUFT Server – A Django/Python-based backend responsible for executing queries, running ETL tasks, handling authentication, and serving API endpoints for the React frontend.
DUFT Config – A structured collection of files that defines dashboards, queries, scheduled data tasks, and navigation. This allows system behaviour to be updated without modifying the server.
DUFT UI – A React-based frontend that dynamically renders dashboards and navigation elements based on DUFT Config.
By separating backend logic, configuration, and UI, DUFT enables a highly adaptable and extensible platform where new dashboards, workflows, and data tasks can be introduced without redeploying the backend.
Engineering Decisions: DUFT Architecture#
DUFT Architecture
Several key engineering decisions shaped the design of DUFT’s architecture:
DUFT Server is generic and configuration-driven to allow dynamic updates without requiring code changes. This ensures that dashboards, workflows, and queries can be modified purely through DUFT Config, making the system highly adaptable.
The architecture follows an app-based structure within DUFT Config, grouping related dashboards, queries, and ETL tasks. This design improves scalability by keeping configurations organised and modular by thematic areas.
The system is designed to be extensible, with planned enhancements such as dynamic role management and support for multiple configuration sets. By avoiding rigid structures, DUFT can evolve to support new governance models and security policies.
Maintenance overhead is minimised by keeping backend logic decoupled from configuration. Since core functionality is defined externally, there is less need for direct backend modifications, reducing deployment complexity and ensuring long-term sustainability.
Key DUFT Concepts#
The three components (DUFT Server, DUFT UI and DUFT Config) collectively implement the following key concepts that DUFT is built on. They are the critical pieces of DUFT, and need to be understood — whether designing dashboards, maintaining the DUFT codebase, or working with data. They are, in no particular order:
3DL – a language to define dashboards, layouts, and data sources declaratively, allowing dynamic dashboard rendering without modifying the frontend.
DUFT Apps & Navigation – a modular system that groups dashboards, queries, and tasks into distinct applications, ensuring scalability and maintainability.
Data Connections – a configuration layer that defines database connections and API endpoints, allowing queries and Data Tasks to dynamically fetch credentials and interact with multiple data sources.
Data Tasks – a mechanism for running scheduled and on-demand ETL processes, data updates, and integrations using Python scripts or Jupyter notebooks.
Data Uploads – a batch-based file transfer system that securely submits structured data from local DUFT Servers to central DUFT Servers, ensuring efficient, reliable, and trackable data transmission.
Who is DUFT built for?#
DUFT is designed for anyone who transforms, analyses, and visualises data—from health facilities to national data repositories. It is lightweight enough to be deployed at individual facilities yet powerful enough to serve as a frontend for large-scale data systems.
Common use cases include:
Integrating data from multiple health facility systems into a unified dataset.
Visualising key insights through interactive, action-driven dashboards.
Uploading local facility data to national repositories.
Downloading and integrating upstream data into local databases.
Remotely updating DUFT’s data transformation scripts and dashboards from a central location.
DUFT Users#
DUFT users generally fall into two categories:
Implementers – Organisations such as Ministries of Health, implementing partners, or technical teams that customise and deploy DUFT on behalf of end users. They configure dashboards, queries, and workflows.
End Users – Individuals or teams who interact with DUFT’s dashboards and data outputs. They rely on insights generated by DUFT but do not configure the system themselves.
For small organisations or independent users, one person may take on both roles, handling both configuration and data consumption.
DUFT File Structure#
The directory tree below shows how the file structure of DUFT Config. It is meant to be at the same level as duft-server, which holds the Dangjo web application.
duft/
│── duft-config/ # Configuration-driven control layer
│ │── apps/ # Modular apps, each in its own directory
│ │ │── app_1/
│ │ │ │── settings.json # App metadata
│ │ │ │── navigation.json # Menu structure for the app
│ │ │ └── dashboards/ # 3DL dashboard definitions for the app
│ │ │ └── sql/ # SQL queries for the app
│ │ │── app_2/ # Another app (structured similarly)
│ │
│ │── system/ # Used when running DUFT as single app
│ │ │── navigation.json # Contains the navigation structure
│ │ │── theme.json # Contains visual theme colours
│ │ │── data_connections.json # Specifies available data connections
│ │ │── data_tasks.json # Specifies data tasks
│ │ │── assets/ # Images and static files
│ │ │── data_tasks/ # Data Task Python/Jupyter scripts
│ │ └── dashboards/ # Dashboard 3DL files
│ │ └── sql/ # Stored SQL queries for dashboards
│ │
│ │── user/ # Installation-specific settings
│ │ └── config.json # Stores passwords and sensitive values
│ │
│ │── settings.json # Implementer-configurable settings
│ │── startup.py # Executed during DUFT Server start
│ │── .env # Implementer-overridable feature flags
│
│── duft-server/ # DUFT Server core (Django backend)
│ ├── app/ # Holds the Django App
│ ├── services/ # Core processing services
│ ├── react_app/ # Serves the React App
│ ├── management/ # Management commands
│ │ ├── commands/ # Custom CLI commands
│ │ └── first_run.py # Runs setup on first launch
│ ├── startup/ # Startup routines & initialisation
│ │ ├── initial_load.py # Pre-loads settings and permissions
│ │ └── fixtures/ # Default system data
│ ├── settings.py # Django project settings
│ ├── urls.py # Routing configuration
│ ├── wsgi.py # WSGI application entry point
│ ├── manage.py # Django management script
│ ├── .env # Server environment variables
│ └── requirements.txt # Python dependencies
How DUFT Server and DUFT Config Interact#
DUFT Server and DUFT Config work together to keep configuration external while processing and execution happen within the backend. This separation ensures flexibility, allowing updates to dashboards, queries, and workflows without modifying the server.
The interaction follows a structured flow:
DUFT Server loads DUFT Config at startup and continuously monitors for changes.
DUFT UI fetches configurations (navigation, dashboards, queries) from DUFT Server and dynamically renders them.
DUFT Server executes queries, retrieving data from the appropriate database and returning results as structured JSON.
DUFT UI visualises the data in real time, allowing users to interact with dashboards without requiring manual updates.
Users trigger new queries or execute Data Tasks, enabling transformations, aggregations, and scheduled processing.
Data Tasks run Python scripts for ETL operations, keeping data fresh and ready for analysis.