Skip to content

DeMi - ATLAS ITk Pixel System Test Microservices

Introduction

The itk-demo-sw GitLab group contains the evolving high-level online software for ITk pixel + strips system tests at CERN and other ITk production and testing sites, also known as DeMi (Detector Microservices).

The general architecture is based on containerized microservices, adapted to the specific needs of the ATLAS ITk system test community1.

Status of adoption in ATLAS.

DeMi has been adopted as top-level software for the qualification of the Loaded Local Supports for the ITk Pixel Outer Barrel as defined in the FDR EDMS 2822664. Adoption for Integration is foreseen as well (see EDMS here and here).

The software integrates tools for configuration, operation, results inspection and analysis into a unified web interface. (Planned) servers and backend interfaces include a distributed controller, a connectivity, configuration and results store, and of course DAQ hardware and software (scan engines) interfaces.

Getting Started

Prerequisites

To get started the only real prerequisite is a working Docker installation.

Install Docker >= 23.0.4 and the docker compose >= v2.17.2 plugin.

Due to heavy containerization the use on any platform that support OCI runtimes such as Docker should be possible. We have not yet extensively tried another platform such as podman or moby, and users are asked to contribute corresponding experience back to DeMi.

  • Initial root access is required to install Docker.
  • In case multiple nodes are planned for deployment passwordless ssh access needs to be set up between them.
  • Further optional native dependencies are needed if trying to run natively instead of using containers or development containers are not used.

To get started, either individual microservices can be run, or a stack of microservices containers can be deployed.

Microservices

A list of available microservices is here.

Setup of the microservices

The best way to get started is by cloning on of the existent deployment repositories. More details on these deployments and their usage is available here. These deployments can than be changed to your specific needs by adding additional microservices or other containers.

Tutorial slides on how to use these deployment directories to setup the optoboard-stack (including a general introduction to docker) can be found here.

If you just want to test a singular microservice most of them include an example directory which offers similar functionality as these deployment repos.

FAQ

A list of frequently asked questions including links on how to best contact us in case of further questions / problems is available here.

For Developers

Every repository in the DeMi project has a standard structure which should make it easy to get started working with each repository.

Overview of development workflow using the Git repos.
  1. Clone the microservice Git repository/ies you want to develop on.
  2. Start the provided devcontainer (if provided)
  3. or install required dependencies for local development:
    • set up / install any backends you are developing against. In most cases a dedicated backend container should already exist, which you "just" need to pull.
    • For each repository the steps are the same as for the pre-defined stack repos.
    • For local API / Flask server development install Python >= 3.9.6.
      • Create a venv and install the dependencies from the requirements or pyproject file
    • For local UI frontend development set up Node.js using nvm.

A meta-repo management repo itk-demo-sw/itk-demo-sw is provided with tools to work with several repos simultaneously.

Software Repositories

Python packages

Python packages have been developed to support the microservices and provide common functionality.

JavaScript packages

npm-packages have been created to simplify the development of microservice UIs and unify their design.

External packages

Forks of external third-party software.

Other top-level repositories

docs repo description
docs docs/ repo that renders this documentation.
defaults defaults/ defaults for all common DeMi scripts and configuration.

Resources

Links to talks etc. can be found on the Resources page.

Contact

  1. Compared to "cloud" orchestrators like OpenShift/Kubernetes this project needs to work on statically assigned nodes following the DAQ hardware topology (deterministic deployment). In fact, the granularity of the microservices should follow the ITk DAQ hardware (and lower-level software) components. Integration with TDAQ is on the menu and kept in mind. ↩