The directory utilities/workflows holds the infrastructure and utilities that power waLBerla's continuous integration and testing (CI) workflows.

Ressources

CMake Presets

All CMake configure-build-test workflows are encoded as CMake presets.

The workflow presets used in the CI are auto-generated by the script generateWorkflows.py. To use them locally, run the following command in waLBerla's project root directory:

python utilities/workflows/generateWorkflows.py cmake-presets ci-workflows.json -p CMakeUserPresets.json

This will generate the file ci-workflows.json and include it into your user presets file. Do not check the generated file into version control, as it is specific to your local machine.

The generated presets defined therein are built as combinations of predefined fragments through inheritance. These fragments are defined in cmake-fragments.json. They can be used to recreate CI build configurations for local development and debugging by setting up equivalent presets in the developer's CMakeUserPresets.json; for example:

{
    "name": "my-build",
    "inherits": [
        ".clang",
        ".hybrid",
        ".python",
        ".hip",
        ".ci-base"
    ],
    "cacheVariables": {
        "CMAKE_PREFIX_PATH": "<your-prefix-path>"
    }
},

This preset definition uses LLVM/Clang as a compiler, enables hybrid parallelization, Python and code generation, and HIP for AMD GPUs. The fragment .ci-base holds the base configuration and must always be listed last, since presets override each other bottom-to-top.

CUDA Compute Capabilities

When building for CUDA, CMake needs to know the target compute capability we are building for. The generated presets expect to find this in the environment variable WALBERLA_CI_CUDA_ARCHITECTURES. When building locally, either set that variable before running CMake, or override CMAKE_CUDA_ARCHITECTURES on the command line or in a derived preset (see above):

WALBERLA_CI_CUDA_ARCHITECTURES=81 cmake --preset .ci-hybrid-cuda
# OR
cmake --preset .ci-hybrid-cuda -DCMAKE_CUDA_ARCHITECTURES=81

GitLab CI Setup

Common Job Definitions (`ci-common.yaml`)

The common job definitions hold the bulk of the CI setup and are meant to be extended and re-used for defining actual tasks. For examples of this, look at the primary .gitlab-ci.yaml at the root of this repository.

The Test Matrix

The waLBerla CI features a test matrix that combines various compilers and build settings. The matrix is defined in generateWorkflows.py which generates a downstream pipeline definition for the test matrix (see Downstream Pipelines in GitLab CI). To generate the matrix file, run

python generateWorkflows.py ci-matrix matrix.yaml

Do not check the generated matrix definition into version control. It is automatically generated during each CI run.

Docker Images

WaLBerla's CI uses the Docker images from the ci/images repository. To recreate the CI build environments locally for debugging, you can pull these images and work inside them (for the CI commands, see the container registry).