mbox series

[0/9] Improve testsuite executability, basic GitHub CI

Message ID cover.1774254639.git.jan.kiszka@siemens.com
Headers show
Series Improve testsuite executability, basic GitHub CI | expand

Message

Jan Kiszka March 23, 2026, 8:30 a.m. UTC 
There were several proposals for wrapping the testsuite execution into
some container. There is even docker-isar already, though this one was
not yet used for a wrapper.

This new approach enhances the existing container to "test-container", a
kas-based environment that contains all required testsuite dependencies
and can run them both as CI base container as well as via a script that
is kas-container-like (actually using it). So, all the setup steps are
history now, just call

    scripts/run-tests.sh -t dev

to execute the minimal testsuite locally. Or just run

    scripts/run-tests.sh testsuite/citest.py:test_dev_apps

to execute a specific test (avocado command line interface).

Along that, kas is updated consistently to 5.2, using the trixie
container now. The testsuite loses some obsolete KFAIL tags. And,
finally, some basic CI step is added as GitHub action.

The latter can be considered an RFC for now. It's unclear how far we can
go on public CI by parallelizing some of the test sets. Avocado lacks
support for test dependencies [1], thus this would require additional
maintenance and may still run into the limitations of public GitHub
runners.

A full run of this series can be found at [2]. This also allows to try
out the container before it was built officially, just set this prior to
calling run-tests.sh:

    CONTAINER_BASENAME=ghcr.io/siemens/isar

Jan

[1] https://github.com/avocado-framework/avocado/issues/6131#issuecomment-2694016818
[2] https://github.com/siemens/isar/actions/runs/23426024062

Jan Kiszka (9):
  Update kas-container to 5.2
  testsuite: Provide a new kas-based test container and wrapper script
  testsuite: Install avocado-framework-plugin-varianter-yaml-to-mux in
    test-container
  testsuite: Add squid to test-container
  ci: Add github workflow for building and deploying test-container
  ci: Switch gitlab-ci to new test-container
  CONTRIBUTING: Switch to new run-tests.sh
  testsuite: Drop KFAIL from trixie tests
  ci: Add github test job to github workflow

 .github/workflows/main.yml                    | 101 ++++++++++++++++++
 .gitlab-ci.yml                                |  22 ++--
 CONTRIBUTING.md                               |  72 +++----------
 kas/kas-container                             |   6 +-
 scripts/ci_setup.sh                           |  32 ------
 scripts/run-tests.sh                          |  18 ++++
 testsuite/citest.py                           |  15 +--
 testsuite/dockerdata/Dockerfile               |  19 +++-
 testsuite/dockerdata/README.md                |  20 ----
 testsuite/dockerdata/build.sh                 |  14 +++
 .../dockerdata/test-container-entrypoint      |  99 +++++++++++++++++
 testsuite/dockerdata/version                  |   1 +
 12 files changed, 280 insertions(+), 139 deletions(-)
 create mode 100644 .github/workflows/main.yml
 delete mode 100755 scripts/ci_setup.sh
 create mode 100755 scripts/run-tests.sh
 delete mode 100644 testsuite/dockerdata/README.md
 create mode 100755 testsuite/dockerdata/build.sh
 create mode 100755 testsuite/dockerdata/test-container-entrypoint
 create mode 100644 testsuite/dockerdata/version