| Message ID | 20260123082501.240751-1-wzh@ilbers.de |
|---|---|
| Headers | show |
| Series | Various improvements to the testsuite | expand |
On 1/23/26 09:24, Zhihang Wei wrote: > This series aims at simplifying and speeding up the testsuite execution. > This should enable contributors to actually run it and test their patches. > > As pointed out in [1], the testsuite currently tests various features over > and over again, while others are not tested at all. We try to improve this > by focusing on "feature" tests, whereby each test case checks a set of > features instead of a whole image. We further identified long running tasks > (like the kernel builds) and move them to a dedicated test. > > The execution time reduction varies greatly depending on the hardware, > network and sstate usage. When applying these changes on recently added > patches, the time was often decreased by more than 50% on "fast". > > Note, that more refactorings of the testsuite are needed - and planned, > but this series should already address the biggest pain points. > > [1] https://groups.google.com/g/isar-users/c/uZBTIHzLp8Q/m/8k0L1vfMDAAJ > > Changes since v2: > > - move startvm tests after each build to avoid inter-testsuite dependency > - remove SstateTest from full v3 has passed fast and full CI. Zhihang > Changes since v1: > > - continued work on testsuite refactoring, this supersedes: > - testsuite: fix SignatureTest by avoiding absolute path in bblayers > - testsuite: add rootfs target for rootfs only tests > - Enable SState cache in GitLab CI > - testsuite: fix typo in log message in perform_signature_lint > - completed exclusion / refactoring of db2m sbuild chroot > - GitLab CI: increased after script timeout to give sstate upload more time > - GitLab CI: do not upload sstate artifacts of images (as almost no reuse) > - do not add SignatureTest to fast tag (as otherwise VM tests break...) > - rebuild container test images in VM test to not depend on test order > - rebased onto next > > Anton Mikanovich (1): > testsuite: Group prebuilt_containers related test cases together > > MOESSBAUER, Felix (17): > testsuite: move targets with custom kernel to separate test > testsuite: enable ccache on kernel tests > testsuite: make prebuilt container a feature test > testsuite: make compat test standalone test > testsuite: handle IMAGE_INSTALL solely in cibuilder.py > testsuite: limit cross_debsrc test to subset of packages > testsuite: forward SSTATE_MIRRORS into CI env on sstate > ci: add support for sstate cache > testsuite: make test_cross_deps more specific > testsuite: fix typo in log message in perform_signature_lint > testsuite: fix SignatureTest by avoiding absolute path in bblayers > testsuite: use more recent distros in SignatureTest > testsuite: make SignatureTest idempotent > testsuite: add rootfs target for rootfs only tests > testsuite: refactor sbom tests to avoid overhead > testsuite: make sbuild-flavor test standalone > testsuite: skip VM tests if images are not available > > Zhihang Wei (2): > testsuite: Execute startvm testcases after building > testsuite: Remove SstateTest from full testsuite > > .gitlab-ci.yml | 36 +- > meta-test/conf/bblayers.conf.sample | 8 +- > meta-test/conf/local.conf.sample | 10 - > .../hello-isar/hello-isar.bbappend | 3 + > .../recipes-app/libhello/libhello.bbappend | 5 + > .../recipes-core/images/isar-image-ci.bb | 2 - > .../recipes-core/images/isar-rootfs-ci.bb | 17 + > testsuite/cibase.py | 34 +- > testsuite/cibuilder.py | 49 +- > testsuite/citest.py | 740 +++++++++++------- > 10 files changed, 602 insertions(+), 302 deletions(-) > create mode 100644 meta-test/recipes-app/libhello/libhello.bbappend > create mode 100644 meta-test/recipes-core/images/isar-rootfs-ci.bb >