From patchwork Mon Mar 23 08:30:30 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jan Kiszka X-Patchwork-Id: 354 Return-Path: Received: from shymkent.ilbers.de ([unix socket]) by shymkent (Cyrus 2.5.10-Debian-2.5.10-3+deb9u2) with LMTPA; Mon, 23 Mar 2026 09:30:53 +0100 X-Sieve: CMU Sieve 2.4 Received: from mail-wm1-f64.google.com (mail-wm1-f64.google.com [209.85.128.64]) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPS id 62N8UoG0028863 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Mon, 23 Mar 2026 09:30:50 +0100 Received: by mail-wm1-f64.google.com with SMTP id 5b1f17b1804b1-48531e6012bsf57672395e9.1 for ; Mon, 23 Mar 2026 01:30:50 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1774254644; cv=pass; d=google.com; s=arc-20240605; b=iBad/gmrx9e7alZ2oBLmHx1RGV7DikmcureUXiVmWDfPFeXYXgrRZJ+E87I7yyNWmu ofNTTontUJ+69Hza88TXKs4V+jeCnodVo7/XCJ1Jk5mJbg/MBe8G3sxUJip56WoZxMTR baTm6Sm6dasYKpHWP/WKuGKjN8K2AjpKln1+7efvuqDvD3P4Apfs9JXFCd0e1LRae9lt ZGB1s8GlJhde+BwnpKFGbQMzHurjkuMUdRdWO9DMcbwoVibqOam8di/i1//qksPGgMvM FkD8INBCOd7juuP83J/5OpdVCGo9P+UQXbxGtVGU/o+Ct+typT9yUZqUJMNFmr+xWEV/ 9Uxw== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20240605; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:reply-to:feedback-id:mime-version :message-id:date:subject:cc:to:from:dkim-signature; bh=y317rZ5NCnVNu8HHUx18yWOkOeh+sWV2rRf8e1dKtrY=; fh=OVUrLhPOIvyd3QYWlPIRJUFMEv8cewDWtqmYqSgfo/8=; b=PBqfKjpZXPvUm00aakwAgi0uP1Cb2E3m0tHhw+V3q8ejIm1NRA0FMn+CR0TA1OS5NC jVo3Gdj6Hxeb/WHlJd6w0QQKxooXfkHgaSzUl7EP2/rmkbAwZeRbImms5de6QtNn2XJ0 RCRPkU4j7s8iHpCLGBb0rs1+psvwH6D1g1A+UdK6BFn6m6YrDkoZDXTuqjIGg0arkzKS fQGaTHwpIimZ4KK2LRVkinOnd+3EbYmV8E7U7c3GFt7xmnYS50trCe22a71BjZf31SvS qPbV+EYaqTjtsTJDTsXE3MmYQ8kwFWgUgt54bo0US9Z1P1XbPV0UOAZcnIhvCrnZTBEv YcDQ==; darn=isar-build.org ARC-Authentication-Results: i=2; gmr-mx.google.com; dkim=pass header.i=@siemens.com header.s=fm1 header.b=NSjTNrh9; spf=pass (google.com: domain of fm-294854-20260323083040a04ddbf2f10002078c-i6ahjo@rts-flowmailer.siemens.com designates 185.136.65.226 as permitted sender) smtp.mailfrom=fm-294854-20260323083040a04ddbf2f10002078c-i6ahJO@rts-flowmailer.siemens.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=siemens.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlegroups.com; s=20251104; t=1774254644; x=1774859444; darn=isar-build.org; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:reply-to :x-original-authentication-results:x-original-sender:feedback-id :mime-version:message-id:date:subject:cc:to:from:from:to:cc:subject :date:message-id:reply-to; bh=y317rZ5NCnVNu8HHUx18yWOkOeh+sWV2rRf8e1dKtrY=; b=HJ5XVP8VgopUt/7x/fnNeWnK8b1rI8OFnVIb0GgHf04ONgMSamiEoSGdP3SKnIdFZc fhdceAgKqg2KOvn1zJaMuLZLmL6VCfuguFuxHQF5DfD7eYD4bNLIt7fUDDlqV0AWzphk EZWku0V3ySovrZNIc9aQAxYnm9tlmUCVdXor4cjB41OcDf6r5grBCkEgBW611vaua9kw A8GcaHJ1KT89eDMqD5ECvspdOxK2t58qgO6M6OvgCuhYWAJDxsb/9+n2QB/arw9cR+/u A8RPSJQfoWsX1gFvRYSZgAIILZNji/y/LF9x2E9Kmcv5lk9c3aHH73rTnvFwTWUaWuxl Cv5A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1774254644; x=1774859444; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :x-spam-checked-in-group:list-id:mailing-list:precedence:reply-to :x-original-authentication-results:x-original-sender:feedback-id :mime-version:message-id:date:subject:cc:to:from:x-beenthere :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=y317rZ5NCnVNu8HHUx18yWOkOeh+sWV2rRf8e1dKtrY=; b=GhkixgncUK5KyRkObI/rE+ml3NdvP3ocvUlmFdCEB1KVc4pJiT9xyBSroemMSu7VL2 fTMclIvLz1IiN62o7FHoy2ElOo36JVSTmM91kVyAXwfkH8ly+iTXWN7BgESs2BxQtMkG WZuNkEFiMyPdOHjc/NQ7ZNmPdf5sezXyILmhrUZrImaNPHDxwCUI1ZpKCjvbSsQJgAZM 4iqkwwYOJdu4d3QTH37UYBKWl7x7qJlzMDOQdxF5GrdaGQRGonGTuRo415MEGPUoIN0l u38fWYo7nosLTChWEN9WZ6GIPa06ipqqYeodgEGau0O3nm/CJuavoSe/PA3lHgyHfJz0 MYxw== X-Forwarded-Encrypted: i=2; AJvYcCX06mDe8iz0vpncfG57kNNudAl4QXLDgZ342/gvlpOB7lNp/j3f5W/z+0GC4Tp0ojqknpVzK9k=@isar-build.org X-Gm-Message-State: AOJu0YxUdUk2H9n90dcq7Kf5COGDQp3qvXZlhQX+BAvZxttOJ2cML2dl C4hdEnvvJAXNLwRWerUHtNyVVJ3LCR0teFrv/u7OBQbct8nbhaF3iuvW X-Received: by 2002:a05:600c:a43:b0:485:3f1c:d897 with SMTP id 5b1f17b1804b1-486fedb586dmr158552785e9.9.1774254643344; Mon, 23 Mar 2026 01:30:43 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com; h="AV1CL+EzB040DPfVB90awMOdrEiNijRRmS6vHF0LACTyXrcX7w==" Received: by 2002:a05:600c:3591:b0:485:3c80:a971 with SMTP id 5b1f17b1804b1-486f8d5cdccls37225995e9.0.-pod-prod-01-eu; Mon, 23 Mar 2026 01:30:41 -0700 (PDT) X-Received: by 2002:a05:6000:290d:b0:43b:4757:cc5 with SMTP id ffacd0b85a97d-43b6424bae2mr18022738f8f.19.1774254641051; Mon, 23 Mar 2026 01:30:41 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1774254641; cv=none; d=google.com; s=arc-20240605; b=U2CrDr5NKsY3UNF0Fzi+y0tb7AQeP2agaGslh58ANVUfrBltURoFer8Zo/1ZzyV9nh 8/Ylo6D37jnvCGscGCsvSn4hIzJbb+wqVs7Vc4BDFjlYSm0gf3cwTdjh/ZICrgbyYH+O xzn7OChzjViaej0ThL2YbA/LH40u7NYAI4bmCOMkttU6uf+NuFiPkLkEON0Xko8rihhu leF+azqw33no/3hPjl7TSEEfJcA1LK64tlnXKm83LMnVWrPbhTXtnzR1GwUWeHK8xmaN kziev5AxNZxjTfkCLP+RGZ8fwtJhinXBYGuLnBP6u0gj88FxbRW34ym7jcZxSYre0nn9 qZBQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20240605; h=feedback-id:content-transfer-encoding:mime-version:message-id:date :subject:cc:to:from:dkim-signature; bh=jrP+WX5OzM9PeIo2zhnStTkP9BqL45n80MisR/xOT1c=; fh=TnBsgKgmUKR6lCgSnR10YW4SMYjU5gXEv4fxP/+RPug=; b=S+G1TpQkuTYnOGHS/Ssj/4bKH3Q6d+eYNJid2OjI4Gs5ivheU2IuLmOE7i2n3lN8Y7 9b/5wA3Pq5W33JTRETZIo6l+HVGUtB25qN5fgjgPjsfRcQCmMYXuk3Ermcq9ccAAcWY8 IF1/EpWynXd80QPd7J6pYzo8TkRMpbzje0UAjpEjfz6l9cvmLmAsCZP+e3z5Kt/8qke5 kU6r7iLl+uuQZEyqRvZxvAHw0/4iNqpZFrwYWEwrf6h4c8ZNxJWmNn9ooyHDDiaM4bEb 0yueTfJT0y3vScODfPRUZM12gKMvmpbJ4p94k3bwHmD82sstMWx2JrChhgA5QJX7DkRo Np6w==; dara=google.com ARC-Authentication-Results: i=1; gmr-mx.google.com; dkim=pass header.i=@siemens.com header.s=fm1 header.b=NSjTNrh9; spf=pass (google.com: domain of fm-294854-20260323083040a04ddbf2f10002078c-i6ahjo@rts-flowmailer.siemens.com designates 185.136.65.226 as permitted sender) smtp.mailfrom=fm-294854-20260323083040a04ddbf2f10002078c-i6ahJO@rts-flowmailer.siemens.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=siemens.com Received: from mta-65-226.siemens.flowmailer.net (mta-65-226.siemens.flowmailer.net. [185.136.65.226]) by gmr-mx.google.com with ESMTPS id ffacd0b85a97d-43b644b7432si177037f8f.1.2026.03.23.01.30.41 for (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Mon, 23 Mar 2026 01:30:41 -0700 (PDT) Received-SPF: pass (google.com: domain of fm-294854-20260323083040a04ddbf2f10002078c-i6ahjo@rts-flowmailer.siemens.com designates 185.136.65.226 as permitted sender) client-ip=185.136.65.226; Received: by mta-65-226.siemens.flowmailer.net with ESMTPSA id 20260323083040a04ddbf2f10002078c for ; Mon, 23 Mar 2026 09:30:40 +0100 X-Patchwork-Original-From: "'Jan Kiszka' via isar-users" From: Jan Kiszka To: isar-users Cc: Felix Moessbauer , Cedric Hombourger Subject: [PATCH 0/9] Improve testsuite executability, basic GitHub CI Date: Mon, 23 Mar 2026 09:30:30 +0100 Message-ID: MIME-Version: 1.0 X-Flowmailer-Platform: Siemens Feedback-ID: 519:519-294854:519-21489:flowmailer X-Original-Sender: jan.kiszka@siemens.com X-Original-Authentication-Results: gmr-mx.google.com; dkim=pass header.i=@siemens.com header.s=fm1 header.b=NSjTNrh9; spf=pass (google.com: domain of fm-294854-20260323083040a04ddbf2f10002078c-i6ahjo@rts-flowmailer.siemens.com designates 185.136.65.226 as permitted sender) smtp.mailfrom=fm-294854-20260323083040a04ddbf2f10002078c-i6ahJO@rts-flowmailer.siemens.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=siemens.com X-Original-From: Jan Kiszka Reply-To: Jan Kiszka Precedence: list Mailing-list: list isar-users@googlegroups.com; contact isar-users+owners@googlegroups.com List-ID: X-Spam-Checked-In-Group: isar-users@googlegroups.com X-Google-Group-Id: 914930254986 List-Post: , List-Help: , List-Archive: , List-Unsubscribe: , X-Spam-Status: No, score=-4.9 required=5.0 tests=DKIMWL_WL_MED,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,MAILING_LIST_MULTI, RCVD_IN_DNSWL_BLOCKED,RCVD_IN_MSPIKE_H2,RCVD_IN_RP_CERTIFIED, RCVD_IN_RP_RNBL,RCVD_IN_RP_SAFE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on shymkent.ilbers.de X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= There were several proposals for wrapping the testsuite execution into some container. There is even docker-isar already, though this one was not yet used for a wrapper. This new approach enhances the existing container to "test-container", a kas-based environment that contains all required testsuite dependencies and can run them both as CI base container as well as via a script that is kas-container-like (actually using it). So, all the setup steps are history now, just call scripts/run-tests.sh -t dev to execute the minimal testsuite locally. Or just run scripts/run-tests.sh testsuite/citest.py:test_dev_apps to execute a specific test (avocado command line interface). Along that, kas is updated consistently to 5.2, using the trixie container now. The testsuite loses some obsolete KFAIL tags. And, finally, some basic CI step is added as GitHub action. The latter can be considered an RFC for now. It's unclear how far we can go on public CI by parallelizing some of the test sets. Avocado lacks support for test dependencies [1], thus this would require additional maintenance and may still run into the limitations of public GitHub runners. A full run of this series can be found at [2]. This also allows to try out the container before it was built officially, just set this prior to calling run-tests.sh: CONTAINER_BASENAME=ghcr.io/siemens/isar Jan [1] https://github.com/avocado-framework/avocado/issues/6131#issuecomment-2694016818 [2] https://github.com/siemens/isar/actions/runs/23426024062 Jan Kiszka (9): Update kas-container to 5.2 testsuite: Provide a new kas-based test container and wrapper script testsuite: Install avocado-framework-plugin-varianter-yaml-to-mux in test-container testsuite: Add squid to test-container ci: Add github workflow for building and deploying test-container ci: Switch gitlab-ci to new test-container CONTRIBUTING: Switch to new run-tests.sh testsuite: Drop KFAIL from trixie tests ci: Add github test job to github workflow .github/workflows/main.yml | 101 ++++++++++++++++++ .gitlab-ci.yml | 22 ++-- CONTRIBUTING.md | 72 +++---------- kas/kas-container | 6 +- scripts/ci_setup.sh | 32 ------ scripts/run-tests.sh | 18 ++++ testsuite/citest.py | 15 +-- testsuite/dockerdata/Dockerfile | 19 +++- testsuite/dockerdata/README.md | 20 ---- testsuite/dockerdata/build.sh | 14 +++ .../dockerdata/test-container-entrypoint | 99 +++++++++++++++++ testsuite/dockerdata/version | 1 + 12 files changed, 280 insertions(+), 139 deletions(-) create mode 100644 .github/workflows/main.yml delete mode 100755 scripts/ci_setup.sh create mode 100755 scripts/run-tests.sh delete mode 100644 testsuite/dockerdata/README.md create mode 100755 testsuite/dockerdata/build.sh create mode 100755 testsuite/dockerdata/test-container-entrypoint create mode 100644 testsuite/dockerdata/version