From patchwork Fri Jan 23 08:24:56 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zhihang Wei X-Patchwork-Id: 4831 Return-Path: Received: from shymkent.ilbers.de ([unix socket]) by shymkent (Cyrus 2.5.10-Debian-2.5.10-3+deb9u2) with LMTPA; Fri, 23 Jan 2026 09:35:47 +0100 X-Sieve: CMU Sieve 2.4 Received: from mail-lj1-f191.google.com (mail-lj1-f191.google.com [209.85.208.191]) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPS id 60N8ZkhR016317 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Fri, 23 Jan 2026 09:35:47 +0100 Received: by mail-lj1-f191.google.com with SMTP id 38308e7fff4ca-385bb7f429csf9029211fa.1 for ; Fri, 23 Jan 2026 00:35:47 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1769157341; cv=pass; d=google.com; s=arc-20240605; b=J89r4Dk9YqBUX8wGvyoiGmZx20rTrtlL7IjCbCk5nZtIdl7Esg/tBCGsr+1H4Q+r80 5LjqdypvOpXv1D9saDESOpeKPWzY1eJ38ffdO9PRgYwFWN2fWPFW13fcFJMhkrGRTJ/y /3Mc1KPrBXeR4ylzi4+8+ZFoVzoo+HiD3Mj6m5ueBNctAx8FYEV3lni7jSvq4vPCa5ow JAmNPpGqvzb4c0hPc7ZTes8JpqM3UK0zSUyn2VYcVNEuiWMbFF6X9XUlWSC24M4K9ii/ 7Gv9vLACyauSRs4ZGm9DnBQRY4cIzlja383yKoV42PTqz6EX1So/L/13WAgMGrH1/qGS IAIg== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20240605; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:dkim-signature; bh=FsWTelvCTgbf+S8C4dNF6azbNrTDS9QAqRFyikHdqwU=; fh=eMPTZPQ6rJDkiC2vKV9NK04sZbMHEIG7rGC9Gl+FdkQ=; b=YlBofeN0+Don2d2ZjRfwOaC7KFrAgI4CzqSxbpx4z4VRra88Oj3lrp12hHPzFgLONl yUpdOFrqfV/8a9kiZCyYiZu57nGR442tEMMIzK/oUzHpbHSzBaLi6p9tpmtLYY81lO7C jHdH1vWeYfL5eXwxvoM2BqYUAjiFbH9OUTw+yJEKaEYKybCLfB8bsop6wXKNNG8OIXic SibbMKtiRwQOzx18AABy1JXBwRXfud6yxned0fixRFwdb2DbXr0SONpd8o+t5wVABHFB LzG4290y9Wurnwm0CndYUl95dLr18/DBnsZQiVcwtwlkmgV7mcgbj09USxXomNq3C9M5 aXnw==; darn=isar-build.org ARC-Authentication-Results: i=2; gmr-mx.google.com; spf=pass (google.com: domain of wzh@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=wzh@ilbers.de DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlegroups.com; s=20230601; t=1769157341; x=1769762141; darn=isar-build.org; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:x-original-authentication-results :x-original-sender:mime-version:references:in-reply-to:message-id :date:subject:cc:to:from:sender:from:to:cc:subject:date:message-id :reply-to; bh=FsWTelvCTgbf+S8C4dNF6azbNrTDS9QAqRFyikHdqwU=; b=CiW8A6v47bI62ThHbnmQw1bSVKZabi/KK1VrBlwtbInBEahs2OZ9FoWHxAj99Gp/kv C+d/G/dyjV/SbXChZtjSEIX2I2SjV997d8INfAhngi7+IL1kseFDkWR14guazydigur4 /dISPvD3Xp8O9b8Vtc9vkI5aza6UU60YOdyIOSU8Gx5HWG6mOjOE1/KP42zuhoOKcJp9 gmgm/VvYaZtgDh2ASxqezDm19/wEXgoolSjsnKXoLjzrY2eSB84LZgNK+U/iz+VSROMH C9wa9bvadTXzaHX3w29Jtuxpd826YPNQOJX2h/KZ9s0dOSrIQdhc49ErVzMUanf1MwBw pRxA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1769157341; x=1769762141; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :x-spam-checked-in-group:list-id:mailing-list:precedence :x-original-authentication-results:x-original-sender:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :x-beenthere:x-gm-message-state:sender:from:to:cc:subject:date :message-id:reply-to; bh=FsWTelvCTgbf+S8C4dNF6azbNrTDS9QAqRFyikHdqwU=; b=dTmyPdvFW7F5Q1p6YrHKrZx/TLFN8M1DGbaV+aFchYXuZfzHiLTQfWLSCP5ykClGXb F6VrLSjRiTR690Gt4fP4gY6Tj65tecFJO+6SToMLVirW8gAFi/6TBYZomS7GUmnjtyqn 4Ermr/nubEycL7WjznOYFjsolTzplNXvRQf48vnCsssjn53HbngWk6xU/9qUuWJkLaFK qJYduNjRtUvfwRJ3YOLposLZAnhQG4d5k/ND0O6JP8pZIcpAUkePgsgIgCu8aiBTaCoV TRbzaNCqVx/7gAdVJc/F01em3B6s1ABNReVSX7qdOx35Dycjp++8WnJnq1eZCYtlioN6 ofKQ== Sender: isar-users@googlegroups.com X-Forwarded-Encrypted: i=2; AJvYcCUK26pOz7ph9S5ZA+zOTr1HRD0RzW/zIW/bj5tHGlINs/+hndLP8GAM9f20N4woiYM+XIKXeWY=@isar-build.org X-Gm-Message-State: AOJu0Yyil4S2isCh2cdp+3kJJE8rNVlMKVBrwgO7y3dguVL2Y5kC4ZEI J+cF91EbY+6zfaOqXuWPfVB63WE4sZPIltrVPZ+SJCK46bIy94FxXRm0 X-Received: by 2002:a05:651c:b25:b0:37b:afdc:bb0d with SMTP id 38308e7fff4ca-385d9ed5f54mr6384961fa.6.1769156710837; Fri, 23 Jan 2026 00:25:10 -0800 (PST) X-BeenThere: isar-users@googlegroups.com; h="AV1CL+HbrdhS8zetyiihHJJmHpqLcxkt989tGHVv5NOryCiV7w==" Received: by 2002:a2e:871a:0:b0:385:ba7e:10fb with SMTP id 38308e7fff4ca-385c269049fls2847631fa.2.-pod-prod-02-eu; Fri, 23 Jan 2026 00:25:08 -0800 (PST) X-Received: by 2002:a05:651c:1992:b0:380:989:f5fb with SMTP id 38308e7fff4ca-385d9e1bff4mr7276531fa.0.1769156708130; Fri, 23 Jan 2026 00:25:08 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1769156708; cv=none; d=google.com; s=arc-20240605; b=BOXffKRfg/JfUJ4LQYH987fidR5RU1b4dRw/vd46PnoCy/vYB4kZ6IxYq8Rb5mq1+4 v9HTqzyGHjTfp3+T+YcWeokFxze6c1AIumUWYoNNJB8kE2LIri017DtIDIF5JQpebp6O SwjejGLJUJkYKBjK6HKY2WZATNe+LsHUQPIWR577B2V9xZ3sjHc68ZQ5ayJYZG/eJgPT XuhrUyJOxkK5ZnY5uK7LE2MUERunRAm2oVm491i2NXl3XEXIV9CRsDhHk7ND4yj1QHEV +9HHlQkIfTpL0FDsTGivg78KhuhgZ3yJ/FTKr2hte5KQJC6tuvRPFazSAw56B17yW/u0 7HAg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20240605; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=bmrc9pIcykHA72MNymnLGtR+iP/RVsJueCR+h+ydxCA=; fh=V42d4Y/lNDOgZ/9/qsphFBeZCU4sz3YugfOOjBc+1H8=; b=hGwCkWKaMtz+wkDPGgjGlb/xQbE0MkvvieM5SDZYp6Go9i8e85KuVsMrZ2EPYooMLy c0ntgNfO6tiErzCL4FcH48FQBYRlA6gs72GSAIvRE21ayZIH9nr08LRy3IPfuRoDFr5g g5M6PByXUTtVqJINq4KAEfDabfzDi02jRlA/n5029bWEMg6JTCvbxV3ObhhVtDqrmIn/ RjpUZf0c5oiThfOMrJZHoCWTb35C0BsvV8uCuMWtbGB+GYHTCUYSm8DC1zRqRGCPgIEq 4ryajThXmGLFwC/FX59I9tE3BbPISmmPi3lWGP6TVKShyDzW/XGUL3yv9IMTlbqmCef2 2VeQ==; dara=google.com ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of wzh@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=wzh@ilbers.de Received: from shymkent.ilbers.de (shymkent.ilbers.de. [85.214.156.166]) by gmr-mx.google.com with ESMTPS id 38308e7fff4ca-385d9f8daf2si458931fa.2.2026.01.23.00.25.07 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 23 Jan 2026 00:25:08 -0800 (PST) Received-SPF: pass (google.com: domain of wzh@ilbers.de designates 85.214.156.166 as permitted sender) client-ip=85.214.156.166; Received: from debian-zwei.m.ilbers.de ([88.130.203.42]) (authenticated bits=0) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPA id 60N8P1Xu014977; Fri, 23 Jan 2026 09:25:07 +0100 From: Zhihang Wei To: isar-users@googlegroups.com, felix.moessbauer@siemens.com, amikan@ilbers.de Cc: wzh@ilbers.de, cedric.hombourger@siemens.com Subject: [PATCH v3 15/20] testsuite: refactor sbom tests to avoid overhead Date: Fri, 23 Jan 2026 09:24:56 +0100 Message-Id: <20260123082501.240751-16-wzh@ilbers.de> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20260123082501.240751-1-wzh@ilbers.de> References: <20260123082501.240751-1-wzh@ilbers.de> MIME-Version: 1.0 X-Spam-Status: No, score=-4.6 required=5.0 tests=DKIMWL_WL_MED,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_EF,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,RCVD_IN_DNSWL_BLOCKED,RCVD_IN_MSPIKE_H2, RCVD_IN_RP_CERTIFIED,RCVD_IN_RP_RNBL,RCVD_IN_RP_SAFE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on shymkent.ilbers.de X-Original-Sender: wzh@ilbers.de X-Original-Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of wzh@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=wzh@ilbers.de Precedence: list Mailing-list: list isar-users@googlegroups.com; contact isar-users+owners@googlegroups.com List-ID: X-Spam-Checked-In-Group: isar-users@googlegroups.com X-Google-Group-Id: 914930254986 List-Post: , List-Help: , List-Archive: , List-Unsubscribe: , X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= From: "MOESSBAUER, Felix" We currently test the SBOM infrastructure in all image builds, which adds a significant overhead. We now change this to not generate SBOMs in general (and by that avoid building the dependencies). To not have a testing gap, we add a dedicated SBOM test that checks the SBOM creation for various targets. In addition, we now also check the content of the SBOM for plausibility. In the future, the SBOM test can be extended without slowing down the overall test execution. Signed-off-by: Felix Moessbauer --- testsuite/cibase.py | 26 ++++++++++++++++++++++++++ testsuite/cibuilder.py | 4 ++++ testsuite/citest.py | 33 +++++++++++++++++++++++++++++++++ 3 files changed, 63 insertions(+) diff --git a/testsuite/cibase.py b/testsuite/cibase.py index 5ef1a5b5..fd6a3df9 100755 --- a/testsuite/cibase.py +++ b/testsuite/cibase.py @@ -140,6 +140,32 @@ class CIBaseTest(CIBuilder): self.delete_from_build_dir('ccache') self.unconfigure() + def perform_sbom_test(self, targets, **kwargs): + """ + Build a rootfs containing a needle package and check if that package + is added to the sbom. + """ + import json + + needle_pkg = 'cowsay' + self.perform_build_test( + targets, image_install=needle_pkg, + generate_sbom=True + ) + + for t in targets: + ds, pn, distro, machine = \ + CIUtils.getVars('DEPLOY_DIR_SBOM', 'PN', 'DISTRO', 'MACHINE', + target=t) + for t in ["cdx", "spdx"]: + sbom_path = os.path.join(ds, f'{pn}-{distro}-{machine}.{t}.json') + self.log.info(f"Check {t} SBOM in {sbom_path}") + with open(sbom_path) as f: + sbom = json.load(f) + pkg_key = 'components' if t == 'cdx' else 'packages' + if not any(c for c in sbom[pkg_key] if c['name'] == needle_pkg): + self.fail(f'{needle_pkg} package not found in SBOM {sbom_path}') + def perform_sstate_populate(self, image_target, **kwargs): # Use a different isar root for populating sstate cache isar_sstate = f"{isar_root}/isar-sstate" diff --git a/testsuite/cibuilder.py b/testsuite/cibuilder.py index 9c97115b..7538ade2 100755 --- a/testsuite/cibuilder.py +++ b/testsuite/cibuilder.py @@ -126,6 +126,7 @@ class CIBuilder(Test): installer_distro=None, installer_device=None, customizations=None, + generate_sbom=False, lines=None, **kwargs, ): @@ -176,6 +177,7 @@ class CIBuilder(Test): f" image_install = {image_install}\n" f" installer_image = {installer_image}\n" f" customizations = {customizations}\n" + f" generate_sbom = {generate_sbom}\n" f" lines = {strlines}\n" f"===================================================" ) @@ -275,6 +277,8 @@ class CIBuilder(Test): 'CUSTOMIZATION_FOR_IMAGES:append = " isar-image-ci"\n' 'HOSTNAME:isar-image-ci = "isar-ci"\n' ) + if generate_sbom is False: + f.write('ROOTFS_FEATURES:remove = "generate-sbom"\n') if lines is not None: f.writelines((line + '\n' if not line.endswith('\n') else line) for line in lines) diff --git a/testsuite/citest.py b/testsuite/citest.py index eaa4c440..d908f9bc 100755 --- a/testsuite/citest.py +++ b/testsuite/citest.py @@ -694,6 +694,39 @@ class CustomizationsTest(CIBaseTest): ) +class SbomTest(CIBaseTest): + """ + Test to check if sbom is generated and contains expected packages. + Most tests are rootfs tests to avoid costly initrd build and imaging. + + :avocado: tags=sbom,fast + """ + + def test_sbom_rootfs_generate(self): + targets = [ + 'mc:qemuamd64-bookworm:isar-rootfs-ci', + 'mc:qemuarm64-bookworm:isar-rootfs-ci', + 'mc:qemuamd64-trixie:isar-rootfs-ci', + 'mc:qemuarm64-trixie:isar-rootfs-ci', + 'mc:qemuamd64-noble:isar-rootfs-ci', + ] + + self.init() + self.perform_sbom_test(targets) + + def test_sbom_unsupported(self): + targets = [ + 'mc:qemuamd64-bullseye:isar-rootfs-ci', + 'mc:qemuamd64-focal:isar-rootfs-ci', + ] + + self.init() + self.perform_build_test( + targets, bitbake_cmd='do_rootfs', image_install='cowsay', + generate_sbom=True + ) + + class SignatureTest(CIBaseTest): """