| Message ID | 20251201085813.1616095-1-felix.moessbauer@siemens.com |
|---|---|
| Headers | show |
| Series | Add SBOM generation with debsbom | expand |
On 01.12.25 09:58, Felix Moessbauer wrote: > This patchset adds proper SBOM generation in the two standard formats > SPDX and CycloneDX during the rootfs generation process. > > The generation is itself is handled by a SBOM generator `debsbom` [1] > which is developed as an open source project at Siemens. It is still > early in development, but it has enough features for what we require > in isar. The required dependencies which are not yet available as > Debian packages were minimally packaged directly in isar too. > > This is a followup of the previous RFC [2]. Since then the series has > changed a lot. The SBOM generation was moved from a simple OE lib to > `debsbom`. This also meant the introduction of a separate chroot was > necessary. The SBOM generation process was also moved from the image > step to the rootfs step, along with a lot of minor changes and > improvements. > > [1] https://github.com/siemens/debsbom > [2] https://groups.google.com/g/isar-users/c/8L-CF4BJY0I/m/p0N3o_zfAAAJ > > Changes since v5: > > - fix isar-image-ci on qemuamd64-bullseye (set IMAGER_BOM according to > machine changes made in image file) > - rebased onto next > > Changes since v4: > > - rebased onto next > - fix race condition on creation of ${DEPLOY_DIR_SBOM} (aka ${DEPLOY_DIR_IMAGE}) > > Changes since v3: > > - fix issue on external bullseye initramfs (we now disable sbom generation > on all unsupported distros rootfs instances) > - update debsbom to v0.4.0 > - rebased onto next > > Changes since v2: > > - fix issues when HOST_ARCH != DISTRO_ARCH on derived distributions > - update debsbom to v0.3.0, which fixes the Origin: bug reported in v2 > - generate SBOM for imager as well and create merged sbom of .wic image > - resend imager manifest + wic manifest patches to reduce conflicts > > Note, that the patches p1-p5 are most important as they add basic SBOM > support. The remaining patches address the imager + .wic bom part, > which also can be merged later on. > > Changes since v1: > > - remove tarball > - refactor packaging (auto-derive python dependencies) > - only build missing packages (varies on bookworm, trixie, noble) > - add ubuntu support > - only generate sboms for supported distributions (bookworm/jammy and > onwards) > - update debsbom (includes bug fixes and more information for source > packages) > > > Christoph Steiger (3): > meta: package python libraries for SBOM generation > meta: package python3-debsbom > meta: add SBOM generation with debsbom > > Felix Moessbauer (7): > refactor: move get_rootfs_distro from sdk into rootfs > override distro vendor in SBOM on Ubuntu > add support to add imager dependencies to BOM > wic: create uniform manifest describing all image components > qemuamd64: add IMAGER_BOM entries > imager: create SBOM of IMAGER_BOM packages > wic: create uniform SBOM describing all image components > > doc/user_manual.md | 1 + > meta-isar/conf/distro/ubuntu-common.inc | 2 + > meta-isar/conf/machine/qemuamd64.conf | 1 + > .../recipes-core/images/isar-image-ci.bb | 1 + > meta/classes/image-tools-extension.bbclass | 29 +++++++++ > meta/classes/image.bbclass | 7 ++ > meta/classes/imagetypes_wic.bbclass | 30 +++++++++ > meta/classes/initramfs.bbclass | 3 +- > meta/classes/rootfs.bbclass | 23 ++++++- > meta/classes/sbom.bbclass | 65 +++++++++++++++++++ New classes should go into the right category already, then rebasing is easier for whoever has to do it in the light of https://patchwork.isar-build.org/project/isar/list/?series=1780. Look at the dracut series. Jan
On Mon, 2025-12-01 at 10:15 +0100, Jan Kiszka wrote: > On 01.12.25 09:58, Felix Moessbauer wrote: > > This patchset adds proper SBOM generation in the two standard formats > > SPDX and CycloneDX during the rootfs generation process. > > > > The generation is itself is handled by a SBOM generator `debsbom` [1] > > which is developed as an open source project at Siemens. It is still > > early in development, but it has enough features for what we require > > in isar. The required dependencies which are not yet available as > > Debian packages were minimally packaged directly in isar too. > > > > This is a followup of the previous RFC [2]. Since then the series has > > changed a lot. The SBOM generation was moved from a simple OE lib to > > `debsbom`. This also meant the introduction of a separate chroot was > > necessary. The SBOM generation process was also moved from the image > > step to the rootfs step, along with a lot of minor changes and > > improvements. > > > > [1] https://github.com/siemens/debsbom > > [2] https://groups.google.com/g/isar-users/c/8L-CF4BJY0I/m/p0N3o_zfAAAJ > > > > Changes since v5: > > > > - fix isar-image-ci on qemuamd64-bullseye (set IMAGER_BOM according to > > machine changes made in image file) > > - rebased onto next > > > > Changes since v4: > > > > - rebased onto next > > - fix race condition on creation of ${DEPLOY_DIR_SBOM} (aka ${DEPLOY_DIR_IMAGE}) > > > > Changes since v3: > > > > - fix issue on external bullseye initramfs (we now disable sbom generation > > on all unsupported distros rootfs instances) > > - update debsbom to v0.4.0 > > - rebased onto next > > > > Changes since v2: > > > > - fix issues when HOST_ARCH != DISTRO_ARCH on derived distributions > > - update debsbom to v0.3.0, which fixes the Origin: bug reported in v2 > > - generate SBOM for imager as well and create merged sbom of .wic image > > - resend imager manifest + wic manifest patches to reduce conflicts > > > > Note, that the patches p1-p5 are most important as they add basic SBOM > > support. The remaining patches address the imager + .wic bom part, > > which also can be merged later on. > > > > Changes since v1: > > > > - remove tarball > > - refactor packaging (auto-derive python dependencies) > > - only build missing packages (varies on bookworm, trixie, noble) > > - add ubuntu support > > - only generate sboms for supported distributions (bookworm/jammy and > > onwards) > > - update debsbom (includes bug fixes and more information for source > > packages) > > > > > > Christoph Steiger (3): > > meta: package python libraries for SBOM generation > > meta: package python3-debsbom > > meta: add SBOM generation with debsbom > > > > Felix Moessbauer (7): > > refactor: move get_rootfs_distro from sdk into rootfs > > override distro vendor in SBOM on Ubuntu > > add support to add imager dependencies to BOM > > wic: create uniform manifest describing all image components > > qemuamd64: add IMAGER_BOM entries > > imager: create SBOM of IMAGER_BOM packages > > wic: create uniform SBOM describing all image components > > > > doc/user_manual.md | 1 + > > meta-isar/conf/distro/ubuntu-common.inc | 2 + > > meta-isar/conf/machine/qemuamd64.conf | 1 + > > .../recipes-core/images/isar-image-ci.bb | 1 + > > meta/classes/image-tools-extension.bbclass | 29 +++++++++ > > meta/classes/image.bbclass | 7 ++ > > meta/classes/imagetypes_wic.bbclass | 30 +++++++++ > > meta/classes/initramfs.bbclass | 3 +- > > meta/classes/rootfs.bbclass | 23 ++++++- > > meta/classes/sbom.bbclass | 65 +++++++++++++++++++ > > New classes should go into the right category already, then rebasing is > easier for whoever has to do it in the light of > https://patchwork.isar-build.org/project/isar/list/?series=1780. Look at > the dracut series. Hi, according to my interpretation of the use-more-classes series and the dracut series, the sbom class is already correct. The sbom series also cleanly applies and works on top of the use-more-classes series. Felix > > Jan > > -- > Siemens AG, Foundational Technologies > Linux Expert Center
Hi,
There are mulltiple test cases failed on CI.
On fast CI:
Failed test case: citest.py:DevTest.test_dev
ERROR: Logfile of failure stored in:
/build/isar_wzh_devel_5b_fast/4/build/tmp/work/debian-bookworm-amd64/isar-image-ci-qemuamd64/1.0-r0/temp/log.do_image_wic.3897481
dpkg-query: no packages found matching systemd-boot-efi:amd64
Failed test case: citest.py:DevTest.test_dev_apps
ERROR: Logfile of failure stored in:
/build/isar_wzh_devel_5b_fast/4/build/tmp/work/debian-bookworm-amd64/isar-image-ci-qemuamd64/1.0-r0/temp/log.do_image_wic.18529
dpkg-query: no packages found matching systemd-boot-efi:amd64
Failed test case: citest.py:DevTest.test_dev_rebuild
ERROR: Logfile of failure stored in:
/build/isar_wzh_devel_5b_fast/4/build/tmp/work/debian-bookworm-amd64/isar-image-ci-qemuamd64/1.0-r0/temp/log.do_image_wic.262594
dpkg-query: no packages found matching systemd-boot-efi:amd64
Failed test case: citest.py:CrossTest.test_cross_debsrc
ERROR: Logfile of failure stored in:
/build/isar_wzh_devel_5b_fast/4/build/tmp/work/debian-bookworm-arm64/isar-image-ci-qemuarm64/1.0-r0/temp/log.do_image_wic.3057235
dpkg-query: no packages found matching systemd-boot-efi:arm64
Failed test case: citest.py:CrossTest.test_cross_dependencies
ERROR: Logfile of failure stored in:
/build/isar_wzh_devel_5b_fast/4/build/tmp/work/debian-bookworm-arm64/isar-image-ci-qemuarm64/1.0-r0/temp/log.do_image_wic.772494
dpkg-query: no packages found matching systemd-boot-efi:arm64
On full CI:
Failed test case: citest.py:ReproTest.test_repro_unsigned:
ERROR: Logfile of failure stored in:
/build/isar_wzh_devel_7b/28/build/tmp/work/debian-bookworm-amd64/isar-image-base-qemuamd64/1.0-r0/temp/log.do_image_wic.862703
bwrap: Can't mkdir /mnt/rootfs: Permission denied
I assume this is related with CI environment. This error is also there
for v5.
Failed test case: citest.py:InstallerTest.test_installer_build:
ERROR: Logfile of failure stored in:
/build/isar_wzh_devel_7b/28/build/tmp/work/debian-bookworm-amd64/isar-image-ci-qemuamd64/1.0-r0/temp/log.do_image_wic.1240668
dpkg-query: no packages found matching systemd-boot-efi:amd64
Failed test case: citest.py:KernelTests.test_per_kernel:
Logfile of failure stored in:
/build/isar_wzh_devel_7b/28/build/tmp/work/debian-bookworm-arm64/isar-image-ci-qemuarm64/1.0-r0/temp/log.do_image_wic.1626092
dpkg-query: no packages found matching systemd-boot-efi:arm64
Failed test case: citest.py:InitRdTest.test_dracut_in_image:
ERROR: Logfile of failure stored in:
/build/isar_wzh_devel_7b/28/build/tmp/work/debian-bookworm-amd64/isar-image-ci-qemuamd64/1.0-r0/temp/log.do_image_wic.1723289
dpkg-query: no packages found matching systemd-boot-efi:amd64
Failed test case: citest.py:InitRdCrossTests.test_dracut_in_image:
ERROR: Logfile of failure stored in:
/build/isar_wzh_devel_7b/28/build/tmp/work/debian-bookworm-amd64/isar-image-ci-qemuamd64/1.0-r0/temp/log.do_image_wic.2466233
dpkg-query: no packages found matching systemd-boot-efi:amd64
Failed test case: citest.py:WicTest.test_wic_nodeploy_partitions:
ERROR: Logfile of failure stored in:
/build/isar_wzh_devel_7b/28/build/tmp/work/debian-bookworm-arm64/isar-image-ci-qemuarm64/1.0-r0/temp/log.do_image_wic.2823952
dpkg-query: no packages found matching systemd-boot-efi:arm64
Failed test case: citest.py:WicTest.test_wic_deploy_partitions:
ERROR: Logfile of failure stored in:
/build/isar_wzh_devel_7b/28/build/tmp/work/debian-bookworm-arm64/isar-image-ci-qemuarm64/1.0-r0/temp/log.do_image_wic.2892279
dpkg-query: no packages found matching systemd-boot-efi:arm64
07:06:42
18-/build/isar_wzh_devel_7b/28/testsuite/citest.py:NoCrossTest.test_nocross:
ERROR: Logfile of failure stored in:
/build/isar_wzh_devel_7b/28/build/tmp/work/debian-bookworm-amd64/isar-image-ci-qemuamd64-iso/1.0-r0/temp/log.do_image_squashfs.1864458
dpkg-query: no packages found matching systemd-boot-efi:amd64
Failed test case: citest.py:SstateTest.test_sstate_populate:
ERROR: Logfile of failure stored in:
/build/isar_wzh_devel_7b/28/build-sstate/tmp/work/debian-bullseye-amd64/isar-image-base-qemuamd64/1.0-r0/temp/log.do_image_wic.1858017
bwrap: Can't find source path
/build/isar_wzh_devel_7b/28/build-sstate/tmp/deploy/sbom/sbom-chroot: No
such file or directory
Failed test case: citest.py:SstateTest.test_sstate:
ERROR: Logfile of failure stored in:
/build/isar_wzh_devel_7b/28/build-sstate/tmp/work/debian-bullseye-amd64/isar-image-base-qemuamd64/1.0-r0/temp/log.do_image_wic.1943793
[stdlog] 2025-12-05 05:40:29,889 avocado.test cibuilder L0321 INFO | |
Traceback (most recent call last):
[stdlog] 2025-12-05 05:40:29,889 avocado.test cibuilder L0321 INFO | |
File "/build/isar_wzh_devel_7b/28/scripts/wic", line 547, in <module>
[stdlog] 2025-12-05 05:40:29,889 avocado.test cibuilder L0321 INFO | |
sys.exit(main(sys.argv[1:]))
[stdlog] 2025-12-05 05:40:29,889 avocado.test cibuilder L0321 INFO | |
File "/build/isar_wzh_devel_7b/28/scripts/wic", line 542, in main
[stdlog] 2025-12-05 05:40:29,889 avocado.test cibuilder L0321 INFO | |
return hlp.invoke_subcommand(args, parser, hlp.wic_help_usage,
subcommands)
[stdlog] 2025-12-05 05:40:29,889 avocado.test cibuilder L0321 INFO | |
File "/build/isar_wzh_devel_7b/28/scripts/lib/wic/help.py", line 83,
in invoke_subcommand
[stdlog] 2025-12-05 05:40:29,889 avocado.test cibuilder L0321 INFO | |
subcmd[0](args, usage)
[stdlog] 2025-12-05 05:40:29,889 avocado.test cibuilder L0321 INFO | |
File "/build/isar_wzh_devel_7b/28/scripts/wic", line 222, in
wic_create_subcommand
[stdlog] 2025-12-05 05:40:29,889 avocado.test cibuilder L0321 INFO | |
engine.wic_create(wks_file, rootfs_dir, bootimg_dir, kernel_dir,
[stdlog] 2025-12-05 05:40:29,889 avocado.test cibuilder L0321 INFO | |
File "/build/isar_wzh_devel_7b/28/scripts/lib/wic/engine.py", line
183, in wic_create
[stdlog] 2025-12-05 05:40:29,889 avocado.test cibuilder L0321 INFO | |
plugin_class = PluginMgr.get_plugins('imager').get(pname)
[stdlog] 2025-12-05 05:40:29,889 avocado.test cibuilder L0321 INFO | |
File "/build/isar_wzh_devel_7b/28/scripts/lib/wic/pluginbase.py", line
61, in get_plugins
[stdlog] 2025-12-05 05:40:29,889 avocado.test cibuilder L0321 INFO | |
spec.loader.exec_module(module)
[stdlog] 2025-12-05 05:40:29,889 avocado.test cibuilder L0321 INFO | |
File "<frozen importlib._bootstrap_external>", line 790, in exec_module
[stdlog] 2025-12-05 05:40:29,889 avocado.test cibuilder L0321 INFO | |
File "<frozen importlib._bootstrap>", line 228, in
_call_with_frames_removed
[stdlog] 2025-12-05 05:40:29,889 avocado.test cibuilder L0321 INFO | |
File
"/build/isar_wzh_devel_7b/28/scripts/lib/wic/plugins/imager/direct.py",
line 22, in <module>
[stdlog] 2025-12-05 05:40:29,889 avocado.test cibuilder L0321 INFO | |
from oe.path import copyhardlinktree
[stdlog] 2025-12-05 05:40:29,889 avocado.test cibuilder L0321 INFO | |
ModuleNotFoundError: No module named 'oe'
[stdlog] 2025-12-05 05:40:29,889 avocado.test cibuilder L0321 INFO |
NOTE: recipe isar-image-base-1.0-r0: task do_image_wic: Failed
[stdlog] 2025-12-05 05:40:29,889 avocado.test cibuilder L0321 INFO |
NOTE: Tasks Summary: Attempted 170 tasks of which 138 didn't need to be
rerun and 1 failed.
Zhihang
On 12/1/25 09:58, 'Felix Moessbauer' via isar-users wrote:
> This patchset adds proper SBOM generation in the two standard formats
> SPDX and CycloneDX during the rootfs generation process.
>
> The generation is itself is handled by a SBOM generator `debsbom` [1]
> which is developed as an open source project at Siemens. It is still
> early in development, but it has enough features for what we require
> in isar. The required dependencies which are not yet available as
> Debian packages were minimally packaged directly in isar too.
>
> This is a followup of the previous RFC [2]. Since then the series has
> changed a lot. The SBOM generation was moved from a simple OE lib to
> `debsbom`. This also meant the introduction of a separate chroot was
> necessary. The SBOM generation process was also moved from the image
> step to the rootfs step, along with a lot of minor changes and
> improvements.
>
> [1] https://github.com/siemens/debsbom
> [2] https://groups.google.com/g/isar-users/c/8L-CF4BJY0I/m/p0N3o_zfAAAJ
>
> Changes since v5:
>
> - fix isar-image-ci on qemuamd64-bullseye (set IMAGER_BOM according to
> machine changes made in image file)
> - rebased onto next
>
> Changes since v4:
>
> - rebased onto next
> - fix race condition on creation of ${DEPLOY_DIR_SBOM} (aka ${DEPLOY_DIR_IMAGE})
>
> Changes since v3:
>
> - fix issue on external bullseye initramfs (we now disable sbom generation
> on all unsupported distros rootfs instances)
> - update debsbom to v0.4.0
> - rebased onto next
>
> Changes since v2:
>
> - fix issues when HOST_ARCH != DISTRO_ARCH on derived distributions
> - update debsbom to v0.3.0, which fixes the Origin: bug reported in v2
> - generate SBOM for imager as well and create merged sbom of .wic image
> - resend imager manifest + wic manifest patches to reduce conflicts
>
> Note, that the patches p1-p5 are most important as they add basic SBOM
> support. The remaining patches address the imager + .wic bom part,
> which also can be merged later on.
>
> Changes since v1:
>
> - remove tarball
> - refactor packaging (auto-derive python dependencies)
> - only build missing packages (varies on bookworm, trixie, noble)
> - add ubuntu support
> - only generate sboms for supported distributions (bookworm/jammy and
> onwards)
> - update debsbom (includes bug fixes and more information for source
> packages)
>
>
> Christoph Steiger (3):
> meta: package python libraries for SBOM generation
> meta: package python3-debsbom
> meta: add SBOM generation with debsbom
>
> Felix Moessbauer (7):
> refactor: move get_rootfs_distro from sdk into rootfs
> override distro vendor in SBOM on Ubuntu
> add support to add imager dependencies to BOM
> wic: create uniform manifest describing all image components
> qemuamd64: add IMAGER_BOM entries
> imager: create SBOM of IMAGER_BOM packages
> wic: create uniform SBOM describing all image components
>
> doc/user_manual.md | 1 +
> meta-isar/conf/distro/ubuntu-common.inc | 2 +
> meta-isar/conf/machine/qemuamd64.conf | 1 +
> .../recipes-core/images/isar-image-ci.bb | 1 +
> meta/classes/image-tools-extension.bbclass | 29 +++++++++
> meta/classes/image.bbclass | 7 ++
> meta/classes/imagetypes_wic.bbclass | 30 +++++++++
> meta/classes/initramfs.bbclass | 3 +-
> meta/classes/rootfs.bbclass | 23 ++++++-
> meta/classes/sbom.bbclass | 65 +++++++++++++++++++
> meta/classes/sdk.bbclass | 10 +--
> .../sbom-chroot/sbom-chroot.bb | 30 +++++++++
> .../python3-beartype/files/rules | 8 +++
> .../python3-beartype_0.19.0.bb | 29 +++++++++
> .../files/pybuild.testfiles | 1 +
> .../python3-cyclonedx-lib/files/rules | 8 +++
> .../python3-cyclonedx-lib_9.1.0.bb | 48 ++++++++++++++
> ...icense-description-in-pyproject.toml.patch | 28 ++++++++
> .../python3-debsbom/files/rules | 8 +++
> .../python3-debsbom/python3-debsbom_0.4.0.bb | 45 +++++++++++++
> .../python3-packageurl/files/rules | 8 +++
> .../python3-packageurl_0.16.0.bb | 33 ++++++++++
> .../python3-py-serializable/files/rules | 8 +++
> .../python3-py-serializable_2.0.0.bb | 38 +++++++++++
> .../python3-spdx-tools/files/rules | 25 +++++++
> .../python3-spdx-tools_0.8.3.bb | 46 +++++++++++++
> 26 files changed, 524 insertions(+), 11 deletions(-)
> create mode 100644 meta/classes/sbom.bbclass
> create mode 100644 meta/recipes-devtools/sbom-chroot/sbom-chroot.bb
> create mode 100644 meta/recipes-support/python3-beartype/files/rules
> create mode 100644 meta/recipes-support/python3-beartype/python3-beartype_0.19.0.bb
> create mode 100644 meta/recipes-support/python3-cyclonedx-lib/files/pybuild.testfiles
> create mode 100644 meta/recipes-support/python3-cyclonedx-lib/files/rules
> create mode 100644 meta/recipes-support/python3-cyclonedx-lib/python3-cyclonedx-lib_9.1.0.bb
> create mode 100644 meta/recipes-support/python3-debsbom/files/0001-Use-old-license-description-in-pyproject.toml.patch
> create mode 100644 meta/recipes-support/python3-debsbom/files/rules
> create mode 100644 meta/recipes-support/python3-debsbom/python3-debsbom_0.4.0.bb
> create mode 100644 meta/recipes-support/python3-packageurl/files/rules
> create mode 100644 meta/recipes-support/python3-packageurl/python3-packageurl_0.16.0.bb
> create mode 100644 meta/recipes-support/python3-py-serializable/files/rules
> create mode 100644 meta/recipes-support/python3-py-serializable/python3-py-serializable_2.0.0.bb
> create mode 100644 meta/recipes-support/python3-spdx-tools/files/rules
> create mode 100644 meta/recipes-support/python3-spdx-tools/python3-spdx-tools_0.8.3.bb
>
On Tue, 2025-12-09 at 15:21 +0100, Zhihang Wei wrote: > Hi, > There are mulltiple test cases failed on CI. Hi, that's very unfortunate. As I'm not able to run the CI, for me it is basically impossible to catch these corner cases (especially the ones in the isar-image-ci, which is somewhat special). This topic also has been extensively discussed at the isar community meetup and identified as a major blocker for contributions. To move forward, we need a different strategy: Patches 1-5 are probably not breaking things. Please consider a partial apply of the series. The series is written in a way that it can be applied linearly. We need a way to get quick feedback regarding which CI tests break. With these one-per-week feedback rounds where other patches are applied in-between, I don't see a chance of getting the sbom feature integrated at all. Finally, we need to reduce the runtime of the testsuite and make it possible to easily execute just the failed tests. The testsuite wrapper script from Cedric could help in this regard. > > > On fast CI: > > Failed test case: citest.py:DevTest.test_dev > ERROR: Logfile of failure stored in: > /build/isar_wzh_devel_5b_fast/4/build/tmp/work/debian-bookworm-amd64/isar-image-ci-qemuamd64/1.0-r0/temp/log.do_image_wic.3897481 > dpkg-query: no packages found matching systemd-boot-efi:amd64 > > Failed test case: citest.py:DevTest.test_dev_apps > ERROR: Logfile of failure stored in: > /build/isar_wzh_devel_5b_fast/4/build/tmp/work/debian-bookworm-amd64/isar-image-ci-qemuamd64/1.0-r0/temp/log.do_image_wic.18529 > dpkg-query: no packages found matching systemd-boot-efi:amd64 > > Failed test case: citest.py:DevTest.test_dev_rebuild > ERROR: Logfile of failure stored in: > /build/isar_wzh_devel_5b_fast/4/build/tmp/work/debian-bookworm-amd64/isar-image-ci-qemuamd64/1.0-r0/temp/log.do_image_wic.262594 > dpkg-query: no packages found matching systemd-boot-efi:amd64 > > Failed test case: citest.py:CrossTest.test_cross_debsrc > ERROR: Logfile of failure stored in: > /build/isar_wzh_devel_5b_fast/4/build/tmp/work/debian-bookworm-arm64/isar-image-ci-qemuarm64/1.0-r0/temp/log.do_image_wic.3057235 > dpkg-query: no packages found matching systemd-boot-efi:arm64 > > Failed test case: citest.py:CrossTest.test_cross_dependencies > ERROR: Logfile of failure stored in: > /build/isar_wzh_devel_5b_fast/4/build/tmp/work/debian-bookworm-arm64/isar-image-ci-qemuarm64/1.0-r0/temp/log.do_image_wic.772494 > dpkg-query: no packages found matching systemd-boot-efi:arm64 This class of failures seems to be related to the anti-pattern of overwriting machine configs in an image recipe. I'll work on a fix. > > > On full CI: > > Failed test case: citest.py:ReproTest.test_repro_unsigned: > ERROR: Logfile of failure stored in: > /build/isar_wzh_devel_7b/28/build/tmp/work/debian-bookworm-amd64/isar-image-base-qemuamd64/1.0-r0/temp/log.do_image_wic.862703 > bwrap: Can't mkdir /mnt/rootfs: Permission denied > I assume this is related with CI environment. This error is also there > for v5. Possible. I never saw it before. > > Failed test case: citest.py:InstallerTest.test_installer_build: > ERROR: Logfile of failure stored in: > /build/isar_wzh_devel_7b/28/build/tmp/work/debian-bookworm-amd64/isar-image-ci-qemuamd64/1.0-r0/temp/log.do_image_wic.1240668 > dpkg-query: no packages found matching systemd-boot-efi:amd64 > > Failed test case: citest.py:KernelTests.test_per_kernel: > Logfile of failure stored in: > /build/isar_wzh_devel_7b/28/build/tmp/work/debian-bookworm-arm64/isar-image-ci-qemuarm64/1.0-r0/temp/log.do_image_wic.1626092 > dpkg-query: no packages found matching systemd-boot-efi:arm64 > > Failed test case: citest.py:InitRdTest.test_dracut_in_image: > ERROR: Logfile of failure stored in: > /build/isar_wzh_devel_7b/28/build/tmp/work/debian-bookworm-amd64/isar-image-ci-qemuamd64/1.0-r0/temp/log.do_image_wic.1723289 > dpkg-query: no packages found matching systemd-boot-efi:amd64 > > Failed test case: citest.py:InitRdCrossTests.test_dracut_in_image: > ERROR: Logfile of failure stored in: > /build/isar_wzh_devel_7b/28/build/tmp/work/debian-bookworm-amd64/isar-image-ci-qemuamd64/1.0-r0/temp/log.do_image_wic.2466233 > dpkg-query: no packages found matching systemd-boot-efi:amd64 > > Failed test case: citest.py:WicTest.test_wic_nodeploy_partitions: > ERROR: Logfile of failure stored in: > /build/isar_wzh_devel_7b/28/build/tmp/work/debian-bookworm-arm64/isar-image-ci-qemuarm64/1.0-r0/temp/log.do_image_wic.2823952 > dpkg-query: no packages found matching systemd-boot-efi:arm64 > > Failed test case: citest.py:WicTest.test_wic_deploy_partitions: > ERROR: Logfile of failure stored in: > /build/isar_wzh_devel_7b/28/build/tmp/work/debian-bookworm-arm64/isar-image-ci-qemuarm64/1.0-r0/temp/log.do_image_wic.2892279 > dpkg-query: no packages found matching systemd-boot-efi:arm64 > > 07:06:42 > 18-/build/isar_wzh_devel_7b/28/testsuite/citest.py:NoCrossTest.test_nocross: > ERROR: Logfile of failure stored in: > /build/isar_wzh_devel_7b/28/build/tmp/work/debian-bookworm-amd64/isar-image-ci-qemuamd64-iso/1.0-r0/temp/log.do_image_squashfs.1864458 > dpkg-query: no packages found matching systemd-boot-efi:amd64 > > Failed test case: citest.py:SstateTest.test_sstate_populate: > ERROR: Logfile of failure stored in: > /build/isar_wzh_devel_7b/28/build-sstate/tmp/work/debian-bullseye-amd64/isar-image-base-qemuamd64/1.0-r0/temp/log.do_image_wic.1858017 > bwrap: Can't find source path > /build/isar_wzh_devel_7b/28/build-sstate/tmp/deploy/sbom/sbom-chroot: No > such file or directory On bullseye there should not be any sbom-chroot. I'm wondering why it even is executed. Investigating... Felix
On 12/9/25 15:49, MOESSBAUER, Felix wrote: > On Tue, 2025-12-09 at 15:21 +0100, Zhihang Wei wrote: >> Hi, >> There are mulltiple test cases failed on CI. > Hi, > > that's very unfortunate. As I'm not able to run the CI, for me it is > basically impossible to catch these corner cases (especially the ones > in the isar-image-ci, which is somewhat special). This topic also has > been extensively discussed at the isar community meetup and identified > as a major blocker for contributions. > > To move forward, we need a different strategy: > > Patches 1-5 are probably not breaking things. Please consider a partial > apply of the series. The series is written in a way that it can be > applied linearly. I'll put p1-5 on CI and get back to you tomorrow. Zhihang > We need a way to get quick feedback regarding which CI tests break. > With these one-per-week feedback rounds where other patches are applied > in-between, I don't see a chance of getting the sbom feature integrated > at all. > > Finally, we need to reduce the runtime of the testsuite and make it > possible to easily execute just the failed tests. The testsuite wrapper > script from Cedric could help in this regard. > >> >> On fast CI: >> >> Failed test case: citest.py:DevTest.test_dev >> ERROR: Logfile of failure stored in: >> /build/isar_wzh_devel_5b_fast/4/build/tmp/work/debian-bookworm-amd64/isar-image-ci-qemuamd64/1.0-r0/temp/log.do_image_wic.3897481 >> dpkg-query: no packages found matching systemd-boot-efi:amd64 >> >> Failed test case: citest.py:DevTest.test_dev_apps >> ERROR: Logfile of failure stored in: >> /build/isar_wzh_devel_5b_fast/4/build/tmp/work/debian-bookworm-amd64/isar-image-ci-qemuamd64/1.0-r0/temp/log.do_image_wic.18529 >> dpkg-query: no packages found matching systemd-boot-efi:amd64 >> >> Failed test case: citest.py:DevTest.test_dev_rebuild >> ERROR: Logfile of failure stored in: >> /build/isar_wzh_devel_5b_fast/4/build/tmp/work/debian-bookworm-amd64/isar-image-ci-qemuamd64/1.0-r0/temp/log.do_image_wic.262594 >> dpkg-query: no packages found matching systemd-boot-efi:amd64 >> >> Failed test case: citest.py:CrossTest.test_cross_debsrc >> ERROR: Logfile of failure stored in: >> /build/isar_wzh_devel_5b_fast/4/build/tmp/work/debian-bookworm-arm64/isar-image-ci-qemuarm64/1.0-r0/temp/log.do_image_wic.3057235 >> dpkg-query: no packages found matching systemd-boot-efi:arm64 >> >> Failed test case: citest.py:CrossTest.test_cross_dependencies >> ERROR: Logfile of failure stored in: >> /build/isar_wzh_devel_5b_fast/4/build/tmp/work/debian-bookworm-arm64/isar-image-ci-qemuarm64/1.0-r0/temp/log.do_image_wic.772494 >> dpkg-query: no packages found matching systemd-boot-efi:arm64 > This class of failures seems to be related to the anti-pattern of > overwriting machine configs in an image recipe. I'll work on a fix. > >> >> On full CI: >> >> Failed test case: citest.py:ReproTest.test_repro_unsigned: >> ERROR: Logfile of failure stored in: >> /build/isar_wzh_devel_7b/28/build/tmp/work/debian-bookworm-amd64/isar-image-base-qemuamd64/1.0-r0/temp/log.do_image_wic.862703 >> bwrap: Can't mkdir /mnt/rootfs: Permission denied >> I assume this is related with CI environment. This error is also there >> for v5. > Possible. I never saw it before. > >> Failed test case: citest.py:InstallerTest.test_installer_build: >> ERROR: Logfile of failure stored in: >> /build/isar_wzh_devel_7b/28/build/tmp/work/debian-bookworm-amd64/isar-image-ci-qemuamd64/1.0-r0/temp/log.do_image_wic.1240668 >> dpkg-query: no packages found matching systemd-boot-efi:amd64 >> >> Failed test case: citest.py:KernelTests.test_per_kernel: >> Logfile of failure stored in: >> /build/isar_wzh_devel_7b/28/build/tmp/work/debian-bookworm-arm64/isar-image-ci-qemuarm64/1.0-r0/temp/log.do_image_wic.1626092 >> dpkg-query: no packages found matching systemd-boot-efi:arm64 >> >> Failed test case: citest.py:InitRdTest.test_dracut_in_image: >> ERROR: Logfile of failure stored in: >> /build/isar_wzh_devel_7b/28/build/tmp/work/debian-bookworm-amd64/isar-image-ci-qemuamd64/1.0-r0/temp/log.do_image_wic.1723289 >> dpkg-query: no packages found matching systemd-boot-efi:amd64 >> >> Failed test case: citest.py:InitRdCrossTests.test_dracut_in_image: >> ERROR: Logfile of failure stored in: >> /build/isar_wzh_devel_7b/28/build/tmp/work/debian-bookworm-amd64/isar-image-ci-qemuamd64/1.0-r0/temp/log.do_image_wic.2466233 >> dpkg-query: no packages found matching systemd-boot-efi:amd64 >> >> Failed test case: citest.py:WicTest.test_wic_nodeploy_partitions: >> ERROR: Logfile of failure stored in: >> /build/isar_wzh_devel_7b/28/build/tmp/work/debian-bookworm-arm64/isar-image-ci-qemuarm64/1.0-r0/temp/log.do_image_wic.2823952 >> dpkg-query: no packages found matching systemd-boot-efi:arm64 >> >> Failed test case: citest.py:WicTest.test_wic_deploy_partitions: >> ERROR: Logfile of failure stored in: >> /build/isar_wzh_devel_7b/28/build/tmp/work/debian-bookworm-arm64/isar-image-ci-qemuarm64/1.0-r0/temp/log.do_image_wic.2892279 >> dpkg-query: no packages found matching systemd-boot-efi:arm64 >> >> 07:06:42 >> 18-/build/isar_wzh_devel_7b/28/testsuite/citest.py:NoCrossTest.test_nocross: >> ERROR: Logfile of failure stored in: >> /build/isar_wzh_devel_7b/28/build/tmp/work/debian-bookworm-amd64/isar-image-ci-qemuamd64-iso/1.0-r0/temp/log.do_image_squashfs.1864458 >> dpkg-query: no packages found matching systemd-boot-efi:amd64 >> >> Failed test case: citest.py:SstateTest.test_sstate_populate: >> ERROR: Logfile of failure stored in: >> /build/isar_wzh_devel_7b/28/build-sstate/tmp/work/debian-bullseye-amd64/isar-image-base-qemuamd64/1.0-r0/temp/log.do_image_wic.1858017 >> bwrap: Can't find source path >> /build/isar_wzh_devel_7b/28/build-sstate/tmp/deploy/sbom/sbom-chroot: No >> such file or directory > On bullseye there should not be any sbom-chroot. I'm wondering why it > even is executed. Investigating... > > Felix >
p1-5 were applied to next, thanks. Zhihang On 12/1/25 09:58, 'Felix Moessbauer' via isar-users wrote: > This patchset adds proper SBOM generation in the two standard formats > SPDX and CycloneDX during the rootfs generation process. > > The generation is itself is handled by a SBOM generator `debsbom` [1] > which is developed as an open source project at Siemens. It is still > early in development, but it has enough features for what we require > in isar. The required dependencies which are not yet available as > Debian packages were minimally packaged directly in isar too. > > This is a followup of the previous RFC [2]. Since then the series has > changed a lot. The SBOM generation was moved from a simple OE lib to > `debsbom`. This also meant the introduction of a separate chroot was > necessary. The SBOM generation process was also moved from the image > step to the rootfs step, along with a lot of minor changes and > improvements. > > [1] https://github.com/siemens/debsbom > [2] https://groups.google.com/g/isar-users/c/8L-CF4BJY0I/m/p0N3o_zfAAAJ > > Changes since v5: > > - fix isar-image-ci on qemuamd64-bullseye (set IMAGER_BOM according to > machine changes made in image file) > - rebased onto next > > Changes since v4: > > - rebased onto next > - fix race condition on creation of ${DEPLOY_DIR_SBOM} (aka ${DEPLOY_DIR_IMAGE}) > > Changes since v3: > > - fix issue on external bullseye initramfs (we now disable sbom generation > on all unsupported distros rootfs instances) > - update debsbom to v0.4.0 > - rebased onto next > > Changes since v2: > > - fix issues when HOST_ARCH != DISTRO_ARCH on derived distributions > - update debsbom to v0.3.0, which fixes the Origin: bug reported in v2 > - generate SBOM for imager as well and create merged sbom of .wic image > - resend imager manifest + wic manifest patches to reduce conflicts > > Note, that the patches p1-p5 are most important as they add basic SBOM > support. The remaining patches address the imager + .wic bom part, > which also can be merged later on. > > Changes since v1: > > - remove tarball > - refactor packaging (auto-derive python dependencies) > - only build missing packages (varies on bookworm, trixie, noble) > - add ubuntu support > - only generate sboms for supported distributions (bookworm/jammy and > onwards) > - update debsbom (includes bug fixes and more information for source > packages) > > > Christoph Steiger (3): > meta: package python libraries for SBOM generation > meta: package python3-debsbom > meta: add SBOM generation with debsbom > > Felix Moessbauer (7): > refactor: move get_rootfs_distro from sdk into rootfs > override distro vendor in SBOM on Ubuntu > add support to add imager dependencies to BOM > wic: create uniform manifest describing all image components > qemuamd64: add IMAGER_BOM entries > imager: create SBOM of IMAGER_BOM packages > wic: create uniform SBOM describing all image components > > doc/user_manual.md | 1 + > meta-isar/conf/distro/ubuntu-common.inc | 2 + > meta-isar/conf/machine/qemuamd64.conf | 1 + > .../recipes-core/images/isar-image-ci.bb | 1 + > meta/classes/image-tools-extension.bbclass | 29 +++++++++ > meta/classes/image.bbclass | 7 ++ > meta/classes/imagetypes_wic.bbclass | 30 +++++++++ > meta/classes/initramfs.bbclass | 3 +- > meta/classes/rootfs.bbclass | 23 ++++++- > meta/classes/sbom.bbclass | 65 +++++++++++++++++++ > meta/classes/sdk.bbclass | 10 +-- > .../sbom-chroot/sbom-chroot.bb | 30 +++++++++ > .../python3-beartype/files/rules | 8 +++ > .../python3-beartype_0.19.0.bb | 29 +++++++++ > .../files/pybuild.testfiles | 1 + > .../python3-cyclonedx-lib/files/rules | 8 +++ > .../python3-cyclonedx-lib_9.1.0.bb | 48 ++++++++++++++ > ...icense-description-in-pyproject.toml.patch | 28 ++++++++ > .../python3-debsbom/files/rules | 8 +++ > .../python3-debsbom/python3-debsbom_0.4.0.bb | 45 +++++++++++++ > .../python3-packageurl/files/rules | 8 +++ > .../python3-packageurl_0.16.0.bb | 33 ++++++++++ > .../python3-py-serializable/files/rules | 8 +++ > .../python3-py-serializable_2.0.0.bb | 38 +++++++++++ > .../python3-spdx-tools/files/rules | 25 +++++++ > .../python3-spdx-tools_0.8.3.bb | 46 +++++++++++++ > 26 files changed, 524 insertions(+), 11 deletions(-) > create mode 100644 meta/classes/sbom.bbclass > create mode 100644 meta/recipes-devtools/sbom-chroot/sbom-chroot.bb > create mode 100644 meta/recipes-support/python3-beartype/files/rules > create mode 100644 meta/recipes-support/python3-beartype/python3-beartype_0.19.0.bb > create mode 100644 meta/recipes-support/python3-cyclonedx-lib/files/pybuild.testfiles > create mode 100644 meta/recipes-support/python3-cyclonedx-lib/files/rules > create mode 100644 meta/recipes-support/python3-cyclonedx-lib/python3-cyclonedx-lib_9.1.0.bb > create mode 100644 meta/recipes-support/python3-debsbom/files/0001-Use-old-license-description-in-pyproject.toml.patch > create mode 100644 meta/recipes-support/python3-debsbom/files/rules > create mode 100644 meta/recipes-support/python3-debsbom/python3-debsbom_0.4.0.bb > create mode 100644 meta/recipes-support/python3-packageurl/files/rules > create mode 100644 meta/recipes-support/python3-packageurl/python3-packageurl_0.16.0.bb > create mode 100644 meta/recipes-support/python3-py-serializable/files/rules > create mode 100644 meta/recipes-support/python3-py-serializable/python3-py-serializable_2.0.0.bb > create mode 100644 meta/recipes-support/python3-spdx-tools/files/rules > create mode 100644 meta/recipes-support/python3-spdx-tools/python3-spdx-tools_0.8.3.bb >
This patchset adds proper SBOM generation in the two standard formats SPDX and CycloneDX during the rootfs generation process. The generation is itself is handled by a SBOM generator `debsbom` [1] which is developed as an open source project at Siemens. It is still early in development, but it has enough features for what we require in isar. The required dependencies which are not yet available as Debian packages were minimally packaged directly in isar too. This is a followup of the previous RFC [2]. Since then the series has changed a lot. The SBOM generation was moved from a simple OE lib to `debsbom`. This also meant the introduction of a separate chroot was necessary. The SBOM generation process was also moved from the image step to the rootfs step, along with a lot of minor changes and improvements. [1] https://github.com/siemens/debsbom [2] https://groups.google.com/g/isar-users/c/8L-CF4BJY0I/m/p0N3o_zfAAAJ Changes since v5: - fix isar-image-ci on qemuamd64-bullseye (set IMAGER_BOM according to machine changes made in image file) - rebased onto next Changes since v4: - rebased onto next - fix race condition on creation of ${DEPLOY_DIR_SBOM} (aka ${DEPLOY_DIR_IMAGE}) Changes since v3: - fix issue on external bullseye initramfs (we now disable sbom generation on all unsupported distros rootfs instances) - update debsbom to v0.4.0 - rebased onto next Changes since v2: - fix issues when HOST_ARCH != DISTRO_ARCH on derived distributions - update debsbom to v0.3.0, which fixes the Origin: bug reported in v2 - generate SBOM for imager as well and create merged sbom of .wic image - resend imager manifest + wic manifest patches to reduce conflicts Note, that the patches p1-p5 are most important as they add basic SBOM support. The remaining patches address the imager + .wic bom part, which also can be merged later on. Changes since v1: - remove tarball - refactor packaging (auto-derive python dependencies) - only build missing packages (varies on bookworm, trixie, noble) - add ubuntu support - only generate sboms for supported distributions (bookworm/jammy and onwards) - update debsbom (includes bug fixes and more information for source packages) Christoph Steiger (3): meta: package python libraries for SBOM generation meta: package python3-debsbom meta: add SBOM generation with debsbom Felix Moessbauer (7): refactor: move get_rootfs_distro from sdk into rootfs override distro vendor in SBOM on Ubuntu add support to add imager dependencies to BOM wic: create uniform manifest describing all image components qemuamd64: add IMAGER_BOM entries imager: create SBOM of IMAGER_BOM packages wic: create uniform SBOM describing all image components doc/user_manual.md | 1 + meta-isar/conf/distro/ubuntu-common.inc | 2 + meta-isar/conf/machine/qemuamd64.conf | 1 + .../recipes-core/images/isar-image-ci.bb | 1 + meta/classes/image-tools-extension.bbclass | 29 +++++++++ meta/classes/image.bbclass | 7 ++ meta/classes/imagetypes_wic.bbclass | 30 +++++++++ meta/classes/initramfs.bbclass | 3 +- meta/classes/rootfs.bbclass | 23 ++++++- meta/classes/sbom.bbclass | 65 +++++++++++++++++++ meta/classes/sdk.bbclass | 10 +-- .../sbom-chroot/sbom-chroot.bb | 30 +++++++++ .../python3-beartype/files/rules | 8 +++ .../python3-beartype_0.19.0.bb | 29 +++++++++ .../files/pybuild.testfiles | 1 + .../python3-cyclonedx-lib/files/rules | 8 +++ .../python3-cyclonedx-lib_9.1.0.bb | 48 ++++++++++++++ ...icense-description-in-pyproject.toml.patch | 28 ++++++++ .../python3-debsbom/files/rules | 8 +++ .../python3-debsbom/python3-debsbom_0.4.0.bb | 45 +++++++++++++ .../python3-packageurl/files/rules | 8 +++ .../python3-packageurl_0.16.0.bb | 33 ++++++++++ .../python3-py-serializable/files/rules | 8 +++ .../python3-py-serializable_2.0.0.bb | 38 +++++++++++ .../python3-spdx-tools/files/rules | 25 +++++++ .../python3-spdx-tools_0.8.3.bb | 46 +++++++++++++ 26 files changed, 524 insertions(+), 11 deletions(-) create mode 100644 meta/classes/sbom.bbclass create mode 100644 meta/recipes-devtools/sbom-chroot/sbom-chroot.bb create mode 100644 meta/recipes-support/python3-beartype/files/rules create mode 100644 meta/recipes-support/python3-beartype/python3-beartype_0.19.0.bb create mode 100644 meta/recipes-support/python3-cyclonedx-lib/files/pybuild.testfiles create mode 100644 meta/recipes-support/python3-cyclonedx-lib/files/rules create mode 100644 meta/recipes-support/python3-cyclonedx-lib/python3-cyclonedx-lib_9.1.0.bb create mode 100644 meta/recipes-support/python3-debsbom/files/0001-Use-old-license-description-in-pyproject.toml.patch create mode 100644 meta/recipes-support/python3-debsbom/files/rules create mode 100644 meta/recipes-support/python3-debsbom/python3-debsbom_0.4.0.bb create mode 100644 meta/recipes-support/python3-packageurl/files/rules create mode 100644 meta/recipes-support/python3-packageurl/python3-packageurl_0.16.0.bb create mode 100644 meta/recipes-support/python3-py-serializable/files/rules create mode 100644 meta/recipes-support/python3-py-serializable/python3-py-serializable_2.0.0.bb create mode 100644 meta/recipes-support/python3-spdx-tools/files/rules create mode 100644 meta/recipes-support/python3-spdx-tools/python3-spdx-tools_0.8.3.bb