mbox series

[0/5] Add iso9660 image support for ISAR

Message ID 20250121101252.324899-1-felix.moessbauer@siemens.com
Headers show
Series Add iso9660 image support for ISAR | expand

Message

Felix Moessbauer Jan. 21, 2025, 10:12 a.m. UTC
This series adds support to create hybrid iso9660 images for ISAR
which can be used as live images for booting from USB sticks or
DVDs. Currently, only syslinux and EFI (grub) bootloaders are supported.

Best regards,
Felix Moessbauer

Felix Moessbauer (5):
  Import isoimage-isohybrid from oe
  wic: add ISAR version of isohybrid-efi plugin
  move squashfs imagetype class from CIP to ISAR
  add qemuamd64 target for hybrid iso9660 image
  add kas menu target for qemuamd64-iso machine

 kas/machine/Kconfig                           |  14 +
 kas/machine/qemuamd64-iso.yaml                |   7 +
 meta-isar/classes/squashfs.bbclass            |  48 ++
 meta-isar/conf/machine/qemuamd64-iso.conf     |  29 +
 .../multiconfig/qemuamd64-iso-bookworm.conf   |   4 +
 .../lib/wic/canned-wks/isohybrid-efi.wks.in   |   2 +
 meta/classes/image.bbclass                    |   2 +-
 .../plugins/source/isoimage-isohybrid-isar.py | 535 ++++++++++++++++++
 8 files changed, 640 insertions(+), 1 deletion(-)
 create mode 100644 kas/machine/qemuamd64-iso.yaml
 create mode 100644 meta-isar/classes/squashfs.bbclass
 create mode 100644 meta-isar/conf/machine/qemuamd64-iso.conf
 create mode 100644 meta-isar/conf/multiconfig/qemuamd64-iso-bookworm.conf
 create mode 100644 meta-isar/scripts/lib/wic/canned-wks/isohybrid-efi.wks.in
 create mode 100644 meta/scripts/lib/wic/plugins/source/isoimage-isohybrid-isar.py

Comments

Jan Kiszka Jan. 21, 2025, 10:24 a.m. UTC | #1
On 21.01.25 11:12, Felix Moessbauer wrote:
> This series adds support to create hybrid iso9660 images for ISAR
> which can be used as live images for booting from USB sticks or
> DVDs. Currently, only syslinux and EFI (grub) bootloaders are supported.
> 
> Best regards,
> Felix Moessbauer
> 
> Felix Moessbauer (5):
>   Import isoimage-isohybrid from oe
>   wic: add ISAR version of isohybrid-efi plugin
>   move squashfs imagetype class from CIP to ISAR
>   add qemuamd64 target for hybrid iso9660 image
>   add kas menu target for qemuamd64-iso machine
> 
>  kas/machine/Kconfig                           |  14 +
>  kas/machine/qemuamd64-iso.yaml                |   7 +
>  meta-isar/classes/squashfs.bbclass            |  48 ++
>  meta-isar/conf/machine/qemuamd64-iso.conf     |  29 +
>  .../multiconfig/qemuamd64-iso-bookworm.conf   |   4 +
>  .../lib/wic/canned-wks/isohybrid-efi.wks.in   |   2 +
>  meta/classes/image.bbclass                    |   2 +-
>  .../plugins/source/isoimage-isohybrid-isar.py | 535 ++++++++++++++++++
>  8 files changed, 640 insertions(+), 1 deletion(-)
>  create mode 100644 kas/machine/qemuamd64-iso.yaml
>  create mode 100644 meta-isar/classes/squashfs.bbclass
>  create mode 100644 meta-isar/conf/machine/qemuamd64-iso.conf
>  create mode 100644 meta-isar/conf/multiconfig/qemuamd64-iso-bookworm.conf
>  create mode 100644 meta-isar/scripts/lib/wic/canned-wks/isohybrid-efi.wks.in
>  create mode 100644 meta/scripts/lib/wic/plugins/source/isoimage-isohybrid-isar.py
> 

I suspect we still missing a hook-up into the isar test suite, aren't we?

Jan
Felix Moessbauer Jan. 21, 2025, 11:42 a.m. UTC | #2
On Tue, 2025-01-21 at 11:24 +0100, Jan Kiszka wrote:
> On 21.01.25 11:12, Felix Moessbauer wrote:
> > This series adds support to create hybrid iso9660 images for ISAR
> > which can be used as live images for booting from USB sticks or
> > DVDs. Currently, only syslinux and EFI (grub) bootloaders are
> > supported.
> > 
> > Best regards,
> > Felix Moessbauer
> > 
> > Felix Moessbauer (5):
> >   Import isoimage-isohybrid from oe
> >   wic: add ISAR version of isohybrid-efi plugin
> >   move squashfs imagetype class from CIP to ISAR
> >   add qemuamd64 target for hybrid iso9660 image
> >   add kas menu target for qemuamd64-iso machine
> > 
> >  kas/machine/Kconfig                           |  14 +
> >  kas/machine/qemuamd64-iso.yaml                |   7 +
> >  meta-isar/classes/squashfs.bbclass            |  48 ++
> >  meta-isar/conf/machine/qemuamd64-iso.conf     |  29 +
> >  .../multiconfig/qemuamd64-iso-bookworm.conf   |   4 +
> >  .../lib/wic/canned-wks/isohybrid-efi.wks.in   |   2 +
> >  meta/classes/image.bbclass                    |   2 +-
> >  .../plugins/source/isoimage-isohybrid-isar.py | 535
> > ++++++++++++++++++
> >  8 files changed, 640 insertions(+), 1 deletion(-)
> >  create mode 100644 kas/machine/qemuamd64-iso.yaml
> >  create mode 100644 meta-isar/classes/squashfs.bbclass
> >  create mode 100644 meta-isar/conf/machine/qemuamd64-iso.conf
> >  create mode 100644 meta-isar/conf/multiconfig/qemuamd64-iso-
> > bookworm.conf
> >  create mode 100644 meta-isar/scripts/lib/wic/canned-wks/isohybrid-
> > efi.wks.in
> >  create mode 100644 meta/scripts/lib/wic/plugins/source/isoimage-
> > isohybrid-isar.py
> > 
> 
> I suspect we still missing a hook-up into the isar test suite, aren't
> we?

Probably yes, but I can't provide that as I'm not familiar with the
testsuite. What we need is a test that builds the target and checks if
systemctl is-system-running reports "running".

Felix

> 
> Jan
>
Jan Kiszka Jan. 21, 2025, 1:17 p.m. UTC | #3
On 21.01.25 12:42, Moessbauer, Felix (FT RPD CED OES-DE) wrote:
> On Tue, 2025-01-21 at 11:24 +0100, Jan Kiszka wrote:
>> On 21.01.25 11:12, Felix Moessbauer wrote:
>>> This series adds support to create hybrid iso9660 images for ISAR
>>> which can be used as live images for booting from USB sticks or
>>> DVDs. Currently, only syslinux and EFI (grub) bootloaders are
>>> supported.
>>>
>>> Best regards,
>>> Felix Moessbauer
>>>
>>> Felix Moessbauer (5):
>>>   Import isoimage-isohybrid from oe
>>>   wic: add ISAR version of isohybrid-efi plugin
>>>   move squashfs imagetype class from CIP to ISAR
>>>   add qemuamd64 target for hybrid iso9660 image
>>>   add kas menu target for qemuamd64-iso machine
>>>
>>>  kas/machine/Kconfig                           |  14 +
>>>  kas/machine/qemuamd64-iso.yaml                |   7 +
>>>  meta-isar/classes/squashfs.bbclass            |  48 ++
>>>  meta-isar/conf/machine/qemuamd64-iso.conf     |  29 +
>>>  .../multiconfig/qemuamd64-iso-bookworm.conf   |   4 +
>>>  .../lib/wic/canned-wks/isohybrid-efi.wks.in   |   2 +
>>>  meta/classes/image.bbclass                    |   2 +-
>>>  .../plugins/source/isoimage-isohybrid-isar.py | 535
>>> ++++++++++++++++++
>>>  8 files changed, 640 insertions(+), 1 deletion(-)
>>>  create mode 100644 kas/machine/qemuamd64-iso.yaml
>>>  create mode 100644 meta-isar/classes/squashfs.bbclass
>>>  create mode 100644 meta-isar/conf/machine/qemuamd64-iso.conf
>>>  create mode 100644 meta-isar/conf/multiconfig/qemuamd64-iso-
>>> bookworm.conf
>>>  create mode 100644 meta-isar/scripts/lib/wic/canned-wks/isohybrid-
>>> efi.wks.in
>>>  create mode 100644 meta/scripts/lib/wic/plugins/source/isoimage-
>>> isohybrid-isar.py
>>>
>>
>> I suspect we still missing a hook-up into the isar test suite, aren't
>> we?
> 
> Probably yes, but I can't provide that as I'm not familiar with the
> testsuite. What we need is a test that builds the target and checks if
> systemctl is-system-running reports "running".
> 

Building an iso would be a starting point. Running it in a VM is more
effort as it actually requires you do fiddle with local execution of the
testsuite.

Jan
Felix Moessbauer Jan. 23, 2025, 7:53 a.m. UTC | #4
On Wed, 2025-01-22 at 17:11 +0300, Uladzimir Bely wrote:
> On Tue, 2025-01-21 at 11:42 +0000, 'MOESSBAUER, Felix' via isar-users
> wrote:
> > On Tue, 2025-01-21 at 11:24 +0100, Jan Kiszka wrote:
> > > On 21.01.25 11:12, Felix Moessbauer wrote:
> > > > This series adds support to create hybrid iso9660 images for
> > > > ISAR
> > > > which can be used as live images for booting from USB sticks or
> > > > DVDs. Currently, only syslinux and EFI (grub) bootloaders are
> > > > supported.
> > > > 
> > > > Best regards,
> > > > Felix Moessbauer
> > > > 
> > > > Felix Moessbauer (5):
> > > >   Import isoimage-isohybrid from oe
> > > >   wic: add ISAR version of isohybrid-efi plugin
> > > >   move squashfs imagetype class from CIP to ISAR
> > > >   add qemuamd64 target for hybrid iso9660 image
> > > >   add kas menu target for qemuamd64-iso machine
> > > > 
> > > >  kas/machine/Kconfig                           |  14 +
> > > >  kas/machine/qemuamd64-iso.yaml                |   7 +
> > > >  meta-isar/classes/squashfs.bbclass            |  48 ++
> > > >  meta-isar/conf/machine/qemuamd64-iso.conf     |  29 +
> > > >  .../multiconfig/qemuamd64-iso-bookworm.conf   |   4 +
> > > >  .../lib/wic/canned-wks/isohybrid-efi.wks.in   |   2 +
> > > >  meta/classes/image.bbclass                    |   2 +-
> > > >  .../plugins/source/isoimage-isohybrid-isar.py | 535
> > > > ++++++++++++++++++
> > > >  8 files changed, 640 insertions(+), 1 deletion(-)
> > > >  create mode 100644 kas/machine/qemuamd64-iso.yaml
> > > >  create mode 100644 meta-isar/classes/squashfs.bbclass
> > > >  create mode 100644 meta-isar/conf/machine/qemuamd64-iso.conf
> > > >  create mode 100644 meta-isar/conf/multiconfig/qemuamd64-iso-
> > > > bookworm.conf
> > > >  create mode 100644 meta-isar/scripts/lib/wic/canned-
> > > > wks/isohybrid-
> > > > efi.wks.in
> > > >  create mode 100644
> > > > meta/scripts/lib/wic/plugins/source/isoimage-
> > > > isohybrid-isar.py
> > > > 
> > > 
> > > I suspect we still missing a hook-up into the isar test suite,
> > > aren't
> > > we?
> > 
> > Probably yes, but I can't provide that as I'm not familiar with the
> > testsuite. What we need is a test that builds the target and checks
> > if
> > systemctl is-system-running reports "running".
> > 
> 
> I tried to check the image (build with kas) under qemu and it fails
> for
> me. My steps (some of them are definitely worth adding to Isar):
> 
> 1. Some local modifications:
> 
> - Added to "start_vm.py" `amd64-iso` "machine"

Hi, I'll add this in the v2 as well.

> 
> - Fixed machine in multiconfig/qemuamd64-iso-bookworm.conf (see my
> comments to patch 4)
> 
> - Added to kas/isar.yaml:
> 
> ```
> local_conf_header:
>   bbmulticonfig: |
>     BBMULTICONFIG += "${MACHINE}-${BASE_DISTRO_CODENAME}"
> ```
> 
> No we can run "start_vm.py" that uses multiconfigs parsing inside.
> 
> 2. Building the image
> 
> Configured `kas-container` menu and run build:
>  - machine: qemuamd64-iso
>  - distro: debian
>  - user `root` enabled
> 
> 
> 3. Running the image with start_vm.py
> 
> # Workaround for bitbake reusing kas' relative paths in
> conf/bblayers.conf
> ln -s . repo
> 
> 
> # Run qemu on the host
> . isar-init-build-env
> ../testsuite/start_vm.py -a amd64-iso -d bookworm -i isar-image-base
> 
> 
> After logging in, `systemd is-system-running` shows "starting" that
> is
> changed to "degraded" after a while.

Please do a systemctl list-units --failed and paste the output here.
I tested with the following (manual) qemu cmdline and there it
correctly worked:

qemu-system-x86_64 -m 1G -snapshot -serial mon:stdio \
    -netdev user,id=net,hostfwd=tcp:127.0.0.1:22222-:22 \
    -cdrom build/tmp/deploy/images/qemuamd64-iso/isar-image-base-
debian-bookworm-qemuamd64-iso.wic \
    -cpu host -smp 4 -machine q35,accel=kvm:tcg \
    -bios /usr/share/ovmf/OVMF.fd -global ICH9-LPC.noreboot=off \
    -device virtio-net-pci,netdev=net -nographic

Best regards,
Felix Moessbauer

> 
> 
> 
> > Felix
> > 
> > > 
> > > Jan
> > > 
> > 
>
Felix Moessbauer Jan. 23, 2025, 8:49 a.m. UTC | #5
On Wed, 2025-01-22 at 17:11 +0300, Uladzimir Bely wrote:
> On Tue, 2025-01-21 at 11:42 +0000, 'MOESSBAUER, Felix' via isar-users
> wrote:
> > On Tue, 2025-01-21 at 11:24 +0100, Jan Kiszka wrote:
> > > On 21.01.25 11:12, Felix Moessbauer wrote:
> > > > This series adds support to create hybrid iso9660 images for
> > > > ISAR
> > > > which can be used as live images for booting from USB sticks or
> > > > DVDs. Currently, only syslinux and EFI (grub) bootloaders are
> > > > supported.
> > > > 
> > > > Best regards,
> > > > Felix Moessbauer
> > > > 
> > > > Felix Moessbauer (5):
> > > >   Import isoimage-isohybrid from oe
> > > >   wic: add ISAR version of isohybrid-efi plugin
> > > >   move squashfs imagetype class from CIP to ISAR
> > > >   add qemuamd64 target for hybrid iso9660 image
> > > >   add kas menu target for qemuamd64-iso machine
> > > > 
> > > >  kas/machine/Kconfig                           |  14 +
> > > >  kas/machine/qemuamd64-iso.yaml                |   7 +
> > > >  meta-isar/classes/squashfs.bbclass            |  48 ++
> > > >  meta-isar/conf/machine/qemuamd64-iso.conf     |  29 +
> > > >  .../multiconfig/qemuamd64-iso-bookworm.conf   |   4 +
> > > >  .../lib/wic/canned-wks/isohybrid-efi.wks.in   |   2 +
> > > >  meta/classes/image.bbclass                    |   2 +-
> > > >  .../plugins/source/isoimage-isohybrid-isar.py | 535
> > > > ++++++++++++++++++
> > > >  8 files changed, 640 insertions(+), 1 deletion(-)
> > > >  create mode 100644 kas/machine/qemuamd64-iso.yaml
> > > >  create mode 100644 meta-isar/classes/squashfs.bbclass
> > > >  create mode 100644 meta-isar/conf/machine/qemuamd64-iso.conf
> > > >  create mode 100644 meta-isar/conf/multiconfig/qemuamd64-iso-
> > > > bookworm.conf
> > > >  create mode 100644 meta-isar/scripts/lib/wic/canned-
> > > > wks/isohybrid-
> > > > efi.wks.in
> > > >  create mode 100644
> > > > meta/scripts/lib/wic/plugins/source/isoimage-
> > > > isohybrid-isar.py
> > > > 
> > > 
> > > I suspect we still missing a hook-up into the isar test suite,
> > > aren't
> > > we?
> > 
> > Probably yes, but I can't provide that as I'm not familiar with the
> > testsuite. What we need is a test that builds the target and checks
> > if
> > systemctl is-system-running reports "running".
> > 
> 
> I tried to check the image (build with kas) under qemu and it fails
> for
> me. My steps (some of them are definitely worth adding to Isar):
> 
> 1. Some local modifications:
> 
> - Added to "start_vm.py" `amd64-iso` "machine"
> 
> - Fixed machine in multiconfig/qemuamd64-iso-bookworm.conf (see my
> comments to patch 4)
> 
> - Added to kas/isar.yaml:
> 
> ```
> local_conf_header:
>   bbmulticonfig: |
>     BBMULTICONFIG += "${MACHINE}-${BASE_DISTRO_CODENAME}"
> ```

This quirk is just needed for the start_vm script, as it checks for
multiconfig target. However, with the build commands from above we do
not build a multiconfig target. If we would, kas would automatically
add this line (with the concrete values instead of vars).

> 
> No we can run "start_vm.py" that uses multiconfigs parsing inside.
> 
> 2. Building the image
> 
> Configured `kas-container` menu and run build:
>  - machine: qemuamd64-iso
>  - distro: debian
>  - user `root` enabled
> 
> 
> 3. Running the image with start_vm.py
> 
> # Workaround for bitbake reusing kas' relative paths in
> conf/bblayers.conf
> ln -s . repo
> 
> 
> # Run qemu on the host
> . isar-init-build-env
> ../testsuite/start_vm.py -a amd64-iso -d bookworm -i isar-image-base
> 
> 
> After logging in, `systemd is-system-running` shows "starting" that
> is
> changed to "degraded" after a while.

When building exactly with the commands from above, the boot fails as
neither podman nor docker can run on the live system (as their storage
backend needs a writable, but non overlayfs mountpoint). We could work
around this by backing the storage dirs with a tmpfs, however I don't
think that's worth the effort. As debians live-boot package which
provides the initrd hooks does not support this, we would have to write
and maintain a lot of infrastructure just for this use-case.

The amount of temporary data the system can hold is anyways limited by
the available system memory (RAM).

In the v2, I'll just disable the problematic packages in our KConfig,
if not requested otherwise.

Felix

> 
> 
> 
> > Felix
> > 
> > > 
> > > Jan
> > > 
> > 
>