| Message ID | 20221215221606.5623-1-henning.schild@siemens.com |
|---|---|
| Headers | show |
| Series | expand-on-first-boot btrfs and CI testing | expand |
In case we keep seeing more trolling on that one i think we should only consider the CI bits and drop btrfs. btrfs has been proposed by two downstream users who already have their own patches and might not care too much about upstream. I am trying hard to cater for them. But one might not care too much, while the other should likely be ignored because the demand and all the reviews seem more like an act of sabotage than actually wanting to help. Please review this if you care about this in a truly relevant use-case. Please stop reviewing this if you are only looking into Isar as a hobby or revenge for something that went south for you. Henning Am Thu, 15 Dec 2022 23:16:00 +0100 schrieb henning.schild@siemens.com: > From: Henning Schild <henning.schild@siemens.com> > > changes since v5: > - drop dead code around mktemp subshell > - revert cosmetic mktemp stuff back to v3 > - remove "Suggested-by" tag to hopefully improve signal to noise > ratio in review process > > changes since v4: > - cosmetic fix in the way we call mktemp > - add a patch that makes sure the service is run only once, also on > error > - bump package version after changes are done > > changes since v3: > - rebased on "expand-on-first-boot: switch back away from > systemd-growfs" > - implement btrfs resizing and CI testing it > - put the padding wks entry into an include > > changes since v2: > - add python namespace prefix for the bitbake get functions > - install only in images that use WKS_FILEs that have the "gap" > > changes since v1: > - complete rewrite to not focus only on the tests but rather the > feature for all examples > > This first was only about testing but meanwhile has a focus on > enabling the expand feature in all example images anyhow, and later > look for traces of that happening in CI test where we can. > We install the application in any image and in the wic images add some > scratch space at the end. That will enable improved interactive use of > all those wic images, so makes sense apart from any testing. > Later use the log reading feature of the qemu tests to look for traces > of successful resizing for non ubuntu images. In ubuntu the > console_loglevel is not verbose enough to find traces. > > Henning Schild (6): > expand-on-first-boot: really only do that once, especially on > failure expand-on-first-boot: support resizing a btrfs > meta-isar: introduce an example to use btrfs > CI: improve cibuilder readability > meta-isar: install expand-on-first-boot in most images and add space > CI: expect a message about filesystem resize vom expand script > > meta-isar/conf/machine/qemuamd64.conf | 6 ++++ > meta-isar/conf/machine/virtualbox.conf | 1 + > meta-isar/conf/machine/vmware.conf | 1 + > .../conf/multiconfig/qemuarm-bookworm.conf | 1 + > .../lib/wic/canned-wks/expand-padding.wks.inc | 5 +++ > .../lib/wic/canned-wks/sdimage-efi-btrfs.wks | 11 +++++++ > .../lib/wic/canned-wks/sdimage-efi-sd.wks | 2 ++ > .../lib/wic/canned-wks/sdimage-efi.wks | 2 ++ > ...oot_1.3.bb => expand-on-first-boot_1.4.bb} | 0 > .../files/expand-last-partition.sh | 31 > ++++++++++++++++--- .../files/expand-on-first-boot.service | > 1 + testsuite/cibuilder.py | 28 > ++++++++++++++--- 12 files changed, 80 insertions(+), 9 deletions(-) > create mode 100644 > meta-isar/scripts/lib/wic/canned-wks/expand-padding.wks.inc create > mode 100644 > meta-isar/scripts/lib/wic/canned-wks/sdimage-efi-btrfs.wks rename > meta/recipes-support/expand-on-first-boot/{expand-on-first-boot_1.3.bb > => expand-on-first-boot_1.4.bb} (100%) >
Hi, I tested this on an Siemens IoT2050 board and can confirm that it properly works. Acked-by: Felix Moessbauer <felix.moessbauer@siemens.com> Felix On Thu, 2022-12-15 at 23:16 +0100, henning.schild@siemens.com wrote: > From: Henning Schild <henning.schild@siemens.com> > > changes since v5: > - drop dead code around mktemp subshell > - revert cosmetic mktemp stuff back to v3 > - remove "Suggested-by" tag to hopefully improve signal to noise > ratio > in review process > > changes since v4: > - cosmetic fix in the way we call mktemp > - add a patch that makes sure the service is run only once, also on > error > - bump package version after changes are done > > changes since v3: > - rebased on "expand-on-first-boot: switch back away from systemd- > growfs" > - implement btrfs resizing and CI testing it > - put the padding wks entry into an include > > changes since v2: > - add python namespace prefix for the bitbake get functions > - install only in images that use WKS_FILEs that have the "gap" > > changes since v1: > - complete rewrite to not focus only on the tests but rather the > feature for all examples > > This first was only about testing but meanwhile has a focus on > enabling > the expand feature in all example images anyhow, and later look for > traces of that happening in CI test where we can. > We install the application in any image and in the wic images add > some > scratch space at the end. That will enable improved interactive use > of > all those wic images, so makes sense apart from any testing. > Later use the log reading feature of the qemu tests to look for > traces > of successful resizing for non ubuntu images. In ubuntu the > console_loglevel is not verbose enough to find traces. > > Henning Schild (6): > expand-on-first-boot: really only do that once, especially on > failure > expand-on-first-boot: support resizing a btrfs > meta-isar: introduce an example to use btrfs > CI: improve cibuilder readability > meta-isar: install expand-on-first-boot in most images and add > space > CI: expect a message about filesystem resize vom expand script > > meta-isar/conf/machine/qemuamd64.conf | 6 ++++ > meta-isar/conf/machine/virtualbox.conf | 1 + > meta-isar/conf/machine/vmware.conf | 1 + > .../conf/multiconfig/qemuarm-bookworm.conf | 1 + > .../lib/wic/canned-wks/expand-padding.wks.inc | 5 +++ > .../lib/wic/canned-wks/sdimage-efi-btrfs.wks | 11 +++++++ > .../lib/wic/canned-wks/sdimage-efi-sd.wks | 2 ++ > .../lib/wic/canned-wks/sdimage-efi.wks | 2 ++ > ...oot_1.3.bb => expand-on-first-boot_1.4.bb} | 0 > .../files/expand-last-partition.sh | 31 ++++++++++++++++- > -- > .../files/expand-on-first-boot.service | 1 + > testsuite/cibuilder.py | 28 ++++++++++++++--- > 12 files changed, 80 insertions(+), 9 deletions(-) > create mode 100644 meta-isar/scripts/lib/wic/canned-wks/expand- > padding.wks.inc > create mode 100644 meta-isar/scripts/lib/wic/canned-wks/sdimage-efi- > btrfs.wks > rename meta/recipes-support/expand-on-first-boot/{expand-on-first- > boot_1.3.bb => expand-on-first-boot_1.4.bb} (100%) > > -- > 2.37.4 >
16.12.2022 01:16, henning.schild@siemens.com wrote: > From: Henning Schild <henning.schild@siemens.com> > > changes since v5: > - drop dead code around mktemp subshell > - revert cosmetic mktemp stuff back to v3 > - remove "Suggested-by" tag to hopefully improve signal to noise ratio > in review process > > changes since v4: > - cosmetic fix in the way we call mktemp > - add a patch that makes sure the service is run only once, also on > error > - bump package version after changes are done > > changes since v3: > - rebased on "expand-on-first-boot: switch back away from systemd-growfs" > - implement btrfs resizing and CI testing it > - put the padding wks entry into an include > > changes since v2: > - add python namespace prefix for the bitbake get functions > - install only in images that use WKS_FILEs that have the "gap" > > changes since v1: > - complete rewrite to not focus only on the tests but rather the > feature for all examples > > This first was only about testing but meanwhile has a focus on enabling > the expand feature in all example images anyhow, and later look for > traces of that happening in CI test where we can. > We install the application in any image and in the wic images add some > scratch space at the end. That will enable improved interactive use of > all those wic images, so makes sense apart from any testing. > Later use the log reading feature of the qemu tests to look for traces > of successful resizing for non ubuntu images. In ubuntu the > console_loglevel is not verbose enough to find traces. > > Henning Schild (6): > expand-on-first-boot: really only do that once, especially on failure > expand-on-first-boot: support resizing a btrfs > meta-isar: introduce an example to use btrfs > CI: improve cibuilder readability > meta-isar: install expand-on-first-boot in most images and add space > CI: expect a message about filesystem resize vom expand script > > meta-isar/conf/machine/qemuamd64.conf | 6 ++++ > meta-isar/conf/machine/virtualbox.conf | 1 + > meta-isar/conf/machine/vmware.conf | 1 + > .../conf/multiconfig/qemuarm-bookworm.conf | 1 + > .../lib/wic/canned-wks/expand-padding.wks.inc | 5 +++ > .../lib/wic/canned-wks/sdimage-efi-btrfs.wks | 11 +++++++ > .../lib/wic/canned-wks/sdimage-efi-sd.wks | 2 ++ > .../lib/wic/canned-wks/sdimage-efi.wks | 2 ++ > ...oot_1.3.bb => expand-on-first-boot_1.4.bb} | 0 > .../files/expand-last-partition.sh | 31 ++++++++++++++++--- > .../files/expand-on-first-boot.service | 1 + > testsuite/cibuilder.py | 28 ++++++++++++++--- > 12 files changed, 80 insertions(+), 9 deletions(-) > create mode 100644 meta-isar/scripts/lib/wic/canned-wks/expand-padding.wks.inc > create mode 100644 meta-isar/scripts/lib/wic/canned-wks/sdimage-efi-btrfs.wks > rename meta/recipes-support/expand-on-first-boot/{expand-on-first-boot_1.3.bb => expand-on-first-boot_1.4.bb} (100%) > Hello Henning, This patchset look really good, but still fails on arm-bookworm startvm test of our fast CI. The issue can be reproduced by just building qemuarm-bookworm target with bitbake (even outside testsuite) and then execute VmBootTestFast.test_arm_bookworm test case. So my proposal is to apply revert expand commit now without waiting for this patchset to be ready.
Am Tue, 20 Dec 2022 22:36:03 +0300 schrieb Anton Mikanovich <amikan@ilbers.de>: > 16.12.2022 01:16, henning.schild@siemens.com wrote: > > From: Henning Schild <henning.schild@siemens.com> > > > > changes since v5: > > - drop dead code around mktemp subshell > > - revert cosmetic mktemp stuff back to v3 > > - remove "Suggested-by" tag to hopefully improve signal to noise > > ratio in review process > > > > changes since v4: > > - cosmetic fix in the way we call mktemp > > - add a patch that makes sure the service is run only once, also > > on error > > - bump package version after changes are done > > > > changes since v3: > > - rebased on "expand-on-first-boot: switch back away from > > systemd-growfs" > > - implement btrfs resizing and CI testing it > > - put the padding wks entry into an include > > > > changes since v2: > > - add python namespace prefix for the bitbake get functions > > - install only in images that use WKS_FILEs that have the "gap" > > > > changes since v1: > > - complete rewrite to not focus only on the tests but rather the > > feature for all examples > > > > This first was only about testing but meanwhile has a focus on > > enabling the expand feature in all example images anyhow, and later > > look for traces of that happening in CI test where we can. > > We install the application in any image and in the wic images add > > some scratch space at the end. That will enable improved > > interactive use of all those wic images, so makes sense apart from > > any testing. Later use the log reading feature of the qemu tests to > > look for traces of successful resizing for non ubuntu images. In > > ubuntu the console_loglevel is not verbose enough to find traces. > > > > Henning Schild (6): > > expand-on-first-boot: really only do that once, especially on > > failure expand-on-first-boot: support resizing a btrfs > > meta-isar: introduce an example to use btrfs > > CI: improve cibuilder readability > > meta-isar: install expand-on-first-boot in most images and add > > space CI: expect a message about filesystem resize vom expand script > > > > meta-isar/conf/machine/qemuamd64.conf | 6 ++++ > > meta-isar/conf/machine/virtualbox.conf | 1 + > > meta-isar/conf/machine/vmware.conf | 1 + > > .../conf/multiconfig/qemuarm-bookworm.conf | 1 + > > .../lib/wic/canned-wks/expand-padding.wks.inc | 5 +++ > > .../lib/wic/canned-wks/sdimage-efi-btrfs.wks | 11 +++++++ > > .../lib/wic/canned-wks/sdimage-efi-sd.wks | 2 ++ > > .../lib/wic/canned-wks/sdimage-efi.wks | 2 ++ > > ...oot_1.3.bb => expand-on-first-boot_1.4.bb} | 0 > > .../files/expand-last-partition.sh | 31 > > ++++++++++++++++--- .../files/expand-on-first-boot.service | > > 1 + testsuite/cibuilder.py | 28 > > ++++++++++++++--- 12 files changed, 80 insertions(+), 9 deletions(-) > > create mode 100644 > > meta-isar/scripts/lib/wic/canned-wks/expand-padding.wks.inc create > > mode 100644 > > meta-isar/scripts/lib/wic/canned-wks/sdimage-efi-btrfs.wks rename > > meta/recipes-support/expand-on-first-boot/{expand-on-first-boot_1.3.bb > > => expand-on-first-boot_1.4.bb} (100%) > Hello Henning, > > This patchset look really good, but still fails on arm-bookworm > startvm test of > our fast CI. The issue can be reproduced by just building > qemuarm-bookworm target with bitbake (even outside testsuite) and > then execute VmBootTestFast.test_arm_bookworm test case. I saw that when i ran ilbers CI once. Was hoping it to be unrelated and maybe a known issue. But if you confirm that these patches introduced it, i will have a look > So my proposal is to apply revert expand commit now without waiting > for this patchset to be ready. Sure. The "revert" patch is important. This stuff can wait. Henning
Am Wed, 21 Dec 2022 09:22:21 +0100 schrieb Henning Schild <henning.schild@siemens.com>: > Am Tue, 20 Dec 2022 22:36:03 +0300 > schrieb Anton Mikanovich <amikan@ilbers.de>: > > > 16.12.2022 01:16, henning.schild@siemens.com wrote: > > > From: Henning Schild <henning.schild@siemens.com> > > > > > > changes since v5: > > > - drop dead code around mktemp subshell > > > - revert cosmetic mktemp stuff back to v3 > > > - remove "Suggested-by" tag to hopefully improve signal to noise > > > ratio in review process > > > > > > changes since v4: > > > - cosmetic fix in the way we call mktemp > > > - add a patch that makes sure the service is run only once, also > > > on error > > > - bump package version after changes are done > > > > > > changes since v3: > > > - rebased on "expand-on-first-boot: switch back away from > > > systemd-growfs" > > > - implement btrfs resizing and CI testing it > > > - put the padding wks entry into an include > > > > > > changes since v2: > > > - add python namespace prefix for the bitbake get functions > > > - install only in images that use WKS_FILEs that have the "gap" > > > > > > changes since v1: > > > - complete rewrite to not focus only on the tests but rather the > > > feature for all examples > > > > > > This first was only about testing but meanwhile has a focus on > > > enabling the expand feature in all example images anyhow, and > > > later look for traces of that happening in CI test where we can. > > > We install the application in any image and in the wic images add > > > some scratch space at the end. That will enable improved > > > interactive use of all those wic images, so makes sense apart from > > > any testing. Later use the log reading feature of the qemu tests > > > to look for traces of successful resizing for non ubuntu images. > > > In ubuntu the console_loglevel is not verbose enough to find > > > traces. > > > > > > Henning Schild (6): > > > expand-on-first-boot: really only do that once, especially on > > > failure expand-on-first-boot: support resizing a btrfs > > > meta-isar: introduce an example to use btrfs > > > CI: improve cibuilder readability > > > meta-isar: install expand-on-first-boot in most images and add > > > space CI: expect a message about filesystem resize vom expand > > > script > > > > > > meta-isar/conf/machine/qemuamd64.conf | 6 ++++ > > > meta-isar/conf/machine/virtualbox.conf | 1 + > > > meta-isar/conf/machine/vmware.conf | 1 + > > > .../conf/multiconfig/qemuarm-bookworm.conf | 1 + > > > .../lib/wic/canned-wks/expand-padding.wks.inc | 5 +++ > > > .../lib/wic/canned-wks/sdimage-efi-btrfs.wks | 11 +++++++ > > > .../lib/wic/canned-wks/sdimage-efi-sd.wks | 2 ++ > > > .../lib/wic/canned-wks/sdimage-efi.wks | 2 ++ > > > ...oot_1.3.bb => expand-on-first-boot_1.4.bb} | 0 > > > .../files/expand-last-partition.sh | 31 > > > ++++++++++++++++--- .../files/expand-on-first-boot.service > > > | 1 + testsuite/cibuilder.py | 28 > > > ++++++++++++++--- 12 files changed, 80 insertions(+), 9 > > > deletions(-) create mode 100644 > > > meta-isar/scripts/lib/wic/canned-wks/expand-padding.wks.inc create > > > mode 100644 > > > meta-isar/scripts/lib/wic/canned-wks/sdimage-efi-btrfs.wks rename > > > meta/recipes-support/expand-on-first-boot/{expand-on-first-boot_1.3.bb > > > => expand-on-first-boot_1.4.bb} (100%) > > Hello Henning, > > > > This patchset look really good, but still fails on arm-bookworm > > startvm test of > > our fast CI. The issue can be reproduced by just building > > qemuarm-bookworm target with bitbake (even outside testsuite) and > > then execute VmBootTestFast.test_arm_bookworm test case. > > I saw that when i ran ilbers CI once. Was hoping it to be unrelated > and maybe a known issue. But if you confirm that these patches > introduced it, i will have a look Found it, reason is that i added test code which expects resize traces in logs when a target used a specific WKS_FILE. Now that qemuarm-bookworm has IMAGE_FSTYPES="ext4 wic" And the CI test does test [0] so it does test ext4 and not wic, in which case we really can not find resize traces. That was all kind of introduced by 8b17d91cc758d173aca9d97961208a17a95f71a2 I will change that last commit to also check IMAGE_FSTYPES and match [0] to be wic Henning > > So my proposal is to apply revert expand commit now without waiting > > for this patchset to be ready. > > Sure. The "revert" patch is important. This stuff can wait. > > Henning > >
From: Henning Schild <henning.schild@siemens.com> changes since v5: - drop dead code around mktemp subshell - revert cosmetic mktemp stuff back to v3 - remove "Suggested-by" tag to hopefully improve signal to noise ratio in review process changes since v4: - cosmetic fix in the way we call mktemp - add a patch that makes sure the service is run only once, also on error - bump package version after changes are done changes since v3: - rebased on "expand-on-first-boot: switch back away from systemd-growfs" - implement btrfs resizing and CI testing it - put the padding wks entry into an include changes since v2: - add python namespace prefix for the bitbake get functions - install only in images that use WKS_FILEs that have the "gap" changes since v1: - complete rewrite to not focus only on the tests but rather the feature for all examples This first was only about testing but meanwhile has a focus on enabling the expand feature in all example images anyhow, and later look for traces of that happening in CI test where we can. We install the application in any image and in the wic images add some scratch space at the end. That will enable improved interactive use of all those wic images, so makes sense apart from any testing. Later use the log reading feature of the qemu tests to look for traces of successful resizing for non ubuntu images. In ubuntu the console_loglevel is not verbose enough to find traces. Henning Schild (6): expand-on-first-boot: really only do that once, especially on failure expand-on-first-boot: support resizing a btrfs meta-isar: introduce an example to use btrfs CI: improve cibuilder readability meta-isar: install expand-on-first-boot in most images and add space CI: expect a message about filesystem resize vom expand script meta-isar/conf/machine/qemuamd64.conf | 6 ++++ meta-isar/conf/machine/virtualbox.conf | 1 + meta-isar/conf/machine/vmware.conf | 1 + .../conf/multiconfig/qemuarm-bookworm.conf | 1 + .../lib/wic/canned-wks/expand-padding.wks.inc | 5 +++ .../lib/wic/canned-wks/sdimage-efi-btrfs.wks | 11 +++++++ .../lib/wic/canned-wks/sdimage-efi-sd.wks | 2 ++ .../lib/wic/canned-wks/sdimage-efi.wks | 2 ++ ...oot_1.3.bb => expand-on-first-boot_1.4.bb} | 0 .../files/expand-last-partition.sh | 31 ++++++++++++++++--- .../files/expand-on-first-boot.service | 1 + testsuite/cibuilder.py | 28 ++++++++++++++--- 12 files changed, 80 insertions(+), 9 deletions(-) create mode 100644 meta-isar/scripts/lib/wic/canned-wks/expand-padding.wks.inc create mode 100644 meta-isar/scripts/lib/wic/canned-wks/sdimage-efi-btrfs.wks rename meta/recipes-support/expand-on-first-boot/{expand-on-first-boot_1.3.bb => expand-on-first-boot_1.4.bb} (100%)