| Message ID | 20220428060615.986332-1-adriaan.schmidt@siemens.com |
|---|---|
| State | Accepted, archived |
| Headers | show |
28.04.2022 09:06, Adriaan Schmidt wrote: > This series introduces image types and conversions as known from OE. > The main motivation is to provide a more flexible way of generating > and processing images, without the need to insert custom tasks (e.g., > currently we have custom classes adding tasks for things like bz2 or xz > compression of images in several downstream layers). > > These patches have been tested on some of our layers, and also > work with some more "challenging" use cases like image-in-image recipes > and custom image types brought by layers. > > This only has a bare minimum of conversions (gz, xz), so there should > probably be follow-ups for additional compressions, or things > like checksums, which can also be nicely done with this. > > - p1 prepares by removing image-type-suffix from image build directories, > which enables sharing of the completed rootfs across image types > - p2 refactors the SDK, and makes it an image feature instead of > a separate recipe (has been discussed here a while ago) > - p3 does the actual work, but keeps all original files > - p4 moves code to different files > - p5 adapts meta-isar > - p6 removes IMAGE_FSTYPES as a dimension in multiconfig. Now that > we can build multiple types from one rootfs, multconfig is > just the wrong way of doing this (also been discussed here). > - p7 documents > - p8 adapts start_vm tests > > changes since v3: > - better handling of xz compression options (taken from OE) > - fix IMAGE_FSTYPES append in qemuamd64-bullseye.conf > - add patch to adapt start_vm to new image type and file names > > changes since v2: > - fix bug where construction of IMAGE_BASETYPES was not recursive > - add comment explaining IMAGE_BASETYPES > - fix ISAR_CROSS_COMPILE exception for ubifs+armhf > - use "IMAGE_CLASSES +=" instead of "INHERIT +=" in machine configs > - fix whitespace errors in documentation > > changes since v1: > - rebase > - fixed naming issue in wic.bmap files > - more reasonable default settings for xz > - add documentation to user manual > > Adriaan Schmidt (7): > image: remove IMAGE_SUFFIX > image: refactor SDK > meta: introduce IMAGE_CMD_* > imagetypes: restructure files > imagetypes: switch meta-isar to the new scheme > meta-isar: remove IMAGE_FSTYPES from multiconfig definitions > docs: add new imagetypes to user manual > > Anton Mikanovich (1): > start_vm: update image types checking > > doc/user_manual.md | 86 ++++--- > meta-isar/classes/rpi-sdimg.bbclass | 2 +- > meta-isar/classes/ubi-ubifs-img.bbclass | 11 +- > meta-isar/conf/local.conf.sample | 4 - > meta-isar/conf/machine/bananapi.conf | 2 +- > meta-isar/conf/machine/container-amd64.conf | 2 +- > meta-isar/conf/machine/de0-nano-soc.conf | 2 +- > meta-isar/conf/machine/hikey.conf | 2 +- > meta-isar/conf/machine/imx6-sabrelite.conf | 3 +- > meta-isar/conf/machine/nanopi-neo.conf | 2 +- > meta-isar/conf/machine/phyboard-mira.conf | 3 +- > meta-isar/conf/machine/qemuamd64.conf | 2 +- > meta-isar/conf/machine/qemuarm.conf | 2 +- > meta-isar/conf/machine/qemuarm64.conf | 2 +- > meta-isar/conf/machine/qemui386.conf | 2 +- > meta-isar/conf/machine/qemumipsel.conf | 2 +- > meta-isar/conf/machine/qemuriscv64.conf | 2 +- > meta-isar/conf/machine/rpi-common.conf | 2 +- > meta-isar/conf/machine/rpi.conf | 2 +- > meta-isar/conf/machine/sifive-fu540.conf | 2 +- > meta-isar/conf/machine/stm32mp15x.conf | 2 +- > meta-isar/conf/machine/virtualbox.conf | 2 +- > meta-isar/conf/machine/vmware.conf | 2 +- > .../qemuamd64-bullseye-cpiogz.conf | 7 - > .../multiconfig/qemuamd64-bullseye-tgz.conf | 7 - > .../conf/multiconfig/qemuamd64-bullseye.conf | 2 + > .../multiconfig/qemuamd64-buster-cpiogz.conf | 9 - > .../multiconfig/qemuamd64-buster-tgz.conf | 9 - > .../conf/multiconfig/qemuamd64-buster.conf | 2 +- > .../recipes-core/images/isar-image-ubi.bb | 4 +- > meta/classes/container-img.bbclass | 17 -- > meta/classes/cpiogz-img.bbclass | 22 -- > meta/classes/ext4-img.bbclass | 24 -- > meta/classes/fit-img.bbclass | 29 --- > .../classes/image-container-extension.bbclass | 83 ------- > meta/classes/image-sdk-extension.bbclass | 87 ------- > meta/classes/image.bbclass | 212 ++++++++++++++++-- > meta/classes/imagetypes.bbclass | 93 ++++++++ > meta/classes/imagetypes_container.bbclass | 95 ++++++++ > .../{vm-img.bbclass => imagetypes_vm.bbclass} | 24 +- > ...wic-img.bbclass => imagetypes_wic.bbclass} | 45 ++-- > meta/classes/rootfs.bbclass | 1 + > meta/classes/sdk.bbclass | 137 +++++++++++ > meta/classes/targz-img.bbclass | 15 -- > meta/classes/ubi-img.bbclass | 32 --- > meta/classes/ubifs-img.bbclass | 32 --- > .../{sdkchroot => sdk-files}/files/README.sdk | 0 > .../files/configscript.sh | 0 > .../files/gcc-sysroot-wrapper.sh | 0 > .../files/relocate-sdk.sh | 0 > meta/recipes-devtools/sdk-files/sdk-files.bb | 26 +++ > meta/recipes-devtools/sdkchroot/sdkchroot.bb | 78 ------- > scripts/start_vm | 8 +- > testsuite/cibase.py | 4 +- > testsuite/citest.py | 2 - > testsuite/start_vm.py | 8 +- > 56 files changed, 673 insertions(+), 584 deletions(-) > delete mode 100644 meta-isar/conf/multiconfig/qemuamd64-bullseye-cpiogz.conf > delete mode 100644 meta-isar/conf/multiconfig/qemuamd64-bullseye-tgz.conf > delete mode 100644 meta-isar/conf/multiconfig/qemuamd64-buster-cpiogz.conf > delete mode 100644 meta-isar/conf/multiconfig/qemuamd64-buster-tgz.conf > delete mode 100644 meta/classes/container-img.bbclass > delete mode 100644 meta/classes/cpiogz-img.bbclass > delete mode 100644 meta/classes/ext4-img.bbclass > delete mode 100644 meta/classes/fit-img.bbclass > delete mode 100644 meta/classes/image-container-extension.bbclass > delete mode 100644 meta/classes/image-sdk-extension.bbclass > create mode 100644 meta/classes/imagetypes.bbclass > create mode 100644 meta/classes/imagetypes_container.bbclass > rename meta/classes/{vm-img.bbclass => imagetypes_vm.bbclass} (89%) > rename meta/classes/{wic-img.bbclass => imagetypes_wic.bbclass} (86%) > create mode 100644 meta/classes/sdk.bbclass > delete mode 100644 meta/classes/targz-img.bbclass > delete mode 100644 meta/classes/ubi-img.bbclass > delete mode 100644 meta/classes/ubifs-img.bbclass > rename meta/recipes-devtools/{sdkchroot => sdk-files}/files/README.sdk (100%) > rename meta/recipes-devtools/{sdkchroot => sdk-files}/files/configscript.sh (100%) > rename meta/recipes-devtools/{sdkchroot => sdk-files}/files/gcc-sysroot-wrapper.sh (100%) > rename meta/recipes-devtools/{sdkchroot => sdk-files}/files/relocate-sdk.sh (100%) > create mode 100644 meta/recipes-devtools/sdk-files/sdk-files.bb > delete mode 100644 meta/recipes-devtools/sdkchroot/sdkchroot.bb > Applied to next, thanks.
On Wed, 2022-05-04 at 10:51 +0300, Anton Mikanovich wrote: > 28.04.2022 09:06, Adriaan Schmidt wrote: > > This series introduces image types and conversions as known from OE. > > The main motivation is to provide a more flexible way of generating > > and processing images, without the need to insert custom tasks (e.g., > > currently we have custom classes adding tasks for things like bz2 or xz > > compression of images in several downstream layers). > > > > These patches have been tested on some of our layers, and also > > work with some more "challenging" use cases like image-in-image recipes > > and custom image types brought by layers. > > > > This only has a bare minimum of conversions (gz, xz), so there should > > probably be follow-ups for additional compressions, or things > > like checksums, which can also be nicely done with this. > > > > - p1 prepares by removing image-type-suffix from image build directories, > > which enables sharing of the completed rootfs across image types > > - p2 refactors the SDK, and makes it an image feature instead of > > a separate recipe (has been discussed here a while ago) > > - p3 does the actual work, but keeps all original files > > - p4 moves code to different files > > - p5 adapts meta-isar > > - p6 removes IMAGE_FSTYPES as a dimension in multiconfig. Now that > > we can build multiple types from one rootfs, multconfig is > > just the wrong way of doing this (also been discussed here). > > - p7 documents > > - p8 adapts start_vm tests > > > > changes since v3: > > - better handling of xz compression options (taken from OE) > > - fix IMAGE_FSTYPES append in qemuamd64-bullseye.conf > > - add patch to adapt start_vm to new image type and file names > > > > changes since v2: > > - fix bug where construction of IMAGE_BASETYPES was not recursive > > - add comment explaining IMAGE_BASETYPES > > - fix ISAR_CROSS_COMPILE exception for ubifs+armhf > > - use "IMAGE_CLASSES +=" instead of "INHERIT +=" in machine configs > > - fix whitespace errors in documentation > > > > changes since v1: > > - rebase > > - fixed naming issue in wic.bmap files > > - more reasonable default settings for xz > > - add documentation to user manual > > > > Adriaan Schmidt (7): > > image: remove IMAGE_SUFFIX > > image: refactor SDK > > meta: introduce IMAGE_CMD_* > > imagetypes: restructure files > > imagetypes: switch meta-isar to the new scheme > > meta-isar: remove IMAGE_FSTYPES from multiconfig definitions > > docs: add new imagetypes to user manual > > > > Anton Mikanovich (1): > > start_vm: update image types checking > > [snip] > > > > > Applied to next, thanks. > My understanding is that all downstream layers have to update IMAGE_FSTYPES (mainly remove "-img"). Isn't that something that should make it into a changelog? Maybe into the "recipe API changelog"? Regards, Florian
On 04.05.22 10:09, Bezdeka, Florian wrote: > On Wed, 2022-05-04 at 10:51 +0300, Anton Mikanovich wrote: >> 28.04.2022 09:06, Adriaan Schmidt wrote: >>> This series introduces image types and conversions as known from OE. >>> The main motivation is to provide a more flexible way of generating >>> and processing images, without the need to insert custom tasks (e.g., >>> currently we have custom classes adding tasks for things like bz2 or xz >>> compression of images in several downstream layers). >>> >>> These patches have been tested on some of our layers, and also >>> work with some more "challenging" use cases like image-in-image recipes >>> and custom image types brought by layers. >>> >>> This only has a bare minimum of conversions (gz, xz), so there should >>> probably be follow-ups for additional compressions, or things >>> like checksums, which can also be nicely done with this. >>> >>> - p1 prepares by removing image-type-suffix from image build directories, >>> which enables sharing of the completed rootfs across image types >>> - p2 refactors the SDK, and makes it an image feature instead of >>> a separate recipe (has been discussed here a while ago) >>> - p3 does the actual work, but keeps all original files >>> - p4 moves code to different files >>> - p5 adapts meta-isar >>> - p6 removes IMAGE_FSTYPES as a dimension in multiconfig. Now that >>> we can build multiple types from one rootfs, multconfig is >>> just the wrong way of doing this (also been discussed here). >>> - p7 documents >>> - p8 adapts start_vm tests >>> >>> changes since v3: >>> - better handling of xz compression options (taken from OE) >>> - fix IMAGE_FSTYPES append in qemuamd64-bullseye.conf >>> - add patch to adapt start_vm to new image type and file names >>> >>> changes since v2: >>> - fix bug where construction of IMAGE_BASETYPES was not recursive >>> - add comment explaining IMAGE_BASETYPES >>> - fix ISAR_CROSS_COMPILE exception for ubifs+armhf >>> - use "IMAGE_CLASSES +=" instead of "INHERIT +=" in machine configs >>> - fix whitespace errors in documentation >>> >>> changes since v1: >>> - rebase >>> - fixed naming issue in wic.bmap files >>> - more reasonable default settings for xz >>> - add documentation to user manual >>> >>> Adriaan Schmidt (7): >>> image: remove IMAGE_SUFFIX >>> image: refactor SDK >>> meta: introduce IMAGE_CMD_* >>> imagetypes: restructure files >>> imagetypes: switch meta-isar to the new scheme >>> meta-isar: remove IMAGE_FSTYPES from multiconfig definitions >>> docs: add new imagetypes to user manual >>> >>> Anton Mikanovich (1): >>> start_vm: update image types checking >>> > > [snip] > >>> >>> >> Applied to next, thanks. >> > > My understanding is that all downstream layers have to update > IMAGE_FSTYPES (mainly remove "-img"). Isn't that something that should > make it into a changelog? Maybe into the "recipe API changelog"? > Yes, please, that will help downstream a lot. Jan
Kiszka, Jan (T CED), 4. Mai 2022 13:33: > On 04.05.22 10:09, Bezdeka, Florian wrote: > > On Wed, 2022-05-04 at 10:51 +0300, Anton Mikanovich wrote: > >> 28.04.2022 09:06, Adriaan Schmidt wrote: > >>> This series introduces image types and conversions as known from OE. > >>> The main motivation is to provide a more flexible way of generating > >>> and processing images, without the need to insert custom tasks (e.g., > >>> currently we have custom classes adding tasks for things like bz2 or xz > >>> compression of images in several downstream layers). > >>> > >>> These patches have been tested on some of our layers, and also > >>> work with some more "challenging" use cases like image-in-image recipes > >>> and custom image types brought by layers. > >>> > >>> This only has a bare minimum of conversions (gz, xz), so there should > >>> probably be follow-ups for additional compressions, or things > >>> like checksums, which can also be nicely done with this. > >>> > >>> - p1 prepares by removing image-type-suffix from image build directories, > >>> which enables sharing of the completed rootfs across image types > >>> - p2 refactors the SDK, and makes it an image feature instead of > >>> a separate recipe (has been discussed here a while ago) > >>> - p3 does the actual work, but keeps all original files > >>> - p4 moves code to different files > >>> - p5 adapts meta-isar > >>> - p6 removes IMAGE_FSTYPES as a dimension in multiconfig. Now that > >>> we can build multiple types from one rootfs, multconfig is > >>> just the wrong way of doing this (also been discussed here). > >>> - p7 documents > >>> - p8 adapts start_vm tests > >>> > >>> changes since v3: > >>> - better handling of xz compression options (taken from OE) > >>> - fix IMAGE_FSTYPES append in qemuamd64-bullseye.conf > >>> - add patch to adapt start_vm to new image type and file names > >>> > >>> changes since v2: > >>> - fix bug where construction of IMAGE_BASETYPES was not recursive > >>> - add comment explaining IMAGE_BASETYPES > >>> - fix ISAR_CROSS_COMPILE exception for ubifs+armhf > >>> - use "IMAGE_CLASSES +=" instead of "INHERIT +=" in machine configs > >>> - fix whitespace errors in documentation > >>> > >>> changes since v1: > >>> - rebase > >>> - fixed naming issue in wic.bmap files > >>> - more reasonable default settings for xz > >>> - add documentation to user manual > >>> > >>> Adriaan Schmidt (7): > >>> image: remove IMAGE_SUFFIX > >>> image: refactor SDK > >>> meta: introduce IMAGE_CMD_* > >>> imagetypes: restructure files > >>> imagetypes: switch meta-isar to the new scheme > >>> meta-isar: remove IMAGE_FSTYPES from multiconfig definitions > >>> docs: add new imagetypes to user manual > >>> > >>> Anton Mikanovich (1): > >>> start_vm: update image types checking > >>> > > > > [snip] > > > >>> > >>> > >> Applied to next, thanks. > >> > > > > My understanding is that all downstream layers have to update > > IMAGE_FSTYPES (mainly remove "-img"). Isn't that something that should > > make it into a changelog? Maybe into the "recipe API changelog"? > > > > Yes, please, that will help downstream a lot. I just posted a proposal. It also has information on what has changed when defining new image classes. Not sure if that is also part of the recipe API... Adriaan
This series introduces image types and conversions as known from OE. The main motivation is to provide a more flexible way of generating and processing images, without the need to insert custom tasks (e.g., currently we have custom classes adding tasks for things like bz2 or xz compression of images in several downstream layers). These patches have been tested on some of our layers, and also work with some more "challenging" use cases like image-in-image recipes and custom image types brought by layers. This only has a bare minimum of conversions (gz, xz), so there should probably be follow-ups for additional compressions, or things like checksums, which can also be nicely done with this. - p1 prepares by removing image-type-suffix from image build directories, which enables sharing of the completed rootfs across image types - p2 refactors the SDK, and makes it an image feature instead of a separate recipe (has been discussed here a while ago) - p3 does the actual work, but keeps all original files - p4 moves code to different files - p5 adapts meta-isar - p6 removes IMAGE_FSTYPES as a dimension in multiconfig. Now that we can build multiple types from one rootfs, multconfig is just the wrong way of doing this (also been discussed here). - p7 documents - p8 adapts start_vm tests changes since v3: - better handling of xz compression options (taken from OE) - fix IMAGE_FSTYPES append in qemuamd64-bullseye.conf - add patch to adapt start_vm to new image type and file names changes since v2: - fix bug where construction of IMAGE_BASETYPES was not recursive - add comment explaining IMAGE_BASETYPES - fix ISAR_CROSS_COMPILE exception for ubifs+armhf - use "IMAGE_CLASSES +=" instead of "INHERIT +=" in machine configs - fix whitespace errors in documentation changes since v1: - rebase - fixed naming issue in wic.bmap files - more reasonable default settings for xz - add documentation to user manual Adriaan Schmidt (7): image: remove IMAGE_SUFFIX image: refactor SDK meta: introduce IMAGE_CMD_* imagetypes: restructure files imagetypes: switch meta-isar to the new scheme meta-isar: remove IMAGE_FSTYPES from multiconfig definitions docs: add new imagetypes to user manual Anton Mikanovich (1): start_vm: update image types checking doc/user_manual.md | 86 ++++--- meta-isar/classes/rpi-sdimg.bbclass | 2 +- meta-isar/classes/ubi-ubifs-img.bbclass | 11 +- meta-isar/conf/local.conf.sample | 4 - meta-isar/conf/machine/bananapi.conf | 2 +- meta-isar/conf/machine/container-amd64.conf | 2 +- meta-isar/conf/machine/de0-nano-soc.conf | 2 +- meta-isar/conf/machine/hikey.conf | 2 +- meta-isar/conf/machine/imx6-sabrelite.conf | 3 +- meta-isar/conf/machine/nanopi-neo.conf | 2 +- meta-isar/conf/machine/phyboard-mira.conf | 3 +- meta-isar/conf/machine/qemuamd64.conf | 2 +- meta-isar/conf/machine/qemuarm.conf | 2 +- meta-isar/conf/machine/qemuarm64.conf | 2 +- meta-isar/conf/machine/qemui386.conf | 2 +- meta-isar/conf/machine/qemumipsel.conf | 2 +- meta-isar/conf/machine/qemuriscv64.conf | 2 +- meta-isar/conf/machine/rpi-common.conf | 2 +- meta-isar/conf/machine/rpi.conf | 2 +- meta-isar/conf/machine/sifive-fu540.conf | 2 +- meta-isar/conf/machine/stm32mp15x.conf | 2 +- meta-isar/conf/machine/virtualbox.conf | 2 +- meta-isar/conf/machine/vmware.conf | 2 +- .../qemuamd64-bullseye-cpiogz.conf | 7 - .../multiconfig/qemuamd64-bullseye-tgz.conf | 7 - .../conf/multiconfig/qemuamd64-bullseye.conf | 2 + .../multiconfig/qemuamd64-buster-cpiogz.conf | 9 - .../multiconfig/qemuamd64-buster-tgz.conf | 9 - .../conf/multiconfig/qemuamd64-buster.conf | 2 +- .../recipes-core/images/isar-image-ubi.bb | 4 +- meta/classes/container-img.bbclass | 17 -- meta/classes/cpiogz-img.bbclass | 22 -- meta/classes/ext4-img.bbclass | 24 -- meta/classes/fit-img.bbclass | 29 --- .../classes/image-container-extension.bbclass | 83 ------- meta/classes/image-sdk-extension.bbclass | 87 ------- meta/classes/image.bbclass | 212 ++++++++++++++++-- meta/classes/imagetypes.bbclass | 93 ++++++++ meta/classes/imagetypes_container.bbclass | 95 ++++++++ .../{vm-img.bbclass => imagetypes_vm.bbclass} | 24 +- ...wic-img.bbclass => imagetypes_wic.bbclass} | 45 ++-- meta/classes/rootfs.bbclass | 1 + meta/classes/sdk.bbclass | 137 +++++++++++ meta/classes/targz-img.bbclass | 15 -- meta/classes/ubi-img.bbclass | 32 --- meta/classes/ubifs-img.bbclass | 32 --- .../{sdkchroot => sdk-files}/files/README.sdk | 0 .../files/configscript.sh | 0 .../files/gcc-sysroot-wrapper.sh | 0 .../files/relocate-sdk.sh | 0 meta/recipes-devtools/sdk-files/sdk-files.bb | 26 +++ meta/recipes-devtools/sdkchroot/sdkchroot.bb | 78 ------- scripts/start_vm | 8 +- testsuite/cibase.py | 4 +- testsuite/citest.py | 2 - testsuite/start_vm.py | 8 +- 56 files changed, 673 insertions(+), 584 deletions(-) delete mode 100644 meta-isar/conf/multiconfig/qemuamd64-bullseye-cpiogz.conf delete mode 100644 meta-isar/conf/multiconfig/qemuamd64-bullseye-tgz.conf delete mode 100644 meta-isar/conf/multiconfig/qemuamd64-buster-cpiogz.conf delete mode 100644 meta-isar/conf/multiconfig/qemuamd64-buster-tgz.conf delete mode 100644 meta/classes/container-img.bbclass delete mode 100644 meta/classes/cpiogz-img.bbclass delete mode 100644 meta/classes/ext4-img.bbclass delete mode 100644 meta/classes/fit-img.bbclass delete mode 100644 meta/classes/image-container-extension.bbclass delete mode 100644 meta/classes/image-sdk-extension.bbclass create mode 100644 meta/classes/imagetypes.bbclass create mode 100644 meta/classes/imagetypes_container.bbclass rename meta/classes/{vm-img.bbclass => imagetypes_vm.bbclass} (89%) rename meta/classes/{wic-img.bbclass => imagetypes_wic.bbclass} (86%) create mode 100644 meta/classes/sdk.bbclass delete mode 100644 meta/classes/targz-img.bbclass delete mode 100644 meta/classes/ubi-img.bbclass delete mode 100644 meta/classes/ubifs-img.bbclass rename meta/recipes-devtools/{sdkchroot => sdk-files}/files/README.sdk (100%) rename meta/recipes-devtools/{sdkchroot => sdk-files}/files/configscript.sh (100%) rename meta/recipes-devtools/{sdkchroot => sdk-files}/files/gcc-sysroot-wrapper.sh (100%) rename meta/recipes-devtools/{sdkchroot => sdk-files}/files/relocate-sdk.sh (100%) create mode 100644 meta/recipes-devtools/sdk-files/sdk-files.bb delete mode 100644 meta/recipes-devtools/sdkchroot/sdkchroot.bb