| Message ID | 20200824092750.19642-1-Vijaikumar_Kanagarajan@mentor.com |
|---|---|
| Headers | show |
| Series | WIC update | expand |
On 24.08.20 11:27, Vijai Kumar K wrote: > The following series updates the wic to the latest one from > openembedded core. The git tree is available here. > > https://github.com/vj-kumar/isar/tree/wic/staging0 > > Below are summary of key changes in the below series. > > P3 & P6: > The latest version of WIC detects the presence of fakeroot using > pseudo_dir(${IMAGE_ROOTFS}/../pseudo). If this is non-existent it doesnot > use fakeroot. P3 handles that quirk. There is also a standalone invocation of > pseudo which P6 tries to handle via wic_fakeroot. > > In case of ISAR, we need wic_fakeroot as fakeroot only for 2 purposes. > 1. To workaround fsck return values for stretch > 2. To handle standalone invocation of pseudo(P4 in series) > > Number 2 is not needed if we dont use fakeroot. > > The alternative approach to fakeroot is to carry patches on top of wic > directory instead of working around such quirks using fakeroot. The downside > being the need to rebase the local changes on future updates of wic. > > > P4 & P5: > These cherry-picked changes address the --exclude-path issues which Henning > has addressed in [2]. Should be dropped once that is merged. > > P9: > P9 is needed for --include-path to work. I have observed that --include-path > does not yet work/broken in the latest OE-core/poky. Posted a query in [1]. In ISAR > it works with P9 fix. P9 fix could also be relevant upstream and can be sent there > for review. > > Apart from the normal qemu build test, Below are the list of features tested. > KS API: > --exclude-path > --include-path > --fsuuid > --overhead-factor > --fixed-size > --use-label > --use-uuid > > Standalone invocations: > wic ls > wic cp > wic rm > > Currently in the process of finding a way to test wic features automatically. Some wic features > like -fsuuid cannot be validated till we boot the target. Need to find a generalized way for > testing the available wic features. The approach in hand is to use a wic-test.wks file and > generate qemu image with that to validate the wic features. > > [1] https://lists.yoctoproject.org/g/poky/message/12151 > [2] https://groups.google.com/g/isar-users/c/xePCyEl2qhM/m/mwFdeou0AQAJ > FWIW, I've tested a couple of images built by jailhouse-images, and no issues so far. Jan
On Monday, August 31, 2020 at 7:33:16 PM UTC+5:30 Jan Kiszka wrote: > On 24.08.20 11:27, Vijai Kumar K wrote: > > The following series updates the wic to the latest one from > > openembedded core. The git tree is available here. > > > > https://github.com/vj-kumar/isar/tree/wic/staging0 > > > > Below are summary of key changes in the below series. > > > > P3 & P6: > > The latest version of WIC detects the presence of fakeroot using > > pseudo_dir(${IMAGE_ROOTFS}/../pseudo). If this is non-existent it > doesnot > > use fakeroot. P3 handles that quirk. There is also a standalone > invocation of > > pseudo which P6 tries to handle via wic_fakeroot. > > > > In case of ISAR, we need wic_fakeroot as fakeroot only for 2 purposes. > > 1. To workaround fsck return values for stretch > > 2. To handle standalone invocation of pseudo(P4 in series) > > > > Number 2 is not needed if we dont use fakeroot. > > > > The alternative approach to fakeroot is to carry patches on top of wic > > directory instead of working around such quirks using fakeroot. The > downside > > being the need to rebase the local changes on future updates of wic. > > > > > > P4 & P5: > > These cherry-picked changes address the --exclude-path issues which > Henning > > has addressed in [2]. Should be dropped once that is merged. > > > > P9: > > P9 is needed for --include-path to work. I have observed that > --include-path > > does not yet work/broken in the latest OE-core/poky. Posted a query in > [1]. In ISAR > > it works with P9 fix. P9 fix could also be relevant upstream and can be > sent there > > for review. > > > > Apart from the normal qemu build test, Below are the list of features > tested. > > KS API: > > --exclude-path > > --include-path > > --fsuuid > > --overhead-factor > > --fixed-size > > --use-label > > --use-uuid > > > > Standalone invocations: > > wic ls > > wic cp > > wic rm > > > > Currently in the process of finding a way to test wic features > automatically. Some wic features > > like -fsuuid cannot be validated till we boot the target. Need to find a > generalized way for > > testing the available wic features. The approach in hand is to use a > wic-test.wks file and > > generate qemu image with that to validate the wic features. > > > > [1] https://lists.yoctoproject.org/g/poky/message/12151 > > [2] https://groups.google.com/g/isar-users/c/xePCyEl2qhM/m/mwFdeou0AQAJ > > > > FWIW, I've tested a couple of images built by jailhouse-images, and no > issues so far. > > Jan > Thank you Jan. Also, I have tested the wic with jailhouse-images & isar-siemens project. No issues were found. https://github.com/vj-kumar/jailhouse-images/commits/wic/update + https://github.com/vj-kumar/isar/commits/wic/staging1 Best, Vijai Kumar K > -- > Siemens AG, Corporate Technology, CT RDA IOT SES-DE > Corporate Competence Center Embedded Linux >
On Monday, August 24, 2020 at 2:58:19 PM UTC+5:30 vijaikumar_...@mentor.com wrote: > The following series updates the wic to the latest one from > openembedded core. The git tree is available here. > > https://github.com/vj-kumar/isar/tree/wic/staging0 > > Below are summary of key changes in the below series. > > P3 & P6: > The latest version of WIC detects the presence of fakeroot using > pseudo_dir(${IMAGE_ROOTFS}/../pseudo). If this is non-existent it doesnot > use fakeroot. P3 handles that quirk. There is also a standalone invocation > of > pseudo which P6 tries to handle via wic_fakeroot. > > In case of ISAR, we need wic_fakeroot as fakeroot only for 2 purposes. > 1. To workaround fsck return values for stretch > 2. To handle standalone invocation of pseudo(P4 in series) > > Number 2 is not needed if we dont use fakeroot. > > The alternative approach to fakeroot is to carry patches on top of wic > directory instead of working around such quirks using fakeroot. The > downside > being the need to rebase the local changes on future updates of wic. > > > P4 & P5: > These cherry-picked changes address the --exclude-path issues which > Henning > has addressed in [2]. Should be dropped once that is merged. > > P9: > P9 is needed for --include-path to work. I have observed that > --include-path > does not yet work/broken in the latest OE-core/poky. Posted a query in > [1]. In ISAR > it works with P9 fix. P9 fix could also be relevant upstream and can be > sent there > for review. > P9 fix is sent to OE-core mailing list. Currently in master-next. Hopefully should get merged. https://lists.openembedded.org/g/openembedded-core/message/142048 I will send a V2 with updated P9 commit. I would cherry-pick and carry P9 version from OE-core once it is merged to master. We could drop that the next time we update wic. FWIW, --include-path is fixed in OE-core with the below patch. It is merged to master. https://lists.openembedded.org/g/openembedded-core/message/141865 Thanks, Vijai Kumar K > > Apart from the normal qemu build test, Below are the list of features > tested. > KS API: > --exclude-path > --include-path > --fsuuid > --overhead-factor > --fixed-size > --use-label > --use-uuid > > Standalone invocations: > wic ls > wic cp > wic rm > > Currently in the process of finding a way to test wic features > automatically. Some wic features > like -fsuuid cannot be validated till we boot the target. Need to find a > generalized way for > testing the available wic features. The approach in hand is to use a > wic-test.wks file and > generate qemu image with that to validate the wic features. > > [1] https://lists.yoctoproject.org/g/poky/message/12151 > [2] https://groups.google.com/g/isar-users/c/xePCyEl2qhM/m/mwFdeou0AQAJ > > > Henning Schild (1): > lib/oe/path: try hardlinking instead of guessing when it might fail > > Paul Barker (1): > oe.path: Add copyhardlink() helper function > > Vijai Kumar K (8): > wic: Update to the latest wic from openembedded core > wic/plugins: Fix wic plugins to work with the latest wic > wic-img: Satisfy the quirks of latest wic > wic_fakeroot: Handle standalone pseudo invocations > meta-isar/conf: Add provision to debug WIC > debian-common: Add tar as a dependency for wic > scripts/lib/wic: Add /bin to the list of searchpaths > meta-isar/canned-wks: Remove /boot mountpoint > > meta-isar/conf/local.conf.sample | 3 + > .../lib/wic/canned-wks/common-isar.wks.inc | 2 +- > .../scripts/lib/wic/canned-wks/hikey.wks | 2 +- > .../lib/wic/canned-wks/sdimage-efi.wks | 2 +- > meta/classes/wic-img.bbclass | 15 +- > meta/conf/distro/debian-common.conf | 3 +- > meta/lib/oe/path.py | 27 +- > .../wic/plugins/source/bootimg-efi-isar.py | 2 +- > .../wic/plugins/source/bootimg-pcbios-isar.py | 9 +- > .../lib/wic/plugins/source/rootfs-u-boot.py | 2 +- > scripts/lib/scriptpath.py | 32 ++ > scripts/lib/wic/__init__.py | 14 +- > scripts/lib/wic/canned-wks/common.wks.inc | 2 +- > .../directdisk-bootloader-config.cfg | 8 +- > .../lib/wic/canned-wks/efi-bootdisk.wks.in | 3 + > scripts/lib/wic/canned-wks/mkhybridiso.wks | 2 +- > scripts/lib/wic/canned-wks/qemuriscv.wks | 3 + > .../lib/wic/canned-wks/qemux86-directdisk.wks | 2 +- > .../lib/wic/canned-wks/sdimage-bootpart.wks | 4 +- > .../lib/wic/canned-wks/systemd-bootdisk.wks | 4 +- > scripts/lib/wic/engine.py | 421 +++++++++++++++- > scripts/lib/wic/filemap.py | 170 ++++--- > scripts/lib/wic/help.py | 401 ++++++++++++++-- > scripts/lib/wic/ksparser.py | 115 +++-- > scripts/lib/wic/{utils => }/misc.py | 100 ++-- > scripts/lib/wic/partition.py | 234 ++++----- > scripts/lib/wic/pluginbase.py | 36 +- > scripts/lib/wic/plugins/imager/direct.py | 175 ++++--- > .../wic/plugins/source/bootimg-biosplusefi.py | 213 +++++++++ > scripts/lib/wic/plugins/source/bootimg-efi.py | 111 +++-- > .../wic/plugins/source/bootimg-partition.py | 153 ++++-- > .../lib/wic/plugins/source/bootimg-pcbios.py | 91 ++-- > scripts/lib/wic/plugins/source/fsimage.py | 56 --- > .../wic/plugins/source/isoimage-isohybrid.py | 185 +++---- > scripts/lib/wic/plugins/source/rawcopy.py | 44 +- > scripts/lib/wic/plugins/source/rootfs.py | 159 ++++-- > scripts/lib/wic/utils/__init__.py | 0 > scripts/lib/wic/utils/runner.py | 114 ----- > scripts/wic | 452 +++++++++++++----- > scripts/wic_fakeroot | 5 + > 40 files changed, 2340 insertions(+), 1036 deletions(-) > create mode 100644 scripts/lib/scriptpath.py > create mode 100644 scripts/lib/wic/canned-wks/efi-bootdisk.wks.in > create mode 100644 scripts/lib/wic/canned-wks/qemuriscv.wks > rename scripts/lib/wic/{utils => }/misc.py (70%) > create mode 100644 scripts/lib/wic/plugins/source/bootimg-biosplusefi.py > delete mode 100644 scripts/lib/wic/plugins/source/fsimage.py > delete mode 100644 scripts/lib/wic/utils/__init__.py > delete mode 100644 scripts/lib/wic/utils/runner.py > > -- > 2.17.1 > >
On 01.09.20 18:53, vijaikumar....@gmail.com wrote: > > > On Monday, August 24, 2020 at 2:58:19 PM UTC+5:30 > vijaikumar_...@mentor.com wrote: > > The following series updates the wic to the latest one from > openembedded core. The git tree is available here. > > https://github.com/vj-kumar/isar/tree/wic/staging0 > > Below are summary of key changes in the below series. > > P3 & P6: > The latest version of WIC detects the presence of fakeroot using > pseudo_dir(${IMAGE_ROOTFS}/../pseudo). If this is non-existent it > doesnot > use fakeroot. P3 handles that quirk. There is also a standalone > invocation of > pseudo which P6 tries to handle via wic_fakeroot. > > In case of ISAR, we need wic_fakeroot as fakeroot only for 2 purposes. > 1. To workaround fsck return values for stretch > 2. To handle standalone invocation of pseudo(P4 in series) > > Number 2 is not needed if we dont use fakeroot. > > The alternative approach to fakeroot is to carry patches on top of wic > directory instead of working around such quirks using fakeroot. The > downside > being the need to rebase the local changes on future updates of wic. > > > P4 & P5: > These cherry-picked changes address the --exclude-path issues which > Henning > has addressed in [2]. Should be dropped once that is merged. > > P9: > P9 is needed for --include-path to work. I have observed that > --include-path > does not yet work/broken in the latest OE-core/poky. Posted a query > in [1]. In ISAR > it works with P9 fix. P9 fix could also be relevant upstream and can > be sent there > for review. > > > P9 fix is sent to OE-core mailing list. Currently in master-next. > Hopefully should get merged. > > https://lists.openembedded.org/g/openembedded-core/message/142048 > > I will send a V2 with updated P9 commit. > I would cherry-pick and carry P9 version from OE-core once it is merged > to master. We could drop that > the next time we update wic. > > FWIW, --include-path is fixed in OE-core with the below patch. It is > merged to master. > > https://lists.openembedded.org/g/openembedded-core/message/141865 > Great, well done! Jan > Thanks, > Vijai Kumar K > > > > Apart from the normal qemu build test, Below are the list of > features tested. > KS API: > --exclude-path > --include-path > --fsuuid > --overhead-factor > --fixed-size > --use-label > --use-uuid > > Standalone invocations: > wic ls > wic cp > wic rm > > Currently in the process of finding a way to test wic features > automatically. Some wic features > like -fsuuid cannot be validated till we boot the target. Need to > find a generalized way for > testing the available wic features. The approach in hand is to use a > wic-test.wks file and > generate qemu image with that to validate the wic features. > > [1] https://lists.yoctoproject.org/g/poky/message/12151 > [2] https://groups.google.com/g/isar-users/c/xePCyEl2qhM/m/mwFdeou0AQAJ > > > Henning Schild (1): > lib/oe/path: try hardlinking instead of guessing when it might fail > > Paul Barker (1): > oe.path: Add copyhardlink() helper function > > Vijai Kumar K (8): > wic: Update to the latest wic from openembedded core > wic/plugins: Fix wic plugins to work with the latest wic > wic-img: Satisfy the quirks of latest wic > wic_fakeroot: Handle standalone pseudo invocations > meta-isar/conf: Add provision to debug WIC > debian-common: Add tar as a dependency for wic > scripts/lib/wic: Add /bin to the list of searchpaths > meta-isar/canned-wks: Remove /boot mountpoint > > meta-isar/conf/local.conf.sample | 3 + > .../lib/wic/canned-wks/common-isar.wks.inc | 2 +- > .../scripts/lib/wic/canned-wks/hikey.wks | 2 +- > .../lib/wic/canned-wks/sdimage-efi.wks | 2 +- > meta/classes/wic-img.bbclass | 15 +- > meta/conf/distro/debian-common.conf | 3 +- > meta/lib/oe/path.py | 27 +- > .../wic/plugins/source/bootimg-efi-isar.py | 2 +- > .../wic/plugins/source/bootimg-pcbios-isar.py | 9 +- > .../lib/wic/plugins/source/rootfs-u-boot.py | 2 +- > scripts/lib/scriptpath.py | 32 ++ > scripts/lib/wic/__init__.py | 14 +- > scripts/lib/wic/canned-wks/common.wks.inc | 2 +- > .../directdisk-bootloader-config.cfg | 8 +- > .../lib/wic/canned-wks/efi-bootdisk.wks.in > <http://efi-bootdisk.wks.in> | 3 + > scripts/lib/wic/canned-wks/mkhybridiso.wks | 2 +- > scripts/lib/wic/canned-wks/qemuriscv.wks | 3 + > .../lib/wic/canned-wks/qemux86-directdisk.wks | 2 +- > .../lib/wic/canned-wks/sdimage-bootpart.wks | 4 +- > .../lib/wic/canned-wks/systemd-bootdisk.wks | 4 +- > scripts/lib/wic/engine.py | 421 +++++++++++++++- > scripts/lib/wic/filemap.py | 170 ++++--- > scripts/lib/wic/help.py | 401 ++++++++++++++-- > scripts/lib/wic/ksparser.py | 115 +++-- > scripts/lib/wic/{utils => }/misc.py | 100 ++-- > scripts/lib/wic/partition.py | 234 ++++----- > scripts/lib/wic/pluginbase.py | 36 +- > scripts/lib/wic/plugins/imager/direct.py | 175 ++++--- > .../wic/plugins/source/bootimg-biosplusefi.py | 213 +++++++++ > scripts/lib/wic/plugins/source/bootimg-efi.py | 111 +++-- > .../wic/plugins/source/bootimg-partition.py | 153 ++++-- > .../lib/wic/plugins/source/bootimg-pcbios.py | 91 ++-- > scripts/lib/wic/plugins/source/fsimage.py | 56 --- > .../wic/plugins/source/isoimage-isohybrid.py | 185 +++---- > scripts/lib/wic/plugins/source/rawcopy.py | 44 +- > scripts/lib/wic/plugins/source/rootfs.py | 159 ++++-- > scripts/lib/wic/utils/__init__.py | 0 > scripts/lib/wic/utils/runner.py | 114 ----- > scripts/wic | 452 +++++++++++++----- > scripts/wic_fakeroot | 5 + > 40 files changed, 2340 insertions(+), 1036 deletions(-) > create mode 100644 scripts/lib/scriptpath.py > create mode 100644 scripts/lib/wic/canned-wks/efi-bootdisk.wks.in > <http://efi-bootdisk.wks.in> > create mode 100644 scripts/lib/wic/canned-wks/qemuriscv.wks > rename scripts/lib/wic/{utils => }/misc.py (70%) > create mode 100644 > scripts/lib/wic/plugins/source/bootimg-biosplusefi.py > delete mode 100644 scripts/lib/wic/plugins/source/fsimage.py > delete mode 100644 scripts/lib/wic/utils/__init__.py > delete mode 100644 scripts/lib/wic/utils/runner.py > > -- > 2.17.1 > > -- > You received this message because you are subscribed to the Google > Groups "isar-users" group. > To unsubscribe from this group and stop receiving emails from it, send > an email to isar-users+unsubscribe@googlegroups.com > <mailto:isar-users+unsubscribe@googlegroups.com>. > To view this discussion on the web visit > https://groups.google.com/d/msgid/isar-users/ce2f12b7-f976-41b3-ac52-977bd2a2b8a3n%40googlegroups.com > <https://groups.google.com/d/msgid/isar-users/ce2f12b7-f976-41b3-ac52-977bd2a2b8a3n%40googlegroups.com?utm_medium=email&utm_source=footer>.
The following series updates the wic to the latest one from openembedded core. The git tree is available here. https://github.com/vj-kumar/isar/tree/wic/staging0 Below are summary of key changes in the below series. P3 & P6: The latest version of WIC detects the presence of fakeroot using pseudo_dir(${IMAGE_ROOTFS}/../pseudo). If this is non-existent it doesnot use fakeroot. P3 handles that quirk. There is also a standalone invocation of pseudo which P6 tries to handle via wic_fakeroot. In case of ISAR, we need wic_fakeroot as fakeroot only for 2 purposes. 1. To workaround fsck return values for stretch 2. To handle standalone invocation of pseudo(P4 in series) Number 2 is not needed if we dont use fakeroot. The alternative approach to fakeroot is to carry patches on top of wic directory instead of working around such quirks using fakeroot. The downside being the need to rebase the local changes on future updates of wic. P4 & P5: These cherry-picked changes address the --exclude-path issues which Henning has addressed in [2]. Should be dropped once that is merged. P9: P9 is needed for --include-path to work. I have observed that --include-path does not yet work/broken in the latest OE-core/poky. Posted a query in [1]. In ISAR it works with P9 fix. P9 fix could also be relevant upstream and can be sent there for review. Apart from the normal qemu build test, Below are the list of features tested. KS API: --exclude-path --include-path --fsuuid --overhead-factor --fixed-size --use-label --use-uuid Standalone invocations: wic ls wic cp wic rm Currently in the process of finding a way to test wic features automatically. Some wic features like -fsuuid cannot be validated till we boot the target. Need to find a generalized way for testing the available wic features. The approach in hand is to use a wic-test.wks file and generate qemu image with that to validate the wic features. [1] https://lists.yoctoproject.org/g/poky/message/12151 [2] https://groups.google.com/g/isar-users/c/xePCyEl2qhM/m/mwFdeou0AQAJ Henning Schild (1): lib/oe/path: try hardlinking instead of guessing when it might fail Paul Barker (1): oe.path: Add copyhardlink() helper function Vijai Kumar K (8): wic: Update to the latest wic from openembedded core wic/plugins: Fix wic plugins to work with the latest wic wic-img: Satisfy the quirks of latest wic wic_fakeroot: Handle standalone pseudo invocations meta-isar/conf: Add provision to debug WIC debian-common: Add tar as a dependency for wic scripts/lib/wic: Add /bin to the list of searchpaths meta-isar/canned-wks: Remove /boot mountpoint meta-isar/conf/local.conf.sample | 3 + .../lib/wic/canned-wks/common-isar.wks.inc | 2 +- .../scripts/lib/wic/canned-wks/hikey.wks | 2 +- .../lib/wic/canned-wks/sdimage-efi.wks | 2 +- meta/classes/wic-img.bbclass | 15 +- meta/conf/distro/debian-common.conf | 3 +- meta/lib/oe/path.py | 27 +- .../wic/plugins/source/bootimg-efi-isar.py | 2 +- .../wic/plugins/source/bootimg-pcbios-isar.py | 9 +- .../lib/wic/plugins/source/rootfs-u-boot.py | 2 +- scripts/lib/scriptpath.py | 32 ++ scripts/lib/wic/__init__.py | 14 +- scripts/lib/wic/canned-wks/common.wks.inc | 2 +- .../directdisk-bootloader-config.cfg | 8 +- .../lib/wic/canned-wks/efi-bootdisk.wks.in | 3 + scripts/lib/wic/canned-wks/mkhybridiso.wks | 2 +- scripts/lib/wic/canned-wks/qemuriscv.wks | 3 + .../lib/wic/canned-wks/qemux86-directdisk.wks | 2 +- .../lib/wic/canned-wks/sdimage-bootpart.wks | 4 +- .../lib/wic/canned-wks/systemd-bootdisk.wks | 4 +- scripts/lib/wic/engine.py | 421 +++++++++++++++- scripts/lib/wic/filemap.py | 170 ++++--- scripts/lib/wic/help.py | 401 ++++++++++++++-- scripts/lib/wic/ksparser.py | 115 +++-- scripts/lib/wic/{utils => }/misc.py | 100 ++-- scripts/lib/wic/partition.py | 234 ++++----- scripts/lib/wic/pluginbase.py | 36 +- scripts/lib/wic/plugins/imager/direct.py | 175 ++++--- .../wic/plugins/source/bootimg-biosplusefi.py | 213 +++++++++ scripts/lib/wic/plugins/source/bootimg-efi.py | 111 +++-- .../wic/plugins/source/bootimg-partition.py | 153 ++++-- .../lib/wic/plugins/source/bootimg-pcbios.py | 91 ++-- scripts/lib/wic/plugins/source/fsimage.py | 56 --- .../wic/plugins/source/isoimage-isohybrid.py | 185 +++---- scripts/lib/wic/plugins/source/rawcopy.py | 44 +- scripts/lib/wic/plugins/source/rootfs.py | 159 ++++-- scripts/lib/wic/utils/__init__.py | 0 scripts/lib/wic/utils/runner.py | 114 ----- scripts/wic | 452 +++++++++++++----- scripts/wic_fakeroot | 5 + 40 files changed, 2340 insertions(+), 1036 deletions(-) create mode 100644 scripts/lib/scriptpath.py create mode 100644 scripts/lib/wic/canned-wks/efi-bootdisk.wks.in create mode 100644 scripts/lib/wic/canned-wks/qemuriscv.wks rename scripts/lib/wic/{utils => }/misc.py (70%) create mode 100644 scripts/lib/wic/plugins/source/bootimg-biosplusefi.py delete mode 100644 scripts/lib/wic/plugins/source/fsimage.py delete mode 100644 scripts/lib/wic/utils/__init__.py delete mode 100644 scripts/lib/wic/utils/runner.py