| Message ID | 20231207154114.20400-1-venkata.pyla@toshiba-tsip.com |
|---|---|
| Headers | show |
| Series | Make ext4 images reproducible | expand |
> -----Original Message----- > From: 'MOESSBAUER, Felix' via isar-users <isar-users@googlegroups.com> > Sent: Friday, December 8, 2023 4:00 PM > To: isar-users@googlegroups.com; pyla venkata(ＴＳＩＰ TMIEC ODG > Porting) <Venkata.Pyla@toshiba-tsip.com> > Cc: Kiszka, Jan <jan.kiszka@siemens.com>; hayashi kazuhiro(林 和宏 ＤＭＥ > ○ＤＩＧ□ＭＰＳ○ＭＰ４) <kazuhiro3.hayashi@toshiba.co.jp>; dinesh > kumar(ＴＳＩＰ TMIEC ODG Porting) <dinesh.kumar@toshiba-tsip.com> > Subject: Re: [PATCH 0/5] Make ext4 images reproducible > > On Thu, 2023-12-07 at 21:11 +0530, venkata.pyla@toshiba-tsip.com wrote: > > From: venkata pyla <venkata.pyla@toshiba-tsip.com> > > > > Hi, > > > > The following series of patches will help to fix reproducible build > > issues in ext4 file systems. > > > > Verified with following steps: > > > > ``` > > > > $ git clone http://github.com/ilbers/isar.git $ cd isar $ . > > isar-init-build-env ../build > > > > configure image parameters > > $ cd isar > > $ cat <<EOT >> conf/local.conf > > SOURCE_DATE_EPOCH = "$(git log -1 --pretty=%ct)" > > IMAGE_FSTYPES = "wic" > > WIC_DEPLOY_PARTITIONS = "1" > > EOT > > > > # First Build > > $ cd ../build > > $ sudo rm -rf tmp sstate-cache > > $ bitbake mc:qemuamd64-bookworm:isar-image-base > > $ mv tmp/deploy/images/qemuamd64/ ../qemuamd64-1 > > > > # Second Build > > $ sudo rm -rf tmp sstate-cache > > Hi, > > just out of curiosity: Did you also have a look a building with sstate cache? Are > there reproducibility issues? I just checked, with sstate cache the images are reproducible as well. In my steps I have cleaned the sstate cache, because it should not hide any issues due to caching. > > In the past I fixed a couple of these, but I'm not 100% sure if we still have > some (esp. around the rootfs caching). > > Happy Coding! > Felix > > > $ bitbake mc:qemuamd64-bookworm:isar-image-base > > $ mv tmp/deploy/images/qemuamd64/ ../qemuamd64-2 > > > > # compare ext4 images with diffoscope > > $ diffoscope qemuamd64-1/isar-image-base-debian-bookworm- > > qemuamd64.wic.p2 \ > > qemuamd64-2/isar-image-base-debian-bookworm-qemuamd64.wic.p2 > > > > ``` > > > > venkata pyla (5): > > linux-module: Use debug-prefix-map to build modules > > deterministically > > wic: Set file system uuid to ext4 partitions > > wic: use E2FSPROGS_FAKE_TIME and hash_seed to generate reproducible > > ext4 images > > wic: honor the SOURCE_DATE_EPOCH in case of updated fstab > > wic: make ext2/3/4 images reproducible > > > > .../lib/wic/canned-wks/common-isar.wks.inc | 2 +- > > .../lib/wic/canned-wks/de0-nano-soc.wks.in | 2 +- > > .../lib/wic/canned-wks/efi-plus-pcbios.wks | 2 +- > > .../scripts/lib/wic/canned-wks/hikey.wks | 2 +- > > .../lib/wic/canned-wks/multipart-efi.wks | 4 +- > > .../lib/wic/canned-wks/nanopi-neo-efi.wks.in | 2 +- > > .../lib/wic/canned-wks/nanopi-neo.wks.in | 2 +- > > .../scripts/lib/wic/canned-wks/rpi-sdimg.wks | 2 +- > > .../wic/canned-wks/sdimage-efi-sb-debian.wks | 2 +- > > .../lib/wic/canned-wks/sdimage-efi-sd.wks | 2 +- > > .../lib/wic/canned-wks/sdimage-efi.wks | 2 +- > > .../lib/wic/canned-wks/sifive-fu540.wks | 2 +- > > .../canned-wks/starfive-visionfive2.wks.in | 2 +- > > .../lib/wic/canned-wks/stm32mp15x.wks.in | 2 +- > > .../linux-module/files/debian/rules.tmpl | 5 ++- > > scripts/lib/wic/partition.py | 37 > > ++++++++++++++++++- > > scripts/lib/wic/plugins/imager/direct.py | 3 ++ > > scripts/lib/wic/plugins/source/rootfs.py | 2 +- > > 18 files changed, 59 insertions(+), 18 deletions(-) > > > > -- > You received this message because you are subscribed to the Google Groups > "isar-users" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to isar-users+unsubscribe@googlegroups.com. > To view this discussion on the web visit > https://groups.google.com/d/msgid/isar- > users/3397cb2d254bc8a2e45c47b5a4094e34c85bec32.camel%40siemens.c > om.
From: venkata pyla <venkata.pyla@toshiba-tsip.com> Hi, The following series of patches will help to fix reproducible build issues in ext4 file systems. Verified with following steps: ``` $ git clone http://github.com/ilbers/isar.git $ cd isar $ . isar-init-build-env ../build configure image parameters $ cd isar $ cat <<EOT >> conf/local.conf SOURCE_DATE_EPOCH = "$(git log -1 --pretty=%ct)" IMAGE_FSTYPES = "wic" WIC_DEPLOY_PARTITIONS = "1" EOT # First Build $ cd ../build $ sudo rm -rf tmp sstate-cache $ bitbake mc:qemuamd64-bookworm:isar-image-base $ mv tmp/deploy/images/qemuamd64/ ../qemuamd64-1 # Second Build $ sudo rm -rf tmp sstate-cache $ bitbake mc:qemuamd64-bookworm:isar-image-base $ mv tmp/deploy/images/qemuamd64/ ../qemuamd64-2 # compare ext4 images with diffoscope $ diffoscope qemuamd64-1/isar-image-base-debian-bookworm-qemuamd64.wic.p2 \ qemuamd64-2/isar-image-base-debian-bookworm-qemuamd64.wic.p2 ``` venkata pyla (5): linux-module: Use debug-prefix-map to build modules deterministically wic: Set file system uuid to ext4 partitions wic: use E2FSPROGS_FAKE_TIME and hash_seed to generate reproducible ext4 images wic: honor the SOURCE_DATE_EPOCH in case of updated fstab wic: make ext2/3/4 images reproducible .../lib/wic/canned-wks/common-isar.wks.inc | 2 +- .../lib/wic/canned-wks/de0-nano-soc.wks.in | 2 +- .../lib/wic/canned-wks/efi-plus-pcbios.wks | 2 +- .../scripts/lib/wic/canned-wks/hikey.wks | 2 +- .../lib/wic/canned-wks/multipart-efi.wks | 4 +- .../lib/wic/canned-wks/nanopi-neo-efi.wks.in | 2 +- .../lib/wic/canned-wks/nanopi-neo.wks.in | 2 +- .../scripts/lib/wic/canned-wks/rpi-sdimg.wks | 2 +- .../wic/canned-wks/sdimage-efi-sb-debian.wks | 2 +- .../lib/wic/canned-wks/sdimage-efi-sd.wks | 2 +- .../lib/wic/canned-wks/sdimage-efi.wks | 2 +- .../lib/wic/canned-wks/sifive-fu540.wks | 2 +- .../canned-wks/starfive-visionfive2.wks.in | 2 +- .../lib/wic/canned-wks/stm32mp15x.wks.in | 2 +- .../linux-module/files/debian/rules.tmpl | 5 ++- scripts/lib/wic/partition.py | 37 ++++++++++++++++++- scripts/lib/wic/plugins/imager/direct.py | 3 ++ scripts/lib/wic/plugins/source/rootfs.py | 2 +- 18 files changed, 59 insertions(+), 18 deletions(-)