| Message ID | 20220410170229.6369-1-venkata.pyla@toshiba-tsip.com |
|---|---|
| Headers | show |
| Series | Fix non-reproducible issue due to localepurge | expand |
Am Sun, 10 Apr 2022 22:32:28 +0530 schrieb venkata.pyla@toshiba-tsip.com: > From: venkata pyla <venkata.pyla@toshiba-tsip.com> > > Hello all, > > Replying to this mail conversation, > (https://groups.google.com/g/isar-users/c/JCgpWU_pFJU/m/IwlYTmzQBQAJ) > > For the problem "debconf cache non-reproducible", I figured out a > solution in isar system itself. Generating configuration items and placing them before a package gets installed is very common. It is not only done by Isar but very likely by many packages in debian that cater for other packages. What you are suggesting is not a generic solution to "debconf cache" being non-reproducible. It is just a quirk to hide the problem in that one case for that one package. Maybe the cache just needs to be excluded from the image, or cleaned or whatever. And i guess we should have some infrastructure in Isar to find "non-reproducible" build issues also available for any layers. Because layers is where the real complexity is, not those relatively boring example images you find in isar. regards, Henning > The problem is that, in isar system, in-order to remove unwanted > locales from the system image, it is installing the localepurge > package after its configuration is copied, > due to this when the package post-installation script triggered the > ucf check, it creates a debconf cache entry with the new configuration > change (which is non-reproducible). > > This would be avoided if localpurge package is installed first and > then copy required configuration to remove unwanted to locales from > the system image. > > Created below patch to fix non-reproducible problem, please review > and provide your feedback. > > venkata pyla (1): > image-locales-extension: Do localepurge configuration after > installation > > meta/classes/image-locales-extension.bbclass | 33 > ++++++++++---------- 1 file changed, 17 insertions(+), 16 deletions(-) >
On 11.04.22 12:53, Henning Schild wrote: > Am Sun, 10 Apr 2022 22:32:28 +0530 > schrieb venkata.pyla@toshiba-tsip.com: > >> From: venkata pyla <venkata.pyla@toshiba-tsip.com> >> >> Hello all, >> >> Replying to this mail conversation, >> (https://groups.google.com/g/isar-users/c/JCgpWU_pFJU/m/IwlYTmzQBQAJ) >> >> For the problem "debconf cache non-reproducible", I figured out a >> solution in isar system itself. > > Generating configuration items and placing them before a package gets > installed is very common. It is not only done by Isar but very likely > by many packages in debian that cater for other packages. > > What you are suggesting is not a generic solution to "debconf cache" > being non-reproducible. It is just a quirk to hide the problem in that > one case for that one package. > > Maybe the cache just needs to be excluded from the image, or cleaned or > whatever. > > And i guess we should have some infrastructure in Isar to find > "non-reproducible" build issues also available for any layers. Because > layers is where the real complexity is, not those relatively boring > example images you find in isar. > Just ran into a more complex scenario where non-reproducibility of the image bites: isar-cip-core [1] in secure boot mode generates dm-verity images and a corresponding initramfs which carries the rootfs hash. As we are not able to re-generate the very same rootfs from the same task(-hashes), bitbake will not rebuild the initramfs, and image and initramfs will no longer match. Jan [1] https://gitlab.com/cip-project/cip-core/isar-cip-core
Hi Henning Schild, Thank you for the comments. Please find my comments inline. >-----Original Message----- >From: isar-users@googlegroups.com <isar-users@googlegroups.com> On Behalf >Of Henning Schild >Sent: 11 April 2022 16:24 >To: pyla venkata(ＴＳＩＰ) <Venkata.Pyla@toshiba-tsip.com> >Cc: isar-users@googlegroups.com; jan.kiszka@siemens.com; dinesh kumar(ＴＳ >ＩＰ) <dinesh.kumar@toshiba-tsip.com>; hayashi kazuhiro(林 和宏 □ＳＷＣ◯Ａ >ＣＴ) <kazuhiro3.hayashi@toshiba.co.jp> >Subject: Re: [isar 0/1] Fix non-reproducible issue due to localepurge > >Am Sun, 10 Apr 2022 22:32:28 +0530 >schrieb venkata.pyla@toshiba-tsip.com: > >> From: venkata pyla <venkata.pyla@toshiba-tsip.com> >> >> Hello all, >> >> Replying to this mail conversation, >> (https://groups.google.com/g/isar-users/c/JCgpWU_pFJU/m/IwlYTmzQBQAJ) >> >> For the problem "debconf cache non-reproducible", I figured out a >> solution in isar system itself. > >Generating configuration items and placing them before a package gets >installed is very common. It is not only done by Isar but very likely by many >packages in debian that cater for other packages. > >What you are suggesting is not a generic solution to "debconf cache" >being non-reproducible. It is just a quirk to hide the problem in that one case for >that one package. Yes I agree with you, Ideally this should be fixed from each package if they have such non-reproducible behaviour. I reported a bug today in the Debian package bug-tracking system, though I don’t have solution for that, but I want to start discussion and get some solution going forward. https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1009639 Meanwhile we can use this trick to bypass non-reproducible problem, what's your opinion? > >Maybe the cache just needs to be excluded from the image, or cleaned or >whatever. > >And i guess we should have some infrastructure in Isar to find "non- >reproducible" build issues also available for any layers. Because layers is where >the real complexity is, not those relatively boring example images you find in >isar. > >regards, >Henning > >> The problem is that, in isar system, in-order to remove unwanted >> locales from the system image, it is installing the localepurge >> package after its configuration is copied, due to this when the >> package post-installation script triggered the ucf check, it creates a >> debconf cache entry with the new configuration change (which is >> non-reproducible). >> >> This would be avoided if localpurge package is installed first and >> then copy required configuration to remove unwanted to locales from >> the system image. >> >> Created below patch to fix non-reproducible problem, please review and >> provide your feedback. >> >> venkata pyla (1): >> image-locales-extension: Do localepurge configuration after >> installation >> >> meta/classes/image-locales-extension.bbclass | 33 >> ++++++++++---------- 1 file changed, 17 insertions(+), 16 deletions(-) >> > >-- >You received this message because you are subscribed to the Google Groups >"isar-users" group. >To unsubscribe from this group and stop receiving emails from it, send an email >to isar-users+unsubscribe@googlegroups.com. >To view this discussion on the web visit https://groups.google.com/d/msgid/isar- >users/20220411125354.7ecbe49c%40md1za8fc.ad001.siemens.net.
Am Wed, 13 Apr 2022 12:15:47 +0000 schrieb <Venkata.Pyla@toshiba-tsip.com>: > Hi Henning Schild, > > Thank you for the comments. > > Please find my comments inline. > > >-----Original Message----- > >From: isar-users@googlegroups.com <isar-users@googlegroups.com> On > >Behalf Of Henning Schild > >Sent: 11 April 2022 16:24 > >To: pyla venkata(ＴＳＩＰ) <Venkata.Pyla@toshiba-tsip.com> > >Cc: isar-users@googlegroups.com; jan.kiszka@siemens.com; dinesh > >kumar(ＴＳＩＰ) <dinesh.kumar@toshiba-tsip.com>; hayashi kazuhiro(林 > >和宏 □ＳＷＣ◯ＡＣＴ) <kazuhiro3.hayashi@toshiba.co.jp> > >Subject: Re: [isar 0/1] Fix non-reproducible issue due to localepurge > > > >Am Sun, 10 Apr 2022 22:32:28 +0530 > >schrieb venkata.pyla@toshiba-tsip.com: > > > >> From: venkata pyla <venkata.pyla@toshiba-tsip.com> > >> > >> Hello all, > >> > >> Replying to this mail conversation, > >> (https://groups.google.com/g/isar-users/c/JCgpWU_pFJU/m/IwlYTmzQBQAJ) > >> > >> For the problem "debconf cache non-reproducible", I figured out a > >> solution in isar system itself. > > > >Generating configuration items and placing them before a package gets > >installed is very common. It is not only done by Isar but very > >likely by many packages in debian that cater for other packages. > > > >What you are suggesting is not a generic solution to "debconf cache" > >being non-reproducible. It is just a quirk to hide the problem in > >that one case for that one package. > > Yes I agree with you, Ideally this should be fixed from each package > if they have such non-reproducible behaviour. I would want to bet that there are some packages which show similar problems directly from upstream, but likely more in downstream isar layers. Because a debian package on its own will simply bring its default config, every deviation will come from something else ... like ansible, puppet, isar ... > I reported a bug today in the Debian package bug-tracking system, > though I don’t have solution for that, but I want to start discussion > and get some solution going forward. > https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1009639 I am not sure that is a valid debian but either, they could just tell you to install first and configure later. The non default config is not coming from debian. > Meanwhile we can use this trick to bypass non-reproducible problem, > what's your opinion? I would prefer a way/idea to deal with such problems in general. Like a quality checker script that would warn about a populated conf cache, or a postprocess rule to clear the cache. That would be a real good thing, while your patch is only a workaround for an exotic potential problem most people do not really care too much about. We know anyways that an isar build is never 100% reproducible ... just try a wic run and binary diff the resulting images instead of diffing the files on mounted partitions. regards, Henning > > > >Maybe the cache just needs to be excluded from the image, or cleaned > >or whatever. > > > >And i guess we should have some infrastructure in Isar to find "non- > >reproducible" build issues also available for any layers. Because > >layers is where the real complexity is, not those relatively boring > >example images you find in isar. > > > >regards, > >Henning > > > >> The problem is that, in isar system, in-order to remove unwanted > >> locales from the system image, it is installing the localepurge > >> package after its configuration is copied, due to this when the > >> package post-installation script triggered the ucf check, it > >> creates a debconf cache entry with the new configuration change > >> (which is non-reproducible). > >> > >> This would be avoided if localpurge package is installed first and > >> then copy required configuration to remove unwanted to locales from > >> the system image. > >> > >> Created below patch to fix non-reproducible problem, please review > >> and provide your feedback. > >> > >> venkata pyla (1): > >> image-locales-extension: Do localepurge configuration after > >> installation > >> > >> meta/classes/image-locales-extension.bbclass | 33 > >> ++++++++++---------- 1 file changed, 17 insertions(+), 16 > >> deletions(-) > > > >-- > >You received this message because you are subscribed to the Google > >Groups "isar-users" group. > >To unsubscribe from this group and stop receiving emails from it, > >send an email to isar-users+unsubscribe@googlegroups.com. > >To view this discussion on the web visit > >https://groups.google.com/d/msgid/isar- > >users/20220411125354.7ecbe49c%40md1za8fc.ad001.siemens.net. >
On 14.04.22 10:08, Henning Schild wrote: > Am Wed, 13 Apr 2022 12:15:47 +0000 > schrieb <Venkata.Pyla@toshiba-tsip.com>: > >> Hi Henning Schild, >> >> Thank you for the comments. >> >> Please find my comments inline. >> >>> -----Original Message----- >>> From: isar-users@googlegroups.com <isar-users@googlegroups.com> On >>> Behalf Of Henning Schild >>> Sent: 11 April 2022 16:24 >>> To: pyla venkata(ＴＳＩＰ) <Venkata.Pyla@toshiba-tsip.com> >>> Cc: isar-users@googlegroups.com; jan.kiszka@siemens.com; dinesh >>> kumar(ＴＳＩＰ) <dinesh.kumar@toshiba-tsip.com>; hayashi kazuhiro(林 >>> 和宏 □ＳＷＣ◯ＡＣＴ) <kazuhiro3.hayashi@toshiba.co.jp> >>> Subject: Re: [isar 0/1] Fix non-reproducible issue due to localepurge >>> >>> Am Sun, 10 Apr 2022 22:32:28 +0530 >>> schrieb venkata.pyla@toshiba-tsip.com: >>> >>>> From: venkata pyla <venkata.pyla@toshiba-tsip.com> >>>> >>>> Hello all, >>>> >>>> Replying to this mail conversation, >>>> (https://groups.google.com/g/isar-users/c/JCgpWU_pFJU/m/IwlYTmzQBQAJ) >>>> >>>> For the problem "debconf cache non-reproducible", I figured out a >>>> solution in isar system itself. >>> >>> Generating configuration items and placing them before a package gets >>> installed is very common. It is not only done by Isar but very >>> likely by many packages in debian that cater for other packages. >>> >>> What you are suggesting is not a generic solution to "debconf cache" >>> being non-reproducible. It is just a quirk to hide the problem in >>> that one case for that one package. >> >> Yes I agree with you, Ideally this should be fixed from each package >> if they have such non-reproducible behaviour. > > I would want to bet that there are some packages which show similar > problems directly from upstream, but likely more in downstream isar > layers. Because a debian package on its own will simply bring its > default config, every deviation will come from something else ... like > ansible, puppet, isar ... > >> I reported a bug today in the Debian package bug-tracking system, >> though I don’t have solution for that, but I want to start discussion >> and get some solution going forward. >> https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1009639 > > I am not sure that is a valid debian but either, they could just tell > you to install first and configure later. The non default config is not > coming from debian. > >> Meanwhile we can use this trick to bypass non-reproducible problem, >> what's your opinion? > > I would prefer a way/idea to deal with such problems in > general. Like a quality checker script that would warn about a > populated conf cache, or a postprocess rule to clear the cache. > > That would be a real good thing, while your patch is only a workaround > for an exotic potential problem most people do not really care too much > about. > We know anyways that an isar build is never 100% reproducible ... just > try a wic run and binary diff the resulting images instead of diffing > the files on mounted partitions. > Are wic-related differences coming from the Isar-way of using wic? Or are we sharing issues with OE/Yocto? They claim to be reproducible. We will have to eventually as well. Jan
From: venkata pyla <venkata.pyla@toshiba-tsip.com> Hello all, Replying to this mail conversation, (https://groups.google.com/g/isar-users/c/JCgpWU_pFJU/m/IwlYTmzQBQAJ) For the problem "debconf cache non-reproducible", I figured out a solution in isar system itself. The problem is that, in isar system, in-order to remove unwanted locales from the system image, it is installing the localepurge package after its configuration is copied, due to this when the package post-installation script triggered the ucf check, it creates a debconf cache entry with the new configuration change (which is non-reproducible). This would be avoided if localpurge package is installed first and then copy required configuration to remove unwanted to locales from the system image. Created below patch to fix non-reproducible problem, please review and provide your feedback. venkata pyla (1): image-locales-extension: Do localepurge configuration after installation meta/classes/image-locales-extension.bbclass | 33 ++++++++++---------- 1 file changed, 17 insertions(+), 16 deletions(-)