diff mbox series

[1/3] mmdebstrap: add "skip=cleanup/reproducible" option to mmdebstrap

Message ID 20250305142513.1841415-1-srinuvasan.a@siemens.com
State Superseded, archived
Headers show
Series [1/3] mmdebstrap: add "skip=cleanup/reproducible" option to mmdebstrap | expand

Commit Message

srinuvasan.a March 5, 2025, 2:25 p.m. UTC 
From: srinuvasan <srinuvasan.a@siemens.com>

Without this option, mmdebstrap deletes the var/log/dpkg.log and its siblings by default,
but this is needed to download deb-src packages for base-apt creation.
Basically, we are referring to the package's status under /var/log/dpkg.log file
and performing the debsrc_download operation.

Without these changes, bootstrap related deb-src files are not downloaded
under the /downloads/deb-src folder, due to this the repo sanity check
functions failed during base-apt caching stage.

Signed-off-by: srinuvasan <srinuvasan.a@siemens.com>
---
 meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc | 6 ++++++
 1 file changed, 6 insertions(+)

Comments

Jan Kiszka March 5, 2025, 2:40 p.m. UTC | #1 
On 05.03.25 15:25, srinuvasan.a via isar-users wrote:
> From: srinuvasan <srinuvasan.a@siemens.com>
> 
> Without this option, mmdebstrap deletes the var/log/dpkg.log and its siblings by default,
> but this is needed to download deb-src packages for base-apt creation.
> Basically, we are referring to the package's status under /var/log/dpkg.log file
> and performing the debsrc_download operation.
> 
> Without these changes, bootstrap related deb-src files are not downloaded
> under the /downloads/deb-src folder, due to this the repo sanity check
> functions failed during base-apt caching stage.
> 
> Signed-off-by: srinuvasan <srinuvasan.a@siemens.com>
> ---
>  meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc | 6 ++++++
>  1 file changed, 6 insertions(+)
> 
> diff --git a/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc b/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc
> index 931f6f13..7380a658 100644
> --- a/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc
> +++ b/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc
> @@ -168,6 +168,11 @@ do_bootstrap() {
>          arch_param="$arch_param,${COMPAT_DISTRO_ARCH}"
>      fi
>  
> +    cleanup=
> +    if [ "${@bb.utils.contains('BASE_REPO_FEATURES', 'cache-deb-src', 'yes', 'no', d)}" = "yes" ]; then
> +        cleanup="--skip=cleanup/reproducible"
> +    fi
> +

Uhh, this is a nasty dependency. We must document this and warn the user
that the generated images are now longer reproducible.

Or am I misinterpreting the impact?

Jan

>      # Cleanup mounts if fails
>      trap 'exit 1' INT HUP QUIT TERM ALRM USR1
>      trap '[ -r "${WORKDIR}/mmtmpdir" ] && tmpdir=$(cat "${WORKDIR}/mmtmpdir") \
> @@ -183,6 +188,7 @@ do_bootstrap() {
>  
>      sudo TMPDIR="${BOOTSTRAP_TMPDIR}" mmdebstrap $bootstrap_args \
>                     $arch_param \
> +                   $cleanup \
>                     --mode=unshare \
>                     ${MMHOOKS} \
>                     --setup-hook='mkdir -p "$1/var/cache/apt/archives/"' \
srinuvasan.a March 6, 2025, 6:08 a.m. UTC | #2 
Hi Jan,

           Yes with this option the images are reproducible, I don’t' find other option to enable /var/log/dpkg.log in mmdebstrap, will capture in the documentation.

Thanks,
Srinu

-----Original Message-----
From: Kiszka, Jan (FT RPD CED) <jan.kiszka@siemens.com> 
Sent: 05 March 2025 20:10
To: Arjunan, Srinu (FT FDS CES LX PBU 2) <srinuvasan.a@siemens.com>; isar-users@googlegroups.com
Cc: Hombourger, Cedric (FT FDS CES LX) <cedric.hombourger@siemens.com>; ubely@ilbers.de; Adithya Balakumar <Adithya.Balakumar@toshiba-tsip.com>
Subject: Re: [PATCH 1/3] mmdebstrap: add "skip=cleanup/reproducible" option to mmdebstrap

On 05.03.25 15:25, srinuvasan.a via isar-users wrote:
> From: srinuvasan <srinuvasan.a@siemens.com>
> 
> Without this option, mmdebstrap deletes the var/log/dpkg.log and its 
> siblings by default, but this is needed to download deb-src packages for base-apt creation.
> Basically, we are referring to the package's status under 
> /var/log/dpkg.log file and performing the debsrc_download operation.
> 
> Without these changes, bootstrap related deb-src files are not 
> downloaded under the /downloads/deb-src folder, due to this the repo 
> sanity check functions failed during base-apt caching stage.
> 
> Signed-off-by: srinuvasan <srinuvasan.a@siemens.com>
> ---
>  meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc | 6 ++++++
>  1 file changed, 6 insertions(+)
> 
> diff --git a/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc 
> b/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc
> index 931f6f13..7380a658 100644
> --- a/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc
> +++ b/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc
> @@ -168,6 +168,11 @@ do_bootstrap() {
>          arch_param="$arch_param,${COMPAT_DISTRO_ARCH}"
>      fi
>  
> +    cleanup=
> +    if [ "${@bb.utils.contains('BASE_REPO_FEATURES', 'cache-deb-src', 'yes', 'no', d)}" = "yes" ]; then
> +        cleanup="--skip=cleanup/reproducible"
> +    fi
> +

Uhh, this is a nasty dependency. We must document this and warn the user that the generated images are now longer reproducible.

Or am I misinterpreting the impact?

Jan

>      # Cleanup mounts if fails
>      trap 'exit 1' INT HUP QUIT TERM ALRM USR1
>      trap '[ -r "${WORKDIR}/mmtmpdir" ] && tmpdir=$(cat 
> "${WORKDIR}/mmtmpdir") \ @@ -183,6 +188,7 @@ do_bootstrap() {
>  
>      sudo TMPDIR="${BOOTSTRAP_TMPDIR}" mmdebstrap $bootstrap_args \
>                     $arch_param \
> +                   $cleanup \
>                     --mode=unshare \
>                     ${MMHOOKS} \
>                     --setup-hook='mkdir -p 
> "$1/var/cache/apt/archives/"' \

--
Siemens AG, Foundational Technologies
Linux Expert Center
Jan Kiszka March 6, 2025, 8:20 a.m. UTC | #3 
On 06.03.25 07:08, Arjunan, Srinu (FT FDS CES LX PBU 2) wrote:
> Hi Jan,
> 
>            Yes with this option the images are reproducible, I don’t' find other option to enable /var/log/dpkg.log in mmdebstrap, will capture in the documentation.

Wait: Images are *still* reproducibe, despite skipping the
"cleanup/reproducible" step in mmdebstrap? Even if so, we cannot be sure
that future changes to mmdebstrap would not break this again.

Jan

> 
> Thanks,
> Srinu
> 
> -----Original Message-----
> From: Kiszka, Jan (FT RPD CED) <jan.kiszka@siemens.com> 
> Sent: 05 March 2025 20:10
> To: Arjunan, Srinu (FT FDS CES LX PBU 2) <srinuvasan.a@siemens.com>; isar-users@googlegroups.com
> Cc: Hombourger, Cedric (FT FDS CES LX) <cedric.hombourger@siemens.com>; ubely@ilbers.de; Adithya Balakumar <Adithya.Balakumar@toshiba-tsip.com>
> Subject: Re: [PATCH 1/3] mmdebstrap: add "skip=cleanup/reproducible" option to mmdebstrap
> 
> On 05.03.25 15:25, srinuvasan.a via isar-users wrote:
>> From: srinuvasan <srinuvasan.a@siemens.com>
>>
>> Without this option, mmdebstrap deletes the var/log/dpkg.log and its 
>> siblings by default, but this is needed to download deb-src packages for base-apt creation.
>> Basically, we are referring to the package's status under 
>> /var/log/dpkg.log file and performing the debsrc_download operation.
>>
>> Without these changes, bootstrap related deb-src files are not 
>> downloaded under the /downloads/deb-src folder, due to this the repo 
>> sanity check functions failed during base-apt caching stage.
>>
>> Signed-off-by: srinuvasan <srinuvasan.a@siemens.com>
>> ---
>>  meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc | 6 ++++++
>>  1 file changed, 6 insertions(+)
>>
>> diff --git a/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc 
>> b/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc
>> index 931f6f13..7380a658 100644
>> --- a/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc
>> +++ b/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc
>> @@ -168,6 +168,11 @@ do_bootstrap() {
>>          arch_param="$arch_param,${COMPAT_DISTRO_ARCH}"
>>      fi
>>  
>> +    cleanup=
>> +    if [ "${@bb.utils.contains('BASE_REPO_FEATURES', 'cache-deb-src', 'yes', 'no', d)}" = "yes" ]; then
>> +        cleanup="--skip=cleanup/reproducible"
>> +    fi
>> +
> 
> Uhh, this is a nasty dependency. We must document this and warn the user that the generated images are now longer reproducible.
> 
> Or am I misinterpreting the impact?
> 
> Jan
> 
>>      # Cleanup mounts if fails
>>      trap 'exit 1' INT HUP QUIT TERM ALRM USR1
>>      trap '[ -r "${WORKDIR}/mmtmpdir" ] && tmpdir=$(cat 
>> "${WORKDIR}/mmtmpdir") \ @@ -183,6 +188,7 @@ do_bootstrap() {
>>  
>>      sudo TMPDIR="${BOOTSTRAP_TMPDIR}" mmdebstrap $bootstrap_args \
>>                     $arch_param \
>> +                   $cleanup \
>>                     --mode=unshare \
>>                     ${MMHOOKS} \
>>                     --setup-hook='mkdir -p 
>> "$1/var/cache/apt/archives/"' \
> 
> --
> Siemens AG, Foundational Technologies
> Linux Expert Center
diff mbox series

Patch

diff --git a/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc b/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc
index 931f6f13..7380a658 100644
--- a/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc
+++ b/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc
@@ -168,6 +168,11 @@  do_bootstrap() {
         arch_param="$arch_param,${COMPAT_DISTRO_ARCH}"
     fi
 
+    cleanup=
+    if [ "${@bb.utils.contains('BASE_REPO_FEATURES', 'cache-deb-src', 'yes', 'no', d)}" = "yes" ]; then
+        cleanup="--skip=cleanup/reproducible"
+    fi
+
     # Cleanup mounts if fails
     trap 'exit 1' INT HUP QUIT TERM ALRM USR1
     trap '[ -r "${WORKDIR}/mmtmpdir" ] && tmpdir=$(cat "${WORKDIR}/mmtmpdir") \
@@ -183,6 +188,7 @@  do_bootstrap() {
 
     sudo TMPDIR="${BOOTSTRAP_TMPDIR}" mmdebstrap $bootstrap_args \
                    $arch_param \
+                   $cleanup \
                    --mode=unshare \
                    ${MMHOOKS} \
                    --setup-hook='mkdir -p "$1/var/cache/apt/archives/"' \