diff mbox series

[1/1] image: move deb-src-caching functionality after image creation

Message ID 20250310115149.1977443-2-srinuvasan.a@siemens.com
State New
Headers show
Series Fix deb-src caching issue | expand

Commit Message

srinuvasan.a March 10, 2025, 11:51 a.m. UTC 
From: srinuvasan <srinuvasan.a@siemens.com>

When we do deb-src caching, we are missing to cache imager related packages, because the deb-src-caching
functionality runs in the stage of ROOTFS_POSTPROCESS_COMAMND, but image creation task runs after the
rootfs creation, hence imager related packages are missing in rootfs's var/cache/apt/archives.

To solve this issue run the deb-src-caching after image creation, now we can able to download deb-src for
all the deb packages.

Signed-off-by: srinuvasan <srinuvasan.a@siemens.com>
---
 meta/classes/image.bbclass  | 8 +++++++-
 meta/classes/rootfs.bbclass | 3 ++-
 2 files changed, 9 insertions(+), 2 deletions(-)

Comments

srinuvasan.a March 18, 2025, 4:45 p.m. UTC | #1 
-----Original Message-----
From: Arjunan, Srinu (FT FDS CES LX PBU 2) <srinuvasan.a@siemens.com> 
Sent: 10 March 2025 17:22
To: isar-users@googlegroups.com
Cc: Kiszka, Jan (FT RPD CED) <jan.kiszka@siemens.com>; Hombourger, Cedric (FT FDS CES LX) <cedric.hombourger@siemens.com>; ubely@ilbers.de; Niedermayr, Benedikt (FT RPD CED OES-DE) <benedikt.niedermayr@siemens.com>; Adithya.Balakumar@toshiba-tsip.com; Arjunan, Srinu (FT FDS CES LX PBU 2) <srinuvasan.a@siemens.com>
Subject: [PATCH 1/1] image: move deb-src-caching functionality after image creation

From: srinuvasan <srinuvasan.a@siemens.com>

When we do deb-src caching, we are missing to cache imager related packages, because the deb-src-caching functionality runs in the stage of ROOTFS_POSTPROCESS_COMAMND, but image creation task runs after the rootfs creation, hence imager related packages are missing in rootfs's var/cache/apt/archives.

To solve this issue run the deb-src-caching after image creation, now we can able to download deb-src for all the deb packages.

Signed-off-by: srinuvasan <srinuvasan.a@siemens.com>
---
 meta/classes/image.bbclass  | 8 +++++++-  meta/classes/rootfs.bbclass | 3 ++-
 2 files changed, 9 insertions(+), 2 deletions(-)

diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass index 56eca202..66b4d313 100644
--- a/meta/classes/image.bbclass
+++ b/meta/classes/image.bbclass
@@ -72,7 +72,13 @@ ROOTFS_MANIFEST_DEPLOY_DIR ?= "${DEPLOY_DIR_IMAGE}"
 ROOTFS_DPKGSTATUS_DEPLOY_DIR ?= "${DEPLOY_DIR_IMAGE}"
 ROOTFS_PACKAGE_SUFFIX ?= "${PN}-${DISTRO}-${MACHINE}"
 
-ROOTFS_POSTPROCESS_COMMAND:prepend = "${@bb.utils.contains('BASE_REPO_FEATURES', 'cache-deb-src', 'cache_deb_src', '', d)} "
+CACHE_DEB_SRC = "${@bb.utils.contains('BASE_REPO_FEATURES', 'cache-deb-src', '1', '0', d)}"
+
+python () {
+    if bb.utils.to_boolean(d.getVar('CACHE_DEB_SRC')):
+        bb.build.addtask('do_cache_deb_src', 'do_deploy', 'do_image', 
+d) }
+
 
 inherit rootfs
 inherit sdk
diff --git a/meta/classes/rootfs.bbclass b/meta/classes/rootfs.bbclass index 2348e269..c9e832dc 100644
--- a/meta/classes/rootfs.bbclass
+++ b/meta/classes/rootfs.bbclass
@@ -272,7 +272,8 @@ python do_rootfs_install() {  }  addtask rootfs_install before do_rootfs_postprocess after do_unpack
 
-cache_deb_src() {
+do_cache_deb_src[network] = "${TASK_USE_SUDO}"
+do_cache_deb_src() {
     if [ -e "${ROOTFSDIR}"/etc/resolv.conf ] ||
        [ -h "${ROOTFSDIR}"/etc/resolv.conf ]; then
         sudo mv "${ROOTFSDIR}"/etc/resolv.conf "${ROOTFSDIR}"/etc/resolv.conf.isar
--
2.39.5

Hi All,

          Can you please update the status of this patch?, or anything missing from my end?

Many thanks,
Srinu
Uladzimir Bely March 20, 2025, 5:31 a.m. UTC | #2 
On Tue, 2025-03-18 at 16:45 +0000, Arjunan, Srinu wrote:
> 
> 
> -----Original Message-----
> From: Arjunan, Srinu (FT FDS CES LX PBU 2) <srinuvasan.a@siemens.com>
> Sent: 10 March 2025 17:22
> To: isar-users@googlegroups.com
> Cc: Kiszka, Jan (FT RPD CED) <jan.kiszka@siemens.com>; Hombourger,
> Cedric (FT FDS CES LX) <cedric.hombourger@siemens.com>;
> ubely@ilbers.de; Niedermayr, Benedikt (FT RPD CED OES-DE)
> <benedikt.niedermayr@siemens.com>;
> Adithya.Balakumar@toshiba-tsip.com; Arjunan, Srinu (FT FDS CES LX PBU
> 2) <srinuvasan.a@siemens.com>
> Subject: [PATCH 1/1] image: move deb-src-caching functionality after
> image creation
> 
> From: srinuvasan <srinuvasan.a@siemens.com>
> 
> When we do deb-src caching, we are missing to cache imager related
> packages, because the deb-src-caching functionality runs in the stage
> of ROOTFS_POSTPROCESS_COMAMND, but image creation task runs after the
> rootfs creation, hence imager related packages are missing in
> rootfs's var/cache/apt/archives.
> 
> To solve this issue run the deb-src-caching after image creation, now
> we can able to download deb-src for all the deb packages.
> 
> Signed-off-by: srinuvasan <srinuvasan.a@siemens.com>
> ---
>  meta/classes/image.bbclass  | 8 +++++++- 
> meta/classes/rootfs.bbclass | 3 ++-
>  2 files changed, 9 insertions(+), 2 deletions(-)
> 
> diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass
> index 56eca202..66b4d313 100644
> --- a/meta/classes/image.bbclass
> +++ b/meta/classes/image.bbclass
> @@ -72,7 +72,13 @@ ROOTFS_MANIFEST_DEPLOY_DIR ?=
> "${DEPLOY_DIR_IMAGE}"
>  ROOTFS_DPKGSTATUS_DEPLOY_DIR ?= "${DEPLOY_DIR_IMAGE}"
>  ROOTFS_PACKAGE_SUFFIX ?= "${PN}-${DISTRO}-${MACHINE}"
>  
> -ROOTFS_POSTPROCESS_COMMAND:prepend =
> "${@bb.utils.contains('BASE_REPO_FEATURES', 'cache-deb-src',
> 'cache_deb_src', '', d)} "
> +CACHE_DEB_SRC = "${@bb.utils.contains('BASE_REPO_FEATURES', 'cache-
> deb-src', '1', '0', d)}"
> +
> +python () {
> +    if bb.utils.to_boolean(d.getVar('CACHE_DEB_SRC')):
> +        bb.build.addtask('do_cache_deb_src', 'do_deploy',
> 'do_image', 
> +d) }
> +
>  
>  inherit rootfs
>  inherit sdk
> diff --git a/meta/classes/rootfs.bbclass
> b/meta/classes/rootfs.bbclass index 2348e269..c9e832dc 100644
> --- a/meta/classes/rootfs.bbclass
> +++ b/meta/classes/rootfs.bbclass
> @@ -272,7 +272,8 @@ python do_rootfs_install() {  }  addtask
> rootfs_install before do_rootfs_postprocess after do_unpack
>  
> -cache_deb_src() {
> +do_cache_deb_src[network] = "${TASK_USE_SUDO}"
> +do_cache_deb_src() {
>      if [ -e "${ROOTFSDIR}"/etc/resolv.conf ] ||
>         [ -h "${ROOTFSDIR}"/etc/resolv.conf ]; then
>          sudo mv "${ROOTFSDIR}"/etc/resolv.conf
> "${ROOTFSDIR}"/etc/resolv.conf.isar
> --
> 2.39.5
> 
> Hi All,
> 
>           Can you please update the status of this patch?, or
> anything missing from my end?
> 

Hello.

Sorry for delays related to "installer" currently breaking full CI on
"next" branch.

Tested recently your patch, it also makes CI fail at:

CrossTest.test_cross_debsrc:

```
[stdlog] 2025-03-19 21:29:50,542 avocado.app ERROR| ERROR: Task
(mc:qemuarm64-bookworm:/workspace/build/isar_ub_devel_fast/214/meta-
test/recipes-core/images/isar-image-ci.bb:do_cache_deb_src) failed with
exit code '1'
[stdlog] 2025-03-19 21:30:44,921 avocado.test INFO | | DEBUG: Executing
shell function do_cache_deb_src
[stdlog] 2025-03-19 21:30:44,921 avocado.test INFO | | Reading package
lists...
[stdlog] 2025-03-19 21:30:44,921 avocado.test INFO | | E: You must put
some 'deb-src' URIs in your sources.list
```

NoCrossTest.test_nocross_debsrc:

```
[stdlog] 2025-03-20 01:09:35,759 avocado.app ERROR| ERROR: Logfile of
failure stored in:
/workspace/build/isar_ub_devel/126/build/tmp/work/debian-bookworm-
armhf/isar-image-ci-qemuarm/1.0-r0/temp/log.do_cache_deb_src.218291
[stdlog] 2025-03-20 01:09:35,832 avocado.app ERROR| ERROR: Task
(mc:qemuarm-bookworm:/workspace/build/isar_ub_devel/126/meta-
test/recipes-core/images/isar-image-ci.bb:do_cache_deb_src) failed with
exit code '1'
[stdlog] 2025-03-20 01:19:35,822 avocado.test INFO | | DEBUG: Executing
shell function do_cache_deb_src
[stdlog] 2025-03-20 01:19:35,823 avocado.test INFO | | Reading package
lists...
[stdlog] 2025-03-20 01:19:35,823 avocado.test INFO | | E: You must put
some 'deb-src' URIs in your sources.list
```

> Many thanks,
> Srinu
>
srinuvasan.a March 20, 2025, 7:18 a.m. UTC | #3 
-----Original Message-----
From: Uladzimir Bely <ubely@ilbers.de> 
Sent: 20 March 2025 11:02
To: Arjunan, Srinu (FT FDS CES LX PBU 2) <srinuvasan.a@siemens.com>; isar-users@googlegroups.com
Cc: Kiszka, Jan (FT RPD CED) <jan.kiszka@siemens.com>; Hombourger, Cedric (FT FDS CES LX) <cedric.hombourger@siemens.com>; Niedermayr, Benedikt (FT RPD CED OES-DE) <benedikt.niedermayr@siemens.com>; Adithya.Balakumar@toshiba-tsip.com
Subject: Re: [PATCH 1/1] image: move deb-src-caching functionality after image creation

On Tue, 2025-03-18 at 16:45 +0000, Arjunan, Srinu wrote:
> 
> 
> -----Original Message-----
> From: Arjunan, Srinu (FT FDS CES LX PBU 2) <srinuvasan.a@siemens.com>
> Sent: 10 March 2025 17:22
> To: isar-users@googlegroups.com
> Cc: Kiszka, Jan (FT RPD CED) <jan.kiszka@siemens.com>; Hombourger, 
> Cedric (FT FDS CES LX) <cedric.hombourger@siemens.com>; 
> ubely@ilbers.de; Niedermayr, Benedikt (FT RPD CED OES-DE) 
> <benedikt.niedermayr@siemens.com>;
> Adithya.Balakumar@toshiba-tsip.com; Arjunan, Srinu (FT FDS CES LX PBU
> 2) <srinuvasan.a@siemens.com>
> Subject: [PATCH 1/1] image: move deb-src-caching functionality after 
> image creation
> 
> From: srinuvasan <srinuvasan.a@siemens.com>
> 
> When we do deb-src caching, we are missing to cache imager related 
> packages, because the deb-src-caching functionality runs in the stage 
> of ROOTFS_POSTPROCESS_COMAMND, but image creation task runs after the 
> rootfs creation, hence imager related packages are missing in rootfs's 
> var/cache/apt/archives.
> 
> To solve this issue run the deb-src-caching after image creation, now 
> we can able to download deb-src for all the deb packages.
> 
> Signed-off-by: srinuvasan <srinuvasan.a@siemens.com>
> ---
>  meta/classes/image.bbclass  | 8 +++++++- meta/classes/rootfs.bbclass 
> | 3 ++-
>  2 files changed, 9 insertions(+), 2 deletions(-)
> 
> diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass 
> index 56eca202..66b4d313 100644
> --- a/meta/classes/image.bbclass
> +++ b/meta/classes/image.bbclass
> @@ -72,7 +72,13 @@ ROOTFS_MANIFEST_DEPLOY_DIR ?= "${DEPLOY_DIR_IMAGE}"
>  ROOTFS_DPKGSTATUS_DEPLOY_DIR ?= "${DEPLOY_DIR_IMAGE}"
>  ROOTFS_PACKAGE_SUFFIX ?= "${PN}-${DISTRO}-${MACHINE}"
>  
> -ROOTFS_POSTPROCESS_COMMAND:prepend =
> "${@bb.utils.contains('BASE_REPO_FEATURES', 'cache-deb-src', 
> 'cache_deb_src', '', d)} "
> +CACHE_DEB_SRC = "${@bb.utils.contains('BASE_REPO_FEATURES', 'cache-
> deb-src', '1', '0', d)}"
> +
> +python () {
> +    if bb.utils.to_boolean(d.getVar('CACHE_DEB_SRC')):
> +        bb.build.addtask('do_cache_deb_src', 'do_deploy',
> 'do_image',
> +d) }
> +
>  
>  inherit rootfs
>  inherit sdk
> diff --git a/meta/classes/rootfs.bbclass b/meta/classes/rootfs.bbclass 
> index 2348e269..c9e832dc 100644
> --- a/meta/classes/rootfs.bbclass
> +++ b/meta/classes/rootfs.bbclass
> @@ -272,7 +272,8 @@ python do_rootfs_install() {  }  addtask 
> rootfs_install before do_rootfs_postprocess after do_unpack
>  
> -cache_deb_src() {
> +do_cache_deb_src[network] = "${TASK_USE_SUDO}"
> +do_cache_deb_src() {
>      if [ -e "${ROOTFSDIR}"/etc/resolv.conf ] ||
>         [ -h "${ROOTFSDIR}"/etc/resolv.conf ]; then
>          sudo mv "${ROOTFSDIR}"/etc/resolv.conf 
> "${ROOTFSDIR}"/etc/resolv.conf.isar
> --
> 2.39.5
> 
> Hi All,
> 
>           Can you please update the status of this patch?, or anything 
> missing from my end?
> 

Hello.

Sorry for delays related to "installer" currently breaking full CI on "next" branch.

Tested recently your patch, it also makes CI fail at:

CrossTest.test_cross_debsrc:

```
[stdlog] 2025-03-19 21:29:50,542 avocado.app ERROR| ERROR: Task
(mc:qemuarm64-bookworm:/workspace/build/isar_ub_devel_fast/214/meta-
test/recipes-core/images/isar-image-ci.bb:do_cache_deb_src) failed with exit code '1'
[stdlog] 2025-03-19 21:30:44,921 avocado.test INFO | | DEBUG: Executing shell function do_cache_deb_src [stdlog] 2025-03-19 21:30:44,921 avocado.test INFO | | Reading package lists...
[stdlog] 2025-03-19 21:30:44,921 avocado.test INFO | | E: You must put some 'deb-src' URIs in your sources.list ```

NoCrossTest.test_nocross_debsrc:

```
[stdlog] 2025-03-20 01:09:35,759 avocado.app ERROR| ERROR: Logfile of failure stored in:
/workspace/build/isar_ub_devel/126/build/tmp/work/debian-bookworm-
armhf/isar-image-ci-qemuarm/1.0-r0/temp/log.do_cache_deb_src.218291
[stdlog] 2025-03-20 01:09:35,832 avocado.app ERROR| ERROR: Task
(mc:qemuarm-bookworm:/workspace/build/isar_ub_devel/126/meta-
test/recipes-core/images/isar-image-ci.bb:do_cache_deb_src) failed with exit code '1'
[stdlog] 2025-03-20 01:19:35,822 avocado.test INFO | | DEBUG: Executing shell function do_cache_deb_src [stdlog] 2025-03-20 01:19:35,823 avocado.test INFO | | Reading package lists...
[stdlog] 2025-03-20 01:19:35,823 avocado.test INFO | | E: You must put some 'deb-src' URIs in your sources.list ```


Hi Uladzimir,

                      Thank you for the reply, I am not able to reproduce the build issue what you reported , I triggered the build for isar-image-ci target and it's getting passed.

This patch depend on Cedric patch [1]
[1]: https://groups.google.com/g/isar-users/c/OwrSBmCPe0Y

I validated my patch along with the [1], it seems working fine.

Or Can you please provide the steps to reproduce the issue?


> Many thanks,
> Srinu
> 

--
Best regards,
Uladzimir.
Cedric Hombourger March 20, 2025, 8:27 a.m. UTC | #4 
On Thu, 2025-03-20 at 07:18 +0000, Arjunan, Srinu (FT FDS CES LX PBU 2)
wrote:
>
>
> -----Original Message-----
> From: Uladzimir Bely <ubely@ilbers.de>
> Sent: 20 March 2025 11:02
> To: Arjunan, Srinu (FT FDS CES LX PBU 2) <srinuvasan.a@siemens.com>;
> isar-users@googlegroups.com
> Cc: Kiszka, Jan (FT RPD CED) <jan.kiszka@siemens.com>; Hombourger,
> Cedric (FT FDS CES LX) <cedric.hombourger@siemens.com>; Niedermayr,
> Benedikt (FT RPD CED OES-DE) <benedikt.niedermayr@siemens.com>;
> Adithya.Balakumar@toshiba-tsip.com
> Subject: Re: [PATCH 1/1] image: move deb-src-caching functionality
> after image creation
>
> On Tue, 2025-03-18 at 16:45 +0000, Arjunan, Srinu wrote:
> >
> >
> > -----Original Message-----
> > From: Arjunan, Srinu (FT FDS CES LX PBU 2)
> > <srinuvasan.a@siemens.com>
> > Sent: 10 March 2025 17:22
> > To: isar-users@googlegroups.com
> > Cc: Kiszka, Jan (FT RPD CED) <jan.kiszka@siemens.com>; Hombourger,
> > Cedric (FT FDS CES LX) <cedric.hombourger@siemens.com>;
> > ubely@ilbers.de; Niedermayr, Benedikt (FT RPD CED OES-DE)
> > <benedikt.niedermayr@siemens.com>;
> > Adithya.Balakumar@toshiba-tsip.com; Arjunan, Srinu (FT FDS CES LX
> > PBU
> > 2) <srinuvasan.a@siemens.com>
> > Subject: [PATCH 1/1] image: move deb-src-caching functionality
> > after
> > image creation
> >
> > From: srinuvasan <srinuvasan.a@siemens.com>
> >
> > When we do deb-src caching, we are missing to cache imager related
> > packages, because the deb-src-caching functionality runs in the
> > stage
> > of ROOTFS_POSTPROCESS_COMAMND, but image creation task runs after
> > the
> > rootfs creation, hence imager related packages are missing in
> > rootfs's
> > var/cache/apt/archives.
> >
> > To solve this issue run the deb-src-caching after image creation,
> > now
> > we can able to download deb-src for all the deb packages.
> >
> > Signed-off-by: srinuvasan <srinuvasan.a@siemens.com>
> > ---
> >  meta/classes/image.bbclass  | 8 +++++++-
> > meta/classes/rootfs.bbclass
> > > 3 ++-
> >  2 files changed, 9 insertions(+), 2 deletions(-)
> >
> > diff --git a/meta/classes/image.bbclass
> > b/meta/classes/image.bbclass
> > index 56eca202..66b4d313 100644
> > --- a/meta/classes/image.bbclass
> > +++ b/meta/classes/image.bbclass
> > @@ -72,7 +72,13 @@ ROOTFS_MANIFEST_DEPLOY_DIR ?=
> > "${DEPLOY_DIR_IMAGE}"
> >  ROOTFS_DPKGSTATUS_DEPLOY_DIR ?= "${DEPLOY_DIR_IMAGE}"
> >  ROOTFS_PACKAGE_SUFFIX ?= "${PN}-${DISTRO}-${MACHINE}"
> >
> > -ROOTFS_POSTPROCESS_COMMAND:prepend =
> > "${@bb.utils.contains('BASE_REPO_FEATURES', 'cache-deb-src',
> > 'cache_deb_src', '', d)} "
> > +CACHE_DEB_SRC = "${@bb.utils.contains('BASE_REPO_FEATURES',
> > 'cache-
> > deb-src', '1', '0', d)}"
> > +
> > +python () {
> > +    if bb.utils.to_boolean(d.getVar('CACHE_DEB_SRC')):
> > +        bb.build.addtask('do_cache_deb_src', 'do_deploy',
> > 'do_image',
> > +d) }
> > +
> >
> >  inherit rootfs
> >  inherit sdk
> > diff --git a/meta/classes/rootfs.bbclass
> > b/meta/classes/rootfs.bbclass
> > index 2348e269..c9e832dc 100644
> > --- a/meta/classes/rootfs.bbclass
> > +++ b/meta/classes/rootfs.bbclass
> > @@ -272,7 +272,8 @@ python do_rootfs_install() {  }  addtask
> > rootfs_install before do_rootfs_postprocess after do_unpack
> >
> > -cache_deb_src() {
> > +do_cache_deb_src[network] = "${TASK_USE_SUDO}"
> > +do_cache_deb_src() {
> >      if [ -e "${ROOTFSDIR}"/etc/resolv.conf ] ||
> >         [ -h "${ROOTFSDIR}"/etc/resolv.conf ]; then
> >          sudo mv "${ROOTFSDIR}"/etc/resolv.conf
> > "${ROOTFSDIR}"/etc/resolv.conf.isar
> > --
> > 2.39.5
> >
> > Hi All,
> >
> >           Can you please update the status of this patch?, or
> > anything
> > missing from my end?
> >
>
> Hello.
>
> Sorry for delays related to "installer" currently breaking full CI on
> "next" branch.
>
> Tested recently your patch, it also makes CI fail at:
>
> CrossTest.test_cross_debsrc:
>
> ```
> [stdlog] 2025-03-19 21:29:50,542 avocado.app ERROR| ERROR: Task
> (mc:qemuarm64-bookworm:/workspace/build/isar_ub_devel_fast/214/meta-
> test/recipes-core/images/isar-image-ci.bb:do_cache_deb_src) failed
> with exit code '1'
> [stdlog] 2025-03-19 21:30:44,921 avocado.test INFO | | DEBUG:
> Executing shell function do_cache_deb_src [stdlog] 2025-03-19
> 21:30:44,921 avocado.test INFO | | Reading package lists...
> [stdlog] 2025-03-19 21:30:44,921 avocado.test INFO | | E: You must
> put some 'deb-src' URIs in your sources.list ```
>
> NoCrossTest.test_nocross_debsrc:
>
> ```
> [stdlog] 2025-03-20 01:09:35,759 avocado.app ERROR| ERROR: Logfile of
> failure stored in:
> /workspace/build/isar_ub_devel/126/build/tmp/work/debian-bookworm-
> armhf/isar-image-ci-qemuarm/1.0-r0/temp/log.do_cache_deb_src.218291
> [stdlog] 2025-03-20 01:09:35,832 avocado.app ERROR| ERROR: Task
> (mc:qemuarm-bookworm:/workspace/build/isar_ub_devel/126/meta-
> test/recipes-core/images/isar-image-ci.bb:do_cache_deb_src) failed
> with exit code '1'
> [stdlog] 2025-03-20 01:19:35,822 avocado.test INFO | | DEBUG:
> Executing shell function do_cache_deb_src [stdlog] 2025-03-20
> 01:19:35,823 avocado.test INFO | | Reading package lists...
> [stdlog] 2025-03-20 01:19:35,823 avocado.test INFO | | E: You must
> put some 'deb-src' URIs in your sources.list ```
>
>
> Hi Uladzimir,
>
>                       Thank you for the reply, I am not able to
> reproduce the build issue what you reported , I triggered the build
> for isar-image-ci target and it's getting passed.
>
> This patch depend on Cedric patch [1]
> [1]:
> https://groups.google.com/g/isar-users/c/OwrSBmCPe0Y
>
> I validated my patch along with the [1], it seems working fine.
>
> Or Can you please provide the steps to reproduce the issue?

See
https://github.com/ilbers/isar/blob/master/CONTRIBUTING.md#development
and in particular section 3

You noted that you ran "build for isar-image-ci target" but that may
not fully reproduce what the CI is doing

In addition, you really need to stop using Outlook (or check how to get
it to properly quote the original message)

>
>
> > Many thanks,
> > Srinu
> >
>
> --
> Best regards,
> Uladzimir.

--
Cedric Hombourger
Siemens AG
http://www.siemens.com/
Uladzimir Bely March 20, 2025, 5:56 p.m. UTC | #5 
On Thu, 2025-03-20 at 07:18 +0000, Arjunan, Srinu wrote:
> 
> 
> -----Original Message-----
> From: Uladzimir Bely <ubely@ilbers.de> 
> Sent: 20 March 2025 11:02
> To: Arjunan, Srinu (FT FDS CES LX PBU 2) <srinuvasan.a@siemens.com>;
> isar-users@googlegroups.com
> Cc: Kiszka, Jan (FT RPD CED) <jan.kiszka@siemens.com>; Hombourger,
> Cedric (FT FDS CES LX) <cedric.hombourger@siemens.com>; Niedermayr,
> Benedikt (FT RPD CED OES-DE) <benedikt.niedermayr@siemens.com>;
> Adithya.Balakumar@toshiba-tsip.com
> Subject: Re: [PATCH 1/1] image: move deb-src-caching functionality
> after image creation
> 
> On Tue, 2025-03-18 at 16:45 +0000, Arjunan, Srinu wrote:
> > 
> > 
> > -----Original Message-----
> > From: Arjunan, Srinu (FT FDS CES LX PBU 2)
> > <srinuvasan.a@siemens.com>
> > Sent: 10 March 2025 17:22
> > To: isar-users@googlegroups.com
> > Cc: Kiszka, Jan (FT RPD CED) <jan.kiszka@siemens.com>; Hombourger, 
> > Cedric (FT FDS CES LX) <cedric.hombourger@siemens.com>; 
> > ubely@ilbers.de; Niedermayr, Benedikt (FT RPD CED OES-DE) 
> > <benedikt.niedermayr@siemens.com>;
> > Adithya.Balakumar@toshiba-tsip.com; Arjunan, Srinu (FT FDS CES LX
> > PBU
> > 2) <srinuvasan.a@siemens.com>
> > Subject: [PATCH 1/1] image: move deb-src-caching functionality
> > after 
> > image creation
> > 
> > From: srinuvasan <srinuvasan.a@siemens.com>
> > 
> > When we do deb-src caching, we are missing to cache imager related 
> > packages, because the deb-src-caching functionality runs in the
> > stage 
> > of ROOTFS_POSTPROCESS_COMAMND, but image creation task runs after
> > the 
> > rootfs creation, hence imager related packages are missing in
> > rootfs's 
> > var/cache/apt/archives.
> > 
> > To solve this issue run the deb-src-caching after image creation,
> > now 
> > we can able to download deb-src for all the deb packages.
> > 
> > Signed-off-by: srinuvasan <srinuvasan.a@siemens.com>
> > ---
> >  meta/classes/image.bbclass  | 8 +++++++-
> > meta/classes/rootfs.bbclass 
> > > 3 ++-
> >  2 files changed, 9 insertions(+), 2 deletions(-)
> > 
> > diff --git a/meta/classes/image.bbclass
> > b/meta/classes/image.bbclass 
> > index 56eca202..66b4d313 100644
> > --- a/meta/classes/image.bbclass
> > +++ b/meta/classes/image.bbclass
> > @@ -72,7 +72,13 @@ ROOTFS_MANIFEST_DEPLOY_DIR ?=
> > "${DEPLOY_DIR_IMAGE}"
> >  ROOTFS_DPKGSTATUS_DEPLOY_DIR ?= "${DEPLOY_DIR_IMAGE}"
> >  ROOTFS_PACKAGE_SUFFIX ?= "${PN}-${DISTRO}-${MACHINE}"
> >  
> > -ROOTFS_POSTPROCESS_COMMAND:prepend =
> > "${@bb.utils.contains('BASE_REPO_FEATURES', 'cache-deb-src', 
> > 'cache_deb_src', '', d)} "
> > +CACHE_DEB_SRC = "${@bb.utils.contains('BASE_REPO_FEATURES',
> > 'cache-
> > deb-src', '1', '0', d)}"
> > +
> > +python () {
> > +    if bb.utils.to_boolean(d.getVar('CACHE_DEB_SRC')):
> > +        bb.build.addtask('do_cache_deb_src', 'do_deploy',
> > 'do_image',
> > +d) }
> > +
> >  
> >  inherit rootfs
> >  inherit sdk
> > diff --git a/meta/classes/rootfs.bbclass
> > b/meta/classes/rootfs.bbclass 
> > index 2348e269..c9e832dc 100644
> > --- a/meta/classes/rootfs.bbclass
> > +++ b/meta/classes/rootfs.bbclass
> > @@ -272,7 +272,8 @@ python do_rootfs_install() {  }  addtask 
> > rootfs_install before do_rootfs_postprocess after do_unpack
> >  
> > -cache_deb_src() {
> > +do_cache_deb_src[network] = "${TASK_USE_SUDO}"
> > +do_cache_deb_src() {
> >      if [ -e "${ROOTFSDIR}"/etc/resolv.conf ] ||
> >         [ -h "${ROOTFSDIR}"/etc/resolv.conf ]; then
> >          sudo mv "${ROOTFSDIR}"/etc/resolv.conf 
> > "${ROOTFSDIR}"/etc/resolv.conf.isar
> > --
> > 2.39.5
> > 
> > Hi All,
> > 
> >           Can you please update the status of this patch?, or
> > anything 
> > missing from my end?
> > 
> 
> Hello.
> 
> Sorry for delays related to "installer" currently breaking full CI on
> "next" branch.
> 
> Tested recently your patch, it also makes CI fail at:
> 
> CrossTest.test_cross_debsrc:
> 
> ```
> [stdlog] 2025-03-19 21:29:50,542 avocado.app ERROR| ERROR: Task
> (mc:qemuarm64-bookworm:/workspace/build/isar_ub_devel_fast/214/meta-
> test/recipes-core/images/isar-image-ci.bb:do_cache_deb_src) failed
> with exit code '1'
> [stdlog] 2025-03-19 21:30:44,921 avocado.test INFO | | DEBUG:
> Executing shell function do_cache_deb_src [stdlog] 2025-03-19
> 21:30:44,921 avocado.test INFO | | Reading package lists...
> [stdlog] 2025-03-19 21:30:44,921 avocado.test INFO | | E: You must
> put some 'deb-src' URIs in your sources.list ```
> 
> NoCrossTest.test_nocross_debsrc:
> 
> ```
> [stdlog] 2025-03-20 01:09:35,759 avocado.app ERROR| ERROR: Logfile of
> failure stored in:
> /workspace/build/isar_ub_devel/126/build/tmp/work/debian-bookworm-
> armhf/isar-image-ci-qemuarm/1.0-r0/temp/log.do_cache_deb_src.218291
> [stdlog] 2025-03-20 01:09:35,832 avocado.app ERROR| ERROR: Task
> (mc:qemuarm-bookworm:/workspace/build/isar_ub_devel/126/meta-
> test/recipes-core/images/isar-image-ci.bb:do_cache_deb_src) failed
> with exit code '1'
> [stdlog] 2025-03-20 01:19:35,822 avocado.test INFO | | DEBUG:
> Executing shell function do_cache_deb_src [stdlog] 2025-03-20
> 01:19:35,823 avocado.test INFO | | Reading package lists...
> [stdlog] 2025-03-20 01:19:35,823 avocado.test INFO | | E: You must
> put some 'deb-src' URIs in your sources.list ```
> 
> 
> Hi Uladzimir,
> 
>                       Thank you for the reply, I am not able to
> reproduce the build issue what you reported , I triggered the build
> for isar-image-ci target and it's getting passed.
> 
> This patch depend on Cedric patch [1]
> [1]: https://groups.google.com/g/isar-users/c/OwrSBmCPe0Y
> 
> I validated my patch along with the [1], it seems working fine.
> 
> Or Can you please provide the steps to reproduce the issue?
> 

I tried to find easy way to reproduce (e.g., one target case), but
didn't find it - build was OK. But it fails in CI - most probably, due
to specific sequence of tests, that share sstate cache and download
directory between each other.

Similar behavior is reproduced in CI also with other patches, like
"deb-dl-dir: remove excessive calls to dpkg-deb in debsrc_download".
Building some target alone is OK, but under CI it fails.

> > Many thanks,
> > Srinu
> > 
> 
> --
> Best regards,
> Uladzimir.
Cedric Hombourger March 20, 2025, 6:02 p.m. UTC | #6 
On Thu, 2025-03-20 at 20:56 +0300, Uladzimir Bely wrote:
> On Thu, 2025-03-20 at 07:18 +0000, Arjunan, Srinu wrote:
> >
> >
> > -----Original Message-----
> > From: Uladzimir Bely <ubely@ilbers.de>
> > Sent: 20 March 2025 11:02
> > To: Arjunan, Srinu (FT FDS CES LX PBU 2)
> > <srinuvasan.a@siemens.com>;
> > isar-users@googlegroups.com
> > Cc: Kiszka, Jan (FT RPD CED) <jan.kiszka@siemens.com>; Hombourger,
> > Cedric (FT FDS CES LX) <cedric.hombourger@siemens.com>; Niedermayr,
> > Benedikt (FT RPD CED OES-DE) <benedikt.niedermayr@siemens.com>;
> > Adithya.Balakumar@toshiba-tsip.com
> > Subject: Re: [PATCH 1/1] image: move deb-src-caching functionality
> > after image creation
> >
> > On Tue, 2025-03-18 at 16:45 +0000, Arjunan, Srinu wrote:
> > >
> > >
> > > -----Original Message-----
> > > From: Arjunan, Srinu (FT FDS CES LX PBU 2)
> > > <srinuvasan.a@siemens.com>
> > > Sent: 10 March 2025 17:22
> > > To: isar-users@googlegroups.com
> > > Cc: Kiszka, Jan (FT RPD CED) <jan.kiszka@siemens.com>;
> > > Hombourger,
> > > Cedric (FT FDS CES LX) <cedric.hombourger@siemens.com>;
> > > ubely@ilbers.de; Niedermayr, Benedikt (FT RPD CED OES-DE)
> > > <benedikt.niedermayr@siemens.com>;
> > > Adithya.Balakumar@toshiba-tsip.com; Arjunan, Srinu (FT FDS CES LX
> > > PBU
> > > 2) <srinuvasan.a@siemens.com>
> > > Subject: [PATCH 1/1] image: move deb-src-caching functionality
> > > after
> > > image creation
> > >
> > > From: srinuvasan <srinuvasan.a@siemens.com>
> > >
> > > When we do deb-src caching, we are missing to cache imager
> > > related
> > > packages, because the deb-src-caching functionality runs in the
> > > stage
> > > of ROOTFS_POSTPROCESS_COMAMND, but image creation task runs after
> > > the
> > > rootfs creation, hence imager related packages are missing in
> > > rootfs's
> > > var/cache/apt/archives.
> > >
> > > To solve this issue run the deb-src-caching after image creation,
> > > now
> > > we can able to download deb-src for all the deb packages.
> > >
> > > Signed-off-by: srinuvasan <srinuvasan.a@siemens.com>
> > > ---
> > >  meta/classes/image.bbclass  | 8 +++++++-
> > > meta/classes/rootfs.bbclass
> > > > 3 ++-
> > >  2 files changed, 9 insertions(+), 2 deletions(-)
> > >
> > > diff --git a/meta/classes/image.bbclass
> > > b/meta/classes/image.bbclass
> > > index 56eca202..66b4d313 100644
> > > --- a/meta/classes/image.bbclass
> > > +++ b/meta/classes/image.bbclass
> > > @@ -72,7 +72,13 @@ ROOTFS_MANIFEST_DEPLOY_DIR ?=
> > > "${DEPLOY_DIR_IMAGE}"
> > >  ROOTFS_DPKGSTATUS_DEPLOY_DIR ?= "${DEPLOY_DIR_IMAGE}"
> > >  ROOTFS_PACKAGE_SUFFIX ?= "${PN}-${DISTRO}-${MACHINE}"
> > >
> > > -ROOTFS_POSTPROCESS_COMMAND:prepend =
> > > "${@bb.utils.contains('BASE_REPO_FEATURES', 'cache-deb-src',
> > > 'cache_deb_src', '', d)} "
> > > +CACHE_DEB_SRC = "${@bb.utils.contains('BASE_REPO_FEATURES',
> > > 'cache-
> > > deb-src', '1', '0', d)}"
> > > +
> > > +python () {
> > > +    if bb.utils.to_boolean(d.getVar('CACHE_DEB_SRC')):
> > > +        bb.build.addtask('do_cache_deb_src', 'do_deploy',
> > > 'do_image',
> > > +d) }
> > > +
> > >
> > >  inherit rootfs
> > >  inherit sdk
> > > diff --git a/meta/classes/rootfs.bbclass
> > > b/meta/classes/rootfs.bbclass
> > > index 2348e269..c9e832dc 100644
> > > --- a/meta/classes/rootfs.bbclass
> > > +++ b/meta/classes/rootfs.bbclass
> > > @@ -272,7 +272,8 @@ python do_rootfs_install() {  }  addtask
> > > rootfs_install before do_rootfs_postprocess after do_unpack
> > >
> > > -cache_deb_src() {
> > > +do_cache_deb_src[network] = "${TASK_USE_SUDO}"
> > > +do_cache_deb_src() {
> > >      if [ -e "${ROOTFSDIR}"/etc/resolv.conf ] ||
> > >         [ -h "${ROOTFSDIR}"/etc/resolv.conf ]; then
> > >          sudo mv "${ROOTFSDIR}"/etc/resolv.conf
> > > "${ROOTFSDIR}"/etc/resolv.conf.isar
> > > --
> > > 2.39.5
> > >
> > > Hi All,
> > >
> > >           Can you please update the status of this patch?, or
> > > anything
> > > missing from my end?
> > >
> >
> > Hello.
> >
> > Sorry for delays related to "installer" currently breaking full CI
> > on
> > "next" branch.
> >
> > Tested recently your patch, it also makes CI fail at:
> >
> > CrossTest.test_cross_debsrc:
> >
> > ```
> > [stdlog] 2025-03-19 21:29:50,542 avocado.app ERROR| ERROR: Task
> > (mc:qemuarm64-
> > bookworm:/workspace/build/isar_ub_devel_fast/214/meta-
> > test/recipes-core/images/isar-image-ci.bb:do_cache_deb_src) failed
> > with exit code '1'
> > [stdlog] 2025-03-19 21:30:44,921 avocado.test INFO | | DEBUG:
> > Executing shell function do_cache_deb_src [stdlog] 2025-03-19
> > 21:30:44,921 avocado.test INFO | | Reading package lists...
> > [stdlog] 2025-03-19 21:30:44,921 avocado.test INFO | | E: You must
> > put some 'deb-src' URIs in your sources.list ```
> >
> > NoCrossTest.test_nocross_debsrc:
> >
> > ```
> > [stdlog] 2025-03-20 01:09:35,759 avocado.app ERROR| ERROR: Logfile
> > of
> > failure stored in:
> > /workspace/build/isar_ub_devel/126/build/tmp/work/debian-bookworm-
> > armhf/isar-image-ci-qemuarm/1.0-r0/temp/log.do_cache_deb_src.218291
> > [stdlog] 2025-03-20 01:09:35,832 avocado.app ERROR| ERROR: Task
> > (mc:qemuarm-bookworm:/workspace/build/isar_ub_devel/126/meta-
> > test/recipes-core/images/isar-image-ci.bb:do_cache_deb_src) failed
> > with exit code '1'
> > [stdlog] 2025-03-20 01:19:35,822 avocado.test INFO | | DEBUG:
> > Executing shell function do_cache_deb_src [stdlog] 2025-03-20
> > 01:19:35,823 avocado.test INFO | | Reading package lists...
> > [stdlog] 2025-03-20 01:19:35,823 avocado.test INFO | | E: You must
> > put some 'deb-src' URIs in your sources.list ```
> >
> >
> > Hi Uladzimir,
> >
> >                       Thank you for the reply, I am not able to
> > reproduce the build issue what you reported , I triggered the build
> > for isar-image-ci target and it's getting passed.
> >
> > This patch depend on Cedric patch [1]
> > [1]:
> > https://groups.google.com/g/isar-users/c/OwrSBmCPe0Y
> >
> > I validated my patch along with the [1], it seems working fine.
> >
> > Or Can you please provide the steps to reproduce the issue?
> >
>
> I tried to find easy way to reproduce (e.g., one target case), but
> didn't find it - build was OK. But it fails in CI - most probably,
> due
> to specific sequence of tests, that share sstate cache and download
> directory between each other.
>
> Similar behavior is reproduced in CI also with other patches, like
> "deb-dl-dir: remove excessive calls to dpkg-deb in debsrc_download".
> Building some target alone is OK, but under CI it fails.

Ah damn it!

I had checked with ./scripts/ci_build.sh -T fast. I may have to try
with the full CI (without -T fast)

Is the error exactly the same when you tried with my patch (and only
that one)?

Would you be able to send a full CI log?

>
> > > Many thanks,
> > > Srinu
> > >
> >
> > --
> > Best regards,
> > Uladzimir.
>

--
Cedric Hombourger
Siemens AG
http://www.siemens.com/
Uladzimir Bely March 21, 2025, 5:32 a.m. UTC | #7 
On Thu, 2025-03-20 at 18:02 +0000, cedric.hombourger@siemens.com wrote:
> On Thu, 2025-03-20 at 20:56 +0300, Uladzimir Bely wrote:
> > On Thu, 2025-03-20 at 07:18 +0000, Arjunan, Srinu wrote:
> > > 
> > > 
> > > -----Original Message-----
> > > From: Uladzimir Bely <ubely@ilbers.de>
> > > Sent: 20 March 2025 11:02
> > > To: Arjunan, Srinu (FT FDS CES LX PBU 2)
> > > <srinuvasan.a@siemens.com>;
> > > isar-users@googlegroups.com
> > > Cc: Kiszka, Jan (FT RPD CED) <jan.kiszka@siemens.com>;
> > > Hombourger,
> > > Cedric (FT FDS CES LX) <cedric.hombourger@siemens.com>;
> > > Niedermayr,
> > > Benedikt (FT RPD CED OES-DE) <benedikt.niedermayr@siemens.com>;
> > > Adithya.Balakumar@toshiba-tsip.com
> > > Subject: Re: [PATCH 1/1] image: move deb-src-caching
> > > functionality
> > > after image creation
> > > 
> > > On Tue, 2025-03-18 at 16:45 +0000, Arjunan, Srinu wrote:
> > > > 
> > > > 
> > > > -----Original Message-----
> > > > From: Arjunan, Srinu (FT FDS CES LX PBU 2)
> > > > <srinuvasan.a@siemens.com>
> > > > Sent: 10 March 2025 17:22
> > > > To: isar-users@googlegroups.com
> > > > Cc: Kiszka, Jan (FT RPD CED) <jan.kiszka@siemens.com>;
> > > > Hombourger,
> > > > Cedric (FT FDS CES LX) <cedric.hombourger@siemens.com>;
> > > > ubely@ilbers.de; Niedermayr, Benedikt (FT RPD CED OES-DE)
> > > > <benedikt.niedermayr@siemens.com>;
> > > > Adithya.Balakumar@toshiba-tsip.com; Arjunan, Srinu (FT FDS CES
> > > > LX
> > > > PBU
> > > > 2) <srinuvasan.a@siemens.com>
> > > > Subject: [PATCH 1/1] image: move deb-src-caching functionality
> > > > after
> > > > image creation
> > > > 
> > > > From: srinuvasan <srinuvasan.a@siemens.com>
> > > > 
> > > > When we do deb-src caching, we are missing to cache imager
> > > > related
> > > > packages, because the deb-src-caching functionality runs in the
> > > > stage
> > > > of ROOTFS_POSTPROCESS_COMAMND, but image creation task runs
> > > > after
> > > > the
> > > > rootfs creation, hence imager related packages are missing in
> > > > rootfs's
> > > > var/cache/apt/archives.
> > > > 
> > > > To solve this issue run the deb-src-caching after image
> > > > creation,
> > > > now
> > > > we can able to download deb-src for all the deb packages.
> > > > 
> > > > Signed-off-by: srinuvasan <srinuvasan.a@siemens.com>
> > > > ---
> > > >  meta/classes/image.bbclass  | 8 +++++++-
> > > > meta/classes/rootfs.bbclass
> > > > > 3 ++-
> > > >  2 files changed, 9 insertions(+), 2 deletions(-)
> > > > 
> > > > diff --git a/meta/classes/image.bbclass
> > > > b/meta/classes/image.bbclass
> > > > index 56eca202..66b4d313 100644
> > > > --- a/meta/classes/image.bbclass
> > > > +++ b/meta/classes/image.bbclass
> > > > @@ -72,7 +72,13 @@ ROOTFS_MANIFEST_DEPLOY_DIR ?=
> > > > "${DEPLOY_DIR_IMAGE}"
> > > >  ROOTFS_DPKGSTATUS_DEPLOY_DIR ?= "${DEPLOY_DIR_IMAGE}"
> > > >  ROOTFS_PACKAGE_SUFFIX ?= "${PN}-${DISTRO}-${MACHINE}"
> > > > 
> > > > -ROOTFS_POSTPROCESS_COMMAND:prepend =
> > > > "${@bb.utils.contains('BASE_REPO_FEATURES', 'cache-deb-src',
> > > > 'cache_deb_src', '', d)} "
> > > > +CACHE_DEB_SRC = "${@bb.utils.contains('BASE_REPO_FEATURES',
> > > > 'cache-
> > > > deb-src', '1', '0', d)}"
> > > > +
> > > > +python () {
> > > > +    if bb.utils.to_boolean(d.getVar('CACHE_DEB_SRC')):
> > > > +        bb.build.addtask('do_cache_deb_src', 'do_deploy',
> > > > 'do_image',
> > > > +d) }
> > > > +
> > > > 
> > > >  inherit rootfs
> > > >  inherit sdk
> > > > diff --git a/meta/classes/rootfs.bbclass
> > > > b/meta/classes/rootfs.bbclass
> > > > index 2348e269..c9e832dc 100644
> > > > --- a/meta/classes/rootfs.bbclass
> > > > +++ b/meta/classes/rootfs.bbclass
> > > > @@ -272,7 +272,8 @@ python do_rootfs_install() {  }  addtask
> > > > rootfs_install before do_rootfs_postprocess after do_unpack
> > > > 
> > > > -cache_deb_src() {
> > > > +do_cache_deb_src[network] = "${TASK_USE_SUDO}"
> > > > +do_cache_deb_src() {
> > > >      if [ -e "${ROOTFSDIR}"/etc/resolv.conf ] ||
> > > >         [ -h "${ROOTFSDIR}"/etc/resolv.conf ]; then
> > > >          sudo mv "${ROOTFSDIR}"/etc/resolv.conf
> > > > "${ROOTFSDIR}"/etc/resolv.conf.isar
> > > > --
> > > > 2.39.5
> > > > 
> > > > Hi All,
> > > > 
> > > >           Can you please update the status of this patch?, or
> > > > anything
> > > > missing from my end?
> > > > 
> > > 
> > > Hello.
> > > 
> > > Sorry for delays related to "installer" currently breaking full
> > > CI
> > > on
> > > "next" branch.
> > > 
> > > Tested recently your patch, it also makes CI fail at:
> > > 
> > > CrossTest.test_cross_debsrc:
> > > 
> > > ```
> > > [stdlog] 2025-03-19 21:29:50,542 avocado.app ERROR| ERROR: Task
> > > (mc:qemuarm64-
> > > bookworm:/workspace/build/isar_ub_devel_fast/214/meta-
> > > test/recipes-core/images/isar-image-ci.bb:do_cache_deb_src)
> > > failed
> > > with exit code '1'
> > > [stdlog] 2025-03-19 21:30:44,921 avocado.test INFO | | DEBUG:
> > > Executing shell function do_cache_deb_src [stdlog] 2025-03-19
> > > 21:30:44,921 avocado.test INFO | | Reading package lists...
> > > [stdlog] 2025-03-19 21:30:44,921 avocado.test INFO | | E: You
> > > must
> > > put some 'deb-src' URIs in your sources.list ```
> > > 
> > > NoCrossTest.test_nocross_debsrc:
> > > 
> > > ```
> > > [stdlog] 2025-03-20 01:09:35,759 avocado.app ERROR| ERROR:
> > > Logfile
> > > of
> > > failure stored in:
> > > /workspace/build/isar_ub_devel/126/build/tmp/work/debian-
> > > bookworm-
> > > armhf/isar-image-ci-qemuarm/1.0-
> > > r0/temp/log.do_cache_deb_src.218291
> > > [stdlog] 2025-03-20 01:09:35,832 avocado.app ERROR| ERROR: Task
> > > (mc:qemuarm-bookworm:/workspace/build/isar_ub_devel/126/meta-
> > > test/recipes-core/images/isar-image-ci.bb:do_cache_deb_src)
> > > failed
> > > with exit code '1'
> > > [stdlog] 2025-03-20 01:19:35,822 avocado.test INFO | | DEBUG:
> > > Executing shell function do_cache_deb_src [stdlog] 2025-03-20
> > > 01:19:35,823 avocado.test INFO | | Reading package lists...
> > > [stdlog] 2025-03-20 01:19:35,823 avocado.test INFO | | E: You
> > > must
> > > put some 'deb-src' URIs in your sources.list ```
> > > 
> > > 
> > > Hi Uladzimir,
> > > 
> > >                       Thank you for the reply, I am not able to
> > > reproduce the build issue what you reported , I triggered the
> > > build
> > > for isar-image-ci target and it's getting passed.
> > > 
> > > This patch depend on Cedric patch [1]
> > > [1]:
> > > https://groups.google.com/g/isar-users/c/OwrSBmCPe0Y
> > > 
> > > I validated my patch along with the [1], it seems working fine.
> > > 
> > > Or Can you please provide the steps to reproduce the issue?
> > > 
> > 
> > I tried to find easy way to reproduce (e.g., one target case), but
> > didn't find it - build was OK. But it fails in CI - most probably,
> > due
> > to specific sequence of tests, that share sstate cache and download
> > directory between each other.
> > 
> > Similar behavior is reproduced in CI also with other patches, like
> > "deb-dl-dir: remove excessive calls to dpkg-deb in
> > debsrc_download".
> > Building some target alone is OK, but under CI it fails.
> 
> Ah damn it!
> 
> I had checked with ./scripts/ci_build.sh -T fast. I may have to try
> with the full CI (without -T fast)
> 
> Is the error exactly the same when you tried with my patch (and only
> that one)?

No, you patch I've mentioned failed in "cache-deb-src" task, but in
NoCrossTest.test_nocross_rpi_debsrc, and with a different error:

[stdlog] 2025-03-07 12:48:01,379 avocado.test INFO | | DEBUG: Executing
shell function cache_deb_src
[stdlog] 2025-03-07 12:48:01,379 avocado.test INFO | |
[stdlog] 2025-03-07 12:48:01,379 avocado.test INFO | | W: Unable to
read directory
/workspace/build/isar_ub_devel/119/build/tmp/work/raspios-bookworm-
armhf/isar-image-base-rpi-arm-v7l/1.0-
r0/rootfs/var/cache/apt/archives/partial
[stdlog] 2025-03-07 12:48:01,379 avocado.test INFO | | Reading package
lists...
[stdlog] 2025-03-07 12:48:01,379 avocado.test INFO | | E: Can not find
version '2023.3+deb12u1' of package 'debian-archive-keyring'
[stdlog] 2025-03-07 12:48:01,379 avocado.test INFO | | E: Unable to
find a source package for debian-archive-keyring


> 
> Would you be able to send a full CI log?
> 

Attached archive with both logs:

isar_ub_devel_119: full CI logs with "deb-dl-dir: remove
excessive calls to dpkg-deb in debsrc_download" applied.

isar_ub_devel_fast_214: fast CI logs with "image: move deb-src-
caching functionality after image creation" applied.


Some additional info.

For the latest patch, I tried to debug it:

- added `BASE_REPO_FEATURES = "cache-deb-src"` to local.conf (that is
the line CrossTest.test_cross_debsrc test adds; the line was removed in
the followed tests).
- chrooted to "isar-image-ci-qemuarm64" rootfs, where error happend.
- executed "apt-get -y --download-only --only-source source gcc-
12=12.2.0-14 download-src" - the same that failed in CI. 

Result:

```
# apt-get -y --download-only --only-source source gcc-12=12.2.0-14
download-src
Reading package lists... Done
E: You must put some 'deb-src' URIs in your sources.list
```

At the same time, sources.list looked fine:

```
# cat /etc/apt/sources.list.d/bootstrap.list 
deb     http://deb.debian.org/debian bookworm main contrib non-free-
firmware non-free
deb-src http://deb.debian.org/debian bookworm main contrib non-free-
firmware non-free

deb     http://deb.debian.org/debian-security bookworm-security main
contrib non-free-firmware non-free
deb-src http://deb.debian.org/debian-security bookworm-security main
contrib non-free-firmware non-free

deb     http://deb.debian.org/debian bookworm-updates main contrib non-
free-firmware non-free
deb-src http://deb.debian.org/debian bookworm-updates main contrib non-
free-firmware non-free
```

After executing "apt-get update" in this chroot, error gone.

It looks like despite proper sources list, internal apt cache was not
complete and didn't know anything about "deb-src" records. Why - I
still don't clearly understand.


> > 
> > > > Many thanks,
> > > > Srinu
> > > > 
> > > 
> > > --
> > > Best regards,
> > > Uladzimir.
> > 
> 
> --
> Cedric Hombourger
> Siemens AG
> http://www.siemens.com/
Uladzimir Bely March 21, 2025, 6:27 a.m. UTC | #8 
On Thu, 2025-03-20 at 07:18 +0000, 'Arjunan, Srinu' via isar-users
wrote:
> 
> 
> -----Original Message-----
> From: Uladzimir Bely <ubely@ilbers.de> 
> Sent: 20 March 2025 11:02
> To: Arjunan, Srinu (FT FDS CES LX PBU 2) <srinuvasan.a@siemens.com>;
> isar-users@googlegroups.com
> Cc: Kiszka, Jan (FT RPD CED) <jan.kiszka@siemens.com>; Hombourger,
> Cedric (FT FDS CES LX) <cedric.hombourger@siemens.com>; Niedermayr,
> Benedikt (FT RPD CED OES-DE) <benedikt.niedermayr@siemens.com>;
> Adithya.Balakumar@toshiba-tsip.com
> Subject: Re: [PATCH 1/1] image: move deb-src-caching functionality
> after image creation
> 
> On Tue, 2025-03-18 at 16:45 +0000, Arjunan, Srinu wrote:
> > 
> > 
> > -----Original Message-----
> > From: Arjunan, Srinu (FT FDS CES LX PBU 2)
> > <srinuvasan.a@siemens.com>
> > Sent: 10 March 2025 17:22
> > To: isar-users@googlegroups.com
> > Cc: Kiszka, Jan (FT RPD CED) <jan.kiszka@siemens.com>; Hombourger, 
> > Cedric (FT FDS CES LX) <cedric.hombourger@siemens.com>; 
> > ubely@ilbers.de; Niedermayr, Benedikt (FT RPD CED OES-DE) 
> > <benedikt.niedermayr@siemens.com>;
> > Adithya.Balakumar@toshiba-tsip.com; Arjunan, Srinu (FT FDS CES LX
> > PBU
> > 2) <srinuvasan.a@siemens.com>
> > Subject: [PATCH 1/1] image: move deb-src-caching functionality
> > after 
> > image creation
> > 
> > From: srinuvasan <srinuvasan.a@siemens.com>
> > 
> > When we do deb-src caching, we are missing to cache imager related 
> > packages, because the deb-src-caching functionality runs in the
> > stage 
> > of ROOTFS_POSTPROCESS_COMAMND, but image creation task runs after
> > the 
> > rootfs creation, hence imager related packages are missing in
> > rootfs's 
> > var/cache/apt/archives.
> > 
> > To solve this issue run the deb-src-caching after image creation,
> > now 
> > we can able to download deb-src for all the deb packages.
> > 
> > Signed-off-by: srinuvasan <srinuvasan.a@siemens.com>
> > ---
> >  meta/classes/image.bbclass  | 8 +++++++-
> > meta/classes/rootfs.bbclass 
> > > 3 ++-
> >  2 files changed, 9 insertions(+), 2 deletions(-)
> > 
> > diff --git a/meta/classes/image.bbclass
> > b/meta/classes/image.bbclass 
> > index 56eca202..66b4d313 100644
> > --- a/meta/classes/image.bbclass
> > +++ b/meta/classes/image.bbclass
> > @@ -72,7 +72,13 @@ ROOTFS_MANIFEST_DEPLOY_DIR ?=
> > "${DEPLOY_DIR_IMAGE}"
> >  ROOTFS_DPKGSTATUS_DEPLOY_DIR ?= "${DEPLOY_DIR_IMAGE}"
> >  ROOTFS_PACKAGE_SUFFIX ?= "${PN}-${DISTRO}-${MACHINE}"
> >  
> > -ROOTFS_POSTPROCESS_COMMAND:prepend =
> > "${@bb.utils.contains('BASE_REPO_FEATURES', 'cache-deb-src', 
> > 'cache_deb_src', '', d)} "
> > +CACHE_DEB_SRC = "${@bb.utils.contains('BASE_REPO_FEATURES',
> > 'cache-
> > deb-src', '1', '0', d)}"
> > +
> > +python () {
> > +    if bb.utils.to_boolean(d.getVar('CACHE_DEB_SRC')):
> > +        bb.build.addtask('do_cache_deb_src', 'do_deploy',
> > 'do_image',
> > +d) }
> > +
> >  
> >  inherit rootfs
> >  inherit sdk
> > diff --git a/meta/classes/rootfs.bbclass
> > b/meta/classes/rootfs.bbclass 
> > index 2348e269..c9e832dc 100644
> > --- a/meta/classes/rootfs.bbclass
> > +++ b/meta/classes/rootfs.bbclass
> > @@ -272,7 +272,8 @@ python do_rootfs_install() {  }  addtask 
> > rootfs_install before do_rootfs_postprocess after do_unpack
> >  
> > -cache_deb_src() {
> > +do_cache_deb_src[network] = "${TASK_USE_SUDO}"
> > +do_cache_deb_src() {
> >      if [ -e "${ROOTFSDIR}"/etc/resolv.conf ] ||
> >         [ -h "${ROOTFSDIR}"/etc/resolv.conf ]; then
> >          sudo mv "${ROOTFSDIR}"/etc/resolv.conf 
> > "${ROOTFSDIR}"/etc/resolv.conf.isar
> > --
> > 2.39.5
> > 
> > Hi All,
> > 
> >           Can you please update the status of this patch?, or
> > anything 
> > missing from my end?
> > 
> 
> Hello.
> 
> Sorry for delays related to "installer" currently breaking full CI on
> "next" branch.
> 
> Tested recently your patch, it also makes CI fail at:
> 
> CrossTest.test_cross_debsrc:
> 
> ```
> [stdlog] 2025-03-19 21:29:50,542 avocado.app ERROR| ERROR: Task
> (mc:qemuarm64-bookworm:/workspace/build/isar_ub_devel_fast/214/meta-
> test/recipes-core/images/isar-image-ci.bb:do_cache_deb_src) failed
> with exit code '1'
> [stdlog] 2025-03-19 21:30:44,921 avocado.test INFO | | DEBUG:
> Executing shell function do_cache_deb_src [stdlog] 2025-03-19
> 21:30:44,921 avocado.test INFO | | Reading package lists...
> [stdlog] 2025-03-19 21:30:44,921 avocado.test INFO | | E: You must
> put some 'deb-src' URIs in your sources.list ```
> 
> NoCrossTest.test_nocross_debsrc:
> 
> ```
> [stdlog] 2025-03-20 01:09:35,759 avocado.app ERROR| ERROR: Logfile of
> failure stored in:
> /workspace/build/isar_ub_devel/126/build/tmp/work/debian-bookworm-
> armhf/isar-image-ci-qemuarm/1.0-r0/temp/log.do_cache_deb_src.218291
> [stdlog] 2025-03-20 01:09:35,832 avocado.app ERROR| ERROR: Task
> (mc:qemuarm-bookworm:/workspace/build/isar_ub_devel/126/meta-
> test/recipes-core/images/isar-image-ci.bb:do_cache_deb_src) failed
> with exit code '1'
> [stdlog] 2025-03-20 01:19:35,822 avocado.test INFO | | DEBUG:
> Executing shell function do_cache_deb_src [stdlog] 2025-03-20
> 01:19:35,823 avocado.test INFO | | Reading package lists...
> [stdlog] 2025-03-20 01:19:35,823 avocado.test INFO | | E: You must
> put some 'deb-src' URIs in your sources.list ```
> 
> 
> Hi Uladzimir,
> 
>                       Thank you for the reply, I am not able to
> reproduce the build issue what you reported , I triggered the build
> for isar-image-ci target and it's getting passed.
> 
> This patch depend on Cedric patch [1]
> [1]: https://groups.google.com/g/isar-users/c/OwrSBmCPe0Y
> 
> I validated my patch along with the [1], it seems working fine.
> 
> Or Can you please provide the steps to reproduce the issue?
> 
> 
> > Many thanks,
> > Srinu
> > 
> 

I finally found root cause and easy steps to reproduce. The patch moves
debsrc caching to later stage and that results in incompatibility with
DISTRO_APT_PREMIRRORS option we are globally using in CI.

Steps to reproduce:

1. Run `./kas/kas-container menu`
2. Change the configuration:
 - `Mirror selection => Use specific Debian mirror`:
http://ftp.de.debian.org
 - `Cache all source packages`: Check the option
 - `Packages selection => Isar packages`: Uncheck everything to speedup
build (not necessary step)
3. Run build

It fails with:

```
ERROR: isar-image-base-1.0-r0 do_cache_deb_src:
ExecutionError('/build/tmp/work/debian-bookworm-amd64/isar-image-base-
qemuamd64/1.0-r0/temp/run.do_cache_deb_src.10979', 100, None, None)
ERROR: Logfile of failure stored in: /build/tmp/work/debian-bookworm-
amd64/isar-image-base-qemuamd64/1.0-r0/temp/log.do_cache_deb_src.10979
Log data follows:
| DEBUG: Executing shell function do_cache_deb_src
| Reading package lists...
| E: You must put some 'deb-src' URIs in your sources.list
ERROR: Task (/build/../repo/meta-isar/recipes-core/images/isar-image-
base.bb:do_cache_deb_src) failed with exit code '1'
```

> --
> Best regards,
> Uladzimir.
>
Cedric Hombourger March 21, 2025, 6:32 a.m. UTC | #9 
On Fri, 2025-03-21 at 09:27 +0300, Uladzimir Bely wrote:
> On Thu, 2025-03-20 at 07:18 +0000, 'Arjunan, Srinu' via isar-users
> wrote:
> >
> >
> > -----Original Message-----
> > From: Uladzimir Bely <ubely@ilbers.de>
> > Sent: 20 March 2025 11:02
> > To: Arjunan, Srinu (FT FDS CES LX PBU 2)
> > <srinuvasan.a@siemens.com>;
> > isar-users@googlegroups.com
> > Cc: Kiszka, Jan (FT RPD CED) <jan.kiszka@siemens.com>; Hombourger,
> > Cedric (FT FDS CES LX) <cedric.hombourger@siemens.com>; Niedermayr,
> > Benedikt (FT RPD CED OES-DE) <benedikt.niedermayr@siemens.com>;
> > Adithya.Balakumar@toshiba-tsip.com
> > Subject: Re: [PATCH 1/1] image: move deb-src-caching functionality
> > after image creation
> >
> > On Tue, 2025-03-18 at 16:45 +0000, Arjunan, Srinu wrote:
> > >
> > >
> > > -----Original Message-----
> > > From: Arjunan, Srinu (FT FDS CES LX PBU 2)
> > > <srinuvasan.a@siemens.com>
> > > Sent: 10 March 2025 17:22
> > > To: isar-users@googlegroups.com
> > > Cc: Kiszka, Jan (FT RPD CED) <jan.kiszka@siemens.com>;
> > > Hombourger,
> > > Cedric (FT FDS CES LX) <cedric.hombourger@siemens.com>;
> > > ubely@ilbers.de; Niedermayr, Benedikt (FT RPD CED OES-DE)
> > > <benedikt.niedermayr@siemens.com>;
> > > Adithya.Balakumar@toshiba-tsip.com; Arjunan, Srinu (FT FDS CES LX
> > > PBU
> > > 2) <srinuvasan.a@siemens.com>
> > > Subject: [PATCH 1/1] image: move deb-src-caching functionality
> > > after
> > > image creation
> > >
> > > From: srinuvasan <srinuvasan.a@siemens.com>
> > >
> > > When we do deb-src caching, we are missing to cache imager
> > > related
> > > packages, because the deb-src-caching functionality runs in the
> > > stage
> > > of ROOTFS_POSTPROCESS_COMAMND, but image creation task runs after
> > > the
> > > rootfs creation, hence imager related packages are missing in
> > > rootfs's
> > > var/cache/apt/archives.
> > >
> > > To solve this issue run the deb-src-caching after image creation,
> > > now
> > > we can able to download deb-src for all the deb packages.
> > >
> > > Signed-off-by: srinuvasan <srinuvasan.a@siemens.com>
> > > ---
> > >  meta/classes/image.bbclass  | 8 +++++++-
> > > meta/classes/rootfs.bbclass
> > > > 3 ++-
> > >  2 files changed, 9 insertions(+), 2 deletions(-)
> > >
> > > diff --git a/meta/classes/image.bbclass
> > > b/meta/classes/image.bbclass
> > > index 56eca202..66b4d313 100644
> > > --- a/meta/classes/image.bbclass
> > > +++ b/meta/classes/image.bbclass
> > > @@ -72,7 +72,13 @@ ROOTFS_MANIFEST_DEPLOY_DIR ?=
> > > "${DEPLOY_DIR_IMAGE}"
> > >  ROOTFS_DPKGSTATUS_DEPLOY_DIR ?= "${DEPLOY_DIR_IMAGE}"
> > >  ROOTFS_PACKAGE_SUFFIX ?= "${PN}-${DISTRO}-${MACHINE}"
> > >
> > > -ROOTFS_POSTPROCESS_COMMAND:prepend =
> > > "${@bb.utils.contains('BASE_REPO_FEATURES', 'cache-deb-src',
> > > 'cache_deb_src', '', d)} "
> > > +CACHE_DEB_SRC = "${@bb.utils.contains('BASE_REPO_FEATURES',
> > > 'cache-
> > > deb-src', '1', '0', d)}"
> > > +
> > > +python () {
> > > +    if bb.utils.to_boolean(d.getVar('CACHE_DEB_SRC')):
> > > +        bb.build.addtask('do_cache_deb_src', 'do_deploy',
> > > 'do_image',
> > > +d) }
> > > +
> > >
> > >  inherit rootfs
> > >  inherit sdk
> > > diff --git a/meta/classes/rootfs.bbclass
> > > b/meta/classes/rootfs.bbclass
> > > index 2348e269..c9e832dc 100644
> > > --- a/meta/classes/rootfs.bbclass
> > > +++ b/meta/classes/rootfs.bbclass
> > > @@ -272,7 +272,8 @@ python do_rootfs_install() {  }  addtask
> > > rootfs_install before do_rootfs_postprocess after do_unpack
> > >
> > > -cache_deb_src() {
> > > +do_cache_deb_src[network] = "${TASK_USE_SUDO}"
> > > +do_cache_deb_src() {
> > >      if [ -e "${ROOTFSDIR}"/etc/resolv.conf ] ||
> > >         [ -h "${ROOTFSDIR}"/etc/resolv.conf ]; then
> > >          sudo mv "${ROOTFSDIR}"/etc/resolv.conf
> > > "${ROOTFSDIR}"/etc/resolv.conf.isar
> > > --
> > > 2.39.5
> > >
> > > Hi All,
> > >
> > >           Can you please update the status of this patch?, or
> > > anything
> > > missing from my end?
> > >
> >
> > Hello.
> >
> > Sorry for delays related to "installer" currently breaking full CI
> > on
> > "next" branch.
> >
> > Tested recently your patch, it also makes CI fail at:
> >
> > CrossTest.test_cross_debsrc:
> >
> > ```
> > [stdlog] 2025-03-19 21:29:50,542 avocado.app ERROR| ERROR: Task
> > (mc:qemuarm64-
> > bookworm:/workspace/build/isar_ub_devel_fast/214/meta-
> > test/recipes-core/images/isar-image-ci.bb:do_cache_deb_src) failed
> > with exit code '1'
> > [stdlog] 2025-03-19 21:30:44,921 avocado.test INFO | | DEBUG:
> > Executing shell function do_cache_deb_src [stdlog] 2025-03-19
> > 21:30:44,921 avocado.test INFO | | Reading package lists...
> > [stdlog] 2025-03-19 21:30:44,921 avocado.test INFO | | E: You must
> > put some 'deb-src' URIs in your sources.list ```
> >
> > NoCrossTest.test_nocross_debsrc:
> >
> > ```
> > [stdlog] 2025-03-20 01:09:35,759 avocado.app ERROR| ERROR: Logfile
> > of
> > failure stored in:
> > /workspace/build/isar_ub_devel/126/build/tmp/work/debian-bookworm-
> > armhf/isar-image-ci-qemuarm/1.0-r0/temp/log.do_cache_deb_src.218291
> > [stdlog] 2025-03-20 01:09:35,832 avocado.app ERROR| ERROR: Task
> > (mc:qemuarm-bookworm:/workspace/build/isar_ub_devel/126/meta-
> > test/recipes-core/images/isar-image-ci.bb:do_cache_deb_src) failed
> > with exit code '1'
> > [stdlog] 2025-03-20 01:19:35,822 avocado.test INFO | | DEBUG:
> > Executing shell function do_cache_deb_src [stdlog] 2025-03-20
> > 01:19:35,823 avocado.test INFO | | Reading package lists...
> > [stdlog] 2025-03-20 01:19:35,823 avocado.test INFO | | E: You must
> > put some 'deb-src' URIs in your sources.list ```
> >
> >
> > Hi Uladzimir,
> >
> >                       Thank you for the reply, I am not able to
> > reproduce the build issue what you reported , I triggered the build
> > for isar-image-ci target and it's getting passed.
> >
> > This patch depend on Cedric patch [1]
> > [1]:
> > https://groups.google.com/g/isar-users/c/OwrSBmCPe0Y
> >
> > I validated my patch along with the [1], it seems working fine.
> >
> > Or Can you please provide the steps to reproduce the issue?
> >
> >
> > > Many thanks,
> > > Srinu
> > >
> >
>
> I finally found root cause and easy steps to reproduce. The patch
> moves
> debsrc caching to later stage and that results in incompatibility
> with
> DISTRO_APT_PREMIRRORS option we are globally using in CI.

Ah thanks. Are you seeing any issues with my patch alone? (was going to
re-run a local CI build to *double* check)

Would appreciate if my changes may be CI-checked alone and merged;
provided there are no issues ofcourse :)

>
> Steps to reproduce:
>
> 1. Run `./kas/kas-container menu`
> 2. Change the configuration:
>  - `Mirror selection => Use specific Debian mirror`:
> http://ftp.de.debian.org/
>  - `Cache all source packages`: Check the option
>  - `Packages selection => Isar packages`: Uncheck everything to
> speedup
> build (not necessary step)
> 3. Run build
>
> It fails with:
>
> ```
> ERROR: isar-image-base-1.0-r0 do_cache_deb_src:
> ExecutionError('/build/tmp/work/debian-bookworm-amd64/isar-image-
> base-
> qemuamd64/1.0-r0/temp/run.do_cache_deb_src.10979', 100, None, None)
> ERROR: Logfile of failure stored in: /build/tmp/work/debian-bookworm-
> amd64/isar-image-base-qemuamd64/1.0-
> r0/temp/log.do_cache_deb_src.10979
> Log data follows:
> > DEBUG: Executing shell function do_cache_deb_src
> > Reading package lists...
> > E: You must put some 'deb-src' URIs in your sources.list
> ERROR: Task (/build/../repo/meta-isar/recipes-core/images/isar-image-
> base.bb:do_cache_deb_src) failed with exit code '1'
> ```
>
> > --
> > Best regards,
> > Uladzimir.
> >
>

--
Cedric Hombourger
Siemens AG
http://www.siemens.com/
Uladzimir Bely March 21, 2025, 7:06 a.m. UTC | #10 
On Fri, 2025-03-21 at 06:32 +0000, cedric.hombourger@siemens.com wrote:
> On Fri, 2025-03-21 at 09:27 +0300, Uladzimir Bely wrote:
> > On Thu, 2025-03-20 at 07:18 +0000, 'Arjunan, Srinu' via isar-users
> > wrote:
> > > 
> > > 
> > > -----Original Message-----
> > > From: Uladzimir Bely <ubely@ilbers.de>
> > > Sent: 20 March 2025 11:02
> > > To: Arjunan, Srinu (FT FDS CES LX PBU 2)
> > > <srinuvasan.a@siemens.com>;
> > > isar-users@googlegroups.com
> > > Cc: Kiszka, Jan (FT RPD CED) <jan.kiszka@siemens.com>;
> > > Hombourger,
> > > Cedric (FT FDS CES LX) <cedric.hombourger@siemens.com>;
> > > Niedermayr,
> > > Benedikt (FT RPD CED OES-DE) <benedikt.niedermayr@siemens.com>;
> > > Adithya.Balakumar@toshiba-tsip.com
> > > Subject: Re: [PATCH 1/1] image: move deb-src-caching
> > > functionality
> > > after image creation
> > > 
> > > On Tue, 2025-03-18 at 16:45 +0000, Arjunan, Srinu wrote:
> > > > 
> > > > 
> > > > -----Original Message-----
> > > > From: Arjunan, Srinu (FT FDS CES LX PBU 2)
> > > > <srinuvasan.a@siemens.com>
> > > > Sent: 10 March 2025 17:22
> > > > To: isar-users@googlegroups.com
> > > > Cc: Kiszka, Jan (FT RPD CED) <jan.kiszka@siemens.com>;
> > > > Hombourger,
> > > > Cedric (FT FDS CES LX) <cedric.hombourger@siemens.com>;
> > > > ubely@ilbers.de; Niedermayr, Benedikt (FT RPD CED OES-DE)
> > > > <benedikt.niedermayr@siemens.com>;
> > > > Adithya.Balakumar@toshiba-tsip.com; Arjunan, Srinu (FT FDS CES
> > > > LX
> > > > PBU
> > > > 2) <srinuvasan.a@siemens.com>
> > > > Subject: [PATCH 1/1] image: move deb-src-caching functionality
> > > > after
> > > > image creation
> > > > 
> > > > From: srinuvasan <srinuvasan.a@siemens.com>
> > > > 
> > > > When we do deb-src caching, we are missing to cache imager
> > > > related
> > > > packages, because the deb-src-caching functionality runs in the
> > > > stage
> > > > of ROOTFS_POSTPROCESS_COMAMND, but image creation task runs
> > > > after
> > > > the
> > > > rootfs creation, hence imager related packages are missing in
> > > > rootfs's
> > > > var/cache/apt/archives.
> > > > 
> > > > To solve this issue run the deb-src-caching after image
> > > > creation,
> > > > now
> > > > we can able to download deb-src for all the deb packages.
> > > > 
> > > > Signed-off-by: srinuvasan <srinuvasan.a@siemens.com>
> > > > ---
> > > >  meta/classes/image.bbclass  | 8 +++++++-
> > > > meta/classes/rootfs.bbclass
> > > > > 3 ++-
> > > >  2 files changed, 9 insertions(+), 2 deletions(-)
> > > > 
> > > > diff --git a/meta/classes/image.bbclass
> > > > b/meta/classes/image.bbclass
> > > > index 56eca202..66b4d313 100644
> > > > --- a/meta/classes/image.bbclass
> > > > +++ b/meta/classes/image.bbclass
> > > > @@ -72,7 +72,13 @@ ROOTFS_MANIFEST_DEPLOY_DIR ?=
> > > > "${DEPLOY_DIR_IMAGE}"
> > > >  ROOTFS_DPKGSTATUS_DEPLOY_DIR ?= "${DEPLOY_DIR_IMAGE}"
> > > >  ROOTFS_PACKAGE_SUFFIX ?= "${PN}-${DISTRO}-${MACHINE}"
> > > > 
> > > > -ROOTFS_POSTPROCESS_COMMAND:prepend =
> > > > "${@bb.utils.contains('BASE_REPO_FEATURES', 'cache-deb-src',
> > > > 'cache_deb_src', '', d)} "
> > > > +CACHE_DEB_SRC = "${@bb.utils.contains('BASE_REPO_FEATURES',
> > > > 'cache-
> > > > deb-src', '1', '0', d)}"
> > > > +
> > > > +python () {
> > > > +    if bb.utils.to_boolean(d.getVar('CACHE_DEB_SRC')):
> > > > +        bb.build.addtask('do_cache_deb_src', 'do_deploy',
> > > > 'do_image',
> > > > +d) }
> > > > +
> > > > 
> > > >  inherit rootfs
> > > >  inherit sdk
> > > > diff --git a/meta/classes/rootfs.bbclass
> > > > b/meta/classes/rootfs.bbclass
> > > > index 2348e269..c9e832dc 100644
> > > > --- a/meta/classes/rootfs.bbclass
> > > > +++ b/meta/classes/rootfs.bbclass
> > > > @@ -272,7 +272,8 @@ python do_rootfs_install() {  }  addtask
> > > > rootfs_install before do_rootfs_postprocess after do_unpack
> > > > 
> > > > -cache_deb_src() {
> > > > +do_cache_deb_src[network] = "${TASK_USE_SUDO}"
> > > > +do_cache_deb_src() {
> > > >      if [ -e "${ROOTFSDIR}"/etc/resolv.conf ] ||
> > > >         [ -h "${ROOTFSDIR}"/etc/resolv.conf ]; then
> > > >          sudo mv "${ROOTFSDIR}"/etc/resolv.conf
> > > > "${ROOTFSDIR}"/etc/resolv.conf.isar
> > > > --
> > > > 2.39.5
> > > > 
> > > > Hi All,
> > > > 
> > > >           Can you please update the status of this patch?, or
> > > > anything
> > > > missing from my end?
> > > > 
> > > 
> > > Hello.
> > > 
> > > Sorry for delays related to "installer" currently breaking full
> > > CI
> > > on
> > > "next" branch.
> > > 
> > > Tested recently your patch, it also makes CI fail at:
> > > 
> > > CrossTest.test_cross_debsrc:
> > > 
> > > ```
> > > [stdlog] 2025-03-19 21:29:50,542 avocado.app ERROR| ERROR: Task
> > > (mc:qemuarm64-
> > > bookworm:/workspace/build/isar_ub_devel_fast/214/meta-
> > > test/recipes-core/images/isar-image-ci.bb:do_cache_deb_src)
> > > failed
> > > with exit code '1'
> > > [stdlog] 2025-03-19 21:30:44,921 avocado.test INFO | | DEBUG:
> > > Executing shell function do_cache_deb_src [stdlog] 2025-03-19
> > > 21:30:44,921 avocado.test INFO | | Reading package lists...
> > > [stdlog] 2025-03-19 21:30:44,921 avocado.test INFO | | E: You
> > > must
> > > put some 'deb-src' URIs in your sources.list ```
> > > 
> > > NoCrossTest.test_nocross_debsrc:
> > > 
> > > ```
> > > [stdlog] 2025-03-20 01:09:35,759 avocado.app ERROR| ERROR:
> > > Logfile
> > > of
> > > failure stored in:
> > > /workspace/build/isar_ub_devel/126/build/tmp/work/debian-
> > > bookworm-
> > > armhf/isar-image-ci-qemuarm/1.0-
> > > r0/temp/log.do_cache_deb_src.218291
> > > [stdlog] 2025-03-20 01:09:35,832 avocado.app ERROR| ERROR: Task
> > > (mc:qemuarm-bookworm:/workspace/build/isar_ub_devel/126/meta-
> > > test/recipes-core/images/isar-image-ci.bb:do_cache_deb_src)
> > > failed
> > > with exit code '1'
> > > [stdlog] 2025-03-20 01:19:35,822 avocado.test INFO | | DEBUG:
> > > Executing shell function do_cache_deb_src [stdlog] 2025-03-20
> > > 01:19:35,823 avocado.test INFO | | Reading package lists...
> > > [stdlog] 2025-03-20 01:19:35,823 avocado.test INFO | | E: You
> > > must
> > > put some 'deb-src' URIs in your sources.list ```
> > > 
> > > 
> > > Hi Uladzimir,
> > > 
> > >                       Thank you for the reply, I am not able to
> > > reproduce the build issue what you reported , I triggered the
> > > build
> > > for isar-image-ci target and it's getting passed.
> > > 
> > > This patch depend on Cedric patch [1]
> > > [1]:
> > > https://groups.google.com/g/isar-users/c/OwrSBmCPe0Y
> > > 
> > > I validated my patch along with the [1], it seems working fine.
> > > 
> > > Or Can you please provide the steps to reproduce the issue?
> > > 
> > > 
> > > > Many thanks,
> > > > Srinu
> > > > 
> > > 
> > 
> > I finally found root cause and easy steps to reproduce. The patch
> > moves
> > debsrc caching to later stage and that results in incompatibility
> > with
> > DISTRO_APT_PREMIRRORS option we are globally using in CI.
> 
> Ah thanks. Are you seeing any issues with my patch alone? (was going
> to
> re-run a local CI build to *double* check)
> 
> Would appreciate if my changes may be CI-checked alone and merged;
> provided there are no issues ofcourse :)
> 

Sure. I've re-run CI with your patch alone on top of 'next'.

Probably, the issue I observed few days ago was caused by (temporary)
non-synchronized debian and raspios repos and it won't be reproduced
again.

> > 
> > Steps to reproduce:
> > 
> > 1. Run `./kas/kas-container menu`
> > 2. Change the configuration:
> >  - `Mirror selection => Use specific Debian mirror`:
> > http://ftp.de.debian.org/
> >  - `Cache all source packages`: Check the option
> >  - `Packages selection => Isar packages`: Uncheck everything to
> > speedup
> > build (not necessary step)
> > 3. Run build
> > 
> > It fails with:
> > 
> > ```
> > ERROR: isar-image-base-1.0-r0 do_cache_deb_src:
> > ExecutionError('/build/tmp/work/debian-bookworm-amd64/isar-image-
> > base-
> > qemuamd64/1.0-r0/temp/run.do_cache_deb_src.10979', 100, None, None)
> > ERROR: Logfile of failure stored in: /build/tmp/work/debian-
> > bookworm-
> > amd64/isar-image-base-qemuamd64/1.0-
> > r0/temp/log.do_cache_deb_src.10979
> > Log data follows:
> > > DEBUG: Executing shell function do_cache_deb_src
> > > Reading package lists...
> > > E: You must put some 'deb-src' URIs in your sources.list
> > ERROR: Task (/build/../repo/meta-isar/recipes-core/images/isar-
> > image-
> > base.bb:do_cache_deb_src) failed with exit code '1'
> > ```
> > 
> > > --
> > > Best regards,
> > > Uladzimir.
> > > 
> > 
> 
> --
> Cedric Hombourger
> Siemens AG
> http://www.siemens.com/
Uladzimir Bely March 21, 2025, 8:39 a.m. UTC | #11 
On Fri, 2025-03-21 at 06:32 +0000, cedric.hombourger@siemens.com wrote:
> On Fri, 2025-03-21 at 09:27 +0300, Uladzimir Bely wrote:
> > On Thu, 2025-03-20 at 07:18 +0000, 'Arjunan, Srinu' via isar-users
> > wrote:
> > > 
> > > 
> > > -----Original Message-----
> > > From: Uladzimir Bely <ubely@ilbers.de>
> > > Sent: 20 March 2025 11:02
> > > To: Arjunan, Srinu (FT FDS CES LX PBU 2)
> > > <srinuvasan.a@siemens.com>;
> > > isar-users@googlegroups.com
> > > Cc: Kiszka, Jan (FT RPD CED) <jan.kiszka@siemens.com>;
> > > Hombourger,
> > > Cedric (FT FDS CES LX) <cedric.hombourger@siemens.com>;
> > > Niedermayr,
> > > Benedikt (FT RPD CED OES-DE) <benedikt.niedermayr@siemens.com>;
> > > Adithya.Balakumar@toshiba-tsip.com
> > > Subject: Re: [PATCH 1/1] image: move deb-src-caching
> > > functionality
> > > after image creation
> > > 
> > > On Tue, 2025-03-18 at 16:45 +0000, Arjunan, Srinu wrote:
> > > > 
> > > > 
> > > > -----Original Message-----
> > > > From: Arjunan, Srinu (FT FDS CES LX PBU 2)
> > > > <srinuvasan.a@siemens.com>
> > > > Sent: 10 March 2025 17:22
> > > > To: isar-users@googlegroups.com
> > > > Cc: Kiszka, Jan (FT RPD CED) <jan.kiszka@siemens.com>;
> > > > Hombourger,
> > > > Cedric (FT FDS CES LX) <cedric.hombourger@siemens.com>;
> > > > ubely@ilbers.de; Niedermayr, Benedikt (FT RPD CED OES-DE)
> > > > <benedikt.niedermayr@siemens.com>;
> > > > Adithya.Balakumar@toshiba-tsip.com; Arjunan, Srinu (FT FDS CES
> > > > LX
> > > > PBU
> > > > 2) <srinuvasan.a@siemens.com>
> > > > Subject: [PATCH 1/1] image: move deb-src-caching functionality
> > > > after
> > > > image creation
> > > > 
> > > > From: srinuvasan <srinuvasan.a@siemens.com>
> > > > 
> > > > When we do deb-src caching, we are missing to cache imager
> > > > related
> > > > packages, because the deb-src-caching functionality runs in the
> > > > stage
> > > > of ROOTFS_POSTPROCESS_COMAMND, but image creation task runs
> > > > after
> > > > the
> > > > rootfs creation, hence imager related packages are missing in
> > > > rootfs's
> > > > var/cache/apt/archives.
> > > > 
> > > > To solve this issue run the deb-src-caching after image
> > > > creation,
> > > > now
> > > > we can able to download deb-src for all the deb packages.
> > > > 
> > > > Signed-off-by: srinuvasan <srinuvasan.a@siemens.com>
> > > > ---
> > > >  meta/classes/image.bbclass  | 8 +++++++-
> > > > meta/classes/rootfs.bbclass
> > > > > 3 ++-
> > > >  2 files changed, 9 insertions(+), 2 deletions(-)
> > > > 
> > > > diff --git a/meta/classes/image.bbclass
> > > > b/meta/classes/image.bbclass
> > > > index 56eca202..66b4d313 100644
> > > > --- a/meta/classes/image.bbclass
> > > > +++ b/meta/classes/image.bbclass
> > > > @@ -72,7 +72,13 @@ ROOTFS_MANIFEST_DEPLOY_DIR ?=
> > > > "${DEPLOY_DIR_IMAGE}"
> > > >  ROOTFS_DPKGSTATUS_DEPLOY_DIR ?= "${DEPLOY_DIR_IMAGE}"
> > > >  ROOTFS_PACKAGE_SUFFIX ?= "${PN}-${DISTRO}-${MACHINE}"
> > > > 
> > > > -ROOTFS_POSTPROCESS_COMMAND:prepend =
> > > > "${@bb.utils.contains('BASE_REPO_FEATURES', 'cache-deb-src',
> > > > 'cache_deb_src', '', d)} "
> > > > +CACHE_DEB_SRC = "${@bb.utils.contains('BASE_REPO_FEATURES',
> > > > 'cache-
> > > > deb-src', '1', '0', d)}"
> > > > +
> > > > +python () {
> > > > +    if bb.utils.to_boolean(d.getVar('CACHE_DEB_SRC')):
> > > > +        bb.build.addtask('do_cache_deb_src', 'do_deploy',
> > > > 'do_image',
> > > > +d) }
> > > > +
> > > > 
> > > >  inherit rootfs
> > > >  inherit sdk
> > > > diff --git a/meta/classes/rootfs.bbclass
> > > > b/meta/classes/rootfs.bbclass
> > > > index 2348e269..c9e832dc 100644
> > > > --- a/meta/classes/rootfs.bbclass
> > > > +++ b/meta/classes/rootfs.bbclass
> > > > @@ -272,7 +272,8 @@ python do_rootfs_install() {  }  addtask
> > > > rootfs_install before do_rootfs_postprocess after do_unpack
> > > > 
> > > > -cache_deb_src() {
> > > > +do_cache_deb_src[network] = "${TASK_USE_SUDO}"
> > > > +do_cache_deb_src() {
> > > >      if [ -e "${ROOTFSDIR}"/etc/resolv.conf ] ||
> > > >         [ -h "${ROOTFSDIR}"/etc/resolv.conf ]; then
> > > >          sudo mv "${ROOTFSDIR}"/etc/resolv.conf
> > > > "${ROOTFSDIR}"/etc/resolv.conf.isar
> > > > --
> > > > 2.39.5
> > > > 
> > > > Hi All,
> > > > 
> > > >           Can you please update the status of this patch?, or
> > > > anything
> > > > missing from my end?
> > > > 
> > > 
> > > Hello.
> > > 
> > > Sorry for delays related to "installer" currently breaking full
> > > CI
> > > on
> > > "next" branch.
> > > 
> > > Tested recently your patch, it also makes CI fail at:
> > > 
> > > CrossTest.test_cross_debsrc:
> > > 
> > > ```
> > > [stdlog] 2025-03-19 21:29:50,542 avocado.app ERROR| ERROR: Task
> > > (mc:qemuarm64-
> > > bookworm:/workspace/build/isar_ub_devel_fast/214/meta-
> > > test/recipes-core/images/isar-image-ci.bb:do_cache_deb_src)
> > > failed
> > > with exit code '1'
> > > [stdlog] 2025-03-19 21:30:44,921 avocado.test INFO | | DEBUG:
> > > Executing shell function do_cache_deb_src [stdlog] 2025-03-19
> > > 21:30:44,921 avocado.test INFO | | Reading package lists...
> > > [stdlog] 2025-03-19 21:30:44,921 avocado.test INFO | | E: You
> > > must
> > > put some 'deb-src' URIs in your sources.list ```
> > > 
> > > NoCrossTest.test_nocross_debsrc:
> > > 
> > > ```
> > > [stdlog] 2025-03-20 01:09:35,759 avocado.app ERROR| ERROR:
> > > Logfile
> > > of
> > > failure stored in:
> > > /workspace/build/isar_ub_devel/126/build/tmp/work/debian-
> > > bookworm-
> > > armhf/isar-image-ci-qemuarm/1.0-
> > > r0/temp/log.do_cache_deb_src.218291
> > > [stdlog] 2025-03-20 01:09:35,832 avocado.app ERROR| ERROR: Task
> > > (mc:qemuarm-bookworm:/workspace/build/isar_ub_devel/126/meta-
> > > test/recipes-core/images/isar-image-ci.bb:do_cache_deb_src)
> > > failed
> > > with exit code '1'
> > > [stdlog] 2025-03-20 01:19:35,822 avocado.test INFO | | DEBUG:
> > > Executing shell function do_cache_deb_src [stdlog] 2025-03-20
> > > 01:19:35,823 avocado.test INFO | | Reading package lists...
> > > [stdlog] 2025-03-20 01:19:35,823 avocado.test INFO | | E: You
> > > must
> > > put some 'deb-src' URIs in your sources.list ```
> > > 
> > > 
> > > Hi Uladzimir,
> > > 
> > >                       Thank you for the reply, I am not able to
> > > reproduce the build issue what you reported , I triggered the
> > > build
> > > for isar-image-ci target and it's getting passed.
> > > 
> > > This patch depend on Cedric patch [1]
> > > [1]:
> > > https://groups.google.com/g/isar-users/c/OwrSBmCPe0Y
> > > 
> > > I validated my patch along with the [1], it seems working fine.
> > > 
> > > Or Can you please provide the steps to reproduce the issue?
> > > 
> > > 
> > > > Many thanks,
> > > > Srinu
> > > > 
> > > 
> > 
> > I finally found root cause and easy steps to reproduce. The patch
> > moves
> > debsrc caching to later stage and that results in incompatibility
> > with
> > DISTRO_APT_PREMIRRORS option we are globally using in CI.
> 
> Ah thanks. Are you seeing any issues with my patch alone? (was going
> to
> re-run a local CI build to *double* check)
> 
> Would appreciate if my changes may be CI-checked alone and merged;
> provided there are no issues ofcourse :)
> 

Actually, found easy way to reproduce the issue with "deb-dl-dir:
remove excessive calls to dpkg-deb in debsrc_download" on top of
'next'.

Steps to reproduce with kas-container:

1. Place the deb we fail to download sources for to downloads
directory. 
```
mkdir -p ./build/downloads/deb/raspios-bookworm
wget
https://deb.debian.org/debian/pool/main/d/debian-archive-keyring/debian-archive-keyring_2023.3+deb12u1_all.deb
-P build/downloads/deb/raspios-bookworm/
```

2. Configure build:

./kas/kas-container menu
 - `Machine selection`: rpi-arm-v7l
 - `Distro selection`: Raspberry Pi OS Debian 12 (bookworm)
 - `Cache all source packages`: check the option

Run build

3. Result:

```
| E: Can not find version '2023.3+deb12u1' of package 'debian-archive-
keyring'
| E: Unable to find a source package for debian-archive-keyring
| DEBUG: Executing shell function rootfs_do_umounts
| DEBUG: Shell function rootfs_do_umounts finished
| DEBUG: Python function do_rootfs_postprocess finished
ERROR: Task (/build/../repo/meta-isar/recipes-core/images/isar-image-
base.bb:do_rootfs_postprocess) failed with exit code '1'
```

The issue is not reproducable on 'next' (since we are parsing dpkg.log
and it has no entry for debian-archive-keyring, since it is not a part
of current build.

> > 
> > Steps to reproduce:
> > 
> > 1. Run `./kas/kas-container menu`
> > 2. Change the configuration:
> >  - `Mirror selection => Use specific Debian mirror`:
> > http://ftp.de.debian.org/
> >  - `Cache all source packages`: Check the option
> >  - `Packages selection => Isar packages`: Uncheck everything to
> > speedup
> > build (not necessary step)
> > 3. Run build
> > 
> > It fails with:
> > 
> > ```
> > ERROR: isar-image-base-1.0-r0 do_cache_deb_src:
> > ExecutionError('/build/tmp/work/debian-bookworm-amd64/isar-image-
> > base-
> > qemuamd64/1.0-r0/temp/run.do_cache_deb_src.10979', 100, None, None)
> > ERROR: Logfile of failure stored in: /build/tmp/work/debian-
> > bookworm-
> > amd64/isar-image-base-qemuamd64/1.0-
> > r0/temp/log.do_cache_deb_src.10979
> > Log data follows:
> > > DEBUG: Executing shell function do_cache_deb_src
> > > Reading package lists...
> > > E: You must put some 'deb-src' URIs in your sources.list
> > ERROR: Task (/build/../repo/meta-isar/recipes-core/images/isar-
> > image-
> > base.bb:do_cache_deb_src) failed with exit code '1'
> > ```
> > 
> > > --
> > > Best regards,
> > > Uladzimir.
> > > 
> > 
> 
> --
> Cedric Hombourger
> Siemens AG
> http://www.siemens.com/
Cedric Hombourger March 21, 2025, 1:50 p.m. UTC | #12 
On Fri, 2025-03-21 at 11:39 +0300, Uladzimir Bely wrote:
> On Fri, 2025-03-21 at 06:32 +0000,
> cedric.hombourger@siemens.com wrote:
> > On Fri, 2025-03-21 at 09:27 +0300, Uladzimir Bely wrote:
> > > On Thu, 2025-03-20 at 07:18 +0000, 'Arjunan, Srinu' via isar-
> > > users
> > > wrote:
> > > >
> > > >
> > > > -----Original Message-----
> > > > From: Uladzimir Bely <ubely@ilbers.de>
> > > > Sent: 20 March 2025 11:02
> > > > To: Arjunan, Srinu (FT FDS CES LX PBU 2)
> > > > <srinuvasan.a@siemens.com>;
> > > > isar-users@googlegroups.com
> > > > Cc: Kiszka, Jan (FT RPD CED) <jan.kiszka@siemens.com>;
> > > > Hombourger,
> > > > Cedric (FT FDS CES LX) <cedric.hombourger@siemens.com>;
> > > > Niedermayr,
> > > > Benedikt (FT RPD CED OES-DE) <benedikt.niedermayr@siemens.com>;
> > > > Adithya.Balakumar@toshiba-tsip.com
> > > > Subject: Re: [PATCH 1/1] image: move deb-src-caching
> > > > functionality
> > > > after image creation
> > > >
> > > > On Tue, 2025-03-18 at 16:45 +0000, Arjunan, Srinu wrote:
> > > > >
> > > > >
> > > > > -----Original Message-----
> > > > > From: Arjunan, Srinu (FT FDS CES LX PBU 2)
> > > > > <srinuvasan.a@siemens.com>
> > > > > Sent: 10 March 2025 17:22
> > > > > To: isar-users@googlegroups.com
> > > > > Cc: Kiszka, Jan (FT RPD CED) <jan.kiszka@siemens.com>;
> > > > > Hombourger,
> > > > > Cedric (FT FDS CES LX) <cedric.hombourger@siemens.com>;
> > > > > ubely@ilbers.de; Niedermayr, Benedikt (FT RPD CED OES-DE)
> > > > > <benedikt.niedermayr@siemens.com>;
> > > > > Adithya.Balakumar@toshiba-tsip.com; Arjunan, Srinu (FT FDS
> > > > > CES
> > > > > LX
> > > > > PBU
> > > > > 2) <srinuvasan.a@siemens.com>
> > > > > Subject: [PATCH 1/1] image: move deb-src-caching
> > > > > functionality
> > > > > after
> > > > > image creation
> > > > >
> > > > > From: srinuvasan <srinuvasan.a@siemens.com>
> > > > >
> > > > > When we do deb-src caching, we are missing to cache imager
> > > > > related
> > > > > packages, because the deb-src-caching functionality runs in
> > > > > the
> > > > > stage
> > > > > of ROOTFS_POSTPROCESS_COMAMND, but image creation task runs
> > > > > after
> > > > > the
> > > > > rootfs creation, hence imager related packages are missing in
> > > > > rootfs's
> > > > > var/cache/apt/archives.
> > > > >
> > > > > To solve this issue run the deb-src-caching after image
> > > > > creation,
> > > > > now
> > > > > we can able to download deb-src for all the deb packages.
> > > > >
> > > > > Signed-off-by: srinuvasan <srinuvasan.a@siemens.com>
> > > > > ---
> > > > >  meta/classes/image.bbclass  | 8 +++++++-
> > > > > meta/classes/rootfs.bbclass
> > > > > > 3 ++-
> > > > >  2 files changed, 9 insertions(+), 2 deletions(-)
> > > > >
> > > > > diff --git a/meta/classes/image.bbclass
> > > > > b/meta/classes/image.bbclass
> > > > > index 56eca202..66b4d313 100644
> > > > > --- a/meta/classes/image.bbclass
> > > > > +++ b/meta/classes/image.bbclass
> > > > > @@ -72,7 +72,13 @@ ROOTFS_MANIFEST_DEPLOY_DIR ?=
> > > > > "${DEPLOY_DIR_IMAGE}"
> > > > >  ROOTFS_DPKGSTATUS_DEPLOY_DIR ?= "${DEPLOY_DIR_IMAGE}"
> > > > >  ROOTFS_PACKAGE_SUFFIX ?= "${PN}-${DISTRO}-${MACHINE}"
> > > > >
> > > > > -ROOTFS_POSTPROCESS_COMMAND:prepend =
> > > > > "${@bb.utils.contains('BASE_REPO_FEATURES', 'cache-deb-src',
> > > > > 'cache_deb_src', '', d)} "
> > > > > +CACHE_DEB_SRC = "${@bb.utils.contains('BASE_REPO_FEATURES',
> > > > > 'cache-
> > > > > deb-src', '1', '0', d)}"
> > > > > +
> > > > > +python () {
> > > > > +    if bb.utils.to_boolean(d.getVar('CACHE_DEB_SRC')):
> > > > > +        bb.build.addtask('do_cache_deb_src', 'do_deploy',
> > > > > 'do_image',
> > > > > +d) }
> > > > > +
> > > > >
> > > > >  inherit rootfs
> > > > >  inherit sdk
> > > > > diff --git a/meta/classes/rootfs.bbclass
> > > > > b/meta/classes/rootfs.bbclass
> > > > > index 2348e269..c9e832dc 100644
> > > > > --- a/meta/classes/rootfs.bbclass
> > > > > +++ b/meta/classes/rootfs.bbclass
> > > > > @@ -272,7 +272,8 @@ python do_rootfs_install() {  }  addtask
> > > > > rootfs_install before do_rootfs_postprocess after do_unpack
> > > > >
> > > > > -cache_deb_src() {
> > > > > +do_cache_deb_src[network] = "${TASK_USE_SUDO}"
> > > > > +do_cache_deb_src() {
> > > > >      if [ -e "${ROOTFSDIR}"/etc/resolv.conf ] ||
> > > > >         [ -h "${ROOTFSDIR}"/etc/resolv.conf ]; then
> > > > >          sudo mv "${ROOTFSDIR}"/etc/resolv.conf
> > > > > "${ROOTFSDIR}"/etc/resolv.conf.isar
> > > > > --
> > > > > 2.39.5
> > > > >
> > > > > Hi All,
> > > > >
> > > > >           Can you please update the status of this patch?, or
> > > > > anything
> > > > > missing from my end?
> > > > >
> > > >
> > > > Hello.
> > > >
> > > > Sorry for delays related to "installer" currently breaking full
> > > > CI
> > > > on
> > > > "next" branch.
> > > >
> > > > Tested recently your patch, it also makes CI fail at:
> > > >
> > > > CrossTest.test_cross_debsrc:
> > > >
> > > > ```
> > > > [stdlog] 2025-03-19 21:29:50,542 avocado.app ERROR| ERROR: Task
> > > > (mc:qemuarm64-
> > > > bookworm:/workspace/build/isar_ub_devel_fast/214/meta-
> > > > test/recipes-core/images/isar-image-ci.bb:do_cache_deb_src)
> > > > failed
> > > > with exit code '1'
> > > > [stdlog] 2025-03-19 21:30:44,921 avocado.test INFO | | DEBUG:
> > > > Executing shell function do_cache_deb_src [stdlog] 2025-03-19
> > > > 21:30:44,921 avocado.test INFO | | Reading package lists...
> > > > [stdlog] 2025-03-19 21:30:44,921 avocado.test INFO | | E: You
> > > > must
> > > > put some 'deb-src' URIs in your sources.list ```
> > > >
> > > > NoCrossTest.test_nocross_debsrc:
> > > >
> > > > ```
> > > > [stdlog] 2025-03-20 01:09:35,759 avocado.app ERROR| ERROR:
> > > > Logfile
> > > > of
> > > > failure stored in:
> > > > /workspace/build/isar_ub_devel/126/build/tmp/work/debian-
> > > > bookworm-
> > > > armhf/isar-image-ci-qemuarm/1.0-
> > > > r0/temp/log.do_cache_deb_src.218291
> > > > [stdlog] 2025-03-20 01:09:35,832 avocado.app ERROR| ERROR: Task
> > > > (mc:qemuarm-bookworm:/workspace/build/isar_ub_devel/126/meta-
> > > > test/recipes-core/images/isar-image-ci.bb:do_cache_deb_src)
> > > > failed
> > > > with exit code '1'
> > > > [stdlog] 2025-03-20 01:19:35,822 avocado.test INFO | | DEBUG:
> > > > Executing shell function do_cache_deb_src [stdlog] 2025-03-20
> > > > 01:19:35,823 avocado.test INFO | | Reading package lists...
> > > > [stdlog] 2025-03-20 01:19:35,823 avocado.test INFO | | E: You
> > > > must
> > > > put some 'deb-src' URIs in your sources.list ```
> > > >
> > > >
> > > > Hi Uladzimir,
> > > >
> > > >                       Thank you for the reply, I am not able to
> > > > reproduce the build issue what you reported , I triggered the
> > > > build
> > > > for isar-image-ci target and it's getting passed.
> > > >
> > > > This patch depend on Cedric patch [1]
> > > > [1]:
> > > > https://groups.google.com/g/isar-users/c/OwrSBmCPe0Y
> > > >
> > > > I validated my patch along with the [1], it seems working fine.
> > > >
> > > > Or Can you please provide the steps to reproduce the issue?
> > > >
> > > >
> > > > > Many thanks,
> > > > > Srinu
> > > > >
> > > >
> > >
> > > I finally found root cause and easy steps to reproduce. The patch
> > > moves
> > > debsrc caching to later stage and that results in incompatibility
> > > with
> > > DISTRO_APT_PREMIRRORS option we are globally using in CI.
> >
> > Ah thanks. Are you seeing any issues with my patch alone? (was
> > going
> > to
> > re-run a local CI build to *double* check)
> >
> > Would appreciate if my changes may be CI-checked alone and merged;
> > provided there are no issues ofcourse :)
> >
>
> Actually, found easy way to reproduce the issue with "deb-dl-dir:
> remove excessive calls to dpkg-deb in debsrc_download" on top of
> 'next'.

Fantastic!

I have revised my patch to combine the list of source packages we'd
like and the list of packages known to apt (apt-cache dumpavail).

I am now going to exercise this revised approach in our downstream
layer (where we are downloading thousands of source packages) but also
with the Isar CI tests.

This will probably take a little while... stay tuned!
and thanks again for all the help

>
> Steps to reproduce with kas-container:
>
> 1. Place the deb we fail to download sources for to downloads
> directory.
> ```
> mkdir -p ./build/downloads/deb/raspios-bookworm
> wget
> https://deb.debian.org/debian/pool/main/d/debian-archive-keyring/debian-archive-keyring_2023.3+deb12u1_all.deb
> -P build/downloads/deb/raspios-bookworm/
> ```
>
> 2. Configure build:
>
> ./kas/kas-container menu
>  - `Machine selection`: rpi-arm-v7l
>  - `Distro selection`: Raspberry Pi OS Debian 12 (bookworm)
>  - `Cache all source packages`: check the option
>
> Run build
>
> 3. Result:
>
> ```
> > E: Can not find version '2023.3+deb12u1' of package 'debian-
> > archive-
> keyring'
> > E: Unable to find a source package for debian-archive-keyring
> > DEBUG: Executing shell function rootfs_do_umounts
> > DEBUG: Shell function rootfs_do_umounts finished
> > DEBUG: Python function do_rootfs_postprocess finished
> ERROR: Task (/build/../repo/meta-isar/recipes-core/images/isar-image-
> base.bb:do_rootfs_postprocess) failed with exit code '1'
> ```
>
> The issue is not reproducable on 'next' (since we are parsing
> dpkg.log
> and it has no entry for debian-archive-keyring, since it is not a
> part
> of current build.
>
> > >
> > > Steps to reproduce:
> > >
> > > 1. Run `./kas/kas-container menu`
> > > 2. Change the configuration:
> > >  - `Mirror selection => Use specific Debian mirror`:
> > > http://ftp.de.debian.org/
> > >  - `Cache all source packages`: Check the option
> > >  - `Packages selection => Isar packages`: Uncheck everything to
> > > speedup
> > > build (not necessary step)
> > > 3. Run build
> > >
> > > It fails with:
> > >
> > > ```
> > > ERROR: isar-image-base-1.0-r0 do_cache_deb_src:
> > > ExecutionError('/build/tmp/work/debian-bookworm-amd64/isar-image-
> > > base-
> > > qemuamd64/1.0-r0/temp/run.do_cache_deb_src.10979', 100, None,
> > > None)
> > > ERROR: Logfile of failure stored in: /build/tmp/work/debian-
> > > bookworm-
> > > amd64/isar-image-base-qemuamd64/1.0-
> > > r0/temp/log.do_cache_deb_src.10979
> > > Log data follows:
> > > > DEBUG: Executing shell function do_cache_deb_src
> > > > Reading package lists...
> > > > E: You must put some 'deb-src' URIs in your sources.list
> > > ERROR: Task (/build/../repo/meta-isar/recipes-core/images/isar-
> > > image-
> > > base.bb:do_cache_deb_src) failed with exit code '1'
> > > ```
> > >
> > > > --
> > > > Best regards,
> > > > Uladzimir.
> > > >
> > >
> >
> > --
> > Cedric Hombourger
> > Siemens AG
> > http://www.siemens.com/
>

--
Cedric Hombourger
Siemens AG
http://www.siemens.com/
Srinuvasan Arjunan March 27, 2025, 4:55 p.m. UTC | #13 
On Fri, Mar 21, 2025 at 7:20 PM 'cedric.hombourger@siemens.com' via
isar-users <isar-users@googlegroups.com> wrote:

> On Fri, 2025-03-21 at 11:39 +0300, Uladzimir Bely wrote:
> > On Fri, 2025-03-21 at 06:32 +0000,
> > cedric.hombourger@siemens.com wrote:
> > > On Fri, 2025-03-21 at 09:27 +0300, Uladzimir Bely wrote:
> > > > On Thu, 2025-03-20 at 07:18 +0000, 'Arjunan, Srinu' via isar-
> > > > users
> > > > wrote:
> > > > >
> > > > >
> > > > > -----Original Message-----
> > > > > From: Uladzimir Bely <ubely@ilbers.de>
> > > > > Sent: 20 March 2025 11:02
> > > > > To: Arjunan, Srinu (FT FDS CES LX PBU 2)
> > > > > <srinuvasan.a@siemens.com>;
> > > > > isar-users@googlegroups.com
> > > > > Cc: Kiszka, Jan (FT RPD CED) <jan.kiszka@siemens.com>;
> > > > > Hombourger,
> > > > > Cedric (FT FDS CES LX) <cedric.hombourger@siemens.com>;
> > > > > Niedermayr,
> > > > > Benedikt (FT RPD CED OES-DE) <benedikt.niedermayr@siemens.com>;
> > > > > Adithya.Balakumar@toshiba-tsip.com
> > > > > Subject: Re: [PATCH 1/1] image: move deb-src-caching
> > > > > functionality
> > > > > after image creation
> > > > >
> > > > > On Tue, 2025-03-18 at 16:45 +0000, Arjunan, Srinu wrote:
> > > > > >
> > > > > >
> > > > > > -----Original Message-----
> > > > > > From: Arjunan, Srinu (FT FDS CES LX PBU 2)
> > > > > > <srinuvasan.a@siemens.com>
> > > > > > Sent: 10 March 2025 17:22
> > > > > > To: isar-users@googlegroups.com
> > > > > > Cc: Kiszka, Jan (FT RPD CED) <jan.kiszka@siemens.com>;
> > > > > > Hombourger,
> > > > > > Cedric (FT FDS CES LX) <cedric.hombourger@siemens.com>;
> > > > > > ubely@ilbers.de; Niedermayr, Benedikt (FT RPD CED OES-DE)
> > > > > > <benedikt.niedermayr@siemens.com>;
> > > > > > Adithya.Balakumar@toshiba-tsip.com; Arjunan, Srinu (FT FDS
> > > > > > CES
> > > > > > LX
> > > > > > PBU
> > > > > > 2) <srinuvasan.a@siemens.com>
> > > > > > Subject: [PATCH 1/1] image: move deb-src-caching
> > > > > > functionality
> > > > > > after
> > > > > > image creation
> > > > > >
> > > > > > From: srinuvasan <srinuvasan.a@siemens.com>
> > > > > >
> > > > > > When we do deb-src caching, we are missing to cache imager
> > > > > > related
> > > > > > packages, because the deb-src-caching functionality runs in
> > > > > > the
> > > > > > stage
> > > > > > of ROOTFS_POSTPROCESS_COMAMND, but image creation task runs
> > > > > > after
> > > > > > the
> > > > > > rootfs creation, hence imager related packages are missing in
> > > > > > rootfs's
> > > > > > var/cache/apt/archives.
> > > > > >
> > > > > > To solve this issue run the deb-src-caching after image
> > > > > > creation,
> > > > > > now
> > > > > > we can able to download deb-src for all the deb packages.
> > > > > >
> > > > > > Signed-off-by: srinuvasan <srinuvasan.a@siemens.com>
> > > > > > ---
> > > > > >  meta/classes/image.bbclass  | 8 +++++++-
> > > > > > meta/classes/rootfs.bbclass
> > > > > > > 3 ++-
> > > > > >  2 files changed, 9 insertions(+), 2 deletions(-)
> > > > > >
> > > > > > diff --git a/meta/classes/image.bbclass
> > > > > > b/meta/classes/image.bbclass
> > > > > > index 56eca202..66b4d313 100644
> > > > > > --- a/meta/classes/image.bbclass
> > > > > > +++ b/meta/classes/image.bbclass
> > > > > > @@ -72,7 +72,13 @@ ROOTFS_MANIFEST_DEPLOY_DIR ?=
> > > > > > "${DEPLOY_DIR_IMAGE}"
> > > > > >  ROOTFS_DPKGSTATUS_DEPLOY_DIR ?= "${DEPLOY_DIR_IMAGE}"
> > > > > >  ROOTFS_PACKAGE_SUFFIX ?= "${PN}-${DISTRO}-${MACHINE}"
> > > > > >
> > > > > > -ROOTFS_POSTPROCESS_COMMAND:prepend =
> > > > > > "${@bb.utils.contains('BASE_REPO_FEATURES', 'cache-deb-src',
> > > > > > 'cache_deb_src', '', d)} "
> > > > > > +CACHE_DEB_SRC = "${@bb.utils.contains('BASE_REPO_FEATURES',
> > > > > > 'cache-
> > > > > > deb-src', '1', '0', d)}"
> > > > > > +
> > > > > > +python () {
> > > > > > +    if bb.utils.to_boolean(d.getVar('CACHE_DEB_SRC')):
> > > > > > +        bb.build.addtask('do_cache_deb_src', 'do_deploy',
> > > > > > 'do_image',
> > > > > > +d) }
> > > > > > +
> > > > > >
> > > > > >  inherit rootfs
> > > > > >  inherit sdk
> > > > > > diff --git a/meta/classes/rootfs.bbclass
> > > > > > b/meta/classes/rootfs.bbclass
> > > > > > index 2348e269..c9e832dc 100644
> > > > > > --- a/meta/classes/rootfs.bbclass
> > > > > > +++ b/meta/classes/rootfs.bbclass
> > > > > > @@ -272,7 +272,8 @@ python do_rootfs_install() {  }  addtask
> > > > > > rootfs_install before do_rootfs_postprocess after do_unpack
> > > > > >
> > > > > > -cache_deb_src() {
> > > > > > +do_cache_deb_src[network] = "${TASK_USE_SUDO}"
> > > > > > +do_cache_deb_src() {
> > > > > >      if [ -e "${ROOTFSDIR}"/etc/resolv.conf ] ||
> > > > > >         [ -h "${ROOTFSDIR}"/etc/resolv.conf ]; then
> > > > > >          sudo mv "${ROOTFSDIR}"/etc/resolv.conf
> > > > > > "${ROOTFSDIR}"/etc/resolv.conf.isar
> > > > > > --
> > > > > > 2.39.5
> > > > > >
> > > > > > Hi All,
> > > > > >
> > > > > >           Can you please update the status of this patch?, or
> > > > > > anything
> > > > > > missing from my end?
> > > > > >
> > > > >
> > > > > Hello.
> > > > >
> > > > > Sorry for delays related to "installer" currently breaking full
> > > > > CI
> > > > > on
> > > > > "next" branch.
> > > > >
> > > > > Tested recently your patch, it also makes CI fail at:
> > > > >
> > > > > CrossTest.test_cross_debsrc:
> > > > >
> > > > > ```
> > > > > [stdlog] 2025-03-19 21:29:50,542 avocado.app ERROR| ERROR: Task
> > > > > (mc:qemuarm64-
> > > > > bookworm:/workspace/build/isar_ub_devel_fast/214/meta-
> > > > > test/recipes-core/images/isar-image-ci.bb:do_cache_deb_src)
> > > > > failed
> > > > > with exit code '1'
> > > > > [stdlog] 2025-03-19 21:30:44,921 avocado.test INFO | | DEBUG:
> > > > > Executing shell function do_cache_deb_src [stdlog] 2025-03-19
> > > > > 21:30:44,921 avocado.test INFO | | Reading package lists...
> > > > > [stdlog] 2025-03-19 21:30:44,921 avocado.test INFO | | E: You
> > > > > must
> > > > > put some 'deb-src' URIs in your sources.list ```
> > > > >
> > > > > NoCrossTest.test_nocross_debsrc:
> > > > >
> > > > > ```
> > > > > [stdlog] 2025-03-20 01:09:35,759 avocado.app ERROR| ERROR:
> > > > > Logfile
> > > > > of
> > > > > failure stored in:
> > > > > /workspace/build/isar_ub_devel/126/build/tmp/work/debian-
> > > > > bookworm-
> > > > > armhf/isar-image-ci-qemuarm/1.0-
> > > > > r0/temp/log.do_cache_deb_src.218291
> > > > > [stdlog] 2025-03-20 01:09:35,832 avocado.app ERROR| ERROR: Task
> > > > > (mc:qemuarm-bookworm:/workspace/build/isar_ub_devel/126/meta-
> > > > > test/recipes-core/images/isar-image-ci.bb:do_cache_deb_src)
> > > > > failed
> > > > > with exit code '1'
> > > > > [stdlog] 2025-03-20 01:19:35,822 avocado.test INFO | | DEBUG:
> > > > > Executing shell function do_cache_deb_src [stdlog] 2025-03-20
> > > > > 01:19:35,823 avocado.test INFO | | Reading package lists...
> > > > > [stdlog] 2025-03-20 01:19:35,823 avocado.test INFO | | E: You
> > > > > must
> > > > > put some 'deb-src' URIs in your sources.list ```
> > > > >
> > > > >
> > > > > Hi Uladzimir,
> > > > >
> > > > >                       Thank you for the reply, I am not able to
> > > > > reproduce the build issue what you reported , I triggered the
> > > > > build
> > > > > for isar-image-ci target and it's getting passed.
> > > > >
> > > > > This patch depend on Cedric patch [1]
> > > > > [1]:
> > > > > https://groups.google.com/g/isar-users/c/OwrSBmCPe0Y
> > > > >
> > > > > I validated my patch along with the [1], it seems working fine.
> > > > >
> > > > > Or Can you please provide the steps to reproduce the issue?
> > > > >
> > > > >
> > > > > > Many thanks,
> > > > > > Srinu
> > > > > >
> > > > >
> > > >
> > > > I finally found root cause and easy steps to reproduce. The patch
> > > > moves
> > > > debsrc caching to later stage and that results in incompatibility
> > > > with
> > > > DISTRO_APT_PREMIRRORS option we are globally using in CI.
> > >
> > > Ah thanks. Are you seeing any issues with my patch alone? (was
> > > going
> > > to
> > > re-run a local CI build to *double* check)
> > >
> > > Would appreciate if my changes may be CI-checked alone and merged;
> > > provided there are no issues ofcourse :)
> > >
> >
> > Actually, found easy way to reproduce the issue with "deb-dl-dir:
> > remove excessive calls to dpkg-deb in debsrc_download" on top of
> > 'next'.
>
> Fantastic!
>
> I have revised my patch to combine the list of source packages we'd
> like and the list of packages known to apt (apt-cache dumpavail).
>
> I am now going to exercise this revised approach in our downstream
> layer (where we are downloading thousands of source packages) but also
> with the Isar CI tests.
>
> This will probably take a little while... stay tuned!
> and thanks again for all the help
>
> >
> > Steps to reproduce with kas-container:
> >
> > 1. Place the deb we fail to download sources for to downloads
> > directory.
> > ```
> > mkdir -p ./build/downloads/deb/raspios-bookworm
> > wget
> >
> https://deb.debian.org/debian/pool/main/d/debian-archive-keyring/debian-archive-keyring_2023.3+deb12u1_all.deb
> > -P build/downloads/deb/raspios-bookworm/
> > ```
> >
> > 2. Configure build:
> >
> > ./kas/kas-container menu
> >  - `Machine selection`: rpi-arm-v7l
> >  - `Distro selection`: Raspberry Pi OS Debian 12 (bookworm)
> >  - `Cache all source packages`: check the option
> >
> > Run build
> >
> > 3. Result:
> >
> > ```
> > > E: Can not find version '2023.3+deb12u1' of package 'debian-
> > > archive-
> > keyring'
> > > E: Unable to find a source package for debian-archive-keyring
> > > DEBUG: Executing shell function rootfs_do_umounts
> > > DEBUG: Shell function rootfs_do_umounts finished
> > > DEBUG: Python function do_rootfs_postprocess finished
> > ERROR: Task (/build/../repo/meta-isar/recipes-core/images/isar-image-
> > base.bb:do_rootfs_postprocess) failed with exit code '1'
> > ```
> >
> > The issue is not reproducable on 'next' (since we are parsing
> > dpkg.log
> > and it has no entry for debian-archive-keyring, since it is not a
> > part
> > of current build.
> >
> > > >
> > > > Steps to reproduce:
> > > >
> > > > 1. Run `./kas/kas-container menu`
> > > > 2. Change the configuration:
> > > >  - `Mirror selection => Use specific Debian mirror`:
> > > > http://ftp.de.debian.org/
> > > >  - `Cache all source packages`: Check the option
> > > >  - `Packages selection => Isar packages`: Uncheck everything to
> > > > speedup
> > > > build (not necessary step)
> > > > 3. Run build
> > > >
> > > > It fails with:
> > > >
> > > > ```
> > > > ERROR: isar-image-base-1.0-r0 do_cache_deb_src:
> > > > ExecutionError('/build/tmp/work/debian-bookworm-amd64/isar-image-
> > > > base-
> > > > qemuamd64/1.0-r0/temp/run.do_cache_deb_src.10979', 100, None,
> > > > None)
> > > > ERROR: Logfile of failure stored in: /build/tmp/work/debian-
> > > > bookworm-
> > > > amd64/isar-image-base-qemuamd64/1.0-
> > > > r0/temp/log.do_cache_deb_src.10979
> > > > Log data follows:
> > > > > DEBUG: Executing shell function do_cache_deb_src
> > > > > Reading package lists...
> > > > > E: You must put some 'deb-src' URIs in your sources.list
> > > > ERROR: Task (/build/../repo/meta-isar/recipes-core/images/isar-
> > > > image-
> > > > base.bb:do_cache_deb_src) failed with exit code '1'
> > > > ```
> > > >
> > > > > --
> > > > > Best regards,
> > > > > Uladzimir.
> > > > >
>

 Hi Uladzimir,

 I triggered the CI build locally with this patch; it looks like CI builds
are getting passed, Here are the logs:

Logs:

builder@0ed36b8446c5:/work/isar/testsuite$ avocado run citest.py -t dev
--max-parallel-tasks=1
JOB ID     : 3ca66c44c8b33b1e2f265f46613f502fa89dbbde
JOB LOG    :
/tmp/tmppvxevsmp/avocado/job-results/job-2025-03-27T15.58-3ca66c4/job.log
 (1/6) citest.py:DevTest.test_dev: STARTED
 (1/6) citest.py:DevTest.test_dev: PASS (1004.16 s)
 (2/6) citest.py:DevTest.test_dev_apps: STARTED
 (2/6) citest.py:DevTest.test_dev_apps: PASS (1153.27 s)
 (3/6) citest.py:DevTest.test_dev_rebuild: STARTED
 (3/6) citest.py:DevTest.test_dev_rebuild: PASS (450.10 s)
 (4/6) citest.py:DevTest.test_dev_run_amd64_bullseye: STARTED
 (4/6) citest.py:DevTest.test_dev_run_amd64_bullseye: PASS (56.52 s)
 (5/6) citest.py:DevTest.test_dev_run_arm64_bullseye: STARTED
 (5/6) citest.py:DevTest.test_dev_run_arm64_bullseye: PASS (45.19 s)
 (6/6) citest.py:DevTest.test_dev_run_arm_bullseye: STARTED
 (6/6) citest.py:DevTest.test_dev_run_arm_bullseye: PASS (49.96 s)
RESULTS    : PASS 6 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 |
CANCEL 0
JOB TIME   : 2765.02 s

Many thanks,
Srinu

> > > >
> > >
> > > --
> > > Cedric Hombourger
> > > Siemens AG
> > > http://www.siemens.com/
> >
>
> --
> Cedric Hombourger
> Siemens AG
> http://www.siemens.com/
>
> --
> You received this message because you are subscribed to the Google Groups
> "isar-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to isar-users+unsubscribe@googlegroups.com.
> To view this discussion visit
> https://groups.google.com/d/msgid/isar-users/1c66d14280d6a27e1529659b2d651f172a32ed87.camel%40siemens.com
> .
>
diff mbox series

Patch

diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass
index 56eca202..66b4d313 100644
--- a/meta/classes/image.bbclass
+++ b/meta/classes/image.bbclass
@@ -72,7 +72,13 @@  ROOTFS_MANIFEST_DEPLOY_DIR ?= "${DEPLOY_DIR_IMAGE}"
 ROOTFS_DPKGSTATUS_DEPLOY_DIR ?= "${DEPLOY_DIR_IMAGE}"
 ROOTFS_PACKAGE_SUFFIX ?= "${PN}-${DISTRO}-${MACHINE}"
 
-ROOTFS_POSTPROCESS_COMMAND:prepend = "${@bb.utils.contains('BASE_REPO_FEATURES', 'cache-deb-src', 'cache_deb_src', '', d)} "
+CACHE_DEB_SRC = "${@bb.utils.contains('BASE_REPO_FEATURES', 'cache-deb-src', '1', '0', d)}"
+
+python () {
+    if bb.utils.to_boolean(d.getVar('CACHE_DEB_SRC')):
+        bb.build.addtask('do_cache_deb_src', 'do_deploy', 'do_image', d)
+}
+
 
 inherit rootfs
 inherit sdk
diff --git a/meta/classes/rootfs.bbclass b/meta/classes/rootfs.bbclass
index 2348e269..c9e832dc 100644
--- a/meta/classes/rootfs.bbclass
+++ b/meta/classes/rootfs.bbclass
@@ -272,7 +272,8 @@  python do_rootfs_install() {
 }
 addtask rootfs_install before do_rootfs_postprocess after do_unpack
 
-cache_deb_src() {
+do_cache_deb_src[network] = "${TASK_USE_SUDO}"
+do_cache_deb_src() {
     if [ -e "${ROOTFSDIR}"/etc/resolv.conf ] ||
        [ -h "${ROOTFSDIR}"/etc/resolv.conf ]; then
         sudo mv "${ROOTFSDIR}"/etc/resolv.conf "${ROOTFSDIR}"/etc/resolv.conf.isar