diff mbox series

[2/2] sstate: guard sstate funcs against execution in wrong context

Message ID 20260625150232.531442-3-felix.moessbauer@siemens.com
State Under Review
Headers show
Series Fix data races in sstate tasks | expand

Commit Message

MOESSBAUER, Felix June 25, 2026, 3:02 p.m. UTC 
In sstate.bbclass, sstate_package() sets SSTATE_CURRTASK from
the sstate task being packaged, then runs every function listed
in SSTATECREATEFUNCS. This happens independent of the context the sstate
task is scoped to for all tasks in a single recipe.

While this pattern is not problematic in isar itself (we only have one
SSTATECREATEFUNC per recipe), it becomes problematic when downstream
users define further manual sstate tasks. In this case, the sstate tasks
are executed in the wrong context, leading to races and failures in all
colors.

We fix this by guarding each manually defined sstate func to only run in
its intended context.

Signed-off-by: Felix Moessbauer <felix.moessbauer@siemens.com>
---
 meta/classes-recipe/dpkg-base.bbclass                 | 4 ++++
 meta/classes-recipe/rootfs.bbclass                    | 4 ++++
 meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc | 4 ++++
 3 files changed, 12 insertions(+)

Comments

Jan Kiszka June 25, 2026, 4:43 p.m. UTC | #1 
On 25.06.26 17:02, 'Felix Moessbauer' via isar-users wrote:
> In sstate.bbclass, sstate_package() sets SSTATE_CURRTASK from
> the sstate task being packaged, then runs every function listed
> in SSTATECREATEFUNCS. This happens independent of the context the sstate
> task is scoped to for all tasks in a single recipe.
> 
> While this pattern is not problematic in isar itself (we only have one
> SSTATECREATEFUNC per recipe), it becomes problematic when downstream
> users define further manual sstate tasks. In this case, the sstate tasks
> are executed in the wrong context, leading to races and failures in all
> colors.
> 
> We fix this by guarding each manually defined sstate func to only run in
> its intended context.
> 
> Signed-off-by: Felix Moessbauer <felix.moessbauer@siemens.com>
> ---
>  meta/classes-recipe/dpkg-base.bbclass                 | 4 ++++
>  meta/classes-recipe/rootfs.bbclass                    | 4 ++++
>  meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc | 4 ++++
>  3 files changed, 12 insertions(+)
> 
> diff --git a/meta/classes-recipe/dpkg-base.bbclass b/meta/classes-recipe/dpkg-base.bbclass
> index f7a12302..85c37a5a 100644
> --- a/meta/classes-recipe/dpkg-base.bbclass
> +++ b/meta/classes-recipe/dpkg-base.bbclass
> @@ -184,6 +184,8 @@ SSTATECREATEFUNCS += "dpkg_build_sstate_prepare"
>  SSTATEPOSTINSTFUNCS += "dpkg_build_sstate_finalize"
>  
>  dpkg_build_sstate_prepare() {
> +    [ "${SSTATE_CURRTASK}" = "dpkg_build" ] || return 0
> +
>      # this runs in SSTATE_BUILDDIR, which will be deleted automatically
>      if [ -n "$(find ${WORKDIR} -maxdepth 1 -name '*.deb' -print -quit)" ]; then
>          cp -f ${WORKDIR}/*.deb -t .
> @@ -191,6 +193,8 @@ dpkg_build_sstate_prepare() {
>  }
>  
>  dpkg_build_sstate_finalize() {
> +    [ "${SSTATE_CURRTASK}" = "dpkg_build" ] || return 0
> +
>      # this runs in SSTATE_INSTDIR
>      if [ -n "$(find . -maxdepth 1 -name '*.deb' -print -quit)" ]; then
>          mv -f ./*.deb -t ${WORKDIR}/
> diff --git a/meta/classes-recipe/rootfs.bbclass b/meta/classes-recipe/rootfs.bbclass
> index 8a4aad16..936f56fe 100644
> --- a/meta/classes-recipe/rootfs.bbclass
> +++ b/meta/classes-recipe/rootfs.bbclass
> @@ -691,6 +691,8 @@ SSTATE_TAR_ATTR_FLAGS ?= "--xattrs --xattrs-include='*'"
>  
>  # the rootfs is owned by root, so we need some sudoing to pack and unpack
>  rootfs_install_sstate_prepare() {
> +    [ "${SSTATE_CURRTASK}" = "rootfs_install" ] || return 0
> +
>      # this runs in SSTATE_BUILDDIR, which will be deleted automatically
>      # tar --one-file-system will cross bind-mounts to the same filesystem,
>      # so we use some mount magic to prevent that
> @@ -705,6 +707,8 @@ rootfs_install_sstate_prepare() {
>  rootfs_install_sstate_prepare[lockfiles] = "${REPO_ISAR_DIR}/isar.lock"
>  
>  rootfs_install_sstate_finalize() {
> +    [ "${SSTATE_CURRTASK}" = "rootfs_install" ] || return 0
> +
>      # this runs in SSTATE_INSTDIR
>      # - after building the rootfs, the tar won't be there, but we also don't need to unpack
>      # - after restoring from cache, there will be a tar which we unpack and then delete
> diff --git a/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc b/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc
> index e746f469..98f6c090 100644
> --- a/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc
> +++ b/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc
> @@ -264,12 +264,16 @@ SSTATECREATEFUNCS += "bootstrap_sstate_prepare"
>  SSTATEPOSTINSTFUNCS += "bootstrap_sstate_finalize"
>  
>  bootstrap_sstate_prepare() {
> +    [ "${SSTATE_CURRTASK}" = "bootstrap" ] || return 0
> +
>      # this runs in SSTATE_BUILDDIR, which will be deleted automatically
>      sudo cp -a "${WORKDIR}/rootfs.tar.zst" ./bootstrap.tar.zst
>      sudo chown $(id -u):$(id -g) bootstrap.tar.zst
>  }
>  
>  bootstrap_sstate_finalize() {
> +    [ "${SSTATE_CURRTASK}" = "bootstrap" ] || return 0
> +
>      # this runs in SSTATE_INSTDIR
>      # we should restore symlinks after using tar
>      if [ -f bootstrap.tar.zst ]; then

Out of curiosity: Does OE have similar setup and solve them the same
way? Or are we doing something ... special here? This pattern looks like
it does not scale very well (easily forgotten when adding another
function in Isar and even more easily in downstream).

Jan
MOESSBAUER, Felix June 25, 2026, 10:16 p.m. UTC | #2 
On Thu, 2026-06-25 at 18:43 +0200, Jan Kiszka wrote:
> On 25.06.26 17:02, 'Felix Moessbauer' via isar-users wrote:
> > In sstate.bbclass, sstate_package() sets SSTATE_CURRTASK from
> > the sstate task being packaged, then runs every function listed
> > in SSTATECREATEFUNCS. This happens independent of the context the sstate
> > task is scoped to for all tasks in a single recipe.
> > 
> > While this pattern is not problematic in isar itself (we only have one
> > SSTATECREATEFUNC per recipe), it becomes problematic when downstream
> > users define further manual sstate tasks. In this case, the sstate tasks
> > are executed in the wrong context, leading to races and failures in all
> > colors.
> > 
> > We fix this by guarding each manually defined sstate func to only run in
> > its intended context.
> > 
> > Signed-off-by: Felix Moessbauer <felix.moessbauer@siemens.com>
> > ---
> >  meta/classes-recipe/dpkg-base.bbclass                 | 4 ++++
> >  meta/classes-recipe/rootfs.bbclass                    | 4 ++++
> >  meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc | 4 ++++
> >  3 files changed, 12 insertions(+)
> > 
> > diff --git a/meta/classes-recipe/dpkg-base.bbclass b/meta/classes-recipe/dpkg-base.bbclass
> > index f7a12302..85c37a5a 100644
> > --- a/meta/classes-recipe/dpkg-base.bbclass
> > +++ b/meta/classes-recipe/dpkg-base.bbclass
> > @@ -184,6 +184,8 @@ SSTATECREATEFUNCS += "dpkg_build_sstate_prepare"
> >  SSTATEPOSTINSTFUNCS += "dpkg_build_sstate_finalize"
> >  
> >  dpkg_build_sstate_prepare() {
> > +    [ "${SSTATE_CURRTASK}" = "dpkg_build" ] || return 0
> > +
> >      # this runs in SSTATE_BUILDDIR, which will be deleted automatically
> >      if [ -n "$(find ${WORKDIR} -maxdepth 1 -name '*.deb' -print -quit)" ]; then
> >          cp -f ${WORKDIR}/*.deb -t .
> > @@ -191,6 +193,8 @@ dpkg_build_sstate_prepare() {
> >  }
> >  
> >  dpkg_build_sstate_finalize() {
> > +    [ "${SSTATE_CURRTASK}" = "dpkg_build" ] || return 0
> > +
> >      # this runs in SSTATE_INSTDIR
> >      if [ -n "$(find . -maxdepth 1 -name '*.deb' -print -quit)" ]; then
> >          mv -f ./*.deb -t ${WORKDIR}/
> > diff --git a/meta/classes-recipe/rootfs.bbclass b/meta/classes-recipe/rootfs.bbclass
> > index 8a4aad16..936f56fe 100644
> > --- a/meta/classes-recipe/rootfs.bbclass
> > +++ b/meta/classes-recipe/rootfs.bbclass
> > @@ -691,6 +691,8 @@ SSTATE_TAR_ATTR_FLAGS ?= "--xattrs --xattrs-include='*'"
> >  
> >  # the rootfs is owned by root, so we need some sudoing to pack and unpack
> >  rootfs_install_sstate_prepare() {
> > +    [ "${SSTATE_CURRTASK}" = "rootfs_install" ] || return 0
> > +
> >      # this runs in SSTATE_BUILDDIR, which will be deleted automatically
> >      # tar --one-file-system will cross bind-mounts to the same filesystem,
> >      # so we use some mount magic to prevent that
> > @@ -705,6 +707,8 @@ rootfs_install_sstate_prepare() {
> >  rootfs_install_sstate_prepare[lockfiles] = "${REPO_ISAR_DIR}/isar.lock"
> >  
> >  rootfs_install_sstate_finalize() {
> > +    [ "${SSTATE_CURRTASK}" = "rootfs_install" ] || return 0
> > +
> >      # this runs in SSTATE_INSTDIR
> >      # - after building the rootfs, the tar won't be there, but we also don't need to unpack
> >      # - after restoring from cache, there will be a tar which we unpack and then delete
> > diff --git a/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc b/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc
> > index e746f469..98f6c090 100644
> > --- a/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc
> > +++ b/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc
> > @@ -264,12 +264,16 @@ SSTATECREATEFUNCS += "bootstrap_sstate_prepare"
> >  SSTATEPOSTINSTFUNCS += "bootstrap_sstate_finalize"
> >  
> >  bootstrap_sstate_prepare() {
> > +    [ "${SSTATE_CURRTASK}" = "bootstrap" ] || return 0
> > +
> >      # this runs in SSTATE_BUILDDIR, which will be deleted automatically
> >      sudo cp -a "${WORKDIR}/rootfs.tar.zst" ./bootstrap.tar.zst
> >      sudo chown $(id -u):$(id -g) bootstrap.tar.zst
> >  }
> >  
> >  bootstrap_sstate_finalize() {
> > +    [ "${SSTATE_CURRTASK}" = "bootstrap" ] || return 0
> > +
> >      # this runs in SSTATE_INSTDIR
> >      # we should restore symlinks after using tar
> >      if [ -f bootstrap.tar.zst ]; then
> 
> Out of curiosity: Does OE have similar setup and solve them the same
> way? Or are we doing something ... special here? 
> 

They use (or nowadays mostly used) this pattern as well. However, this
is only needed when implementing the low-level sstate functions
manually, which should be avoided. When using the task flags (sstate-
inputdirs / sstate-outputdirs) this is not needed. Unfortunately we
need to implement the rootfs part manually, as tarring and untarring
the rootfs can only be performed by root (due to the permissions) and
we also want to avoid having rootfs' tarballs lingering in the workdir.

My hope is that sometime in the future we can solely rely on the task
flags, but this requires more refactoring. The rootless story is part
of it, but does not fully solve this until we drop the old privileged
part (and more ...).

You can also call it technical dept.

> This pattern looks like
> it does not scale very well (easily forgotten when adding another
> function in Isar and even more easily in downstream).

Well... you rarely need to write sstate functions by yourself, but rely
on the inputdirs / outputdirs flags instead. Then it is not needed.

Felix

> 
> Jan
> 
> -- 
> Siemens AG, Foundational Technologies
> Linux Expert Center
diff mbox series

Patch

diff --git a/meta/classes-recipe/dpkg-base.bbclass b/meta/classes-recipe/dpkg-base.bbclass
index f7a12302..85c37a5a 100644
--- a/meta/classes-recipe/dpkg-base.bbclass
+++ b/meta/classes-recipe/dpkg-base.bbclass
@@ -184,6 +184,8 @@  SSTATECREATEFUNCS += "dpkg_build_sstate_prepare"
 SSTATEPOSTINSTFUNCS += "dpkg_build_sstate_finalize"
 
 dpkg_build_sstate_prepare() {
+    [ "${SSTATE_CURRTASK}" = "dpkg_build" ] || return 0
+
     # this runs in SSTATE_BUILDDIR, which will be deleted automatically
     if [ -n "$(find ${WORKDIR} -maxdepth 1 -name '*.deb' -print -quit)" ]; then
         cp -f ${WORKDIR}/*.deb -t .
@@ -191,6 +193,8 @@  dpkg_build_sstate_prepare() {
 }
 
 dpkg_build_sstate_finalize() {
+    [ "${SSTATE_CURRTASK}" = "dpkg_build" ] || return 0
+
     # this runs in SSTATE_INSTDIR
     if [ -n "$(find . -maxdepth 1 -name '*.deb' -print -quit)" ]; then
         mv -f ./*.deb -t ${WORKDIR}/
diff --git a/meta/classes-recipe/rootfs.bbclass b/meta/classes-recipe/rootfs.bbclass
index 8a4aad16..936f56fe 100644
--- a/meta/classes-recipe/rootfs.bbclass
+++ b/meta/classes-recipe/rootfs.bbclass
@@ -691,6 +691,8 @@  SSTATE_TAR_ATTR_FLAGS ?= "--xattrs --xattrs-include='*'"
 
 # the rootfs is owned by root, so we need some sudoing to pack and unpack
 rootfs_install_sstate_prepare() {
+    [ "${SSTATE_CURRTASK}" = "rootfs_install" ] || return 0
+
     # this runs in SSTATE_BUILDDIR, which will be deleted automatically
     # tar --one-file-system will cross bind-mounts to the same filesystem,
     # so we use some mount magic to prevent that
@@ -705,6 +707,8 @@  rootfs_install_sstate_prepare() {
 rootfs_install_sstate_prepare[lockfiles] = "${REPO_ISAR_DIR}/isar.lock"
 
 rootfs_install_sstate_finalize() {
+    [ "${SSTATE_CURRTASK}" = "rootfs_install" ] || return 0
+
     # this runs in SSTATE_INSTDIR
     # - after building the rootfs, the tar won't be there, but we also don't need to unpack
     # - after restoring from cache, there will be a tar which we unpack and then delete
diff --git a/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc b/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc
index e746f469..98f6c090 100644
--- a/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc
+++ b/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc
@@ -264,12 +264,16 @@  SSTATECREATEFUNCS += "bootstrap_sstate_prepare"
 SSTATEPOSTINSTFUNCS += "bootstrap_sstate_finalize"
 
 bootstrap_sstate_prepare() {
+    [ "${SSTATE_CURRTASK}" = "bootstrap" ] || return 0
+
     # this runs in SSTATE_BUILDDIR, which will be deleted automatically
     sudo cp -a "${WORKDIR}/rootfs.tar.zst" ./bootstrap.tar.zst
     sudo chown $(id -u):$(id -g) bootstrap.tar.zst
 }
 
 bootstrap_sstate_finalize() {
+    [ "${SSTATE_CURRTASK}" = "bootstrap" ] || return 0
+
     # this runs in SSTATE_INSTDIR
     # we should restore symlinks after using tar
     if [ -f bootstrap.tar.zst ]; then