| Message ID | 20240716202021.226020-1-benedikt.niedermayr@siemens.com |
|---|---|
| State | Superseded, archived |
| Headers | show |
| Series | bitbake.conf: fix vardepsexclude for ZSTD_DEFAULTS | expand |
On 16.07.24 22:20, 'Benedikt Niedermayr' via isar-users wrote: > This fixes a rebuild in case ZSTD_LEVEL changes. This is valid since > a different compression level leads to different outputs. > > Signed-off-by: Benedikt Niedermayr <benedikt.niedermayr@siemens.com> > --- > meta/conf/bitbake.conf | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > > diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf > index 4cfa8b10b58b..2ecba508935c 100644 > --- a/meta/conf/bitbake.conf > +++ b/meta/conf/bitbake.conf > @@ -161,7 +161,7 @@ ZSTD_THREADS ?= "${@oe.utils.cpu_count(at_least=2)}" > ZSTD_THREADS[vardepvalue] = "1" > ZSTD_LEVEL ?= "19" > ZSTD_DEFAULTS ?= "--rsyncable -${ZSTD_LEVEL} --threads=${ZSTD_THREADS}" > -ZSTD_DEFAULTS[vardepsexclude] += "ZSTD_LEVEL ZSTD_THREADS" > +ZSTD_DEFAULTS[vardepsexclude] += "ZSTD_THREADS" Checking what OE does by now: # Default parallelism for zstd ZSTD_THREADS ?= "${@oe.utils.cpu_count(at_least=2)}" ZSTD_THREADS[vardepvalue] = "1" ZSTD_COMPRESSION_LEVEL ?= "-3" ZSTD_DEFAULTS ?= "--threads=${ZSTD_THREADS} ${ZSTD_COMPRESSION_LEVEL}" ZSTD_DEFAULTS[vardepsexclude] = "ZSTD_THREADS" We should likely follow that, also naming-wise. Reducing the compression level to 3 is probably more CPU friendly as well as we have seen with larger artifacts. Just keep --rsyncable, see 241155f7. What I don't get yet: ZSTD_THREADS is pinned to 1 for the purpose of dependency calculations. Why do they additionally use vardepsexclude? Jan
On Wed, 2024-07-17 at 07:24 +0200, Jan Kiszka wrote: > On 16.07.24 22:20, 'Benedikt Niedermayr' via isar-users wrote: > > This fixes a rebuild in case ZSTD_LEVEL changes. This is valid since > > a different compression level leads to different outputs. > > > > Signed-off-by: Benedikt Niedermayr <benedikt.niedermayr@siemens.com> > > --- > > meta/conf/bitbake.conf | 2 +- > > 1 file changed, 1 insertion(+), 1 deletion(-) > > > > diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf > > index 4cfa8b10b58b..2ecba508935c 100644 > > --- a/meta/conf/bitbake.conf > > +++ b/meta/conf/bitbake.conf > > @@ -161,7 +161,7 @@ ZSTD_THREADS ?= "${@oe.utils.cpu_count(at_least=2)}" > > ZSTD_THREADS[vardepvalue] = "1" > > ZSTD_LEVEL ?= "19" > > ZSTD_DEFAULTS ?= "--rsyncable -${ZSTD_LEVEL} --threads=${ZSTD_THREADS}" > > -ZSTD_DEFAULTS[vardepsexclude] += "ZSTD_LEVEL ZSTD_THREADS" > > +ZSTD_DEFAULTS[vardepsexclude] += "ZSTD_THREADS" > > Checking what OE does by now: > > # Default parallelism for zstd > ZSTD_THREADS ?= "${@oe.utils.cpu_count(at_least=2)}" > ZSTD_THREADS[vardepvalue] = "1" > > ZSTD_COMPRESSION_LEVEL ?= "-3" > ZSTD_DEFAULTS ?= "--threads=${ZSTD_THREADS} ${ZSTD_COMPRESSION_LEVEL}" > ZSTD_DEFAULTS[vardepsexclude] = "ZSTD_THREADS" > > We should likely follow that, also naming-wise. Reducing the compression > level to 3 is probably more CPU friendly as well as we have seen with > larger artifacts. Just keep --rsyncable, see 241155f7. > > What I don't get yet: ZSTD_THREADS is pinned to 1 for the purpose of > dependency calculations. Why do they additionally use vardepsexclude? I'm also not sure. Maybe some sstate magic going on of which I'm not aware of. We can try to copy everything but the vardepsexclude with ZSTD_THREADS and see if any issues occur. If not we could fix/refactor that in OE later on. Benedikt > > Jan >
diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf index 4cfa8b10b58b..2ecba508935c 100644 --- a/meta/conf/bitbake.conf +++ b/meta/conf/bitbake.conf @@ -161,7 +161,7 @@ ZSTD_THREADS ?= "${@oe.utils.cpu_count(at_least=2)}" ZSTD_THREADS[vardepvalue] = "1" ZSTD_LEVEL ?= "19" ZSTD_DEFAULTS ?= "--rsyncable -${ZSTD_LEVEL} --threads=${ZSTD_THREADS}" -ZSTD_DEFAULTS[vardepsexclude] += "ZSTD_LEVEL ZSTD_THREADS" +ZSTD_DEFAULTS[vardepsexclude] += "ZSTD_THREADS" # Default compression settings for zchunk ZCK_DEFAULTS ?= ""
This fixes a rebuild in case ZSTD_LEVEL changes. This is valid since a different compression level leads to different outputs. Signed-off-by: Benedikt Niedermayr <benedikt.niedermayr@siemens.com> --- meta/conf/bitbake.conf | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)