[1/1] make compressed images better syncable

Message ID 20230425101216.1693998-1-felix.moessbauer@siemens.com
State Accepted, archived
Headers show
Series [1/1] make compressed images better syncable | expand

Commit Message

MOESSBAUER, Felix April 25, 2023, 10:12 a.m. UTC
For compression formats that support these options (currently only zstd),
we enable to include the sparseness information into the archive. By that,
disk IO on decompression of large zero blocks is speedup and storage usage
is decreased. Usually, this is handled automatically by the compression
tools, but not when sending output to stdout.

In addition, we add the rsyncable flag to reduce the number
of changed blocks in the output when only changing some input blocks.

Signed-off-by: Felix Moessbauer <felix.moessbauer@siemens.com>
---
 meta/classes/imagetypes.bbclass | 2 +-
 meta/conf/bitbake.conf          | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

Comments

Anton Mikanovich May 8, 2023, 5:58 p.m. UTC | #1
25/04/2023 13:12, 'Felix Moessbauer' via isar-users wrote:
> For compression formats that support these options (currently only zstd),
> we enable to include the sparseness information into the archive. By that,
> disk IO on decompression of large zero blocks is speedup and storage usage
> is decreased. Usually, this is handled automatically by the compression
> tools, but not when sending output to stdout.
>
> In addition, we add the rsyncable flag to reduce the number
> of changed blocks in the output when only changing some input blocks.
>
> Signed-off-by: Felix Moessbauer <felix.moessbauer@siemens.com>

Applied to next, thanks.

Patch

diff --git a/meta/classes/imagetypes.bbclass b/meta/classes/imagetypes.bbclass
index 2ee3240b..a80a6da5 100644
--- a/meta/classes/imagetypes.bbclass
+++ b/meta/classes/imagetypes.bbclass
@@ -101,5 +101,5 @@  CONVERSION_DEPS:gz = "gzip"
 CONVERSION_CMD:xz = "${SUDO_CHROOT} sh -c 'xz -c ${XZ_DEFAULTS} ${IMAGE_FILE_CHROOT} > ${IMAGE_FILE_CHROOT}.xz'"
 CONVERSION_DEPS:xz = "xz-utils"
 
-CONVERSION_CMD:zst = "${SUDO_CHROOT} sh -c 'zstd -c ${ZSTD_DEFAULTS} ${IMAGE_FILE_CHROOT} > ${IMAGE_FILE_CHROOT}.zst'"
+CONVERSION_CMD:zst = "${SUDO_CHROOT} sh -c 'zstd -c --sparse ${ZSTD_DEFAULTS} ${IMAGE_FILE_CHROOT} > ${IMAGE_FILE_CHROOT}.zst'"
 CONVERSION_DEPS:zst = "zstd"
diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf
index f421050e..c660660a 100644
--- a/meta/conf/bitbake.conf
+++ b/meta/conf/bitbake.conf
@@ -141,7 +141,7 @@  XZ_DEFAULTS[vardepsexclude] += "XZ_MEMLIMIT XZ_THREADS"
 ZSTD_THREADS ?= "${@oe.utils.cpu_count(at_least=2)}"
 ZSTD_THREADS[vardepvalue] = "1"
 ZSTD_LEVEL ?= "19"
-ZSTD_DEFAULTS ?= "-${ZSTD_LEVEL} --threads=${ZSTD_THREADS}"
+ZSTD_DEFAULTS ?= "--rsyncable -${ZSTD_LEVEL} --threads=${ZSTD_THREADS}"
 ZSTD_DEFAULTS[vardepsexclude] += "ZSTD_LEVEL ZSTD_THREADS"
 
 BBINCLUDELOGS ??= "yes"