From patchwork Tue Apr 25 10:12:16 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "MOESSBAUER, Felix" X-Patchwork-Id: 2776 Return-Path: Received: from shymkent.ilbers.de ([unix socket]) by shymkent (Cyrus 2.5.10-Debian-2.5.10-3+deb9u2) with LMTPA; Tue, 25 Apr 2023 12:12:35 +0200 X-Sieve: CMU Sieve 2.4 Received: from mail-lj1-f187.google.com (mail-lj1-f187.google.com [209.85.208.187]) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPS id 33PACYXm027792 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 25 Apr 2023 12:12:34 +0200 Received: by mail-lj1-f187.google.com with SMTP id 38308e7fff4ca-2a8a54fc72csf21447971fa.3; Tue, 25 Apr 2023 03:12:34 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1682417549; cv=pass; d=google.com; s=arc-20160816; b=u7+LqyXgWIE6J41Nt4x+TMGUZv+P9+wppa+x+mETpavFTM263YKPB9HLrZfTb6tnGO ulApxyirf+QxWUikLDzpPz9XZZaoNYGPVezUiK/iNer0xEAqGtKNjzgbfvKd+VDwy81o 0LJICVorRiC2VM9zEhxf+zbv+q8Hh7Cd8lmnfU0bt1C2qvqf0eLqr21GE8HUuUgOMeKl Giclx9FzfTZq4BxLKq6mzBiPvN7Lq/BOyhhJHzXTdQs7SYBCaIUgBoLgX9qdwfvOhs9t GKA+YhsgfK8dO7yaP/ZV2Pj4S8sbmPihFre70yiMoS/615pEMrvA8KCdXmAaxspOwUhG 38vg== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:reply-to:feedback-id:mime-version :message-id:date:subject:cc:to:from:dkim-signature; bh=nooj0mlgL4KwFfqw3HOanSmJraWaYsbVcwI9YFqhyRM=; b=IB/juI+nsHwJZCg+AJOJV457gRDkHipf0+BF+hI3KZtcKZyLBCsK+eWUxeonQdEL0G d7psFAG/AkLZ5Eo62d6J4ymbM8v7o6KhAxcZx6nebN0qOWM3PKeuy8KZvL0mnDE8LW2U j51YJ/4NZAQBNmsuozT6Mnl4f2crWae0tlYpxRDLN1EhK9t+vObQXLBSjhNRiLtVEMs+ ZcyahZCTcOnHEd+1TRITneGfIME3qdCCA0BMWuc4m3Px9VL/qNjxTTF+nZylp0QT532D +2K0Hx5ntv/cYChfuYfhNF0ekIc8S7tPmiBpXjTsAlJzzVdsB2QakQZPYux5RtKTwxqP nVJg== ARC-Authentication-Results: i=2; gmr-mx.google.com; dkim=pass header.i=@siemens.com header.s=fm1 header.b=PiKQhi61; spf=pass (google.com: domain of fm-72506-20230425101226e14462ed4c50a9fb6e-8cp8og@rts-flowmailer.siemens.com designates 185.136.64.227 as permitted sender) smtp.mailfrom=fm-72506-20230425101226e14462ed4c50a9fb6e-8cP8oG@rts-flowmailer.siemens.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=siemens.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlegroups.com; s=20221208; t=1682417549; x=1685009549; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:reply-to :x-original-authentication-results:x-original-sender:feedback-id :mime-version:message-id:date:subject:cc:to:from:from:to:cc:subject :date:message-id:reply-to; bh=nooj0mlgL4KwFfqw3HOanSmJraWaYsbVcwI9YFqhyRM=; b=j/1nT8jAYs/ygUk/zSz+wvoxohsvM6Lj9oeszqfBkjvBYaKp/V7tOYFq4cbvRbNVqV IwENn640Iky4BsOcJAwQZl4Ps/rK+Jt/KBBe97E5sB441mDILJouRrgXCAx1tSbV9dtl R/PriqQla7S4tj+OL3aylmFLkKt3zmERKUF3bmU420eirl0wg5xLo4d0qU08GbT3SAzO se9z7RMsd1RvDanRFwZ8QbK5QNG3sb/4n9UedkrbgTthHnt7BVfqIbwyDxoKFItru541 JyJkW3nY9LcEJuaVOaGprElKlR0NJ167ZhEq1v/3ovaQuzKCqESRoHN3qGM5Ll4G4y4G 7Efg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1682417549; x=1685009549; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :x-spam-checked-in-group:list-id:mailing-list:precedence:reply-to :x-original-authentication-results:x-original-sender:feedback-id :mime-version:message-id:date:subject:cc:to:from:x-beenthere :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=nooj0mlgL4KwFfqw3HOanSmJraWaYsbVcwI9YFqhyRM=; b=PtIQl+BpznhTiGgAfk5sCvjqCQ2z7ev8jAAHaNUmRJui9HXqhNmfITxlEzy0YwCE2i Wd9R3wu0LFv5MYX3Q9CizXh/a3ZD10/GQf6pUHlfh9lKxQFowmXzM9X2+JlYl1yw5jXt H01Tb7GM/Vg8q4VvELnAuvuIB/t03TQVzkzUQgvFTxPVo0gWnpkCGEMh6n+EHtWxYcTk RAbbbblI/3yDbgfkGlCaDlQpjYFhgHyLs//0SziTm6R2xqwmJFlPiLAxJuzoWImrq54+ mz9FePkLEWD/eksiS78NqkyT74qhqL+3lXuX2TQQfivaq7sOeIkogH02Ja/LF9MCLyNL cVQA== X-Gm-Message-State: AAQBX9elPQ+vYsvJwcLwwhvP9LLBF7raVVfWsWc2LspbfdITDW9CpX0a gKYdcnIQTk+o18xiqoDqSAc= X-Google-Smtp-Source: AKy350ZCC+BoJ1rFNdMnHWdtSS5xCq3tWajDnLu35Vt4iZr2QycztiX7J+ulx4y1cy5nAzMD3c3Dzw== X-Received: by 2002:a05:651c:154:b0:2a7:8607:2e64 with SMTP id c20-20020a05651c015400b002a786072e64mr3294826ljd.3.1682417548641; Tue, 25 Apr 2023 03:12:28 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a05:6512:4013:b0:4ec:6fe6:9f26 with SMTP id br19-20020a056512401300b004ec6fe69f26ls718970lfb.0.-pod-prod-gmail; Tue, 25 Apr 2023 03:12:26 -0700 (PDT) X-Received: by 2002:ac2:4204:0:b0:4ea:e296:fe9e with SMTP id y4-20020ac24204000000b004eae296fe9emr4134177lfh.9.1682417546915; Tue, 25 Apr 2023 03:12:26 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1682417546; cv=none; d=google.com; s=arc-20160816; b=Oz6jCrVY5igFWk5s4XTQZRH0DwZVClrKEglcTia+JgPSMWjqpvfp5U2DUnLdw0/dO4 s8c4va/iURcbJD1IBrZcJ1zJYmLYtNXU/yPIFsf6qS08OlFKkJM8pb2mXc9R7Q6TtqzI Vs2GdpncQFn0A04FUIeXcRs9Mray5ZCLIBETe68i4ePznCVuGZFfRO1Z+v0YJl1440yV L5bEoES9ORXWgfM3H4tdY4MDyOziSzR4ckfCCl9BdIl96OMk5A063H/VI+DUaBeKS2F0 jciS7TOUiS+s5NDf+wMd3BWnKjEsywHy1U92kwFSje7eECD8towLbM49htFI7i0faJlc bROw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=feedback-id:content-transfer-encoding:mime-version:message-id:date :subject:cc:to:from:dkim-signature; bh=CRGr1P1DYNHw0s6xBVDq8rro+GasTxCCzHfJfRC9jaI=; b=UcWtlFBTQl7H6MzAoImOkfj4Ffb0EoDlDGJFfnfFZosIermwrVqDvV5Q3+D12NwwVK kLt7pTvNGffiDnpfFvbb7RJW9dGbrAvT0NTFWUcMeRFKYGKcW2cVI7xvHgmhZw2Gvixm c1wkt99ZaUC330hG0k4yReFqfzjMsS949RYTw6njHWdKfS/cnWUuphltSu4qB8dphIsw x7ahx6ISrGmkfsakgT4kLvoaAVCOqv1ng/BwwrP9vp93ko3dFmIEP17Lv6vF8evjTcI+ qWNGXNoYBIQiLy9XZfDAXGbvOJwiBPnkily9mBJz2ujW2q5WTNrczcs/lRdIPa50cZCM SuXA== ARC-Authentication-Results: i=1; gmr-mx.google.com; dkim=pass header.i=@siemens.com header.s=fm1 header.b=PiKQhi61; spf=pass (google.com: domain of fm-72506-20230425101226e14462ed4c50a9fb6e-8cp8og@rts-flowmailer.siemens.com designates 185.136.64.227 as permitted sender) smtp.mailfrom=fm-72506-20230425101226e14462ed4c50a9fb6e-8cP8oG@rts-flowmailer.siemens.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=siemens.com Received: from mta-64-227.siemens.flowmailer.net (mta-64-227.siemens.flowmailer.net. [185.136.64.227]) by gmr-mx.google.com with ESMTPS id d29-20020a0565123d1d00b004dd84067a4asi861022lfv.4.2023.04.25.03.12.26 for (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 25 Apr 2023 03:12:26 -0700 (PDT) Received-SPF: pass (google.com: domain of fm-72506-20230425101226e14462ed4c50a9fb6e-8cp8og@rts-flowmailer.siemens.com designates 185.136.64.227 as permitted sender) client-ip=185.136.64.227; Received: by mta-64-227.siemens.flowmailer.net with ESMTPSA id 20230425101226e14462ed4c50a9fb6e for ; Tue, 25 Apr 2023 12:12:26 +0200 X-Patchwork-Original-From: "'Felix Moessbauer' via isar-users" From: "MOESSBAUER, Felix" To: isar-users@googlegroups.com Cc: michael.adler@siemens.com, Felix Moessbauer Subject: [PATCH 1/1] make compressed images better syncable Date: Tue, 25 Apr 2023 10:12:16 +0000 Message-Id: <20230425101216.1693998-1-felix.moessbauer@siemens.com> MIME-Version: 1.0 X-Flowmailer-Platform: Siemens Feedback-ID: 519:519-72506:519-21489:flowmailer X-Original-Sender: felix.moessbauer@siemens.com X-Original-Authentication-Results: gmr-mx.google.com; dkim=pass header.i=@siemens.com header.s=fm1 header.b=PiKQhi61; spf=pass (google.com: domain of fm-72506-20230425101226e14462ed4c50a9fb6e-8cp8og@rts-flowmailer.siemens.com designates 185.136.64.227 as permitted sender) smtp.mailfrom=fm-72506-20230425101226e14462ed4c50a9fb6e-8cP8oG@rts-flowmailer.siemens.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=siemens.com X-Original-From: Felix Moessbauer Reply-To: Felix Moessbauer Precedence: list Mailing-list: list isar-users@googlegroups.com; contact isar-users+owners@googlegroups.com List-ID: X-Spam-Checked-In-Group: isar-users@googlegroups.com X-Google-Group-Id: 914930254986 List-Post: , List-Help: , List-Archive: , List-Unsubscribe: , X-Spam-Status: No, score=-1.2 required=5.0 tests=DKIMWL_WL_MED,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,MAILING_LIST_MULTI, RCVD_IN_MSPIKE_H2,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on shymkent.ilbers.de X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= For compression formats that support these options (currently only zstd), we enable to include the sparseness information into the archive. By that, disk IO on decompression of large zero blocks is speedup and storage usage is decreased. Usually, this is handled automatically by the compression tools, but not when sending output to stdout. In addition, we add the rsyncable flag to reduce the number of changed blocks in the output when only changing some input blocks. Signed-off-by: Felix Moessbauer --- meta/classes/imagetypes.bbclass | 2 +- meta/conf/bitbake.conf | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/meta/classes/imagetypes.bbclass b/meta/classes/imagetypes.bbclass index 2ee3240b..a80a6da5 100644 --- a/meta/classes/imagetypes.bbclass +++ b/meta/classes/imagetypes.bbclass @@ -101,5 +101,5 @@ CONVERSION_DEPS:gz = "gzip" CONVERSION_CMD:xz = "${SUDO_CHROOT} sh -c 'xz -c ${XZ_DEFAULTS} ${IMAGE_FILE_CHROOT} > ${IMAGE_FILE_CHROOT}.xz'" CONVERSION_DEPS:xz = "xz-utils" -CONVERSION_CMD:zst = "${SUDO_CHROOT} sh -c 'zstd -c ${ZSTD_DEFAULTS} ${IMAGE_FILE_CHROOT} > ${IMAGE_FILE_CHROOT}.zst'" +CONVERSION_CMD:zst = "${SUDO_CHROOT} sh -c 'zstd -c --sparse ${ZSTD_DEFAULTS} ${IMAGE_FILE_CHROOT} > ${IMAGE_FILE_CHROOT}.zst'" CONVERSION_DEPS:zst = "zstd" diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf index f421050e..c660660a 100644 --- a/meta/conf/bitbake.conf +++ b/meta/conf/bitbake.conf @@ -141,7 +141,7 @@ XZ_DEFAULTS[vardepsexclude] += "XZ_MEMLIMIT XZ_THREADS" ZSTD_THREADS ?= "${@oe.utils.cpu_count(at_least=2)}" ZSTD_THREADS[vardepvalue] = "1" ZSTD_LEVEL ?= "19" -ZSTD_DEFAULTS ?= "-${ZSTD_LEVEL} --threads=${ZSTD_THREADS}" +ZSTD_DEFAULTS ?= "--rsyncable -${ZSTD_LEVEL} --threads=${ZSTD_THREADS}" ZSTD_DEFAULTS[vardepsexclude] += "ZSTD_LEVEL ZSTD_THREADS" BBINCLUDELOGS ??= "yes"