From patchwork Mon Apr 22 18:58:42 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "MOESSBAUER, Felix" X-Patchwork-Id: 3502 Return-Path: Received: from shymkent.ilbers.de ([unix socket]) by shymkent (Cyrus 2.5.10-Debian-2.5.10-3+deb9u2) with LMTPA; Mon, 22 Apr 2024 20:59:14 +0200 X-Sieve: CMU Sieve 2.4 Received: from mail-wm1-f64.google.com (mail-wm1-f64.google.com [209.85.128.64]) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPS id 43MIxDg5021593 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Mon, 22 Apr 2024 20:59:13 +0200 Received: by mail-wm1-f64.google.com with SMTP id 5b1f17b1804b1-4167351545csf37426375e9.1 for ; Mon, 22 Apr 2024 11:59:13 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1713812348; cv=pass; d=google.com; s=arc-20160816; b=lVtuHTrPni3ZT/xHrgjXRLIxVITzzfJ915J7wH3wt04xacHBjOLoGtXxNAZY8XIcas izH11tHg19gwfKU0UDNEh244PQN7S55yaOu2vY8Fso89J0yahuTAvchi9cwSCPyp0b+M zV+he2mqYFP46g+hzmXZVTxMTlPkfqHKekP3mZsux2vWIZHyRh6/vzlp5YfA70nioBLs k/ed+Wa35ddlVNRfLiM2kNv1W8EG6lIHxSiRh7JNu/omLG7mdXKLtPMRoE10B2la4ILS IHz5TiY1+y/R4P9+iMyZ9P5DQKcvJiYpQ2glGaCNNnOXxn6zMAxcmtsMlGuglRw37HcL fBYg== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:reply-to:feedback-id:mime-version :message-id:date:subject:cc:to:from:dkim-signature; bh=qOun//jD0Zf3ILABiTGKzEiYttp7Wf2DlkZOyjPcZH8=; fh=gsgUU5QY0gYE+BnDGTEL9QF228qh3E0y8gNpVF24ILc=; b=vy0p6juEqBlLkJXb5IVUTmA2AWOG9AYAkkmOhaeD3WYS5tpcDriCDtAI3J//rhcPMa WpML0PDyx0VMTpEZsNL0/JaTb/FHGKr4oFbzSKXnHG1jF5ADo3VUhFq0sfoj2dVQGklr d91ndrta/lBu+VmVEXXNJKDUhWWgpk8v8d5RxQ4xk0A2km+VmO+JsbfBz3wEBYe05Cnt qB4ikk1+xgB34C4M7zSBQ+rkxd9B5gLPXtBeLRJDz6azNrkUttBolTBs4CgyeklSJAO3 +HxR3XrPdNNpVhYq0rphRb/qt5Q/cq1QQ78K3lvlGyYJhU3RR8SPsKP2KdyTbypCc94B R1ug==; darn=isar-build.org ARC-Authentication-Results: i=2; gmr-mx.google.com; dkim=pass header.i=@siemens.com header.s=fm1 header.b=isFIjRfC; spf=pass (google.com: domain of fm-1321639-20240422185904e726bf16374199206b-key6ws@rts-flowmailer.siemens.com designates 185.136.64.227 as permitted sender) smtp.mailfrom=fm-1321639-20240422185904e726bf16374199206b-Key6ws@rts-flowmailer.siemens.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=siemens.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlegroups.com; s=20230601; t=1713812348; x=1714417148; darn=isar-build.org; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:reply-to :x-original-authentication-results:x-original-sender:feedback-id :mime-version:message-id:date:subject:cc:to:from:from:to:cc:subject :date:message-id:reply-to; bh=qOun//jD0Zf3ILABiTGKzEiYttp7Wf2DlkZOyjPcZH8=; b=M2lN958vu4j0vlqdcHcy6tdHCothPqjqCTdOpHJF85FOCGzIBd0lXDZE6/pHT6Ja6+ ZFeFlnahWE9p1cY18HX9JzzRYiRaT+tRI0wTFFWfXuOr1I7M7ACUkIuIL4nSmtV5m5Hl 8v9nC5+OUvixFNWTUQyIV1aFLpi7KaE+iGgXkEqyWpbQNACSg/hGdrqhJ0FgJGlInsXx wcH9TUlVdtgNCBOZ8OEvfZc2tCO8SHQak68hqgs+JDA19TiMe9nmeV+4j+7v2r6dc4Sd N0g8lgcYcOXUXNY82bgdZkF8U327O8nQdmPOcUFtsWSKghsyaV1z91jJHqKV3Cf89+qK ds3g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1713812348; x=1714417148; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :x-spam-checked-in-group:list-id:mailing-list:precedence:reply-to :x-original-authentication-results:x-original-sender:feedback-id :mime-version:message-id:date:subject:cc:to:from:x-beenthere :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=qOun//jD0Zf3ILABiTGKzEiYttp7Wf2DlkZOyjPcZH8=; b=PmlBIXmwd+GYy9wWsU/IcOHfYG2LbFyPxErEjSMjy6+420IVnpELlINnvVm1FNwaAf rBGEYEh2X1QTKVUd3RJZ562Fuz34nSaeGuEQbsNYbgoICCa+h9xYv8nvEFx0iKR5igza ch7H1vyB1+yozH+ZPGhcbMGYZ9kZxYzSI65t4imtqszvkA6WQkG1hnz0pgeUtPNGrR7U Mwjt60PQvjT9azsEuMvvxIJznUx6vC88pgzNI3GlFyDNzKf5pKKBwLc6qgRgaBUJf/el 5/GOHdB4SbaPrIp1zohFWwtjt0On4q9vOrJeHgDgzJYG5FBiferAA7r6kO6qI6XFLNUU 460g== X-Forwarded-Encrypted: i=2; AJvYcCWkSWnEQLFyGtM1t4G1t+S0mMGuVw8Ys2yKMfaa52y5dIq3bo2oWora7dhcOGLLAjZs8lyygKV+JAU8sS7WWVbg0pPcmP4= X-Gm-Message-State: AOJu0YwUD7fxPcrqLptetXqSLPuuKBBpe6fv/CMddBFBkFY/RutidkFm b88u0//fi14YzVFUf6N1puMo/1lYBlp7JNAs6VumiWDluG7Bh8W8 X-Google-Smtp-Source: AGHT+IH6Fm/9U6tLA6YKR1I2uIQgpL1o6Gsj/zYoVowzcMsPwc+mwC51Kfibsj123yiCt8wtHHtPrw== X-Received: by 2002:a05:600c:358e:b0:418:9dd2:fd04 with SMTP id p14-20020a05600c358e00b004189dd2fd04mr10289222wmq.13.1713812347354; Mon, 22 Apr 2024 11:59:07 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a05:600c:4ab0:b0:41a:3868:d22f with SMTP id 5b1f17b1804b1-41a3868d4abls7916065e9.0.-pod-prod-05-eu; Mon, 22 Apr 2024 11:59:05 -0700 (PDT) X-Received: by 2002:a05:600c:470e:b0:418:4aac:a576 with SMTP id v14-20020a05600c470e00b004184aaca576mr9478603wmo.39.1713812345073; Mon, 22 Apr 2024 11:59:05 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1713812345; cv=none; d=google.com; s=arc-20160816; b=Pfxwkpnp4FKT9nzlEWgfyTeIUeqK2I0VVen6BwNQ9jv53b7kNpdtoDW5mf1eOI/fKU Wz/P8Qo+vBOddyKcSlUEuxgwxKQaEHLrlAxF6hsWEI2VX4CSv0FU2Di3jGVfxIYfHX3/ cy5ttFAxDVFKqTxEja2olXTZFvmmLN87vZ6pIQ/m7xAuHog/Dk1nfzMiMIvQW6UP9O74 Woryu3oB4XEoBAJSMV9pgtLOOd2yLxzBjTUzpU1AkglYbakmJTFm/v9Ln56rQxIffHwq MxWsifpnLcdAOKnsvb5192VqljMUMQrgionZtjXDd4/gdtLi1i1/Q66EFl6ufcCOW+la p5HQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=feedback-id:content-transfer-encoding:mime-version:message-id:date :subject:cc:to:from:dkim-signature; bh=OlWP/1Xo5gMN7q4w8nyLn+BRc0CPlrSSICqGVuRR7sI=; fh=Lx2rbPd59kjPGSEMqxhNf/sNnSLG/zARlEzKSFlpdzg=; b=p92N/478sby3RNJmEaxjraBcJ4GRwURJlN5553vcCj5Rfd0pm1B4mdrWtw2BQyBtE2 Ugv1Yg2fVSCQ7OkD68rYgH8itRXe4zdWG7n7cZhFfbU65K9a9lX3HzUmk34rRT8bDp8E GXxLn/IV17w9eLLiv+DYaQEku1li8mhBf60QbDLkSGLdHfYLadRRJ0dtY+hOjKCvxBc6 F8FSubRKr9BdIwjQs7aDi++RxiN74D4+C3gpvY2LfbM0BNKGqRtt6oyUOIuYMQM53mss E+Fm4G4ylwc6fwwTtQqazmdhiH8IAj6UiL56xD+WJdbakYhrZK7iBv2q5Ddfge+Xy38B Z57Q==; dara=google.com ARC-Authentication-Results: i=1; gmr-mx.google.com; dkim=pass header.i=@siemens.com header.s=fm1 header.b=isFIjRfC; spf=pass (google.com: domain of fm-1321639-20240422185904e726bf16374199206b-key6ws@rts-flowmailer.siemens.com designates 185.136.64.227 as permitted sender) smtp.mailfrom=fm-1321639-20240422185904e726bf16374199206b-Key6ws@rts-flowmailer.siemens.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=siemens.com Received: from mta-64-227.siemens.flowmailer.net (mta-64-227.siemens.flowmailer.net. [185.136.64.227]) by gmr-mx.google.com with ESMTPS id q6-20020a05600c46c600b00418318566adsi345038wmo.2.2024.04.22.11.59.05 for (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Mon, 22 Apr 2024 11:59:05 -0700 (PDT) Received-SPF: pass (google.com: domain of fm-1321639-20240422185904e726bf16374199206b-key6ws@rts-flowmailer.siemens.com designates 185.136.64.227 as permitted sender) client-ip=185.136.64.227; Received: by mta-64-227.siemens.flowmailer.net with ESMTPSA id 20240422185904e726bf16374199206b for ; Mon, 22 Apr 2024 20:59:04 +0200 X-Patchwork-Original-From: "'Felix Moessbauer' via isar-users" From: "MOESSBAUER, Felix" To: isar-users@googlegroups.com Cc: venkata.pyla@toshiba-tsip.com, quirin.gylstorff@siemens.com, Felix Moessbauer Subject: [PATCH 1/2] remove conditional SDE paths Date: Mon, 22 Apr 2024 20:58:42 +0200 Message-Id: <20240422185843.550406-1-felix.moessbauer@siemens.com> MIME-Version: 1.0 X-Flowmailer-Platform: Siemens Feedback-ID: 519:519-1321639:519-21489:flowmailer X-Original-Sender: felix.moessbauer@siemens.com X-Original-Authentication-Results: gmr-mx.google.com; dkim=pass header.i=@siemens.com header.s=fm1 header.b=isFIjRfC; spf=pass (google.com: domain of fm-1321639-20240422185904e726bf16374199206b-key6ws@rts-flowmailer.siemens.com designates 185.136.64.227 as permitted sender) smtp.mailfrom=fm-1321639-20240422185904e726bf16374199206b-Key6ws@rts-flowmailer.siemens.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=siemens.com X-Original-From: Felix Moessbauer Reply-To: Felix Moessbauer Precedence: list Mailing-list: list isar-users@googlegroups.com; contact isar-users+owners@googlegroups.com List-ID: X-Spam-Checked-In-Group: isar-users@googlegroups.com X-Google-Group-Id: 914930254986 List-Post: , List-Help: , List-Archive: , List-Unsubscribe: , X-Spam-Status: No, score=-1.2 required=5.0 tests=DKIMWL_WL_MED,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,MAILING_LIST_MULTI, RCVD_IN_MSPIKE_H2,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on shymkent.ilbers.de X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= The SOURCE_DATE_EPOCH variable is now always available, both as bitbake variable, as well as as env-var. By that, we can remove all conditional paths that checked if the variable is available. Also, manual exports of the bitbake variable into the env is no longer required. Signed-off-by: Felix Moessbauer --- meta/classes/dpkg-source.bbclass | 3 --- meta/classes/image-account-extension.bbclass | 13 +++------ meta/classes/image.bbclass | 27 +++++-------------- meta/classes/imagetypes_wic.bbclass | 3 --- meta/classes/initramfs.bbclass | 5 ---- meta/classes/rootfs-add-files.bbclass | 4 --- .../isar-bootstrap/isar-bootstrap.inc | 4 --- 7 files changed, 10 insertions(+), 49 deletions(-) diff --git a/meta/classes/dpkg-source.bbclass b/meta/classes/dpkg-source.bbclass index 7e3868f6..7fd5d2ed 100644 --- a/meta/classes/dpkg-source.bbclass +++ b/meta/classes/dpkg-source.bbclass @@ -11,9 +11,6 @@ do_dpkg_source() { # Create a .dsc file from source directory to use it with sbuild DEB_SOURCE_NAME=$(dpkg-parsechangelog --show-field Source --file ${WORKDIR}/${PPS}/debian/changelog) find ${WORKDIR} -name "${DEB_SOURCE_NAME}*.dsc" -maxdepth 1 -delete - if [ ! -z "${SOURCE_DATE_EPOCH}" ]; then - export SOURCE_DATE_EPOCH="${SOURCE_DATE_EPOCH}" - fi sh -c "cd ${WORKDIR}; dpkg-source ${DPKG_SOURCE_EXTRA_ARGS} -b ${PPS}" } addtask dpkg_source after do_prepare_build before do_dpkg_build diff --git a/meta/classes/image-account-extension.bbclass b/meta/classes/image-account-extension.bbclass index 3d4e1d1d..b63fff5c 100644 --- a/meta/classes/image-account-extension.bbclass +++ b/meta/classes/image-account-extension.bbclass @@ -114,11 +114,10 @@ def image_create_users(d: "DataSmart") -> None: # chpasswd adds a random salt when running against a clear-text password. # For reproducible images, we manually generate the password and use the # SOURCE_DATE_EPOCH to generate the salt in a deterministic way. - source_date_epoch = d.getVar("SOURCE_DATE_EPOCH") or "" - if source_date_epoch: - command.append("-e") - salt = hashlib.sha256("{}\n".format(source_date_epoch).encode()).hexdigest()[0:15] - password = bb.process.run('openssl passwd -6 --salt {} {}'.format(salt, password))[0].strip() + source_date_epoch = d.getVar("SOURCE_DATE_EPOCH") + command.append("-e") + salt = hashlib.sha256("{}\n".format(source_date_epoch).encode()).hexdigest()[0:15] + password = bb.process.run('openssl passwd -6 --salt {} {}'.format(salt, password))[0].strip() else: command.append("-e") @@ -131,10 +130,6 @@ def image_create_users(d: "DataSmart") -> None: ROOTFS_POSTPROCESS_COMMAND += "image_postprocess_accounts" python image_postprocess_accounts() { - import os - if d.getVar("SOURCE_DATE_EPOCH") != None: - os.environ["SOURCE_DATE_EPOCH"] = d.getVar("SOURCE_DATE_EPOCH") - image_create_groups(d) image_create_users(d) } diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass index 98741da0..34faa6af 100644 --- a/meta/classes/image.bbclass +++ b/meta/classes/image.bbclass @@ -289,18 +289,6 @@ python() { d.appendVar('IMAGER_BUILD_DEPS', ' ' + ' '.join(sorted(imager_build_deps))) } - -# make generation of initramfs reproducible -# note: this function is shared across multiple rootfs, but we only want to make the -# image rootfs reproducible. Otherwise changes of SOURCE_DATE_EPOCH would -# invalidate the SSTATE entries for most packages, even if they don't use the -# global SOURCE_DATE_EPOCH variable. -rootfs_install_pkgs_install:prepend() { - if [ ! -z "${SOURCE_DATE_EPOCH}" ]; then - export SOURCE_DATE_EPOCH="${SOURCE_DATE_EPOCH}" - fi -} - # here we call a command that should describe your whole build system, # this could be "git describe" or something similar. # set ISAR_RELEASE_CMD to customize, or override do_mark_rootfs to do something @@ -451,17 +439,14 @@ EOSUDO # Set same time-stamps to the newly generated file/folders in the # rootfs image for the purpose of reproducible builds. - if [ -n "${SOURCE_DATE_EPOCH}" ]; then - fn="${DEPLOY_DIR_IMAGE}/files.modified_timestamps" - if sudo find ${ROOTFSDIR} -newermt "$(date -d@${SOURCE_DATE_EPOCH} '+%Y-%m-%d %H:%M:%S')" \ - -printf "%y %p\n" -exec touch '{}' -h -d@${SOURCE_DATE_EPOCH} ';' | egrep ^f >"$fn"; then - if [ -e "$fn" ]; then - bbwarn "modified timestamp (${SOURCE_DATE_EPOCH}) of $(cat "$fn" | wc -l) files for image reproducibly." \ - "List of files modified can be found in: .${DEPLOY_DIR_IMAGE}/files.modified_timestamps" - fi + fn="${DEPLOY_DIR_IMAGE}/files.modified_timestamps" + if sudo find ${ROOTFSDIR} -newermt "$(date -d@${SOURCE_DATE_EPOCH} '+%Y-%m-%d %H:%M:%S')" \ + -printf "%y %p\n" -exec touch '{}' -h -d@${SOURCE_DATE_EPOCH} ';' | egrep ^f >"$fn"; then + if [ -e "$fn" ]; then + bbwarn "modified timestamp (${SOURCE_DATE_EPOCH}) of $(cat "$fn" | wc -l) files for image reproducibly." \ + "List of files modified can be found in: .${DEPLOY_DIR_IMAGE}/files.modified_timestamps" fi fi - } do_rootfs_finalize[network] = "${TASK_USE_SUDO}" addtask rootfs_finalize before do_rootfs after do_rootfs_postprocess diff --git a/meta/classes/imagetypes_wic.bbclass b/meta/classes/imagetypes_wic.bbclass index bce881ed..3b697cdd 100644 --- a/meta/classes/imagetypes_wic.bbclass +++ b/meta/classes/imagetypes_wic.bbclass @@ -157,9 +157,6 @@ generate_wic_image() { export FAKEROOTCMD=${FAKEROOTCMD} export BUILDDIR=${TOPDIR} export MTOOLS_SKIP_CHECK=1 - if [ ! -z "${SOURCE_DATE_EPOCH}" ]; then - export SOURCE_DATE_EPOCH="${SOURCE_DATE_EPOCH}" - fi mkdir -p ${IMAGE_ROOTFS}/../pseudo touch ${IMAGE_ROOTFS}/../pseudo/files.db diff --git a/meta/classes/initramfs.bbclass b/meta/classes/initramfs.bbclass index 925c3f6d..5b5943f2 100644 --- a/meta/classes/initramfs.bbclass +++ b/meta/classes/initramfs.bbclass @@ -38,11 +38,6 @@ do_generate_initramfs() { rootfs_do_mounts rootfs_do_qemu - # generate reproducible initrd if requested - if [ ! -z "${SOURCE_DATE_EPOCH}" ]; then - export SOURCE_DATE_EPOCH="${SOURCE_DATE_EPOCH}" - fi - sudo -E chroot "${INITRAMFS_ROOTFS}" sh -c '\ export kernel_version=$(basename /boot/vmlinu[xz]* | cut -d'-' -f2-); \ if [ -n "$kernel_version" ]; then \ diff --git a/meta/classes/rootfs-add-files.bbclass b/meta/classes/rootfs-add-files.bbclass index 4125caf5..8ebd25cb 100644 --- a/meta/classes/rootfs-add-files.bbclass +++ b/meta/classes/rootfs-add-files.bbclass @@ -16,10 +16,6 @@ ROOTFS_ADDITIONAL_FILES ??= "" python rootfs_add_files() { - import os - if d.getVar("SOURCE_DATE_EPOCH") != None: - os.environ["SOURCE_DATE_EPOCH"] = d.getVar("SOURCE_DATE_EPOCH") - postprocess_additional_files = d.getVar('ROOTFS_ADDITIONAL_FILES').split() rootfsdir = d.getVar("ROOTFSDIR") diff --git a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc index b1567d7b..faf22a50 100644 --- a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc +++ b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc @@ -310,10 +310,6 @@ do_bootstrap() { sudo rm -rf --one-file-system "${ROOTFSDIR}" deb_dl_dir_import "${ROOTFSDIR}" "${BOOTSTRAP_BASE_DISTRO}-${BASE_DISTRO_CODENAME}" - if [ -n "${SOURCE_DATE_EPOCH}" ]; then - export SOURCE_DATE_EPOCH="${SOURCE_DATE_EPOCH}" - fi - sudo -E -s <<'EOSUDO' set -e if [ "${BOOTSTRAP_FOR_HOST}" = "0" ]; then