From patchwork Tue Apr 9 15:55:45 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "MOESSBAUER, Felix" X-Patchwork-Id: 3487 Return-Path: Received: from shymkent.ilbers.de ([unix socket]) by shymkent (Cyrus 2.5.10-Debian-2.5.10-3+deb9u2) with LMTPA; Tue, 09 Apr 2024 17:56:13 +0200 X-Sieve: CMU Sieve 2.4 Received: from mail-oo1-f59.google.com (mail-oo1-f59.google.com [209.85.161.59]) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPS id 439FuAX8022950 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Tue, 9 Apr 2024 17:56:11 +0200 Received: by mail-oo1-f59.google.com with SMTP id 006d021491bc7-5aa350b446dsf2169065eaf.1 for ; Tue, 09 Apr 2024 08:56:11 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1712678165; cv=pass; d=google.com; s=arc-20160816; b=Xb7Uy5oKQNuTCRpGPb59dh5543qm76s4MY5IFVQaI9PUy5whSTQ4IT0GIVQH2sIjMz NSSMIv+2vGi3/4//piZTuHLtwjEpevrnuBjYGjM3yaDvl6Ups9wOMN8T+epC7pJwll8I WgrRy3pDTinvQVX0q+svS18VX50YJaqU9O4S8PuuafT058vOFtAAdMMpI8h1YG1+LjhV OHL/NpnzxuXWsh+9PnpcNjFCiCOeKCUrrUTCJP7aZY/ZE3pSEKZ7QySnEo6n5ZzBrbNO 9HMhGX7cPPdWMnjdYxxoX/eg1MoeR5bwredhivaIQdC5/TgVl+JTNOBz/DqpAgGscsRp t4cA== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:reply-to:feedback-id:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=+kb96aBNNkdwKYgr3Gbzm8JQA7VdNHXiHky32hUQfs8=; fh=uN+nBdDv8loP8Kq1bGiikLxU1oLmj0Musk3jkxAREGw=; b=TxFy/2KpudXE1KUtKiVBtVXHfgxZHBtWlTC+33wqpzfwSoA5kWrmWnApnJ0CSXxorW vEbxmYK4NCm8u6tv3qRMo6ZX4Yvu3PUGQ7Q+L4nHWq5TCO/0KsieWtWAfcahAtAxVreE v9wLAtmgEGnhkpfwbbhCGlBZHGJGDoPKJmJVfSglzhVwflZ89G0Hv7G8MRLCTl/vSXOB 1Fd9kpztbm3XtJsr0iL0REjzPU0v7hDHQwnQ3CuXdbTSF0xf0USYEIATwPHeKRNQiG9d FixRnhBmGKm2sTkF9k9fPlLGFTlX0A/z/lMevJYJbaaqjzjRp25ztmv+XNFVmzNADt6m YmwQ==; darn=isar-build.org ARC-Authentication-Results: i=2; gmr-mx.google.com; dkim=pass header.i=@siemens.com header.s=fm1 header.b=o5bvaMdV; spf=pass (google.com: domain of fm-1321639-20240409155601d80ed3c77cbb2655c6-mjy35y@rts-flowmailer.siemens.com designates 185.136.65.227 as permitted sender) smtp.mailfrom=fm-1321639-20240409155601d80ed3c77cbb2655c6-mJy35y@rts-flowmailer.siemens.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=siemens.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlegroups.com; s=20230601; t=1712678165; x=1713282965; darn=isar-build.org; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:reply-to :x-original-authentication-results:x-original-sender:feedback-id :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:from:to:cc:subject:date:message-id:reply-to; bh=+kb96aBNNkdwKYgr3Gbzm8JQA7VdNHXiHky32hUQfs8=; b=jyRU23vKWavhGLyldM6SW17nCGpkq83LmWeDXmlSMpCjaaZC9MrNeYV/dzvaccUAak vQA0yVfulLY53MCA8XJMFjGFEC92MI9AUyx4x2uln+PlSu1HesSjV76fvITy6VgL2m88 8KAGcqNoDGJDDaMDaL7sejMb7dpmksHEY8rKg8MH7qzJmMi05csOL2BGSY5aaBOVwEKa ZqkTJR69vziMYhu0NlUgJeQ3PPzPijA4kA1eKw0gGctHCcfw3vsZf/njQtRdl+SKtXhK j2Cau2Rev0RStqhIrmA0I6KOBLxWfUIVGVr2XcRRr/udJ+YQkKRZxgoTVnVxcU+uJ8SN M9ZQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1712678165; x=1713282965; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :x-spam-checked-in-group:list-id:mailing-list:precedence:reply-to :x-original-authentication-results:x-original-sender:feedback-id :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:x-beenthere:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=+kb96aBNNkdwKYgr3Gbzm8JQA7VdNHXiHky32hUQfs8=; b=IZgqewW8mCgxI5ad0i6NieUurZF9Y71kvqlLW8JUQsZkZPt0egJOLd5bHB/2jjfDGr YkJiV1gqn57p4vebJmq9tkUu3hmwSMwX3TqsmZ2Ad7oaWu3DrXd/7njDUK81U/RjUCNU LgrkS4s4bhLZp1OAa5CB04u7rxgr/vKUUOjJVKReTjMettnBFSHl0AylP/d6hrpPgnQl +dNkAwBS9/YFug5FXQxoa64PvvjqWzd4MCcyinc5KMc7KVAxBRuh8kAoTFh19fympPBt VYNFVQ7cuZzc5aBM7aDF9ulS7rz08wBVR8eOFaXc8njVjB5OEtdyDko9r74cp+IJsVzA lh8w== X-Forwarded-Encrypted: i=2; AJvYcCU5BVNJjAY+f09/o2mbjdYC4UazGU/PNRkwDOt79iOesIPcfcUtg4vGKlxdbF88Hahwg+lb70DVEjxIHMwmeSWBGNdKycA= X-Gm-Message-State: AOJu0YwgPJ4y+IyiIoCSHBbHSX60aOmNlDV7EwHjzIerAz4wFLM5WNd2 xnco5iFc9ay6UfERmLAS2ce+QzU1PmlOTMLAKd7kLXx8+vBVF5Us X-Google-Smtp-Source: AGHT+IGKGTbU0RkLbrkiqBISTXKhMyi1lQk9lxbv6slLDMTHHsC7eZgd55RPdCZj4viWYUnSBT+dEg== X-Received: by 2002:a05:6870:63a4:b0:229:929a:b127 with SMTP id t36-20020a05687063a400b00229929ab127mr14317437oap.37.1712678164922; Tue, 09 Apr 2024 08:56:04 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a05:6870:1648:b0:22f:1b54:b7ca with SMTP id c8-20020a056870164800b0022f1b54b7cals3048181oae.0.-pod-prod-07-us; Tue, 09 Apr 2024 08:56:03 -0700 (PDT) X-Received: by 2002:aca:2313:0:b0:3c5:e553:475a with SMTP id e19-20020aca2313000000b003c5e553475amr9001982oie.45.1712678163551; Tue, 09 Apr 2024 08:56:03 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1712678163; cv=none; d=google.com; s=arc-20160816; b=zBT3ChUirhmxXwAkViAAaLSLAsU2OYPqM/HMVPodwAx4y4W1FtuekmlwapsCRhckXH WBD6EMgFbGTEVLAwpDpnCrkaRzy115X78Ma6f1d+ErR5eELyyL+2g5Ia2H4w0BOxO/Bg PyUbaQJwDjWwCwy8dfD1RuPYtGGcHnLtqjJNGAPuzQfc3RTTIUjSFVZDnsL05pFn6Uc/ IHejJ4XpvbZKHaVlve+j7wIiuPrQrCGfe5so/wPealHINRR/NUPte9Et8EXqjGDcXnWg pEy173/RXMN7dmdWG0GCoQCJcwddrdHOiwS8oXoQZxw5M/8ybjTEIYXW/HvH/Z1LY+fJ xhiA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=feedback-id:content-transfer-encoding:mime-version:references :in-reply-to:message-id:date:subject:cc:to:from:dkim-signature; bh=x74ASns5mdLMMVRYE5JA6iV/n2XbnOrouqPaD7Qxc1g=; fh=WkhL8kaJc+l2wQon1t06Ej3uvBGj9sVhNcE8PaS/XbI=; b=ZJI2xSOYLupthXJc+rhcTTYXYQbawSROBeYcuDlkBMHhnxiT0IhA9diwjX9HqftoGB jCIdCsGa2xGz26SzxTJD7pf1YE3hyxkk0CCBuC3oAS7BpUIP5fPeCmHRMNzfmznxvuv6 00Oe08IjN2cgtXtY26p+bTSFb4tuypDIzc9wTdfUpad65UzcKCJPwWkoRUygJl8LRDz2 IEo9kBmUIiMEia9MCNFSdjAlBCxV057qeqAc3NUfjBIdwvWQPnPgGfnEvT8fXwaaZEkZ nVMlk4p+gY3EMysL9pMlALd9+1130CTR2igAZo4ojF9bBwInAbs+4+OKbVN9c5eYFr8L BbZw==; dara=google.com ARC-Authentication-Results: i=1; gmr-mx.google.com; dkim=pass header.i=@siemens.com header.s=fm1 header.b=o5bvaMdV; spf=pass (google.com: domain of fm-1321639-20240409155601d80ed3c77cbb2655c6-mjy35y@rts-flowmailer.siemens.com designates 185.136.65.227 as permitted sender) smtp.mailfrom=fm-1321639-20240409155601d80ed3c77cbb2655c6-mJy35y@rts-flowmailer.siemens.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=siemens.com Received: from mta-65-227.siemens.flowmailer.net (mta-65-227.siemens.flowmailer.net. [185.136.65.227]) by gmr-mx.google.com with ESMTPS id bf8-20020a056808190800b003c5f1bfdb6esi385091oib.0.2024.04.09.08.56.02 for (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 09 Apr 2024 08:56:03 -0700 (PDT) Received-SPF: pass (google.com: domain of fm-1321639-20240409155601d80ed3c77cbb2655c6-mjy35y@rts-flowmailer.siemens.com designates 185.136.65.227 as permitted sender) client-ip=185.136.65.227; Received: by mta-65-227.siemens.flowmailer.net with ESMTPSA id 20240409155601d80ed3c77cbb2655c6 for ; Tue, 09 Apr 2024 17:56:01 +0200 X-Patchwork-Original-From: "'Felix Moessbauer' via isar-users" From: "MOESSBAUER, Felix" To: isar-users@googlegroups.com Cc: jan.kiszka@siemens.com, quirin.gylstorff@siemens.com, Felix Moessbauer Subject: [PATCH v3 1/5] add reproducible builds infrastructure from oe Date: Tue, 9 Apr 2024 17:55:45 +0200 Message-Id: <20240409155549.826454-2-felix.moessbauer@siemens.com> In-Reply-To: <20240409155549.826454-1-felix.moessbauer@siemens.com> References: <20240409155549.826454-1-felix.moessbauer@siemens.com> MIME-Version: 1.0 X-Flowmailer-Platform: Siemens Feedback-ID: 519:519-1321639:519-21489:flowmailer X-Original-Sender: felix.moessbauer@siemens.com X-Original-Authentication-Results: gmr-mx.google.com; dkim=pass header.i=@siemens.com header.s=fm1 header.b=o5bvaMdV; spf=pass (google.com: domain of fm-1321639-20240409155601d80ed3c77cbb2655c6-mjy35y@rts-flowmailer.siemens.com designates 185.136.65.227 as permitted sender) smtp.mailfrom=fm-1321639-20240409155601d80ed3c77cbb2655c6-mJy35y@rts-flowmailer.siemens.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=siemens.com X-Original-From: Felix Moessbauer Reply-To: Felix Moessbauer Precedence: list Mailing-list: list isar-users@googlegroups.com; contact isar-users+owners@googlegroups.com List-ID: X-Spam-Checked-In-Group: isar-users@googlegroups.com X-Google-Group-Id: 914930254986 List-Post: , List-Help: , List-Archive: , List-Unsubscribe: , X-Spam-Status: No, score=-1.2 required=5.0 tests=DKIMWL_WL_MED,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,MAILING_LIST_MULTI, RCVD_IN_MSPIKE_H3,RCVD_IN_MSPIKE_WL,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on shymkent.ilbers.de X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= This patch adds the reproducible builds infrastructure from OE. This includes python helpers to determine the SOURCE_DATE_EPOCH per component (if not set). Once determined, these values are written to the SDE_FILE and picked up on the next build as a fallback. Both this and the reproducible section of the bitbake.conf are taken 1:1 from OE 6548354 (corresponding commit to bitbake version currently used by ISAR). Signed-off-by: Felix Moessbauer --- meta/classes/base.bbclass | 17 +++- meta/conf/bitbake.conf | 12 +++ meta/lib/oe/reproducible.py | 197 ++++++++++++++++++++++++++++++++++++ 3 files changed, 225 insertions(+), 1 deletion(-) create mode 100644 meta/lib/oe/reproducible.py diff --git a/meta/classes/base.bbclass b/meta/classes/base.bbclass index 529811af..b8825bd3 100644 --- a/meta/classes/base.bbclass +++ b/meta/classes/base.bbclass @@ -21,7 +21,7 @@ THISDIR = "${@os.path.dirname(d.getVar('FILE'))}" FILESPATH = "${@base_set_filespath(["${FILE_DIRNAME}/${PF}", "${FILE_DIRNAME}/${P}", "${FILE_DIRNAME}/${PN}", "${FILE_DIRNAME}/files", "${FILE_DIRNAME}"], d)}" -OE_IMPORTS += "os sys time oe.path oe.patch oe.sstatesig oe.utils" +OE_IMPORTS += "os sys time oe.path oe.patch oe.reproducible oe.sstatesig oe.utils" OE_IMPORTS[type] = "list" def oe_import(d): @@ -318,3 +318,18 @@ def calculate_build_uuid(d): # Unique ID for this build, used to avoid name clashes on external mountpoints # When running parallel builds in different PID namespaces ISAR_BUILD_UUID = "${@ calculate_build_uuid(d)}" + +do_deploy_source_date_epoch[dirs] = "${SDE_DEPLOYDIR}" +do_deploy_source_date_epoch[sstate-plaindirs] = "${SDE_DEPLOYDIR}" +addtask do_deploy_source_date_epoch_setscene +addtask do_deploy_source_date_epoch before do_configure after do_patch + +python create_source_date_epoch_stamp() { + # Version: 1 + source_date_epoch = oe.reproducible.get_source_date_epoch(d, d.getVar('S')) + oe.reproducible.epochfile_write(source_date_epoch, d.getVar('SDE_FILE'), d) +} +do_unpack[postfuncs] += "create_source_date_epoch_stamp" + +def get_source_date_epoch_value(d): + return oe.reproducible.epochfile_read(d.getVar('SDE_FILE'), d) diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf index 91c5c815..1da3ecac 100644 --- a/meta/conf/bitbake.conf +++ b/meta/conf/bitbake.conf @@ -134,6 +134,18 @@ BB_NUMBER_THREADS ?= "${@bb.utils.cpu_count()}" # Default to setting automatically based on cpu count PARALLEL_MAKE ?= "-j ${@bb.utils.cpu_count()}" +# Reproducibility (taken 1:1 from oe) +SDE_DIR = "${WORKDIR}/source-date-epoch" +SDE_FILE = "${SDE_DIR}/__source_date_epoch.txt" +SDE_DEPLOYDIR = "${WORKDIR}/deploy-source-date-epoch" + +export PYTHONHASHSEED = "0" +export PERL_HASH_SEED = "0" +export SOURCE_DATE_EPOCH ?= "${@get_source_date_epoch_value(d)}" +# A SOURCE_DATE_EPOCH of '0' might be misinterpreted as no SDE +# ISAR: set value to date of latest release +SOURCE_DATE_EPOCH_FALLBACK ??= "1709565251" + # Default parallelism and resource usage for xz XZ_MEMLIMIT ?= "50%" XZ_THREADS ?= "${@oe.utils.cpu_count(at_least=2)}" diff --git a/meta/lib/oe/reproducible.py b/meta/lib/oe/reproducible.py new file mode 100644 index 00000000..448befce --- /dev/null +++ b/meta/lib/oe/reproducible.py @@ -0,0 +1,197 @@ +# +# Copyright OpenEmbedded Contributors +# +# SPDX-License-Identifier: GPL-2.0-only +# +import os +import subprocess +import bb + +# For reproducible builds, this code sets the default SOURCE_DATE_EPOCH in each +# component's build environment. The format is number of seconds since the +# system epoch. +# +# Upstream components (generally) respect this environment variable, +# using it in place of the "current" date and time. +# See https://reproducible-builds.org/specs/source-date-epoch/ +# +# The default value of SOURCE_DATE_EPOCH comes from the function +# get_source_date_epoch_value which reads from the SDE_FILE, or if the file +# is not available will use the fallback of SOURCE_DATE_EPOCH_FALLBACK. +# +# The SDE_FILE is normally constructed from the function +# create_source_date_epoch_stamp which is typically added as a postfuncs to +# the do_unpack task. If a recipe does NOT have do_unpack, it should be added +# to a task that runs after the source is available and before the +# do_deploy_source_date_epoch task is executed. +# +# If a recipe wishes to override the default behavior it should set it's own +# SOURCE_DATE_EPOCH or override the do_deploy_source_date_epoch_stamp task +# with recipe-specific functionality to write the appropriate +# SOURCE_DATE_EPOCH into the SDE_FILE. +# +# SOURCE_DATE_EPOCH is intended to be a reproducible value. This value should +# be reproducible for anyone who builds the same revision from the same +# sources. +# +# There are 4 ways the create_source_date_epoch_stamp function determines what +# becomes SOURCE_DATE_EPOCH: +# +# 1. Use the value from __source_date_epoch.txt file if this file exists. +# This file was most likely created in the previous build by one of the +# following methods 2,3,4. +# Alternatively, it can be provided by a recipe via SRC_URI. +# +# If the file does not exist: +# +# 2. If there is a git checkout, use the last git commit timestamp. +# Git does not preserve file timestamps on checkout. +# +# 3. Use the mtime of "known" files such as NEWS, CHANGLELOG, ... +# This works for well-kept repositories distributed via tarball. +# +# 4. Use the modification time of the youngest file in the source tree, if +# there is one. +# This will be the newest file from the distribution tarball, if any. +# +# 5. Fall back to a fixed timestamp (SOURCE_DATE_EPOCH_FALLBACK). +# +# Once the value is determined, it is stored in the recipe's SDE_FILE. + +def get_source_date_epoch_from_known_files(d, sourcedir): + source_date_epoch = None + newest_file = None + known_files = set(["NEWS", "ChangeLog", "Changelog", "CHANGES"]) + for file in known_files: + filepath = os.path.join(sourcedir, file) + if os.path.isfile(filepath): + mtime = int(os.lstat(filepath).st_mtime) + # There may be more than one "known_file" present, if so, use the youngest one + if not source_date_epoch or mtime > source_date_epoch: + source_date_epoch = mtime + newest_file = filepath + if newest_file: + bb.debug(1, "SOURCE_DATE_EPOCH taken from: %s" % newest_file) + return source_date_epoch + +def find_git_folder(d, sourcedir): + # First guess: WORKDIR/git + # This is the default git fetcher unpack path + workdir = d.getVar('WORKDIR') + gitpath = os.path.join(workdir, "git/.git") + if os.path.isdir(gitpath): + return gitpath + + # Second guess: ${S} + gitpath = os.path.join(sourcedir, ".git") + if os.path.isdir(gitpath): + return gitpath + + # Perhaps there was a subpath or destsuffix specified. + # Go looking in the WORKDIR + exclude = set(["build", "image", "license-destdir", "patches", "pseudo", + "recipe-sysroot", "recipe-sysroot-native", "sysroot-destdir", "temp"]) + for root, dirs, files in os.walk(workdir, topdown=True): + dirs[:] = [d for d in dirs if d not in exclude] + if '.git' in dirs: + return os.path.join(root, ".git") + + bb.warn("Failed to find a git repository in WORKDIR: %s" % workdir) + return None + +def get_source_date_epoch_from_git(d, sourcedir): + if not "git://" in d.getVar('SRC_URI') and not "gitsm://" in d.getVar('SRC_URI'): + return None + + gitpath = find_git_folder(d, sourcedir) + if not gitpath: + return None + + # Check that the repository has a valid HEAD; it may not if subdir is used + # in SRC_URI + p = subprocess.run(['git', '--git-dir', gitpath, 'rev-parse', 'HEAD'], stdout=subprocess.PIPE, stderr=subprocess.STDOUT) + if p.returncode != 0: + bb.debug(1, "%s does not have a valid HEAD: %s" % (gitpath, p.stdout.decode('utf-8'))) + return None + + bb.debug(1, "git repository: %s" % gitpath) + p = subprocess.run(['git', '-c', 'log.showSignature=false', '--git-dir', gitpath, 'log', '-1', '--pretty=%ct'], + check=True, stdout=subprocess.PIPE) + return int(p.stdout.decode('utf-8')) + +def get_source_date_epoch_from_youngest_file(d, sourcedir): + if sourcedir == d.getVar('WORKDIR'): + # These sources are almost certainly not from a tarball + return None + + # Do it the hard way: check all files and find the youngest one... + source_date_epoch = None + newest_file = None + for root, dirs, files in os.walk(sourcedir, topdown=True): + files = [f for f in files if not f[0] == '.'] + + for fname in files: + if fname == "singletask.lock": + # Ignore externalsrc/devtool lockfile [YOCTO #14921] + continue + filename = os.path.join(root, fname) + try: + mtime = int(os.lstat(filename).st_mtime) + except ValueError: + mtime = 0 + if not source_date_epoch or mtime > source_date_epoch: + source_date_epoch = mtime + newest_file = filename + + if newest_file: + bb.debug(1, "Newest file found: %s" % newest_file) + return source_date_epoch + +def fixed_source_date_epoch(d): + bb.debug(1, "No tarball or git repo found to determine SOURCE_DATE_EPOCH") + source_date_epoch = d.getVar('SOURCE_DATE_EPOCH_FALLBACK') + if source_date_epoch: + bb.debug(1, "Using SOURCE_DATE_EPOCH_FALLBACK") + return int(source_date_epoch) + return 0 + +def get_source_date_epoch(d, sourcedir): + return ( + get_source_date_epoch_from_git(d, sourcedir) or + get_source_date_epoch_from_youngest_file(d, sourcedir) or + fixed_source_date_epoch(d) # Last resort + ) + +def epochfile_read(epochfile, d): + cached, efile = d.getVar('__CACHED_SOURCE_DATE_EPOCH') or (None, None) + if cached and efile == epochfile: + return cached + + if cached and epochfile != efile: + bb.debug(1, "Epoch file changed from %s to %s" % (efile, epochfile)) + + source_date_epoch = int(d.getVar('SOURCE_DATE_EPOCH_FALLBACK')) + try: + with open(epochfile, 'r') as f: + s = f.read() + try: + source_date_epoch = int(s) + except ValueError: + bb.warn("SOURCE_DATE_EPOCH value '%s' is invalid. Reverting to SOURCE_DATE_EPOCH_FALLBACK" % s) + source_date_epoch = int(d.getVar('SOURCE_DATE_EPOCH_FALLBACK')) + bb.debug(1, "SOURCE_DATE_EPOCH: %d" % source_date_epoch) + except FileNotFoundError: + bb.debug(1, "Cannot find %s. SOURCE_DATE_EPOCH will default to %d" % (epochfile, source_date_epoch)) + + d.setVar('__CACHED_SOURCE_DATE_EPOCH', (str(source_date_epoch), epochfile)) + return str(source_date_epoch) + +def epochfile_write(source_date_epoch, epochfile, d): + + bb.debug(1, "SOURCE_DATE_EPOCH: %d" % source_date_epoch) + bb.utils.mkdirhier(os.path.dirname(epochfile)) + + tmp_file = "%s.new" % epochfile + with open(tmp_file, 'w') as f: + f.write(str(source_date_epoch)) + os.rename(tmp_file, epochfile) From patchwork Tue Apr 9 15:55:46 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "MOESSBAUER, Felix" X-Patchwork-Id: 3488 Return-Path: Received: from shymkent.ilbers.de ([unix socket]) by shymkent (Cyrus 2.5.10-Debian-2.5.10-3+deb9u2) with LMTPA; Tue, 09 Apr 2024 17:56:13 +0200 X-Sieve: CMU Sieve 2.4 Received: from mail-pj1-f60.google.com (mail-pj1-f60.google.com [209.85.216.60]) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPS id 439FuBeH022954 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Tue, 9 Apr 2024 17:56:12 +0200 Received: by mail-pj1-f60.google.com with SMTP id 98e67ed59e1d1-2a2c2b0d82asf5199846a91.3 for ; Tue, 09 Apr 2024 08:56:12 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1712678165; cv=pass; d=google.com; s=arc-20160816; b=B+mbH3SXXcwoBJODbamMK66c8NbePOhe26QpGPAoT+oLaaLsQHassLxq7JNR3dkAT+ gQh0BTNvhixdV0PJ3KDCr2OpmleYmQPQmx9mD9wZ0cskmWzj5ECuW5iFdKOGQkjB7o9l OBlwkGOjmHe7euG2OJIpLS3yb4g3J033xeFNxtdZrlS4EUP4zHTy9ATFIaIMlr6DdKAT Va3Jr3GhxCmYHlYPEvvE8tN7gxTNPtb+90v6Ds1dkDXvxCaH0U4kyX5Wx024n7DxXjPy UdExkW/9KBlpw9D05VTTfIkgzG2jsOLb2pZURMUgawNDgCXLqO+5p952/jvdpCKucL5j zrHw== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:reply-to:feedback-id :content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature; bh=0IWWmF4q/vlKzzC6OiXmIj1nrKq9AqoRdWAEa32xC3o=; fh=z70l4yF9oGbrYh4Q4jHu+xSbANNdMEW34xb4IWwS1qk=; b=A9cILA62sjYThIS6d0VprIRMoMS8gZXEbcx0+e26xlvrpe7faNlf64WvKj6t7LlpVG WGcqj0URaEkBKIRwkuq4syCzTbpDq0Mp2M/m/ikp4l+oE4j05C06Z6ckY4hf+cX/Ehzy PfnZUBhFkUINO1ivYy74H9nspe/gSLWITRmv+om59eqqshY23iT7tZ34Zgbh6cdDROtx S4LC0CFDloZW496rCyN3/EghsdlSiHAm/ztthqaZR5mSoiXFa/KXOkmu9vp9NRXnwsZE rU4CahwigPhh3RFvU4IGsdck5a86Hm0+tSQ8AxaPFlrXHT2X4+GRMUce34VNkEesKkYT HFkQ==; darn=isar-build.org ARC-Authentication-Results: i=2; gmr-mx.google.com; dkim=pass header.i=@siemens.com header.s=fm1 header.b=J+cL1W+A; spf=pass (google.com: domain of fm-1321639-20240409155601abb2507ec2bc3d7b23-x_jmhs@rts-flowmailer.siemens.com designates 185.136.64.226 as permitted sender) smtp.mailfrom=fm-1321639-20240409155601abb2507ec2bc3d7b23-X_jMHs@rts-flowmailer.siemens.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=siemens.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlegroups.com; s=20230601; t=1712678165; x=1713282965; darn=isar-build.org; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:reply-to :x-original-authentication-results:x-original-sender:feedback-id :content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=0IWWmF4q/vlKzzC6OiXmIj1nrKq9AqoRdWAEa32xC3o=; b=nUHSccij2ZDVEPG5KYtczRyQ4dGDCCMGvZweCiM7uqf3GhniHKI6Ez79RZDzzTHzks 9P8QNGAnP1n5ccgV0W82FlDYfmtFYJ3tB5RE46nL/X7dT3l90aISXXGICKMjKDN8jDpt 1jLQXl0TkS7e1GVJG2gWncfhwjjrSlWJLv+DIiy3sapTAoxu+zUzcoUOAZpeFhmlR074 0LlsCcrYiRw9pfm1BD+Dpk/sEoAbZo3JWmg/j0rC6CZRG5GcAQIXAlp2g/8Mlpu9Tgd1 4tkeRvxYOTtpD/OFUhBpRgV8tB+wP+JpLYiRJSUzHzq97rjzxBihXBQJbHiy+FNqZ7Xf v45A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1712678165; x=1713282965; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :x-spam-checked-in-group:list-id:mailing-list:precedence:reply-to :x-original-authentication-results:x-original-sender:feedback-id :content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-beenthere:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=0IWWmF4q/vlKzzC6OiXmIj1nrKq9AqoRdWAEa32xC3o=; b=woaXghlw4dZM9XcDNLGWishGXgNP8YDPWKMILToD3qz1aG7VCHlaGP5UErTHF5Ih0c AHIG9CvmSp72Owt3JA2JPPVr1WCbtW0MBBKUuvZCTDp1kaHJn0ZImnYA+9x7UVA8zjFV uT5Dx9tnTfapCtSKUzLQh5RGwSfDmeanSQqUcI3OwhY1Bh5gfGB008tKv2g6UOW6Q6SB qV3LxnMksubCS/QWQ5WE1mJpVz8Ige0DnAkO1bJgoS4JQgQsf2wG3eAExDORmtARv4Gd sumk9sCbTypiH5Q5osZ4i7FeLdf2TflUpQlMEYHZJKKsXkhqXzxpnEFRSOezCgSq+mQh dhoQ== X-Forwarded-Encrypted: i=2; AJvYcCXvou8UChpbVxjSWqa8Hoe3Xq/wVRtF7gjxIuOM0Tp0O6sIXjBlkE9AXlvZgaeVvpmKaK9EXfaiYi9cz2UsBJxIjI6QDMQ= X-Gm-Message-State: AOJu0Yz0eKjQfVHekV1vIYTXkI0l552zAn2MGoUFX6tOpeYQH7oWGsVr C82J8iYwUU6cHta1K18jXXslWNMTPBHALS9rd/LWDu09VXuT58bR X-Google-Smtp-Source: AGHT+IHnDvN3CXo6Wn4r+TpuxGwyLq41sAR0SbmmLggPGM1T+DnXVdH0EGattOYIzAj9MCIZLKq4MA== X-Received: by 2002:a17:90a:c16:b0:2a3:be59:e969 with SMTP id 22-20020a17090a0c1600b002a3be59e969mr15152pjs.47.1712678165132; Tue, 09 Apr 2024 08:56:05 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a17:90a:9cc:b0:2a0:9806:220f with SMTP id 70-20020a17090a09cc00b002a09806220fls2938197pjo.0.-pod-prod-09-us; Tue, 09 Apr 2024 08:56:04 -0700 (PDT) X-Received: by 2002:a17:90a:d493:b0:2a3:4544:ea80 with SMTP id s19-20020a17090ad49300b002a34544ea80mr47123pju.5.1712678163735; Tue, 09 Apr 2024 08:56:03 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1712678163; cv=none; d=google.com; s=arc-20160816; b=BbFyWQgW9mAfoZl5WHgeuh30bsbvh1+O/a/HrUzgwstf3pV9nXI2Y2qIqgWpe5pVo9 yO2rNfDLD6QThQVC106oAXv2L84AuS7CO1qNGGxqbNUPondngbqWQgMccrTcsouWaD0/ 3Ib61PYJJyqFe88GrDueTQj6OWk8R2v21RwywIOGxeMjt6bRY7B79W0Ibukc1zg+sFaw UNoBnV/1QGAMoGzkhbbt9dH5t4DR6MZU9p1+sKQW9mEGo4dwLFjBQC+aAxgtDgmJPkC+ ffNzHY/1NHHDDk6ucvcst59XMgbGhQUzdbzLPlpyA4cxljul3PyrvuRjysrbt7JwYOI8 K+Eg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=feedback-id:content-transfer-encoding:mime-version:references :in-reply-to:message-id:date:subject:cc:to:from:dkim-signature; bh=QRZZOm4OI7CdXbKlI70a6zKsDBS65rwqhT4WNgY1gv0=; fh=WkhL8kaJc+l2wQon1t06Ej3uvBGj9sVhNcE8PaS/XbI=; b=p+CANZqzf8g9aqvbY5cjzLFXrZ3QZaOQqbMV/PS5YHkLUTdB2R6PqsICDIY3QFSVm6 Zho4/2XAfZhFPJuXBjXkWu36k3Axvp+qkmRWDZNi9Li8ZKzzRp4bVB2zCntd+4PwWk+A 5OnWtRMOdtciApuTI4FLZr8G2NpP9ENNotIYsg19+Cj4pnKd42878CEadzjGtrYvEi+m r00Xbtu/2e6qkmBGj1wWAX6b34LWKs3f5Y47f30uH4CQH90zDUNUGClsRuNPyCrl5WJ/ IgFVX0Xl/FidzXRQHa6THAYmACZWhx44mL9ppWi94sDGgNz7GCZxGBUEJSw2hwfmvMBS HNKA==; dara=google.com ARC-Authentication-Results: i=1; gmr-mx.google.com; dkim=pass header.i=@siemens.com header.s=fm1 header.b=J+cL1W+A; spf=pass (google.com: domain of fm-1321639-20240409155601abb2507ec2bc3d7b23-x_jmhs@rts-flowmailer.siemens.com designates 185.136.64.226 as permitted sender) smtp.mailfrom=fm-1321639-20240409155601abb2507ec2bc3d7b23-X_jMHs@rts-flowmailer.siemens.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=siemens.com Received: from mta-64-226.siemens.flowmailer.net (mta-64-226.siemens.flowmailer.net. [185.136.64.226]) by gmr-mx.google.com with ESMTPS id w12-20020a17090a8a0c00b002a290bec184si518118pjn.2.2024.04.09.08.56.03 for (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 09 Apr 2024 08:56:03 -0700 (PDT) Received-SPF: pass (google.com: domain of fm-1321639-20240409155601abb2507ec2bc3d7b23-x_jmhs@rts-flowmailer.siemens.com designates 185.136.64.226 as permitted sender) client-ip=185.136.64.226; Received: by mta-64-226.siemens.flowmailer.net with ESMTPSA id 20240409155601abb2507ec2bc3d7b23 for ; Tue, 09 Apr 2024 17:56:01 +0200 X-Patchwork-Original-From: "'Felix Moessbauer' via isar-users" From: "MOESSBAUER, Felix" To: isar-users@googlegroups.com Cc: jan.kiszka@siemens.com, quirin.gylstorff@siemens.com, Felix Moessbauer Subject: [PATCH v3 2/5] use apt snapshot mirror if ISAR_USE_APT_SNAPSHOT is set Date: Tue, 9 Apr 2024 17:55:46 +0200 Message-Id: <20240409155549.826454-3-felix.moessbauer@siemens.com> In-Reply-To: <20240409155549.826454-1-felix.moessbauer@siemens.com> References: <20240409155549.826454-1-felix.moessbauer@siemens.com> MIME-Version: 1.0 X-Flowmailer-Platform: Siemens Feedback-ID: 519:519-1321639:519-21489:flowmailer X-Original-Sender: felix.moessbauer@siemens.com X-Original-Authentication-Results: gmr-mx.google.com; dkim=pass header.i=@siemens.com header.s=fm1 header.b=J+cL1W+A; spf=pass (google.com: domain of fm-1321639-20240409155601abb2507ec2bc3d7b23-x_jmhs@rts-flowmailer.siemens.com designates 185.136.64.226 as permitted sender) smtp.mailfrom=fm-1321639-20240409155601abb2507ec2bc3d7b23-X_jMHs@rts-flowmailer.siemens.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=siemens.com X-Original-From: Felix Moessbauer Reply-To: Felix Moessbauer Precedence: list Mailing-list: list isar-users@googlegroups.com; contact isar-users+owners@googlegroups.com List-ID: X-Spam-Checked-In-Group: isar-users@googlegroups.com X-Google-Group-Id: 914930254986 List-Post: , List-Help: , List-Archive: , List-Unsubscribe: , X-Spam-Status: No, score=-1.2 required=5.0 tests=DKIMWL_WL_MED,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,MAILING_LIST_MULTI, RCVD_IN_MSPIKE_H3,RCVD_IN_MSPIKE_WL,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on shymkent.ilbers.de X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= This patch adds infrastructure to switch the apt sources to a frozen snapshot mirror. To build against a mirror, set ISAR_USE_APT_SNAPSHOT=1. As the mirror is distro specific, it is configured in the distro config for all supported distros (currently only debian and ubuntu). For unsupported distros, a meaningful error message is emitted on enabling the snapshot build. Similar to the DISTRO_APT_PREMIRRORS, this mirror is only injected temporarily and does not end up in the final apt sources list. To further control the behavior, we introduce the following variables: - DISTRO_APT_SNAPSHOT_PREMIRROR: The snapshot mirror to use. Syntax identical to DISTRO_APT_PREMIRRORS. - ISAR_APT_SNAPSHOT_TIMESTAMP: Unix timestamp of the snapshot. This is automatically derived from the SOURCE_DATE_EPOCH if not set. Signed-off-by: Felix Moessbauer --- RECIPE-API-CHANGELOG.md | 6 ++++++ doc/user_manual.md | 3 +++ meta-isar/conf/distro/ubuntu-common.inc | 3 +++ meta/conf/bitbake.conf | 3 +++ meta/conf/distro/debian-common.conf | 3 +++ .../isar-bootstrap/isar-bootstrap.inc | 16 ++++++++++++++++ 6 files changed, 34 insertions(+) diff --git a/RECIPE-API-CHANGELOG.md b/RECIPE-API-CHANGELOG.md index 6653ab43..e6861523 100644 --- a/RECIPE-API-CHANGELOG.md +++ b/RECIPE-API-CHANGELOG.md @@ -583,3 +583,9 @@ Cross compiling kernel modules for distro kernels is not supported in debian. To simplify downstream kernel module builds, we automatically turn of cross compilation for a user-provided module when building it for a distro kernel. + +### Build against debian snapshot mirror + +To build against a distributions snapshot mirror, set `ISAR_USE_APT_SNAPSHOT="1"`. +The mirror to use is specified in `DISTRO_APT_SNAPSHOT_PREMIRROR` and usually +pre-defined in the distro config. diff --git a/doc/user_manual.md b/doc/user_manual.md index 419d5339..70741968 100644 --- a/doc/user_manual.md +++ b/doc/user_manual.md @@ -431,6 +431,9 @@ Some other variables include: - `HOST_DISTRO_APT_PREFERENCES` - List of apt preference files for SDK root filesystem. This variable is optional. - `HOST_DISTRO_BOOTSTRAP_KEYS` - Analogously to DISTRO_BOOTSTRAP_KEYS: List of gpg key URIs used to verify apt bootstrap repo for the host. - `DISTRO_APT_PREMIRRORS` - The preferred mirror (append it to the default URI in the format `ftp.debian.org my.preferred.mirror`. This variable is optional. PREMIRRORS will be used only for the build. The final images will have the sources list as mentioned in DISTRO_APT_SOURCES. + - `ISAR_USE_APT_SNAPSHOT` - Use a frozen apt snapshot instead of the live mirror. Optional. + - `DISTRO_APT_SNAPSHOT_PREMIRROR` - Similar to `DISTRO_APT_PREMIRRORS` but for a snapshot, pre-defined for supported distros. + - `ISAR_APT_SNAPSHOT_TIMESTAMP` - Timestamp of the apt snapshot. Automatically derived from `SOURCE_DATE_EPOCH` if not overwritten. - `THIRD_PARTY_APT_KEYS` - List of gpg key URIs used to verify apt repos for apt installation after bootstrapping. - `FILESEXTRAPATHS` - The default directories BitBake uses when it processes recipes are initially defined by the FILESPATH variable. You can extend FILESPATH variable by using FILESEXTRAPATHS. - `FILESOVERRIDES` - A subset of OVERRIDES used by the build system for creating FILESPATH. The FILESOVERRIDES variable uses overrides to automatically extend the FILESPATH variable. diff --git a/meta-isar/conf/distro/ubuntu-common.inc b/meta-isar/conf/distro/ubuntu-common.inc index 9d8a843b..54bb747a 100644 --- a/meta-isar/conf/distro/ubuntu-common.inc +++ b/meta-isar/conf/distro/ubuntu-common.inc @@ -32,3 +32,6 @@ IMAGE_PREINSTALL += "init" IMAGE_PREINSTALL += "initramfs-tools" IMAGER_INSTALL:wic += "python3-distutils" + +# snapshot mirror for reproducible builds +DISTRO_APT_SNAPSHOT_PREMIRROR ??= "(http|https)://archive.ubuntu.com/(.*) https://snapshot.ubuntu.com/\2/${APT_SNAPSHOT_DATE}/\n" diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf index 1da3ecac..4cfa8b10 100644 --- a/meta/conf/bitbake.conf +++ b/meta/conf/bitbake.conf @@ -145,6 +145,9 @@ export SOURCE_DATE_EPOCH ?= "${@get_source_date_epoch_value(d)}" # A SOURCE_DATE_EPOCH of '0' might be misinterpreted as no SDE # ISAR: set value to date of latest release SOURCE_DATE_EPOCH_FALLBACK ??= "1709565251" +# Debian snapshots +ISAR_USE_APT_SNAPSHOT ??= "0" +ISAR_APT_SNAPSHOT_TIMESTAMP ??= "${SOURCE_DATE_EPOCH}" # Default parallelism and resource usage for xz XZ_MEMLIMIT ?= "50%" diff --git a/meta/conf/distro/debian-common.conf b/meta/conf/distro/debian-common.conf index 1e1dfc83..db538510 100644 --- a/meta/conf/distro/debian-common.conf +++ b/meta/conf/distro/debian-common.conf @@ -39,3 +39,6 @@ SYSTEMD_BOOTLOADER_INSTALL:sid = "systemd-boot-efi:${DISTRO_ARCH}" COMPAT_DISTRO_ARCH:amd64 = "i386" COMPAT_DISTRO_ARCH:arm64 = "armhf" + +# snapshot mirror for reproducible builds +DISTRO_APT_SNAPSHOT_PREMIRROR ??= "deb.debian.org/(.*) snapshot-cloudflare.debian.org/archive/\1/${APT_SNAPSHOT_DATE}/\n" \ No newline at end of file diff --git a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc index 17f19fd8..733a23df 100644 --- a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc +++ b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc @@ -30,6 +30,9 @@ DISTRO_VARS_PREFIX ?= "${@'HOST_' if bb.utils.to_boolean(d.getVar('BOOTSTRAP_FOR BOOTSTRAP_DISTRO = "${@d.getVar('HOST_DISTRO' if bb.utils.to_boolean(d.getVar('BOOTSTRAP_FOR_HOST')) else 'DISTRO')}" BOOTSTRAP_BASE_DISTRO = "${@d.getVar('HOST_BASE_DISTRO' if bb.utils.to_boolean(d.getVar('BOOTSTRAP_FOR_HOST')) else 'BASE_DISTRO')}" FILESEXTRAPATHS:append = ":${BBPATH}" +# reproducible builds, only enabled if ISAR_USE_APT_SNAPSHOT +ISAR_APT_SNAPSHOT_MIRROR ??= "" +APT_SNAPSHOT_DATE = "${@ get_apt_snapshot_date(d)}" inherit deb-dl-dir @@ -107,11 +110,21 @@ def parse_aptsources_list_line(source_list_line): return [type, options, source, suite, components] +def get_apt_snapshot_date(d): + import time + source_date_epoch = d.getVar('ISAR_APT_SNAPSHOT_TIMESTAMP') + return time.strftime('%Y%m%dT%H%M%SZ', time.gmtime(int(source_date_epoch))) + def get_apt_source_mirror(d, aptsources_entry_list): import re + # this is executed during parsing. No error checking possible + use_snapshot = bb.utils.to_boolean(d.getVar('ISAR_USE_APT_SNAPSHOT')) + snapshot_mirror = d.getVar('DISTRO_APT_SNAPSHOT_PREMIRROR') if bb.utils.to_boolean(d.getVar('ISAR_USE_CACHED_BASE_REPO')): premirrors = "\S* file://${REPO_BASE_DIR}/${BOOTSTRAP_BASE_DISTRO}\n" + elif use_snapshot and snapshot_mirror: + premirrors = snapshot_mirror else: premirrors = d.getVar('DISTRO_APT_PREMIRRORS') or "" mirror_list = [entry.split() @@ -126,6 +139,8 @@ def get_apt_source_mirror(d, aptsources_entry_list): new_aptsources_entry_list[2] = re.sub(regex, replace, aptsources_entry_list[2], count = 1) + if use_snapshot: + new_aptsources_entry_list[1] = "[check-valid-until=no]" return new_aptsources_entry_list return aptsources_entry_list @@ -240,6 +255,7 @@ do_apt_config_prepare[vardeps] += " \ APTSRCS \ ${DISTRO_VARS_PREFIX}DISTRO_APT_SOURCES \ DEPLOY_ISAR_BOOTSTRAP \ + ${@'DISTRO_APT_SNAPSHOT_PREMIRROR' if bb.utils.to_boolean(d.getVar('ISAR_USE_APT_SNAPSHOT')) else ''} \ " python do_apt_config_prepare() { apt_preferences_out = d.getVar("APTPREFS") From patchwork Tue Apr 9 15:55:47 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "MOESSBAUER, Felix" X-Patchwork-Id: 3490 Return-Path: Received: from shymkent.ilbers.de ([unix socket]) by shymkent (Cyrus 2.5.10-Debian-2.5.10-3+deb9u2) with LMTPA; Tue, 09 Apr 2024 17:56:19 +0200 X-Sieve: CMU Sieve 2.4 Received: from mail-pl1-f192.google.com (mail-pl1-f192.google.com [209.85.214.192]) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPS id 439FuHbo023043 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Tue, 9 Apr 2024 17:56:18 +0200 Received: by mail-pl1-f192.google.com with SMTP id d9443c01a7336-1e3f2261895sf19909775ad.3 for ; Tue, 09 Apr 2024 08:56:18 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1712678172; cv=pass; d=google.com; s=arc-20160816; b=lEQvAXPDBhfPHeVBzWTRu7uw8iZnQc3xpyjWtJ6euA/Ta6kyVfOfmhaVUo8KoLupyg WYg1me8mhLLONHltt7MOUi3wMxRcLnZqbV6w+e2s4aHvB5rgvIWPVn/RRUf9iRHl/xFp I+pS1I5q6Cdn8PlgS38zuWgJMqt3NcEHKb9XNZDM1R9w8RR5nJucWTeZQ/QNBYUxj5n+ LR/wsTD32AXMIwALFL2jh1aa5iuEIryE1LBtVDXLKAEEwHmh5QDd6SRlgDiiG00j0hFE R6EWCwi2lqNtKAHQ1miuPt+Q8414m83e4+wxQXeojI22AjZubQlFFup/IPXsosHi9/+d sSJw== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:reply-to:feedback-id:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=QGXDCt1LjEDqlD5i9rIrfjkRbXp4OkgZwsik5hH8HXs=; fh=lyy98DH4z2twHIDk0LLu84GVicVUjVXNgQW++DJYu9U=; b=OLnuJwRJ0mwddAfwP69C/UvO5Hg2oEL5Lh0af0yGADm5dmW6gZYlCjYl6qvZhEpBk4 nEV+XQkUwXErHl7nsBBR7K6/5Z9X0VqfuXds23mo4rj5fwFVbey4NHFHPDO54pmt0ngT ns/eHMX70It/I1j++oLQwwHv1B6xjC+4Y/GurOtatrXZS9gckIm1xVkNb7KbkM+3ywUp Ck9sT4dMx+MZYydIBVVv179oeqiZV4Oynxeu0dsYCTwgOPKcTnltSXlMEmcGVqrNC+wP HyQeB1BAtFtD1HHS4K9ZhHiqsLRRSPU09OhvAeS/1k1mFiIj7GAYiof1jqW0kgK2J+7t 4uiA==; darn=isar-build.org ARC-Authentication-Results: i=2; gmr-mx.google.com; dkim=pass header.i=@siemens.com header.s=fm1 header.b=RiZrWsnJ; spf=pass (google.com: domain of fm-1321639-20240409155608bb67d020dc880b7d8c-rizctb@rts-flowmailer.siemens.com designates 185.136.64.225 as permitted sender) smtp.mailfrom=fm-1321639-20240409155608bb67d020dc880b7d8c-RIZcTB@rts-flowmailer.siemens.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=siemens.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlegroups.com; s=20230601; t=1712678172; x=1713282972; darn=isar-build.org; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:reply-to :x-original-authentication-results:x-original-sender:feedback-id :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:from:to:cc:subject:date:message-id:reply-to; bh=QGXDCt1LjEDqlD5i9rIrfjkRbXp4OkgZwsik5hH8HXs=; b=W+xZOdlxQx3iaOvDLRAJGIfK8CHdGZDAKNq2iwe6anHmUsVXCUtEcUhvvkCF9I4ncU C8Tg9tphLIQYbI1PN9Jfd2TdZJF3uXnWMxF/6vexY6oddKKE2YItHxY1KeRdjn8MuK1P qQZX+rasTB1KVlNtjp0n4KbDahIwCixIGCYYN3SfTjy1TyzOnvowghk/oht4k28kibwz awo6YblvjWwMcy8IxDSfKAUaQGzBZGWC8AMg1LM1dRhHl6vnJ6Oigzs3+OCezJKMuz+l iAd0uvSJyl6B4wQK6tF2VuqxP8Aos5z3s0PBxAodtluBsEKk81MQTz4cmCKwkWLWk1sM qplw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1712678172; x=1713282972; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :x-spam-checked-in-group:list-id:mailing-list:precedence:reply-to :x-original-authentication-results:x-original-sender:feedback-id :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:x-beenthere:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=QGXDCt1LjEDqlD5i9rIrfjkRbXp4OkgZwsik5hH8HXs=; b=Yw6gXqgq3lE7jcc80lSgxLjAPsDzHo6WAk2G1RdNy7sfWwnuJ7/DEM11OLlL11WudE Cga2ZPhyThCYtIcniFoCoztIgCwFoe1LrngbWs+jcKa27eWt0QVHTXzIyJR73ILZykoF ZJOCS1y3DUpFsCXCw9S2IV+3I9ubACzEo4Bb1oP9cJFlpJw8h+ImeBxlAmCkcV1Kea7r D7ZUEfJJCtfZ26Eirx5C4Ps+gYbTmfe2QyqMlaz7AuzHr7F2l8TdqdZMB8l8EnZ6B/dp J/KIVPibKuW3sG1YQa9tpqGDHvmfMn7Bzea/Zkohpsn0HyqPP216XohZZc3/zmDVUWJD CKmQ== X-Forwarded-Encrypted: i=2; AJvYcCU6Z5tJo0JdED9Js4MEpBOhp7fGRoUNxBkRbh3FpkY74Dcwqy/juIF9bkqmFEKmE3UhuLjLovtYWSuz0U44uSL76ziRYQQ= X-Gm-Message-State: AOJu0YzDEN9fIxJVUdkC6CGJ8GsKCON7LSxFVNXG0DWKPlAqKNPwimwc Sa+LdQ3kPPE25yFwU5hZLcruP56ddW8EeXKQgOJaXkIFk+CCczZ4 X-Google-Smtp-Source: AGHT+IHWmeAioDnVID5rNUTkKlesBe8gOLgISi9lBIBqrgsPrZ/r9ec9/VuYthAqxaP7XzvXSm1Wmg== X-Received: by 2002:a17:902:d4c8:b0:1e4:7850:efe8 with SMTP id o8-20020a170902d4c800b001e47850efe8mr197326plg.10.1712678171604; Tue, 09 Apr 2024 08:56:11 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a17:902:d2cf:b0:1e2:a5cc:a39c with SMTP id n15-20020a170902d2cf00b001e2a5cca39cls1823464plc.2.-pod-prod-07-us; Tue, 09 Apr 2024 08:56:10 -0700 (PDT) X-Received: by 2002:a17:902:f682:b0:1dd:81a3:8dc3 with SMTP id l2-20020a170902f68200b001dd81a38dc3mr174296plg.46.1712678170272; Tue, 09 Apr 2024 08:56:10 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1712678170; cv=none; d=google.com; s=arc-20160816; b=ftOqpqCgAFpJGvRBcN+M66UX7N/4axNhHAbtkWbPfTUUKk9+Y28MuybinVrFbhE5+m QEA+tFJMJ6nkLsIqcJdSOnVQdULMFaeolG5MXyHKfMlQpJ8aSAUPLnljiPOcXOpJqPYe af4MJjkZgwTwztgx7qwOZrkf5Y8OBx9a2V6K72O4rSnqvGGGOaxA73fzTHARKpwZLBp8 JK0ySdnM48a+BJobZqATnCGtZo+8RhCqmMUZUJa3uHWEf0HNv1gLaG21NgIy61GLuLEG geJ0fz/1j/3gZqtvcZ3Ubrsu2Hw+JYdfTvfST59AmW/PlPYHXtwsYouw2qdEE63/EzYF Ohcg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=feedback-id:content-transfer-encoding:mime-version:references :in-reply-to:message-id:date:subject:cc:to:from:dkim-signature; bh=mkJGMVF1HHDPcREG0klULGPgQGYuqK0Gx4dM8H3DBEo=; fh=WkhL8kaJc+l2wQon1t06Ej3uvBGj9sVhNcE8PaS/XbI=; b=o5rVosz2tTIOS9ucE6nuH0v56S5UQrBX1kduC6Drn+Mc5KssZoDy15/Rv418gotD+w mkGzcJYuUtyz0IEZTQypB/12IYnR7EmwXxyKFIejRAzRUGqjDd2S86flx1AtWE7TiDwO 1d5FVT3byM23uMda2SbwrxtFBhIHojgJfv84EMD16TaRrSVFHFHBafIQCPXe35GNAjBs GCwe0bx2gjAUOhs/pkU+Vu+D+PWBXHT4eTDA6chMC5OM6h7zVPqdrqGjkmGAkmJHLmnc pFkSma+sE1Q4Q2xYFUQuQsZ2+D45TqH0X48VmRqVmlv5Czly4wRnH8PSOrF40wPwadAJ mMig==; dara=google.com ARC-Authentication-Results: i=1; gmr-mx.google.com; dkim=pass header.i=@siemens.com header.s=fm1 header.b=RiZrWsnJ; spf=pass (google.com: domain of fm-1321639-20240409155608bb67d020dc880b7d8c-rizctb@rts-flowmailer.siemens.com designates 185.136.64.225 as permitted sender) smtp.mailfrom=fm-1321639-20240409155608bb67d020dc880b7d8c-RIZcTB@rts-flowmailer.siemens.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=siemens.com Received: from mta-64-225.siemens.flowmailer.net (mta-64-225.siemens.flowmailer.net. [185.136.64.225]) by gmr-mx.google.com with ESMTPS id n19-20020a170903405300b001e3cfc2a3c4si518562pla.10.2024.04.09.08.56.10 for (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 09 Apr 2024 08:56:10 -0700 (PDT) Received-SPF: pass (google.com: domain of fm-1321639-20240409155608bb67d020dc880b7d8c-rizctb@rts-flowmailer.siemens.com designates 185.136.64.225 as permitted sender) client-ip=185.136.64.225; Received: by mta-64-225.siemens.flowmailer.net with ESMTPSA id 20240409155608bb67d020dc880b7d8c for ; Tue, 09 Apr 2024 17:56:08 +0200 X-Patchwork-Original-From: "'Felix Moessbauer' via isar-users" From: "MOESSBAUER, Felix" To: isar-users@googlegroups.com Cc: jan.kiszka@siemens.com, quirin.gylstorff@siemens.com, Felix Moessbauer Subject: [PATCH v3 3/5] add kas menu options to build against snapshots Date: Tue, 9 Apr 2024 17:55:47 +0200 Message-Id: <20240409155549.826454-4-felix.moessbauer@siemens.com> In-Reply-To: <20240409155549.826454-1-felix.moessbauer@siemens.com> References: <20240409155549.826454-1-felix.moessbauer@siemens.com> MIME-Version: 1.0 X-Flowmailer-Platform: Siemens Feedback-ID: 519:519-1321639:519-21489:flowmailer X-Original-Sender: felix.moessbauer@siemens.com X-Original-Authentication-Results: gmr-mx.google.com; dkim=pass header.i=@siemens.com header.s=fm1 header.b=RiZrWsnJ; spf=pass (google.com: domain of fm-1321639-20240409155608bb67d020dc880b7d8c-rizctb@rts-flowmailer.siemens.com designates 185.136.64.225 as permitted sender) smtp.mailfrom=fm-1321639-20240409155608bb67d020dc880b7d8c-RIZcTB@rts-flowmailer.siemens.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=siemens.com X-Original-From: Felix Moessbauer Reply-To: Felix Moessbauer Precedence: list Mailing-list: list isar-users@googlegroups.com; contact isar-users+owners@googlegroups.com List-ID: X-Spam-Checked-In-Group: isar-users@googlegroups.com X-Google-Group-Id: 914930254986 List-Post: , List-Help: , List-Archive: , List-Unsubscribe: , X-Spam-Status: No, score=-1.2 required=5.0 tests=DKIMWL_WL_MED,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,MAILING_LIST_MULTI, RCVD_IN_MSPIKE_H2,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on shymkent.ilbers.de X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= Signed-off-by: Felix Moessbauer --- kas/opt/Kconfig | 11 +++++++++++ kas/opt/mirror-snapshot.yaml | 9 +++++++++ 2 files changed, 20 insertions(+) create mode 100644 kas/opt/mirror-snapshot.yaml diff --git a/kas/opt/Kconfig b/kas/opt/Kconfig index 1619a4cf..701bd263 100644 --- a/kas/opt/Kconfig +++ b/kas/opt/Kconfig @@ -40,6 +40,17 @@ config KAS_INCLUDE_MIRROR_UBUNTU default "kas/opt/mirror-ubuntu.yaml" depends on MIRROR_UBUNTU +config USE_APT_SNAPSHOT + bool "Use the distros snapshot mirror" + depends on !MIRROR_DEBIAN && (DEBIAN_BUSTER || DEBIAN_BULLSEYE || DEBIAN_BOOKWORM || DEBIAN_TRIXIE || UBUNTU_FOCAL || UBUNTU_JAMMY) + help + Use a snapshot mirror for the selected distribution. The date is defined by ISAR_APT_SNAPSHOT_TIMESTAMP. + +config KAS_INCLUDE_APT_SNAPSHOT + string + default "kas/opt/mirror-snapshot.yaml" + depends on USE_APT_SNAPSHOT + endmenu diff --git a/kas/opt/mirror-snapshot.yaml b/kas/opt/mirror-snapshot.yaml new file mode 100644 index 00000000..59fae192 --- /dev/null +++ b/kas/opt/mirror-snapshot.yaml @@ -0,0 +1,9 @@ +# This software is a part of ISAR. +# Copyright (C) 2024 Siemens AG + +header: + version: 14 + +local_conf_header: + mirror-debian: | + ISAR_USE_APT_SNAPSHOT = "1" From patchwork Tue Apr 9 15:55:48 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "MOESSBAUER, Felix" X-Patchwork-Id: 3491 Return-Path: Received: from shymkent.ilbers.de ([unix socket]) by shymkent (Cyrus 2.5.10-Debian-2.5.10-3+deb9u2) with LMTPA; Tue, 09 Apr 2024 17:56:19 +0200 X-Sieve: CMU Sieve 2.4 Received: from mail-pj1-f57.google.com (mail-pj1-f57.google.com [209.85.216.57]) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPS id 439FuHxu023094 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Tue, 9 Apr 2024 17:56:18 +0200 Received: by mail-pj1-f57.google.com with SMTP id 98e67ed59e1d1-2a2dbaacff8sf4874407a91.2 for ; Tue, 09 Apr 2024 08:56:18 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1712678172; cv=pass; d=google.com; s=arc-20160816; b=rwNtu8pETX3a1ZZ/zcotdWO6LnydAXV0Q6TpPGAlcIcd7/pmjRjS68hsYtSXGUvPLr RXu2NQW5Yiepyy+D34wq8qTWI2IiBlUN/sUmFhXiyruwWdYqRv3EN6OPf/05u1PDsqS8 grvJErfmIFiH7tEkDmvsczc605bxPnuCgiYMmM9LjlYaT7kGyahxShPfYL1UvYh+dokM M7ILOqeAa4EqOWg7Tmxje3zu7cMHX22Juzz+JM7fAsuYPMvFp8nePpZrlz0MeVDDjJ1t ZthHqZQuoaPY53K0DhBTlxgP/AwUNvPUbNR8ORHmTwQ4cPg5k/ZdJJbcgZoeLlD9G9YV vN/Q== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:reply-to:feedback-id:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=4wXwQgZWiUUpdxjXIn8+1yCulZ4k1EHR8y2QGj4DLuA=; fh=2nVnaIBgsA2CzMXWV5/aKhS+ch2EVsUbvKh0SajOMiA=; b=XPu0PoDvQ/GnuSIn1MFsalvym9eAQwWg7ogqk+A6RF0YlXSpcVxKwwFxSunZVQFyY+ jDS/ozd8On+TsaPbBFFXq4//qW49u+pIcmT6fOwK2ZBERPJNUaXEUMJDhgUsiip9XAl9 ta5+BPsY8sNeLOMVxGvCDzvHWs/4DV5pwZt+AE4Dr+jxy2avZbn6UHqf8G2Ywj8UX5f8 TlifDV0QZQNwevwaK0Fk6GQIc9eJMmmLYH7omkERzjxT2E94+erwoa2bauSllKf8k9Af t06CddlZGjW1htbMJnjkrgNdSuVYkuYW8biSk3VM0FkACtWKWtjXejWW+Lt2EXRBZeeC zqUg==; darn=isar-build.org ARC-Authentication-Results: i=2; gmr-mx.google.com; dkim=pass header.i=@siemens.com header.s=fm1 header.b=Knjflh69; spf=pass (google.com: domain of fm-1321639-202404091556085e5053408d64d9c91c-mm_jwi@rts-flowmailer.siemens.com designates 185.136.64.228 as permitted sender) smtp.mailfrom=fm-1321639-202404091556085e5053408d64d9c91c-MM_JwI@rts-flowmailer.siemens.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=siemens.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlegroups.com; s=20230601; t=1712678172; x=1713282972; darn=isar-build.org; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:reply-to :x-original-authentication-results:x-original-sender:feedback-id :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:from:to:cc:subject:date:message-id:reply-to; bh=4wXwQgZWiUUpdxjXIn8+1yCulZ4k1EHR8y2QGj4DLuA=; b=xmo44mZv1NsUlAnuuoERP9PVT62DRvYriihyKzY9tBUJa2FtTPxkZNPQVAJG9L9WBH XRzD683xHpO6Ab4m4Nt7VMVomwVp7OUOIFELQn42qH9YTrLrEbGNFjNaTqM8PKy8opTC olXW2y6F/FoXIsnteTk8tNwsAxEijpIsJJNn7w8IVzLZqovN58vLrAfPG0PGzYk6/dIi P4GfA8BeZmjNdlQVSB+ecnzKmVHKv3cyzZ/k3nISSbUlsONU+gnVWVxJmH/QIOZNYvLu EaH0AxquzRvSzred6+6I1RX3Og8fvVlF/XdHG0Ln/1G+AyFWwcJEJ0eMIIePJthrM1hl HaVQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1712678172; x=1713282972; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :x-spam-checked-in-group:list-id:mailing-list:precedence:reply-to :x-original-authentication-results:x-original-sender:feedback-id :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:x-beenthere:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=4wXwQgZWiUUpdxjXIn8+1yCulZ4k1EHR8y2QGj4DLuA=; b=Bps45pfpobvN9mnFvSqZ5l1TDcJPB7NEDNgeOYNQOhOECwHpnWJuywB8W2Pq1j3SzD 4p10tHER3uAK5AP7IvuGRR88h4yASvGLWTsZ2IR/GTW0Bo0B65PGB6fsKlaEGIu0LDyi lUGenbXD9Dj86/KXvqbbYR4QMn99kVyof3XH3CrUUEHP8KVjD2nq7W3XOUqPAsPjb56M H+eo4c87PiL/KPUK7Zs3zeeoxWLmWge7+7n/4hFh6yuFpg7tpdnpbBgElREJzByO9DEp HchYZ5fd/wscHs/t+ClMxkMpFZ8icSEp9ZrY5M0pclH2FM8Pk9xPPJS8N3NIK8i6WThD Em2g== X-Forwarded-Encrypted: i=2; AJvYcCUwJ7gtDHl0rzJrFduDZjhQYS5z7CmqIZVYkpUh3kPJ4OahNU432D3rZYopQuqqspFLBI/pRXAJESFDWbsJ6OuVNM/LQiQ= X-Gm-Message-State: AOJu0YzFNCkNwnMsM4PQwSfChWT8+6DqhrTCzHStvi5xdTFak6Gqu0LK uTdONN0NJgKNfYbpJvtoLCtx82XdErKHrP6B9Bzm+iSEArwk/LOD X-Google-Smtp-Source: AGHT+IEmTvSsBCno0n6Pbl8AiNxWDOWsi+hJeqZgxi9Cf9D2wYkEOdTqiXEIcxCKKiqaKVIp4oUeMw== X-Received: by 2002:a17:90b:188b:b0:2a2:afb8:372a with SMTP id mn11-20020a17090b188b00b002a2afb8372amr41104pjb.17.1712678172128; Tue, 09 Apr 2024 08:56:12 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a17:90a:8c6:b0:2a2:937e:f959 with SMTP id 6-20020a17090a08c600b002a2937ef959ls2928059pjn.2.-pod-prod-09-us; Tue, 09 Apr 2024 08:56:11 -0700 (PDT) X-Received: by 2002:a17:90a:d910:b0:2a4:6a31:600c with SMTP id c16-20020a17090ad91000b002a46a31600cmr42318pjv.14.1712678170639; Tue, 09 Apr 2024 08:56:10 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1712678170; cv=none; d=google.com; s=arc-20160816; b=YgaQvu4Zy7cjjidxuHYC8r5q+FD/Hay0RJS1ZnupRrG+Vs2DY2gl/6JZ0OF7/6XdlD KF1tHbpfXIKiKGSfHwu/bApUoT5NY5IuiiRRqpiswl1qf0Md3BotT2OocH0GRt4Wx+bg MCWxagtEzzuAXiZ+wnqUeF1vcjIzPRMsNxFbPQg5rZvJxvoEl7Yw8hxH2p+RV89okucW uN+Gw6OojFlwLo57gBduvfq+GW+4j/WF8rwzjlH01l0OwQ9zatnO1GqKEa1tkKe0ysyW 6xpmidkk37Uqh6XgwM1kKTG3Hwu53MvfJNronRQdUBgBnlYto261xo7UMRQowhOGZT8x Xf4A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=feedback-id:content-transfer-encoding:mime-version:references :in-reply-to:message-id:date:subject:cc:to:from:dkim-signature; bh=ilQMryfxvzUvG4DkYfuoXCHe2z5Q+t43t3HIp4OQ/kI=; fh=WkhL8kaJc+l2wQon1t06Ej3uvBGj9sVhNcE8PaS/XbI=; b=XklFiJLRX0wi/95UVk1uwTwS17rZ4CueqsUkIFPjl0BDPNo5m2DU4I44Lz/D0jN0fi JbSyLd4x7hjZTm/fgUuSBsSsrQuuK9w3mVQZ1LWDjfY6lEaeCJEZqHIp0aqWFDG/skc6 RTc73WvXU7GSbW/pfITVe+MgFCy8/CRv5sFJC7pDaU55tMRMKNmo5KI6mimcBUByS6kS CZzEJFc6ig0L+voCmbsPlwhc4c7Qlj01YZyGCcErYU4WZKVUaTVqsbzzjhSpgt3DfPee 05n7X3vH3jkKbOKtxz+J6PGVrgsTlNhF+QOkQwriV1xI3Vtpg4kDCl8pimcWiLj6SATT McBA==; dara=google.com ARC-Authentication-Results: i=1; gmr-mx.google.com; dkim=pass header.i=@siemens.com header.s=fm1 header.b=Knjflh69; spf=pass (google.com: domain of fm-1321639-202404091556085e5053408d64d9c91c-mm_jwi@rts-flowmailer.siemens.com designates 185.136.64.228 as permitted sender) smtp.mailfrom=fm-1321639-202404091556085e5053408d64d9c91c-MM_JwI@rts-flowmailer.siemens.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=siemens.com Received: from mta-64-228.siemens.flowmailer.net (mta-64-228.siemens.flowmailer.net. [185.136.64.228]) by gmr-mx.google.com with ESMTPS id mu11-20020a17090b388b00b002a5aaa7bd84si12525pjb.0.2024.04.09.08.56.10 for (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 09 Apr 2024 08:56:10 -0700 (PDT) Received-SPF: pass (google.com: domain of fm-1321639-202404091556085e5053408d64d9c91c-mm_jwi@rts-flowmailer.siemens.com designates 185.136.64.228 as permitted sender) client-ip=185.136.64.228; Received: by mta-64-228.siemens.flowmailer.net with ESMTPSA id 202404091556085e5053408d64d9c91c for ; Tue, 09 Apr 2024 17:56:08 +0200 X-Patchwork-Original-From: "'Felix Moessbauer' via isar-users" From: "MOESSBAUER, Felix" To: isar-users@googlegroups.com Cc: jan.kiszka@siemens.com, quirin.gylstorff@siemens.com, Felix Moessbauer Subject: [PATCH v3 4/5] change the proposed interface to set the SDE Date: Tue, 9 Apr 2024 17:55:48 +0200 Message-Id: <20240409155549.826454-5-felix.moessbauer@siemens.com> In-Reply-To: <20240409155549.826454-1-felix.moessbauer@siemens.com> References: <20240409155549.826454-1-felix.moessbauer@siemens.com> MIME-Version: 1.0 X-Flowmailer-Platform: Siemens Feedback-ID: 519:519-1321639:519-21489:flowmailer X-Original-Sender: felix.moessbauer@siemens.com X-Original-Authentication-Results: gmr-mx.google.com; dkim=pass header.i=@siemens.com header.s=fm1 header.b=Knjflh69; spf=pass (google.com: domain of fm-1321639-202404091556085e5053408d64d9c91c-mm_jwi@rts-flowmailer.siemens.com designates 185.136.64.228 as permitted sender) smtp.mailfrom=fm-1321639-202404091556085e5053408d64d9c91c-MM_JwI@rts-flowmailer.siemens.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=siemens.com X-Original-From: Felix Moessbauer Reply-To: Felix Moessbauer Precedence: list Mailing-list: list isar-users@googlegroups.com; contact isar-users+owners@googlegroups.com List-ID: X-Spam-Checked-In-Group: isar-users@googlegroups.com X-Google-Group-Id: 914930254986 List-Post: , List-Help: , List-Archive: , List-Unsubscribe: , X-Spam-Status: No, score=-1.2 required=5.0 tests=DKIMWL_WL_MED,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,MAILING_LIST_MULTI, RCVD_IN_MSPIKE_H3,RCVD_IN_MSPIKE_WL,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on shymkent.ilbers.de X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= To align the handling of source timestamps with OE, we propose to not set the SOURCE_DATE_EPOCH variable directly, but via `SOURCE_DATE_EPOCH_FALLBACK`. By that, we internally can use heuristics to better estimate the value of the SDE, while still having a global fallback if it cannot be estimated otherwise. Please note, that change is backwards compatible. You can still set the SOURCE_DATE_EPOCH variable directly and get the same behavior as before. This patch also updates the reproducibility test according to the new interface. Signed-off-by: Felix Moessbauer --- RECIPE-API-CHANGELOG.md | 6 ++++++ doc/user_manual.md | 1 + testsuite/cibuilder.py | 2 +- 3 files changed, 8 insertions(+), 1 deletion(-) diff --git a/RECIPE-API-CHANGELOG.md b/RECIPE-API-CHANGELOG.md index e6861523..4afe8b16 100644 --- a/RECIPE-API-CHANGELOG.md +++ b/RECIPE-API-CHANGELOG.md @@ -589,3 +589,9 @@ compilation for a user-provided module when building it for a distro kernel. To build against a distributions snapshot mirror, set `ISAR_USE_APT_SNAPSHOT="1"`. The mirror to use is specified in `DISTRO_APT_SNAPSHOT_PREMIRROR` and usually pre-defined in the distro config. + +### Use OE interface to set timestamp for reproducible builds + +The `SOURCE_DATE_EPOCH` (SDE) should not be set globally, but on a per-recipe basis +and to meaningful values. As a global fallback, set the `SOURCE_DATE_EPOCH_FALLBACK` +bitbake variable to the desired unix timestamp. diff --git a/doc/user_manual.md b/doc/user_manual.md index 70741968..b12c7692 100644 --- a/doc/user_manual.md +++ b/doc/user_manual.md @@ -425,6 +425,7 @@ Some other variables include: - `IMAGE_INSTALL` - The list of custom packages to build and install to target image, please refer to relative chapter for more information. - `BB_NUMBER_THREADS` - The number of `bitbake` jobs that can be run in parallel. Please set this option according to your host CPU cores number. + - `SOURCE_DATE_EPOCH_FALLBACK` - The unix timestamp passed to all tooling to make the results reproducible. This variable is optional. - `HOST_DISTRO` - The distro to use for SDK root filesystem. This variable is optional. - `HOST_ARCH` - The Debian architecture of SDK root filesystem (e.g., `amd64`). By default set to current Debian host architecture. This variable is optional. - `HOST_DISTRO_APT_SOURCES` - List of apt source files for SDK root filesystem. This variable is optional. diff --git a/testsuite/cibuilder.py b/testsuite/cibuilder.py index fa30c2f5..7e69dafe 100755 --- a/testsuite/cibuilder.py +++ b/testsuite/cibuilder.py @@ -159,7 +159,7 @@ class CIBuilder(Test): f.write('USE_CCACHE = "1"\n') f.write('CCACHE_TOP_DIR = "%s"\n' % ccache_dir) if source_date_epoch: - f.write('SOURCE_DATE_EPOCH = "%s"\n' % source_date_epoch) + f.write('SOURCE_DATE_EPOCH_FALLBACK = "%s"\n' % source_date_epoch) if dl_dir: f.write('DL_DIR = "%s"\n' % dl_dir) if sstate_dir: From patchwork Tue Apr 9 15:55:49 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "MOESSBAUER, Felix" X-Patchwork-Id: 3489 Return-Path: Received: from shymkent.ilbers.de ([unix socket]) by shymkent (Cyrus 2.5.10-Debian-2.5.10-3+deb9u2) with LMTPA; Tue, 09 Apr 2024 17:56:18 +0200 X-Sieve: CMU Sieve 2.4 Received: from mail-oa1-f63.google.com (mail-oa1-f63.google.com [209.85.160.63]) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPS id 439FuHYR023014 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Tue, 9 Apr 2024 17:56:17 +0200 Received: by mail-oa1-f63.google.com with SMTP id 586e51a60fabf-22a210cda85sf3405295fac.1 for ; Tue, 09 Apr 2024 08:56:17 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1712678171; cv=pass; d=google.com; s=arc-20160816; b=AwM93jXO9wspJxpBSZB92aQAIwh9k55yCSThxikxiFr93HWm9dMqIkx+LT1I3VfTVu fteEIngovQhHfLL/uXM1sZgSu3A4bJ8wcNWplQdPpXMe8Oh0GIzTe/7rIvUTdDegY7S0 juqcLnwji3Kyqk34Y01vSZ2tTt2M7ItPNp9ficge9O1Hl920EDfVOX+5dVZNVYMDpzfd /+JMq34VJj3wc/+/q4Ep5X89Pkzd7jswYaYhMVc/10R7SX/qVLqu7GSIs7UFkzNT76JE /qwKOtqKz4ARuUUX3b8Rn/M5S1RxzyfZDGGbNCdhktspLBw5A24XEQefAsOQ9Tea9qtF x0qw== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:reply-to:feedback-id:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=UJJcix9QEh14J8uYiMwuG9bIOLXRRz96MgUPqy7PpEo=; fh=PXz3TFDE5mjud0SQEDyvbvDcsQM/FlwoYtWLKhbTIhw=; b=BH4Zjl39lYT/ynLSehZNWZyKjGwYw3vhqMivGkG9NZZ4HdNF8p6PHrI57c4aJ+93LW 6CZpOiR0BkBRrFjf9nTSBW3Ewu5lVjDjUFUOjTM9a2OOaJznSnn0znW4tJAqSWvH0iGi hsvGBNwhC6TodBB3dckDjAaw1Qsp1kE1ikE3kPVNDLD/vqKlxMRh6nwnBo8izDOHq734 S1m926IjToA3Vf+oItrXrFhi0s5qiq8OxmMzsDC5q+ifZs31WuR251bmPSpsXj2ADNRN kj8feD0qL9tcsem7mF+q4gdIiYWh9NFbID/65trWgI80N1lrpO/tp+XMFY37JG0a2Hnz 80TQ==; darn=isar-build.org ARC-Authentication-Results: i=2; gmr-mx.google.com; dkim=pass header.i=@siemens.com header.s=fm1 header.b=GSoaPQpv; spf=pass (google.com: domain of fm-1321639-20240409155608e8da9c026e5a643812-rg_fv0@rts-flowmailer.siemens.com designates 185.136.65.227 as permitted sender) smtp.mailfrom=fm-1321639-20240409155608e8da9c026e5a643812-RG_FV0@rts-flowmailer.siemens.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=siemens.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlegroups.com; s=20230601; t=1712678171; x=1713282971; darn=isar-build.org; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:reply-to :x-original-authentication-results:x-original-sender:feedback-id :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:from:to:cc:subject:date:message-id:reply-to; bh=UJJcix9QEh14J8uYiMwuG9bIOLXRRz96MgUPqy7PpEo=; b=OuzSFk3c7qZXYvX9wdExtjx0rnezm+m2wt0qE3+s85zszRw3KIRJgu799+G4CJOK41 OjtxeoLagRALMpDVG6WWzAFBNLlW679dir6SUXgnqPmYk19Io8ainwr9bJs9Oi5dMQkp b30ppomdzPctInJ39A8oZlQW9kKoQl7FhO3wsFEyAnrpIsGsiARJt1vaYQWWW9mYDdii 2OqyC58dtdgXNuR7y/TZBX0+jjt04yasF1QdFn4JEQhLKmpvTW6cWDuCIQeFZdWJ1i9k 0nBBfmuA4U+5NGIBpBuze4pwbti5lofQmuBMVkSK9FWF2iMuf6FrQiWHWR7jJsrdIW6k hjFg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1712678171; x=1713282971; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :x-spam-checked-in-group:list-id:mailing-list:precedence:reply-to :x-original-authentication-results:x-original-sender:feedback-id :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:x-beenthere:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=UJJcix9QEh14J8uYiMwuG9bIOLXRRz96MgUPqy7PpEo=; b=AYBh45f1NPqrMEwfeKlkUt8kRGz4R0FOpevgX9nOPe/IJzC7Tj2TW9uC/jIDipIJW8 s+HVYdNtzsUIQrTZwTF2oBxyjjn/IMy/WRlhjilpMELhHF8CLtb/NzQbh12U27f/SEsl LP9W1rz7hmj2b/qX59h1+A0vuVvZgpNvJoDL/6ohmJh3CEEZWw5usom1EkE+UFT1oAlC VlhJgaPYQIPJxCX5GRgLcg5Or39N1TQL03GGJ09RqLxIlm98FcoEPrNeAnpc0wGRkCkg B/wgqA8eWPDS78vwsxEOPTev5y4mvRtM5g9mQI7Wmc/jShQECTZo+aRHT6oi0IHqW5S5 vk2w== X-Forwarded-Encrypted: i=2; AJvYcCXLgHHLYmw4GM2ZGRsFVsLM3MS+HTBk0gsX7bLTA+2e26UOYXPTlVwAb3lxRLfAE2LHO+sKkdNJ4SPvvWjlhJExIHRR31M= X-Gm-Message-State: AOJu0YyWqXW6DTkGbWq8L0r/r65GO/PJ/XgiS8l/JUGOlTAG8hp4/NeM wlkeVrSU0Wn+5oGDKqqA+OxSw824qsdDCN34JC41xysccE7vPtzo X-Google-Smtp-Source: AGHT+IGh/TTrVaOJiIRvAwixnn5hQbH+fC0+sT8ZJkUP2ZybUMBW6tL7gult5lV4fCaAgsWZMcss3A== X-Received: by 2002:a05:6871:887:b0:21e:b8f7:9a1b with SMTP id r7-20020a056871088700b0021eb8f79a1bmr12579809oaq.25.1712678171577; Tue, 09 Apr 2024 08:56:11 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a05:6870:724b:b0:22e:bed9:4695 with SMTP id y11-20020a056870724b00b0022ebed94695ls3850577oaf.1.-pod-prod-06-us; Tue, 09 Apr 2024 08:56:10 -0700 (PDT) X-Received: by 2002:a05:6870:c087:b0:229:e6c6:1f13 with SMTP id c7-20020a056870c08700b00229e6c61f13mr12581060oad.14.1712678170439; Tue, 09 Apr 2024 08:56:10 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1712678170; cv=none; d=google.com; s=arc-20160816; b=VcE793qg0Ied07UiBQJLiqQ8J1W7jtYAmd7Gt7hA5HGtMhDY6Eq6aM3xRq7cAHGxN/ kpPnu4VH4Lj8YX+TrAUvmlmKdeyP5SXt3RW92D20b3Fg0kyGc2wsekLe5gslBUkDoY92 f77n23KDskN8rZd4wFNX2zbHIZ5N+8lPYWQEVGkc92oci0OFa1hlCSO/X0EKxY+wY/Ht VtJ+VCqqQdhje4VKC1iOXRY+F2dvJ67SS0+WNuOWw3AqzLmzYmUzt0N/U3GpFvhfyf9y WUdUQP8PUdk9qa3P4XrIYmvCvu5oAbt++AAPK+hETgzSvVYGbVsYmiSOrscbGYtm3xCz Q0Eg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=feedback-id:content-transfer-encoding:mime-version:references :in-reply-to:message-id:date:subject:cc:to:from:dkim-signature; bh=Tk9K1sWITZ8ojr8cF6KomEYzaqy9Ai63TQSVjrLK5Ao=; fh=WkhL8kaJc+l2wQon1t06Ej3uvBGj9sVhNcE8PaS/XbI=; b=QYVy8lvbddW8sFm2byYMxrfMP66rohwG8qdl6ByRo6qSLaSCHoNrr93iL8hVQZN7EE g4dyAWlImLXrmFW4ie9+k1lh6fpBnjfuPU/xRUBXe/LumY8BMC8ie5llByw/wnYIzWjB ZRPnqQdM+G2ivZjLTqQOMfuAzBYBtAF3L80McMZpCyVs4CqcECLx+qEC6EtafE8kmpFy NiX9loh3Vi+9IYSUmFI17ZO2+qWzp9yiLa+Q0E3zxi0R0Kt6aGm+aA6JkkzoYv1bKQh+ yiS644szP5urGw67F9DiLZyGm83GM1z2UBDi7IvV4+ydsIdWD7VY7FCRNq95PNbummaY MxsA==; dara=google.com ARC-Authentication-Results: i=1; gmr-mx.google.com; dkim=pass header.i=@siemens.com header.s=fm1 header.b=GSoaPQpv; spf=pass (google.com: domain of fm-1321639-20240409155608e8da9c026e5a643812-rg_fv0@rts-flowmailer.siemens.com designates 185.136.65.227 as permitted sender) smtp.mailfrom=fm-1321639-20240409155608e8da9c026e5a643812-RG_FV0@rts-flowmailer.siemens.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=siemens.com Received: from mta-65-227.siemens.flowmailer.net (mta-65-227.siemens.flowmailer.net. [185.136.65.227]) by gmr-mx.google.com with ESMTPS id du9-20020a0568703a0900b0022ef31f7182si456830oab.2.2024.04.09.08.56.10 for (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 09 Apr 2024 08:56:10 -0700 (PDT) Received-SPF: pass (google.com: domain of fm-1321639-20240409155608e8da9c026e5a643812-rg_fv0@rts-flowmailer.siemens.com designates 185.136.65.227 as permitted sender) client-ip=185.136.65.227; Received: by mta-65-227.siemens.flowmailer.net with ESMTPSA id 20240409155608e8da9c026e5a643812 for ; Tue, 09 Apr 2024 17:56:08 +0200 X-Patchwork-Original-From: "'Felix Moessbauer' via isar-users" From: "MOESSBAUER, Felix" To: isar-users@googlegroups.com Cc: jan.kiszka@siemens.com, quirin.gylstorff@siemens.com, Felix Moessbauer Subject: [PATCH v3 5/5] ci: use snapshot in reproducible builds test Date: Tue, 9 Apr 2024 17:55:49 +0200 Message-Id: <20240409155549.826454-6-felix.moessbauer@siemens.com> In-Reply-To: <20240409155549.826454-1-felix.moessbauer@siemens.com> References: <20240409155549.826454-1-felix.moessbauer@siemens.com> MIME-Version: 1.0 X-Flowmailer-Platform: Siemens Feedback-ID: 519:519-1321639:519-21489:flowmailer X-Original-Sender: felix.moessbauer@siemens.com X-Original-Authentication-Results: gmr-mx.google.com; dkim=pass header.i=@siemens.com header.s=fm1 header.b=GSoaPQpv; spf=pass (google.com: domain of fm-1321639-20240409155608e8da9c026e5a643812-rg_fv0@rts-flowmailer.siemens.com designates 185.136.65.227 as permitted sender) smtp.mailfrom=fm-1321639-20240409155608e8da9c026e5a643812-RG_FV0@rts-flowmailer.siemens.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=siemens.com X-Original-From: Felix Moessbauer Reply-To: Felix Moessbauer Precedence: list Mailing-list: list isar-users@googlegroups.com; contact isar-users+owners@googlegroups.com List-ID: X-Spam-Checked-In-Group: isar-users@googlegroups.com X-Google-Group-Id: 914930254986 List-Post: , List-Help: , List-Archive: , List-Unsubscribe: , X-Spam-Status: No, score=-1.2 required=5.0 tests=DKIMWL_WL_MED,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,MAILING_LIST_MULTI, RCVD_IN_MSPIKE_H2,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on shymkent.ilbers.de X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= As we now have easy to use snapshot support, we also switch over the reproducible builds test to use a snapshot. By that, we both test the snapshot support, as well as make sure to always build against a common set of packages. Signed-off-by: Felix Moessbauer --- testsuite/cibuilder.py | 6 +++++- testsuite/repro-build-test.py | 2 +- 2 files changed, 6 insertions(+), 2 deletions(-) diff --git a/testsuite/cibuilder.py b/testsuite/cibuilder.py index 7e69dafe..039355e5 100755 --- a/testsuite/cibuilder.py +++ b/testsuite/cibuilder.py @@ -78,7 +78,8 @@ class CIBuilder(Test): container=False, ccache=False, sstate=False, offline=False, gpg_pub_key=None, wic_deploy_parts=False, dl_dir=None, sstate_dir=None, ccache_dir=None, - source_date_epoch=None, image_install=None, **kwargs): + source_date_epoch=None, use_apt_snapshot=False, + image_install=None, **kwargs): # write configuration file and set bitbake_args # can run multiple times per test case self.check_init() @@ -116,6 +117,7 @@ class CIBuilder(Test): f' gpg_pub_key = {gpg_pub_key}\n' f' wic_deploy_parts = {wic_deploy_parts}\n' f' source_date_epoch = {source_date_epoch} \n' + f' use_apt_snapshot = {use_apt_snapshot} \n' f' dl_dir = {dl_dir}\n' f' sstate_dir = {sstate_dir}\n' f' ccache_dir = {ccache_dir}\n' @@ -160,6 +162,8 @@ class CIBuilder(Test): f.write('CCACHE_TOP_DIR = "%s"\n' % ccache_dir) if source_date_epoch: f.write('SOURCE_DATE_EPOCH_FALLBACK = "%s"\n' % source_date_epoch) + if use_apt_snapshot: + f.write('ISAR_USE_APT_SNAPSHOT = "1"\n') if dl_dir: f.write('DL_DIR = "%s"\n' % dl_dir) if sstate_dir: diff --git a/testsuite/repro-build-test.py b/testsuite/repro-build-test.py index 040a844e..843f3bc6 100755 --- a/testsuite/repro-build-test.py +++ b/testsuite/repro-build-test.py @@ -46,7 +46,7 @@ class ReproBuild(CIBuilder): # Build self.log.info("Started Build " + image_name) - self.configure(source_date_epoch=source_date_epoch) + self.configure(source_date_epoch=source_date_epoch, use_apt_snapshot=True) self.bitbake(target) # copy the artifacts image name with given name