From patchwork Wed Nov 6 08:21:10 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anton Mikanovich X-Patchwork-Id: 3894 Return-Path: Received: from shymkent.ilbers.de ([unix socket]) by shymkent (Cyrus 2.5.10-Debian-2.5.10-3+deb9u2) with LMTPA; Wed, 06 Nov 2024 09:21:37 +0100 X-Sieve: CMU Sieve 2.4 Received: from mail-wr1-f64.google.com (mail-wr1-f64.google.com [209.85.221.64]) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPS id 4A68LZWH017008 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Wed, 6 Nov 2024 09:21:36 +0100 Received: by mail-wr1-f64.google.com with SMTP id ffacd0b85a97d-37d4cf04be1sf3393308f8f.2 for ; Wed, 06 Nov 2024 00:21:36 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1730881289; cv=pass; d=google.com; s=arc-20240605; b=W+Em9DcdOMdzDW4FWk/Sz6K+UkCwDZQ7jgQC3ztB0IfSY5puNB/6XG+nqhHgFytvbO 1N1DvxMcScMujdx0g4AahB9tfRH+Wta1ZN6VLCfLBFhWtCNaAupyKOKAlox1/Fbo9qwI +6qh0CDJvqH3ZQrm25LBMTildLmvb3AKKSyeATBtrIfrz22xAWN2rCAHG+GuxGIZ8EMU x8b12kO7ggxStDFdkIEsBt5at3vvpI8HLh2n/tlVpJkUraMhR8o3aaZLS/nqxoki1xiM ce1SXEmM+8xprg1wXCtESj4Yq5ML3QWgpKByDOpGI25zcgqi/bwfS2lSZhlEgPj1FR6o zIlQ== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20240605; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:dkim-signature; bh=EuRTUONUeJ/2BNLp9TxuE5zZassyloqOuCbb3TCEFMI=; fh=1L5HGb6QbC1WZBHmPhspVdHwCQa78w1vLbiWn1E+Mk8=; b=DhRnvvafj9PHgb00KtoRp2cLVnfPlJNyBCROM07PXUUL3QgKpZlCHfPtZeoZJ/V75T IZ95ouGSvt/j3GigTGXFq5TPeBsZJFa9hQ+R82nezEzsGe7e7mDdrUqgJNC/UMDAYFCY qlhBt/MlSpDOcvrtbgXU8ATA258mIPNUA7VpG7hXd16cX9ZrqpevqtSUQxN0kizevYLw flJuRc6FTHs5BQ0OME8+fzaPu7vW9hm2uimRg5WVfSWSNfzKO84IPvWiyoCJmjkuX4+f gh5+5jA/kNHeZm2nMC5hW9eMRfezbsFfqMjyKXl0MqKaa/GgeHgad/uw/FwBrFBqHmmg meqg==; darn=isar-build.org ARC-Authentication-Results: i=2; gmr-mx.google.com; spf=pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=amikan@ilbers.de DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlegroups.com; s=20230601; t=1730881289; x=1731486089; darn=isar-build.org; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:x-original-authentication-results :x-original-sender:mime-version:references:in-reply-to:message-id :date:subject:cc:to:from:sender:from:to:cc:subject:date:message-id :reply-to; bh=EuRTUONUeJ/2BNLp9TxuE5zZassyloqOuCbb3TCEFMI=; b=rX69d/qbxHPL7tK6gFzc8I2R4lE9NJ5L3yam/C5MQqN8Z3dleypMkPmDMHE1e6m+IC iOWpaFG4XLgguekzBW5Iw66tSddzhyuRsF3keZ5cyJOhM15fz//5u+E2gA0p05N+iyx4 wBxWyeHGqDekbpSQE1jQwTeDIDkfWFZ6YIhmPtTZvXNkhXpfXjy2liIcQ93nsLQlVZtp t08o2dL2tBGOonx8Ev3nYlsBEcGDsveAchwdPZMyobnN7m3/kNsOBxD1ajGwFqMufKGq M/WwhSRE5bltXRBdomRDnbn7a6hYO5tiE2DRPK6JXuXNON6/BDrfs85Fu9Bpr/0dZGh+ 2aEQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1730881289; x=1731486089; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :x-spam-checked-in-group:list-id:mailing-list:precedence :x-original-authentication-results:x-original-sender:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :x-beenthere:x-gm-message-state:sender:from:to:cc:subject:date :message-id:reply-to; bh=EuRTUONUeJ/2BNLp9TxuE5zZassyloqOuCbb3TCEFMI=; b=Lj0sorPjxklz2g26hNXnkoebN9N3u4xWT2g671XlxvFVtlHOyWh85f8faWca2YltHy So7rxywCO1ll6YgKJSPbk4XrmHR7UHhTQIggYQ4i2ao8oRzpx6K0UnpETv6EBdh9UUzQ JN45QLP/Ibmk3yWo0n477M0DPwAAdRoPv5KevQcVJ5Y97/N8iJy2GOVN/4/HsxeFutGN A5pgP98BYa0E7SJ715Tjl0d/MK0mRIVIwEBU1+0lmBR0UfJIr6QpEbCszjNI0cWG6yyR 4G3qadteVdz0QNz5Zf1R26fG10hX4quQuvCC7AN55DmIEOLSeeycZnm74sxLi2o+D4Kt C6Jg== Sender: isar-users@googlegroups.com X-Forwarded-Encrypted: i=2; AJvYcCW6PxKGZP9KyEtGanPYPDnIhZlhg2YW/GrWMYyrVvHV0xp4FYWsGhl1rs0Z4YFjs3wCbi1xPJc=@isar-build.org X-Gm-Message-State: AOJu0YyRWr0HvYzLPDriYDzTXfst9PZUt2md3ubEOBnKM0MHUXkSqn7a HxV+rmGg5YR3XR/iXhqIOCxyq+6R/uQyuziZiI9Ajdiamwst2p+F X-Google-Smtp-Source: AGHT+IHnKuwfH+LOGpeGvyE/HiwlqHzXiabHpLR/Bs1vwrtVh49s1pWrlrpV8U8PaxhL0baL5MS6zQ== X-Received: by 2002:a5d:6c68:0:b0:381:b68f:d14b with SMTP id ffacd0b85a97d-381bea1bfc4mr18994930f8f.45.1730881288945; Wed, 06 Nov 2024 00:21:28 -0800 (PST) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a05:600c:1c81:b0:431:405a:f96f with SMTP id 5b1f17b1804b1-4327b6d53d0ls34604405e9.0.-pod-prod-02-eu; Wed, 06 Nov 2024 00:21:27 -0800 (PST) X-Received: by 2002:a05:600c:a09:b0:42c:b52b:4335 with SMTP id 5b1f17b1804b1-4327b6fda39mr185824965e9.10.1730881286641; Wed, 06 Nov 2024 00:21:26 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1730881286; cv=none; d=google.com; s=arc-20240605; b=D/Fo53aMXCtfKVyijoXEyZTcN/L7Dq9v3wFYjiTjeBfOAeBpIUS/NnTZO1EWPYxTsb QcSvdLhaOtbMjOwHzuXu65YZHRTFjPGRoYMmjMWpNe4uubqOW84r4IoGSmPUr1yXK9SK yBEWMkJDB+3+MEgrTxRYEfYQ5wlxWhkzSTFxZmu7yAwmDBHcK+uIYOt4+gNPRWHyUBYE l019DJyuEt0VuqzhSCRVObJF3Gf5xjOf4ucNNCqMQXzfJk1j3F9K9lcPWvgTxYI3M7LH dsAkondAmzzNBOZXr+ATn9IQA4wLxjg9jg19J+Z09ht6xorDtjezrvQVBXjkbTiDgovM MLyw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20240605; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=c6mda56arW427J628WW74gm4Nr7s//H874fbor8aOGc=; fh=/h9QQkzJ8EboVkWg45aWwpaUro6WMavIVd2OhN45RtE=; b=huRpDvxYwK4BiBzVmA21Fg5Ze37b3aLgUw7gw+Aits1hZPAGzXqa0Dc2Idadd123Zy lWar/l6Wiyaxhv0RZlf1Q4Sjzu0HD/nm6YCWTj1XmPTfCrHsy23bz/+yWhoHkQJ14Bwd fQFJ+lWD1b3bYeeCbfgWFHLk9pJbfm4kJJRUxUR9X95EPvmXhwArSdK/zMF+j3R7Q6Vf Jtk92PnzEbbNCWukT1Iq2Sp2iMooWGsOcbkXARLgP8HdtP1VIJ+sglS3IRchsAfjVzPM sTr9CO2lR/X10AXjYWeVIkGwPsxtGTbchHd5tkYq1y1p4yy4Bm6N3kq8lHejNqFjPHEc Bb1A==; dara=google.com ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=amikan@ilbers.de Received: from shymkent.ilbers.de (shymkent.ilbers.de. [85.214.156.166]) by gmr-mx.google.com with ESMTPS id 5b1f17b1804b1-432a368745esi1572995e9.1.2024.11.06.00.21.26 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Wed, 06 Nov 2024 00:21:26 -0800 (PST) Received-SPF: pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) client-ip=85.214.156.166; Received: from user-B660.promwad.corp ([159.148.83.114]) (authenticated bits=0) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPSA id 4A68LOjq016956 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 6 Nov 2024 09:21:25 +0100 From: Anton Mikanovich To: isar-users@googlegroups.com Cc: Anton Mikanovich Subject: [PATCH v11 1/8] isar-bootstrap: Move common parts to bbclass Date: Wed, 6 Nov 2024 10:21:10 +0200 Message-Id: <20241106082117.1089554-2-amikan@ilbers.de> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20241106082117.1089554-1-amikan@ilbers.de> References: <20241106082117.1089554-1-amikan@ilbers.de> MIME-Version: 1.0 X-Spam-Status: No, score=-4.6 required=5.0 tests=DKIMWL_WL_MED,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_EF,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,RCVD_IN_DNSWL_BLOCKED,RCVD_IN_MSPIKE_H2, RCVD_IN_RP_CERTIFIED,RCVD_IN_RP_RNBL,RCVD_IN_RP_SAFE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on shymkent.ilbers.de X-Original-Sender: amikan@ilbers.de X-Original-Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=amikan@ilbers.de Precedence: list Mailing-list: list isar-users@googlegroups.com; contact isar-users+owners@googlegroups.com List-ID: X-Spam-Checked-In-Group: isar-users@googlegroups.com X-Google-Group-Id: 914930254986 List-Post: , List-Help: , List-Archive: , List-Unsubscribe: , X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= To allow implementing other bootstrap recipes move some functionality supposed to be reusable to bbclass. Signed-off-by: Anton Mikanovich --- meta/classes/bootstrap.bbclass | 225 ++++++++++++++++++ .../isar-bootstrap/isar-bootstrap.inc | 219 +---------------- 2 files changed, 227 insertions(+), 217 deletions(-) create mode 100644 meta/classes/bootstrap.bbclass diff --git a/meta/classes/bootstrap.bbclass b/meta/classes/bootstrap.bbclass new file mode 100644 index 00000000..870465a9 --- /dev/null +++ b/meta/classes/bootstrap.bbclass @@ -0,0 +1,225 @@ +# This software is a part of Isar. +# Copyright (c) Siemens AG, 2018-2021 +# Copyright (C) 2024 ilbers GmbH +# +# SPDX-License-Identifier: MIT + +FILESEXTRAPATHS:append = ":${BBPATH}" + +SRC_URI = " \ + file://locale \ + file://chroot-setup.sh \ + ${DISTRO_BOOTSTRAP_KEYS} \ + ${THIRD_PARTY_APT_KEYS}" + +BOOTSTRAP_FOR_HOST ?= "0" + +APTPREFS = "${WORKDIR}/apt-preferences" +APTSRCS = "${WORKDIR}/apt-sources" +APTSRCS_INIT = "${WORKDIR}/apt-sources-init" +DISTRO_BOOTSTRAP_KEYFILES = "" +THIRD_PARTY_APT_KEYFILES = "" +DISTRO_BOOTSTRAP_KEYS ?= "" +THIRD_PARTY_APT_KEYS ?= "" +DEPLOY_ISAR_BOOTSTRAP ?= "" +DISTRO_BOOTSTRAP_BASE_PACKAGES ??= "" +DISTRO_VARS_PREFIX ?= "${@'HOST_' if bb.utils.to_boolean(d.getVar('BOOTSTRAP_FOR_HOST')) else ''}" +BOOTSTRAP_DISTRO = "${@d.getVar('HOST_DISTRO' if bb.utils.to_boolean(d.getVar('BOOTSTRAP_FOR_HOST')) else 'DISTRO')}" +BOOTSTRAP_BASE_DISTRO = "${@d.getVar('HOST_BASE_DISTRO' if bb.utils.to_boolean(d.getVar('BOOTSTRAP_FOR_HOST')) else 'BASE_DISTRO')}" +ISAR_APT_SNAPSHOT_DATE ?= "${@ get_isar_apt_snapshot_date(d)}" + +python () { + distro_bootstrap_keys = (d.getVar("DISTRO_BOOTSTRAP_KEYS") or "").split() + third_party_apt_keys = (d.getVar("THIRD_PARTY_APT_KEYS") or "").split() + topdir = d.getVar("TOPDIR") + + # The cached repo key can be both for bootstrapping and apt package + # installation afterwards. However, bootstrap will include the key into + # the rootfs automatically thus the right place is distro_bootstrap_keys. + + if bb.utils.to_boolean(d.getVar('ISAR_USE_CACHED_BASE_REPO')): + own_pub_key = d.getVar("BASE_REPO_KEY") + if own_pub_key: + distro_bootstrap_keys += own_pub_key.split() + + for key in distro_bootstrap_keys: + fetcher = bb.fetch2.Fetch([key], d) + filename = os.path.relpath(fetcher.localpath(key), topdir) + d.appendVar("DISTRO_BOOTSTRAP_KEYFILES", " ${TOPDIR}/%s" % filename) + + for key in third_party_apt_keys: + fetcher = bb.fetch2.Fetch([key], d) + filename = os.path.relpath(fetcher.localpath(key), topdir) + d.appendVar("THIRD_PARTY_APT_KEYFILES", " ${TOPDIR}/%s" % filename) + + distro_apt_sources = get_aptsources_list(d) + for file in distro_apt_sources: + d.appendVar("SRC_URI", " file://%s" % file) + + distro_apt_preferences = d.getVar(d.getVar("DISTRO_VARS_PREFIX") + "DISTRO_APT_PREFERENCES") or "" + for file in distro_apt_preferences.split(): + d.appendVar("SRC_URI", " file://%s" % file) +} + +def aggregate_files(d, file_list, file_out): + import shutil + + with open(file_out, "wb") as out_fd: + for entry in file_list: + entry_real = bb.parse.resolve_file(entry, d) + with open(entry_real, "rb") as in_fd: + shutil.copyfileobj(in_fd, out_fd, 1024*1024*10) + out_fd.write("\n".encode()) + +def parse_aptsources_list_line(source_list_line): + import re + + s = source_list_line.strip() + + if not s or s.startswith("#"): + return None + + type, s = re.split("\s+", s, maxsplit=1) + if type not in ["deb", "deb-src"]: + return None + + options = "" + options_match = re.match("\[\s*(\S+=\S+(?=\s))*\s*(\S+=\S+)\s*\]\s+", s) + if options_match: + options = options_match.group(0).strip() + s = s[options_match.end():] + + source, s = re.split("\s+", s, maxsplit=1) + + if s.startswith("/"): + suite = "" + else: + suite, s = re.split("\s+", s, maxsplit=1) + + components = " ".join(s.split()) + + return [type, options, source, suite, components] + +def get_isar_apt_snapshot_date(d): + import time + source_date_epoch = d.getVar('ISAR_APT_SNAPSHOT_TIMESTAMP') + return time.strftime('%Y%m%dT%H%M%SZ', time.gmtime(int(source_date_epoch))) + +def get_apt_source_mirror(d, aptsources_entry_list): + import re + + # this is executed during parsing. No error checking possible + use_snapshot = bb.utils.to_boolean(d.getVar('ISAR_USE_APT_SNAPSHOT')) + snapshot_mirror = d.getVar('DISTRO_APT_SNAPSHOT_PREMIRROR') + if bb.utils.to_boolean(d.getVar('ISAR_USE_CACHED_BASE_REPO')): + premirrors = "\S* file://${REPO_BASE_DIR}/${BOOTSTRAP_BASE_DISTRO}\n" + elif use_snapshot and snapshot_mirror: + premirrors = snapshot_mirror + else: + premirrors = d.getVar('DISTRO_APT_PREMIRRORS') or "" + mirror_list = [entry.split() + for entry in premirrors.split('\\n') + if any(entry)] + + for regex, replace in mirror_list: + match = re.search(regex, aptsources_entry_list[2]) + + if match: + new_aptsources_entry_list = aptsources_entry_list.copy() + new_aptsources_entry_list[2] = re.sub(regex, replace, + aptsources_entry_list[2], + count = 1) + if use_snapshot: + new_aptsources_entry_list[1] = "[check-valid-until=no]" + return new_aptsources_entry_list + + return aptsources_entry_list + +def aggregate_aptsources_list(d, file_list, file_out): + import shutil + + with open(file_out, "wb") as out_fd: + for entry in file_list: + entry_real = bb.parse.resolve_file(entry, d) + with open(entry_real, "r") as in_fd: + for line in in_fd: + parsed = parse_aptsources_list_line(line) + if parsed: + parsed = get_apt_source_mirror(d, parsed) + out_fd.write(" ".join(parsed).encode()) + else: + out_fd.write(line.encode()) + out_fd.write("\n".encode()) + out_fd.write("\n".encode()) + +def get_aptsources_list(d): + import errno + from collections import OrderedDict + apt_sources_var = d.getVar("DISTRO_VARS_PREFIX") + "DISTRO_APT_SOURCES" + apt_sources_list = list(OrderedDict.fromkeys((d.getVar(apt_sources_var) or "").split())) + for p in apt_sources_list: + try: + bb.parse.resolve_file(p, d) + except FileNotFoundError as e: + bb.fatal(os.strerror(errno.ENOENT) + ' "' + p + '"') + return apt_sources_list + +def generate_distro_sources(d): + apt_sources_list = get_aptsources_list(d) + for entry in apt_sources_list: + with open(bb.parse.resolve_file(entry, d), "r") as in_fd: + for line in in_fd: + parsed = parse_aptsources_list_line(line) + if parsed: + parsed = get_apt_source_mirror(d, parsed) + yield parsed + +def get_distro_primary_source_entry(d): + for source in generate_distro_sources(d): + if source[0] == "deb": + return source[2:] + bb.fatal('Invalid apt sources list') + +def get_distro_source(d): + return get_distro_primary_source_entry(d)[0] + +def get_distro_suite(d): + return get_distro_primary_source_entry(d)[1] + +def get_distro_components_argument(d): + components = get_distro_primary_source_entry(d)[2] + if components and components.strip(): + return "--components=" + ",".join(components.split()) + else: + return "" + +do_apt_config_prepare[dirs] = "${WORKDIR}" +do_apt_config_prepare[vardeps] += " \ + APTPREFS \ + ${DISTRO_VARS_PREFIX}DISTRO_APT_PREFERENCES \ + DEBDISTRONAME \ + APTSRCS \ + ${DISTRO_VARS_PREFIX}DISTRO_APT_SOURCES \ + DEPLOY_ISAR_BOOTSTRAP \ + ${@'DISTRO_APT_SNAPSHOT_PREMIRROR' if bb.utils.to_boolean(d.getVar('ISAR_USE_APT_SNAPSHOT')) else ''} \ + " +python do_apt_config_prepare() { + apt_preferences_out = d.getVar("APTPREFS") + apt_preferences_list = ( + d.getVar(d.getVar("DISTRO_VARS_PREFIX") + "DISTRO_APT_PREFERENCES") or "" + ).split() + aggregate_files(d, apt_preferences_list, apt_preferences_out) + + apt_sources_out = d.getVar("APTSRCS") + apt_sources_init_out = d.getVar("APTSRCS_INIT") + apt_sources_list = get_aptsources_list(d) + + aggregate_files(d, apt_sources_list, apt_sources_init_out) + aggregate_aptsources_list(d, apt_sources_list, apt_sources_out) +} +addtask apt_config_prepare before do_bootstrap after do_unpack + +CLEANFUNCS = "clean_deploy" +clean_deploy() { + rm -f "${DEPLOY_ISAR_BOOTSTRAP}" +} diff --git a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc index e155aade..0e5ab57d 100644 --- a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc +++ b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc @@ -5,191 +5,20 @@ # # SPDX-License-Identifier: MIT +inherit bootstrap + LICENSE = "gpl-2.0" LIC_FILES_CHKSUM = "file://${LAYERDIR_core}/licenses/COPYING.GPLv2;md5=751419260aa954499f7abaabaa882bbe" -FILESPATH:prepend := "${THISDIR}/files:" -SRC_URI = " \ - file://locale \ - file://chroot-setup.sh \ - ${DISTRO_BOOTSTRAP_KEYS} \ - ${THIRD_PARTY_APT_KEYS}" PV = "1.0" -BOOTSTRAP_FOR_HOST ?= "0" - DEBOOTSTRAP ?= "qemu-debootstrap" ROOTFSDIR = "${WORKDIR}/rootfs" -APTPREFS = "${WORKDIR}/apt-preferences" -APTSRCS = "${WORKDIR}/apt-sources" -APTSRCS_INIT = "${WORKDIR}/apt-sources-init" -DISTRO_BOOTSTRAP_KEYFILES = "" -THIRD_PARTY_APT_KEYFILES = "" -DISTRO_BOOTSTRAP_KEYS ?= "" -THIRD_PARTY_APT_KEYS ?= "" -DEPLOY_ISAR_BOOTSTRAP ?= "" DISTRO_BOOTSTRAP_BASE_PACKAGES = "locales" DISTRO_BOOTSTRAP_BASE_PACKAGES:append:gnupg = ",gnupg" DISTRO_BOOTSTRAP_BASE_PACKAGES:append:https-support = ",ca-certificates" -DISTRO_VARS_PREFIX ?= "${@'HOST_' if bb.utils.to_boolean(d.getVar('BOOTSTRAP_FOR_HOST')) else ''}" -BOOTSTRAP_DISTRO = "${@d.getVar('HOST_DISTRO' if bb.utils.to_boolean(d.getVar('BOOTSTRAP_FOR_HOST')) else 'DISTRO')}" -BOOTSTRAP_BASE_DISTRO = "${@d.getVar('HOST_BASE_DISTRO' if bb.utils.to_boolean(d.getVar('BOOTSTRAP_FOR_HOST')) else 'BASE_DISTRO')}" -FILESEXTRAPATHS:append = ":${BBPATH}" -ISAR_APT_SNAPSHOT_DATE ?= "${@ get_isar_apt_snapshot_date(d)}" inherit deb-dl-dir -python () { - distro_bootstrap_keys = (d.getVar("DISTRO_BOOTSTRAP_KEYS") or "").split() - third_party_apt_keys = (d.getVar("THIRD_PARTY_APT_KEYS") or "").split() - topdir = d.getVar("TOPDIR") - - # The cached repo key can be both for bootstrapping and apt package - # installation afterwards. However, debootstrap will include the key into - # the rootfs automatically thus the right place is distro_bootstrap_keys. - - if bb.utils.to_boolean(d.getVar('ISAR_USE_CACHED_BASE_REPO')): - own_pub_key = d.getVar("BASE_REPO_KEY") - if own_pub_key: - distro_bootstrap_keys += own_pub_key.split() - - for key in distro_bootstrap_keys: - fetcher = bb.fetch2.Fetch([key], d) - filename = os.path.relpath(fetcher.localpath(key), topdir) - d.appendVar("DISTRO_BOOTSTRAP_KEYFILES", " ${TOPDIR}/%s" % filename) - - for key in third_party_apt_keys: - fetcher = bb.fetch2.Fetch([key], d) - filename = os.path.relpath(fetcher.localpath(key), topdir) - d.appendVar("THIRD_PARTY_APT_KEYFILES", " ${TOPDIR}/%s" % filename) - - distro_apt_sources = get_aptsources_list(d) - for file in distro_apt_sources: - d.appendVar("SRC_URI", " file://%s" % file) - - distro_apt_preferences = d.getVar(d.getVar("DISTRO_VARS_PREFIX") + "DISTRO_APT_PREFERENCES") or "" - for file in distro_apt_preferences.split(): - d.appendVar("SRC_URI", " file://%s" % file) -} - -def aggregate_files(d, file_list, file_out): - import shutil - - with open(file_out, "wb") as out_fd: - for entry in file_list: - entry_real = bb.parse.resolve_file(entry, d) - with open(entry_real, "rb") as in_fd: - shutil.copyfileobj(in_fd, out_fd, 1024*1024*10) - out_fd.write("\n".encode()) - -def parse_aptsources_list_line(source_list_line): - import re - - s = source_list_line.strip() - - if not s or s.startswith("#"): - return None - - type, s = re.split("\s+", s, maxsplit=1) - if type not in ["deb", "deb-src"]: - return None - - options = "" - options_match = re.match("\[\s*(\S+=\S+(?=\s))*\s*(\S+=\S+)\s*\]\s+", s) - if options_match: - options = options_match.group(0).strip() - s = s[options_match.end():] - - source, s = re.split("\s+", s, maxsplit=1) - - if s.startswith("/"): - suite = "" - else: - suite, s = re.split("\s+", s, maxsplit=1) - - components = " ".join(s.split()) - - return [type, options, source, suite, components] - -def get_isar_apt_snapshot_date(d): - import time - source_date_epoch = d.getVar('ISAR_APT_SNAPSHOT_TIMESTAMP') - return time.strftime('%Y%m%dT%H%M%SZ', time.gmtime(int(source_date_epoch))) - -def get_apt_source_mirror(d, aptsources_entry_list): - import re - - # this is executed during parsing. No error checking possible - use_snapshot = bb.utils.to_boolean(d.getVar('ISAR_USE_APT_SNAPSHOT')) - snapshot_mirror = d.getVar('DISTRO_APT_SNAPSHOT_PREMIRROR') - if bb.utils.to_boolean(d.getVar('ISAR_USE_CACHED_BASE_REPO')): - premirrors = "\S* file://${REPO_BASE_DIR}/${BOOTSTRAP_BASE_DISTRO}\n" - elif use_snapshot and snapshot_mirror: - premirrors = snapshot_mirror - else: - premirrors = d.getVar('DISTRO_APT_PREMIRRORS') or "" - mirror_list = [entry.split() - for entry in premirrors.split('\\n') - if any(entry)] - - for regex, replace in mirror_list: - match = re.search(regex, aptsources_entry_list[2]) - - if match: - new_aptsources_entry_list = aptsources_entry_list.copy() - new_aptsources_entry_list[2] = re.sub(regex, replace, - aptsources_entry_list[2], - count = 1) - if use_snapshot: - new_aptsources_entry_list[1] = "[check-valid-until=no]" - return new_aptsources_entry_list - - return aptsources_entry_list - -def aggregate_aptsources_list(d, file_list, file_out): - import shutil - - with open(file_out, "wb") as out_fd: - for entry in file_list: - entry_real = bb.parse.resolve_file(entry, d) - with open(entry_real, "r") as in_fd: - for line in in_fd: - parsed = parse_aptsources_list_line(line) - if parsed: - parsed = get_apt_source_mirror(d, parsed) - out_fd.write(" ".join(parsed).encode()) - else: - out_fd.write(line.encode()) - out_fd.write("\n".encode()) - out_fd.write("\n".encode()) - -def get_aptsources_list(d): - import errno - from collections import OrderedDict - apt_sources_var = d.getVar("DISTRO_VARS_PREFIX") + "DISTRO_APT_SOURCES" - apt_sources_list = list(OrderedDict.fromkeys((d.getVar(apt_sources_var) or "").split())) - for p in apt_sources_list: - try: - bb.parse.resolve_file(p, d) - except FileNotFoundError as e: - bb.fatal(os.strerror(errno.ENOENT) + ' "' + p + '"') - return apt_sources_list - -def generate_distro_sources(d): - apt_sources_list = get_aptsources_list(d) - for entry in apt_sources_list: - with open(bb.parse.resolve_file(entry, d), "r") as in_fd: - for line in in_fd: - parsed = parse_aptsources_list_line(line) - if parsed: - parsed = get_apt_source_mirror(d, parsed) - yield parsed - -def get_distro_primary_source_entry(d): - for source in generate_distro_sources(d): - if source[0] == "deb": - return source[2:] - bb.fatal('Invalid apt sources list') - def get_distro_have_https_source(d): return any(source[2].startswith("https://") for source in generate_distro_sources(d)) @@ -211,19 +40,6 @@ def get_distro_needs_gpg_support(d): OVERRIDES:append = ":${@get_distro_needs_gpg_support(d)}" -def get_distro_source(d): - return get_distro_primary_source_entry(d)[0] - -def get_distro_suite(d): - return get_distro_primary_source_entry(d)[1] - -def get_distro_components_argument(d): - components = get_distro_primary_source_entry(d)[2] - if components and components.strip(): - return "--components=" + ",".join(components.split()) - else: - return "" - APT_KEYS_DIR = "${WORKDIR}/aptkeys" DISTRO_BOOTSTRAP_KEYRING = "${WORKDIR}/distro-keyring.gpg" @@ -247,32 +63,6 @@ do_generate_keyrings() { } addtask generate_keyrings before do_build after do_unpack -do_apt_config_prepare[dirs] = "${WORKDIR}" -do_apt_config_prepare[vardeps] += " \ - APTPREFS \ - ${DISTRO_VARS_PREFIX}DISTRO_APT_PREFERENCES \ - DEBDISTRONAME \ - APTSRCS \ - ${DISTRO_VARS_PREFIX}DISTRO_APT_SOURCES \ - DEPLOY_ISAR_BOOTSTRAP \ - ${@'DISTRO_APT_SNAPSHOT_PREMIRROR' if bb.utils.to_boolean(d.getVar('ISAR_USE_APT_SNAPSHOT')) else ''} \ - " -python do_apt_config_prepare() { - apt_preferences_out = d.getVar("APTPREFS") - apt_preferences_list = ( - d.getVar(d.getVar("DISTRO_VARS_PREFIX") + "DISTRO_APT_PREFERENCES") or "" - ).split() - aggregate_files(d, apt_preferences_list, apt_preferences_out) - - apt_sources_out = d.getVar("APTSRCS") - apt_sources_init_out = d.getVar("APTSRCS_INIT") - apt_sources_list = get_aptsources_list(d) - - aggregate_files(d, apt_sources_list, apt_sources_init_out) - aggregate_aptsources_list(d, apt_sources_list, apt_sources_out) -} -addtask apt_config_prepare before do_bootstrap after do_unpack - def get_host_release(): import platform rel = platform.release() @@ -448,8 +238,3 @@ python do_bootstrap_setscene() { addtask do_bootstrap_setscene do_bootstrap_setscene[dirs] = "${DEPLOY_DIR_BOOTSTRAP}" - -CLEANFUNCS = "clean_deploy" -clean_deploy() { - rm -f "${DEPLOY_ISAR_BOOTSTRAP}" -} From patchwork Wed Nov 6 08:21:11 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anton Mikanovich X-Patchwork-Id: 3895 Return-Path: Received: from shymkent.ilbers.de ([unix socket]) by shymkent (Cyrus 2.5.10-Debian-2.5.10-3+deb9u2) with LMTPA; Wed, 06 Nov 2024 09:21:37 +0100 X-Sieve: CMU Sieve 2.4 Received: from mail-ed1-f56.google.com (mail-ed1-f56.google.com [209.85.208.56]) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPS id 4A68LaSL017024 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Wed, 6 Nov 2024 09:21:36 +0100 Received: by mail-ed1-f56.google.com with SMTP id 4fb4d7f45d1cf-5c999f254aesf5073713a12.0 for ; Wed, 06 Nov 2024 00:21:36 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1730881290; cv=pass; d=google.com; s=arc-20240605; b=MOhcbxpXaC/sxf5vhJftZ34mtedIr6gcr/YwPpicBxJkmXtFdqM60egV6gyqCn+d89 UCsE5K8fbtDnGKzyV6B10zaE5P2KFOjYJkVmaK40gktNke0yqcbg2hAxgZdmBuqnhCn8 Ap0cw5z1/2bi5pxrph5AdCbuMKAhfW3RCL5BXuBzuF/xQtvx8dmJYm56Bj+2PkK6grpW wcVwMCFHXSQT9HNWy1auARSfCyr1bmHwyAmOfaUCIm1jtpBDts3g9OAzFgaDyvguhhpb /joK8Rpn5+sx0n29B5IxfOVRUOh45Xq1DAKE9tSuCEo+R+XbqxrgmaFX8+r/k/ZpNbMW Q02A== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20240605; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:dkim-signature; bh=VIKJJV6HXCvXFAP+vlK9MEbCCDHUQmekPzgj9Ydx9rw=; fh=s8qSy9jn0EhxwTd0eUsU+poBR3CZANallfjeQKXFeSM=; b=b2Z1KaCycQ/BLJgL6LSMYkGqpMI0ZSy483NDWjZaQg3qdSpMzMIUrCUpmWcDyIxeGm rm6CHU1Rrt64SpeH5fvOQZa5KbQoOtBMDNBOkaZIrf8OtWd+Ni7YV40y9d0GMd44FJ7h tDFzOfmgj/6Vzqy6gJNVPvaH7buS7aOapg/Grh66i40SwI1LLy/meq59XkFm2D7h9/pd Z6lPMHF6XgnbzuTOcUiRic9XHslxQ6MinsFMpYw3JIeA5HzEiPn4bHTJiqB+9L59005W hBvKcfVAk8BhcsZeBxQ5s/ZXwZSQTkCiqpZhje0rQGNY7Y4YCJgd1kgLl3PbppNw4IkP /H5w==; darn=isar-build.org ARC-Authentication-Results: i=2; gmr-mx.google.com; spf=pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=amikan@ilbers.de DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlegroups.com; s=20230601; t=1730881290; x=1731486090; darn=isar-build.org; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:x-original-authentication-results :x-original-sender:mime-version:references:in-reply-to:message-id :date:subject:cc:to:from:sender:from:to:cc:subject:date:message-id :reply-to; bh=VIKJJV6HXCvXFAP+vlK9MEbCCDHUQmekPzgj9Ydx9rw=; b=NH98mbE89j00YyqYOpyKmb2x/tZp4fXyLcsH6XpSipsulP4OYLlTPuLPHyQ4p8ZLm3 AzRaZi+Atw81Mb9P4m031giBk/HTcPJg/VSq+tMcXkfGOYMvusEX4FgeTnvnQ9zvMhSw GRpuwzDDlwqZb5WEcsxxapP36cOxMgesa3jE+vFxcz7IwJ3xv1IkAIuaH5aXq8AFHxEv 6/Fxn/Ep/KABkZ/NswP8FNnB23VbtcpbLgfIjtjRkb0UfhG3ajSJs0aPC8CfZ+MDyJyp 33s/xNpYN9O89GEBF4lAsCH5+EBS8slTJo6Xi2CSgKwyjAqsj1NgbXrCPt4C2ZejFEiY h9Jg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1730881290; x=1731486090; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :x-spam-checked-in-group:list-id:mailing-list:precedence :x-original-authentication-results:x-original-sender:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :x-beenthere:x-gm-message-state:sender:from:to:cc:subject:date :message-id:reply-to; bh=VIKJJV6HXCvXFAP+vlK9MEbCCDHUQmekPzgj9Ydx9rw=; b=od75NBAiDksc5hujBBL8+U/6+4oYZcE7HUkAlctYlq7FXTydX8UjlLbP6D6xWzMwb5 Uixx7ITcYiAjFh/jFVjjpS+ZqkAK9nCRSChe4MF80lrdPvKOddProv3NPiFcxnn/u63T cGTu7Xu1KxvnY8+xx800QxRMXHVig+enMrOj+sWZpq+sGgxEuHiyiN7DFPYi4H7DOJnM RwniTfqFEmA2yTFZLW3EIQ9/uZ35x0EHpf2DWyd/a6kuvXTbSpDh97ZlPzfsaK2rQISw L3MeVFpt8uEWtJes1Cuy+DHG/zMleoXuU8GM08G7eGiQSWnsNM6KoL/rMQRGAVN+T00q LGyA== Sender: isar-users@googlegroups.com X-Forwarded-Encrypted: i=2; AJvYcCXC4JAZ9DxGCZYBRikYL1NUePynvr5ZVwjWYX3w+TVU2jitpBh0SNxsnUDJ0rCDPA9tkqNSCzQ=@isar-build.org X-Gm-Message-State: AOJu0YycDmrJpT9qDG74qWJuNluZY8RSiCNWjrzCEN1vfBv6XfDr7oDU CSuAQWpIC6l8PmKnbr9iM6B1X6ju4xSVGyiMgbFuvQ/iTNShy/pb X-Google-Smtp-Source: AGHT+IFRkXvUlQPUop8qUI4UxFu3Hg990WwXS+cFYYL5YUJbYwVLey9HRQLOYvfV4bCJdAdOqOaluw== X-Received: by 2002:a05:6402:2708:b0:5ce:dd38:ed6a with SMTP id 4fb4d7f45d1cf-5cedd38eeacmr6034049a12.25.1730881289760; Wed, 06 Nov 2024 00:21:29 -0800 (PST) X-BeenThere: isar-users@googlegroups.com Received: by 2002:aa7:d9d7:0:b0:5ce:e5c9:38e1 with SMTP id 4fb4d7f45d1cf-5cee5c943a0ls87108a12.2.-pod-prod-06-eu; Wed, 06 Nov 2024 00:21:27 -0800 (PST) X-Received: by 2002:a05:6402:3586:b0:5c8:bb09:b417 with SMTP id 4fb4d7f45d1cf-5ceb91254b4mr13371100a12.0.1730881287397; Wed, 06 Nov 2024 00:21:27 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1730881287; cv=none; d=google.com; s=arc-20240605; b=gsggvs2l9ioG8aNd01kuwZ7jqc5pBqrLT4vKz3pKNSDrdl/c8/80INpByMQh2+Wb4U 6m/PYw9B7J2/lUPF8eAt0aD6z7Kgkd1uRQepg17p6p90AVbhHlrn5toLVnisOdudMCUO m02lyjMNrQKSQE1/Ztdm556LhpNMvTajXxbL4c/WKdLCA+1tF70z9mJERvQ+hhQvWmt8 XJ0uAsIgoKyag/89zfp4jFpsJk7X4HFmjYW/uK9SDwVQatL9RIiWxNHr+9pU758SKubv lRrpgq8vIsQ5UlrNnO7Ir8+JBMR2y3yoMbDuDgVAgWuNS4gVNn1B9w2Ya4a7D4iFEwxo yZgg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20240605; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=CmT+JaKH1aoTpSnWASTf6Cd+qZ1omMoxyhKg1qXhK8Q=; fh=/h9QQkzJ8EboVkWg45aWwpaUro6WMavIVd2OhN45RtE=; b=Abq7yU+BO/Ji2r9SGrCtf7hYpR0pPzZ6UZIcOYUqfE6Jk9mYL9cA6muiQqSUIzT9Q0 vWi6XECaS0NyX4JoO4Tg1OyWLT/uCfOwnf9MsTtKK8smjnPNHW6ZeTAlANV4f3a8CkX9 omAuQkWGqUet/mmBl3Xln7UtmtO592wan/FcY5W7lPupf11q6nYQ81GJ+LzM8AWiyVNe 9GNvFzUAdM2A9IheNDHEW2JCbGPQposgA4Y3xMFREAk70MtgI/UEH4Y0uFugnlJxYB2c LWnkKeGhttYBpOhA6NZPOEq5PlBqYYwzlLzpYXL00ydLx590DxzijmSukCDTFx7Bb7D4 SB+g==; dara=google.com ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=amikan@ilbers.de Received: from shymkent.ilbers.de (shymkent.ilbers.de. [85.214.156.166]) by gmr-mx.google.com with ESMTPS id 4fb4d7f45d1cf-5cee6b05269si128497a12.3.2024.11.06.00.21.27 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Wed, 06 Nov 2024 00:21:27 -0800 (PST) Received-SPF: pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) client-ip=85.214.156.166; Received: from user-B660.promwad.corp ([159.148.83.114]) (authenticated bits=0) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPSA id 4A68LOjr016956 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 6 Nov 2024 09:21:26 +0100 From: Anton Mikanovich To: isar-users@googlegroups.com Cc: Anton Mikanovich Subject: [PATCH v11 2/8] meta: Add mmdebstrap recipe Date: Wed, 6 Nov 2024 10:21:11 +0200 Message-Id: <20241106082117.1089554-3-amikan@ilbers.de> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20241106082117.1089554-1-amikan@ilbers.de> References: <20241106082117.1089554-1-amikan@ilbers.de> MIME-Version: 1.0 X-Spam-Status: No, score=-4.6 required=5.0 tests=DKIMWL_WL_MED,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_EF,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,RCVD_IN_DNSWL_BLOCKED,RCVD_IN_MSPIKE_H2, RCVD_IN_RP_CERTIFIED,RCVD_IN_RP_RNBL,RCVD_IN_RP_SAFE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on shymkent.ilbers.de X-Original-Sender: amikan@ilbers.de X-Original-Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=amikan@ilbers.de Precedence: list Mailing-list: list isar-users@googlegroups.com; contact isar-users+owners@googlegroups.com List-ID: X-Spam-Checked-In-Group: isar-users@googlegroups.com X-Google-Group-Id: 914930254986 List-Post: , List-Help: , List-Archive: , List-Unsubscribe: , X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= It can be used as debootstrap alternative for rootfs prepare. Internally, it uses apt and allows to bootstrap the distro from multiple repositories. chroot-setup.sh and locale are copied from debootstrap recipe. Signed-off-by: Anton Mikanovich --- meta/classes/bootstrap.bbclass | 1 + .../isar-mmdebstrap/isar-mmdebstrap-host.bb | 17 ++ .../isar-mmdebstrap/isar-mmdebstrap-target.bb | 12 + .../isar-mmdebstrap/isar-mmdebstrap.inc | 235 ++++++++++++++++++ 4 files changed, 265 insertions(+) create mode 100644 meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap-host.bb create mode 100644 meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap-target.bb create mode 100644 meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc diff --git a/meta/classes/bootstrap.bbclass b/meta/classes/bootstrap.bbclass index 870465a9..f5b92808 100644 --- a/meta/classes/bootstrap.bbclass +++ b/meta/classes/bootstrap.bbclass @@ -26,6 +26,7 @@ DISTRO_BOOTSTRAP_BASE_PACKAGES ??= "" DISTRO_VARS_PREFIX ?= "${@'HOST_' if bb.utils.to_boolean(d.getVar('BOOTSTRAP_FOR_HOST')) else ''}" BOOTSTRAP_DISTRO = "${@d.getVar('HOST_DISTRO' if bb.utils.to_boolean(d.getVar('BOOTSTRAP_FOR_HOST')) else 'DISTRO')}" BOOTSTRAP_BASE_DISTRO = "${@d.getVar('HOST_BASE_DISTRO' if bb.utils.to_boolean(d.getVar('BOOTSTRAP_FOR_HOST')) else 'BASE_DISTRO')}" +BOOTSTRAP_DISTRO_ARCH = "${@d.getVar('HOST_ARCH' if bb.utils.to_boolean(d.getVar('BOOTSTRAP_FOR_HOST')) else 'DISTRO_ARCH')}" ISAR_APT_SNAPSHOT_DATE ?= "${@ get_isar_apt_snapshot_date(d)}" python () { diff --git a/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap-host.bb b/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap-host.bb new file mode 100644 index 00000000..66c8d11e --- /dev/null +++ b/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap-host.bb @@ -0,0 +1,17 @@ +# Minimal host Debian root file system +# +# This software is a part of Isar. +# Copyright (C) 2024 ilbers GmbH +# +# SPDX-License-Identifier: MIT + +Description = "Minimal host Debian root file system" + +DEPLOY_ISAR_BOOTSTRAP = "${DEPLOY_DIR_BOOTSTRAP}/${HOST_DISTRO}-host_${DISTRO}-${DISTRO_ARCH}" + +BOOTSTRAP_FOR_HOST = "1" + +require isar-mmdebstrap.inc + +HOST_DISTRO_BOOTSTRAP_KEYS ?= "" +DISTRO_BOOTSTRAP_KEYS = "${HOST_DISTRO_BOOTSTRAP_KEYS}" diff --git a/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap-target.bb b/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap-target.bb new file mode 100644 index 00000000..84a89ff1 --- /dev/null +++ b/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap-target.bb @@ -0,0 +1,12 @@ +# Minimal target Debian root file system +# +# This software is a part of Isar. +# Copyright (C) 2024 ilbers GmbH +# +# SPDX-License-Identifier: MIT + +Description = "Minimal target Debian root file system" + +DEPLOY_ISAR_BOOTSTRAP = "${DEPLOY_DIR_BOOTSTRAP}/${DISTRO}-${DISTRO_ARCH}" + +require isar-mmdebstrap.inc diff --git a/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc b/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc new file mode 100644 index 00000000..658d45be --- /dev/null +++ b/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc @@ -0,0 +1,235 @@ +# Minimal debian root file system +# +# This software is a part of Isar. +# Copyright (C) 2024 ilbers GmbH +# +# SPDX-License-Identifier: MIT + +inherit bootstrap +inherit compat +inherit deb-dl-dir + +FILESEXTRAPATHS:append = ":${LAYERDIR_core}/recipes-core/isar-bootstrap/files" + +ROOTFSDIR = "${WORKDIR}/rootfs" +DISTRO_BOOTSTRAP_BASE_PACKAGES = "locales,apt,usrmerge" +DISTRO_BOOTSTRAP_BASE_PACKAGES:append:gnupg = ",gnupg" +DISTRO_BOOTSTRAP_BASE_PACKAGES:append:https-support = ",ca-certificates" +BOOTSTRAP_TMPDIR = "${WORKDIR}/tempdir" + +def get_distro_primary_source_entry(d): + for source in generate_distro_sources(d): + if source[0] == "deb": + return source[2:] + bb.fatal('Invalid apt sources list') + +def get_distro_have_https_source(d): + return any(source[2].startswith("https://") for source in generate_distro_sources(d)) + +def get_distro_needs_https_support(d): + if get_distro_have_https_source(d): + return "https-support" + else: + return "" + +OVERRIDES:append = ":${@get_distro_needs_https_support(d)}" + +def get_distro_needs_gpg_support(d): + if d.getVar("DISTRO_BOOTSTRAP_KEYS") or \ + d.getVar("THIRD_PARTY_APT_KEYS") or \ + d.getVar("BASE_REPO_KEY"): + return "gnupg" + else: + return "" + +OVERRIDES:append = ":${@get_distro_needs_gpg_support(d)}" + +APT_KEYS_DIR = "${WORKDIR}/aptkeys" +DISTRO_BOOTSTRAP_KEYRING = "${WORKDIR}/distro-keyring.gpg" + +do_generate_keyrings[cleandirs] = "${APT_KEYS_DIR}" +do_generate_keyrings[dirs] = "${DL_DIR}" +do_generate_keyrings[vardeps] += "DISTRO_BOOTSTRAP_KEYS THIRD_PARTY_APT_KEYS" +do_generate_keyrings[network] = "${TASK_USE_SUDO}" +do_generate_keyrings() { + if [ -n "${@d.getVar("THIRD_PARTY_APT_KEYFILES") or ""}" ]; then + chmod 777 "${APT_KEYS_DIR}" + for keyfile in ${@d.getVar("THIRD_PARTY_APT_KEYFILES")}; do + cp "$keyfile" "${APT_KEYS_DIR}"/"$(basename "$keyfile")" + done + fi + if [ -n "${@d.getVar("DISTRO_BOOTSTRAP_KEYFILES") or ""}" ]; then + for keyfile in ${@d.getVar("DISTRO_BOOTSTRAP_KEYFILES")}; do + sudo apt-key --keyring "${DISTRO_BOOTSTRAP_KEYRING}" add $keyfile + cp "$keyfile" "${APT_KEYS_DIR}"/"$(basename "$keyfile")" + done + fi +} +addtask generate_keyrings before do_build after do_unpack + +do_bootstrap[vardeps] += " \ + DISTRO_APT_PREMIRRORS \ + ISAR_ENABLE_COMPAT_ARCH \ + ${DISTRO_VARS_PREFIX}DISTRO_APT_SOURCES \ + " +do_bootstrap[dirs] = "${DEPLOY_DIR_BOOTSTRAP} ${BOOTSTRAP_TMPDIR}" +do_bootstrap[depends] = "base-apt:do_cache isar-apt:do_cache_config" +do_bootstrap[network] = "${TASK_USE_NETWORK_AND_SUDO}" + +do_bootstrap() { + if [ "${ISAR_ENABLE_COMPAT_ARCH}" = "1" ]; then + if [ -z "${COMPAT_DISTRO_ARCH}" ]; then + bbfatal "${DISTRO_ARCH} does not have a compat arch" + fi + fi + bootstrap_args="--verbose --variant=minbase --include=${DISTRO_BOOTSTRAP_BASE_PACKAGES}" + if [ -f "${DISTRO_BOOTSTRAP_KEYRING}" ]; then + bootstrap_args="$bootstrap_args --keyring=${DISTRO_BOOTSTRAP_KEYRING}" + fi + E="${@ isar_export_proxies(d)}" + export BOOTSTRAP_FOR_HOST + + deb_dl_dir_import "${ROOTFSDIR}" "${BOOTSTRAP_BASE_DISTRO}-${BASE_DISTRO_CODENAME}" + sudo rm -rf --one-file-system "${ROOTFSDIR}" + mkdir -p "${ROOTFSDIR}" + + arch_param="--arch=${BOOTSTRAP_DISTRO_ARCH},${DISTRO_ARCH}" + + sudo TMPDIR="${BOOTSTRAP_TMPDIR}" mmdebstrap $bootstrap_args \ + $arch_param \ + --mode=unshare \ + ${@get_distro_components_argument(d)} \ + "${@get_distro_suite(d)}" \ + "${WORKDIR}/rootfs.tar.zst" \ + "${@get_distro_source(d)}" + + sudo -E -s <<'EOSUDO' + set -e + + tar -xf "${WORKDIR}/rootfs.tar.zst" -C "${ROOTFSDIR}" --exclude="./dev/console" + + # Install apt config + mkdir -p "${ROOTFSDIR}/etc/apt/preferences.d" + install -v -m644 "${APTPREFS}" \ + "${ROOTFSDIR}/etc/apt/preferences.d/bootstrap" + mkdir -p "${ROOTFSDIR}/etc/apt/sources.list.d" + if [ "${ISAR_USE_CACHED_BASE_REPO}" = "1" ]; then + line="file:///base-apt/${BOOTSTRAP_BASE_DISTRO} ${BASE_DISTRO_CODENAME} main" + if [ -z "${BASE_REPO_KEY}" ]; then + line="[trusted=yes] ${line}" + fi + echo "deb ${line}" > "${ROOTFSDIR}/etc/apt/sources.list.d/base-apt.list" + line="file:///base-apt/${BASE_DISTRO} ${BASE_DISTRO_CODENAME} main" + if [ -z "${BASE_REPO_KEY}" ]; then + line="[trusted=yes] ${line}" + fi + echo "deb-src ${line}" >> "${ROOTFSDIR}/etc/apt/sources.list.d/base-apt.list" + + mkdir -p ${ROOTFSDIR}/base-apt + mount -o bind,private ${REPO_BASE_DIR} ${ROOTFSDIR}/base-apt + else + install -v -m644 "${APTSRCS}" \ + "${ROOTFSDIR}/etc/apt/sources.list.d/bootstrap.list" + fi + install -v -m644 "${APTSRCS_INIT}" "${ROOTFSDIR}/etc/apt/sources-list" + rm -f "${ROOTFSDIR}/etc/apt/sources.list" + rm -rf "${ROOTFSDIR}/var/lib/apt/lists/"* + find ${APT_KEYS_DIR}/ -type f | while read keyfile + do + MY_GPGHOME="$(chroot "${ROOTFSDIR}" mktemp -d /tmp/gpghomeXXXXXXXXXX)" + echo "Created temporary directory ${MY_GPGHOME} for gpg-agent" + export GNUPGHOME="${MY_GPGHOME}" + APT_KEY_APPEND="--homedir ${MY_GPGHOME}" + + kfn="$(basename $keyfile)" + cp $keyfile "${ROOTFSDIR}/tmp/$kfn" + chroot "${ROOTFSDIR}" /usr/bin/gpg-agent --daemon -- /usr/bin/apt-key \ + --keyring ${THIRD_PARTY_APT_KEYRING} ${APT_KEY_APPEND} add "/tmp/$kfn" + rm "${ROOTFSDIR}/tmp/$kfn" + + echo "Removing ${MY_GPGHOME}" + rm -rf "${ROOTFSDIR}${MY_GPGHOME}" + done + + # Set locale + install -v -m644 "${WORKDIR}/locale" "${ROOTFSDIR}/etc/locale" + + sed -i '/en_US.UTF-8 UTF-8/s/^#//g' "${ROOTFSDIR}/etc/locale.gen" + chroot "${ROOTFSDIR}" /usr/sbin/locale-gen + + # update APT + mount -o bind,private /dev ${ROOTFSDIR}/dev + mount -o bind,private /dev/pts ${ROOTFSDIR}/dev/pts + mount -t tmpfs none "${ROOTFSDIR}/dev/shm" + mount -t proc none ${ROOTFSDIR}/proc + mount -o bind,private /sys ${ROOTFSDIR}/sys + mount --make-rslave ${ROOTFSDIR}/sys + + export DEBIAN_FRONTEND=noninteractive + + if [ "${BOOTSTRAP_FOR_HOST}" = "1" ]; then + chroot "${ROOTFSDIR}" /usr/bin/dpkg --add-architecture ${DISTRO_ARCH} + fi + + if [ "${ISAR_ENABLE_COMPAT_ARCH}" = "1" ]; then + chroot "${ROOTFSDIR}" /usr/bin/dpkg --add-architecture ${COMPAT_DISTRO_ARCH} + fi + + chroot "${ROOTFSDIR}" /usr/bin/apt-get update -y \ + -o APT::Update::Error-Mode=any + + chroot "${ROOTFSDIR}" /usr/bin/apt-get install -y dpkg + + # setup chroot + install -v -m755 "${WORKDIR}/chroot-setup.sh" "${ROOTFSDIR}/chroot-setup.sh" + "${ROOTFSDIR}/chroot-setup.sh" "setup" "${ROOTFSDIR}" + + chroot "${ROOTFSDIR}" /usr/bin/apt-get install -y -f + chroot "${ROOTFSDIR}" /usr/bin/apt-get dist-upgrade -y \ + -o Debug::pkgProblemResolver=yes + + umount "${ROOTFSDIR}/dev/shm" + umount "${ROOTFSDIR}/dev/pts" + umount "${ROOTFSDIR}/dev" + umount "${ROOTFSDIR}/proc" + umount "${ROOTFSDIR}/sys" + if mountpoint -q "${ROOTFSDIR}/base-apt"; then + umount "${ROOTFSDIR}/base-apt" + fi + + # Finalize bootstrap by setting the link in deploy + ln -Tfsr "${ROOTFSDIR}" "${DEPLOY_ISAR_BOOTSTRAP}" +EOSUDO + deb_dl_dir_export "${ROOTFSDIR}" "${BOOTSTRAP_BASE_DISTRO}-${BASE_DISTRO_CODENAME}" + + # Cleanup apt cache + sudo -Es chroot "${ROOTFSDIR}" /usr/bin/apt-get -y clean +} +addtask bootstrap before do_build after do_generate_keyrings + +SSTATETASKS += "do_bootstrap" +SSTATECREATEFUNCS += "bootstrap_sstate_prepare" +SSTATEPOSTINSTFUNCS += "bootstrap_sstate_finalize" + +bootstrap_sstate_prepare() { + # this runs in SSTATE_BUILDDIR, which will be deleted automatically + sudo cp -a "$(dirname "${ROOTFSDIR}")/rootfs.tar.zst" ./bootstrap.tar.zst + sudo chown $(id -u):$(id -g) bootstrap.tar.zst +} + +bootstrap_sstate_finalize() { + # this runs in SSTATE_INSTDIR + # we should restore symlinks after using tar + if [ -f bootstrap.tar.zst ]; then + mv bootstrap.tar.zst "$(dirname "${ROOTFSDIR}")/rootfs.tar.zst" + sudo ln -Tfsr "$(dirname "${ROOTFSDIR}")/rootfs.tar.zst" \ + "${DEPLOY_ISAR_BOOTSTRAP}.tar.zst" + fi +} + +python do_bootstrap_setscene() { + sstate_setscene(d) +} + +addtask do_bootstrap_setscene +do_bootstrap_setscene[dirs] = "${DEPLOY_DIR_BOOTSTRAP}" From patchwork Wed Nov 6 08:21:12 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anton Mikanovich X-Patchwork-Id: 3896 Return-Path: Received: from shymkent.ilbers.de ([unix socket]) by shymkent (Cyrus 2.5.10-Debian-2.5.10-3+deb9u2) with LMTPA; Wed, 06 Nov 2024 09:21:38 +0100 X-Sieve: CMU Sieve 2.4 Received: from mail-lf1-f64.google.com (mail-lf1-f64.google.com [209.85.167.64]) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPS id 4A68Lbwp017041 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Wed, 6 Nov 2024 09:21:37 +0100 Received: by mail-lf1-f64.google.com with SMTP id 2adb3069b0e04-539ea0fcd4bsf4362608e87.2 for ; Wed, 06 Nov 2024 00:21:37 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1730881292; cv=pass; d=google.com; s=arc-20240605; b=gD6gYleb8ljtul39aBUAibMN5w2DeSrNvCZyoCe3/fYQ8MBugZgygQlMf3k1cpybfk BvLz5vAMZA9lkMnnE7zaU2pEY9PQkSOOnIzsHKjItlb7ek81rpMmGsHrKHCHJi9YN6uq HQIecyijC0bmZ4PRSlKQHnVGcKHNYizIeyz3a9t5BUdVHw32hMB95qU4JB1VCT/7flL0 VWZePI4uu/tbNrtkrCVFz6j4lXD28q7CjPbTL5VOnuu2YpDkR0I7XA5SkYrnrLA4PJLs qWvn8Rb04WQ2f14iUAeVqig+vS3fdk6NfYEzmjsBaa7t028b6IwidtkIpLLXaQh1+ZGi FPfA== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20240605; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:dkim-signature; bh=SrpCGGG/YELYoSGPY2fPek7098bS7QmmFigGylc796g=; fh=aQIRjz5UyFllxCPCfVkamW8n45TLdbUFXj09SpK0luE=; b=b9ind8KnH4LmPiPAHpZJzlp0v46t2p4zRmShJ8jmT+PC4DYH3zFySauv38T1oOKaB+ wNbZGPk28Yxc+853adMtPNGTfYtwa5l/eToyktx2AXGNSihDKk8hbWG5EZKCKyrRGqT5 fstn7qzY6tpvnkX8oemp+bHlbD09SvzqbHU8ltXm6M9xN/tmfjPmtJ+1R9Fx6PKT9fWK 6I0/p5Q03Z69pDkRn9fyV3xDE/zIzcqyzKnvMn7JWVZCnoCoqQheZ2WTgZpXEr8GCl2W N3r1XiHiS+NfA5xbThzaVKhNSZ7jpzc0ir/JFZ0NnO/Dt+jVuOxCAob7bg3Ic2WY64+3 wnvw==; darn=isar-build.org ARC-Authentication-Results: i=2; gmr-mx.google.com; spf=pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=amikan@ilbers.de DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlegroups.com; s=20230601; t=1730881292; x=1731486092; darn=isar-build.org; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:x-original-authentication-results :x-original-sender:mime-version:references:in-reply-to:message-id :date:subject:cc:to:from:sender:from:to:cc:subject:date:message-id :reply-to; bh=SrpCGGG/YELYoSGPY2fPek7098bS7QmmFigGylc796g=; b=f8Aoz6ONrcpDm1yAEYiWS/FDFwDE4a06wmD2TxefHAwolL6tPPdPTe+NtpKMVwR/VL sAUHBipWVDb00xFDFNWRrSsZwMO6WWckSK2sAIgwYkHgCdVnwG3v4yQQ5xNigRR3vn6Q SsaUv6XuTB+p2XgmvDBD2Y44XD8lEPrPIItex77SkzI+eSdvNlH35x6PS46LR+UfzuC9 bOZz8gSCF3i9/X2XlTulBtt7R1CQA5pqkL5661vXs5/w4k6MOC1mH8p/9xoyY6MI6eWg cNLWbIbYpp6TZwTVMWP3YZlvBZ14XGK4Bfrf6rT+Onj3acXAkQFAgkEcsZeteRCNThBG A13Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1730881292; x=1731486092; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :x-spam-checked-in-group:list-id:mailing-list:precedence :x-original-authentication-results:x-original-sender:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :x-beenthere:x-gm-message-state:sender:from:to:cc:subject:date :message-id:reply-to; bh=SrpCGGG/YELYoSGPY2fPek7098bS7QmmFigGylc796g=; b=N40MQxyQbi1ckWPmtD0uHGnIQqxqU1PtAIlrDGb/TQv4B08Yx4ZeGCXoLl6q+HTqbx C32+IKR9QlcEFhCteY2zEsHM+HmOxb7IKk/Xvlr1lTMWsFkiuSYHwBBX7mZkRdcrlKJ5 xnBOjnctWoAY2XY/nlpJgNo49xXswQ0DJS0XPhcxnhmrK+sjjtFvlDNXSEpixLQQ92fP SIugdgfEjDo2MSlPps4ECx3u+BoIsocb1zhKrIJpSmWymORUKp6o2SrejfSJkeyhiA1i FWJM7q9Dr5inNUA8xiY6OOJ8iMxK3Ie9RDHuRERacyWn6RHpiYkZQYeo2Cq/LMXV0Uf5 F8Kw== Sender: isar-users@googlegroups.com X-Forwarded-Encrypted: i=2; AJvYcCXxdKaCpn4ZcLuW6vKANqGiKsZdAvJi5c5Wcw+rumAx+UlgEXucNPkflVNPbdcgRznGbSmIi0Q=@isar-build.org X-Gm-Message-State: AOJu0YxA53n+VdL2lBGozHowfsnwQOtzt4ez5A5ch/h9wOmKp4pSzSCg ULRrHpFtCbSKgkRHs4A+Os07e3zlE0xBsKqh5wf1G1Ai806MzBQ7 X-Google-Smtp-Source: AGHT+IGCxEr7kTjr0bPUpJOtLgzwkNJFJGncSzR7Xar0LeFadFIoGBlkNgw5dIJxbT2J/4ghml7sRA== X-Received: by 2002:a05:6512:23a9:b0:539:9f52:9e4 with SMTP id 2adb3069b0e04-53c79ea5736mr10904945e87.48.1730881291089; Wed, 06 Nov 2024 00:21:31 -0800 (PST) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a05:6512:a84:b0:53c:7504:2de5 with SMTP id 2adb3069b0e04-53c7971e264ls663157e87.2.-pod-prod-03-eu; Wed, 06 Nov 2024 00:21:29 -0800 (PST) X-Received: by 2002:a2e:be2c:0:b0:2fb:57b7:5cd with SMTP id 38308e7fff4ca-2fdec4ca568mr106974971fa.7.1730881288805; Wed, 06 Nov 2024 00:21:28 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1730881288; cv=none; d=google.com; s=arc-20240605; b=kc0AhDNJdPAEyMzOmvP3ao8aYM4GkQQ6h9jc4/cYapfro6wPTDacOZSLu9Wm0U2UF4 wEbfucC6x9froo2l18m4miqZSQnPzMjwOKEf922u2Nnt9w7ZhGxAwQeHQFOmzXkZnUOU D5isznP7jQ5EIhJ6rcw9v2cO08baFM7xwl5VU/lXaFbMeeD/oBIXBurgufOhmYjI2Unu lJRepm81SE0amuge1N0KyqLDFJMm/X03rlCS31YS5ipwK7VuYb2XM85qCwk2aLFoDTlS goC7MP7DK+hUG6VSye94DQr4hoNdOhp8VV1CdOqQJSWEY88pxE9He3z+YH9GyLYdEmqn Zcvg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20240605; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=Rlre3fVu+/Oi4URRnZqDx7S4Bl41A3nSOqKZ52UtkAM=; fh=/h9QQkzJ8EboVkWg45aWwpaUro6WMavIVd2OhN45RtE=; b=FUNWjGCvAdw6c1mWTJQfKSgKcJrkYp8qIe3OhFOYZH7t0olYVzYnBuqqiFwSDDerpS hgXgEwZkP8Ai2Np6DHA5ODblGATFfeehJjAkg700xNQ5wsCEd+cUnlM99ngs9+NiDfu+ Ry4G3uSS+XrXQ/s2f7MM0Ol6srEP6HegQNXH4v2J+U/KOQ9Q5nE15b2Olcyh/Lu00vb/ eGbLAIhNV/W4S8TzpHWPb5TVSn0iRweEZ8ADlidegeZ01hCySGcgzA+zMICl43P8sQQY ULm/nHbh6iQaAT707Sj257vQIXiZn2a25SFmbJcanbu02u6rAIQqy81oJ1u8Ug2eLsB0 Gj/w==; dara=google.com ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=amikan@ilbers.de Received: from shymkent.ilbers.de (shymkent.ilbers.de. [85.214.156.166]) by gmr-mx.google.com with ESMTPS id 38308e7fff4ca-2fdef8aab2asi2500161fa.5.2024.11.06.00.21.28 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Wed, 06 Nov 2024 00:21:28 -0800 (PST) Received-SPF: pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) client-ip=85.214.156.166; Received: from user-B660.promwad.corp ([159.148.83.114]) (authenticated bits=0) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPSA id 4A68LOjs016956 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 6 Nov 2024 09:21:27 +0100 From: Anton Mikanovich To: isar-users@googlegroups.com Cc: Anton Mikanovich Subject: [PATCH v11 3/8] meta: Allow selecting bootstrap providers Date: Wed, 6 Nov 2024 10:21:12 +0200 Message-Id: <20241106082117.1089554-4-amikan@ilbers.de> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20241106082117.1089554-1-amikan@ilbers.de> References: <20241106082117.1089554-1-amikan@ilbers.de> MIME-Version: 1.0 X-Spam-Status: No, score=-4.6 required=5.0 tests=DKIMWL_WL_MED,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_EF,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,RCVD_IN_DNSWL_BLOCKED,RCVD_IN_MSPIKE_H2, RCVD_IN_RP_CERTIFIED,RCVD_IN_RP_RNBL,RCVD_IN_RP_SAFE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on shymkent.ilbers.de X-Original-Sender: amikan@ilbers.de X-Original-Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=amikan@ilbers.de Precedence: list Mailing-list: list isar-users@googlegroups.com; contact isar-users+owners@googlegroups.com List-ID: X-Spam-Checked-In-Group: isar-users@googlegroups.com X-Google-Group-Id: 914930254986 List-Post: , List-Help: , List-Archive: , List-Unsubscribe: , X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= Both isar-bootstrap or isar-mmdebstrap can be used for initial rootfs preparation. Signed-off-by: Anton Mikanovich --- meta-isar/conf/local.conf.sample | 4 ++++ meta/classes/rootfs.bbclass | 2 +- meta/conf/bitbake.conf | 3 +++ meta/recipes-core/isar-bootstrap/isar-bootstrap-host.bb | 2 ++ meta/recipes-core/isar-bootstrap/isar-bootstrap-target.bb | 2 ++ meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap-host.bb | 2 ++ meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap-target.bb | 2 ++ 7 files changed, 16 insertions(+), 1 deletion(-) diff --git a/meta-isar/conf/local.conf.sample b/meta-isar/conf/local.conf.sample index 17455015..a7862352 100644 --- a/meta-isar/conf/local.conf.sample +++ b/meta-isar/conf/local.conf.sample @@ -221,3 +221,7 @@ USER_isar[flags] += "clear-text-password" # To know more details about this variable and how to set the value refer below # https://reproducible-builds.org/docs/source-date-epoch/ #SOURCE_DATE_EPOCH = + +# Uncomment this to use old isar-bootstrap provider for rootfs prepare +#PREFERRED_PROVIDER_bootstrap-host ?= "isar-bootstrap-host" +#PREFERRED_PROVIDER_bootstrap-target ?= "isar-bootstrap-target" diff --git a/meta/classes/rootfs.bbclass b/meta/classes/rootfs.bbclass index c7011508..e339d24f 100644 --- a/meta/classes/rootfs.bbclass +++ b/meta/classes/rootfs.bbclass @@ -227,7 +227,7 @@ rootfs_install_pkgs_install() { do_rootfs_install[root_cleandirs] = "${ROOTFSDIR}" do_rootfs_install[vardeps] += "${ROOTFS_CONFIGURE_COMMAND} ${ROOTFS_INSTALL_COMMAND}" do_rootfs_install[vardepsexclude] += "IMAGE_ROOTFS" -do_rootfs_install[depends] = "isar-bootstrap-${@'target' if d.getVar('ROOTFS_ARCH') == d.getVar('DISTRO_ARCH') else 'host'}:do_build" +do_rootfs_install[depends] = "bootstrap-${@'target' if d.getVar('ROOTFS_ARCH') == d.getVar('DISTRO_ARCH') else 'host'}:do_build" do_rootfs_install[recrdeptask] = "do_deploy_deb" do_rootfs_install[network] = "${TASK_USE_SUDO}" python do_rootfs_install() { diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf index 13966a62..289c90b7 100644 --- a/meta/conf/bitbake.conf +++ b/meta/conf/bitbake.conf @@ -195,3 +195,6 @@ include conf/machine/${MACHINE}.conf include conf/distro/${DISTRO}.conf PATCHRESOLVE ?= "noop" + +PREFERRED_PROVIDER_bootstrap-host ??= "isar-mmdebstrap-host" +PREFERRED_PROVIDER_bootstrap-target ??= "isar-mmdebstrap-target" diff --git a/meta/recipes-core/isar-bootstrap/isar-bootstrap-host.bb b/meta/recipes-core/isar-bootstrap/isar-bootstrap-host.bb index 4f90fd01..12798488 100644 --- a/meta/recipes-core/isar-bootstrap/isar-bootstrap-host.bb +++ b/meta/recipes-core/isar-bootstrap/isar-bootstrap-host.bb @@ -9,6 +9,8 @@ Description = "Minimal host Debian root file system" DEPLOY_ISAR_BOOTSTRAP = "${DEPLOY_DIR_BOOTSTRAP}/${HOST_DISTRO}-host_${DISTRO}-${DISTRO_ARCH}" +PROVIDES += "bootstrap-host" + BOOTSTRAP_FOR_HOST = "1" require isar-bootstrap.inc diff --git a/meta/recipes-core/isar-bootstrap/isar-bootstrap-target.bb b/meta/recipes-core/isar-bootstrap/isar-bootstrap-target.bb index c66cb3b3..69e97b33 100644 --- a/meta/recipes-core/isar-bootstrap/isar-bootstrap-target.bb +++ b/meta/recipes-core/isar-bootstrap/isar-bootstrap-target.bb @@ -9,4 +9,6 @@ Description = "Minimal target Debian root file system" DEPLOY_ISAR_BOOTSTRAP = "${DEPLOY_DIR_BOOTSTRAP}/${DISTRO}-${DISTRO_ARCH}" +PROVIDES += "bootstrap-target" + require isar-bootstrap.inc diff --git a/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap-host.bb b/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap-host.bb index 66c8d11e..6de9c21a 100644 --- a/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap-host.bb +++ b/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap-host.bb @@ -9,6 +9,8 @@ Description = "Minimal host Debian root file system" DEPLOY_ISAR_BOOTSTRAP = "${DEPLOY_DIR_BOOTSTRAP}/${HOST_DISTRO}-host_${DISTRO}-${DISTRO_ARCH}" +PROVIDES += "bootstrap-host" + BOOTSTRAP_FOR_HOST = "1" require isar-mmdebstrap.inc diff --git a/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap-target.bb b/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap-target.bb index 84a89ff1..227ff04c 100644 --- a/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap-target.bb +++ b/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap-target.bb @@ -9,4 +9,6 @@ Description = "Minimal target Debian root file system" DEPLOY_ISAR_BOOTSTRAP = "${DEPLOY_DIR_BOOTSTRAP}/${DISTRO}-${DISTRO_ARCH}" +PROVIDES += "bootstrap-target" + require isar-mmdebstrap.inc From patchwork Wed Nov 6 08:21:13 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anton Mikanovich X-Patchwork-Id: 3897 Return-Path: Received: from shymkent.ilbers.de ([unix socket]) by shymkent (Cyrus 2.5.10-Debian-2.5.10-3+deb9u2) with LMTPA; Wed, 06 Nov 2024 09:21:39 +0100 X-Sieve: CMU Sieve 2.4 Received: from mail-lj1-f187.google.com (mail-lj1-f187.google.com [209.85.208.187]) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPS id 4A68LbYC017026 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Wed, 6 Nov 2024 09:21:37 +0100 Received: by mail-lj1-f187.google.com with SMTP id 38308e7fff4ca-2fb58c0df21sf30883121fa.2 for ; Wed, 06 Nov 2024 00:21:37 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1730881292; cv=pass; d=google.com; s=arc-20240605; b=T/9MxZJA2x3bmhBMouO4eC/dJQjbAiW+UfXfTSnvWvwjBSNtZ7yLQ/Dbk/xV0houW9 HcCOoMsfOCO2hJ9B8OKSS5Ukzq4vjw/M9yCTSMfKAS5+CJAETm/tKxW9wltwlwOtfrPi X0NN983rz7rcJ8jUcijrED1JHSh7Pgu9BjzSvNcU8yUPKTag5AxpcHWgAREysNMhpx92 c4d41DE5kJsd9kr4+OTjIbEPsB0z25sDVU56zVVdB4OS3dQijAzuXSCJWATyuTT1G3Go D/88D8KzGroXyErzPw76NMPlE7NAo32VPeRqia6crfZk0aq+oXFw0sKdFWVzXJHBTbIS qPpg== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20240605; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:dkim-signature; bh=7PvZRxHE6F8Sh02WOZ739DcXAg3uUWW3Nj5LHUmNEvI=; fh=cwzcE3Qt5j2CoEwvXAymHc4HQLkJVUUpr2rpWLAbQnY=; b=GLgh1SzSxSA3Jpxpy6m1KPMJ0JTZZeH/FZttBS/6RLei2Ub15WnTyVVtnCluTi7Grr vBk5r/RAhTo4gPV70hikyMS9ii/uPvJ1Jv0qYAJ5tSBzyhi6980/FIUEKyWB3aoakLr2 xx4CISLNer9iiD5mAIknFT7OhBnwrI2ezuR0Wg4MoUESBfFyIOFWjgR2pJt4Tr+pxdaN Amu0TM9mdiPlmg8NOhP06QA6kfpy1Abg8Fk5JrvrsmwxAB1lOO0ewU/V18tyHeomvTLb f9VEUY777QfFwNk6XxaLTqhSWotPrHrWpx9/D8E7V/866lWoM8w3p1GNg/XCGDv6Wovo ad4w==; darn=isar-build.org ARC-Authentication-Results: i=2; gmr-mx.google.com; spf=pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=amikan@ilbers.de DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlegroups.com; s=20230601; t=1730881292; x=1731486092; darn=isar-build.org; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:x-original-authentication-results :x-original-sender:mime-version:references:in-reply-to:message-id :date:subject:cc:to:from:sender:from:to:cc:subject:date:message-id :reply-to; bh=7PvZRxHE6F8Sh02WOZ739DcXAg3uUWW3Nj5LHUmNEvI=; b=L9yMwsGGqlwHNDMHg9OfHFbS5ljBd8vbNmK/tPv47NVFexb1tD1nTQxHVGzlkMHd8K 62Eo0mo8Zcy3bAxyvEXHI8ghpL61w1ryCEWl8A64VGbzJiL84Jn4N9sW9evlytsgABCP ud84Q3HKNuxpWEErC1zUca1uXnCEWwzeuK2zqx6X16K8XEsisr5fImWJC55L9Lqi3SI6 nVLm7mi174owci4mUcD1vS+ZPqTkfRXF6o0PGXg+Bwb+xLjn/EDUCK5vNWo1EpeUp6eh iOiSd9PXTl8b3F4I7NNLTej0B89HCgcDNxYPMeddUIhfKgmUT/9JEmHtjwjZdvgPV6fa QqQg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1730881292; x=1731486092; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :x-spam-checked-in-group:list-id:mailing-list:precedence :x-original-authentication-results:x-original-sender:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :x-beenthere:x-gm-message-state:sender:from:to:cc:subject:date :message-id:reply-to; bh=7PvZRxHE6F8Sh02WOZ739DcXAg3uUWW3Nj5LHUmNEvI=; b=cmyDG5n/1aXayag2kXS6iePxI7sRiiMsiQhZHRow6XH06vWvzWvogDvkDAX/uhsZHs SdCV2IShzxn0ExIyxa/BDnSSjNLveJ8cQMF0zsKqTj58JOiiK/4Rids2gWd6wJiuMZIc ifYkOtdeTo6tq0H+mk9llJIZGPI8L0ex9Uj4SNAfqLTb3cP409vfc9fmjs1B77aXZWEb SVV1koaq1MuPKRlFBoYqcmq+ixqws6x0nAqLIHuCkwNYa3lozVUBAaXsBMle3ImcBcOc HD/MTXZ1NqM0c4LB1U9bjmxpFsmVRt2fZYlm1lu93Q3nYZALkrUAY6Pd/PHXYG/3SFi4 TAmA== Sender: isar-users@googlegroups.com X-Forwarded-Encrypted: i=2; AJvYcCXCCicKdV0aPQqpQbxa/p/V3WZC02q20aTbvXJD6IvGum+jr1S9ranOCm4WCwgZWzTFgDodack=@isar-build.org X-Gm-Message-State: AOJu0YzoqUg4ORYxK+WJGveocbHHYzveDhL9Dnq3pzXFYC+NMrOWkdAA hrfEjkJu5YjR2a02R38y5XRM0Y961QurIU5OK1fq8M8zyxyIt6J5 X-Google-Smtp-Source: AGHT+IEdoPpfZRZVivyU6IcNNNjr/QAPhF2MrfRd1MiBEe+wZQ69sgsHF2sAKoDxj1ledQWHPthu9g== X-Received: by 2002:a05:651c:2209:b0:2fa:d31a:1b84 with SMTP id 38308e7fff4ca-2fcbdf5fa0bmr180598711fa.9.1730881291385; Wed, 06 Nov 2024 00:21:31 -0800 (PST) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a2e:22c5:0:b0:2fa:1c89:d423 with SMTP id 38308e7fff4ca-2fdeb674ae0ls3518851fa.1.-pod-prod-09-eu; Wed, 06 Nov 2024 00:21:29 -0800 (PST) X-Received: by 2002:a2e:be20:0:b0:2fb:4951:172 with SMTP id 38308e7fff4ca-2fcbdf5f742mr176155371fa.2.1730881289113; Wed, 06 Nov 2024 00:21:29 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1730881289; cv=none; d=google.com; s=arc-20240605; b=DoG+F2Qj//tDBDzPzrydLUn/ms0oqBSMjPUSXdSMPpm9BVyoRgZThFk/m5pxQIXA82 Ev3yq/V5fX78nVBz0VQfRczzUjHcLqdD8V3+QI2cYgd/iEn7q0cPnQew/ZTaWi9Iov/e GNoC7fwTHHPgJ/umZ+KPCRV9a8ej1p8/5jhJ/Sdy/xIx9b1q/GNnB6bRWk3Rc8+aADrJ THOVtdwSkneQpK1qoy+g/sPqv+jVdQ45vDuWONEDu6Lw+BOflSRqbTZsDoToFmQC2HwF BmRjYKh58Getwcpdh45D7pdDBZMacFftr41m/mZG+k/3DK6xvt6nLnE0f299ZBr5jsU8 6xCQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20240605; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=QJoiNIoUazrM7NxTXosWWmTxeKtmCD1h7UZJG1bxO5U=; fh=/h9QQkzJ8EboVkWg45aWwpaUro6WMavIVd2OhN45RtE=; b=BR0ZNHdHlXDAGrrihbI0z+IiDAP1xgC+9ka7+niNHY/j/xWSMDDN6xX2kHEIaIfkMe M6z+/977BOwDrgp77/jDWgQJtrWFKb8aXOTDcR30WdEEyJxMUe7CH4YtQIJCOTc4regx o27NT3Wq0PqLC9HUlkUwisg+0NefEAyRR4DRqUvTw/55amxPI/YvzR0hs6asOyA4HuP0 nqS/dkndGQsqxADjH0sSxNAWfT1nnn7jXyeiW3/omNLasVsrhqtAgjq5FnCPNWfefGCm wypvGCZ1w/ul6p1ulImPVwu/BsSXQZoQ0A4K1YIg/ugIzvn3kwgIxpqwmWkvk2Sv9Ox0 MGKQ==; dara=google.com ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=amikan@ilbers.de Received: from shymkent.ilbers.de (shymkent.ilbers.de. [85.214.156.166]) by gmr-mx.google.com with ESMTPS id 38308e7fff4ca-2fdef38d7d0si2469481fa.2.2024.11.06.00.21.28 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Wed, 06 Nov 2024 00:21:29 -0800 (PST) Received-SPF: pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) client-ip=85.214.156.166; Received: from user-B660.promwad.corp ([159.148.83.114]) (authenticated bits=0) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPSA id 4A68LOjt016956 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 6 Nov 2024 09:21:28 +0100 From: Anton Mikanovich To: isar-users@googlegroups.com Cc: Anton Mikanovich Subject: [PATCH v11 4/8] testsuite: Allow variable bootstrap providers Date: Wed, 6 Nov 2024 10:21:13 +0200 Message-Id: <20241106082117.1089554-5-amikan@ilbers.de> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20241106082117.1089554-1-amikan@ilbers.de> References: <20241106082117.1089554-1-amikan@ilbers.de> MIME-Version: 1.0 X-Spam-Status: No, score=-4.6 required=5.0 tests=DKIMWL_WL_MED,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_EF,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,RCVD_IN_DNSWL_BLOCKED,RCVD_IN_MSPIKE_H2, RCVD_IN_RP_CERTIFIED,RCVD_IN_RP_RNBL,RCVD_IN_RP_SAFE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on shymkent.ilbers.de X-Original-Sender: amikan@ilbers.de X-Original-Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=amikan@ilbers.de Precedence: list Mailing-list: list isar-users@googlegroups.com; contact isar-users+owners@googlegroups.com List-ID: X-Spam-Checked-In-Group: isar-users@googlegroups.com X-Google-Group-Id: 914930254986 List-Post: , List-Help: , List-Archive: , List-Unsubscribe: , X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= If we can use different recipes provide bootstrap, recipe name should be obtained from target environment. Signed-off-by: Anton Mikanovich --- testsuite/cibase.py | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-) diff --git a/testsuite/cibase.py b/testsuite/cibase.py index cccac86c..075535b1 100755 --- a/testsuite/cibase.py +++ b/testsuite/cibase.py @@ -216,10 +216,13 @@ class CIBaseTest(CIBuilder): # Rebuild image self.move_in_build_dir('tmp', 'tmp_before_sstate') self.bitbake(image_target, **kwargs) + bootstrap_target_task = CIUtils.getVars( + 'PREFERRED_PROVIDER_bootstrap-target', target=image_target + ) if not all( [ check_executed_tasks( - 'isar-bootstrap-target', + bootstrap_target_task, ['do_bootstrap_setscene', '!do_bootstrap'], ), check_executed_tasks( @@ -257,7 +260,7 @@ class CIBaseTest(CIBuilder): if not all( [ check_executed_tasks( - 'isar-bootstrap-target', ['do_bootstrap_setscene'] + bootstrap_target_task, ['do_bootstrap_setscene'] ), check_executed_tasks( 'sbuild-chroot-target', ['!do_sbuildchroot_deploy'] @@ -282,7 +285,7 @@ class CIBaseTest(CIBuilder): if not all( [ check_executed_tasks( - 'isar-bootstrap-target', + bootstrap_target_task, ['do_bootstrap_setscene', '!do_bootstrap'], ), check_executed_tasks( From patchwork Wed Nov 6 08:21:14 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anton Mikanovich X-Patchwork-Id: 3898 Return-Path: Received: from shymkent.ilbers.de ([unix socket]) by shymkent (Cyrus 2.5.10-Debian-2.5.10-3+deb9u2) with LMTPA; Wed, 06 Nov 2024 09:21:39 +0100 X-Sieve: CMU Sieve 2.4 Received: from mail-lf1-f62.google.com (mail-lf1-f62.google.com [209.85.167.62]) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPS id 4A68Lcb9017043 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Wed, 6 Nov 2024 09:21:38 +0100 Received: by mail-lf1-f62.google.com with SMTP id 2adb3069b0e04-539ea3d778dsf4064694e87.1 for ; Wed, 06 Nov 2024 00:21:38 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1730881293; cv=pass; d=google.com; s=arc-20240605; b=gQxOe+5eKVLcfFnEM+VKNeoJ96bzWQh21Zwy8kdhxW0gtebkwK/YVWVhni+q0Ron17 gZrw6rN30Le5aEigkycQeZK7nNoq30KTn9AtaQLy8c6Eth8kS3O12QpRu5eQys4o8s8Z ZSQ8VuQbstcwF2S5kST4apyggZDTqzCC70Osw+GvPeRsujx5QnsLvn5XO+pzPDkP5FIt gxCd1lb5wzzp76dkewjptKgvco10hyqaY1uBVXIYB1BmX8ZeYeNTp9woFrVecggdLBGQ FF6Bt+dli1Da+Kh9eIAHZ8wSl4+Ga/NIFEMTsSFyDZwcvo5t8rQ0GBLKHOJbQCBSU53f c/OQ== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20240605; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:dkim-signature; bh=miozGLssYB6qsyrH4K0bgplURLEcdvk3KzXgSxFjNMM=; fh=DTd+Atx7bkBKchr5FZVCoSirY85txfO9oGPse2x5/A8=; b=iMpRqvPEGh1bBpKRGINPI/VVE+lqxx6v59AVvBJGlItPOHK7zQTkMes2GROWdO72/S 5juDXw61sEyQEJbX/lzivvSjknftIAKRXu3RV8z6Qnhe5ti/cjt2A/ZpanGp2bn3D/hD 1TlN0EBbkghcerwT1AHAPx60DSOXIAwtKplkhhDmMe5RMQv7IowwXB7FVnVkl5znA0Wu awPT8CNk3OBsjUuE0QYhaWxdPUYfleWmQNGdNMNGPOqSQx24VOboOJ63J5QcGutSHxL/ UMHzm2w9Byh/kyFWdQzgtdNPoDCfs1amOutZedH7N/iy3u8b7xFOHhacI+8QO6N2fDUH NMyQ==; darn=isar-build.org ARC-Authentication-Results: i=2; gmr-mx.google.com; spf=pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=amikan@ilbers.de DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlegroups.com; s=20230601; t=1730881293; x=1731486093; darn=isar-build.org; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:x-original-authentication-results :x-original-sender:mime-version:references:in-reply-to:message-id :date:subject:cc:to:from:sender:from:to:cc:subject:date:message-id :reply-to; bh=miozGLssYB6qsyrH4K0bgplURLEcdvk3KzXgSxFjNMM=; b=h3YG+vGYiiOhjnXO/pLm5Lz1WReAq4xHXpTe1FEQjiV/maMd+a0BuS0BVEpKexBHnb myPonGNNrswvyhdgrHlFhKk62I2wiYvtSayb8UFCoOnEzxN18jP6Pc3FnzB+g4AyAVhv nXMoM54c/0Y6XsIRgPDcn3EhxCIVqTXIJjxD7cqSihmFCuzjZTz/aOo9orNmbm2aFOXe RQzyaV0nlnIiFonhZpERD0z7/OfwnoXSxT4SXXGTiPigB61IZXseIji8KXaVeaXErCTg Bo3Et3r7Qq/Bk09ZfnDNjh6iTLOmum58AH5EmE1kAiUU/pG4B99gFHo4pNeKy4f1WfaA 8TRw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1730881293; x=1731486093; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :x-spam-checked-in-group:list-id:mailing-list:precedence :x-original-authentication-results:x-original-sender:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :x-beenthere:x-gm-message-state:sender:from:to:cc:subject:date :message-id:reply-to; bh=miozGLssYB6qsyrH4K0bgplURLEcdvk3KzXgSxFjNMM=; b=jgQrENNuObOK9OM2Y4Xf8TwkoTzx8+Q4/Wz1KMis4CK+EW+OE61poZfWOI1k1l8KQN vDrCh1adxchyM9k5JT6GGeeBFR08l3CB6NwRVIaFqAw3BKltsuzI6hbXAFjgMpMsxKJo R5izWe/MeELTf7bFWsydnUquSau9HMm1J6Lc30kwkacAe7hFNBytWYp0UyVVoCwzYAFP Zurh/NDBm+EYhb2/3Hsjnk0hdLGgRzl+gPwEPBuhgWqnMbh48bgYPiaXSi3/tSlcupIv EsOhg0/nsf9sWci8Q2gbavKvdP9ExmsD4zSZtCPLkZfE8iuHyuNpb3ieFw7v7o3dqS6O hdqg== Sender: isar-users@googlegroups.com X-Forwarded-Encrypted: i=2; AJvYcCWKnjYz5plnjXfuyYxTH2eRkhn6jdVqUs7ml3Iy3N+KHFoTYtg5duoFQmRy5BMR8vAKWaYyOsI=@isar-build.org X-Gm-Message-State: AOJu0YzuP+GlZtqQvunNvhoaNyVzRcsKdDB6hwrDO5VP/c7wvPtUDw8+ 40UcqwUDlLZ3ZpbVWmdLzeUKJjkncpg4V0cpUJZgl8YABZN77WpI X-Google-Smtp-Source: AGHT+IFvyAO89+4YlmWaPuOt+c//aRG39A0stCMGQqdzyxE/0hlBuqYVeNCMtJwxEDFi6WI64m+3xA== X-Received: by 2002:a05:6512:b11:b0:535:d4e9:28bf with SMTP id 2adb3069b0e04-53d65e12357mr9602168e87.46.1730881292191; Wed, 06 Nov 2024 00:21:32 -0800 (PST) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a05:6512:ea8:b0:53a:7e:bcd3 with SMTP id 2adb3069b0e04-53c794fdd70ls453050e87.1.-pod-prod-06-eu; Wed, 06 Nov 2024 00:21:30 -0800 (PST) X-Received: by 2002:a2e:bd08:0:b0:2fa:cf5b:1e8e with SMTP id 38308e7fff4ca-2fedb758b32mr87918751fa.2.1730881289811; Wed, 06 Nov 2024 00:21:29 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1730881289; cv=none; d=google.com; s=arc-20240605; b=JWXsgR7q4mg29xZ4bGL6PQEwyDfY3BG0mK49UR9bhLcgvum+L9LbWWJQ2h2iDbCnIU vcyd3XlXlgUOKJEvBZcFxBtCzF8eD2uG4G0qSnX+4trDTVzyArQCvriO6E2rft0oaCDB u/pXzCziz02YUNXdisWPmeF7yugPw5H39JsfmMOCoapp6QZra3NMYffDG7IIUroVkZTu 8Pwqg9tclbjgpw0iuSvMY8XdaE+UuZw/WkO4FWM17nyrUJiTrNbj3UrihblHpZwDdBOF DaGTYYy3ZRL5lPKbJO2Ez4Tx3ZyutV71CPl8IjmN1+xMuKpkT2Bn20y5kQzPCYluAH+g VKcQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20240605; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=zvbBnfT4xT0RIRGHztwdSNJYoyshqv732ubMcCz6VBo=; fh=/h9QQkzJ8EboVkWg45aWwpaUro6WMavIVd2OhN45RtE=; b=LQT9k78Sek5rn9tcA5yqM8Wq8p/OJBEgLpZY18gcr+3TiHYW48pBC8GDFIsPi3eSSY Br0+u5y2M9x2RE207D0ndrQocMnZyXgCgvUQulOmbKQTqZ0e6l6sszH8OhIh7/o7WRcH yqmoJbPVjTP11EI0r5GGNwQvw//7jF8Oqs6FfwimOF0nBL+yhS1qKjFcNFO6bNZH54wy 6graIeKu8eHFChH2ctsrqLX/MkWh4I0HbPvEfjeKxKgmTVPK81mJ/JVerZXnVOQ130c7 E9lg8k9VuMq5pyefbk6AoAEHIzHEBO2IhDQL0cdjdmLRawTSALuQb8OYdYdg99HCCtTV JTnQ==; dara=google.com ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=amikan@ilbers.de Received: from shymkent.ilbers.de (shymkent.ilbers.de. [85.214.156.166]) by gmr-mx.google.com with ESMTPS id 38308e7fff4ca-2fdef8aab2asi2500171fa.5.2024.11.06.00.21.29 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Wed, 06 Nov 2024 00:21:29 -0800 (PST) Received-SPF: pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) client-ip=85.214.156.166; Received: from user-B660.promwad.corp ([159.148.83.114]) (authenticated bits=0) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPSA id 4A68LOju016956 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 6 Nov 2024 09:21:28 +0100 From: Anton Mikanovich To: isar-users@googlegroups.com Cc: Anton Mikanovich Subject: [PATCH v11 5/8] mmdebstrap: Fix missing dpkg available Date: Wed, 6 Nov 2024 10:21:14 +0200 Message-Id: <20241106082117.1089554-6-amikan@ilbers.de> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20241106082117.1089554-1-amikan@ilbers.de> References: <20241106082117.1089554-1-amikan@ilbers.de> MIME-Version: 1.0 X-Spam-Status: No, score=-4.6 required=5.0 tests=DKIMWL_WL_MED,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_EF,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,RCVD_IN_DNSWL_BLOCKED,RCVD_IN_MSPIKE_H2, RCVD_IN_RP_CERTIFIED,RCVD_IN_RP_RNBL,RCVD_IN_RP_SAFE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on shymkent.ilbers.de X-Original-Sender: amikan@ilbers.de X-Original-Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=amikan@ilbers.de Precedence: list Mailing-list: list isar-users@googlegroups.com; contact isar-users+owners@googlegroups.com List-ID: X-Spam-Checked-In-Group: isar-users@googlegroups.com X-Google-Group-Id: 914930254986 List-Post: , List-Help: , List-Archive: , List-Unsubscribe: , X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= In some distros /var/lib/dpkg/available is required for do_rootfs_install task. The fix for it is present in jessie-or-older mmdebstrap hook in Bookworm and later, but maybe-jessie-or-older wrapper do not handle distro version check correctly. That's why we need to check and apply hook manually or create /var/lib/dpkg/available by hand for older distros. Signed-off-by: Anton Mikanovich --- meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc b/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc index 658d45be..e1c3832d 100644 --- a/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc +++ b/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc @@ -17,6 +17,14 @@ DISTRO_BOOTSTRAP_BASE_PACKAGES:append:gnupg = ",gnupg" DISTRO_BOOTSTRAP_BASE_PACKAGES:append:https-support = ",ca-certificates" BOOTSTRAP_TMPDIR = "${WORKDIR}/tempdir" +# Fix for /var/lib/apt/available while maybe-jessie-or-older hook do not work +MM_HOOK_JESSIE = "/usr/share/mmdebstrap/hooks/jessie-or-older" +DPKG_HOOKS ?= "${@'--hook-dir='+d.getVar('MM_HOOK_JESSIE') \ + if os.path.isdir(d.getVar('MM_HOOK_JESSIE')) \ + else '--customize-hook=\'touch /var/lib/dpkg/available\''}" +MMHOOKS:ubuntu-focal ?= "${DPKG_HOOKS}" +MMHOOKS:debian-buster ?= "${DPKG_HOOKS}" + def get_distro_primary_source_entry(d): for source in generate_distro_sources(d): if source[0] == "deb": @@ -98,6 +106,7 @@ do_bootstrap() { sudo TMPDIR="${BOOTSTRAP_TMPDIR}" mmdebstrap $bootstrap_args \ $arch_param \ --mode=unshare \ + ${MMHOOKS} \ ${@get_distro_components_argument(d)} \ "${@get_distro_suite(d)}" \ "${WORKDIR}/rootfs.tar.zst" \ From patchwork Wed Nov 6 08:21:15 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anton Mikanovich X-Patchwork-Id: 3900 Return-Path: Received: from shymkent.ilbers.de ([unix socket]) by shymkent (Cyrus 2.5.10-Debian-2.5.10-3+deb9u2) with LMTPA; Wed, 06 Nov 2024 09:21:41 +0100 X-Sieve: CMU Sieve 2.4 Received: from mail-lf1-f62.google.com (mail-lf1-f62.google.com [209.85.167.62]) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPS id 4A68Ld4Q017098 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Wed, 6 Nov 2024 09:21:40 +0100 Received: by mail-lf1-f62.google.com with SMTP id 2adb3069b0e04-539fbbadca7sf6077410e87.3 for ; Wed, 06 Nov 2024 00:21:40 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1730881293; cv=pass; d=google.com; s=arc-20240605; b=JnII8UflU0W5bzCEQBk8dYwBlDKhwh2av9SnoZnHMBIUFxYa+U/NZaUYbbh4qateuf P67Tey1SPhyI9GxjwcUuAepASgFTbKA3GB/zN77tyYR3/2hEVTFzZc0EoVxBYpyHixuc qjAS5Olfpi2CUmhaUwl00mdNewklZ3QgLAbBU7kiZCHg67WyxxEpYN1EXqKnNnBW9qdb U43duN9Pj8EtXp7QGwSfrapZpa1+Yv+u+g8o0RUmwtOqK3bcoiguSizXCYTQHNPILK9O cV/ZZPL2fL2Ph6GZRNE/MGrPLh39LHoebcwwmQ6NS4/zmy6wCiA4srL++2llAbxRHWZH Qmxg== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20240605; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:dkim-signature; bh=51vSoTxZEcpv/ff8baGrTTAQ/e/M5ucuALuXAsu+Jhs=; fh=WwEAQWaYJFdaw/vUOgvVmj1DMd7QDrvBJw+9be7emRA=; b=aS+ZUdeF2m3ApPYGoOQe6AvEuStj7qoQ+ILOgTBLbXBkEQk8khvN//trqgTkUlx4sk wqQNIDW65YFU8xZPHQOcZtQ2yS62J5vLq/83MxLdil2JuECzLsaeTZGUJfgvafc1AgEP zGOn1ga8vcxyOMuXFBA6+cppzN4roGpt22wtziI2oSCjafFVft08cGsIpxFqpt9ZNSNf 74wyHFLsLunGtzEQBOazuf2GTLzk/krsJPIrKKHvAw68O9UQln3ZxDIMAIR08QzJeC3V RQAO0pxVcts2OY+Y26dzXigsDVq4A+zrS6IBwacOosK+ppT8Dgo1EZpcE3tgEXqKem1X 8ZoA==; darn=isar-build.org ARC-Authentication-Results: i=2; gmr-mx.google.com; spf=pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=amikan@ilbers.de DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlegroups.com; s=20230601; t=1730881293; x=1731486093; darn=isar-build.org; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:x-original-authentication-results :x-original-sender:mime-version:references:in-reply-to:message-id :date:subject:cc:to:from:sender:from:to:cc:subject:date:message-id :reply-to; bh=51vSoTxZEcpv/ff8baGrTTAQ/e/M5ucuALuXAsu+Jhs=; b=BX66mNx4wSaW4l9y64X6Bex7QX8lr0aRgMZYXKvT1GrdR336EKHpUOAgtn67C0B0ky y2zU/wL5Zoxw+46E+4/zpDP0tcnEJAGxR2uAiDqQekzfBK6+eogYdrIFWeoVeJgIqjV7 s5CX+IMsVB1jjPI+dqTqu3AKzs42KtfejqkjVuvAQzfOJ9/DrboMfx85tJUsly50wDnZ DJmhqPmQ5LqyGVZaHevu3DFtdivJxtanyzV43Ae8WITQ/+8BYm2QPJ81G7eNI+1LsKnd 4u/LEs5XldVmhXVbHjcKCAwjDu8RjAoR/pwUmJPFE/gBC/kh5XUGOWG+xzbHkXE0k3Am P1WA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1730881293; x=1731486093; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :x-spam-checked-in-group:list-id:mailing-list:precedence :x-original-authentication-results:x-original-sender:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :x-beenthere:x-gm-message-state:sender:from:to:cc:subject:date :message-id:reply-to; bh=51vSoTxZEcpv/ff8baGrTTAQ/e/M5ucuALuXAsu+Jhs=; b=iA/JkQD9/xCNEMpD2tAh5P1Yp7fe4TwZLstt6OMXTApHXE9SwPYDK61fHaz8sohH2F G6bmr8gxEeX7SBWasia3ZscUUkkbBY/oJmqOdIpHtK29zqG1NFlBRx4kf17QhsbHLlCU GPKJ3rWwPVMVsqpFXPUbaujFoQob3NnPNESS6cmThlN8YSNx9/Bscv0fJDAIzNGJHvln Ki8vKM6AzHPdaTPyplZPPChBPPCtQtvP3ejAdix2bbUsTRO4lBN9TxmEWA9wILanM88J RPe+UruP1Ji5njXs7mbrf7L3+2I2BHI5Ia+5BWEhXI3wzDDRpp5bSqXcIndptDe7r+OR UyjA== Sender: isar-users@googlegroups.com X-Forwarded-Encrypted: i=2; AJvYcCXUbUXmr+//D/WcldE15LnxtCvnh+CMVcKTlcVNZSeCkO9w3DYTnv+bgFI3LOvomp9jfzXUJNk=@isar-build.org X-Gm-Message-State: AOJu0YwqqK63YoaLH8ZvfRAPDQOAVMW8CG/FgcSc4U3JxeyO4LgxVCMl I832HapWt/F+y7GNjDvHcVMAPpuj+fpQQjHd5wgx6Sl8Pys8/Eb/ X-Google-Smtp-Source: AGHT+IF9RezwmqgHPKB7pCw0Tuw638UtFGxjjhHgl1cI+yPNbAxhe8JSEBJo/1FgDEdN6miPTBN9/Q== X-Received: by 2002:a05:6512:124b:b0:53a:3a:f4f4 with SMTP id 2adb3069b0e04-53c79e4c400mr10080792e87.31.1730881292865; Wed, 06 Nov 2024 00:21:32 -0800 (PST) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a05:6512:3993:b0:539:fc31:fc7b with SMTP id 2adb3069b0e04-53c791f9e43ls336278e87.0.-pod-prod-02-eu; Wed, 06 Nov 2024 00:21:31 -0800 (PST) X-Received: by 2002:ac2:4e14:0:b0:539:d9e2:9d15 with SMTP id 2adb3069b0e04-53c79e4b459mr11103958e87.29.1730881290629; Wed, 06 Nov 2024 00:21:30 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1730881290; cv=none; d=google.com; s=arc-20240605; b=CjL531GqpgmSxA6lBiD1liEQE8UDDjdA64lkI+xyHXG7SypFGwTKOJKHcljWO1RLEn tVf/M1DOb92FlE92HO0sewVMsyY2rV/a6p6XfKwiCDb1lgCIXcgSn+mN+YhvYTY/RfFQ EAU4sNnZRrJJiOn4RgTdrXHhu4lIlZB0mPLSlHkiOdwMgS4gX1z/MQ4QIvVsMX5P94w6 8khkDr5weUMUyGjQq79VdKB5BJyx5Ze4QHY/uLcV+x/1BMmbAZHOhNQugDb537Qd2wlD Xnz82e/ClyduJjlnF26gVUbmJyzK+dlUVTCIr1HjsD4iHxlmGdVgf3NFL2MHgMNVoDBL XmtA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20240605; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=EcO0BBUZurkMFuUMYbrT06ViPUO76krUApGcQep/SoY=; fh=/h9QQkzJ8EboVkWg45aWwpaUro6WMavIVd2OhN45RtE=; b=hc2ydAZ7+nWJz9FrZsnZESDINrogEE1coiagGm/4ojShVmDa6KDEVipUjvsy4uhPuY Dpap/hdFbkGGVJIGtAza2LIUo49rBxv4ZtG5w1R9PWdejv84DTo6ML5YkWyIrPMg7Qrq IBiEQ3KFwDe6bJjInUxP8IpQ5FElemgWhOfdgZrBRLbI0xBkFJp89VzMl/ceo/tDVGx/ 8jrvwQTXnHnD/zRrnGdWWr2+a3jT0a92eTLQGtOBqdOypdMUg44alAZ71V7IDhHX7HBi /HQW4b2tKZQcjQEcuYg/SUtj8ZvN69fHgeV4CYq7B3weXmOaIOWQATNGz3vzw5y25KpS 8aHQ==; dara=google.com ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=amikan@ilbers.de Received: from shymkent.ilbers.de (shymkent.ilbers.de. [85.214.156.166]) by gmr-mx.google.com with ESMTPS id 2adb3069b0e04-53c7bccc719si508517e87.10.2024.11.06.00.21.30 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Wed, 06 Nov 2024 00:21:30 -0800 (PST) Received-SPF: pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) client-ip=85.214.156.166; Received: from user-B660.promwad.corp ([159.148.83.114]) (authenticated bits=0) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPSA id 4A68LOjv016956 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 6 Nov 2024 09:21:29 +0100 From: Anton Mikanovich To: isar-users@googlegroups.com Cc: Anton Mikanovich Subject: [PATCH v11 6/8] mmdebstrap: Move preparations to hooks Date: Wed, 6 Nov 2024 10:21:15 +0200 Message-Id: <20241106082117.1089554-7-amikan@ilbers.de> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20241106082117.1089554-1-amikan@ilbers.de> References: <20241106082117.1089554-1-amikan@ilbers.de> MIME-Version: 1.0 X-Spam-Status: No, score=-4.6 required=5.0 tests=DKIMWL_WL_MED,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_EF,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,RCVD_IN_DNSWL_BLOCKED,RCVD_IN_MSPIKE_H2, RCVD_IN_RP_CERTIFIED,RCVD_IN_RP_RNBL,RCVD_IN_RP_SAFE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on shymkent.ilbers.de X-Original-Sender: amikan@ilbers.de X-Original-Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=amikan@ilbers.de Precedence: list Mailing-list: list isar-users@googlegroups.com; contact isar-users+owners@googlegroups.com List-ID: X-Spam-Checked-In-Group: isar-users@googlegroups.com X-Google-Group-Id: 914930254986 List-Post: , List-Help: , List-Archive: , List-Unsubscribe: , X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= Move all bootstrap rootfs prepare logic into mmdebstrap hooks. Also migrate from apt-key to gpg. Signed-off-by: Anton Mikanovich --- .../isar-mmdebstrap/isar-mmdebstrap.inc | 244 ++++++++---------- 1 file changed, 114 insertions(+), 130 deletions(-) diff --git a/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc b/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc index e1c3832d..4128ad80 100644 --- a/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc +++ b/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc @@ -13,8 +13,6 @@ FILESEXTRAPATHS:append = ":${LAYERDIR_core}/recipes-core/isar-bootstrap/files" ROOTFSDIR = "${WORKDIR}/rootfs" DISTRO_BOOTSTRAP_BASE_PACKAGES = "locales,apt,usrmerge" -DISTRO_BOOTSTRAP_BASE_PACKAGES:append:gnupg = ",gnupg" -DISTRO_BOOTSTRAP_BASE_PACKAGES:append:https-support = ",ca-certificates" BOOTSTRAP_TMPDIR = "${WORKDIR}/tempdir" # Fix for /var/lib/apt/available while maybe-jessie-or-older hook do not work @@ -25,53 +23,29 @@ DPKG_HOOKS ?= "${@'--hook-dir='+d.getVar('MM_HOOK_JESSIE') \ MMHOOKS:ubuntu-focal ?= "${DPKG_HOOKS}" MMHOOKS:debian-buster ?= "${DPKG_HOOKS}" -def get_distro_primary_source_entry(d): - for source in generate_distro_sources(d): - if source[0] == "deb": - return source[2:] - bb.fatal('Invalid apt sources list') - -def get_distro_have_https_source(d): - return any(source[2].startswith("https://") for source in generate_distro_sources(d)) - -def get_distro_needs_https_support(d): - if get_distro_have_https_source(d): - return "https-support" - else: - return "" - -OVERRIDES:append = ":${@get_distro_needs_https_support(d)}" - -def get_distro_needs_gpg_support(d): - if d.getVar("DISTRO_BOOTSTRAP_KEYS") or \ - d.getVar("THIRD_PARTY_APT_KEYS") or \ - d.getVar("BASE_REPO_KEY"): - return "gnupg" - else: - return "" - -OVERRIDES:append = ":${@get_distro_needs_gpg_support(d)}" - -APT_KEYS_DIR = "${WORKDIR}/aptkeys" DISTRO_BOOTSTRAP_KEYRING = "${WORKDIR}/distro-keyring.gpg" -do_generate_keyrings[cleandirs] = "${APT_KEYS_DIR}" -do_generate_keyrings[dirs] = "${DL_DIR}" +do_generate_keyrings[cleandirs] = "${WORKDIR}/trusted.gpg.d" +do_generate_keyrings[dirs] = "${DEBDIR}" do_generate_keyrings[vardeps] += "DISTRO_BOOTSTRAP_KEYS THIRD_PARTY_APT_KEYS" do_generate_keyrings[network] = "${TASK_USE_SUDO}" do_generate_keyrings() { + export GNUPGHOME="$(mktemp -td gpghomeXXXXXXXXXX)" if [ -n "${@d.getVar("THIRD_PARTY_APT_KEYFILES") or ""}" ]; then - chmod 777 "${APT_KEYS_DIR}" for keyfile in ${@d.getVar("THIRD_PARTY_APT_KEYFILES")}; do - cp "$keyfile" "${APT_KEYS_DIR}"/"$(basename "$keyfile")" + gpg --no-autostart --keyring "gnupg-ring:${DISTRO_BOOTSTRAP_KEYRING}" \ + --no-default-keyring --import $keyfile done fi if [ -n "${@d.getVar("DISTRO_BOOTSTRAP_KEYFILES") or ""}" ]; then for keyfile in ${@d.getVar("DISTRO_BOOTSTRAP_KEYFILES")}; do - sudo apt-key --keyring "${DISTRO_BOOTSTRAP_KEYRING}" add $keyfile - cp "$keyfile" "${APT_KEYS_DIR}"/"$(basename "$keyfile")" + gpg --no-autostart --keyring "gnupg-ring:${DISTRO_BOOTSTRAP_KEYRING}" \ + --no-default-keyring --import $keyfile done fi + if [ -r "${DISTRO_BOOTSTRAP_KEYRING}" ]; then + chmod o+r "${DISTRO_BOOTSTRAP_KEYRING}" + fi } addtask generate_keyrings before do_build after do_unpack @@ -80,10 +54,12 @@ do_bootstrap[vardeps] += " \ ISAR_ENABLE_COMPAT_ARCH \ ${DISTRO_VARS_PREFIX}DISTRO_APT_SOURCES \ " -do_bootstrap[dirs] = "${DEPLOY_DIR_BOOTSTRAP} ${BOOTSTRAP_TMPDIR}" +do_bootstrap[dirs] = "${DEPLOY_DIR_BOOTSTRAP} ${BOOTSTRAP_TMPDIR} ${WORKDIR}/trusted.gpg.d ${WORKDIR}/sources.list.d" do_bootstrap[depends] = "base-apt:do_cache isar-apt:do_cache_config" do_bootstrap[network] = "${TASK_USE_NETWORK_AND_SUDO}" +DEB_DL_LOCK ?= "${DEBDIR}/${BOOTSTRAP_BASE_DISTRO}-${BASE_DISTRO_CODENAME}.lock" + do_bootstrap() { if [ "${ISAR_ENABLE_COMPAT_ARCH}" = "1" ]; then if [ -z "${COMPAT_DISTRO_ARCH}" ]; then @@ -93,126 +69,134 @@ do_bootstrap() { bootstrap_args="--verbose --variant=minbase --include=${DISTRO_BOOTSTRAP_BASE_PACKAGES}" if [ -f "${DISTRO_BOOTSTRAP_KEYRING}" ]; then bootstrap_args="$bootstrap_args --keyring=${DISTRO_BOOTSTRAP_KEYRING}" + cp "${DISTRO_BOOTSTRAP_KEYRING}" "${WORKDIR}/trusted.gpg.d/" fi E="${@ isar_export_proxies(d)}" - export BOOTSTRAP_FOR_HOST - deb_dl_dir_import "${ROOTFSDIR}" "${BOOTSTRAP_BASE_DISTRO}-${BASE_DISTRO_CODENAME}" sudo rm -rf --one-file-system "${ROOTFSDIR}" mkdir -p "${ROOTFSDIR}" + if [ "${ISAR_USE_CACHED_BASE_REPO}" = "1" ]; then + base_apt_tmp="$(mktemp -d /tmp/isar-base-aptXXXXXXXXXX)" + bootstrap_list="${WORKDIR}/sources.list.d/base-apt.list" + line="copy://$base_apt_tmp/${BOOTSTRAP_BASE_DISTRO} ${BASE_DISTRO_CODENAME} main" + if [ -z "${BASE_REPO_KEY}" ]; then + line="[trusted=yes] ${line}" + fi + echo "deb ${line}" > "${WORKDIR}/sources.list.d/base-apt.list" + line="copy://$base_apt_tmp/${BASE_DISTRO} ${BASE_DISTRO_CODENAME} main" + if [ -z "${BASE_REPO_KEY}" ]; then + line="[trusted=yes] ${line}" + fi + echo "deb-src ${line}" >> "${WORKDIR}/sources.list.d/base-apt.list" + + # no need to sync /var/cache/apt/archives if base-apt used + syncin='echo skip sync-in' + syncout='echo skip sync-out' + extra_setup="mount --bind '${REPO_BASE_DIR}' $base_apt_tmp" + extra_extract="$syncout" + # save mmdebstrap tempdir for cleanup + extra_essential="mkdir -p \$1/$base_apt_tmp && \ + echo \$1 > ${WORKDIR}/mmtmpdir && \ + mount -o bind,private '${REPO_BASE_DIR}' \$1/$base_apt_tmp" + # replace base-apt mount in tmp with /base-apt mount + extra_customize="sed -i \"s|copy://$base_apt_tmp|file:///base-apt|g\" \ + \$1/etc/apt/sources.list.d/*.list && \ + mkdir -p \$1/base-apt && \ + mount -o bind,private '${REPO_BASE_DIR}' \$1/base-apt && \ + chroot \$1 apt-get update -y \ + -o APT::Update::Error-Mode=any && \ + chroot \$1 apt-get install -y dpkg && \ + umount \$1/base-apt && \ + umount \$1/$base_apt_tmp && rm ${WORKDIR}/mmtmpdir && \ + umount $base_apt_tmp && rm -rf --one-file-system $base_apt_tmp" + else + deb_dl_dir_import "${WORKDIR}/dl_dir" "${BOOTSTRAP_BASE_DISTRO}-${BASE_DISTRO_CODENAME}" + + bootstrap_list="${WORKDIR}/sources.list.d/bootstrap.list" + install -v -m644 "${APTSRCS}" \ + "${WORKDIR}/sources.list.d/bootstrap.list" + + syncin='flock -s ${DEB_DL_LOCK} cp -n --no-preserve=owner \ + "${WORKDIR}/dl_dir/var/cache/apt/archives/"*.deb \ + "$1/var/cache/apt/archives/" || true' + syncout='flock -s ${DEB_DL_LOCK} cp -n --no-preserve=owner \ + "$1/var/cache/apt/archives/"*.deb \ + "${WORKDIR}/dl_dir/var/cache/apt/archives/"' + extra_setup="$syncin" + extra_extract="$syncout" + # prefetch apt debs because mmdebstrap will clean them on next stage + extra_essential='apt-get install apt -y -d \ + -o Dir::State="$1/var/lib/apt" \ + -o Dir::Etc="$1/etc/apt" \ + -o Dir::Cache="$1/var/cache/apt" \ + -o Apt::Architecture="${BOOTSTRAP_DISTRO_ARCH}"' + extra_essential="$extra_essential && $syncout" + extra_customize="$syncout" + fi + + if [ ! -z "${SOURCE_DATE_EPOCH}" ]; then + export SOURCE_DATE_EPOCH="${SOURCE_DATE_EPOCH}" + fi + arch_param="--arch=${BOOTSTRAP_DISTRO_ARCH},${DISTRO_ARCH}" + if [ "${ISAR_ENABLE_COMPAT_ARCH}" = "1" ]; then + arch_param="$arch_param,${COMPAT_DISTRO_ARCH}" + fi + + # Cleanup mounts if fails + trap 'exit 1' INT HUP QUIT TERM ALRM USR1 + trap '[ -r "${WORKDIR}/mmtmpdir" ] && tmpdir=$(cat "${WORKDIR}/mmtmpdir") \ + && rm "${WORKDIR}/mmtmpdir"; \ + [ -d "$tmpdir" ] && mountpoint -q $tmpdir/$base_apt_tmp \ + && sudo umount $tmpdir/$base_apt_tmp; \ + [ -d "$tmpdir" ] && mountpoint -q $tmpdir/base-apt \ + && sudo umount $tmpdir/base-apt; \ + [ -d "$tmpdir" ] && sudo rm -rf --one-file-system $tmpdir; \ + [ -n "$base_apt_tmp" ] && mountpoint -q $base_apt_tmp \ + && sudo umount $base_apt_tmp \ + && rm -rf --one-file-system $base_apt_tmp' EXIT sudo TMPDIR="${BOOTSTRAP_TMPDIR}" mmdebstrap $bootstrap_args \ $arch_param \ --mode=unshare \ ${MMHOOKS} \ + --setup-hook='mkdir -p "$1/var/cache/apt/archives/"' \ + --setup-hook="$extra_setup" \ + --setup-hook='upload "${APTPREFS}" /etc/apt/preferences.d/bootstrap' \ + --setup-hook='upload "${APTSRCS_INIT}" /etc/apt/sources-list' \ + --setup-hook='upload "${WORKDIR}/locale" /etc/locale' \ + --setup-hook='mkdir -p "$1/etc/apt/trusted.gpg.d"' \ + --setup-hook='sync-in "${WORKDIR}/trusted.gpg.d" /etc/apt/trusted.gpg.d' \ + --setup-hook='install -v -m755 "${WORKDIR}/chroot-setup.sh" "$1/chroot-setup.sh"' \ + --extract-hook="$extra_extract" \ + --essential-hook="$extra_essential" \ + --customize-hook="$extra_customize" \ + --customize-hook='sed -i "/en_US.UTF-8 UTF-8/s/^#//g" "$1/etc/locale.gen"' \ + --customize-hook='chroot "$1" /usr/sbin/locale-gen' \ + --customize-hook='chroot "$1" /usr/bin/apt-get -y clean' \ + --skip=cleanup/apt \ + --skip=download/empty \ ${@get_distro_components_argument(d)} \ "${@get_distro_suite(d)}" \ "${WORKDIR}/rootfs.tar.zst" \ - "${@get_distro_source(d)}" + "$bootstrap_list" sudo -E -s <<'EOSUDO' set -e tar -xf "${WORKDIR}/rootfs.tar.zst" -C "${ROOTFSDIR}" --exclude="./dev/console" - # Install apt config - mkdir -p "${ROOTFSDIR}/etc/apt/preferences.d" - install -v -m644 "${APTPREFS}" \ - "${ROOTFSDIR}/etc/apt/preferences.d/bootstrap" - mkdir -p "${ROOTFSDIR}/etc/apt/sources.list.d" - if [ "${ISAR_USE_CACHED_BASE_REPO}" = "1" ]; then - line="file:///base-apt/${BOOTSTRAP_BASE_DISTRO} ${BASE_DISTRO_CODENAME} main" - if [ -z "${BASE_REPO_KEY}" ]; then - line="[trusted=yes] ${line}" - fi - echo "deb ${line}" > "${ROOTFSDIR}/etc/apt/sources.list.d/base-apt.list" - line="file:///base-apt/${BASE_DISTRO} ${BASE_DISTRO_CODENAME} main" - if [ -z "${BASE_REPO_KEY}" ]; then - line="[trusted=yes] ${line}" - fi - echo "deb-src ${line}" >> "${ROOTFSDIR}/etc/apt/sources.list.d/base-apt.list" - - mkdir -p ${ROOTFSDIR}/base-apt - mount -o bind,private ${REPO_BASE_DIR} ${ROOTFSDIR}/base-apt - else - install -v -m644 "${APTSRCS}" \ - "${ROOTFSDIR}/etc/apt/sources.list.d/bootstrap.list" - fi - install -v -m644 "${APTSRCS_INIT}" "${ROOTFSDIR}/etc/apt/sources-list" - rm -f "${ROOTFSDIR}/etc/apt/sources.list" - rm -rf "${ROOTFSDIR}/var/lib/apt/lists/"* - find ${APT_KEYS_DIR}/ -type f | while read keyfile - do - MY_GPGHOME="$(chroot "${ROOTFSDIR}" mktemp -d /tmp/gpghomeXXXXXXXXXX)" - echo "Created temporary directory ${MY_GPGHOME} for gpg-agent" - export GNUPGHOME="${MY_GPGHOME}" - APT_KEY_APPEND="--homedir ${MY_GPGHOME}" - - kfn="$(basename $keyfile)" - cp $keyfile "${ROOTFSDIR}/tmp/$kfn" - chroot "${ROOTFSDIR}" /usr/bin/gpg-agent --daemon -- /usr/bin/apt-key \ - --keyring ${THIRD_PARTY_APT_KEYRING} ${APT_KEY_APPEND} add "/tmp/$kfn" - rm "${ROOTFSDIR}/tmp/$kfn" - - echo "Removing ${MY_GPGHOME}" - rm -rf "${ROOTFSDIR}${MY_GPGHOME}" - done - - # Set locale - install -v -m644 "${WORKDIR}/locale" "${ROOTFSDIR}/etc/locale" - - sed -i '/en_US.UTF-8 UTF-8/s/^#//g' "${ROOTFSDIR}/etc/locale.gen" - chroot "${ROOTFSDIR}" /usr/sbin/locale-gen - - # update APT - mount -o bind,private /dev ${ROOTFSDIR}/dev - mount -o bind,private /dev/pts ${ROOTFSDIR}/dev/pts - mount -t tmpfs none "${ROOTFSDIR}/dev/shm" - mount -t proc none ${ROOTFSDIR}/proc - mount -o bind,private /sys ${ROOTFSDIR}/sys - mount --make-rslave ${ROOTFSDIR}/sys - - export DEBIAN_FRONTEND=noninteractive - - if [ "${BOOTSTRAP_FOR_HOST}" = "1" ]; then - chroot "${ROOTFSDIR}" /usr/bin/dpkg --add-architecture ${DISTRO_ARCH} - fi - - if [ "${ISAR_ENABLE_COMPAT_ARCH}" = "1" ]; then - chroot "${ROOTFSDIR}" /usr/bin/dpkg --add-architecture ${COMPAT_DISTRO_ARCH} - fi - - chroot "${ROOTFSDIR}" /usr/bin/apt-get update -y \ - -o APT::Update::Error-Mode=any - - chroot "${ROOTFSDIR}" /usr/bin/apt-get install -y dpkg - # setup chroot - install -v -m755 "${WORKDIR}/chroot-setup.sh" "${ROOTFSDIR}/chroot-setup.sh" "${ROOTFSDIR}/chroot-setup.sh" "setup" "${ROOTFSDIR}" - chroot "${ROOTFSDIR}" /usr/bin/apt-get install -y -f - chroot "${ROOTFSDIR}" /usr/bin/apt-get dist-upgrade -y \ - -o Debug::pkgProblemResolver=yes - - umount "${ROOTFSDIR}/dev/shm" - umount "${ROOTFSDIR}/dev/pts" - umount "${ROOTFSDIR}/dev" - umount "${ROOTFSDIR}/proc" - umount "${ROOTFSDIR}/sys" - if mountpoint -q "${ROOTFSDIR}/base-apt"; then - umount "${ROOTFSDIR}/base-apt" - fi - # Finalize bootstrap by setting the link in deploy ln -Tfsr "${ROOTFSDIR}" "${DEPLOY_ISAR_BOOTSTRAP}" EOSUDO - deb_dl_dir_export "${ROOTFSDIR}" "${BOOTSTRAP_BASE_DISTRO}-${BASE_DISTRO_CODENAME}" - - # Cleanup apt cache - sudo -Es chroot "${ROOTFSDIR}" /usr/bin/apt-get -y clean + if [ "${ISAR_USE_CACHED_BASE_REPO}" != "1" ]; then + deb_dl_dir_export "${WORKDIR}/dl_dir" "${BOOTSTRAP_BASE_DISTRO}-${BASE_DISTRO_CODENAME}" + sudo rm -rf --one-file-system "${WORKDIR}/dl_dir" + fi } addtask bootstrap before do_build after do_generate_keyrings From patchwork Wed Nov 6 08:21:16 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anton Mikanovich X-Patchwork-Id: 3899 Return-Path: Received: from shymkent.ilbers.de ([unix socket]) by shymkent (Cyrus 2.5.10-Debian-2.5.10-3+deb9u2) with LMTPA; Wed, 06 Nov 2024 09:21:41 +0100 X-Sieve: CMU Sieve 2.4 Received: from mail-lj1-f190.google.com (mail-lj1-f190.google.com [209.85.208.190]) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPS id 4A68LdGm017102 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Wed, 6 Nov 2024 09:21:40 +0100 Received: by mail-lj1-f190.google.com with SMTP id 38308e7fff4ca-2fb3e48fb66sf3743151fa.2 for ; Wed, 06 Nov 2024 00:21:40 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1730881294; cv=pass; d=google.com; s=arc-20240605; b=iXflTOcQy/QqteL4vMaYyrYoFSzSCGacelDK9V3XGNLwNeXpNrMczzDofOOL9K3SOo UIwHfPp+fdn8LOQx6nRjf1bAHO9AyJtxQ9A2tjG1oredvYoNgAcV0XPHI07a3MR2rQqz fxb9VX7aVqvZgHzC+owTsDY2GT3oNgwU5DcfQkxruS0HtHshOo/ioUoOx7s2wwF2co3X H5CuPXYpSd8R8eibUbgJd7FJHEr279Dh3p+uTRlsCqgTNSRcrP0gtWrW+sSMBKoXVbVY 93CmbB7iJ9WKwbv/ZbxJPFnuXseTFGKdHtfdk2FNd7S02zIZ0OjiCG3d9P4s+6B9Vu3Y wbMA== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20240605; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:dkim-signature; bh=fNbPD93Ni6IXi4nxpMBsz/N/ABtxqebG0Tfp1ktjE/M=; fh=bdNVqBCHITAaFdTaWFRpq8r+nugql3AX/3Gszv6CCZs=; b=MGHL2vGDRv7eh9NhyLDlKEU0Mb22ZqDMyPbw67UpxDPrBttu7AX1tiqwNTFvoOEi/l pFmEqxtRq4ATfCyAoRIqdp0lAACeuIDlODOQLWvunpMpsha6Y8WdDwsC58ElRZOl9NVw LPbI8qjoMVw+51d+r0ewb3RCF3NoouX5Ei6lQ9/fadBwbfUzt6QQbPBI6DAGCrymMoxV 2v+x76y0kGVgt7LHiCtxMCJQyI5CO0hFgnlaTw/87e7AOzdFVeJ39nirgy2TuMtL3fQY /xmSk+1L6PuRx2ccRJOIwLnWjiy4B3i7mG7NDnuAF7TowzgpoRW5bJ7nIMVwed0ZyLZZ fh0w==; darn=isar-build.org ARC-Authentication-Results: i=2; gmr-mx.google.com; spf=pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=amikan@ilbers.de DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlegroups.com; s=20230601; t=1730881294; x=1731486094; darn=isar-build.org; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:x-original-authentication-results :x-original-sender:mime-version:references:in-reply-to:message-id :date:subject:cc:to:from:sender:from:to:cc:subject:date:message-id :reply-to; bh=fNbPD93Ni6IXi4nxpMBsz/N/ABtxqebG0Tfp1ktjE/M=; b=nc3EmsDGkJGLW7gPleATSDAJj0kvh0r+P9ehtu905PTbvFvCtsvgHQuUWPIJplyGRd XKe9fla+F3GMwK+vFoFPujKi4NtyVTry566SoeAYAs/qJlQRFlyHXxg2K2hSBPNL9kkM WWci8IqqwfG0jsqkRGgVD2YY08VUpIIHuzI0bQoOx6kVQB/mqQpsF+bXGZQ+8iA94bfp JyYNggJPHxg2f6Xb/jKa4g7hZymCmQ8UmVSt9EuxKRKdVZtO0n69k0leo4gDVwfoL9t/ Os39cCjqAPQ65OksPmlge1ykDLRpgobhXkTK3mjMtDgUS//kkSkTGFbxR8arlJi+VKMp XjMQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1730881294; x=1731486094; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :x-spam-checked-in-group:list-id:mailing-list:precedence :x-original-authentication-results:x-original-sender:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :x-beenthere:x-gm-message-state:sender:from:to:cc:subject:date :message-id:reply-to; bh=fNbPD93Ni6IXi4nxpMBsz/N/ABtxqebG0Tfp1ktjE/M=; b=ZG5iPS1XAttTg4AnWSUjJxi0dgCP5L1pDlFP1wkWyzgUVgo89pBwXW5HwNNMIQdo02 BaaychgI510NuKttz/tv/Z/SbP5NyuJvBPEUHAALjMx+dnQsH0+P+cn+9GJ6suhIe18r 8OkgZG1LAryC7u0kF94MdGetuWScAKf/R/rOPSAk4DOtinrJtNl3XGiDkdpHTm58okvs 8e5ExDxmbUNbicrpXREwvTbrNDDL1XWgyzUUg7L/UGeYCsIcPkftO0/zBQTWqMcJXZgi +KbQIjXn23pKCIdhAPx7XV76MhJe1ARghXpcO4ev+Qk3Dq2/Q1Kg3//tvEve3ynLrdlW EkuQ== Sender: isar-users@googlegroups.com X-Forwarded-Encrypted: i=2; AJvYcCWlxdTWCygFpGJna+husvThgx4y/Iy4lypofyQ9VzpjvYnTuYYCv3xaw4vREfccp3k1AwzvMnk=@isar-build.org X-Gm-Message-State: AOJu0Yyac2raiqtGjRP7KOnX+fVp3QlucV6cIAaVq+tUeRncWegrACzF 1ftxH/pBmu0N7GnldUERhAgBMSq/A14X7dhmzcFwg7RV9N/xxHir X-Google-Smtp-Source: AGHT+IEjsq1CstabVEN64wMMpTbEpFgDkxkyoBbfSMvcBx4jphXoFqMkAYjgAqgmArfSAAtbW7S6Cw== X-Received: by 2002:a2e:be85:0:b0:2fb:56c1:bf with SMTP id 38308e7fff4ca-2fcbe04bb4dmr61645511fa.7.1730881293473; Wed, 06 Nov 2024 00:21:33 -0800 (PST) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a2e:1f01:0:b0:2fb:358c:f74d with SMTP id 38308e7fff4ca-2fdeb667f6als13144761fa.2.-pod-prod-04-eu; Wed, 06 Nov 2024 00:21:31 -0800 (PST) X-Received: by 2002:a2e:b8c5:0:b0:2fc:9afe:1157 with SMTP id 38308e7fff4ca-2fedb794dc0mr87374451fa.2.1730881291091; Wed, 06 Nov 2024 00:21:31 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1730881291; cv=none; d=google.com; s=arc-20240605; b=Vldtgr24Fn9c8yJTA0niovUVIXpqRhvzEroZ3jAweNWkpZiDoHFvsEX1qfbn/oAdhf hFZpVVdSzyO/1ZhUUfZkxFZ83HeORDZs2c0BOyZFykNPNLn705oKWZSpTK1YQNmKw+Hn HKnxeyiQqLPimobc1mwp8e+QSqR9wjlAuzTT4iT72yLdV9Ttlx/qXzgUujal6fKbIiDz zYgEOJyokxvzoVCvVQZled3yuXAAA15k7t+f8r4UDR4aOXKCSc1mvpy/aTMCEUbPaN4D oiUWJk1/OG5qlWMcQgJ0DnjYD45jNKuZaQO4vV7gfnwzZ3JeOWhxiodvEMMjEGvTPnBs gcAg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20240605; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=FTzM+SpSU14IEU9VKNUCKCG3FornmhDhsewSCHJ+vT8=; fh=/h9QQkzJ8EboVkWg45aWwpaUro6WMavIVd2OhN45RtE=; b=FLXGTupw0m7Pv7IbWcrWpDEHbD9D6GJNJfsjsxeDi9XvjhD+GathRlWt7djULV+wUN U9bMp897i+jqf97dPR6RB7tPhNqIaTCv94bxj4Z4+M0hcaWn0MuUZ/Z8dRuKhwv1qubW HKwsuH6ZwDNI5l9ZYfcDFtxHqfVQpeQBFlnOSEcKF/ydmquV0pEMSMl7k68ALcZThDV8 LWwBm96BxDQ1e0gP4Zp3UnK0Ea3JRYVeQhSuWhb3hiPEUzhZPIn6c2fmVl1ngpI8KRTf 2YBW6jV42fT1kAs/HbfEDItYmFoykUBYhVwXqE7OWcOTXn0Y2cChqAj1rQ+eKviYQFQx 9Tsg==; dara=google.com ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=amikan@ilbers.de Received: from shymkent.ilbers.de (shymkent.ilbers.de. [85.214.156.166]) by gmr-mx.google.com with ESMTPS id 4fb4d7f45d1cf-5cee6b05269si128500a12.3.2024.11.06.00.21.30 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Wed, 06 Nov 2024 00:21:31 -0800 (PST) Received-SPF: pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) client-ip=85.214.156.166; Received: from user-B660.promwad.corp ([159.148.83.114]) (authenticated bits=0) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPSA id 4A68LOjw016956 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 6 Nov 2024 09:21:30 +0100 From: Anton Mikanovich To: isar-users@googlegroups.com Cc: Anton Mikanovich Subject: [PATCH v11 7/8] isar-bootstrap: Use tar output instead of directory Date: Wed, 6 Nov 2024 10:21:16 +0200 Message-Id: <20241106082117.1089554-8-amikan@ilbers.de> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20241106082117.1089554-1-amikan@ilbers.de> References: <20241106082117.1089554-1-amikan@ilbers.de> MIME-Version: 1.0 X-Spam-Status: No, score=-4.6 required=5.0 tests=DKIMWL_WL_MED,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_EF,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,RCVD_IN_DNSWL_BLOCKED,RCVD_IN_MSPIKE_H2, RCVD_IN_RP_CERTIFIED,RCVD_IN_RP_RNBL,RCVD_IN_RP_SAFE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on shymkent.ilbers.de X-Original-Sender: amikan@ilbers.de X-Original-Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=amikan@ilbers.de Precedence: list Mailing-list: list isar-users@googlegroups.com; contact isar-users+owners@googlegroups.com List-ID: X-Spam-Checked-In-Group: isar-users@googlegroups.com X-Google-Group-Id: 914930254986 List-Post: , List-Help: , List-Archive: , List-Unsubscribe: , X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= Do not unpack tar with rootfs content inside bootstrap recipe. This change can allow to remove sudo usage during the bootstrap later. Using tar output requires moving chroot prepare step to the later unpack step. Signed-off-by: Anton Mikanovich --- meta/classes/rootfs.bbclass | 11 +++++--- .../isar-bootstrap/isar-bootstrap.inc | 25 ++++++++++--------- .../isar-mmdebstrap/isar-mmdebstrap.inc | 22 ++++------------ 3 files changed, 25 insertions(+), 33 deletions(-) diff --git a/meta/classes/rootfs.bbclass b/meta/classes/rootfs.bbclass index e339d24f..dee724a0 100644 --- a/meta/classes/rootfs.bbclass +++ b/meta/classes/rootfs.bbclass @@ -105,8 +105,8 @@ rootfs_do_qemu() { fi } -BOOTSTRAP_SRC = "${DEPLOY_DIR_BOOTSTRAP}/${ROOTFS_DISTRO}-host_${DISTRO}-${DISTRO_ARCH}" -BOOTSTRAP_SRC:${ROOTFS_ARCH} = "${DEPLOY_DIR_BOOTSTRAP}/${ROOTFS_DISTRO}-${ROOTFS_ARCH}" +BOOTSTRAP_SRC = "${DEPLOY_DIR_BOOTSTRAP}/${ROOTFS_DISTRO}-host_${DISTRO}-${DISTRO_ARCH}.tar.zst" +BOOTSTRAP_SRC:${ROOTFS_ARCH} = "${DEPLOY_DIR_BOOTSTRAP}/${ROOTFS_DISTRO}-${ROOTFS_ARCH}.tar.zst" def rootfs_extra_import(d): bb.utils._context["rootfs_progress"] = __import__("rootfs_progress") @@ -116,7 +116,10 @@ ROOTFS_EXTRA_IMPORTED := "${@rootfs_extra_import(d)}" rootfs_prepare[weight] = "25" rootfs_prepare(){ - sudo cp -Trpfx --reflink=auto '${BOOTSTRAP_SRC}/' '${ROOTFSDIR}' + sudo tar -xf "${BOOTSTRAP_SRC}" -C "${ROOTFSDIR}" --exclude="./dev/console" + + # setup chroot + sudo "${ROOTFSDIR}/chroot-setup.sh" "setup" "${ROOTFSDIR}" } ROOTFS_CONFIGURE_COMMAND += "rootfs_configure_isar_apt" @@ -275,7 +278,7 @@ cache_deb_src() { # Note: ISAR updates the apt state information(apt-get update) only once during bootstrap and # relies on that through out the build. Copy that state information instead of apt-get update # which generates a new state from upstream. - sudo cp -Trpn --reflink=auto "${BOOTSTRAP_SRC}/var/lib/apt/lists/" "${ROOTFSDIR}/var/lib/apt/lists/" + sudo tar -xf "${BOOTSTRAP_SRC}" ./var/lib/apt/lists --one-top-level="${ROOTFSDIR}" deb_dl_dir_import ${ROOTFSDIR} ${ROOTFS_BASE_DISTRO}-${BASE_DISTRO_CODENAME} debsrc_download ${ROOTFSDIR} ${ROOTFS_BASE_DISTRO}-${BASE_DISTRO_CODENAME} diff --git a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc index 0e5ab57d..f9dae26a 100644 --- a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc +++ b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc @@ -183,9 +183,8 @@ do_bootstrap() { chroot "${ROOTFSDIR}" /usr/bin/apt-get install -y dpkg - # setup chroot + # prepare setup chroot script install -v -m755 "${WORKDIR}/chroot-setup.sh" "${ROOTFSDIR}/chroot-setup.sh" - "${ROOTFSDIR}/chroot-setup.sh" "setup" "${ROOTFSDIR}" chroot "${ROOTFSDIR}" /usr/bin/apt-get install -y -f chroot "${ROOTFSDIR}" /usr/bin/apt-get dist-upgrade -y \ @@ -199,8 +198,12 @@ do_bootstrap() { mountpoint -q "${ROOTFSDIR}/base-apt" && \ umount "${ROOTFSDIR}/base-apt" + # Compress rootfs for compatibility + lopts="--one-file-system --exclude=var/cache/apt/archives" + tar --zstd -cf "${WORKDIR}/rootfs.tar.zst" $lopts -C "${ROOTFSDIR}" . + # Finalize debootstrap by setting the link in deploy - ln -Tfsr "${ROOTFSDIR}" "${DEPLOY_ISAR_BOOTSTRAP}" + ln -Tfsr "${WORKDIR}/rootfs.tar.zst" "${DEPLOY_ISAR_BOOTSTRAP}.tar.zst" EOSUDO deb_dl_dir_export "${ROOTFSDIR}" "${BOOTSTRAP_BASE_DISTRO}-${BASE_DISTRO_CODENAME}" @@ -216,19 +219,17 @@ SSTATEPOSTINSTFUNCS += "bootstrap_sstate_finalize" bootstrap_sstate_prepare() { # this runs in SSTATE_BUILDDIR, which will be deleted automatically - lopts="--one-file-system --exclude=var/cache/apt/archives" - sudo tar -C $(dirname "${ROOTFSDIR}") -cpSf bootstrap.tar $lopts $(basename "${ROOTFSDIR}") - sudo chown $(id -u):$(id -g) bootstrap.tar + sudo cp -a "${WORKDIR}/rootfs.tar.zst" ./bootstrap.tar.zst + sudo chown $(id -u):$(id -g) bootstrap.tar.zst } bootstrap_sstate_finalize() { # this runs in SSTATE_INSTDIR - # - after building the bootstrap, the tar won't be there, but we also don't need to unpack - # - after restoring from cache, there will be a tar which we unpack and then delete - if [ -f bootstrap.tar ]; then - sudo tar -C $(dirname "${ROOTFSDIR}") -xpf bootstrap.tar - sudo ln -Tfsr "${ROOTFSDIR}" "${DEPLOY_ISAR_BOOTSTRAP}" - rm bootstrap.tar + # we should restore symlinks after using tar + if [ -f bootstrap.tar.zst ]; then + mv bootstrap.tar.zst "${WORKDIR}/rootfs.tar.zst" + sudo ln -Tfsr "${WORKDIR}/rootfs.tar.zst" \ + "${DEPLOY_ISAR_BOOTSTRAP}.tar.zst" fi } diff --git a/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc b/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc index 4128ad80..1043f2d1 100644 --- a/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc +++ b/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc @@ -11,7 +11,6 @@ inherit deb-dl-dir FILESEXTRAPATHS:append = ":${LAYERDIR_core}/recipes-core/isar-bootstrap/files" -ROOTFSDIR = "${WORKDIR}/rootfs" DISTRO_BOOTSTRAP_BASE_PACKAGES = "locales,apt,usrmerge" BOOTSTRAP_TMPDIR = "${WORKDIR}/tempdir" @@ -73,9 +72,6 @@ do_bootstrap() { fi E="${@ isar_export_proxies(d)}" - sudo rm -rf --one-file-system "${ROOTFSDIR}" - mkdir -p "${ROOTFSDIR}" - if [ "${ISAR_USE_CACHED_BASE_REPO}" = "1" ]; then base_apt_tmp="$(mktemp -d /tmp/isar-base-aptXXXXXXXXXX)" bootstrap_list="${WORKDIR}/sources.list.d/base-apt.list" @@ -182,17 +178,9 @@ do_bootstrap() { "${WORKDIR}/rootfs.tar.zst" \ "$bootstrap_list" - sudo -E -s <<'EOSUDO' - set -e - - tar -xf "${WORKDIR}/rootfs.tar.zst" -C "${ROOTFSDIR}" --exclude="./dev/console" - - # setup chroot - "${ROOTFSDIR}/chroot-setup.sh" "setup" "${ROOTFSDIR}" + # Finalize bootstrap by setting the link in deploy + sudo ln -Tfsr "${WORKDIR}/rootfs.tar.zst" "${DEPLOY_ISAR_BOOTSTRAP}.tar.zst" - # Finalize bootstrap by setting the link in deploy - ln -Tfsr "${ROOTFSDIR}" "${DEPLOY_ISAR_BOOTSTRAP}" -EOSUDO if [ "${ISAR_USE_CACHED_BASE_REPO}" != "1" ]; then deb_dl_dir_export "${WORKDIR}/dl_dir" "${BOOTSTRAP_BASE_DISTRO}-${BASE_DISTRO_CODENAME}" sudo rm -rf --one-file-system "${WORKDIR}/dl_dir" @@ -206,7 +194,7 @@ SSTATEPOSTINSTFUNCS += "bootstrap_sstate_finalize" bootstrap_sstate_prepare() { # this runs in SSTATE_BUILDDIR, which will be deleted automatically - sudo cp -a "$(dirname "${ROOTFSDIR}")/rootfs.tar.zst" ./bootstrap.tar.zst + sudo cp -a "${WORKDIR}/rootfs.tar.zst" ./bootstrap.tar.zst sudo chown $(id -u):$(id -g) bootstrap.tar.zst } @@ -214,8 +202,8 @@ bootstrap_sstate_finalize() { # this runs in SSTATE_INSTDIR # we should restore symlinks after using tar if [ -f bootstrap.tar.zst ]; then - mv bootstrap.tar.zst "$(dirname "${ROOTFSDIR}")/rootfs.tar.zst" - sudo ln -Tfsr "$(dirname "${ROOTFSDIR}")/rootfs.tar.zst" \ + mv bootstrap.tar.zst "${WORKDIR}/rootfs.tar.zst" + sudo ln -Tfsr "${WORKDIR}/rootfs.tar.zst" \ "${DEPLOY_ISAR_BOOTSTRAP}.tar.zst" fi } From patchwork Wed Nov 6 08:21:17 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anton Mikanovich X-Patchwork-Id: 3901 Return-Path: Received: from shymkent.ilbers.de ([unix socket]) by shymkent (Cyrus 2.5.10-Debian-2.5.10-3+deb9u2) with LMTPA; Wed, 06 Nov 2024 09:21:41 +0100 X-Sieve: CMU Sieve 2.4 Received: from mail-lf1-f62.google.com (mail-lf1-f62.google.com [209.85.167.62]) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPS id 4A68Le05017133 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Wed, 6 Nov 2024 09:21:40 +0100 Received: by mail-lf1-f62.google.com with SMTP id 2adb3069b0e04-539eeb63cc3sf4131721e87.2 for ; Wed, 06 Nov 2024 00:21:40 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1730881295; cv=pass; d=google.com; s=arc-20240605; b=iNgsy4OJmqBp6MNXnFwfW8GtNC6PIhP4Ir/+uWfTcVEXVNWOvt4PidnVenVSW/XMDb ALdQdrn9RC4VttHX7EXV3MAo+LDUBiWmWxxNvrgHI9xRb5tUXfFak/atdYyW+19ekHJe mu367DJ455ClKPWoD8y9oRW79YmuE5kqVIQKDV6HfZ2aMntaiLzuphkRWs/ImsYPTD+m eAdo/+75ZcP9DIbOPk7ntxGRCK4lJ7bS07LIDiWz67xCO9Pq6dwekSLKlBUKY7KBvFLX zsmHoWSQ/VPA36nKw/kBvfoe5K3l50nlIv6D40wRUhVNHooLVcQXWO066ZMQ1+OYBM/u IXhQ== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20240605; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:dkim-signature; bh=jUbMfEEIU3rM9+TzA0RjLa/62jdLxlPLUagS/F1ddYc=; fh=N3SyreFDZwAbNlhiK55RZRyZTdEtGInIuIjZ4T9oAE0=; b=QGl8iRfK7W5phWTQedekecanXUah/61khz+hAHoi7KaZzJq/X5y+0bQ/CCoKXoT7oK 8dTkfsP6MPvpqx4uXpCG0UrLwMhS76aBUWnTqHn/hx+/OSbnlsmoH/Y6qVABTmh1BHqs vS+wv+b9wWxTUdK0DvpBhvIa3sIS1fvGyhm3YZcHM1pwXldKIje/lw8HkDzqb8OsvIu5 0AVI05/lTIAyQsWoRbPfylslwHzJaLFTrYK+u4jwAp2KuwIg03alVLsZzownpkHxF4GO T1OlX0vkeS32mx838eoRMeB4nZ2Z2C53G+ETd78WXWdBsTvmDO4NwBs2c9YTrQb2voYa th/A==; darn=isar-build.org ARC-Authentication-Results: i=2; gmr-mx.google.com; spf=pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=amikan@ilbers.de DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlegroups.com; s=20230601; t=1730881295; x=1731486095; darn=isar-build.org; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:x-original-authentication-results :x-original-sender:mime-version:references:in-reply-to:message-id :date:subject:cc:to:from:sender:from:to:cc:subject:date:message-id :reply-to; bh=jUbMfEEIU3rM9+TzA0RjLa/62jdLxlPLUagS/F1ddYc=; b=k+/URlozrmfAY5atLEcfPFIRtGupROGI7sI7ioryEWO/rGhRRIkUl2dFxYTEQBkGEZ ohPdR84tHmoEz9ZhQ1Md6JGYhRzjCbbkpGhyT7XcoT208RbomqmapiA0z834ohyg+t5p 2ea1S+qGgGh87a6QZmSs56bsfWc5yW87I3H44uu6JV9cMpEH+lRaIBhLCBp3BHCUviot BCgqb4Y5EcqdMvAutYcRywg/5DAOEZRUORHVvo03lMn2I6I7ouRTRkV+ZZlZAbB46vO+ jM9xOMeZFiPru1J5sMfr5B7x3Ad61sJn1ZWbtj4ltfeR2J3piGmAnroSpxuEnx6ID9Jy ZmUg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1730881295; x=1731486095; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :x-spam-checked-in-group:list-id:mailing-list:precedence :x-original-authentication-results:x-original-sender:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :x-beenthere:x-gm-message-state:sender:from:to:cc:subject:date :message-id:reply-to; bh=jUbMfEEIU3rM9+TzA0RjLa/62jdLxlPLUagS/F1ddYc=; b=e5m7F1GUV3D/uPGMkvBo1c+uDxnuHz+ix3j12RWYxwb11gyeGx0w4TlAy0reWwtxW6 Uqpul7HNq4oVqRQlR//DBEIHKrqLTk3cPVoM4UpTR1bGSyDyzysIXOMNlvbVCpYsX5Ok Agtynus1oOjbXjhQefN1+ojFI2tOCV+I99uD9MV8eNonwFZS1ybSZ1qiJO6vhKo5mmo/ lFnF0ZCoUwfftUYimyLY0on6RkCR9XJ07vc/NK7NpnjJSzt8YvgeMSRScys8yuSH1KeE JrUgPnZneTyDaSR9I0Fm8ptftbcF7mWnLgtW2rJjuZH9z4mhN0IEXtoQt2gmNLJCzrNQ ftfw== Sender: isar-users@googlegroups.com X-Forwarded-Encrypted: i=2; AJvYcCXgaNHKCXY2unlFMjxJlYWPh+VJumJyNLxMGrVgCgTo0Alu13JEnmkk7MDsZBpzWuoMa0QRo8A=@isar-build.org X-Gm-Message-State: AOJu0YwJqkWq6kjxKLSaopL3C0e9A0/b4yvyqaLkgPoZK+RAEUHONced HgQsDYfC0n2qIsRpAvacc/U7BtER+rBYCjnlmxW4JrNFPlsccW/Y X-Google-Smtp-Source: AGHT+IE94Z6bYK50jZzOdc4BM5c/zB6BgoqPoZu5nT7M1WVOR9iz6UqXJ34zh2T0FrFsy3tvg68eZQ== X-Received: by 2002:a05:6512:a87:b0:53b:27ba:2d11 with SMTP id 2adb3069b0e04-53b7ece09bamr14500165e87.16.1730881294207; Wed, 06 Nov 2024 00:21:34 -0800 (PST) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a05:6512:2215:b0:539:fdc6:d67e with SMTP id 2adb3069b0e04-53c7972c266ls996064e87.2.-pod-prod-01-eu; Wed, 06 Nov 2024 00:21:32 -0800 (PST) X-Received: by 2002:a05:6512:3b1e:b0:539:f630:f233 with SMTP id 2adb3069b0e04-53b88f406e2mr14234293e87.57.1730881292026; Wed, 06 Nov 2024 00:21:32 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1730881292; cv=none; d=google.com; s=arc-20240605; b=P4WIGXTDuxeL9qPUn6M6VGFvUlQcl1rsKEJAm3I8f+yYkT3pNut7EpBng7HEnPzkp8 gc2GQNw2ZgTMh6DqFQ2iaVQ6rDNK7Aov6IP2+2g7ut1LUBbzO5MWG/Z9QLl2ftw3pj00 +wCOYg/R6ufVOcx5+Hl5qKuNMyMjAVRvkvnSbUKa2Df0CrVErEiIRP7tx4UDybDhqbqi UjFbxllOTq5JzPYs8yMHwoExnYImDGTLwYqBOnEksBN0bq2IaqR8zQa82BEJRjeG/ypx Lv1jPinEWwDSrra1rjdcu4MDswveco+LVeAakngOymUl1zdYblSdYZy5KMUJ+EUXpkaT jNcA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20240605; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=MKoTmwDjs9D84BMIr779E6Areb5Ua0PXxvZZWTGivxM=; fh=/h9QQkzJ8EboVkWg45aWwpaUro6WMavIVd2OhN45RtE=; b=gJAOhu6b61xfDCJ0lV9wNR/mtmVypjJ6i/aQqVgbPqOzRPnF1Otc+0zxpRQV1XF965 1oWTB/M7jPotH5R+qbJp5JDVp7h3jParKd2E0f+AzwaVcUW45zwFPGVgtPGTFDTNe9vh PV8u3beEM8bNiFD08TgImFlUA5CN6Ow1eX6dejs+aGggP9AgcY6a67aw1tbN6H1S4xG7 fCvWHTg4lYmj5DEapUdpiYeZE9PUStDxzkdtA7ES8r2yd93e7mny6GmOHa2Ovm5dKR3p bJBoloSr4ixyVjyWe4CgwXR7VI0G7dY6eIU7KlSrSHZktEbtCbz/LBNPCZG1BpGETr5G jWnA==; dara=google.com ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=amikan@ilbers.de Received: from shymkent.ilbers.de (shymkent.ilbers.de. [85.214.156.166]) by gmr-mx.google.com with ESMTPS id 2adb3069b0e04-53c7bccc4e7si504280e87.8.2024.11.06.00.21.31 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Wed, 06 Nov 2024 00:21:32 -0800 (PST) Received-SPF: pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) client-ip=85.214.156.166; Received: from user-B660.promwad.corp ([159.148.83.114]) (authenticated bits=0) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPSA id 4A68LOjx016956 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 6 Nov 2024 09:21:31 +0100 From: Anton Mikanovich To: isar-users@googlegroups.com Cc: Anton Mikanovich Subject: [PATCH v11 8/8] user_manual.md: Update boostrap related documentation Date: Wed, 6 Nov 2024 10:21:17 +0200 Message-Id: <20241106082117.1089554-9-amikan@ilbers.de> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20241106082117.1089554-1-amikan@ilbers.de> References: <20241106082117.1089554-1-amikan@ilbers.de> MIME-Version: 1.0 X-Spam-Status: No, score=-4.6 required=5.0 tests=DKIMWL_WL_MED,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_EF,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,RCVD_IN_DNSWL_BLOCKED,RCVD_IN_MSPIKE_H2, RCVD_IN_RP_CERTIFIED,RCVD_IN_RP_RNBL,RCVD_IN_RP_SAFE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on shymkent.ilbers.de X-Original-Sender: amikan@ilbers.de X-Original-Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=amikan@ilbers.de Precedence: list Mailing-list: list isar-users@googlegroups.com; contact isar-users+owners@googlegroups.com List-ID: X-Spam-Checked-In-Group: isar-users@googlegroups.com X-Google-Group-Id: 914930254986 List-Post: , List-Help: , List-Archive: , List-Unsubscribe: , X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= Update documentation in user manual and recipes API changelog. Signed-off-by: Anton Mikanovich --- RECIPE-API-CHANGELOG.md | 13 +++++++++++++ doc/user_manual.md | 15 ++++++++++++--- 2 files changed, 25 insertions(+), 3 deletions(-) diff --git a/RECIPE-API-CHANGELOG.md b/RECIPE-API-CHANGELOG.md index 56ebfeb7..deb1c4e1 100644 --- a/RECIPE-API-CHANGELOG.md +++ b/RECIPE-API-CHANGELOG.md @@ -680,3 +680,16 @@ Bitbake exit status depends on ISAR_FAIL_ON_CLEANUP bitbake variable: rootfs_do_umounts is not called from do_rootfs_finalize anymore. Every individual task that does mounting must also do the umounting at its end. + +### Default boostrap recipe changed to mmdebstrap + +New virtual packages bootstrap-host and bootstrap-target are introduced. +There are two providers of bootstrap-host/-target currently: + * isar-mmdebstrap: deafult one using mmdebstrap to prepare rootfs + * isar-bootstrap: previous bootstrap implementation left for compatibility + +So default bootstrap procedure is now performing with mmdebstrap. +Previous implementation still can be selected by setting in local.conf: + +PREFERRED_PROVIDER_bootstrap-host ?= "isar-bootstrap-host" +PREFERRED_PROVIDER_bootstrap-target ?= "isar-bootstrap-target" diff --git a/doc/user_manual.md b/doc/user_manual.md index 284847c8..06561247 100644 --- a/doc/user_manual.md +++ b/doc/user_manual.md @@ -66,7 +66,8 @@ The steps below describe how to build the images provided by default. ### Install Host Tools -The supported host system is >= buster. +The supported host system is >= buster for debootstrap and >= bullseye for +default mmdebstrap provider. Building `debian-trixie` requires host system >= bookworm. @@ -75,7 +76,9 @@ Install the following packages: apt install \ binfmt-support \ bzip2 \ - debootstrap \ + mmdebstrap \ + arch-test \ + apt-utils \ dpkg-dev \ gettext-base \ git \ @@ -93,6 +96,12 @@ apt install \ zstd ``` +If using isar-bootstrap provider, debootstrap should be installed instead of +mmdebstrap: +``` +apt install debootstrap +``` + If your host is >= buster, also install the following package. ``` apt install python3-distutils @@ -135,7 +144,7 @@ apt install qemu ### Setup Sudo -Isar requires `sudo` rights without password to work with `chroot` and `debootstrap`. To add them, use the following steps: +Isar requires `sudo` rights without password to work with `chroot`. To add them, use the following steps: ``` # visudo ```