From patchwork Mon Jul 8 06:17:31 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anton Mikanovich X-Patchwork-Id: 3664 Return-Path: Received: from shymkent.ilbers.de ([unix socket]) by shymkent (Cyrus 2.5.10-Debian-2.5.10-3+deb9u2) with LMTPA; Mon, 08 Jul 2024 08:18:00 +0200 X-Sieve: CMU Sieve 2.4 Received: from mail-lj1-f186.google.com (mail-lj1-f186.google.com [209.85.208.186]) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPS id 4686HxhT030731 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Mon, 8 Jul 2024 08:17:59 +0200 Received: by mail-lj1-f186.google.com with SMTP id 38308e7fff4ca-2ee806f5a66sf37046221fa.2 for ; Sun, 07 Jul 2024 23:17:59 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1720419473; cv=pass; d=google.com; s=arc-20160816; b=nK4MhAZzuTuvAO3pjU2kUycQI7rVFqkJCtxFVXfsYCLbL10Mi+pjyeWKaZrqEeaLfC ulatKhH/LaW6ITEe+09BCU72oAh0rnvhJBEjIsxUsl5bDsJkyuz56MEyBYiSie4RChGt nP8VGsDeJ4IlUPjcZFKvqtFscGk8NXOAxC1FlrMaEbpCOCS6Dtako7lI0pv7faFFcQ8q Ql6315ArfzbX8K3rY+0Dk99s0WeWLFgc4bQmjMHaATYmJ2IhCd9rHGGa3QitNjJXC4cX 43Z47Lg7XgZEb7NCKyug0lKGXp4ugKybtMn0bztsE5h0Sfz8vVaFwvgbkCLZJBDAv29N jn9Q== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:dkim-signature; bh=a9KStk/ATgD4vXr2hzRFBUVxgcP+rozk+bLIdN4S+6M=; fh=Hi7Ab8LL9zGQBJFpOvZRy084boksSghOJUf7CTCfQfs=; b=ExC0FPeCvhlIoAUv1IeLpUACm6wujmsFfjotZ3vGwzgSJHzlc44Ot16trmb22SuFOJ SdKyiobdvh0wLam0canoEL8VI//mJtsRo2cVGfQ6vWDjAAyBkZM/k342ht+faNk2kmAq TOtJcxRQjpchy3lY1tchTlSLaaN5E0KkTVCJVaGgDEO7BBEqV/MmtQxg6Crqw9Y7twUo 0x0wY17JuazirsxxVESs9uJJDaBu3yOoDGIonTH4arMz/ay0uESNQj02Q3WhjbxswJOy ZDTKmhn2HO0eIwNiU5dJW2HL2FkqZNFq0XAzg8rmHibc45zQ97sBDiTlslFSCIKafyDS JSLw==; darn=isar-build.org ARC-Authentication-Results: i=2; gmr-mx.google.com; spf=pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=amikan@ilbers.de DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlegroups.com; s=20230601; t=1720419473; x=1721024273; darn=isar-build.org; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:x-original-authentication-results :x-original-sender:mime-version:references:in-reply-to:message-id :date:subject:cc:to:from:sender:from:to:cc:subject:date:message-id :reply-to; bh=a9KStk/ATgD4vXr2hzRFBUVxgcP+rozk+bLIdN4S+6M=; b=kt0TNPiACMD4hxc98MCeUbJzO6TmTrMxTe8lm4TMFe2hSZ8aAIV1bl4yGrWemxsQDa drnuR//KTwpbwlPGRZkUbmvQC81zyvMXEFTBLWDHOf9b5yCpn+cqEC+TTwtm1P4QM947 +/VkwNMZA5S39rXlel7HtCx0ygo+2zXvNfssc+frksSXBK14F3BfRksVlvcGkr8HKJHm 0EriG8ITAv0mW3n/akHwSyt48wOZzFW1VwvHziYnpVSlpVYj6IJWrT0iF9wLiE0cv8zl +U4r43h6WTVpe9vEdE34r6cYIUZgxdqsHRzVl2OseWZfGj3KF+VwX7R2LDcOPZz0njHs Wt3w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1720419473; x=1721024273; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :x-spam-checked-in-group:list-id:mailing-list:precedence :x-original-authentication-results:x-original-sender:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :x-beenthere:x-gm-message-state:sender:from:to:cc:subject:date :message-id:reply-to; bh=a9KStk/ATgD4vXr2hzRFBUVxgcP+rozk+bLIdN4S+6M=; b=IHuip1XiDGAVMLiHog9w5PFAHN36KYeg8NkL6mJELpcxC+bm7ksp6R+934qB9St+dl U5rYOtpLer0s3Z7sOVds5Rcesw6XsgwX6aqLE81VJFOpObACmWLTpGlfxobE20ON/oIT ERX/e4Dxp2ZK18beEo92zaClN6FHDQu4xxI2n0PT9alDxd6r5VrEjuEBAAWFrAnS82pO MVwbCHWRyAdSUQ1IfsIqYfNBgsdK/9WmohSApjG7roiKRyqyESWUWBtAKsGn2EyS/tJf nO9RbaDb/OaZSL9iqmYVsP68IifogDLsaMj0kUCjEYWVz/iDX4fvhii5nzTAcdizt+QN JQnw== Sender: isar-users@googlegroups.com X-Forwarded-Encrypted: i=2; AJvYcCXBW/xcIf1o8t529aNM+LBosAfKLSc8qaBqvwpRK2EfAdVf/sQddihX5ewh59manQXnzm/RAZLj69tNLNrMQHs0sF04HD0= X-Gm-Message-State: AOJu0Yxrq7CjDU1knqZ/mNSBpswdTnYZOP4HtJTpxQA6ybrSL0F6faCN NdUsAxHjuR8VWWpgsVOD7SFxNehyCqFqvOOvlWIdylZvT/RnhuqL X-Google-Smtp-Source: AGHT+IHuoL3ckNMdOA8J66w+yNHl5BygIuU9myORDP88FIq29xU/wY7C9Lu994tmQUHmV9n+HN5Ouw== X-Received: by 2002:a2e:be0f:0:b0:2ec:4d48:75f3 with SMTP id 38308e7fff4ca-2ee8edd77dcmr84436781fa.45.1720419473180; Sun, 07 Jul 2024 23:17:53 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a2e:b80d:0:b0:2ee:8c6b:1209 with SMTP id 38308e7fff4ca-2ee8ccf485dls16102901fa.0.-pod-prod-04-eu; Sun, 07 Jul 2024 23:17:51 -0700 (PDT) X-Received: by 2002:a2e:9d0c:0:b0:2ee:813b:cd8f with SMTP id 38308e7fff4ca-2ee8ed5fb4fmr69602191fa.23.1720419470476; Sun, 07 Jul 2024 23:17:50 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1720419470; cv=none; d=google.com; s=arc-20160816; b=TrWgFtq0IHZZWztVdNpkAvHHpgEiJYLCt6c0ZxRccI55k2Uzz8Omv/Zh/OHmyDHzSA jVBY8TiTHh9Pr6TGh15LxfzYVSse2/g14FXRjJlEDorLoFMBmvQG6xqK6rlRSovdw1s3 1aALBBDCOTU+ttZjeiJh7ii4ooJpC52HGFiJLIF3QxNyFButOZFlrPRVdOj5dnwnhgqS VERE03S6deLt7WX0twDDHDJDRi2/0lyq08UXUc/6GuzY0/Ahpv6SI7Ex5t1KcifC/NIz CQ62cKat8cZQ7sHbkf3qMWEF+WjaiubIge1vsxSv15yf8bBEvlMBC6P3KvWwDtq8wXLa EQ+Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=DUKda8dthFuQSor/ddlT8khsbVp/bKzdKwUabTHw2Fg=; fh=/h9QQkzJ8EboVkWg45aWwpaUro6WMavIVd2OhN45RtE=; b=A7bIGzZLakbChOs1rCl3k4jm/DBwHck5rzgC8WBriTcGyuhBQi+WXG5b1wiYBAlX8e /KjTVpSouGEqepBtQf6m5LoW/kpPAasp1gTQtFUaGIRJJZTxFmjo1uN5P0ORS57CWqrm F+dpY3ip7CCq8pGeBv5+fBoBvh0z51sm9VZhQ0Yay68/xGgAgFCv3fVvvaqfF7pBMKU9 KG0QoD1AtXQJcnkhM+0bUo5ikrSQ5XGvPIb/gdULEL6sAnT72H2A1EsOPcV7h/5dPT0l W6EuuhmnnMsd6AS31ueW0Fh4Lf+/A3rfNHuPMukinwnjV+ZEKu5BPPPz0YZ8+wChQBXm WkfQ==; dara=google.com ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=amikan@ilbers.de Received: from shymkent.ilbers.de (shymkent.ilbers.de. [85.214.156.166]) by gmr-mx.google.com with ESMTPS id 4fb4d7f45d1cf-58e929fc74csi130694a12.5.2024.07.07.23.17.50 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Sun, 07 Jul 2024 23:17:50 -0700 (PDT) Received-SPF: pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) client-ip=85.214.156.166; Received: from user-B660.promwad.corp ([159.148.83.114]) (authenticated bits=0) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPSA id 4686HmUr030688 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 8 Jul 2024 08:17:49 +0200 From: Anton Mikanovich To: isar-users@googlegroups.com Cc: Anton Mikanovich Subject: [PATCH v10 1/8] isar-bootstrap: Move common parts to bbclass Date: Mon, 8 Jul 2024 09:17:31 +0300 Message-Id: <20240708061738.311938-2-amikan@ilbers.de> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240708061738.311938-1-amikan@ilbers.de> References: <20240708061738.311938-1-amikan@ilbers.de> MIME-Version: 1.0 X-Spam-Status: No, score=-4.6 required=5.0 tests=DKIMWL_WL_MED,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_EF,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,RCVD_IN_MSPIKE_H2,RCVD_IN_RP_CERTIFIED, RCVD_IN_RP_RNBL,RCVD_IN_RP_SAFE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on shymkent.ilbers.de X-Original-Sender: amikan@ilbers.de X-Original-Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=amikan@ilbers.de Precedence: list Mailing-list: list isar-users@googlegroups.com; contact isar-users+owners@googlegroups.com List-ID: X-Spam-Checked-In-Group: isar-users@googlegroups.com X-Google-Group-Id: 914930254986 List-Post: , List-Help: , List-Archive: , List-Unsubscribe: , X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= To allow implementing other bootstrap recipes move some functionality supposed to be reusable to bbclass. Signed-off-by: Anton Mikanovich --- meta/classes/bootstrap.bbclass | 225 ++++++++++++++++++ .../isar-bootstrap/isar-bootstrap.inc | 219 +---------------- 2 files changed, 227 insertions(+), 217 deletions(-) create mode 100644 meta/classes/bootstrap.bbclass diff --git a/meta/classes/bootstrap.bbclass b/meta/classes/bootstrap.bbclass new file mode 100644 index 00000000..a2c2ee1e --- /dev/null +++ b/meta/classes/bootstrap.bbclass @@ -0,0 +1,225 @@ +# This software is a part of Isar. +# Copyright (c) Siemens AG, 2018-2021 +# Copyright (C) 2024 ilbers GmbH +# +# SPDX-License-Identifier: MIT + +FILESEXTRAPATHS:append = ":${BBPATH}" + +SRC_URI = " \ + file://locale \ + file://chroot-setup.sh \ + ${DISTRO_BOOTSTRAP_KEYS} \ + ${THIRD_PARTY_APT_KEYS}" + +BOOTSTRAP_FOR_HOST ?= "0" + +APTPREFS = "${WORKDIR}/apt-preferences" +APTSRCS = "${WORKDIR}/apt-sources" +APTSRCS_INIT = "${WORKDIR}/apt-sources-init" +DISTRO_BOOTSTRAP_KEYFILES = "" +THIRD_PARTY_APT_KEYFILES = "" +DISTRO_BOOTSTRAP_KEYS ?= "" +THIRD_PARTY_APT_KEYS ?= "" +DEPLOY_ISAR_BOOTSTRAP ?= "" +DISTRO_BOOTSTRAP_BASE_PACKAGES ??= "" +DISTRO_VARS_PREFIX ?= "${@'HOST_' if bb.utils.to_boolean(d.getVar('BOOTSTRAP_FOR_HOST')) else ''}" +BOOTSTRAP_DISTRO = "${@d.getVar('HOST_DISTRO' if bb.utils.to_boolean(d.getVar('BOOTSTRAP_FOR_HOST')) else 'DISTRO')}" +BOOTSTRAP_BASE_DISTRO = "${@d.getVar('HOST_BASE_DISTRO' if bb.utils.to_boolean(d.getVar('BOOTSTRAP_FOR_HOST')) else 'BASE_DISTRO')}" +APT_SNAPSHOT_DATE = "${@ get_apt_snapshot_date(d)}" + +python () { + distro_bootstrap_keys = (d.getVar("DISTRO_BOOTSTRAP_KEYS") or "").split() + third_party_apt_keys = (d.getVar("THIRD_PARTY_APT_KEYS") or "").split() + topdir = d.getVar("TOPDIR") + + # The cached repo key can be both for bootstrapping and apt package + # installation afterwards. However, bootstrap will include the key into + # the rootfs automatically thus the right place is distro_bootstrap_keys. + + if bb.utils.to_boolean(d.getVar('ISAR_USE_CACHED_BASE_REPO')): + own_pub_key = d.getVar("BASE_REPO_KEY") + if own_pub_key: + distro_bootstrap_keys += own_pub_key.split() + + for key in distro_bootstrap_keys: + fetcher = bb.fetch2.Fetch([key], d) + filename = os.path.relpath(fetcher.localpath(key), topdir) + d.appendVar("DISTRO_BOOTSTRAP_KEYFILES", " ${TOPDIR}/%s" % filename) + + for key in third_party_apt_keys: + fetcher = bb.fetch2.Fetch([key], d) + filename = os.path.relpath(fetcher.localpath(key), topdir) + d.appendVar("THIRD_PARTY_APT_KEYFILES", " ${TOPDIR}/%s" % filename) + + distro_apt_sources = get_aptsources_list(d) + for file in distro_apt_sources: + d.appendVar("SRC_URI", " file://%s" % file) + + distro_apt_preferences = d.getVar(d.getVar("DISTRO_VARS_PREFIX") + "DISTRO_APT_PREFERENCES") or "" + for file in distro_apt_preferences.split(): + d.appendVar("SRC_URI", " file://%s" % file) +} + +def aggregate_files(d, file_list, file_out): + import shutil + + with open(file_out, "wb") as out_fd: + for entry in file_list: + entry_real = bb.parse.resolve_file(entry, d) + with open(entry_real, "rb") as in_fd: + shutil.copyfileobj(in_fd, out_fd, 1024*1024*10) + out_fd.write("\n".encode()) + +def parse_aptsources_list_line(source_list_line): + import re + + s = source_list_line.strip() + + if not s or s.startswith("#"): + return None + + type, s = re.split("\s+", s, maxsplit=1) + if type not in ["deb", "deb-src"]: + return None + + options = "" + options_match = re.match("\[\s*(\S+=\S+(?=\s))*\s*(\S+=\S+)\s*\]\s+", s) + if options_match: + options = options_match.group(0).strip() + s = s[options_match.end():] + + source, s = re.split("\s+", s, maxsplit=1) + + if s.startswith("/"): + suite = "" + else: + suite, s = re.split("\s+", s, maxsplit=1) + + components = " ".join(s.split()) + + return [type, options, source, suite, components] + +def get_apt_snapshot_date(d): + import time + source_date_epoch = d.getVar('ISAR_APT_SNAPSHOT_TIMESTAMP') + return time.strftime('%Y%m%dT%H%M%SZ', time.gmtime(int(source_date_epoch))) + +def get_apt_source_mirror(d, aptsources_entry_list): + import re + + # this is executed during parsing. No error checking possible + use_snapshot = bb.utils.to_boolean(d.getVar('ISAR_USE_APT_SNAPSHOT')) + snapshot_mirror = d.getVar('DISTRO_APT_SNAPSHOT_PREMIRROR') + if bb.utils.to_boolean(d.getVar('ISAR_USE_CACHED_BASE_REPO')): + premirrors = "\S* file://${REPO_BASE_DIR}/${BOOTSTRAP_BASE_DISTRO}\n" + elif use_snapshot and snapshot_mirror: + premirrors = snapshot_mirror + else: + premirrors = d.getVar('DISTRO_APT_PREMIRRORS') or "" + mirror_list = [entry.split() + for entry in premirrors.split('\\n') + if any(entry)] + + for regex, replace in mirror_list: + match = re.search(regex, aptsources_entry_list[2]) + + if match: + new_aptsources_entry_list = aptsources_entry_list.copy() + new_aptsources_entry_list[2] = re.sub(regex, replace, + aptsources_entry_list[2], + count = 1) + if use_snapshot: + new_aptsources_entry_list[1] = "[check-valid-until=no]" + return new_aptsources_entry_list + + return aptsources_entry_list + +def aggregate_aptsources_list(d, file_list, file_out): + import shutil + + with open(file_out, "wb") as out_fd: + for entry in file_list: + entry_real = bb.parse.resolve_file(entry, d) + with open(entry_real, "r") as in_fd: + for line in in_fd: + parsed = parse_aptsources_list_line(line) + if parsed: + parsed = get_apt_source_mirror(d, parsed) + out_fd.write(" ".join(parsed).encode()) + else: + out_fd.write(line.encode()) + out_fd.write("\n".encode()) + out_fd.write("\n".encode()) + +def get_aptsources_list(d): + import errno + from collections import OrderedDict + apt_sources_var = d.getVar("DISTRO_VARS_PREFIX") + "DISTRO_APT_SOURCES" + apt_sources_list = list(OrderedDict.fromkeys((d.getVar(apt_sources_var) or "").split())) + for p in apt_sources_list: + try: + bb.parse.resolve_file(p, d) + except FileNotFoundError as e: + bb.fatal(os.strerror(errno.ENOENT) + ' "' + p + '"') + return apt_sources_list + +def generate_distro_sources(d): + apt_sources_list = get_aptsources_list(d) + for entry in apt_sources_list: + with open(bb.parse.resolve_file(entry, d), "r") as in_fd: + for line in in_fd: + parsed = parse_aptsources_list_line(line) + if parsed: + parsed = get_apt_source_mirror(d, parsed) + yield parsed + +def get_distro_primary_source_entry(d): + for source in generate_distro_sources(d): + if source[0] == "deb": + return source[2:] + bb.fatal('Invalid apt sources list') + +def get_distro_source(d): + return get_distro_primary_source_entry(d)[0] + +def get_distro_suite(d): + return get_distro_primary_source_entry(d)[1] + +def get_distro_components_argument(d): + components = get_distro_primary_source_entry(d)[2] + if components and components.strip(): + return "--components=" + ",".join(components.split()) + else: + return "" + +do_apt_config_prepare[dirs] = "${WORKDIR}" +do_apt_config_prepare[vardeps] += " \ + APTPREFS \ + ${DISTRO_VARS_PREFIX}DISTRO_APT_PREFERENCES \ + DEBDISTRONAME \ + APTSRCS \ + ${DISTRO_VARS_PREFIX}DISTRO_APT_SOURCES \ + DEPLOY_ISAR_BOOTSTRAP \ + ${@'DISTRO_APT_SNAPSHOT_PREMIRROR' if bb.utils.to_boolean(d.getVar('ISAR_USE_APT_SNAPSHOT')) else ''} \ + " +python do_apt_config_prepare() { + apt_preferences_out = d.getVar("APTPREFS") + apt_preferences_list = ( + d.getVar(d.getVar("DISTRO_VARS_PREFIX") + "DISTRO_APT_PREFERENCES") or "" + ).split() + aggregate_files(d, apt_preferences_list, apt_preferences_out) + + apt_sources_out = d.getVar("APTSRCS") + apt_sources_init_out = d.getVar("APTSRCS_INIT") + apt_sources_list = get_aptsources_list(d) + + aggregate_files(d, apt_sources_list, apt_sources_init_out) + aggregate_aptsources_list(d, apt_sources_list, apt_sources_out) +} +addtask apt_config_prepare before do_bootstrap after do_unpack + +CLEANFUNCS = "clean_deploy" +clean_deploy() { + rm -f "${DEPLOY_ISAR_BOOTSTRAP}" +} diff --git a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc index 12f32ff0..0855875b 100644 --- a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc +++ b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc @@ -5,191 +5,20 @@ # # SPDX-License-Identifier: MIT +inherit bootstrap + LICENSE = "gpl-2.0" LIC_FILES_CHKSUM = "file://${LAYERDIR_core}/licenses/COPYING.GPLv2;md5=751419260aa954499f7abaabaa882bbe" -FILESPATH:prepend := "${THISDIR}/files:" -SRC_URI = " \ - file://locale \ - file://chroot-setup.sh \ - ${DISTRO_BOOTSTRAP_KEYS} \ - ${THIRD_PARTY_APT_KEYS}" PV = "1.0" -BOOTSTRAP_FOR_HOST ?= "0" - DEBOOTSTRAP ?= "qemu-debootstrap" ROOTFSDIR = "${WORKDIR}/rootfs" -APTPREFS = "${WORKDIR}/apt-preferences" -APTSRCS = "${WORKDIR}/apt-sources" -APTSRCS_INIT = "${WORKDIR}/apt-sources-init" -DISTRO_BOOTSTRAP_KEYFILES = "" -THIRD_PARTY_APT_KEYFILES = "" -DISTRO_BOOTSTRAP_KEYS ?= "" -THIRD_PARTY_APT_KEYS ?= "" -DEPLOY_ISAR_BOOTSTRAP ?= "" DISTRO_BOOTSTRAP_BASE_PACKAGES = "locales" DISTRO_BOOTSTRAP_BASE_PACKAGES:append:gnupg = ",gnupg" DISTRO_BOOTSTRAP_BASE_PACKAGES:append:https-support = ",ca-certificates" -DISTRO_VARS_PREFIX ?= "${@'HOST_' if bb.utils.to_boolean(d.getVar('BOOTSTRAP_FOR_HOST')) else ''}" -BOOTSTRAP_DISTRO = "${@d.getVar('HOST_DISTRO' if bb.utils.to_boolean(d.getVar('BOOTSTRAP_FOR_HOST')) else 'DISTRO')}" -BOOTSTRAP_BASE_DISTRO = "${@d.getVar('HOST_BASE_DISTRO' if bb.utils.to_boolean(d.getVar('BOOTSTRAP_FOR_HOST')) else 'BASE_DISTRO')}" -FILESEXTRAPATHS:append = ":${BBPATH}" -APT_SNAPSHOT_DATE = "${@ get_apt_snapshot_date(d)}" inherit deb-dl-dir -python () { - distro_bootstrap_keys = (d.getVar("DISTRO_BOOTSTRAP_KEYS") or "").split() - third_party_apt_keys = (d.getVar("THIRD_PARTY_APT_KEYS") or "").split() - topdir = d.getVar("TOPDIR") - - # The cached repo key can be both for bootstrapping and apt package - # installation afterwards. However, debootstrap will include the key into - # the rootfs automatically thus the right place is distro_bootstrap_keys. - - if bb.utils.to_boolean(d.getVar('ISAR_USE_CACHED_BASE_REPO')): - own_pub_key = d.getVar("BASE_REPO_KEY") - if own_pub_key: - distro_bootstrap_keys += own_pub_key.split() - - for key in distro_bootstrap_keys: - fetcher = bb.fetch2.Fetch([key], d) - filename = os.path.relpath(fetcher.localpath(key), topdir) - d.appendVar("DISTRO_BOOTSTRAP_KEYFILES", " ${TOPDIR}/%s" % filename) - - for key in third_party_apt_keys: - fetcher = bb.fetch2.Fetch([key], d) - filename = os.path.relpath(fetcher.localpath(key), topdir) - d.appendVar("THIRD_PARTY_APT_KEYFILES", " ${TOPDIR}/%s" % filename) - - distro_apt_sources = get_aptsources_list(d) - for file in distro_apt_sources: - d.appendVar("SRC_URI", " file://%s" % file) - - distro_apt_preferences = d.getVar(d.getVar("DISTRO_VARS_PREFIX") + "DISTRO_APT_PREFERENCES") or "" - for file in distro_apt_preferences.split(): - d.appendVar("SRC_URI", " file://%s" % file) -} - -def aggregate_files(d, file_list, file_out): - import shutil - - with open(file_out, "wb") as out_fd: - for entry in file_list: - entry_real = bb.parse.resolve_file(entry, d) - with open(entry_real, "rb") as in_fd: - shutil.copyfileobj(in_fd, out_fd, 1024*1024*10) - out_fd.write("\n".encode()) - -def parse_aptsources_list_line(source_list_line): - import re - - s = source_list_line.strip() - - if not s or s.startswith("#"): - return None - - type, s = re.split("\s+", s, maxsplit=1) - if type not in ["deb", "deb-src"]: - return None - - options = "" - options_match = re.match("\[\s*(\S+=\S+(?=\s))*\s*(\S+=\S+)\s*\]\s+", s) - if options_match: - options = options_match.group(0).strip() - s = s[options_match.end():] - - source, s = re.split("\s+", s, maxsplit=1) - - if s.startswith("/"): - suite = "" - else: - suite, s = re.split("\s+", s, maxsplit=1) - - components = " ".join(s.split()) - - return [type, options, source, suite, components] - -def get_apt_snapshot_date(d): - import time - source_date_epoch = d.getVar('ISAR_APT_SNAPSHOT_TIMESTAMP') - return time.strftime('%Y%m%dT%H%M%SZ', time.gmtime(int(source_date_epoch))) - -def get_apt_source_mirror(d, aptsources_entry_list): - import re - - # this is executed during parsing. No error checking possible - use_snapshot = bb.utils.to_boolean(d.getVar('ISAR_USE_APT_SNAPSHOT')) - snapshot_mirror = d.getVar('DISTRO_APT_SNAPSHOT_PREMIRROR') - if bb.utils.to_boolean(d.getVar('ISAR_USE_CACHED_BASE_REPO')): - premirrors = "\S* file://${REPO_BASE_DIR}/${BOOTSTRAP_BASE_DISTRO}\n" - elif use_snapshot and snapshot_mirror: - premirrors = snapshot_mirror - else: - premirrors = d.getVar('DISTRO_APT_PREMIRRORS') or "" - mirror_list = [entry.split() - for entry in premirrors.split('\\n') - if any(entry)] - - for regex, replace in mirror_list: - match = re.search(regex, aptsources_entry_list[2]) - - if match: - new_aptsources_entry_list = aptsources_entry_list.copy() - new_aptsources_entry_list[2] = re.sub(regex, replace, - aptsources_entry_list[2], - count = 1) - if use_snapshot: - new_aptsources_entry_list[1] = "[check-valid-until=no]" - return new_aptsources_entry_list - - return aptsources_entry_list - -def aggregate_aptsources_list(d, file_list, file_out): - import shutil - - with open(file_out, "wb") as out_fd: - for entry in file_list: - entry_real = bb.parse.resolve_file(entry, d) - with open(entry_real, "r") as in_fd: - for line in in_fd: - parsed = parse_aptsources_list_line(line) - if parsed: - parsed = get_apt_source_mirror(d, parsed) - out_fd.write(" ".join(parsed).encode()) - else: - out_fd.write(line.encode()) - out_fd.write("\n".encode()) - out_fd.write("\n".encode()) - -def get_aptsources_list(d): - import errno - from collections import OrderedDict - apt_sources_var = d.getVar("DISTRO_VARS_PREFIX") + "DISTRO_APT_SOURCES" - apt_sources_list = list(OrderedDict.fromkeys((d.getVar(apt_sources_var) or "").split())) - for p in apt_sources_list: - try: - bb.parse.resolve_file(p, d) - except FileNotFoundError as e: - bb.fatal(os.strerror(errno.ENOENT) + ' "' + p + '"') - return apt_sources_list - -def generate_distro_sources(d): - apt_sources_list = get_aptsources_list(d) - for entry in apt_sources_list: - with open(bb.parse.resolve_file(entry, d), "r") as in_fd: - for line in in_fd: - parsed = parse_aptsources_list_line(line) - if parsed: - parsed = get_apt_source_mirror(d, parsed) - yield parsed - -def get_distro_primary_source_entry(d): - for source in generate_distro_sources(d): - if source[0] == "deb": - return source[2:] - bb.fatal('Invalid apt sources list') - def get_distro_have_https_source(d): return any(source[2].startswith("https://") for source in generate_distro_sources(d)) @@ -211,19 +40,6 @@ def get_distro_needs_gpg_support(d): OVERRIDES:append = ":${@get_distro_needs_gpg_support(d)}" -def get_distro_source(d): - return get_distro_primary_source_entry(d)[0] - -def get_distro_suite(d): - return get_distro_primary_source_entry(d)[1] - -def get_distro_components_argument(d): - components = get_distro_primary_source_entry(d)[2] - if components and components.strip(): - return "--components=" + ",".join(components.split()) - else: - return "" - APT_KEYS_DIR = "${WORKDIR}/aptkeys" DISTRO_BOOTSTRAP_KEYRING = "${WORKDIR}/distro-keyring.gpg" @@ -247,32 +63,6 @@ do_generate_keyrings() { } addtask generate_keyrings before do_build after do_unpack -do_apt_config_prepare[dirs] = "${WORKDIR}" -do_apt_config_prepare[vardeps] += " \ - APTPREFS \ - ${DISTRO_VARS_PREFIX}DISTRO_APT_PREFERENCES \ - DEBDISTRONAME \ - APTSRCS \ - ${DISTRO_VARS_PREFIX}DISTRO_APT_SOURCES \ - DEPLOY_ISAR_BOOTSTRAP \ - ${@'DISTRO_APT_SNAPSHOT_PREMIRROR' if bb.utils.to_boolean(d.getVar('ISAR_USE_APT_SNAPSHOT')) else ''} \ - " -python do_apt_config_prepare() { - apt_preferences_out = d.getVar("APTPREFS") - apt_preferences_list = ( - d.getVar(d.getVar("DISTRO_VARS_PREFIX") + "DISTRO_APT_PREFERENCES") or "" - ).split() - aggregate_files(d, apt_preferences_list, apt_preferences_out) - - apt_sources_out = d.getVar("APTSRCS") - apt_sources_init_out = d.getVar("APTSRCS_INIT") - apt_sources_list = get_aptsources_list(d) - - aggregate_files(d, apt_sources_list, apt_sources_init_out) - aggregate_aptsources_list(d, apt_sources_list, apt_sources_out) -} -addtask apt_config_prepare before do_bootstrap after do_unpack - def get_host_release(): import platform rel = platform.release() @@ -445,8 +235,3 @@ python do_bootstrap_setscene() { addtask do_bootstrap_setscene do_bootstrap_setscene[dirs] = "${DEPLOY_DIR_BOOTSTRAP}" - -CLEANFUNCS = "clean_deploy" -clean_deploy() { - rm -f "${DEPLOY_ISAR_BOOTSTRAP}" -} From patchwork Mon Jul 8 06:17:32 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anton Mikanovich X-Patchwork-Id: 3666 Return-Path: Received: from shymkent.ilbers.de ([unix socket]) by shymkent (Cyrus 2.5.10-Debian-2.5.10-3+deb9u2) with LMTPA; Mon, 08 Jul 2024 08:18:01 +0200 X-Sieve: CMU Sieve 2.4 Received: from mail-wr1-f55.google.com (mail-wr1-f55.google.com [209.85.221.55]) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPS id 4686I0U1030749 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Mon, 8 Jul 2024 08:18:00 +0200 Received: by mail-wr1-f55.google.com with SMTP id ffacd0b85a97d-3678e549a1esf1776077f8f.0 for ; Sun, 07 Jul 2024 23:18:00 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1720419474; cv=pass; d=google.com; s=arc-20160816; b=Il3WcvVbXyRWl4A83MWqr1hLftZw/DREmxpkQte4g/xH3QplVY3I7NRupjpEfZHy2h 3qsOfjQfRWB4daYFLzzU7FzmXE5cocU75d1aOkk4JfUibN71yu4paWvj3O9D+NM7PKbI jvbAVjusP3wPPwZOhtwtQKcGgoGjL12Bst11E+PdjUtHoVuslAzNbhoK0RIIvGVI4BPH /NmBnhtSe8YZtxPJ+9RLqF2lzqFUxC5Fw3LddWe8DrH7jiLJEiCfEfzTa6RGOoyd2KeF 21pstAh/e/qawzzKI+EeQ9WZhCGqamzYh85Loqb/b2dHFAqDJbU+8OJS02DbJMFszoFn hjxw== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:dkim-signature; bh=fjhU4xtkoWvtA7QhYikfL6Kgi2dOl+rQ9/nNbyOy3wA=; fh=88ZOAxQgaFtqthjZfPYM1J0bo+sMh0tYBmYrBIKdgdk=; b=eUPWRAHfOBK4nEqtk2XnVptGhEqv96qAIbycHHBloQMXTZBiWcWY5PiMWKJcUjIQBr DnA1ChZushn+QaIrBwATlSQWxrW2KoxEbA4un0JUUeOKcYkLkcqBfX14HaMulFKI2h2A cUkaX5Gkq4AFEEeNK9L7kmpAgr4FRMR0Dg2ISndwcPtOybh9jW3Mabzzy3GyCCRTTj2N 8zeILCxEn4ftHt4JC4osmM+0+KBlAqN4MBttR5BuSFw3FBmmDEbMdFp78QWT3A5OC2kV 3NHaBWBGfF2MD0sT7pD2KqTcS3uobRlL+KPxP215n8X9nPRZuEO2KikGRoKfYrIU/HxT PubQ==; darn=isar-build.org ARC-Authentication-Results: i=2; gmr-mx.google.com; spf=pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=amikan@ilbers.de DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlegroups.com; s=20230601; t=1720419474; x=1721024274; darn=isar-build.org; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:x-original-authentication-results :x-original-sender:mime-version:references:in-reply-to:message-id :date:subject:cc:to:from:sender:from:to:cc:subject:date:message-id :reply-to; bh=fjhU4xtkoWvtA7QhYikfL6Kgi2dOl+rQ9/nNbyOy3wA=; b=d5Mim6lLhRC6ceTU4Fuqd41YjSNfoOq/aOjoZaLcdGMfy5bip16y5EwABICdhoL7pJ KpO8h3EJ6MCdHHhf/PDNBoq7RKI5uRD8f8SyiWUq/DAL3PzMqsoGE24LRPhRIZWt2JDa 0Psm6HL9LDTdRi4SRztET7ScP4W7BfGq/2lBbSooigwKUHNQ88KIuCw8EjofJ7eqntlt mgHl72Uk9Gnk9+f7ZlkSXxqqvLYePvr6TTdm4lEbDff9POy5J2Xbls6Qu+LoBZFPdPqd SBLSDe7z/+h5qzm2zzXSShe4L0QAmLKXDrYaL13XluaZ/0ftzdlKSmQ4B/NtUD09sYgQ 1LEg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1720419474; x=1721024274; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :x-spam-checked-in-group:list-id:mailing-list:precedence :x-original-authentication-results:x-original-sender:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :x-beenthere:x-gm-message-state:sender:from:to:cc:subject:date :message-id:reply-to; bh=fjhU4xtkoWvtA7QhYikfL6Kgi2dOl+rQ9/nNbyOy3wA=; b=HzTrZjgE/OJxUtNUS4Hl6WvVxdvrH2vh1elbhTm7iNMzDc7fIO4T8/ydI5YeceVVQk PCYaNYqiaog8bB6ba7WWS9t/vx5mhuybkBGqQjxqQ8mbreE743Zaqj5pasaeZHgowa0X flN0m8cTcZI+ckrelRsnv+hGhCJDUdyDsFs/6SEJs+77bRFsMQny8ZJIwlUJV4IiF+u/ tZpC70IUeNKqYWlU/LeE9psmfc0wHm0RaIxgQT6DfcwATktHe757ypAYO02WgF8Muii5 LILiXBnPSOAtX38AQWKqf/OBYM5SMnzmx/Na/ghk4wGYr65zNePGDxXLTGYVDB1Nisn1 eo3w== Sender: isar-users@googlegroups.com X-Forwarded-Encrypted: i=2; AJvYcCUzpvLfWXgmOtOY/AQXXU9sXTGHKWmWyF6rIJ4wFXlU7x5Ulz3vhWpLGKPuAFMRdKozr5jiLz5BKoV/C6lGDBc2+DPQ9ME= X-Gm-Message-State: AOJu0Yw/qEedBcU+yO1KTFnjE7SeX+ALfcbyiOLQ51PGtlaLAOz9pRRz d9LTdLLcVrarB0Etay0H/K9FkLXBIivVV98i1amu2zD1K2TSekiD X-Google-Smtp-Source: AGHT+IFmOd81ngoa0T6/01EApo+WFxr1BfrWyhd9JrK4HOTFbS56uODWgQLNI1sNkNyBgS4ZbvnKQA== X-Received: by 2002:a5d:59a6:0:b0:362:ff95:5697 with SMTP id ffacd0b85a97d-3679f75c493mr11670826f8f.28.1720419473480; Sun, 07 Jul 2024 23:17:53 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a05:600c:16d3:b0:426:62f0:5d9f with SMTP id 5b1f17b1804b1-42662f06144ls4659275e9.1.-pod-prod-00-eu-canary; Sun, 07 Jul 2024 23:17:51 -0700 (PDT) X-Received: by 2002:a05:600c:510b:b0:426:686f:7ad with SMTP id 5b1f17b1804b1-426686f09a5mr18394115e9.10.1720419471604; Sun, 07 Jul 2024 23:17:51 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1720419471; cv=none; d=google.com; s=arc-20160816; b=xR6WIf8OEsglWUViYZzN4syBlQb/QgRz/nUcCqQWEOa0nvcBV4Q4Nv3eciKML4UDt1 MpH8j65p65kPukQKjYwEV1Ntn6SF83spJQDdDMSQaoAF1O1N1LwGVtrWvE82qCG7Ik9Z 5+JTHZh5zuMOP9qLYJo7bD5us8LqOxhW6czPZyHLVFAtA/X15tlyPBL8JUso/o55mhsC 8sikKEdhuu+MW1ahdDiAPohy7ZfyUnh49V+K8Yc/MvZ29JK32ZzBcgX7O5w1fR0ptO/+ G2SlhX/3d2STpaCGpVfe6rQMzp6/MoWVM+N3ENWnkqCLBzAAvjEej2E1sLP7oDBm0Zw/ QIwg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=B4Y/JHERytGORiJbLjTmBSq47BqC7BhNQN+txHti0tY=; fh=/h9QQkzJ8EboVkWg45aWwpaUro6WMavIVd2OhN45RtE=; b=DXN1lW2/QszBsxiwnn3gNoAw4t6hwdpqNfia3O1/dmvuVoaKJ6v4ESu6st1cPB8dtu B4+lJAm9cl7vcrIL0ZTzDasR7WFTFXug1ndazqlAAuNFWCO5oO82HIxL/fpAAZxwDYF1 ChWO/1uKjdyyGfciRDH/VRqjclXE4SVyftGcXUeKYsnX/bfXVcfejB1zL4E+OIv6ghuQ rvqzKQnrlQIAylKlijdtVNYwv1OvtDgwzcBn+qgGJKh0eYxiXxC4k619PZRjb6+x8FfN TVybFDWnu/wKh4j+JLojnWh130BMlivgFAT9rJMouoxP+QsHHv00xeWGLGbewLhapTug KVYQ==; dara=google.com ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=amikan@ilbers.de Received: from shymkent.ilbers.de (shymkent.ilbers.de. [85.214.156.166]) by gmr-mx.google.com with ESMTPS id 5b1f17b1804b1-4265ee3246bsi1864315e9.1.2024.07.07.23.17.51 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Sun, 07 Jul 2024 23:17:51 -0700 (PDT) Received-SPF: pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) client-ip=85.214.156.166; Received: from user-B660.promwad.corp ([159.148.83.114]) (authenticated bits=0) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPSA id 4686HmUs030688 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 8 Jul 2024 08:17:50 +0200 From: Anton Mikanovich To: isar-users@googlegroups.com Cc: Anton Mikanovich Subject: [PATCH v10 2/8] meta: Add mmdebstrap recipe Date: Mon, 8 Jul 2024 09:17:32 +0300 Message-Id: <20240708061738.311938-3-amikan@ilbers.de> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240708061738.311938-1-amikan@ilbers.de> References: <20240708061738.311938-1-amikan@ilbers.de> MIME-Version: 1.0 X-Spam-Status: No, score=-4.6 required=5.0 tests=DKIMWL_WL_MED,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_EF,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,RCVD_IN_MSPIKE_H2,RCVD_IN_RP_CERTIFIED, RCVD_IN_RP_RNBL,RCVD_IN_RP_SAFE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on shymkent.ilbers.de X-Original-Sender: amikan@ilbers.de X-Original-Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=amikan@ilbers.de Precedence: list Mailing-list: list isar-users@googlegroups.com; contact isar-users+owners@googlegroups.com List-ID: X-Spam-Checked-In-Group: isar-users@googlegroups.com X-Google-Group-Id: 914930254986 List-Post: , List-Help: , List-Archive: , List-Unsubscribe: , X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= It can be used as debootstrap alternative for rootfs prepare. Internally, it uses apt and allows to bootstrap the distro from multiple repositories. chroot-setup.sh and locale are copied from debootstrap recipe. Signed-off-by: Anton Mikanovich --- meta/classes/bootstrap.bbclass | 1 + .../isar-mmdebstrap/isar-mmdebstrap-host.bb | 17 ++ .../isar-mmdebstrap/isar-mmdebstrap-target.bb | 12 + .../isar-mmdebstrap/isar-mmdebstrap.inc | 230 ++++++++++++++++++ 4 files changed, 260 insertions(+) create mode 100644 meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap-host.bb create mode 100644 meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap-target.bb create mode 100644 meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc diff --git a/meta/classes/bootstrap.bbclass b/meta/classes/bootstrap.bbclass index a2c2ee1e..e0ccd041 100644 --- a/meta/classes/bootstrap.bbclass +++ b/meta/classes/bootstrap.bbclass @@ -26,6 +26,7 @@ DISTRO_BOOTSTRAP_BASE_PACKAGES ??= "" DISTRO_VARS_PREFIX ?= "${@'HOST_' if bb.utils.to_boolean(d.getVar('BOOTSTRAP_FOR_HOST')) else ''}" BOOTSTRAP_DISTRO = "${@d.getVar('HOST_DISTRO' if bb.utils.to_boolean(d.getVar('BOOTSTRAP_FOR_HOST')) else 'DISTRO')}" BOOTSTRAP_BASE_DISTRO = "${@d.getVar('HOST_BASE_DISTRO' if bb.utils.to_boolean(d.getVar('BOOTSTRAP_FOR_HOST')) else 'BASE_DISTRO')}" +BOOTSTRAP_DISTRO_ARCH = "${@d.getVar('HOST_ARCH' if bb.utils.to_boolean(d.getVar('BOOTSTRAP_FOR_HOST')) else 'DISTRO_ARCH')}" APT_SNAPSHOT_DATE = "${@ get_apt_snapshot_date(d)}" python () { diff --git a/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap-host.bb b/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap-host.bb new file mode 100644 index 00000000..66c8d11e --- /dev/null +++ b/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap-host.bb @@ -0,0 +1,17 @@ +# Minimal host Debian root file system +# +# This software is a part of Isar. +# Copyright (C) 2024 ilbers GmbH +# +# SPDX-License-Identifier: MIT + +Description = "Minimal host Debian root file system" + +DEPLOY_ISAR_BOOTSTRAP = "${DEPLOY_DIR_BOOTSTRAP}/${HOST_DISTRO}-host_${DISTRO}-${DISTRO_ARCH}" + +BOOTSTRAP_FOR_HOST = "1" + +require isar-mmdebstrap.inc + +HOST_DISTRO_BOOTSTRAP_KEYS ?= "" +DISTRO_BOOTSTRAP_KEYS = "${HOST_DISTRO_BOOTSTRAP_KEYS}" diff --git a/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap-target.bb b/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap-target.bb new file mode 100644 index 00000000..84a89ff1 --- /dev/null +++ b/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap-target.bb @@ -0,0 +1,12 @@ +# Minimal target Debian root file system +# +# This software is a part of Isar. +# Copyright (C) 2024 ilbers GmbH +# +# SPDX-License-Identifier: MIT + +Description = "Minimal target Debian root file system" + +DEPLOY_ISAR_BOOTSTRAP = "${DEPLOY_DIR_BOOTSTRAP}/${DISTRO}-${DISTRO_ARCH}" + +require isar-mmdebstrap.inc diff --git a/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc b/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc new file mode 100644 index 00000000..72096ae2 --- /dev/null +++ b/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc @@ -0,0 +1,230 @@ +# Minimal debian root file system +# +# This software is a part of Isar. +# Copyright (C) 2024 ilbers GmbH +# +# SPDX-License-Identifier: MIT + +inherit bootstrap +inherit compat +inherit deb-dl-dir + +FILESEXTRAPATHS:append = ":${LAYERDIR_core}/recipes-core/isar-bootstrap/files" + +ROOTFSDIR = "${WORKDIR}/rootfs" +DISTRO_BOOTSTRAP_BASE_PACKAGES = "locales,apt,usrmerge" +DISTRO_BOOTSTRAP_BASE_PACKAGES:append:gnupg = ",gnupg" +DISTRO_BOOTSTRAP_BASE_PACKAGES:append:https-support = ",ca-certificates" +BOOTSTRAP_TMPDIR = "${WORKDIR}/tempdir" + +def get_distro_primary_source_entry(d): + for source in generate_distro_sources(d): + if source[0] == "deb": + return source[2:] + bb.fatal('Invalid apt sources list') + +def get_distro_have_https_source(d): + return any(source[2].startswith("https://") for source in generate_distro_sources(d)) + +def get_distro_needs_https_support(d): + if get_distro_have_https_source(d): + return "https-support" + else: + return "" + +OVERRIDES:append = ":${@get_distro_needs_https_support(d)}" + +def get_distro_needs_gpg_support(d): + if d.getVar("DISTRO_BOOTSTRAP_KEYS") or \ + d.getVar("THIRD_PARTY_APT_KEYS") or \ + d.getVar("BASE_REPO_KEY"): + return "gnupg" + else: + return "" + +OVERRIDES:append = ":${@get_distro_needs_gpg_support(d)}" + +APT_KEYS_DIR = "${WORKDIR}/aptkeys" +DISTRO_BOOTSTRAP_KEYRING = "${WORKDIR}/distro-keyring.gpg" + +do_generate_keyrings[cleandirs] = "${APT_KEYS_DIR}" +do_generate_keyrings[dirs] = "${DL_DIR}" +do_generate_keyrings[vardeps] += "DISTRO_BOOTSTRAP_KEYS THIRD_PARTY_APT_KEYS" +do_generate_keyrings[network] = "${TASK_USE_SUDO}" +do_generate_keyrings() { + if [ -n "${@d.getVar("THIRD_PARTY_APT_KEYFILES") or ""}" ]; then + chmod 777 "${APT_KEYS_DIR}" + for keyfile in ${@d.getVar("THIRD_PARTY_APT_KEYFILES")}; do + cp "$keyfile" "${APT_KEYS_DIR}"/"$(basename "$keyfile")" + done + fi + if [ -n "${@d.getVar("DISTRO_BOOTSTRAP_KEYFILES") or ""}" ]; then + for keyfile in ${@d.getVar("DISTRO_BOOTSTRAP_KEYFILES")}; do + sudo apt-key --keyring "${DISTRO_BOOTSTRAP_KEYRING}" add $keyfile + cp "$keyfile" "${APT_KEYS_DIR}"/"$(basename "$keyfile")" + done + fi +} +addtask generate_keyrings before do_build after do_unpack + +do_bootstrap[vardeps] += " \ + DISTRO_APT_PREMIRRORS \ + ISAR_ENABLE_COMPAT_ARCH \ + ${DISTRO_VARS_PREFIX}DISTRO_APT_SOURCES \ + " +do_bootstrap[dirs] = "${DEPLOY_DIR_BOOTSTRAP} ${BOOTSTRAP_TMPDIR}" +do_bootstrap[depends] = "base-apt:do_cache isar-apt:do_cache_config" +do_bootstrap[network] = "${TASK_USE_NETWORK_AND_SUDO}" + +do_bootstrap() { + if [ "${ISAR_ENABLE_COMPAT_ARCH}" = "1" ]; then + if [ -z "${COMPAT_DISTRO_ARCH}" ]; then + bbfatal "${DISTRO_ARCH} does not have a compat arch" + fi + fi + bootstrap_args="--verbose --variant=minbase --include=${DISTRO_BOOTSTRAP_BASE_PACKAGES}" + if [ -f "${DISTRO_BOOTSTRAP_KEYRING}" ]; then + bootstrap_args="$bootstrap_args --keyring=${DISTRO_BOOTSTRAP_KEYRING}" + fi + E="${@ isar_export_proxies(d)}" + export BOOTSTRAP_FOR_HOST + + deb_dl_dir_import "${ROOTFSDIR}" "${BOOTSTRAP_BASE_DISTRO}-${BASE_DISTRO_CODENAME}" + sudo rm -rf --one-file-system "${ROOTFSDIR}" + mkdir -p "${ROOTFSDIR}" + + arch_param="--arch=${BOOTSTRAP_DISTRO_ARCH},${DISTRO_ARCH}" + + sudo TMPDIR="${BOOTSTRAP_TMPDIR}" mmdebstrap $bootstrap_args \ + $arch_param \ + --mode=unshare \ + ${@get_distro_components_argument(d)} \ + "${@get_distro_suite(d)}" \ + "${WORKDIR}/rootfs.tar.zst" \ + "${@get_distro_source(d)}" + + sudo -E -s <<'EOSUDO' + set -e + + tar -xf "${WORKDIR}/rootfs.tar.zst" -C "${ROOTFSDIR}" --exclude="./dev/console" + + # Install apt config + mkdir -p "${ROOTFSDIR}/etc/apt/preferences.d" + install -v -m644 "${APTPREFS}" \ + "${ROOTFSDIR}/etc/apt/preferences.d/bootstrap" + mkdir -p "${ROOTFSDIR}/etc/apt/sources.list.d" + if [ "${ISAR_USE_CACHED_BASE_REPO}" = "1" ]; then + line="file:///base-apt/${BOOTSTRAP_BASE_DISTRO} ${BASE_DISTRO_CODENAME} main" + if [ -z "${BASE_REPO_KEY}" ]; then + line="[trusted=yes] ${line}" + fi + echo "deb ${line}" > "${ROOTFSDIR}/etc/apt/sources.list.d/base-apt.list" + line="file:///base-apt/${BASE_DISTRO} ${BASE_DISTRO_CODENAME} main" + if [ -z "${BASE_REPO_KEY}" ]; then + line="[trusted=yes] ${line}" + fi + echo "deb-src ${line}" >> "${ROOTFSDIR}/etc/apt/sources.list.d/base-apt.list" + + mkdir -p ${ROOTFSDIR}/base-apt + mount --bind ${REPO_BASE_DIR} ${ROOTFSDIR}/base-apt + else + install -v -m644 "${APTSRCS}" \ + "${ROOTFSDIR}/etc/apt/sources.list.d/bootstrap.list" + fi + install -v -m644 "${APTSRCS_INIT}" "${ROOTFSDIR}/etc/apt/sources-list" + rm -f "${ROOTFSDIR}/etc/apt/sources.list" + rm -rf "${ROOTFSDIR}/var/lib/apt/lists/"* + find ${APT_KEYS_DIR}/ -type f | while read keyfile + do + MY_GPGHOME="$(chroot "${ROOTFSDIR}" mktemp -d /tmp/gpghomeXXXXXXXXXX)" + echo "Created temporary directory ${MY_GPGHOME} for gpg-agent" + export GNUPGHOME="${MY_GPGHOME}" + APT_KEY_APPEND="--homedir ${MY_GPGHOME}" + + kfn="$(basename $keyfile)" + cp $keyfile "${ROOTFSDIR}/tmp/$kfn" + chroot "${ROOTFSDIR}" /usr/bin/gpg-agent --daemon -- /usr/bin/apt-key \ + --keyring ${THIRD_PARTY_APT_KEYRING} ${APT_KEY_APPEND} add "/tmp/$kfn" + rm "${ROOTFSDIR}/tmp/$kfn" + + echo "Removing ${MY_GPGHOME}" + rm -rf "${ROOTFSDIR}${MY_GPGHOME}" + done + + # Set locale + install -v -m644 "${WORKDIR}/locale" "${ROOTFSDIR}/etc/locale" + + sed -i '/en_US.UTF-8 UTF-8/s/^#//g' "${ROOTFSDIR}/etc/locale.gen" + chroot "${ROOTFSDIR}" /usr/sbin/locale-gen + + # setup chroot + install -v -m755 "${WORKDIR}/chroot-setup.sh" "${ROOTFSDIR}/chroot-setup.sh" + "${ROOTFSDIR}/chroot-setup.sh" "setup" "${ROOTFSDIR}" + + # update APT + mount -o bind,private /dev ${ROOTFSDIR}/dev + mount --bind /dev/pts ${ROOTFSDIR}/dev/pts + mount -t tmpfs none "${ROOTFSDIR}/dev/shm" + mount -t proc none ${ROOTFSDIR}/proc + mount --rbind /sys ${ROOTFSDIR}/sys + mount --make-rslave ${ROOTFSDIR}/sys + + export DEBIAN_FRONTEND=noninteractive + + if [ "${BOOTSTRAP_FOR_HOST}" = "1" ]; then + chroot "${ROOTFSDIR}" /usr/bin/dpkg --add-architecture ${DISTRO_ARCH} + fi + + if [ "${ISAR_ENABLE_COMPAT_ARCH}" = "1" ]; then + chroot "${ROOTFSDIR}" /usr/bin/dpkg --add-architecture ${COMPAT_DISTRO_ARCH} + fi + + chroot "${ROOTFSDIR}" /usr/bin/apt-get update -y \ + -o APT::Update::Error-Mode=any + chroot "${ROOTFSDIR}" /usr/bin/apt-get install -y -f + chroot "${ROOTFSDIR}" /usr/bin/apt-get dist-upgrade -y \ + -o Debug::pkgProblemResolver=yes + + umount -l "${ROOTFSDIR}/dev/shm" + umount -l "${ROOTFSDIR}/dev/pts" + umount -l "${ROOTFSDIR}/dev" + umount -l "${ROOTFSDIR}/proc" + umount -l "${ROOTFSDIR}/sys" + umount -l "${ROOTFSDIR}/base-apt" || true + + # Finalize bootstrap by setting the link in deploy + ln -Tfsr "${ROOTFSDIR}" "${DEPLOY_ISAR_BOOTSTRAP}" +EOSUDO + deb_dl_dir_export "${ROOTFSDIR}" "${BOOTSTRAP_BASE_DISTRO}-${BASE_DISTRO_CODENAME}" + + # Cleanup apt cache + sudo -Es chroot "${ROOTFSDIR}" /usr/bin/apt-get -y clean +} +addtask bootstrap before do_build after do_generate_keyrings + +SSTATETASKS += "do_bootstrap" +SSTATECREATEFUNCS += "bootstrap_sstate_prepare" +SSTATEPOSTINSTFUNCS += "bootstrap_sstate_finalize" + +bootstrap_sstate_prepare() { + # this runs in SSTATE_BUILDDIR, which will be deleted automatically + sudo cp -a "$(dirname "${ROOTFSDIR}")/rootfs.tar.zst" ./bootstrap.tar.zst + sudo chown $(id -u):$(id -g) bootstrap.tar.zst +} + +bootstrap_sstate_finalize() { + # this runs in SSTATE_INSTDIR + # we should restore symlinks after using tar + if [ -f bootstrap.tar.zst ]; then + mv bootstrap.tar.zst "$(dirname "${ROOTFSDIR}")/rootfs.tar.zst" + sudo ln -Tfsr "$(dirname "${ROOTFSDIR}")/rootfs.tar.zst" \ + "${DEPLOY_ISAR_BOOTSTRAP}.tar.zst" + fi +} + +python do_bootstrap_setscene() { + sstate_setscene(d) +} + +addtask do_bootstrap_setscene +do_bootstrap_setscene[dirs] = "${DEPLOY_DIR_BOOTSTRAP}" From patchwork Mon Jul 8 06:17:33 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anton Mikanovich X-Patchwork-Id: 3665 Return-Path: Received: from shymkent.ilbers.de ([unix socket]) by shymkent (Cyrus 2.5.10-Debian-2.5.10-3+deb9u2) with LMTPA; Mon, 08 Jul 2024 08:18:01 +0200 X-Sieve: CMU Sieve 2.4 Received: from mail-wm1-f56.google.com (mail-wm1-f56.google.com [209.85.128.56]) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPS id 4686I0Qw030754 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Mon, 8 Jul 2024 08:18:00 +0200 Received: by mail-wm1-f56.google.com with SMTP id 5b1f17b1804b1-42666ed2d5fsf6659895e9.3 for ; Sun, 07 Jul 2024 23:18:00 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1720419474; cv=pass; d=google.com; s=arc-20160816; b=bZRj329kd36X+WB+zERvs/T2U6SVaz5+9rX3aKcKD1NA7SmILQZ+magS6I9+0rWJie SpeqyaxCN/f5hbWVyvRwMURbXpxSzs0jFK/K+sIMkuJDSPdCA3+NvxxOV/JloTsClRhs 9ytm3i/N6fE+vNiBEoR1y7vtBpzyLM/NbaPvAJ6KSz9NvuHROF+5H8Vxh9LWGcQSlFTI rVlmt8mVZ7NpSZaSziccRhG4skFf22h1Yr/UP+gE7JbvTfS94D6ft4WTelTfLN2T3FK+ RXrH1jzNUl3z5bHZKvsgFI3W4Ig0owpvjTgCwHpRhculnigouqYoJSw64IEKCVQG4PlP LCtw== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:dkim-signature; bh=qaTAvxfpFL/oaGCKzgCS4SCdkjfxwImwhsF2++lYCvA=; fh=VPe2DAycU4SXAMOmcnfK4UH7Jio83M1E388XPiUnOAM=; b=yFJQTK26sO6Xd2RkuQPWTnCIvdjA2qTCWP61Sb4uqv4QzYrkHcMOCq74GxKxljnd4i cCfwgn1TtO0b2p6EBXveUh8EFfFTvbEDtQxBZErIMA9vozRQ6dktAsq8y30lqmFBtzMb wfPnO5pW+ppTo3bmAWC2qQMZTqo3Cq+L+4t6yYizuiNGzbfww45x8myCdaqgG7AxGTUx ffgZVyDlM2HMKm3xAVCXG7PuPYX+Rb+P2NSP8scf5q6ZKmB0EADEyiYUu62hokNkGwQr WhIM5vB04Mc3FWWWh6KqigSB4zPaZYYew4RCwNHF0pbscUhWejbv5DL7pnjxlKI81C6O kb9w==; darn=isar-build.org ARC-Authentication-Results: i=2; gmr-mx.google.com; spf=pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=amikan@ilbers.de DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlegroups.com; s=20230601; t=1720419474; x=1721024274; darn=isar-build.org; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:x-original-authentication-results :x-original-sender:mime-version:references:in-reply-to:message-id :date:subject:cc:to:from:sender:from:to:cc:subject:date:message-id :reply-to; bh=qaTAvxfpFL/oaGCKzgCS4SCdkjfxwImwhsF2++lYCvA=; b=w62Y49QnGPSdjwL+FCZwT7thSpZWqbIve+cm1Sjiynv07SgJGDxnLtMBfBfuY0w+4F uaLWX9wUv/tRdxd8HFuESk06sQhZ3Iob6wXuDUvS4Pk+ZdXRJOnnVVLLFoiTZK/s4HEu yf9jVIc6zhyToY+FbMtZMaFeB6LOiiLZOhgmECfRAQEf5LyPfpDfEzlTroUHp0714bmM otOaWsfDN/fkoIPyOwzU/mQEvucF40yVzLnuyqSQ8A43beF7jf8Rtv5OvUDB9/cykFP4 i9erqau7q9460NGWiMfc0iUOpv672inUAPCvHcL7Y3y88iUALktvX6RFiQ0CrK5U3Bub BL6w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1720419474; x=1721024274; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :x-spam-checked-in-group:list-id:mailing-list:precedence :x-original-authentication-results:x-original-sender:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :x-beenthere:x-gm-message-state:sender:from:to:cc:subject:date :message-id:reply-to; bh=qaTAvxfpFL/oaGCKzgCS4SCdkjfxwImwhsF2++lYCvA=; b=G3JXrCaWDwyEU8CBuwSBNuafFZJxs2CKUFs0cGTeAp3xrJmfKPq1up0v84Fuc+vCDR qFgf+FsOLhWdzS4zdEbr6bfVZbH5I9tzrW1zGsJiVbRR16HAF6bY66L30NeECNMAc/q0 m4sINolEiA7qSyQZ4FOjBnsDfnXjQLZbnJtROB6Rt7FBKHbIVSKzDjoo2O3VMHC/BgLR 9wRuSCm/KBhLKxMH+5m7lcCDuq6a052YyinELx4KFO0HJXRkopXL58DmvlgW6P2HjhHc pt5toKU1wcCN5xG0R/v9PRhFmT4uIbFsNpy+2qHbkAFD+z/JNZisuve8T6srTS1iGemZ /GoQ== Sender: isar-users@googlegroups.com X-Forwarded-Encrypted: i=2; AJvYcCU5WAMQlyU9mWIub0zBiZHZfr82vRi1LNKHKXPsMsnR7jsjziMV1fSQRHUD30jQ9uAXpCdTbOJLpE6bSNNxTS3k3cU7xbQ= X-Gm-Message-State: AOJu0Yyi97Rr31odfjq1+iIjn0CISw9pB2X7kTqPPOU4xVBHqazbIZ20 5aBrBatpwGAxqBOcKxEdkfzgvH3OnM0YIrAQZL2ROM4+Vug0Ecvu X-Google-Smtp-Source: AGHT+IHNwLcJTQnq0e84vlE3RVrDRyIRElQ0Dzhn1qsYawfhvm9jTmQCk+aZ/Ix5S6GUewtAj/az+A== X-Received: by 2002:a05:600c:43c4:b0:426:5b28:3468 with SMTP id 5b1f17b1804b1-4265b283635mr45222635e9.12.1720419474266; Sun, 07 Jul 2024 23:17:54 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a05:600c:511f:b0:426:68ce:c970 with SMTP id 5b1f17b1804b1-42668cece73ls2522025e9.1.-pod-prod-04-eu; Sun, 07 Jul 2024 23:17:52 -0700 (PDT) X-Received: by 2002:a05:600c:43c4:b0:426:5b28:3468 with SMTP id 5b1f17b1804b1-4265b283635mr45222165e9.12.1720419472443; Sun, 07 Jul 2024 23:17:52 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1720419472; cv=none; d=google.com; s=arc-20160816; b=WG7B0Sqa2oinCXcn+s2l96xmLlg83E3FXu3YCFJVTff7484iNx64206cxce3WOWOsW j1Gbz4eLqncClGkkB9ikB3NFeLACpffGPKJAo/UQ1SAREsqiTJRdoAM+L9YvDwUndEta O2mE0kmXw3q0JSDvmw5c1/H+GZiSKMjYoxf1qrqaXplC99YyBGeKgcIuxIn81oJNemSb BlAOZtBEPIP88+SmAkgAyTPE+yJYDmUCtt3QE5So7BWRH74QK+h3mzMBMYzPcp18a5j+ 9wjgXfLh6lLJT/RkA7U3L+G3Bf6kIem2gXumbL+Dd4JHQBfe4XKlf6uEXQm8mmPoWHv3 +NBw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=iygpaPvr9s+HOaRDmyrAnyExbTTT2vl8cTlQAgoPvuc=; fh=/h9QQkzJ8EboVkWg45aWwpaUro6WMavIVd2OhN45RtE=; b=m450AV+I1DcOKa55wLD3/NbQHJT3PG7yPYLm08tfH6cHve0SM0J+oWPaf122IHRIuQ +fTwI2YEaHdnAiV2d+AY7BZTsVc/QDLFLpake8wv5f6JNPXqtARsVngpFHq6ZzJe66Ef nAXLxqJIZbugqMop+LTP1mfJHS+LbpGT4YtuSGmcNL94jlq++hoBDAhJOPt+ULd63RZw wOkkDweKfBqV2H05561aPC0r5ZWZ6Q8e+Z4hmJWWYBuHcdjlrKASumaEX7Si+dUa3nuB DbLhfUSoM9vAQ22NOiyGS1ckGpKIph+tecSqy/yabgPvrqTVURDqb2XQ5F1fLvlf7L6E yiOg==; dara=google.com ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=amikan@ilbers.de Received: from shymkent.ilbers.de (shymkent.ilbers.de. [85.214.156.166]) by gmr-mx.google.com with ESMTPS id 5b1f17b1804b1-42666fd630dsi1114635e9.1.2024.07.07.23.17.52 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Sun, 07 Jul 2024 23:17:52 -0700 (PDT) Received-SPF: pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) client-ip=85.214.156.166; Received: from user-B660.promwad.corp ([159.148.83.114]) (authenticated bits=0) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPSA id 4686HmUt030688 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 8 Jul 2024 08:17:51 +0200 From: Anton Mikanovich To: isar-users@googlegroups.com Cc: Anton Mikanovich Subject: [PATCH v10 3/8] meta: Allow selecting bootstrap providers Date: Mon, 8 Jul 2024 09:17:33 +0300 Message-Id: <20240708061738.311938-4-amikan@ilbers.de> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240708061738.311938-1-amikan@ilbers.de> References: <20240708061738.311938-1-amikan@ilbers.de> MIME-Version: 1.0 X-Spam-Status: No, score=-4.6 required=5.0 tests=DKIMWL_WL_MED,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_EF,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,RCVD_IN_MSPIKE_H2,RCVD_IN_RP_CERTIFIED, RCVD_IN_RP_RNBL,RCVD_IN_RP_SAFE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on shymkent.ilbers.de X-Original-Sender: amikan@ilbers.de X-Original-Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=amikan@ilbers.de Precedence: list Mailing-list: list isar-users@googlegroups.com; contact isar-users+owners@googlegroups.com List-ID: X-Spam-Checked-In-Group: isar-users@googlegroups.com X-Google-Group-Id: 914930254986 List-Post: , List-Help: , List-Archive: , List-Unsubscribe: , X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= Both isar-bootstrap or isar-mmdebstrap can be used for initial rootfs preparation. Signed-off-by: Anton Mikanovich --- meta-isar/conf/local.conf.sample | 4 ++++ meta/classes/rootfs.bbclass | 2 +- meta/conf/bitbake.conf | 3 +++ meta/recipes-core/isar-bootstrap/isar-bootstrap-host.bb | 2 ++ meta/recipes-core/isar-bootstrap/isar-bootstrap-target.bb | 2 ++ meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap-host.bb | 2 ++ meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap-target.bb | 2 ++ 7 files changed, 16 insertions(+), 1 deletion(-) diff --git a/meta-isar/conf/local.conf.sample b/meta-isar/conf/local.conf.sample index 17455015..a7862352 100644 --- a/meta-isar/conf/local.conf.sample +++ b/meta-isar/conf/local.conf.sample @@ -221,3 +221,7 @@ USER_isar[flags] += "clear-text-password" # To know more details about this variable and how to set the value refer below # https://reproducible-builds.org/docs/source-date-epoch/ #SOURCE_DATE_EPOCH = + +# Uncomment this to use old isar-bootstrap provider for rootfs prepare +#PREFERRED_PROVIDER_bootstrap-host ?= "isar-bootstrap-host" +#PREFERRED_PROVIDER_bootstrap-target ?= "isar-bootstrap-target" diff --git a/meta/classes/rootfs.bbclass b/meta/classes/rootfs.bbclass index 2e091e0c..aa3f6baa 100644 --- a/meta/classes/rootfs.bbclass +++ b/meta/classes/rootfs.bbclass @@ -183,7 +183,7 @@ rootfs_install_pkgs_install() { do_rootfs_install[root_cleandirs] = "${ROOTFSDIR}" do_rootfs_install[vardeps] += "${ROOTFS_CONFIGURE_COMMAND} ${ROOTFS_INSTALL_COMMAND}" do_rootfs_install[vardepsexclude] += "IMAGE_ROOTFS" -do_rootfs_install[depends] = "isar-bootstrap-${@'target' if d.getVar('ROOTFS_ARCH') == d.getVar('DISTRO_ARCH') else 'host'}:do_build" +do_rootfs_install[depends] = "bootstrap-${@'target' if d.getVar('ROOTFS_ARCH') == d.getVar('DISTRO_ARCH') else 'host'}:do_build" do_rootfs_install[recrdeptask] = "do_deploy_deb" do_rootfs_install[network] = "${TASK_USE_SUDO}" python do_rootfs_install() { diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf index 4cfa8b10..faaeede3 100644 --- a/meta/conf/bitbake.conf +++ b/meta/conf/bitbake.conf @@ -193,3 +193,6 @@ include conf/machine/${MACHINE}.conf include conf/distro/${DISTRO}.conf PATCHRESOLVE ?= "noop" + +PREFERRED_PROVIDER_bootstrap-host ??= "isar-mmdebstrap-host" +PREFERRED_PROVIDER_bootstrap-target ??= "isar-mmdebstrap-target" diff --git a/meta/recipes-core/isar-bootstrap/isar-bootstrap-host.bb b/meta/recipes-core/isar-bootstrap/isar-bootstrap-host.bb index 4f90fd01..12798488 100644 --- a/meta/recipes-core/isar-bootstrap/isar-bootstrap-host.bb +++ b/meta/recipes-core/isar-bootstrap/isar-bootstrap-host.bb @@ -9,6 +9,8 @@ Description = "Minimal host Debian root file system" DEPLOY_ISAR_BOOTSTRAP = "${DEPLOY_DIR_BOOTSTRAP}/${HOST_DISTRO}-host_${DISTRO}-${DISTRO_ARCH}" +PROVIDES += "bootstrap-host" + BOOTSTRAP_FOR_HOST = "1" require isar-bootstrap.inc diff --git a/meta/recipes-core/isar-bootstrap/isar-bootstrap-target.bb b/meta/recipes-core/isar-bootstrap/isar-bootstrap-target.bb index c66cb3b3..69e97b33 100644 --- a/meta/recipes-core/isar-bootstrap/isar-bootstrap-target.bb +++ b/meta/recipes-core/isar-bootstrap/isar-bootstrap-target.bb @@ -9,4 +9,6 @@ Description = "Minimal target Debian root file system" DEPLOY_ISAR_BOOTSTRAP = "${DEPLOY_DIR_BOOTSTRAP}/${DISTRO}-${DISTRO_ARCH}" +PROVIDES += "bootstrap-target" + require isar-bootstrap.inc diff --git a/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap-host.bb b/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap-host.bb index 66c8d11e..6de9c21a 100644 --- a/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap-host.bb +++ b/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap-host.bb @@ -9,6 +9,8 @@ Description = "Minimal host Debian root file system" DEPLOY_ISAR_BOOTSTRAP = "${DEPLOY_DIR_BOOTSTRAP}/${HOST_DISTRO}-host_${DISTRO}-${DISTRO_ARCH}" +PROVIDES += "bootstrap-host" + BOOTSTRAP_FOR_HOST = "1" require isar-mmdebstrap.inc diff --git a/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap-target.bb b/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap-target.bb index 84a89ff1..227ff04c 100644 --- a/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap-target.bb +++ b/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap-target.bb @@ -9,4 +9,6 @@ Description = "Minimal target Debian root file system" DEPLOY_ISAR_BOOTSTRAP = "${DEPLOY_DIR_BOOTSTRAP}/${DISTRO}-${DISTRO_ARCH}" +PROVIDES += "bootstrap-target" + require isar-mmdebstrap.inc From patchwork Mon Jul 8 06:17:34 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anton Mikanovich X-Patchwork-Id: 3669 Return-Path: Received: from shymkent.ilbers.de ([unix socket]) by shymkent (Cyrus 2.5.10-Debian-2.5.10-3+deb9u2) with LMTPA; Mon, 08 Jul 2024 08:18:04 +0200 X-Sieve: CMU Sieve 2.4 Received: from mail-lf1-f64.google.com (mail-lf1-f64.google.com [209.85.167.64]) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPS id 4686I24x030780 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Mon, 8 Jul 2024 08:18:03 +0200 Received: by mail-lf1-f64.google.com with SMTP id 2adb3069b0e04-52ea88005bfsf2324700e87.2 for ; Sun, 07 Jul 2024 23:18:03 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1720419476; cv=pass; d=google.com; s=arc-20160816; b=jR1c+agUUhi8c7J8JCpcm/7YfyxnFZSVvUGd9KQt0BmE1tXl+MdMIbRfsZVRqDm1Ik yRI5kwJ+IEFaxXKmXkPwkG462lvqm6OXGQf9X3U51kJqZxTB7H3y8j5Vxv6z+s3a8Lzz Mz7muTRHZEITpHOIbgki0OfubqnSTCk4dzBZLiyxW7BCoMQqkBGf7Mgjlb0pyDpoO7Lk dSJPSY5ne5PDyQ9R+dVLNRk7J5c5z2qmZgNO7uYcFyJWkHu57HiqJRyzZnsfHXSoiyGf Ym33sxZ4K0IJtTxIyKyLLwoLykktI9EcQTxWxCE4014SY+UmfNFqlY6LlGzfh2QxCGH5 K3Wg== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:dkim-signature; bh=gPf4YIdXJ53sy/zIOcShisZ0Z7uUgLbfZo7FXWsH6ig=; fh=yPiiAlKXl0hjsCJtYUiFNDmY9HsgUcbVlBduQo5etDk=; b=Rvom5MZxijXvH2tS9WCxV188jRGconejnGlA1pIfOjgzZvja4e2pPvl8yeJRd3md9l KvpW9yXrbQv82sBTL/ektPLZiVpkouGYglEElRLhIEnTJ5hqfHbAX+b7Ews6trpD5QKX oUS1AQUeDTorliSCh9j7DKmt9GMYS8tCiNFGwp+RbpCEAbFRADXTSouEFNjPoLbRApNE r1cO24WwUKu5p7i5Nvv5zTHir+COXYMie2U8+Btbx8As/CuKpEpeU7inUNCjNKYmJfIJ bdJACz94+GCKh9LRkNc6KJvTf7GDsQ6zQIMWwwPjYd0OalpRZ6xrXWGKGj2x6dUgKt0Q NAaw==; darn=isar-build.org ARC-Authentication-Results: i=2; gmr-mx.google.com; spf=pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=amikan@ilbers.de DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlegroups.com; s=20230601; t=1720419476; x=1721024276; darn=isar-build.org; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:x-original-authentication-results :x-original-sender:mime-version:references:in-reply-to:message-id :date:subject:cc:to:from:sender:from:to:cc:subject:date:message-id :reply-to; bh=gPf4YIdXJ53sy/zIOcShisZ0Z7uUgLbfZo7FXWsH6ig=; b=RMeDlDK5zBu+dBtH0dc9z5GylmfjdHbzY4R7Mig21oog/ADnOtrXdcixd5uQz2mDIZ vkjk3YDhoDXE/7InMjCW21z1Fv6QAn+D4Ekqg/gGHa5f3VEn/vG1Y35Wr5RI1SyGP5QQ eko7U1sISsO7il3hP7aWf7nuoTDo79HAbSfmmM5ONDhFXnzE4oaTG8yRxmSZu9yRhpYy mFkuqU0xar57e1uxyPRNoPGA9Ndug16FnBJ02gnfdPWGICl7MnS6wGI1/1tygfneV7Gl u/XtqQjF6p1VsZBn8Cbcj8DW4RfGrf99lkS1HOKoVnpUnVDS3Zpq2bPOiwYQOPtSWu0d JbPg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1720419476; x=1721024276; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :x-spam-checked-in-group:list-id:mailing-list:precedence :x-original-authentication-results:x-original-sender:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :x-beenthere:x-gm-message-state:sender:from:to:cc:subject:date :message-id:reply-to; bh=gPf4YIdXJ53sy/zIOcShisZ0Z7uUgLbfZo7FXWsH6ig=; b=bjAwP507zlfocNR/BX2T4CByJvd2C8SNg/TZIBuXLMtc2aPiF6oZM58ltnGniSo+8d PAZYPbW6KnhGGtsa43XnNxHR7doJr0Bls/p+FJTV1eCpGiEESjqPYxAGvcvawFHOr6AI N1eA0Ih/buMF+7D7Jb3vXN92gjx5YUB6h9ct94KyjsVp05Js8FCcJJFtD50HKsbIix94 gZWu+L4W3zWkxITv0B39/r0gpZf6VXSFXeBGFu0esVbC5RbdL6h5DD1sSuXtGJnqLqo5 176DTX/xTpcLD9L/P+wOmuz5VnQdNqwk7irEp64OsO5YtedC+C+ylJ4egfecPR4mVhol Ch9g== Sender: isar-users@googlegroups.com X-Forwarded-Encrypted: i=2; AJvYcCXvx8Fv0oNJL5aOWLk6dLtsVyETlsOhgfsvHAJhpcwzpX1SJfqK77h3qKPwngVCG1pF+bvHEu/CekiaLYKcpOtJp9mm9MU= X-Gm-Message-State: AOJu0YyvP6bHXssCN0uoQIYXeWAu1h4dhZBEJY1IIRiQ4qAmsM5E9sWg CWB/FEKWNMoaRq899Z6ZuOZirl715LMUnQ/Zf3SmVRC5C4Hylq4C X-Google-Smtp-Source: AGHT+IExHyg7LY3bRrGGyrCaGcoU8Kb/5frcTIJ5qZXSPUrwM9WNISM++V5DZaOHO1hp8ZdJEt0HNQ== X-Received: by 2002:a05:6512:3da3:b0:52c:d967:79c6 with SMTP id 2adb3069b0e04-52ea062dceamr8478505e87.20.1720419475672; Sun, 07 Jul 2024 23:17:55 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com Received: by 2002:ac2:5ded:0:b0:52e:9d2f:233 with SMTP id 2adb3069b0e04-52e9f16bcb7ls1743092e87.0.-pod-prod-02-eu; Sun, 07 Jul 2024 23:17:54 -0700 (PDT) X-Received: by 2002:a19:910b:0:b0:52c:c032:538d with SMTP id 2adb3069b0e04-52ea062e3bbmr6565467e87.27.1720419473625; Sun, 07 Jul 2024 23:17:53 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1720419473; cv=none; d=google.com; s=arc-20160816; b=S1l/uXUWiLy8lva3a96UZoNbBjh+955fLa3P0Shgmf+eTzikQkX0wjOyW4tpnjj9oq LYxwytk4Ss2bC8SaRLGLmb/rYHVmyQQsXAPiJCRU9+wNh2J0dfNCXwyvVIWQp8c7jDij bbIH3UElsO60Qfp35TDbLWruCRg5y2HyvkuyOf7nfSKibe7eohPxnwLmzvvsKbVy3d1U PcTu+ai7Bs8N1L0KVXF2TC+VS65D8wb2V21Qki+/dUsyCAiVZy8DNYXeseuZ+GCpv/vB G3So6oN5D6Nda1G53tkIsh4vY9DuhweR+SMZZBqfZbfO3/TwGmIg0IC1yDVOQv7l6mvC pOWQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=sBHue/9A62wDtl9abyp77JTYIgWpPHTnzbErTm5jZwE=; fh=/h9QQkzJ8EboVkWg45aWwpaUro6WMavIVd2OhN45RtE=; b=GvLqNWD71At3kCjVxVkje5rbNShKf7hcj80HfASukrdVHGGNKZB/7Lbwhf6Ud1Qp4y 7f5WjyeYNLDKFB6pl4JOIPCfFNlK7xPr2/S/5uPy6r8mTkHgK5kExDfb2x6t5n1m7RDD auQ52MlY/wujNnQyG2tYNtj5AeXM9AlP99GzksqhyKa8bv0VugU6W5wAmbO3GpSsmUOh Oi+M4P58B23ufCcyHBFI8oGeP455Q3TtbIvzr6KkvlAHWHIBC+UFCSII21AvICODyJ9M ZuwO56D3YuG0d9mZItDHpTd6YBKTQ1N1GYYdpX2ijFp1txcC83j8/oEwP7yr0VU2fdi8 wJmw==; dara=google.com ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=amikan@ilbers.de Received: from shymkent.ilbers.de (shymkent.ilbers.de. [85.214.156.166]) by gmr-mx.google.com with ESMTPS id 2adb3069b0e04-52eaf790c66si40704e87.10.2024.07.07.23.17.53 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Sun, 07 Jul 2024 23:17:53 -0700 (PDT) Received-SPF: pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) client-ip=85.214.156.166; Received: from user-B660.promwad.corp ([159.148.83.114]) (authenticated bits=0) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPSA id 4686HmUu030688 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 8 Jul 2024 08:17:52 +0200 From: Anton Mikanovich To: isar-users@googlegroups.com Cc: Anton Mikanovich Subject: [PATCH v10 4/8] testsuite: Allow variable bootstrap providers Date: Mon, 8 Jul 2024 09:17:34 +0300 Message-Id: <20240708061738.311938-5-amikan@ilbers.de> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240708061738.311938-1-amikan@ilbers.de> References: <20240708061738.311938-1-amikan@ilbers.de> MIME-Version: 1.0 X-Spam-Status: No, score=-4.6 required=5.0 tests=DKIMWL_WL_MED,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_EF,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,RCVD_IN_MSPIKE_H2,RCVD_IN_RP_CERTIFIED, RCVD_IN_RP_RNBL,RCVD_IN_RP_SAFE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on shymkent.ilbers.de X-Original-Sender: amikan@ilbers.de X-Original-Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=amikan@ilbers.de Precedence: list Mailing-list: list isar-users@googlegroups.com; contact isar-users+owners@googlegroups.com List-ID: X-Spam-Checked-In-Group: isar-users@googlegroups.com X-Google-Group-Id: 914930254986 List-Post: , List-Help: , List-Archive: , List-Unsubscribe: , X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= If we can use different recipes provide bootstrap, recipe name should be obtained from target environment. Signed-off-by: Anton Mikanovich --- testsuite/cibase.py | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-) diff --git a/testsuite/cibase.py b/testsuite/cibase.py index fd571c29..d8106941 100755 --- a/testsuite/cibase.py +++ b/testsuite/cibase.py @@ -181,8 +181,11 @@ class CIBaseTest(CIBuilder): # Rebuild image self.move_in_build_dir('tmp', 'tmp_before_sstate') self.bitbake(image_target, **kwargs) + bootstrap_target_task = CIUtils.getVars( + 'PREFERRED_PROVIDER_bootstrap-target', target=image_target + ) if not all([ - check_executed_tasks('isar-bootstrap-target', + check_executed_tasks(bootstrap_target_task, ['do_bootstrap_setscene', '!do_bootstrap']), check_executed_tasks('sbuild-chroot-target', ['do_rootfs_install_setscene', '!do_rootfs_install']), @@ -206,7 +209,7 @@ class CIBaseTest(CIBuilder): self.move_in_build_dir('tmp', 'tmp_middle_sstate') self.bitbake(package_target, **kwargs) if not all([ - check_executed_tasks('isar-bootstrap-target', + check_executed_tasks(bootstrap_target_task, ['do_bootstrap_setscene']), check_executed_tasks('sbuild-chroot-target', ['!do_sbuildchroot_deploy']), @@ -220,7 +223,7 @@ class CIBaseTest(CIBuilder): process.run(f'find {self.build_dir}/sstate-cache/ -name sstate:hello:* -delete') self.bitbake(image_target, **kwargs) if not all([ - check_executed_tasks('isar-bootstrap-target', + check_executed_tasks(bootstrap_target_task, ['do_bootstrap_setscene', '!do_bootstrap']), check_executed_tasks('sbuild-chroot-target', ['do_rootfs_install_setscene', '!do_rootfs_install']), From patchwork Mon Jul 8 06:17:35 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anton Mikanovich X-Patchwork-Id: 3667 Return-Path: Received: from shymkent.ilbers.de ([unix socket]) by shymkent (Cyrus 2.5.10-Debian-2.5.10-3+deb9u2) with LMTPA; Mon, 08 Jul 2024 08:18:02 +0200 X-Sieve: CMU Sieve 2.4 Received: from mail-wm1-f64.google.com (mail-wm1-f64.google.com [209.85.128.64]) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPS id 4686I1v4030766 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Mon, 8 Jul 2024 08:18:01 +0200 Received: by mail-wm1-f64.google.com with SMTP id 5b1f17b1804b1-4265f9cd54csf6596835e9.0 for ; Sun, 07 Jul 2024 23:18:01 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1720419476; cv=pass; d=google.com; s=arc-20160816; b=Ih/YLOxJgSWtBHfbBQgP7Yn519eu0tWJ7uxNT7jhsPdOKJKoGtIwNOCte+v6Su7Aj+ M3YMBo3rZeN9skqhLcDjpmehsW5xmij0NBR2feIOxh6k7IhJp+/GXE2mBws6SctJa/y1 XCa3m8b7WBY86MO4mTNSTa1LSvnNZf5HriuU/UiOPmKTc+Oz+sMfsImYGl0eiJI/SGHw KWL8CeZObFaRrmPkDSgC0AQQYIsCI+z6OVIK5ILFl5YlDpDAFU00vBXd1zA2i6Idz/lw VGaDxZIm+6ITC6s7AMRTBvrom7nk9LaFiZYyFeYswXSUjqZHNi8GQSbNL/hHfqrD8bjf fPUQ== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:dkim-signature; bh=mmB7gdKdlYGhQWLRBlai1TLm4Pa8yrPiWepBNhrveJU=; fh=ZIqoXvkOKgazGp5fK82NHbSxUFzHEL+FQn5lDCg7PO4=; b=DSoRkFgU20UGIJrouG4Wy5P5cm6x/LF9NPIHGjj0hzxHJXtQ8n1TiNobKzjFktocJn /e+xREbFAH0umezH9itYunogxEggUsJb5JiZrkYChE9sbGOu6UOpJxYoaaqqKSmGIs1o fo1Zg73fdz7gvv/Gr+oo9Pl1ZgPxTShER+ZUiZhOzZkLPiYs+5oCXKcrzZ9M8Ckbls9n Lkln5n4joiS4286rsrNCfxU7FZsdgOS3ps3OpxNIH5Bgzhp9Xn7WnP/lEDAq6/lRB+h/ pH9wGc3yj3AFZzlq1OqMyEoQo+EVmQsF+Av6LAWi4Inh7ShAdU5FvfTSp36j2aDd7YPx MyGg==; darn=isar-build.org ARC-Authentication-Results: i=2; gmr-mx.google.com; spf=pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=amikan@ilbers.de DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlegroups.com; s=20230601; t=1720419476; x=1721024276; darn=isar-build.org; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:x-original-authentication-results :x-original-sender:mime-version:references:in-reply-to:message-id :date:subject:cc:to:from:sender:from:to:cc:subject:date:message-id :reply-to; bh=mmB7gdKdlYGhQWLRBlai1TLm4Pa8yrPiWepBNhrveJU=; b=D6BPNW1Xk3DZp1KP/d33A3tpuSWyQi/dzssgseqBRpEIh5asyh9cGkqF/9sO478Gs7 Rzw6c2ljOidJQcoYYC9fPab/k9jHD9mO+YrJAk5oUdReACHjWv6gYnu1ypLOQF1NsPiI SRZcNxiHWVjh+cLtTEtZXYGiCP4gpUN8uqwmZgG5u6zwhIjFoS+cy24FrYk+7luX08RU AZ4SdsDeR75dPPrRqFZaqsALXMkENZnfE+sj1GIiakvepihTIBEhce5PcGtlCtD3zAfe TVYBlcXvQ7Fj/hCtBoB/HLAZO57IdQeyh9dg/qHRH+KINiBFlgcHempsECYuABCbmU3S Xekw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1720419476; x=1721024276; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :x-spam-checked-in-group:list-id:mailing-list:precedence :x-original-authentication-results:x-original-sender:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :x-beenthere:x-gm-message-state:sender:from:to:cc:subject:date :message-id:reply-to; bh=mmB7gdKdlYGhQWLRBlai1TLm4Pa8yrPiWepBNhrveJU=; b=iglWqMiMojxLY+v89erulrpq8xeGDz7b8gi/pdb5Yjb8fcUNYhyL9qK5k1HxQLNoqD Lv9VhTPeLbdnQFo3P0nDI20rNZOGPwbNz2bJf6xcBc4Yv3NTdiOPI4SUw5WVVF+xzxot kaiaRX0yR2eYyg2XB4YnXGx1k6qLASeVRntEHGvm+MUFo0yDMJvSMCh7vCf7Jyx5ghP3 ijCcc1gVXHHwd3MP3ZGagVezXexThyMiSi4/voY90RFoVOCupmGoQoP7L9/nzikxIfQz YU2d+tiOyBx3+0pGAeFkDj1La6aQIsQH4zPbIC8k8BRJTbCXpAAD0CAnIv9K7QP/Vc3m YAvQ== Sender: isar-users@googlegroups.com X-Forwarded-Encrypted: i=2; AJvYcCWSnTGslfEdEX3AkyzWbmSeKaQ203fARP39pu/EC/Qa4F39tC5Zh5otcsA1zauJ3qqSI4F6gO76MBFkRhQAjpxi3SQMlwc= X-Gm-Message-State: AOJu0YzIuGILI5OeLNdWwy0vFP43zLnwiX41fsKEPBMHopD6t+PWl0Np bKEK21EE+xziKWMK/O55RIUuKCJwpRUy/6/UDjNMN3g3zoBGf4bE X-Google-Smtp-Source: AGHT+IGbwFxYmkQwlFHqw/PJYO/7I0Oj5ulNjjkF4EMYhIHUMegv5thGa+7aFuCKHIKYsn+wIUB6ag== X-Received: by 2002:a05:600c:4f42:b0:424:8be4:f2c with SMTP id 5b1f17b1804b1-4264b0cc4b0mr94723755e9.2.1720419475778; Sun, 07 Jul 2024 23:17:55 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a05:600c:308a:b0:426:5216:9af1 with SMTP id 5b1f17b1804b1-42652169b58ls10522035e9.1.-pod-prod-00-eu; Sun, 07 Jul 2024 23:17:54 -0700 (PDT) X-Received: by 2002:a5d:6e5e:0:b0:363:337a:3e0 with SMTP id ffacd0b85a97d-3679f6c48ecmr8500441f8f.1.1720419473916; Sun, 07 Jul 2024 23:17:53 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1720419473; cv=none; d=google.com; s=arc-20160816; b=u2UgK6gx9E76unfSisKvKssGJLG5GODnZT8tilUZUqRA81HshOVKltrPCmj6dX0ko2 Bx3vZbnchc3bryElr1/xARxawoefGL7+YmOJExfH/jgjBE3QyCJGam9mxe0LF66ONXrK eFOfqJoStiQjHavTkIFQ5RuC1dTKGippj46UGnowd9bpZSvwD37js1EB63dRtmBKTF33 VpAazLGFvB/dWEz086a3POTXDRWW+ekaolO/FvCbXBCPsjpVJPa01k6mDOiIU4Yro0uR +O3l6AOEXGg0VlkSPhU+zrd9jTdA7MWyu/MvioOzDkiqbNpMMDfsRCkFcHEFR0cr11QH bRnA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=3Lz6cPDKcV5Rgf80zj6VnfUs8knvXQN69JfYYVHa8yY=; fh=/h9QQkzJ8EboVkWg45aWwpaUro6WMavIVd2OhN45RtE=; b=laEpNwkq2Ny9nVXeUnrP2NGAWBOYM3kbU2mCWXQx/ioeF5mi0CnaYbahxiHwmr6ZOR 6k2ebWhpl8VKv2INmcc8YJ8mGOwSRzS2+R/NjSpGDbTmXtCJHIRoNuAFRSZakMfj/lJT g0iIDtwd2+hGzyCUlKQKAfS8I86gIShHrj8OjJ76avY/RJKz1Rv4QM0TMxRuBIzhMZjA XVp1rETLfaRmMQbaP+gHdgKYEZqv4uqTBlEHh+YjMedWPxBXUWv9a69Jh8l53CRHGZzx y0s2YFxxc6kgI0z+16yrm/VWs9mI8uC+xA3w5WwuzZZugf3hfe85TYbWmznT/7GAMQXl AKKQ==; dara=google.com ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=amikan@ilbers.de Received: from shymkent.ilbers.de (shymkent.ilbers.de. [85.214.156.166]) by gmr-mx.google.com with ESMTPS id 5b1f17b1804b1-42662c6a252si1532715e9.0.2024.07.07.23.17.53 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Sun, 07 Jul 2024 23:17:53 -0700 (PDT) Received-SPF: pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) client-ip=85.214.156.166; Received: from user-B660.promwad.corp ([159.148.83.114]) (authenticated bits=0) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPSA id 4686HmUv030688 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 8 Jul 2024 08:17:53 +0200 From: Anton Mikanovich To: isar-users@googlegroups.com Cc: Anton Mikanovich Subject: [PATCH v10 5/8] mmdebstrap: Fix missing dpkg available Date: Mon, 8 Jul 2024 09:17:35 +0300 Message-Id: <20240708061738.311938-6-amikan@ilbers.de> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240708061738.311938-1-amikan@ilbers.de> References: <20240708061738.311938-1-amikan@ilbers.de> MIME-Version: 1.0 X-Spam-Status: No, score=-4.6 required=5.0 tests=DKIMWL_WL_MED,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_EF,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,RCVD_IN_MSPIKE_H2,RCVD_IN_RP_CERTIFIED, RCVD_IN_RP_RNBL,RCVD_IN_RP_SAFE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on shymkent.ilbers.de X-Original-Sender: amikan@ilbers.de X-Original-Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=amikan@ilbers.de Precedence: list Mailing-list: list isar-users@googlegroups.com; contact isar-users+owners@googlegroups.com List-ID: X-Spam-Checked-In-Group: isar-users@googlegroups.com X-Google-Group-Id: 914930254986 List-Post: , List-Help: , List-Archive: , List-Unsubscribe: , X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= In some distros /var/lib/dpkg/available is required for do_rootfs_install task. The fix for it is present in jessie-or-older mmdebstrap hook in Bookworm and later, but maybe-jessie-or-older wrapper do not handle distro version check correctly. That's why we need to check and apply hook manually or create /var/lib/dpkg/available by hand for older distros. Signed-off-by: Anton Mikanovich --- meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc b/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc index 72096ae2..4f56a0b8 100644 --- a/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc +++ b/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc @@ -17,6 +17,14 @@ DISTRO_BOOTSTRAP_BASE_PACKAGES:append:gnupg = ",gnupg" DISTRO_BOOTSTRAP_BASE_PACKAGES:append:https-support = ",ca-certificates" BOOTSTRAP_TMPDIR = "${WORKDIR}/tempdir" +# Fix for /var/lib/apt/available while maybe-jessie-or-older hook do not work +MM_HOOK_JESSIE = "/usr/share/mmdebstrap/hooks/jessie-or-older" +DPKG_HOOKS ?= "${@'--hook-dir='+d.getVar('MM_HOOK_JESSIE') \ + if os.path.isdir(d.getVar('MM_HOOK_JESSIE')) \ + else '--customize-hook=\'touch /var/lib/dpkg/available\''}" +MMHOOKS:ubuntu-focal ?= "${DPKG_HOOKS}" +MMHOOKS:debian-buster ?= "${DPKG_HOOKS}" + def get_distro_primary_source_entry(d): for source in generate_distro_sources(d): if source[0] == "deb": @@ -98,6 +106,7 @@ do_bootstrap() { sudo TMPDIR="${BOOTSTRAP_TMPDIR}" mmdebstrap $bootstrap_args \ $arch_param \ --mode=unshare \ + ${MMHOOKS} \ ${@get_distro_components_argument(d)} \ "${@get_distro_suite(d)}" \ "${WORKDIR}/rootfs.tar.zst" \ From patchwork Mon Jul 8 06:17:36 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anton Mikanovich X-Patchwork-Id: 3668 Return-Path: Received: from shymkent.ilbers.de ([unix socket]) by shymkent (Cyrus 2.5.10-Debian-2.5.10-3+deb9u2) with LMTPA; Mon, 08 Jul 2024 08:18:03 +0200 X-Sieve: CMU Sieve 2.4 Received: from mail-ed1-f61.google.com (mail-ed1-f61.google.com [209.85.208.61]) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPS id 4686I243030790 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Mon, 8 Jul 2024 08:18:03 +0200 Received: by mail-ed1-f61.google.com with SMTP id 4fb4d7f45d1cf-58d134ecd3dsf3048137a12.1 for ; Sun, 07 Jul 2024 23:18:03 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1720419477; cv=pass; d=google.com; s=arc-20160816; b=UU9YO3gjlrgLV3HCHpnDH42ImNIvktIsAgOVP5N7bjeJNYjeQYwAsyafsv3DXtjGvP G7oqh8ftC9fNUt1YvCITjW4RpB9poIDeE0o5gh5M6fdJa+Mbgt5VzStF1x9qd6lDlcea 4jL5wJxOywSQEU7v6H0yI/J31+ZkAwQVlRnMAIiA7CV7ELhbY0Skd8ujH83A7+fzdagJ yzux1YmuXst6F//YJ5/SpYT0iBNDZdOmGQfsPJro/3lSdDO6oN21akGJU8v5L2qz7Dua aeAtY0U7jFf8wJmmsJ7wFiTRFKDigmICxfrmUrrxxQ3ynYuuSIJfL6Drd+0bxyqvMNz3 OgxA== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:dkim-signature; bh=VnzwuO2A5sIl/Bhi3h2mqqY/ufKxMQK1J1u/W08PKlw=; fh=Ezjv/6LW2SqBxBI5CiHC7jQoQXiYdSH9jp37H+/s/uU=; b=OGqHvfJuK3R6yZk2l+XJ/LJuX0oZ7vu6r8pvyAs5q4+jVC4kjquAOPjdjKIctkfyP1 ESm+8PcMdk+QeO1wFE7RodiMNiH6MKMsDoZ2+iZMUhLaMrhzmr5y+BToTbCpUetAStf2 spM2xVGTS6gVTUI9cfISzHwg6vop70ehwokTzIckRCltw/IlPGD+5UA7Je8oxXt7XLp3 h6JiwMBUScQJQRPrvgZNPLFnP4kdT8pY19PMDQRuCf4UHFo5q3uGNijH6nNVrhkyW8Bd Mw3VqoCbYPeFEVftkMZIdfWACADlt4xsqGP11KRYnQkIya/woMJSiwpSnwN3UWo6lVDD skjg==; darn=isar-build.org ARC-Authentication-Results: i=2; gmr-mx.google.com; spf=pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=amikan@ilbers.de DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlegroups.com; s=20230601; t=1720419477; x=1721024277; darn=isar-build.org; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:x-original-authentication-results :x-original-sender:mime-version:references:in-reply-to:message-id :date:subject:cc:to:from:sender:from:to:cc:subject:date:message-id :reply-to; bh=VnzwuO2A5sIl/Bhi3h2mqqY/ufKxMQK1J1u/W08PKlw=; b=TfBdUuVaRoxS2Df56zI6HRNgq6F+VC1Tm7n4pugBgn9MeGhhBOKZYuPxIDTyrp5FVQ Bfc1074yXcCRCE/ESie5rcvLs93dlLuo9nRbyKf01tzS0BJFK8gHXR/QG5YuZeL6N8E2 NUv11H6ckQPVEn3hUj/o733jSKCvEYdiZMSydzDijMObfUI31WI7Kd77RbHt5ID6Nn4O DGuvrLpC7cFr5AdRcU4I8oHH8d7AAxnr2LOszeQE1iSWuyE40v1ZZsmjhgaxk2WwZwM1 ZsQ+jzHSlGkwELml5AHn/THxKalUqoAnjBaC6ytokvWpuQcbwuJ6OZaIsnZZXW/sSAOf IkgQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1720419477; x=1721024277; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :x-spam-checked-in-group:list-id:mailing-list:precedence :x-original-authentication-results:x-original-sender:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :x-beenthere:x-gm-message-state:sender:from:to:cc:subject:date :message-id:reply-to; bh=VnzwuO2A5sIl/Bhi3h2mqqY/ufKxMQK1J1u/W08PKlw=; b=pOUmBJzs3f4llm3e4xH6mE8nIwwR1GLxIUBl+8yIfhqQXaMDkSsmH+yjJJ9ZXHnC9f J9GY6fMjd0GfgZ5gjxuR6aO0SRsrIgiEXCSxYapbG4wM5b+SwVcKzGdgdZFuDeELn1X1 0OWml6GlcmjCg/sFAObLW7gEVKBKm6yizci0LunbStbsqftmReZL97ysfWWpT5ofBOMW iAQ4WtzMA3vjK6JEDcqZ1WMBC7mZfHn1YvXzfvQg9s72GORFxYZtlR5Dxha9hwEvviAJ 0F7ZocoZ7Bo3WiZ+dpcNCUfeVPEK/dfPrR0ecwKfAbLstJ9tylko7RKG8PqJ3gWmz6x2 Lqgw== Sender: isar-users@googlegroups.com X-Forwarded-Encrypted: i=2; AJvYcCV4PrRrfnbl6ayeHemdsCWlvWhaTkFF0JN+DvEduNNHCNeL9Vqm5X9sWIzhqKz92EdNRbxS/B3O7jxJ483O7d0XEaXomPw= X-Gm-Message-State: AOJu0Yyr4cs39LW6cJ+zmwskUE/6hI+pHGuKdgH4Xbm9mpx6eQ7R1MpL Rb/PWWAneD2qsfto+THnIsOdPPwDtco0eN4toogRkNYC00NpiuuS X-Google-Smtp-Source: AGHT+IE5GmrX2CtEFDRN/LJGKiaxYLlvV17rqzMFzzHPpxnhN/bjuL+ZyfGSVEYq2KWd5nHBS3OeXg== X-Received: by 2002:a05:6402:2803:b0:58e:450b:da08 with SMTP id 4fb4d7f45d1cf-58e5b4aefdfmr8107525a12.31.1720419476669; Sun, 07 Jul 2024 23:17:56 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a05:6402:34c1:b0:58b:b493:fcc1 with SMTP id 4fb4d7f45d1cf-58e20192d0fls1705475a12.0.-pod-prod-06-eu; Sun, 07 Jul 2024 23:17:55 -0700 (PDT) X-Received: by 2002:a05:6402:27d0:b0:58d:10a0:36e2 with SMTP id 4fb4d7f45d1cf-58e59c40320mr7932350a12.20.1720419474737; Sun, 07 Jul 2024 23:17:54 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1720419474; cv=none; d=google.com; s=arc-20160816; b=IkV4kvFDXnJ6NYrUnAIoUxMIf+JpZ9jO31mYPcosBK0HuLFmnBlX5UGzPqIkvYQHeH BeW6GDLRGFBj2RaCY3zbqtMfk4ZSZM42egxsFaVVHgNVHHa1PdBxEh4U7DwiBpfbhs4u TcXvJ7a/271+p/9LW59zn0ad0iT2fTQK+GUclgx7mlxStHqxMWuNAQAAcBiLURap3WPI 8aoPvFxg0qQaqoAcXIE3CL3293uQ9SNKAoera0g9Y+Hj4U8W+guQz4UcFEUCUnl/x90/ QgcqjbS6Q+IWqbG0bK+TDrWrJ8xbYYZ/ubNbdR5SfdcPDPl+wiYPxTop0vsOb7b/kaPl QLKQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=r26ZZDuAPq7EDxfs9w7PznlG/tFWE7XmeH0N52DJOWg=; fh=/h9QQkzJ8EboVkWg45aWwpaUro6WMavIVd2OhN45RtE=; b=tjH2H+AtEoHFTWcsK5nso1Y9LyxUxUiZYI4jn6cY/0PS5McwCvJf8O4mwWw9sJloZw 8nnNplsUXmLSCKUp9vhIDmfAxzpMuKn97eh0t+ZXNW62kh1DwWNaZl/0pFl9cVpTLF6Q x24XJ2YUAb+Xq+ah473TtGGHX2t8NgNw8jzPCtdr3xpl2kI6cNGk/nYtoYiugEzhrQxD OFShgRbOmjrc7Ijap7hkMr72tIurRx078Ae+BMVnCfQ7zfcBoPU+IdHUr8obYsqMnVyb leyrTUuD2biwZbJTl1tHwYVU7h1Vr4omvwSFvT66YXwVGJXFj+ETpSSJ1LAWvVNlVBvt 39XA==; dara=google.com ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=amikan@ilbers.de Received: from shymkent.ilbers.de (shymkent.ilbers.de. [85.214.156.166]) by gmr-mx.google.com with ESMTPS id 4fb4d7f45d1cf-58b396553a5si291369a12.0.2024.07.07.23.17.54 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Sun, 07 Jul 2024 23:17:54 -0700 (PDT) Received-SPF: pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) client-ip=85.214.156.166; Received: from user-B660.promwad.corp ([159.148.83.114]) (authenticated bits=0) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPSA id 4686HmUw030688 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 8 Jul 2024 08:17:54 +0200 From: Anton Mikanovich To: isar-users@googlegroups.com Cc: Anton Mikanovich Subject: [PATCH v10 6/8] mmdebstrap: Move preparations to hooks Date: Mon, 8 Jul 2024 09:17:36 +0300 Message-Id: <20240708061738.311938-7-amikan@ilbers.de> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240708061738.311938-1-amikan@ilbers.de> References: <20240708061738.311938-1-amikan@ilbers.de> MIME-Version: 1.0 X-Spam-Status: No, score=-4.6 required=5.0 tests=DKIMWL_WL_MED,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_EF,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,RCVD_IN_MSPIKE_H2,RCVD_IN_RP_CERTIFIED, RCVD_IN_RP_RNBL,RCVD_IN_RP_SAFE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on shymkent.ilbers.de X-Original-Sender: amikan@ilbers.de X-Original-Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=amikan@ilbers.de Precedence: list Mailing-list: list isar-users@googlegroups.com; contact isar-users+owners@googlegroups.com List-ID: X-Spam-Checked-In-Group: isar-users@googlegroups.com X-Google-Group-Id: 914930254986 List-Post: , List-Help: , List-Archive: , List-Unsubscribe: , X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= Move all bootstrap rootfs prepare logic into mmdebstrap hooks. Also migrate from apt-key to gpg. Signed-off-by: Anton Mikanovich --- .../isar-mmdebstrap/isar-mmdebstrap.inc | 237 +++++++++--------- 1 file changed, 113 insertions(+), 124 deletions(-) diff --git a/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc b/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc index 4f56a0b8..c41dd3a1 100644 --- a/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc +++ b/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc @@ -13,8 +13,6 @@ FILESEXTRAPATHS:append = ":${LAYERDIR_core}/recipes-core/isar-bootstrap/files" ROOTFSDIR = "${WORKDIR}/rootfs" DISTRO_BOOTSTRAP_BASE_PACKAGES = "locales,apt,usrmerge" -DISTRO_BOOTSTRAP_BASE_PACKAGES:append:gnupg = ",gnupg" -DISTRO_BOOTSTRAP_BASE_PACKAGES:append:https-support = ",ca-certificates" BOOTSTRAP_TMPDIR = "${WORKDIR}/tempdir" # Fix for /var/lib/apt/available while maybe-jessie-or-older hook do not work @@ -25,53 +23,29 @@ DPKG_HOOKS ?= "${@'--hook-dir='+d.getVar('MM_HOOK_JESSIE') \ MMHOOKS:ubuntu-focal ?= "${DPKG_HOOKS}" MMHOOKS:debian-buster ?= "${DPKG_HOOKS}" -def get_distro_primary_source_entry(d): - for source in generate_distro_sources(d): - if source[0] == "deb": - return source[2:] - bb.fatal('Invalid apt sources list') - -def get_distro_have_https_source(d): - return any(source[2].startswith("https://") for source in generate_distro_sources(d)) - -def get_distro_needs_https_support(d): - if get_distro_have_https_source(d): - return "https-support" - else: - return "" - -OVERRIDES:append = ":${@get_distro_needs_https_support(d)}" - -def get_distro_needs_gpg_support(d): - if d.getVar("DISTRO_BOOTSTRAP_KEYS") or \ - d.getVar("THIRD_PARTY_APT_KEYS") or \ - d.getVar("BASE_REPO_KEY"): - return "gnupg" - else: - return "" - -OVERRIDES:append = ":${@get_distro_needs_gpg_support(d)}" - -APT_KEYS_DIR = "${WORKDIR}/aptkeys" DISTRO_BOOTSTRAP_KEYRING = "${WORKDIR}/distro-keyring.gpg" -do_generate_keyrings[cleandirs] = "${APT_KEYS_DIR}" -do_generate_keyrings[dirs] = "${DL_DIR}" +do_generate_keyrings[cleandirs] = "${WORKDIR}/trusted.gpg.d" +do_generate_keyrings[dirs] = "${DEBDIR}" do_generate_keyrings[vardeps] += "DISTRO_BOOTSTRAP_KEYS THIRD_PARTY_APT_KEYS" do_generate_keyrings[network] = "${TASK_USE_SUDO}" do_generate_keyrings() { + export GNUPGHOME="$(mktemp -td gpghomeXXXXXXXXXX)" if [ -n "${@d.getVar("THIRD_PARTY_APT_KEYFILES") or ""}" ]; then - chmod 777 "${APT_KEYS_DIR}" for keyfile in ${@d.getVar("THIRD_PARTY_APT_KEYFILES")}; do - cp "$keyfile" "${APT_KEYS_DIR}"/"$(basename "$keyfile")" + gpg --no-autostart --keyring "gnupg-ring:${DISTRO_BOOTSTRAP_KEYRING}" \ + --no-default-keyring --import $keyfile done fi if [ -n "${@d.getVar("DISTRO_BOOTSTRAP_KEYFILES") or ""}" ]; then for keyfile in ${@d.getVar("DISTRO_BOOTSTRAP_KEYFILES")}; do - sudo apt-key --keyring "${DISTRO_BOOTSTRAP_KEYRING}" add $keyfile - cp "$keyfile" "${APT_KEYS_DIR}"/"$(basename "$keyfile")" + gpg --no-autostart --keyring "gnupg-ring:${DISTRO_BOOTSTRAP_KEYRING}" \ + --no-default-keyring --import $keyfile done fi + if [ -r "${DISTRO_BOOTSTRAP_KEYRING}" ]; then + chmod o+r "${DISTRO_BOOTSTRAP_KEYRING}" + fi } addtask generate_keyrings before do_build after do_unpack @@ -80,10 +54,12 @@ do_bootstrap[vardeps] += " \ ISAR_ENABLE_COMPAT_ARCH \ ${DISTRO_VARS_PREFIX}DISTRO_APT_SOURCES \ " -do_bootstrap[dirs] = "${DEPLOY_DIR_BOOTSTRAP} ${BOOTSTRAP_TMPDIR}" +do_bootstrap[dirs] = "${DEPLOY_DIR_BOOTSTRAP} ${BOOTSTRAP_TMPDIR} ${WORKDIR}/trusted.gpg.d ${WORKDIR}/sources.list.d" do_bootstrap[depends] = "base-apt:do_cache isar-apt:do_cache_config" do_bootstrap[network] = "${TASK_USE_NETWORK_AND_SUDO}" +DEB_DL_LOCK ?= "${DEBDIR}/${BOOTSTRAP_BASE_DISTRO}-${BASE_DISTRO_CODENAME}.lock" + do_bootstrap() { if [ "${ISAR_ENABLE_COMPAT_ARCH}" = "1" ]; then if [ -z "${COMPAT_DISTRO_ARCH}" ]; then @@ -93,121 +69,134 @@ do_bootstrap() { bootstrap_args="--verbose --variant=minbase --include=${DISTRO_BOOTSTRAP_BASE_PACKAGES}" if [ -f "${DISTRO_BOOTSTRAP_KEYRING}" ]; then bootstrap_args="$bootstrap_args --keyring=${DISTRO_BOOTSTRAP_KEYRING}" + cp "${DISTRO_BOOTSTRAP_KEYRING}" "${WORKDIR}/trusted.gpg.d/" fi E="${@ isar_export_proxies(d)}" - export BOOTSTRAP_FOR_HOST - deb_dl_dir_import "${ROOTFSDIR}" "${BOOTSTRAP_BASE_DISTRO}-${BASE_DISTRO_CODENAME}" sudo rm -rf --one-file-system "${ROOTFSDIR}" mkdir -p "${ROOTFSDIR}" + if [ "${ISAR_USE_CACHED_BASE_REPO}" = "1" ]; then + base_apt_tmp="$(mktemp -d /tmp/isar-base-aptXXXXXXXXXX)" + bootstrap_list="${WORKDIR}/sources.list.d/base-apt.list" + line="copy://$base_apt_tmp/${BOOTSTRAP_BASE_DISTRO} ${BASE_DISTRO_CODENAME} main" + if [ -z "${BASE_REPO_KEY}" ]; then + line="[trusted=yes] ${line}" + fi + echo "deb ${line}" > "${WORKDIR}/sources.list.d/base-apt.list" + line="copy://$base_apt_tmp/${BASE_DISTRO} ${BASE_DISTRO_CODENAME} main" + if [ -z "${BASE_REPO_KEY}" ]; then + line="[trusted=yes] ${line}" + fi + echo "deb-src ${line}" >> "${WORKDIR}/sources.list.d/base-apt.list" + + # no need to sync /var/cache/apt/archives if base-apt used + syncin='echo skip sync-in' + syncout='echo skip sync-out' + extra_setup="mount --bind '${REPO_BASE_DIR}' $base_apt_tmp" + extra_extract="$syncout" + # save mmdebstrap tempdir for cleanup + extra_essential="mkdir -p \$1/$base_apt_tmp && \ + echo \$1 > ${WORKDIR}/mmtmpdir && \ + mount --bind '${REPO_BASE_DIR}' \$1/$base_apt_tmp" + # replace base-apt mount in tmp with /base-apt mount + extra_customize="sed -i \"s|copy://$base_apt_tmp|file:///base-apt|g\" \ + \$1/etc/apt/sources.list.d/*.list && \ + mkdir -p \$1/base-apt && \ + mount --bind '${REPO_BASE_DIR}' \$1/base-apt && \ + chroot \$1 apt-get update -y \ + -o APT::Update::Error-Mode=any && \ + umount \$1/base-apt && \ + umount \$1/$base_apt_tmp && rm ${WORKDIR}/mmtmpdir && \ + umount $base_apt_tmp && rm -rf --one-file-system $base_apt_tmp" + else + deb_dl_dir_import "${WORKDIR}/dl_dir" "${BOOTSTRAP_BASE_DISTRO}-${BASE_DISTRO_CODENAME}" + + bootstrap_list="${WORKDIR}/sources.list.d/bootstrap.list" + install -v -m644 "${APTSRCS}" \ + "${WORKDIR}/sources.list.d/bootstrap.list" + + syncin='flock -s ${DEB_DL_LOCK} cp -n --no-preserve=owner \ + "${WORKDIR}/dl_dir/var/cache/apt/archives/"*.deb \ + "$1/var/cache/apt/archives/" || true' + syncout='flock -s ${DEB_DL_LOCK} cp -n --no-preserve=owner \ + "$1/var/cache/apt/archives/"*.deb \ + "${WORKDIR}/dl_dir/var/cache/apt/archives/"' + extra_setup="$syncin" + extra_extract="$syncout" + # prefetch apt debs because mmdebstrap will clean them on next stage + extra_essential='apt-get install apt -y -d \ + -o Dir::State="$1/var/lib/apt" \ + -o Dir::Etc="$1/etc/apt" \ + -o Dir::Cache="$1/var/cache/apt" \ + -o Apt::Architecture="${BOOTSTRAP_DISTRO_ARCH}"' + extra_essential="$extra_essential && $syncout" + extra_customize="$syncout" + fi + + if [ ! -z "${SOURCE_DATE_EPOCH}" ]; then + export SOURCE_DATE_EPOCH="${SOURCE_DATE_EPOCH}" + fi + arch_param="--arch=${BOOTSTRAP_DISTRO_ARCH},${DISTRO_ARCH}" + if [ "${ISAR_ENABLE_COMPAT_ARCH}" = "1" ]; then + arch_param="$arch_param,${COMPAT_DISTRO_ARCH}" + fi + + # Cleanup mounts if fails + trap 'exit 1' INT HUP QUIT TERM ALRM USR1 + trap '[ -r "${WORKDIR}/mmtmpdir" ] && tmpdir=$(cat "${WORKDIR}/mmtmpdir") \ + && rm "${WORKDIR}/mmtmpdir"; \ + [ -d "$tmpdir" ] && mountpoint -q $tmpdir/$base_apt_tmp \ + && sudo umount $tmpdir/$base_apt_tmp; \ + [ -d "$tmpdir" ] && mountpoint -q $tmpdir/base-apt \ + && sudo umount $tmpdir/base-apt; \ + [ -d "$tmpdir" ] && sudo rm -rf --one-file-system $tmpdir; \ + [ -n "$base_apt_tmp" ] && mountpoint -q $base_apt_tmp \ + && sudo umount $base_apt_tmp \ + && rm -rf --one-file-system $base_apt_tmp' EXIT sudo TMPDIR="${BOOTSTRAP_TMPDIR}" mmdebstrap $bootstrap_args \ $arch_param \ --mode=unshare \ ${MMHOOKS} \ + --setup-hook='mkdir -p "$1/var/cache/apt/archives/"' \ + --setup-hook="$extra_setup" \ + --setup-hook='upload "${APTPREFS}" /etc/apt/preferences.d/bootstrap' \ + --setup-hook='upload "${APTSRCS_INIT}" /etc/apt/sources-list' \ + --setup-hook='upload "${WORKDIR}/locale" /etc/locale' \ + --setup-hook='mkdir -p "$1/etc/apt/trusted.gpg.d"' \ + --setup-hook='sync-in "${WORKDIR}/trusted.gpg.d" /etc/apt/trusted.gpg.d' \ + --setup-hook='install -v -m755 "${WORKDIR}/chroot-setup.sh" "$1/chroot-setup.sh"' \ + --extract-hook="$extra_extract" \ + --essential-hook="$extra_essential" \ + --customize-hook="$extra_customize" \ + --customize-hook='sed -i "/en_US.UTF-8 UTF-8/s/^#//g" "$1/etc/locale.gen"' \ + --customize-hook='chroot "$1" /usr/sbin/locale-gen' \ + --customize-hook='chroot "$1" /usr/bin/apt-get -y clean' \ + --skip=cleanup/apt \ + --skip=download/empty \ ${@get_distro_components_argument(d)} \ "${@get_distro_suite(d)}" \ "${WORKDIR}/rootfs.tar.zst" \ - "${@get_distro_source(d)}" + "$bootstrap_list" sudo -E -s <<'EOSUDO' set -e tar -xf "${WORKDIR}/rootfs.tar.zst" -C "${ROOTFSDIR}" --exclude="./dev/console" - # Install apt config - mkdir -p "${ROOTFSDIR}/etc/apt/preferences.d" - install -v -m644 "${APTPREFS}" \ - "${ROOTFSDIR}/etc/apt/preferences.d/bootstrap" - mkdir -p "${ROOTFSDIR}/etc/apt/sources.list.d" - if [ "${ISAR_USE_CACHED_BASE_REPO}" = "1" ]; then - line="file:///base-apt/${BOOTSTRAP_BASE_DISTRO} ${BASE_DISTRO_CODENAME} main" - if [ -z "${BASE_REPO_KEY}" ]; then - line="[trusted=yes] ${line}" - fi - echo "deb ${line}" > "${ROOTFSDIR}/etc/apt/sources.list.d/base-apt.list" - line="file:///base-apt/${BASE_DISTRO} ${BASE_DISTRO_CODENAME} main" - if [ -z "${BASE_REPO_KEY}" ]; then - line="[trusted=yes] ${line}" - fi - echo "deb-src ${line}" >> "${ROOTFSDIR}/etc/apt/sources.list.d/base-apt.list" - - mkdir -p ${ROOTFSDIR}/base-apt - mount --bind ${REPO_BASE_DIR} ${ROOTFSDIR}/base-apt - else - install -v -m644 "${APTSRCS}" \ - "${ROOTFSDIR}/etc/apt/sources.list.d/bootstrap.list" - fi - install -v -m644 "${APTSRCS_INIT}" "${ROOTFSDIR}/etc/apt/sources-list" - rm -f "${ROOTFSDIR}/etc/apt/sources.list" - rm -rf "${ROOTFSDIR}/var/lib/apt/lists/"* - find ${APT_KEYS_DIR}/ -type f | while read keyfile - do - MY_GPGHOME="$(chroot "${ROOTFSDIR}" mktemp -d /tmp/gpghomeXXXXXXXXXX)" - echo "Created temporary directory ${MY_GPGHOME} for gpg-agent" - export GNUPGHOME="${MY_GPGHOME}" - APT_KEY_APPEND="--homedir ${MY_GPGHOME}" - - kfn="$(basename $keyfile)" - cp $keyfile "${ROOTFSDIR}/tmp/$kfn" - chroot "${ROOTFSDIR}" /usr/bin/gpg-agent --daemon -- /usr/bin/apt-key \ - --keyring ${THIRD_PARTY_APT_KEYRING} ${APT_KEY_APPEND} add "/tmp/$kfn" - rm "${ROOTFSDIR}/tmp/$kfn" - - echo "Removing ${MY_GPGHOME}" - rm -rf "${ROOTFSDIR}${MY_GPGHOME}" - done - - # Set locale - install -v -m644 "${WORKDIR}/locale" "${ROOTFSDIR}/etc/locale" - - sed -i '/en_US.UTF-8 UTF-8/s/^#//g' "${ROOTFSDIR}/etc/locale.gen" - chroot "${ROOTFSDIR}" /usr/sbin/locale-gen - # setup chroot install -v -m755 "${WORKDIR}/chroot-setup.sh" "${ROOTFSDIR}/chroot-setup.sh" "${ROOTFSDIR}/chroot-setup.sh" "setup" "${ROOTFSDIR}" - # update APT - mount -o bind,private /dev ${ROOTFSDIR}/dev - mount --bind /dev/pts ${ROOTFSDIR}/dev/pts - mount -t tmpfs none "${ROOTFSDIR}/dev/shm" - mount -t proc none ${ROOTFSDIR}/proc - mount --rbind /sys ${ROOTFSDIR}/sys - mount --make-rslave ${ROOTFSDIR}/sys - - export DEBIAN_FRONTEND=noninteractive - - if [ "${BOOTSTRAP_FOR_HOST}" = "1" ]; then - chroot "${ROOTFSDIR}" /usr/bin/dpkg --add-architecture ${DISTRO_ARCH} - fi - - if [ "${ISAR_ENABLE_COMPAT_ARCH}" = "1" ]; then - chroot "${ROOTFSDIR}" /usr/bin/dpkg --add-architecture ${COMPAT_DISTRO_ARCH} - fi - - chroot "${ROOTFSDIR}" /usr/bin/apt-get update -y \ - -o APT::Update::Error-Mode=any - chroot "${ROOTFSDIR}" /usr/bin/apt-get install -y -f - chroot "${ROOTFSDIR}" /usr/bin/apt-get dist-upgrade -y \ - -o Debug::pkgProblemResolver=yes - - umount -l "${ROOTFSDIR}/dev/shm" - umount -l "${ROOTFSDIR}/dev/pts" - umount -l "${ROOTFSDIR}/dev" - umount -l "${ROOTFSDIR}/proc" - umount -l "${ROOTFSDIR}/sys" - umount -l "${ROOTFSDIR}/base-apt" || true - # Finalize bootstrap by setting the link in deploy ln -Tfsr "${ROOTFSDIR}" "${DEPLOY_ISAR_BOOTSTRAP}" EOSUDO - deb_dl_dir_export "${ROOTFSDIR}" "${BOOTSTRAP_BASE_DISTRO}-${BASE_DISTRO_CODENAME}" - - # Cleanup apt cache - sudo -Es chroot "${ROOTFSDIR}" /usr/bin/apt-get -y clean + if [ "${ISAR_USE_CACHED_BASE_REPO}" != "1" ]; then + deb_dl_dir_export "${WORKDIR}/dl_dir" "${BOOTSTRAP_BASE_DISTRO}-${BASE_DISTRO_CODENAME}" + sudo rm -rf --one-file-system "${WORKDIR}/dl_dir" + fi } addtask bootstrap before do_build after do_generate_keyrings From patchwork Mon Jul 8 06:17:37 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anton Mikanovich X-Patchwork-Id: 3671 Return-Path: Received: from shymkent.ilbers.de ([unix socket]) by shymkent (Cyrus 2.5.10-Debian-2.5.10-3+deb9u2) with LMTPA; Mon, 08 Jul 2024 08:18:06 +0200 X-Sieve: CMU Sieve 2.4 Received: from mail-lj1-f188.google.com (mail-lj1-f188.google.com [209.85.208.188]) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPS id 4686I5Du030928 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Mon, 8 Jul 2024 08:18:05 +0200 Received: by mail-lj1-f188.google.com with SMTP id 38308e7fff4ca-2ee87546e5esf38428821fa.0 for ; Sun, 07 Jul 2024 23:18:05 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1720419479; cv=pass; d=google.com; s=arc-20160816; b=LH1vso9R/94bHf18dWNkWkNrGg8cB1xQWtKx/3c1nGIbUmmOph02Iw30furHXv2d7q vebngFBVJQHN1x9d7QPz6U88jhcACDCjaCF2jnr6nvTV8zilRrgMpElUbvj206PHJ01F NRolu0u8gwdS6xrL+7/LhrL70oVBEWJFDwDrCrSxnwiJiUYh59f8EelIscDs/2yjJFEq P/6/kctQTaQdD0lZ2IHM2SWa5frDov1QmDcDJt+6hxo4c4j+g58Pizv1V6GtV3ZG+qru fk9+SjbJ48JHb/1StRL9IREqDUBltth0Oq5Xu55N9XU7r8THCWmNEfHgi8vImNfTevre nGHQ== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:dkim-signature; bh=6KeDG8EnGr4dmaEUO0kinkyPEGDCXbH9Nk1rp98cbZA=; fh=raoxeF1K3M0mLn5ZagIrqNNYn5ahm7Ggn4Hb1O4Z9tk=; b=waSTaDZvwJUVDizlYEZfGXNwxsFFSZ9F7X5J+a6CKWRHKq/WnFaj7ZYATzd1Lf6Lz1 nxTjI3dwa8Hvj8JNVuPXLnRSZoSrD3WuzWSfAEPF5XC87QUALxGtUrjqYkP1IuzI7R86 oa04Duvio05VJE1Nbe1gK02WVqcNKRps95CjkwHBkstG7GMO3CzzRn5kXj7aqQi6J/Ro dkvnYwgRyZ1elVj0BHIG7dnxbFVQY/zoGi62sa7BDyoDTklhcBwm3pEW7NLSHDMODgtk PUk4Z827zbWc85NNg4cEtS9vhrojkyT4o8SyUGF2D2z6J/mXrvt/+pHDraV6EA32YHfs dF9g==; darn=isar-build.org ARC-Authentication-Results: i=2; gmr-mx.google.com; spf=pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=amikan@ilbers.de DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlegroups.com; s=20230601; t=1720419479; x=1721024279; darn=isar-build.org; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:x-original-authentication-results :x-original-sender:mime-version:references:in-reply-to:message-id :date:subject:cc:to:from:sender:from:to:cc:subject:date:message-id :reply-to; bh=6KeDG8EnGr4dmaEUO0kinkyPEGDCXbH9Nk1rp98cbZA=; b=smFWOVy65+u5bhrwqASOUvFrzbwqaTQUWz0az+TaBzLMzMmevnhjDmW+xdtsBmTX7q 859fN/FSxSqxM2hLwwRHaGE/ONmOwVgMqXZza/jZmRDYPCzZUJxBspng3Pm3VbF5GkST 8z59oWHrA1dWSCUZ6d1NooYj1h8IMWoelLVLIcq/YAKwvwpfrnhZvweEWKM6y6jpW+2G zqJF01i3P7qARQyh9guB+G+3ixCyy5j7ytVmRLdPB4O4a5dEsgZKpGcDj0cSv6225Bn5 6JEKyfQGHEHHJD61ODrz8NQ/XuuCNhotl/FSK2W39D4ESCV3Prg++hIvnxg/QRDGiM8W y1rg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1720419479; x=1721024279; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :x-spam-checked-in-group:list-id:mailing-list:precedence :x-original-authentication-results:x-original-sender:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :x-beenthere:x-gm-message-state:sender:from:to:cc:subject:date :message-id:reply-to; bh=6KeDG8EnGr4dmaEUO0kinkyPEGDCXbH9Nk1rp98cbZA=; b=pqRCSWRztnZarOvbB+ZPsKWB5C1QjB7nPGgQKD/Dah96MyuDtFxukYYjrR++l+eNI/ kfQsBSZ7ELbRNiP0Y6VtjpUfTeRqYCmSdn4Tzbfy1IXdemEGFxo1mXMzHtFWMdFMU4uC r0Ndhi2wP4mOIP4XcmuwKp/IQuMwAu9S2iHV+ZQzzxC/19atzEtoHCmZCttDEojL9ord /rQ7FtvskT8glxo8FdrMkcCt3ZLBh6YVhGOHobkFIzWZoJe/y6amP4aSy5/Vvok7fS7k jDnXaKljjFgwOkDrjBB/wWQWS9xb1QggFf1CaKyIK1TtQEWxchIdUKrcFtSoO0eyEfyJ 7Aew== Sender: isar-users@googlegroups.com X-Forwarded-Encrypted: i=2; AJvYcCVNGg8JqkQdjnKboz0rytiO88aAta6DXeOR67q+teN78kbiLBTC70kR7ZmLMoQUXA8a+dl5mK7cgs9jn60/GejYFnoohD8= X-Gm-Message-State: AOJu0Yz4YwfsTwerMYoC69DO/0cD6DMcTuzXOpKx63Ge4ScjHlT2l6p1 pUpSKiSeijER3gevA/cnb3t7gtxIZL4epf37EoU3ewAVC0T63CCj X-Google-Smtp-Source: AGHT+IH1cSi6pnyB0DQca0odZnarkpLruuw2hkFU6xt7XxZCB4Snxr1NyH7XzhG1e3rMOQhmVf/3xg== X-Received: by 2002:a2e:a58a:0:b0:2ee:7d37:498b with SMTP id 38308e7fff4ca-2ee8ed8cf11mr92339071fa.14.1720419478929; Sun, 07 Jul 2024 23:17:58 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a2e:aa26:0:b0:2ee:89a7:12ac with SMTP id 38308e7fff4ca-2ee8cd96adfls18073181fa.1.-pod-prod-06-eu; Sun, 07 Jul 2024 23:17:57 -0700 (PDT) X-Received: by 2002:a05:651c:601:b0:2ee:4f22:33f9 with SMTP id 38308e7fff4ca-2ee8eda42f0mr63774631fa.24.1720419475918; Sun, 07 Jul 2024 23:17:55 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1720419475; cv=none; d=google.com; s=arc-20160816; b=ix4HviUdwA1EEZBEKxNsNpDOVRLzCbKrvcOTu6v0/xTlSIigFpi40t9EbatepSw6tA H6FvyLdaP3zpPRoruJyw/0hfqs8Dr6gDVuwv0icahguesEUF646caU4+DQrLuuaPp8ID 8JjeVHEXxlzAfvoshj+I+IXw3IRVGPW54OVZV9k0RD9PDSMUaAxKHM1O/fYZ8hj1eUhH GDjdnqMrfmzfXiYNsRtfUc7S+PBJp9RIQ/X7fGpyNjddD0PGmkRZheLtUgZpMUx1QOuv X3UL7Vc+z9qSgHhhKvykd4zKFiBSTKwrR7AyIKJKCMBh66JDfjhIlJHBvvH3L7Jv3dWT F8bw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=5nPiJF+WzumJAVVCCb60FHItiJ2GgvRB52oI2WIygYs=; fh=/h9QQkzJ8EboVkWg45aWwpaUro6WMavIVd2OhN45RtE=; b=lOyxHloPrbAZEsLO9mcuwW/9SRoTRWS2SOoo7xdMHG2POubl8oCclgeZAbZLsv2dy2 HuNly0y2tRRSi1DwEHnLpHrur2m+fY1pB0oaZee1bo0TMgYJ3ErcrLV03IsarsZ7rgV1 LP5D2SEnDldsZ4TfT4monbYOizoYDFVqxizMZeSgvpCz2deLtQMUB62+/T5f+dIp6P7h V/yKNXn/W65DHCSgRrbfx8kemcGB2slqJRc3GhPqGoKRL6QaV/OKs3kEpPwHETl69/Ia hjMXVRTHavwL4ubUewuauFA0hnckG14foXTBIQjt2A5DclJ93XSeRG7E/Y3vWLZdr7yg I0NQ==; dara=google.com ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=amikan@ilbers.de Received: from shymkent.ilbers.de (shymkent.ilbers.de. [85.214.156.166]) by gmr-mx.google.com with ESMTPS id 38308e7fff4ca-2ee941090e9si1239231fa.6.2024.07.07.23.17.55 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Sun, 07 Jul 2024 23:17:55 -0700 (PDT) Received-SPF: pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) client-ip=85.214.156.166; Received: from user-B660.promwad.corp ([159.148.83.114]) (authenticated bits=0) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPSA id 4686HmUx030688 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 8 Jul 2024 08:17:54 +0200 From: Anton Mikanovich To: isar-users@googlegroups.com Cc: Anton Mikanovich Subject: [PATCH v10 7/8] isar-bootstrap: Use tar output instead of directory Date: Mon, 8 Jul 2024 09:17:37 +0300 Message-Id: <20240708061738.311938-8-amikan@ilbers.de> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240708061738.311938-1-amikan@ilbers.de> References: <20240708061738.311938-1-amikan@ilbers.de> MIME-Version: 1.0 X-Spam-Status: No, score=-4.6 required=5.0 tests=DKIMWL_WL_MED,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_EF,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,RCVD_IN_MSPIKE_H2,RCVD_IN_RP_CERTIFIED, RCVD_IN_RP_RNBL,RCVD_IN_RP_SAFE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on shymkent.ilbers.de X-Original-Sender: amikan@ilbers.de X-Original-Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=amikan@ilbers.de Precedence: list Mailing-list: list isar-users@googlegroups.com; contact isar-users+owners@googlegroups.com List-ID: X-Spam-Checked-In-Group: isar-users@googlegroups.com X-Google-Group-Id: 914930254986 List-Post: , List-Help: , List-Archive: , List-Unsubscribe: , X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= Do not unpack tar with rootfs content inside bootstrap recipe. This change can allow to remove sudo usage during the bootstrap later. Using tar output requires moving chroot prepare step to the later unpack step. Signed-off-by: Anton Mikanovich --- meta/classes/rootfs.bbclass | 11 +++++---- .../isar-bootstrap/isar-bootstrap.inc | 23 ++++++++++--------- .../isar-mmdebstrap/isar-mmdebstrap.inc | 23 ++++--------------- 3 files changed, 24 insertions(+), 33 deletions(-) diff --git a/meta/classes/rootfs.bbclass b/meta/classes/rootfs.bbclass index aa3f6baa..0a1e64e5 100644 --- a/meta/classes/rootfs.bbclass +++ b/meta/classes/rootfs.bbclass @@ -73,12 +73,15 @@ rootfs_do_qemu() { fi } -BOOTSTRAP_SRC = "${DEPLOY_DIR_BOOTSTRAP}/${ROOTFS_DISTRO}-host_${DISTRO}-${DISTRO_ARCH}" -BOOTSTRAP_SRC:${ROOTFS_ARCH} = "${DEPLOY_DIR_BOOTSTRAP}/${ROOTFS_DISTRO}-${ROOTFS_ARCH}" +BOOTSTRAP_SRC = "${DEPLOY_DIR_BOOTSTRAP}/${ROOTFS_DISTRO}-host_${DISTRO}-${DISTRO_ARCH}.tar.zst" +BOOTSTRAP_SRC:${ROOTFS_ARCH} = "${DEPLOY_DIR_BOOTSTRAP}/${ROOTFS_DISTRO}-${ROOTFS_ARCH}.tar.zst" rootfs_prepare[weight] = "25" rootfs_prepare(){ - sudo cp -Trpfx --reflink=auto '${BOOTSTRAP_SRC}/' '${ROOTFSDIR}' + sudo tar -xf "${BOOTSTRAP_SRC}" -C "${ROOTFSDIR}" --exclude="./dev/console" + + # setup chroot + sudo "${ROOTFSDIR}/chroot-setup.sh" "setup" "${ROOTFSDIR}" } ROOTFS_CONFIGURE_COMMAND += "rootfs_configure_isar_apt" @@ -227,7 +230,7 @@ cache_deb_src() { # Note: ISAR updates the apt state information(apt-get update) only once during bootstrap and # relies on that through out the build. Copy that state information instead of apt-get update # which generates a new state from upstream. - sudo cp -Trpn --reflink=auto "${BOOTSTRAP_SRC}/var/lib/apt/lists/" "${ROOTFSDIR}/var/lib/apt/lists/" + sudo tar -xf "${BOOTSTRAP_SRC}" ./var/lib/apt/lists --one-top-level="${ROOTFSDIR}" deb_dl_dir_import ${ROOTFSDIR} ${ROOTFS_BASE_DISTRO}-${BASE_DISTRO_CODENAME} debsrc_download ${ROOTFSDIR} ${ROOTFS_BASE_DISTRO}-${BASE_DISTRO_CODENAME} diff --git a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc index 0855875b..4e0ffc0d 100644 --- a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc +++ b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc @@ -162,7 +162,6 @@ do_bootstrap() { # setup chroot install -v -m755 "${WORKDIR}/chroot-setup.sh" "${ROOTFSDIR}/chroot-setup.sh" - "${ROOTFSDIR}/chroot-setup.sh" "setup" "${ROOTFSDIR}" # update APT mount -o bind,private /dev ${ROOTFSDIR}/dev @@ -196,8 +195,12 @@ do_bootstrap() { mountpoint -q "${ROOTFSDIR}/base-apt" && \ umount "${ROOTFSDIR}/base-apt" + # Compress rootfs for compatibility + lopts="--one-file-system --exclude=var/cache/apt/archives" + tar --zstd -cf "${WORKDIR}/rootfs.tar.zst" $lopts -C "${ROOTFSDIR}" . + # Finalize debootstrap by setting the link in deploy - ln -Tfsr "${ROOTFSDIR}" "${DEPLOY_ISAR_BOOTSTRAP}" + ln -Tfsr "${WORKDIR}/rootfs.tar.zst" "${DEPLOY_ISAR_BOOTSTRAP}.tar.zst" EOSUDO deb_dl_dir_export "${ROOTFSDIR}" "${BOOTSTRAP_BASE_DISTRO}-${BASE_DISTRO_CODENAME}" @@ -213,19 +216,17 @@ SSTATEPOSTINSTFUNCS += "bootstrap_sstate_finalize" bootstrap_sstate_prepare() { # this runs in SSTATE_BUILDDIR, which will be deleted automatically - lopts="--one-file-system --exclude=var/cache/apt/archives" - sudo tar -C $(dirname "${ROOTFSDIR}") -cpSf bootstrap.tar $lopts $(basename "${ROOTFSDIR}") - sudo chown $(id -u):$(id -g) bootstrap.tar + sudo cp -a "${WORKDIR}/rootfs.tar.zst" ./bootstrap.tar.zst + sudo chown $(id -u):$(id -g) bootstrap.tar.zst } bootstrap_sstate_finalize() { # this runs in SSTATE_INSTDIR - # - after building the bootstrap, the tar won't be there, but we also don't need to unpack - # - after restoring from cache, there will be a tar which we unpack and then delete - if [ -f bootstrap.tar ]; then - sudo tar -C $(dirname "${ROOTFSDIR}") -xpf bootstrap.tar - sudo ln -Tfsr "${ROOTFSDIR}" "${DEPLOY_ISAR_BOOTSTRAP}" - rm bootstrap.tar + # we should restore symlinks after using tar + if [ -f bootstrap.tar.zst ]; then + mv bootstrap.tar.zst "${WORKDIR}/rootfs.tar.zst" + sudo ln -Tfsr "${WORKDIR}/rootfs.tar.zst" \ + "${DEPLOY_ISAR_BOOTSTRAP}.tar.zst" fi } diff --git a/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc b/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc index c41dd3a1..f1ad891b 100644 --- a/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc +++ b/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc @@ -11,7 +11,6 @@ inherit deb-dl-dir FILESEXTRAPATHS:append = ":${LAYERDIR_core}/recipes-core/isar-bootstrap/files" -ROOTFSDIR = "${WORKDIR}/rootfs" DISTRO_BOOTSTRAP_BASE_PACKAGES = "locales,apt,usrmerge" BOOTSTRAP_TMPDIR = "${WORKDIR}/tempdir" @@ -73,9 +72,6 @@ do_bootstrap() { fi E="${@ isar_export_proxies(d)}" - sudo rm -rf --one-file-system "${ROOTFSDIR}" - mkdir -p "${ROOTFSDIR}" - if [ "${ISAR_USE_CACHED_BASE_REPO}" = "1" ]; then base_apt_tmp="$(mktemp -d /tmp/isar-base-aptXXXXXXXXXX)" bootstrap_list="${WORKDIR}/sources.list.d/base-apt.list" @@ -181,18 +177,9 @@ do_bootstrap() { "${WORKDIR}/rootfs.tar.zst" \ "$bootstrap_list" - sudo -E -s <<'EOSUDO' - set -e - - tar -xf "${WORKDIR}/rootfs.tar.zst" -C "${ROOTFSDIR}" --exclude="./dev/console" - - # setup chroot - install -v -m755 "${WORKDIR}/chroot-setup.sh" "${ROOTFSDIR}/chroot-setup.sh" - "${ROOTFSDIR}/chroot-setup.sh" "setup" "${ROOTFSDIR}" + # Finalize bootstrap by setting the link in deploy + sudo ln -Tfsr "${WORKDIR}/rootfs.tar.zst" "${DEPLOY_ISAR_BOOTSTRAP}.tar.zst" - # Finalize bootstrap by setting the link in deploy - ln -Tfsr "${ROOTFSDIR}" "${DEPLOY_ISAR_BOOTSTRAP}" -EOSUDO if [ "${ISAR_USE_CACHED_BASE_REPO}" != "1" ]; then deb_dl_dir_export "${WORKDIR}/dl_dir" "${BOOTSTRAP_BASE_DISTRO}-${BASE_DISTRO_CODENAME}" sudo rm -rf --one-file-system "${WORKDIR}/dl_dir" @@ -206,7 +193,7 @@ SSTATEPOSTINSTFUNCS += "bootstrap_sstate_finalize" bootstrap_sstate_prepare() { # this runs in SSTATE_BUILDDIR, which will be deleted automatically - sudo cp -a "$(dirname "${ROOTFSDIR}")/rootfs.tar.zst" ./bootstrap.tar.zst + sudo cp -a "${WORKDIR}/rootfs.tar.zst" ./bootstrap.tar.zst sudo chown $(id -u):$(id -g) bootstrap.tar.zst } @@ -214,8 +201,8 @@ bootstrap_sstate_finalize() { # this runs in SSTATE_INSTDIR # we should restore symlinks after using tar if [ -f bootstrap.tar.zst ]; then - mv bootstrap.tar.zst "$(dirname "${ROOTFSDIR}")/rootfs.tar.zst" - sudo ln -Tfsr "$(dirname "${ROOTFSDIR}")/rootfs.tar.zst" \ + mv bootstrap.tar.zst "${WORKDIR}/rootfs.tar.zst" + sudo ln -Tfsr "${WORKDIR}/rootfs.tar.zst" \ "${DEPLOY_ISAR_BOOTSTRAP}.tar.zst" fi } From patchwork Mon Jul 8 06:17:38 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anton Mikanovich X-Patchwork-Id: 3670 Return-Path: Received: from shymkent.ilbers.de ([unix socket]) by shymkent (Cyrus 2.5.10-Debian-2.5.10-3+deb9u2) with LMTPA; Mon, 08 Jul 2024 08:18:04 +0200 X-Sieve: CMU Sieve 2.4 Received: from mail-wm1-f59.google.com (mail-wm1-f59.google.com [209.85.128.59]) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPS id 4686I3Hq030835 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Mon, 8 Jul 2024 08:18:03 +0200 Received: by mail-wm1-f59.google.com with SMTP id 5b1f17b1804b1-4265b751469sf9343845e9.1 for ; Sun, 07 Jul 2024 23:18:03 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1720419478; cv=pass; d=google.com; s=arc-20160816; b=bnYd3O5m47Az8Xkwr86nHoVU5+XPPQlzTTrAoOnZrXmdz0NgGv/8/5Qkt7S+G4Zn7Z mWyjguleVo8IzXYHK7HBz4kZVDw7jJ7y6l5Yjnqn4plObVPfV02o744Ge9PXo3dqQFGx T9nSrL22ujXZeOxhFUULCon428PzzO/lG7lzR1yDc/VcCJRyUzg2Sv/fhAjmF6UxXGjY qwTjRjDw2NhzTgFenEiMvR1/UPHIKGjtubNg0fxRdMQle68YRuRKHPc8VZV7foOFomY9 RGqPxJ7bvvzKO3M7osk+5NS9ZNWJztDC5p6nUMzojcXz21JabviUwa46lxWe7nEmMHgQ ZxRA== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:dkim-signature; bh=Nn+Yv1DSoZM/Q4z65nG43aWLTz6JCC0hBcY1DUuhNqk=; fh=vEt2qBpkNuQf/JQJU1OaIeIxlX1LE01+FC7ApSKut/0=; b=xoXhq7CxB4QrlqS5hZRjFMsNEGy4xvQE2dP9L6uaksje8kFuEWEOaMuk8vWvBDArBp PBGKA8V19fifnMGOXK9/U9zr8e/1+SiXFZ4o6ttEJ7v60IIpKcZPTrW15B46Crfl2ac6 rd45YRDjgb5p4XTlp6N6qg/E8dqTRwMqY2lJ7b5l47YA4exfeDsZuuNNuoSrBScFcacY GXT2ghNpdQGqJUmWgLvDfJ54pWMRhn7pIG4wWfjBOiQz/otXWeQabwe6E1nX16l8hJZ/ cssaabUeaBRWs/Lxy5hsSldPi+qaydXujUD2uW1Kap4lDYzF3SAAoGjA9hPrPrwewhSc jpsg==; darn=isar-build.org ARC-Authentication-Results: i=2; gmr-mx.google.com; spf=pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=amikan@ilbers.de DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlegroups.com; s=20230601; t=1720419478; x=1721024278; darn=isar-build.org; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:x-original-authentication-results :x-original-sender:mime-version:references:in-reply-to:message-id :date:subject:cc:to:from:sender:from:to:cc:subject:date:message-id :reply-to; bh=Nn+Yv1DSoZM/Q4z65nG43aWLTz6JCC0hBcY1DUuhNqk=; b=mtCGYfFYGOxJhYpveEbY4mJvRAhxV5cmiOEBRD4VPxWYtx8jjtYSE0nXkK7wgfgcb1 iu4/Mc97wx6r5qUqMfXgbvmHRQe/488BXRJtGUQ5Rzmmu3fIqZK64IVtVRVreNZ8ON7A DFq4f2k/l0/8DoIHkjac4DYaJ6zTigMqTHVjt0t7OLLbHbRZtLrFrIFcnBitBZTz7Vy9 udm71/IUpKeipn5xS0h5bA8oUTq1cX9DFCHT8ZHx8SCTc6ebKRajV2Bwk5rbWzdpMczS iJ7Y1Ei1DpW1eF3ahYYrW6loVre40mf+PXRSWFhqu8CWgsPJYMpxXAH6mGSjn8Zokj4v 2fZA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1720419478; x=1721024278; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :x-spam-checked-in-group:list-id:mailing-list:precedence :x-original-authentication-results:x-original-sender:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :x-beenthere:x-gm-message-state:sender:from:to:cc:subject:date :message-id:reply-to; bh=Nn+Yv1DSoZM/Q4z65nG43aWLTz6JCC0hBcY1DUuhNqk=; b=vN71Oa245eH8mdMlSlChniQwnMuwLwFndxM2L+VNr1I8lgT1oAGjmdMdQ4lPNhAQY+ H1CVhj3v0CiZlUlivcku8V+kCXKip0uGQyBAGGxnX80G5GEqCoXrK+K7QS9TXjaL/gxc bt1Wl+UTBW/EGEzPgIM3jhB/Jhlf1bry0RV7L4+GD4MiUCYlSrsS9k+0RtJYP21inUvl Ywv3vDHyDqwC08RktrdQVDYyYMtXVfzghnvFSrQ+qonj05C2UsUXDIMXVTcdc7z1BDS7 3aFkf8IuOaXP0zjBeLaAgSYWQztpxk9YgT7n0Tb+Kl71cINR10kkQ+agJk9hcmaZzURU 2I4Q== Sender: isar-users@googlegroups.com X-Forwarded-Encrypted: i=2; AJvYcCUK6hOuvIe9zXmgfPkEI3GYwMoKyqdvYMv5jvuobnq9UtgoJvYSisF3yNaGTe8o6bwespVhQ12i+rYMuO0eBB//WZEmaew= X-Gm-Message-State: AOJu0Yxsfrv7C0HMBGLYXCMAQ/gKsc9vzkZs4PCXQVDDIwYubcFYRmLO +DO6XPgueLIRLV/Y1o3V7inFfZBGmGNEoIe9ZF6SIRFSZrLIB3Fb X-Google-Smtp-Source: AGHT+IEX8/dWJezSs/Ka7QTEcZSZm8WVgnzjUFAI87pD6sz/pER2hXQ7C1U2gveyDzZ6PVtFLT6ARw== X-Received: by 2002:a05:600c:2d16:b0:426:5b52:84fb with SMTP id 5b1f17b1804b1-4265b528599mr55693085e9.7.1720419478190; Sun, 07 Jul 2024 23:17:58 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a05:600c:1c24:b0:426:67f1:ab2d with SMTP id 5b1f17b1804b1-42667f1bd30ls2709355e9.0.-pod-prod-00-eu-canary; Sun, 07 Jul 2024 23:17:56 -0700 (PDT) X-Received: by 2002:a05:600c:3510:b0:424:aa83:ef27 with SMTP id 5b1f17b1804b1-4264b0c5393mr98305225e9.1.1720419476450; Sun, 07 Jul 2024 23:17:56 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1720419476; cv=none; d=google.com; s=arc-20160816; b=ivUEFs+kukrsBtJ+4joZ1Luf/8fafPDfRBMYXdUubeiq0Sni9VDx0uVton2eLGoaVa w4NlIPJvfYqJHKRzGtHDkWXnO0TK+DabWDLJqXM3cQ8yKJ59N2nYKLGPbukl1CbhMsmN hHFdubLBfDmVMSku32MnrJKVqurvK7qbWbVhRnepcocN4kGWiXaGVgoSwIechTvzE9B7 aIfUlWOWGGONvQy5XrjZ/6GxFLFDHzrcM544C1SsAizuayJP+8LhxnBe9rzVTx43Ja2n jGhLmix9SZgvdfOZoaVKXrfNKOY9A2Y4ZnRf5aMOpWF6rz4AgjNMAo9yQf0Ra4fpUCkS hPwQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=BeEF2Nxzmm9+xvUdvNprhcPi5wLiRAU0ezaVIGWeyhA=; fh=/h9QQkzJ8EboVkWg45aWwpaUro6WMavIVd2OhN45RtE=; b=QU4YX1Q6EB8dBMKljtS9S0XemCB+N3JlbD3AkjugM0t1a4n+CM26Z3qPcuPjjUNv0y jy4QKdd/eUVukDQB+2/e7ksg/TOaWFbPLO4y8gktCH0jaR056DmFHjB2mMGiW8MRsfjs kVvBraqpB8oDNqNsCmIaZCjKUuH+1RZplKCKpzTDZr7dW3Y4ZsfhEVeRWCZAxmbc2ekQ 79oD9IiEqvm+6W+YWS8IuTxuOuDiYHIr6zZ/uHz7RC1qRCcbhVN0KoOWnC1J3M6adVKu 2WjIYcV+7Thved9RpgC6QsIdS5ylIb43ZnVvcBZlRrXZXVFql4Zr5QS6EoUP2II8leJh +POw==; dara=google.com ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=amikan@ilbers.de Received: from shymkent.ilbers.de (shymkent.ilbers.de. [85.214.156.166]) by gmr-mx.google.com with ESMTPS id 5b1f17b1804b1-42661583512si2510275e9.0.2024.07.07.23.17.56 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Sun, 07 Jul 2024 23:17:56 -0700 (PDT) Received-SPF: pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) client-ip=85.214.156.166; Received: from user-B660.promwad.corp ([159.148.83.114]) (authenticated bits=0) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPSA id 4686HmV0030688 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 8 Jul 2024 08:17:55 +0200 From: Anton Mikanovich To: isar-users@googlegroups.com Cc: Anton Mikanovich Subject: [PATCH v10 8/8] user_manual.md: Update boostrap related documentation Date: Mon, 8 Jul 2024 09:17:38 +0300 Message-Id: <20240708061738.311938-9-amikan@ilbers.de> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240708061738.311938-1-amikan@ilbers.de> References: <20240708061738.311938-1-amikan@ilbers.de> MIME-Version: 1.0 X-Spam-Status: No, score=-4.6 required=5.0 tests=DKIMWL_WL_MED,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_EF,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,RCVD_IN_MSPIKE_H2,RCVD_IN_RP_CERTIFIED, RCVD_IN_RP_RNBL,RCVD_IN_RP_SAFE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on shymkent.ilbers.de X-Original-Sender: amikan@ilbers.de X-Original-Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=amikan@ilbers.de Precedence: list Mailing-list: list isar-users@googlegroups.com; contact isar-users+owners@googlegroups.com List-ID: X-Spam-Checked-In-Group: isar-users@googlegroups.com X-Google-Group-Id: 914930254986 List-Post: , List-Help: , List-Archive: , List-Unsubscribe: , X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= Update documentation in user manual and recipes API changelog. Signed-off-by: Anton Mikanovich --- RECIPE-API-CHANGELOG.md | 13 +++++++++++++ doc/user_manual.md | 15 ++++++++++++--- 2 files changed, 25 insertions(+), 3 deletions(-) diff --git a/RECIPE-API-CHANGELOG.md b/RECIPE-API-CHANGELOG.md index 12ea93ec..589dae66 100644 --- a/RECIPE-API-CHANGELOG.md +++ b/RECIPE-API-CHANGELOG.md @@ -629,3 +629,16 @@ into kernel kbuild package. Only the "host" specific package is built automatically at cross builds. * Support emulated module build with cross-compiled kernel for linux-module + +### Default boostrap recipe changed to mmdebstrap + +New virtual packages bootstrap-host and bootstrap-target are introduced. +There are two providers of bootstrap-host/-target currently: + * isar-mmdebstrap: deafult one using mmdebstrap to prepare rootfs + * isar-bootstrap: previous bootstrap implementation left for compatibility + +So default bootstrap procedure is now performing with mmdebstrap. +Previous implementation still can be selected by setting in local.conf: + +PREFERRED_PROVIDER_bootstrap-host ?= "isar-bootstrap-host" +PREFERRED_PROVIDER_bootstrap-target ?= "isar-bootstrap-target" diff --git a/doc/user_manual.md b/doc/user_manual.md index 776ae52c..552b1d8c 100644 --- a/doc/user_manual.md +++ b/doc/user_manual.md @@ -66,7 +66,8 @@ The steps below describe how to build the images provided by default. ### Install Host Tools -The supported host system is >= buster. +The supported host system is >= buster for debootstrap and >= bullseye for +default mmdebstrap provider. Building `debian-trixie` requires host system >= bookworm. @@ -75,7 +76,9 @@ Install the following packages: apt install \ binfmt-support \ bzip2 \ - debootstrap \ + mmdebstrap \ + arch-test \ + apt-utils \ dpkg-dev \ gettext-base \ git \ @@ -93,6 +96,12 @@ apt install \ zstd ``` +If using isar-bootstrap provider, debootstrap should be installed instead of +mmdebstrap: +``` +apt install debootstrap +``` + If your host is >= buster, also install the following package. ``` apt install python3-distutils @@ -135,7 +144,7 @@ apt install qemu ### Setup Sudo -Isar requires `sudo` rights without password to work with `chroot` and `debootstrap`. To add them, use the following steps: +Isar requires `sudo` rights without password to work with `chroot`. To add them, use the following steps: ``` # visudo ```