From patchwork Fri Jun 14 11:34:18 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anton Mikanovich X-Patchwork-Id: 3606 Return-Path: Received: from shymkent.ilbers.de ([unix socket]) by shymkent (Cyrus 2.5.10-Debian-2.5.10-3+deb9u2) with LMTPA; Fri, 14 Jun 2024 13:34:47 +0200 X-Sieve: CMU Sieve 2.4 Received: from mail-wm1-f56.google.com (mail-wm1-f56.google.com [209.85.128.56]) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPS id 45EBYjdO032088 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Fri, 14 Jun 2024 13:34:45 +0200 Received: by mail-wm1-f56.google.com with SMTP id 5b1f17b1804b1-42180d2a0d6sf3244395e9.1 for ; Fri, 14 Jun 2024 04:34:45 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1718364880; cv=pass; d=google.com; s=arc-20160816; b=RgtYclReBVTnflvPguAJ+JUaoQGFJv1lG/aZlISHq+fxAJyD97sIGHUT00AP78V1qW qOsH5aYEoX1+HqyISZEcix7yfkWCZm0SPVLtVg6uaVhyqhrUEZXnAN5kEYEio39mPtk8 TouCkqRYxwRCF43KuUUFpsewdNuhCPJKiyVuoglDk+5qMITUmmO2lwmwgVeI04xL1F/l TVSOp9OkjSdHho+ThZusijeAvHtC01f5V2Wd8k7Dj1dxZlp+RBa0LR10ZncvfxWeh72V lhLFLxQjdGHYluJYkkLYaik1tIYnTUUzPKHPxITBsivIVXecoZeAG1R8AZ4n9ejmxWoW 2IQQ== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:dkim-signature; bh=cG+GDJ8kiICOrYECAkGZxrE8cZvAd3Qn5iD2wrjO/Dk=; fh=Ukc4hVcxY/3Vx7/ZE2P3H8PwmW8Lrg9SjqOYXL0jFm8=; b=JEDrj2O5YdmBQBLrc34pKc5Ej/qK4Zj1kKPGd0/1Ts7BjyN6X3WYrooft1DIemnnc6 vtPBlVCX/QGBdgwp1Cj/GUTHiYj3aKeXjAouiGpddritdGm7uggLUUuEEsKYS3S7qw7u Q0aAq3pX0io2T5rmFIZmhVOvOphfSOE4BrEgEOJk8NRNxCEqE/FuRul7kagNzHNu5dWV xc9amAZLJ7Q0SZuL4ist41ndKttvNjBVBo81Dm/N33AuEuHXfFynV33k+kryXTY4bCWm HRGF15UDZTP087G6n5VUioRK/2FdT3jgISIKyfkEykKMd5lo+X9jIkyTtZ4DA6U+3W0T 4N4A==; darn=isar-build.org ARC-Authentication-Results: i=2; gmr-mx.google.com; spf=pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=amikan@ilbers.de DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlegroups.com; s=20230601; t=1718364880; x=1718969680; darn=isar-build.org; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:x-original-authentication-results :x-original-sender:mime-version:references:in-reply-to:message-id :date:subject:cc:to:from:sender:from:to:cc:subject:date:message-id :reply-to; bh=cG+GDJ8kiICOrYECAkGZxrE8cZvAd3Qn5iD2wrjO/Dk=; b=r3jOWd2ITj8bsTYkMvhc7xoCta9t7+tzoPEzIkPq/xImrpYIQ914tfRbq4HDeILLlg Kf0ZKjxUQbsTwxMTuAsVDbn31aWUjovUrEqJG1NrppEDaawIPuarZuN88ZC/ycD2FJhT +Kb7ba9X7trL2lmHvjY6QRVUr+5Tobx3u6wjVOBE48+ILo/qBreXaW+/F8SBbBh+EKdC 91sXbHENhvktnwQAWfIfe+MkG51o0df9UMlqebKh6+/lzUW2SKKK4LXhvkMrzg/nuswB 8bjEcuBLBQS8BAdioUkv06YOX7jwWrDP1B9r6Fs7yJLWyVWh1bo2kkzvuYFQqG+Nj0Pr HUQg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1718364880; x=1718969680; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :x-spam-checked-in-group:list-id:mailing-list:precedence :x-original-authentication-results:x-original-sender:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :x-beenthere:x-gm-message-state:sender:from:to:cc:subject:date :message-id:reply-to; bh=cG+GDJ8kiICOrYECAkGZxrE8cZvAd3Qn5iD2wrjO/Dk=; b=GYGKI/y3dhl3sC0u/HDCjUtm5MxXo124JtvDfj9bFjhPP5qXHVedu9zEQgWya/teYQ bvy8hkAhVropDSBT+vwT2fZiLNG5VhxciIU8NxCGQg1kWGrbGyDazquwulB5q4TgezxC 2HL/GjpOKZX35oFXXN4DG01m+IDRgT6llJ3Hcg1z27FD0LNknveP/CmrRrTkSyUopdT8 lEWng6uMwXyvRI0UgBxsI3q/15BY1h5ytmXeXh/MwQmjErqkgx7eIYchGBRQye2OWsCy p7eTpU+gB1Qfj/3Qino3jA/A3+kHB3EvrgmVwrtAerx6JiuhIIcKnxyOrFhZ1VnBwuTu SfRQ== Sender: isar-users@googlegroups.com X-Forwarded-Encrypted: i=2; AJvYcCWqDueP5G6ksye8Qdn13q+8OOtGhgEGTdnItTVII3B3czxMO8PSd1/lRrWJlIbiOI0hMY5xrahLTef82nJQSq9pRSyd2Gc= X-Gm-Message-State: AOJu0Yz9zWmgU0PK9mNNU7sp6IJY+RJGmAuOYqrtUhydZ1aakpdEAHxJ wkkPOb3tEjdXWjgaKTwo4++eRy4ysuJeAdUloYb9vqvMTT4aJMPd X-Google-Smtp-Source: AGHT+IF1xwHVUgvPJvTZnp0N4gEonuPuWAMGQK+D/+oX35uJhzf/qHs/VxOwwLoVRknYirjX4oJ4tw== X-Received: by 2002:a5d:6da4:0:b0:35f:2287:7b97 with SMTP id ffacd0b85a97d-3607a7804efmr1835116f8f.4.1718364879529; Fri, 14 Jun 2024 04:34:39 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a5d:5704:0:b0:357:d332:f77 with SMTP id ffacd0b85a97d-3607194f3ddls898197f8f.1.-pod-prod-05-eu; Fri, 14 Jun 2024 04:34:37 -0700 (PDT) X-Received: by 2002:adf:f54d:0:b0:35f:c82:e7b5 with SMTP id ffacd0b85a97d-3607a768bbemr2264638f8f.42.1718364877531; Fri, 14 Jun 2024 04:34:37 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1718364877; cv=none; d=google.com; s=arc-20160816; b=zB6sG/+NQZyxZ5IIEqQ3SizJkFcxBZ+AR6eYmgnR5e+QVsn3nwXnjk1jYnTmeC8nB8 a7LhPFZniPBjLG19M3joQQPc7NPXkCxml1UPZLBCf3dXGdGYHpS8P3eUWkBPTkk4ZuUU 4oUsMzTbMuJeHqmtEcIpuNQMmJ6zt9noEDNZcEGJcKlLHqrgoI6b9rHi8vwGQnsDbwcB 9xVAdqOuAAnUt+f5QXL3lkLXc926N5FX0h+tI0b2HyFHAj7jzmjDnad3QIU8/q48prxU cpC8rKrnmXCo6JM+YJpbJdPjzZi5nJQo7DnLJE2qmqR/RWf3iZDUcGsbG36Ymr4PDsPf 2JQA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=+YZUR7d/Gifuq1dptnHD4e1ThZoIgNK0nMRC7vEtAMs=; fh=/h9QQkzJ8EboVkWg45aWwpaUro6WMavIVd2OhN45RtE=; b=Bf/+iZP/kYdHTVxUtQOmRBg5sLFEgEEhfYgV/Nmc3lX0BMW8ogSS9zbbmaTx2uqpnJ INvCbOTKkx3eOIly6UEKvPGbyLF+M5XwqXFhoh2iEvRNnLTXwDWvHTuvu10e/ScQ3m/R CZGotuh/X7VzKTtDyIFM2Jo2IzFJQOVfHb1yiH4H+Qb3T/QK3QBiS0Cvp5OpuELmui91 wEkq+w3tUzItKe8f2ROV6gWECEOWS1TZcuUvVerqSIY/mBneaM6UtImDv1SSepxxE8sa ufNCb3u3oPEUGESoIkLRWy9oBUW+wt7GYwOxi5lJSRiwMkBsH5VekawPv74HvGL0SAC7 1K2g==; dara=google.com ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=amikan@ilbers.de Received: from shymkent.ilbers.de (shymkent.ilbers.de. [85.214.156.166]) by gmr-mx.google.com with ESMTPS id ffacd0b85a97d-36074e0e07fsi74396f8f.0.2024.06.14.04.34.37 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 14 Jun 2024 04:34:37 -0700 (PDT) Received-SPF: pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) client-ip=85.214.156.166; Received: from localhost.localdomain (r154-240-38-77-broadband.btv.lv [77.38.240.154] (may be forged)) (authenticated bits=0) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPSA id 45EBYZOw032043 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 14 Jun 2024 13:34:36 +0200 From: Anton Mikanovich To: isar-users@googlegroups.com Cc: Anton Mikanovich Subject: [PATCH v9 1/8] isar-bootstrap: Move common parts to bbclass Date: Fri, 14 Jun 2024 14:34:18 +0300 Message-Id: <20240614113425.122722-2-amikan@ilbers.de> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240614113425.122722-1-amikan@ilbers.de> References: <20240614113425.122722-1-amikan@ilbers.de> MIME-Version: 1.0 X-Spam-Status: No, score=-4.6 required=5.0 tests=DKIMWL_WL_MED,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_EF,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,RCVD_IN_MSPIKE_H2,RCVD_IN_RP_CERTIFIED, RCVD_IN_RP_RNBL,RCVD_IN_RP_SAFE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on shymkent.ilbers.de X-Original-Sender: amikan@ilbers.de X-Original-Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=amikan@ilbers.de Precedence: list Mailing-list: list isar-users@googlegroups.com; contact isar-users+owners@googlegroups.com List-ID: X-Spam-Checked-In-Group: isar-users@googlegroups.com X-Google-Group-Id: 914930254986 List-Post: , List-Help: , List-Archive: , List-Unsubscribe: , X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= To allow implementing other bootstrap recipes move some functionality supposed to be reusable to bbclass. Signed-off-by: Anton Mikanovich --- meta/classes/bootstrap.bbclass | 228 ++++++++++++++++++ .../isar-bootstrap/isar-bootstrap.inc | 221 +---------------- 2 files changed, 230 insertions(+), 219 deletions(-) create mode 100644 meta/classes/bootstrap.bbclass diff --git a/meta/classes/bootstrap.bbclass b/meta/classes/bootstrap.bbclass new file mode 100644 index 00000000..b392677a --- /dev/null +++ b/meta/classes/bootstrap.bbclass @@ -0,0 +1,228 @@ +# This software is a part of Isar. +# Copyright (c) Siemens AG, 2018-2021 +# Copyright (C) 2024 ilbers GmbH +# +# SPDX-License-Identifier: MIT + +FILESEXTRAPATHS:append = ":${BBPATH}" + +SRC_URI = " \ + file://locale \ + file://chroot-setup.sh \ + ${DISTRO_BOOTSTRAP_KEYS} \ + ${THIRD_PARTY_APT_KEYS}" + +BOOTSTRAP_FOR_HOST ?= "0" + +APTPREFS = "${WORKDIR}/apt-preferences" +APTSRCS = "${WORKDIR}/apt-sources" +APTSRCS_INIT = "${WORKDIR}/apt-sources-init" +DISTRO_BOOTSTRAP_KEYFILES = "" +THIRD_PARTY_APT_KEYFILES = "" +DISTRO_BOOTSTRAP_KEYS ?= "" +THIRD_PARTY_APT_KEYS ?= "" +DEPLOY_ISAR_BOOTSTRAP ?= "" +DISTRO_BOOTSTRAP_BASE_PACKAGES ??= "" +DISTRO_VARS_PREFIX ?= "${@'HOST_' if bb.utils.to_boolean(d.getVar('BOOTSTRAP_FOR_HOST')) else ''}" +BOOTSTRAP_DISTRO = "${@d.getVar('HOST_DISTRO' if bb.utils.to_boolean(d.getVar('BOOTSTRAP_FOR_HOST')) else 'DISTRO')}" +BOOTSTRAP_BASE_DISTRO = "${@d.getVar('HOST_BASE_DISTRO' if bb.utils.to_boolean(d.getVar('BOOTSTRAP_FOR_HOST')) else 'BASE_DISTRO')}" + +# reproducible builds, only enabled if ISAR_USE_APT_SNAPSHOT +ISAR_APT_SNAPSHOT_MIRROR ??= "" +APT_SNAPSHOT_DATE = "${@ get_apt_snapshot_date(d)}" + +python () { + distro_bootstrap_keys = (d.getVar("DISTRO_BOOTSTRAP_KEYS") or "").split() + third_party_apt_keys = (d.getVar("THIRD_PARTY_APT_KEYS") or "").split() + topdir = d.getVar("TOPDIR") + + # The cached repo key can be both for bootstrapping and apt package + # installation afterwards. However, bootstrap will include the key into + # the rootfs automatically thus the right place is distro_bootstrap_keys. + + if bb.utils.to_boolean(d.getVar('ISAR_USE_CACHED_BASE_REPO')): + own_pub_key = d.getVar("BASE_REPO_KEY") + if own_pub_key: + distro_bootstrap_keys += own_pub_key.split() + + for key in distro_bootstrap_keys: + fetcher = bb.fetch2.Fetch([key], d) + filename = os.path.relpath(fetcher.localpath(key), topdir) + d.appendVar("DISTRO_BOOTSTRAP_KEYFILES", " ${TOPDIR}/%s" % filename) + + for key in third_party_apt_keys: + fetcher = bb.fetch2.Fetch([key], d) + filename = os.path.relpath(fetcher.localpath(key), topdir) + d.appendVar("THIRD_PARTY_APT_KEYFILES", " ${TOPDIR}/%s" % filename) + + distro_apt_sources = get_aptsources_list(d) + for file in distro_apt_sources: + d.appendVar("SRC_URI", " file://%s" % file) + + distro_apt_preferences = d.getVar(d.getVar("DISTRO_VARS_PREFIX") + "DISTRO_APT_PREFERENCES") or "" + for file in distro_apt_preferences.split(): + d.appendVar("SRC_URI", " file://%s" % file) +} + +def aggregate_files(d, file_list, file_out): + import shutil + + with open(file_out, "wb") as out_fd: + for entry in file_list: + entry_real = bb.parse.resolve_file(entry, d) + with open(entry_real, "rb") as in_fd: + shutil.copyfileobj(in_fd, out_fd, 1024*1024*10) + out_fd.write("\n".encode()) + +def parse_aptsources_list_line(source_list_line): + import re + + s = source_list_line.strip() + + if not s or s.startswith("#"): + return None + + type, s = re.split("\s+", s, maxsplit=1) + if type not in ["deb", "deb-src"]: + return None + + options = "" + options_match = re.match("\[\s*(\S+=\S+(?=\s))*\s*(\S+=\S+)\s*\]\s+", s) + if options_match: + options = options_match.group(0).strip() + s = s[options_match.end():] + + source, s = re.split("\s+", s, maxsplit=1) + + if s.startswith("/"): + suite = "" + else: + suite, s = re.split("\s+", s, maxsplit=1) + + components = " ".join(s.split()) + + return [type, options, source, suite, components] + +def get_apt_snapshot_date(d): + import time + source_date_epoch = d.getVar('ISAR_APT_SNAPSHOT_TIMESTAMP') + return time.strftime('%Y%m%dT%H%M%SZ', time.gmtime(int(source_date_epoch))) + +def get_apt_source_mirror(d, aptsources_entry_list): + import re + + # this is executed during parsing. No error checking possible + use_snapshot = bb.utils.to_boolean(d.getVar('ISAR_USE_APT_SNAPSHOT')) + snapshot_mirror = d.getVar('DISTRO_APT_SNAPSHOT_PREMIRROR') + if bb.utils.to_boolean(d.getVar('ISAR_USE_CACHED_BASE_REPO')): + premirrors = "\S* file://${REPO_BASE_DIR}/${BOOTSTRAP_BASE_DISTRO}\n" + elif use_snapshot and snapshot_mirror: + premirrors = snapshot_mirror + else: + premirrors = d.getVar('DISTRO_APT_PREMIRRORS') or "" + mirror_list = [entry.split() + for entry in premirrors.split('\\n') + if any(entry)] + + for regex, replace in mirror_list: + match = re.search(regex, aptsources_entry_list[2]) + + if match: + new_aptsources_entry_list = aptsources_entry_list.copy() + new_aptsources_entry_list[2] = re.sub(regex, replace, + aptsources_entry_list[2], + count = 1) + if use_snapshot: + new_aptsources_entry_list[1] = "[check-valid-until=no]" + return new_aptsources_entry_list + + return aptsources_entry_list + +def aggregate_aptsources_list(d, file_list, file_out): + import shutil + + with open(file_out, "wb") as out_fd: + for entry in file_list: + entry_real = bb.parse.resolve_file(entry, d) + with open(entry_real, "r") as in_fd: + for line in in_fd: + parsed = parse_aptsources_list_line(line) + if parsed: + parsed = get_apt_source_mirror(d, parsed) + out_fd.write(" ".join(parsed).encode()) + else: + out_fd.write(line.encode()) + out_fd.write("\n".encode()) + out_fd.write("\n".encode()) + +def get_aptsources_list(d): + import errno + from collections import OrderedDict + apt_sources_var = d.getVar("DISTRO_VARS_PREFIX") + "DISTRO_APT_SOURCES" + apt_sources_list = list(OrderedDict.fromkeys((d.getVar(apt_sources_var) or "").split())) + for p in apt_sources_list: + try: + bb.parse.resolve_file(p, d) + except FileNotFoundError as e: + bb.fatal(os.strerror(errno.ENOENT) + ' "' + p + '"') + return apt_sources_list + +def generate_distro_sources(d): + apt_sources_list = get_aptsources_list(d) + for entry in apt_sources_list: + with open(bb.parse.resolve_file(entry, d), "r") as in_fd: + for line in in_fd: + parsed = parse_aptsources_list_line(line) + if parsed: + parsed = get_apt_source_mirror(d, parsed) + yield parsed + +def get_distro_primary_source_entry(d): + for source in generate_distro_sources(d): + if source[0] == "deb": + return source[2:] + bb.fatal('Invalid apt sources list') + +def get_distro_source(d): + return get_distro_primary_source_entry(d)[0] + +def get_distro_suite(d): + return get_distro_primary_source_entry(d)[1] + +def get_distro_components_argument(d): + components = get_distro_primary_source_entry(d)[2] + if components and components.strip(): + return "--components=" + ",".join(components.split()) + else: + return "" + +do_apt_config_prepare[dirs] = "${WORKDIR}" +do_apt_config_prepare[vardeps] += " \ + APTPREFS \ + ${DISTRO_VARS_PREFIX}DISTRO_APT_PREFERENCES \ + DEBDISTRONAME \ + APTSRCS \ + ${DISTRO_VARS_PREFIX}DISTRO_APT_SOURCES \ + DEPLOY_ISAR_BOOTSTRAP \ + ${@'DISTRO_APT_SNAPSHOT_PREMIRROR' if bb.utils.to_boolean(d.getVar('ISAR_USE_APT_SNAPSHOT')) else ''} \ + " +python do_apt_config_prepare() { + apt_preferences_out = d.getVar("APTPREFS") + apt_preferences_list = ( + d.getVar(d.getVar("DISTRO_VARS_PREFIX") + "DISTRO_APT_PREFERENCES") or "" + ).split() + aggregate_files(d, apt_preferences_list, apt_preferences_out) + + apt_sources_out = d.getVar("APTSRCS") + apt_sources_init_out = d.getVar("APTSRCS_INIT") + apt_sources_list = get_aptsources_list(d) + + aggregate_files(d, apt_sources_list, apt_sources_init_out) + aggregate_aptsources_list(d, apt_sources_list, apt_sources_out) +} +addtask apt_config_prepare before do_bootstrap after do_unpack + +CLEANFUNCS = "clean_deploy" +clean_deploy() { + rm -f "${DEPLOY_ISAR_BOOTSTRAP}" +} diff --git a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc index faf22a50..5c9eb5c8 100644 --- a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc +++ b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc @@ -5,193 +5,20 @@ # # SPDX-License-Identifier: MIT +inherit bootstrap + LICENSE = "gpl-2.0" LIC_FILES_CHKSUM = "file://${LAYERDIR_core}/licenses/COPYING.GPLv2;md5=751419260aa954499f7abaabaa882bbe" -FILESPATH:prepend := "${THISDIR}/files:" -SRC_URI = " \ - file://locale \ - file://chroot-setup.sh \ - ${DISTRO_BOOTSTRAP_KEYS} \ - ${THIRD_PARTY_APT_KEYS}" PV = "1.0" -BOOTSTRAP_FOR_HOST ?= "0" - DEBOOTSTRAP ?= "qemu-debootstrap" ROOTFSDIR = "${WORKDIR}/rootfs" -APTPREFS = "${WORKDIR}/apt-preferences" -APTSRCS = "${WORKDIR}/apt-sources" -APTSRCS_INIT = "${WORKDIR}/apt-sources-init" -DISTRO_BOOTSTRAP_KEYFILES = "" -THIRD_PARTY_APT_KEYFILES = "" -DISTRO_BOOTSTRAP_KEYS ?= "" -THIRD_PARTY_APT_KEYS ?= "" -DEPLOY_ISAR_BOOTSTRAP ?= "" DISTRO_BOOTSTRAP_BASE_PACKAGES = "locales" DISTRO_BOOTSTRAP_BASE_PACKAGES:append:gnupg = ",gnupg" DISTRO_BOOTSTRAP_BASE_PACKAGES:append:https-support = ",ca-certificates" -DISTRO_VARS_PREFIX ?= "${@'HOST_' if bb.utils.to_boolean(d.getVar('BOOTSTRAP_FOR_HOST')) else ''}" -BOOTSTRAP_DISTRO = "${@d.getVar('HOST_DISTRO' if bb.utils.to_boolean(d.getVar('BOOTSTRAP_FOR_HOST')) else 'DISTRO')}" -BOOTSTRAP_BASE_DISTRO = "${@d.getVar('HOST_BASE_DISTRO' if bb.utils.to_boolean(d.getVar('BOOTSTRAP_FOR_HOST')) else 'BASE_DISTRO')}" -FILESEXTRAPATHS:append = ":${BBPATH}" -# reproducible builds, only enabled if ISAR_USE_APT_SNAPSHOT -ISAR_APT_SNAPSHOT_MIRROR ??= "" -APT_SNAPSHOT_DATE = "${@ get_apt_snapshot_date(d)}" inherit deb-dl-dir -python () { - distro_bootstrap_keys = (d.getVar("DISTRO_BOOTSTRAP_KEYS") or "").split() - third_party_apt_keys = (d.getVar("THIRD_PARTY_APT_KEYS") or "").split() - topdir = d.getVar("TOPDIR") - - # The cached repo key can be both for bootstrapping and apt package - # installation afterwards. However, debootstrap will include the key into - # the rootfs automatically thus the right place is distro_bootstrap_keys. - - if bb.utils.to_boolean(d.getVar('ISAR_USE_CACHED_BASE_REPO')): - own_pub_key = d.getVar("BASE_REPO_KEY") - if own_pub_key: - distro_bootstrap_keys += own_pub_key.split() - - for key in distro_bootstrap_keys: - fetcher = bb.fetch2.Fetch([key], d) - filename = os.path.relpath(fetcher.localpath(key), topdir) - d.appendVar("DISTRO_BOOTSTRAP_KEYFILES", " ${TOPDIR}/%s" % filename) - - for key in third_party_apt_keys: - fetcher = bb.fetch2.Fetch([key], d) - filename = os.path.relpath(fetcher.localpath(key), topdir) - d.appendVar("THIRD_PARTY_APT_KEYFILES", " ${TOPDIR}/%s" % filename) - - distro_apt_sources = get_aptsources_list(d) - for file in distro_apt_sources: - d.appendVar("SRC_URI", " file://%s" % file) - - distro_apt_preferences = d.getVar(d.getVar("DISTRO_VARS_PREFIX") + "DISTRO_APT_PREFERENCES") or "" - for file in distro_apt_preferences.split(): - d.appendVar("SRC_URI", " file://%s" % file) -} - -def aggregate_files(d, file_list, file_out): - import shutil - - with open(file_out, "wb") as out_fd: - for entry in file_list: - entry_real = bb.parse.resolve_file(entry, d) - with open(entry_real, "rb") as in_fd: - shutil.copyfileobj(in_fd, out_fd, 1024*1024*10) - out_fd.write("\n".encode()) - -def parse_aptsources_list_line(source_list_line): - import re - - s = source_list_line.strip() - - if not s or s.startswith("#"): - return None - - type, s = re.split("\s+", s, maxsplit=1) - if type not in ["deb", "deb-src"]: - return None - - options = "" - options_match = re.match("\[\s*(\S+=\S+(?=\s))*\s*(\S+=\S+)\s*\]\s+", s) - if options_match: - options = options_match.group(0).strip() - s = s[options_match.end():] - - source, s = re.split("\s+", s, maxsplit=1) - - if s.startswith("/"): - suite = "" - else: - suite, s = re.split("\s+", s, maxsplit=1) - - components = " ".join(s.split()) - - return [type, options, source, suite, components] - -def get_apt_snapshot_date(d): - import time - source_date_epoch = d.getVar('ISAR_APT_SNAPSHOT_TIMESTAMP') - return time.strftime('%Y%m%dT%H%M%SZ', time.gmtime(int(source_date_epoch))) - -def get_apt_source_mirror(d, aptsources_entry_list): - import re - - # this is executed during parsing. No error checking possible - use_snapshot = bb.utils.to_boolean(d.getVar('ISAR_USE_APT_SNAPSHOT')) - snapshot_mirror = d.getVar('DISTRO_APT_SNAPSHOT_PREMIRROR') - if bb.utils.to_boolean(d.getVar('ISAR_USE_CACHED_BASE_REPO')): - premirrors = "\S* file://${REPO_BASE_DIR}/${BOOTSTRAP_BASE_DISTRO}\n" - elif use_snapshot and snapshot_mirror: - premirrors = snapshot_mirror - else: - premirrors = d.getVar('DISTRO_APT_PREMIRRORS') or "" - mirror_list = [entry.split() - for entry in premirrors.split('\\n') - if any(entry)] - - for regex, replace in mirror_list: - match = re.search(regex, aptsources_entry_list[2]) - - if match: - new_aptsources_entry_list = aptsources_entry_list.copy() - new_aptsources_entry_list[2] = re.sub(regex, replace, - aptsources_entry_list[2], - count = 1) - if use_snapshot: - new_aptsources_entry_list[1] = "[check-valid-until=no]" - return new_aptsources_entry_list - - return aptsources_entry_list - -def aggregate_aptsources_list(d, file_list, file_out): - import shutil - - with open(file_out, "wb") as out_fd: - for entry in file_list: - entry_real = bb.parse.resolve_file(entry, d) - with open(entry_real, "r") as in_fd: - for line in in_fd: - parsed = parse_aptsources_list_line(line) - if parsed: - parsed = get_apt_source_mirror(d, parsed) - out_fd.write(" ".join(parsed).encode()) - else: - out_fd.write(line.encode()) - out_fd.write("\n".encode()) - out_fd.write("\n".encode()) - -def get_aptsources_list(d): - import errno - from collections import OrderedDict - apt_sources_var = d.getVar("DISTRO_VARS_PREFIX") + "DISTRO_APT_SOURCES" - apt_sources_list = list(OrderedDict.fromkeys((d.getVar(apt_sources_var) or "").split())) - for p in apt_sources_list: - try: - bb.parse.resolve_file(p, d) - except FileNotFoundError as e: - bb.fatal(os.strerror(errno.ENOENT) + ' "' + p + '"') - return apt_sources_list - -def generate_distro_sources(d): - apt_sources_list = get_aptsources_list(d) - for entry in apt_sources_list: - with open(bb.parse.resolve_file(entry, d), "r") as in_fd: - for line in in_fd: - parsed = parse_aptsources_list_line(line) - if parsed: - parsed = get_apt_source_mirror(d, parsed) - yield parsed - -def get_distro_primary_source_entry(d): - for source in generate_distro_sources(d): - if source[0] == "deb": - return source[2:] - bb.fatal('Invalid apt sources list') - def get_distro_have_https_source(d): return any(source[2].startswith("https://") for source in generate_distro_sources(d)) @@ -213,19 +40,6 @@ def get_distro_needs_gpg_support(d): OVERRIDES:append = ":${@get_distro_needs_gpg_support(d)}" -def get_distro_source(d): - return get_distro_primary_source_entry(d)[0] - -def get_distro_suite(d): - return get_distro_primary_source_entry(d)[1] - -def get_distro_components_argument(d): - components = get_distro_primary_source_entry(d)[2] - if components and components.strip(): - return "--components=" + ",".join(components.split()) - else: - return "" - APT_KEYS_DIR = "${WORKDIR}/aptkeys" DISTRO_BOOTSTRAP_KEYRING = "${WORKDIR}/distro-keyring.gpg" @@ -249,32 +63,6 @@ do_generate_keyrings() { } addtask generate_keyrings before do_build after do_unpack -do_apt_config_prepare[dirs] = "${WORKDIR}" -do_apt_config_prepare[vardeps] += " \ - APTPREFS \ - ${DISTRO_VARS_PREFIX}DISTRO_APT_PREFERENCES \ - DEBDISTRONAME \ - APTSRCS \ - ${DISTRO_VARS_PREFIX}DISTRO_APT_SOURCES \ - DEPLOY_ISAR_BOOTSTRAP \ - ${@'DISTRO_APT_SNAPSHOT_PREMIRROR' if bb.utils.to_boolean(d.getVar('ISAR_USE_APT_SNAPSHOT')) else ''} \ - " -python do_apt_config_prepare() { - apt_preferences_out = d.getVar("APTPREFS") - apt_preferences_list = ( - d.getVar(d.getVar("DISTRO_VARS_PREFIX") + "DISTRO_APT_PREFERENCES") or "" - ).split() - aggregate_files(d, apt_preferences_list, apt_preferences_out) - - apt_sources_out = d.getVar("APTSRCS") - apt_sources_init_out = d.getVar("APTSRCS_INIT") - apt_sources_list = get_aptsources_list(d) - - aggregate_files(d, apt_sources_list, apt_sources_init_out) - aggregate_aptsources_list(d, apt_sources_list, apt_sources_out) -} -addtask apt_config_prepare before do_bootstrap after do_unpack - def get_host_release(): import platform rel = platform.release() @@ -446,8 +234,3 @@ python do_bootstrap_setscene() { addtask do_bootstrap_setscene do_bootstrap_setscene[dirs] = "${DEPLOY_DIR_BOOTSTRAP}" - -CLEANFUNCS = "clean_deploy" -clean_deploy() { - rm -f "${DEPLOY_ISAR_BOOTSTRAP}" -} From patchwork Fri Jun 14 11:34:19 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anton Mikanovich X-Patchwork-Id: 3608 Return-Path: Received: from shymkent.ilbers.de ([unix socket]) by shymkent (Cyrus 2.5.10-Debian-2.5.10-3+deb9u2) with LMTPA; Fri, 14 Jun 2024 13:34:48 +0200 X-Sieve: CMU Sieve 2.4 Received: from mail-lj1-f190.google.com (mail-lj1-f190.google.com [209.85.208.190]) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPS id 45EBYluS032113 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Fri, 14 Jun 2024 13:34:47 +0200 Received: by mail-lj1-f190.google.com with SMTP id 38308e7fff4ca-2ebddcae207sf15539321fa.2 for ; Fri, 14 Jun 2024 04:34:47 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1718364881; cv=pass; d=google.com; s=arc-20160816; b=A3RcZ57uSWZbxH7LQvdst2HOPJgwpHvp/N3BFPcyxKg9fqfCLbkRjeSb3zTABnV/zp L5eUWPMzqlXH/ePTGyAe7iP1mJqurangnTpiYo3NaUZwdudut4igJSgaE68Vl5i9qlgF IiQnD2SjyMZBjT5CQbmaZK6rTkM5x4jZsitfRKi91BVkM6WzEtqlMoj+MwAKZzvwuj9n E18Rhz6fDX/VrDBT6swLlKRjzTfGtgBiFdo8erw+KL8hOnKDXhHWpCDoBnGc7RB0HqH9 8LwynV0p/P7S3GjkmumCv9Xto8nJ9kt0bx7wp1X4ir1iuQ0EbPIi8fY2Zoccg5izfsWt usSQ== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:dkim-signature; bh=SbwvZdcBof2/ZsfJtNVWWvrvIpmyi8n94+dg+QUEwTk=; fh=jegAJ7X1tfTXYS5ZxZuVcsc2Q5BzkeW2CNGxBMfvzN8=; b=AiBzK4NdYI7R+aAv1JhMTLyZTVKuxBJKgJxbrNFwgnUXS+1Ov96OTCXUjDPO/LAsKU 521k+hOLw/tNNZ1QbB8B2fzWBc2SW1nMroP69s8Ct+ljeGG9IOJeA4splG/RlXPPTB2k 4X7BEGzMdHzNCJ2e6rtDEyZWVgkB9kT9Dq2Uyh9N9M7DDYPgDXeCvoRNphOu/b3DLc7d Var+N2oBNABmISaLBf0LmtrEyJyzOCXkm5ss018KslwZWWDsws9NgmRNyNBuFACETNkV ycqQqaa+dTn1eBbSToX2VzIfq1k8guXEhYB39Y5zXlIxIpjl2WtJoHMCtIekqkGNeRLn QK3g==; darn=isar-build.org ARC-Authentication-Results: i=2; gmr-mx.google.com; spf=pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=amikan@ilbers.de DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlegroups.com; s=20230601; t=1718364881; x=1718969681; darn=isar-build.org; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:x-original-authentication-results :x-original-sender:mime-version:references:in-reply-to:message-id :date:subject:cc:to:from:sender:from:to:cc:subject:date:message-id :reply-to; bh=SbwvZdcBof2/ZsfJtNVWWvrvIpmyi8n94+dg+QUEwTk=; b=gSuNFNi43QPFn9wjzPoCtoG174RVfzwzaI0JcGMsPfmcuCaqPYhorAaWf5+NXb7xbc hRbNuR4zaN3RcsexoXAQPE7+0ddEdLyvy1HIk6GjUoOdXuBRNKbc91BxO7VqGF82BXdd qf7F6ceRtpNzO4m07GRvWs1g/vRz0cHMbSSiU0i0ygTI7nC0vWGJhmFd/BvFD8IAFZZK KZXASMO7Mkf1NCyxqs2VGuy2yF5fH3kAnbargDVJGnGcd02q7KwKnZY+AsYRME5jnRcq X7Kp/tvMCO30aRBQ+paJZOp2EPSn2FCcCSYvez8PsUXPlcxvS99YblrmxThdR3x+BPyY c+XA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1718364881; x=1718969681; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :x-spam-checked-in-group:list-id:mailing-list:precedence :x-original-authentication-results:x-original-sender:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :x-beenthere:x-gm-message-state:sender:from:to:cc:subject:date :message-id:reply-to; bh=SbwvZdcBof2/ZsfJtNVWWvrvIpmyi8n94+dg+QUEwTk=; b=VdL0mjatHW3vrjNdkXC9/bB3ytfK9hFHf4MAovyWBaiha+sxpzZPuKxqVfc9simXfe p9b+4y8t6U2hcpJGl5kZjOrchcbS2bUUjjvdBSgM3hkVg1bgq6flEOpIwUAfB5/wA682 1/3VD4oUkZTrP7UvQRbNLpRUf5Ej8JXg2XDaXCd4yfrOuZkcb/kPNY0T3rb4Vn5oOYH8 r79ZJZm0ZBWvPKv3ySHT9m8438M4k2Rx8TVZq45SCLj0gCF5lBO2820DBKwNf4TiVQPf nvgFm+OohZGKL4cqz8dMsTPsbOHqns+Z6eMAFqNCuaSfgWKF+A03ai1/VM8csKlr4g7+ t/ZA== Sender: isar-users@googlegroups.com X-Forwarded-Encrypted: i=2; AJvYcCVUOs4nwI2XJYNmB5uCo0qOJlWU0L7I6uUsXOKYMFtLqv/cfxAiHc6dyVq7DG/ZFrl3gLcK9MGhFn11bRf30BzhZTiJgb4= X-Gm-Message-State: AOJu0Yy7SmURojoamk79LNWfKYif9ycAYQWHYPhAgu3CEroh0aEWMWwF M8WgrYzXj/3voh9ywiu/Jxbs4dCfRAUspz9X9obBw03zBX5A4dDZ X-Google-Smtp-Source: AGHT+IF7OrHaa+S69zoioTU1O7AS4R3VF5hMzF+R3N9sqFRS6JmKaKn0ppkyXFeLxJtzdf7lT4B+/w== X-Received: by 2002:a2e:b0f7:0:b0:2eb:d9f0:87d3 with SMTP id 38308e7fff4ca-2ec0e5ae60cmr15298071fa.53.1718364880653; Fri, 14 Jun 2024 04:34:40 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a05:651c:313:b0:2eb:ff5d:15d with SMTP id 38308e7fff4ca-2ec02936c1dls8894431fa.2.-pod-prod-03-eu; Fri, 14 Jun 2024 04:34:38 -0700 (PDT) X-Received: by 2002:a2e:99d8:0:b0:2ea:eb96:b952 with SMTP id 38308e7fff4ca-2ec0e46ec08mr17675351fa.22.1718364878440; Fri, 14 Jun 2024 04:34:38 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1718364878; cv=none; d=google.com; s=arc-20160816; b=UW8HyP+ISxJnhsEl6eWjkv7PkXGg8qg7f9k2huisxgQq4rhdAzqvNo++tzCh2S69no KZN7zNMmNizn3+9vf80K7oDXyYjYmdtLDwOmKlCt4DVM/lIEzMvCaZgZi3HJn1CsUvPO hJVyDZlkghKetTiIM7P13mbWcd87s5xF1+0FpW1XlkJkfdRAq2PUOliUIQf0T5iVTUpf yf1/jPDXuSvYKrzWxxEH08JsgvEfArhgO+xKrBGhJJbZdijaLwcBWgiMCEKT5UEFWfhe rIcQaRbtCKm/zomXCSnCCv0dSB1yLKbvtqJMfReO0UuhijtP2LJoBn0LxQDV7XTBXloT LzmQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=XGoSey23MmGfWxb+QhdSEo89ZAoBGDs+5t0yTJoYugQ=; fh=/h9QQkzJ8EboVkWg45aWwpaUro6WMavIVd2OhN45RtE=; b=Gzo0jHOXFTz9j6ZdYb9NSWpaB8nvUbIYw3pA6aOA7y3uSXJGvyrt/yZAYYAciCKWaf BfEP4cFRna1t8kgtpI/w6eRufYsMTgc6G4aZqI3iX8VR3GPEzrFy6vHH83xAR0oMdlZ5 Ff3huNddrjjNQe8reJ3FcYl2amuoshxF+b17Jmv9V2sBzntO+viZq3tjIKWtbB9CDxXx Gn8tm28XCjrLRXKtcir4KTwoEeEYuTkI2o/ccY/H7byiybSXXfT8JaPT5NL51AiEJLhO g8LSXc5y/AY2vmSEfxYqPmjlqRDwPwbZQFXmIoNPW+zkWObv8OTJcj1zK9cxDJ8Xduqi SOVg==; dara=google.com ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=amikan@ilbers.de Received: from shymkent.ilbers.de (shymkent.ilbers.de. [85.214.156.166]) by gmr-mx.google.com with ESMTPS id 38308e7fff4ca-2ec05cc82efsi789541fa.7.2024.06.14.04.34.38 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 14 Jun 2024 04:34:38 -0700 (PDT) Received-SPF: pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) client-ip=85.214.156.166; Received: from localhost.localdomain (r154-240-38-77-broadband.btv.lv [77.38.240.154] (may be forged)) (authenticated bits=0) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPSA id 45EBYZOx032043 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 14 Jun 2024 13:34:37 +0200 From: Anton Mikanovich To: isar-users@googlegroups.com Cc: Anton Mikanovich Subject: [PATCH v9 2/8] meta: Add mmdebstrap recipe Date: Fri, 14 Jun 2024 14:34:19 +0300 Message-Id: <20240614113425.122722-3-amikan@ilbers.de> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240614113425.122722-1-amikan@ilbers.de> References: <20240614113425.122722-1-amikan@ilbers.de> MIME-Version: 1.0 X-Spam-Status: No, score=-4.6 required=5.0 tests=DKIMWL_WL_MED,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_EF,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,RCVD_IN_MSPIKE_H2,RCVD_IN_RP_CERTIFIED, RCVD_IN_RP_RNBL,RCVD_IN_RP_SAFE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on shymkent.ilbers.de X-Original-Sender: amikan@ilbers.de X-Original-Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=amikan@ilbers.de Precedence: list Mailing-list: list isar-users@googlegroups.com; contact isar-users+owners@googlegroups.com List-ID: X-Spam-Checked-In-Group: isar-users@googlegroups.com X-Google-Group-Id: 914930254986 List-Post: , List-Help: , List-Archive: , List-Unsubscribe: , X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= It can be used as debootstrap alternative for rootfs prepare. Internally, it uses apt and allows to bootstrap the distro from multiple repositories. chroot-setup.sh and locale are copied from debootstrap recipe. Signed-off-by: Anton Mikanovich --- meta/classes/bootstrap.bbclass | 1 + .../isar-mmdebstrap/isar-mmdebstrap-host.bb | 17 ++ .../isar-mmdebstrap/isar-mmdebstrap-target.bb | 12 + .../isar-mmdebstrap/isar-mmdebstrap.inc | 230 ++++++++++++++++++ 4 files changed, 260 insertions(+) create mode 100644 meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap-host.bb create mode 100644 meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap-target.bb create mode 100644 meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc diff --git a/meta/classes/bootstrap.bbclass b/meta/classes/bootstrap.bbclass index b392677a..deb00b23 100644 --- a/meta/classes/bootstrap.bbclass +++ b/meta/classes/bootstrap.bbclass @@ -26,6 +26,7 @@ DISTRO_BOOTSTRAP_BASE_PACKAGES ??= "" DISTRO_VARS_PREFIX ?= "${@'HOST_' if bb.utils.to_boolean(d.getVar('BOOTSTRAP_FOR_HOST')) else ''}" BOOTSTRAP_DISTRO = "${@d.getVar('HOST_DISTRO' if bb.utils.to_boolean(d.getVar('BOOTSTRAP_FOR_HOST')) else 'DISTRO')}" BOOTSTRAP_BASE_DISTRO = "${@d.getVar('HOST_BASE_DISTRO' if bb.utils.to_boolean(d.getVar('BOOTSTRAP_FOR_HOST')) else 'BASE_DISTRO')}" +BOOTSTRAP_DISTRO_ARCH = "${@d.getVar('HOST_ARCH' if bb.utils.to_boolean(d.getVar('BOOTSTRAP_FOR_HOST')) else 'DISTRO_ARCH')}" # reproducible builds, only enabled if ISAR_USE_APT_SNAPSHOT ISAR_APT_SNAPSHOT_MIRROR ??= "" diff --git a/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap-host.bb b/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap-host.bb new file mode 100644 index 00000000..66c8d11e --- /dev/null +++ b/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap-host.bb @@ -0,0 +1,17 @@ +# Minimal host Debian root file system +# +# This software is a part of Isar. +# Copyright (C) 2024 ilbers GmbH +# +# SPDX-License-Identifier: MIT + +Description = "Minimal host Debian root file system" + +DEPLOY_ISAR_BOOTSTRAP = "${DEPLOY_DIR_BOOTSTRAP}/${HOST_DISTRO}-host_${DISTRO}-${DISTRO_ARCH}" + +BOOTSTRAP_FOR_HOST = "1" + +require isar-mmdebstrap.inc + +HOST_DISTRO_BOOTSTRAP_KEYS ?= "" +DISTRO_BOOTSTRAP_KEYS = "${HOST_DISTRO_BOOTSTRAP_KEYS}" diff --git a/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap-target.bb b/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap-target.bb new file mode 100644 index 00000000..84a89ff1 --- /dev/null +++ b/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap-target.bb @@ -0,0 +1,12 @@ +# Minimal target Debian root file system +# +# This software is a part of Isar. +# Copyright (C) 2024 ilbers GmbH +# +# SPDX-License-Identifier: MIT + +Description = "Minimal target Debian root file system" + +DEPLOY_ISAR_BOOTSTRAP = "${DEPLOY_DIR_BOOTSTRAP}/${DISTRO}-${DISTRO_ARCH}" + +require isar-mmdebstrap.inc diff --git a/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc b/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc new file mode 100644 index 00000000..72096ae2 --- /dev/null +++ b/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc @@ -0,0 +1,230 @@ +# Minimal debian root file system +# +# This software is a part of Isar. +# Copyright (C) 2024 ilbers GmbH +# +# SPDX-License-Identifier: MIT + +inherit bootstrap +inherit compat +inherit deb-dl-dir + +FILESEXTRAPATHS:append = ":${LAYERDIR_core}/recipes-core/isar-bootstrap/files" + +ROOTFSDIR = "${WORKDIR}/rootfs" +DISTRO_BOOTSTRAP_BASE_PACKAGES = "locales,apt,usrmerge" +DISTRO_BOOTSTRAP_BASE_PACKAGES:append:gnupg = ",gnupg" +DISTRO_BOOTSTRAP_BASE_PACKAGES:append:https-support = ",ca-certificates" +BOOTSTRAP_TMPDIR = "${WORKDIR}/tempdir" + +def get_distro_primary_source_entry(d): + for source in generate_distro_sources(d): + if source[0] == "deb": + return source[2:] + bb.fatal('Invalid apt sources list') + +def get_distro_have_https_source(d): + return any(source[2].startswith("https://") for source in generate_distro_sources(d)) + +def get_distro_needs_https_support(d): + if get_distro_have_https_source(d): + return "https-support" + else: + return "" + +OVERRIDES:append = ":${@get_distro_needs_https_support(d)}" + +def get_distro_needs_gpg_support(d): + if d.getVar("DISTRO_BOOTSTRAP_KEYS") or \ + d.getVar("THIRD_PARTY_APT_KEYS") or \ + d.getVar("BASE_REPO_KEY"): + return "gnupg" + else: + return "" + +OVERRIDES:append = ":${@get_distro_needs_gpg_support(d)}" + +APT_KEYS_DIR = "${WORKDIR}/aptkeys" +DISTRO_BOOTSTRAP_KEYRING = "${WORKDIR}/distro-keyring.gpg" + +do_generate_keyrings[cleandirs] = "${APT_KEYS_DIR}" +do_generate_keyrings[dirs] = "${DL_DIR}" +do_generate_keyrings[vardeps] += "DISTRO_BOOTSTRAP_KEYS THIRD_PARTY_APT_KEYS" +do_generate_keyrings[network] = "${TASK_USE_SUDO}" +do_generate_keyrings() { + if [ -n "${@d.getVar("THIRD_PARTY_APT_KEYFILES") or ""}" ]; then + chmod 777 "${APT_KEYS_DIR}" + for keyfile in ${@d.getVar("THIRD_PARTY_APT_KEYFILES")}; do + cp "$keyfile" "${APT_KEYS_DIR}"/"$(basename "$keyfile")" + done + fi + if [ -n "${@d.getVar("DISTRO_BOOTSTRAP_KEYFILES") or ""}" ]; then + for keyfile in ${@d.getVar("DISTRO_BOOTSTRAP_KEYFILES")}; do + sudo apt-key --keyring "${DISTRO_BOOTSTRAP_KEYRING}" add $keyfile + cp "$keyfile" "${APT_KEYS_DIR}"/"$(basename "$keyfile")" + done + fi +} +addtask generate_keyrings before do_build after do_unpack + +do_bootstrap[vardeps] += " \ + DISTRO_APT_PREMIRRORS \ + ISAR_ENABLE_COMPAT_ARCH \ + ${DISTRO_VARS_PREFIX}DISTRO_APT_SOURCES \ + " +do_bootstrap[dirs] = "${DEPLOY_DIR_BOOTSTRAP} ${BOOTSTRAP_TMPDIR}" +do_bootstrap[depends] = "base-apt:do_cache isar-apt:do_cache_config" +do_bootstrap[network] = "${TASK_USE_NETWORK_AND_SUDO}" + +do_bootstrap() { + if [ "${ISAR_ENABLE_COMPAT_ARCH}" = "1" ]; then + if [ -z "${COMPAT_DISTRO_ARCH}" ]; then + bbfatal "${DISTRO_ARCH} does not have a compat arch" + fi + fi + bootstrap_args="--verbose --variant=minbase --include=${DISTRO_BOOTSTRAP_BASE_PACKAGES}" + if [ -f "${DISTRO_BOOTSTRAP_KEYRING}" ]; then + bootstrap_args="$bootstrap_args --keyring=${DISTRO_BOOTSTRAP_KEYRING}" + fi + E="${@ isar_export_proxies(d)}" + export BOOTSTRAP_FOR_HOST + + deb_dl_dir_import "${ROOTFSDIR}" "${BOOTSTRAP_BASE_DISTRO}-${BASE_DISTRO_CODENAME}" + sudo rm -rf --one-file-system "${ROOTFSDIR}" + mkdir -p "${ROOTFSDIR}" + + arch_param="--arch=${BOOTSTRAP_DISTRO_ARCH},${DISTRO_ARCH}" + + sudo TMPDIR="${BOOTSTRAP_TMPDIR}" mmdebstrap $bootstrap_args \ + $arch_param \ + --mode=unshare \ + ${@get_distro_components_argument(d)} \ + "${@get_distro_suite(d)}" \ + "${WORKDIR}/rootfs.tar.zst" \ + "${@get_distro_source(d)}" + + sudo -E -s <<'EOSUDO' + set -e + + tar -xf "${WORKDIR}/rootfs.tar.zst" -C "${ROOTFSDIR}" --exclude="./dev/console" + + # Install apt config + mkdir -p "${ROOTFSDIR}/etc/apt/preferences.d" + install -v -m644 "${APTPREFS}" \ + "${ROOTFSDIR}/etc/apt/preferences.d/bootstrap" + mkdir -p "${ROOTFSDIR}/etc/apt/sources.list.d" + if [ "${ISAR_USE_CACHED_BASE_REPO}" = "1" ]; then + line="file:///base-apt/${BOOTSTRAP_BASE_DISTRO} ${BASE_DISTRO_CODENAME} main" + if [ -z "${BASE_REPO_KEY}" ]; then + line="[trusted=yes] ${line}" + fi + echo "deb ${line}" > "${ROOTFSDIR}/etc/apt/sources.list.d/base-apt.list" + line="file:///base-apt/${BASE_DISTRO} ${BASE_DISTRO_CODENAME} main" + if [ -z "${BASE_REPO_KEY}" ]; then + line="[trusted=yes] ${line}" + fi + echo "deb-src ${line}" >> "${ROOTFSDIR}/etc/apt/sources.list.d/base-apt.list" + + mkdir -p ${ROOTFSDIR}/base-apt + mount --bind ${REPO_BASE_DIR} ${ROOTFSDIR}/base-apt + else + install -v -m644 "${APTSRCS}" \ + "${ROOTFSDIR}/etc/apt/sources.list.d/bootstrap.list" + fi + install -v -m644 "${APTSRCS_INIT}" "${ROOTFSDIR}/etc/apt/sources-list" + rm -f "${ROOTFSDIR}/etc/apt/sources.list" + rm -rf "${ROOTFSDIR}/var/lib/apt/lists/"* + find ${APT_KEYS_DIR}/ -type f | while read keyfile + do + MY_GPGHOME="$(chroot "${ROOTFSDIR}" mktemp -d /tmp/gpghomeXXXXXXXXXX)" + echo "Created temporary directory ${MY_GPGHOME} for gpg-agent" + export GNUPGHOME="${MY_GPGHOME}" + APT_KEY_APPEND="--homedir ${MY_GPGHOME}" + + kfn="$(basename $keyfile)" + cp $keyfile "${ROOTFSDIR}/tmp/$kfn" + chroot "${ROOTFSDIR}" /usr/bin/gpg-agent --daemon -- /usr/bin/apt-key \ + --keyring ${THIRD_PARTY_APT_KEYRING} ${APT_KEY_APPEND} add "/tmp/$kfn" + rm "${ROOTFSDIR}/tmp/$kfn" + + echo "Removing ${MY_GPGHOME}" + rm -rf "${ROOTFSDIR}${MY_GPGHOME}" + done + + # Set locale + install -v -m644 "${WORKDIR}/locale" "${ROOTFSDIR}/etc/locale" + + sed -i '/en_US.UTF-8 UTF-8/s/^#//g' "${ROOTFSDIR}/etc/locale.gen" + chroot "${ROOTFSDIR}" /usr/sbin/locale-gen + + # setup chroot + install -v -m755 "${WORKDIR}/chroot-setup.sh" "${ROOTFSDIR}/chroot-setup.sh" + "${ROOTFSDIR}/chroot-setup.sh" "setup" "${ROOTFSDIR}" + + # update APT + mount -o bind,private /dev ${ROOTFSDIR}/dev + mount --bind /dev/pts ${ROOTFSDIR}/dev/pts + mount -t tmpfs none "${ROOTFSDIR}/dev/shm" + mount -t proc none ${ROOTFSDIR}/proc + mount --rbind /sys ${ROOTFSDIR}/sys + mount --make-rslave ${ROOTFSDIR}/sys + + export DEBIAN_FRONTEND=noninteractive + + if [ "${BOOTSTRAP_FOR_HOST}" = "1" ]; then + chroot "${ROOTFSDIR}" /usr/bin/dpkg --add-architecture ${DISTRO_ARCH} + fi + + if [ "${ISAR_ENABLE_COMPAT_ARCH}" = "1" ]; then + chroot "${ROOTFSDIR}" /usr/bin/dpkg --add-architecture ${COMPAT_DISTRO_ARCH} + fi + + chroot "${ROOTFSDIR}" /usr/bin/apt-get update -y \ + -o APT::Update::Error-Mode=any + chroot "${ROOTFSDIR}" /usr/bin/apt-get install -y -f + chroot "${ROOTFSDIR}" /usr/bin/apt-get dist-upgrade -y \ + -o Debug::pkgProblemResolver=yes + + umount -l "${ROOTFSDIR}/dev/shm" + umount -l "${ROOTFSDIR}/dev/pts" + umount -l "${ROOTFSDIR}/dev" + umount -l "${ROOTFSDIR}/proc" + umount -l "${ROOTFSDIR}/sys" + umount -l "${ROOTFSDIR}/base-apt" || true + + # Finalize bootstrap by setting the link in deploy + ln -Tfsr "${ROOTFSDIR}" "${DEPLOY_ISAR_BOOTSTRAP}" +EOSUDO + deb_dl_dir_export "${ROOTFSDIR}" "${BOOTSTRAP_BASE_DISTRO}-${BASE_DISTRO_CODENAME}" + + # Cleanup apt cache + sudo -Es chroot "${ROOTFSDIR}" /usr/bin/apt-get -y clean +} +addtask bootstrap before do_build after do_generate_keyrings + +SSTATETASKS += "do_bootstrap" +SSTATECREATEFUNCS += "bootstrap_sstate_prepare" +SSTATEPOSTINSTFUNCS += "bootstrap_sstate_finalize" + +bootstrap_sstate_prepare() { + # this runs in SSTATE_BUILDDIR, which will be deleted automatically + sudo cp -a "$(dirname "${ROOTFSDIR}")/rootfs.tar.zst" ./bootstrap.tar.zst + sudo chown $(id -u):$(id -g) bootstrap.tar.zst +} + +bootstrap_sstate_finalize() { + # this runs in SSTATE_INSTDIR + # we should restore symlinks after using tar + if [ -f bootstrap.tar.zst ]; then + mv bootstrap.tar.zst "$(dirname "${ROOTFSDIR}")/rootfs.tar.zst" + sudo ln -Tfsr "$(dirname "${ROOTFSDIR}")/rootfs.tar.zst" \ + "${DEPLOY_ISAR_BOOTSTRAP}.tar.zst" + fi +} + +python do_bootstrap_setscene() { + sstate_setscene(d) +} + +addtask do_bootstrap_setscene +do_bootstrap_setscene[dirs] = "${DEPLOY_DIR_BOOTSTRAP}" From patchwork Fri Jun 14 11:34:20 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anton Mikanovich X-Patchwork-Id: 3609 Return-Path: Received: from shymkent.ilbers.de ([unix socket]) by shymkent (Cyrus 2.5.10-Debian-2.5.10-3+deb9u2) with LMTPA; Fri, 14 Jun 2024 13:34:49 +0200 X-Sieve: CMU Sieve 2.4 Received: from mail-lj1-f184.google.com (mail-lj1-f184.google.com [209.85.208.184]) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPS id 45EBYlm8032125 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Fri, 14 Jun 2024 13:34:47 +0200 Received: by mail-lj1-f184.google.com with SMTP id 38308e7fff4ca-2eae267d112sf3377251fa.1 for ; Fri, 14 Jun 2024 04:34:47 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1718364882; cv=pass; d=google.com; s=arc-20160816; b=VLLeknz6CigG9hpBd8lwL5mspzrX8pX8jkpvFBwXJPqMQKCltM+Aw9ujC2ZpEQZgbr M6p63FIKbDCSnF4hWO5LbmxIigizEUWZHCbk7Q04m25QURoMP3K9P1h0rVYvlDfbJvMw /tDLXWbN+fYrYbFCCXIdHRBSDNUrvP5tIVd3GBUofciCiHYMTZgfDn/6+0E91q/RnrqC 4CDxO0ZRH8QjzU7oEgAwLLG99zSYiCeqnohKUx9u99mI13LPUZmlL53jnOO7BEWZZXHQ 2unU1SqiSlCMRAfvHTJ1JXauYGYTY3ttUUmv6YeQoOtANMnU7RV4DlGakSm9A4KUgaqg 5SWg== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:dkim-signature; bh=vkfC9lSo8iuGLb2Z9L6XEFiS4VxGLHsboyI3Hg2jqaI=; fh=UqU218yQs0gci8aKc5y1geIeTV0iAwkMnpqzOrGI4q0=; b=ECO0Lkh/cy7rkpr1+XKdrRULZLyHCZdF7buT8m4u5mc6Qkg8vsG6DY1BOuHKw9Zbri sIoxLdoxKoBTlh3I1GDXPTSiETcitEwwkoSz/Edk0u6vT2ilmj7eSV2HGM1qOqxYpr3m 7G/VICWqyrdHX3g236ZDHVldyIWeQiBpx77QoWcVrVHo9ibwcJZ+aHv7KK6l+13Q+gUi ukBDn1Sc9Cbv94LyF4R81UB0+cW46NO9nx6FooyesGcEQMM9VTlHURbkaHKuRv5b3YHY w5nn8YRVZoe0o3DCk4okonzmxQCNwu++baPuW2kaXyTYgmjdD4qK+WQIvbDcFBEzham3 FwXw==; darn=isar-build.org ARC-Authentication-Results: i=2; gmr-mx.google.com; spf=pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=amikan@ilbers.de DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlegroups.com; s=20230601; t=1718364882; x=1718969682; darn=isar-build.org; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:x-original-authentication-results :x-original-sender:mime-version:references:in-reply-to:message-id :date:subject:cc:to:from:sender:from:to:cc:subject:date:message-id :reply-to; bh=vkfC9lSo8iuGLb2Z9L6XEFiS4VxGLHsboyI3Hg2jqaI=; b=ALpd/fhcFpOAKni0/RwAJR1yhuAK7NKK2sTvl+CYGktPXQRg3nUM9Uel20oJjMGYg3 onG7wXOHCubgvy4u9rgSeSsUyTA6T9x6VlRK2y38EGnN5VdHDLtyoELa7DcnBf6KxFxM XxU38g84lX/s5pfoTH4LofPNkpk16DBF1JKIVpk8gdZ+m9nzYJgvTlRYVU58HwalKam5 MIcJMQoBx6bkKRB5dJRGItkxhVkttgWujuDbxc1N154B918hmu5Owc/wtSaSkKjkx750 cRzyR8z4VGSTLSgogm4c/uPWvFdnyMVGBUNxLCDyXX+oDfBWCDnoWNPjpMagSvPeFoqT bPoA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1718364882; x=1718969682; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :x-spam-checked-in-group:list-id:mailing-list:precedence :x-original-authentication-results:x-original-sender:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :x-beenthere:x-gm-message-state:sender:from:to:cc:subject:date :message-id:reply-to; bh=vkfC9lSo8iuGLb2Z9L6XEFiS4VxGLHsboyI3Hg2jqaI=; b=KdJB0jX2KA7itaObSqZKaKSEahjrXfp2Ysk1uJBHjvy6RjSgrssopXqnp6cDSpNK56 6qdgGa9tethurWSFNByYjM3/uPgcnSLrv79GoVHv5ZrXJ+APyJff+xQ2OJPNLft/QjCT WLzF1xHi65SG6AIEKZYGC9PYtMIPp9C4+ZZmMkofeaq3tYLMHhQfWubJrgNeGAu1U3CH TYXYGk1n5vKB+3UPRvDJDIN3z5fRKwwnPX7Sd89X+TAxTFt9mfB3aGrMNGmFA4g33lup jVndeRjOp3BvuuxfPb40Fo5ueaS8ico3qSSXIH8UP3h8IaPNRXhNJo7NTZ+i6y9YJVvG wJOg== Sender: isar-users@googlegroups.com X-Forwarded-Encrypted: i=2; AJvYcCVg5nEnc24XpXgGCwW4Lq2RA6ouj6a45YCbBM7xSx05SJN32R1dJAYBI6iBPFg94A7FC2J1epUEpoFhw0FG3j0Pj/shvtc= X-Gm-Message-State: AOJu0YwDhHSpVnrzxkVpbhNACdQK5O0lLMPXinhZ1n5P6IzUd75WaR8o VmShhDfak/CuPBDanCTLe/lOQk/vO8Ffh4POfXG2q3AIQke/SrSw X-Google-Smtp-Source: AGHT+IE9mcT9i6Ai1paPcyWw56nyhZ2WnMrAZmtSxp9/bgua4iizaIhl+VA1klLrh2PEAqQEpH3gXw== X-Received: by 2002:a05:6512:1051:b0:52c:af45:ed99 with SMTP id 2adb3069b0e04-52caf45ee94mr414810e87.0.1718364880919; Fri, 14 Jun 2024 04:34:40 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a05:6512:753:b0:52c:842a:a28 with SMTP id 2adb3069b0e04-52ca050245cls1000891e87.0.-pod-prod-09-eu; Fri, 14 Jun 2024 04:34:39 -0700 (PDT) X-Received: by 2002:a05:6512:24c:b0:52c:84a5:6617 with SMTP id 2adb3069b0e04-52ca6e9eb89mr1410505e87.68.1718364878714; Fri, 14 Jun 2024 04:34:38 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1718364878; cv=none; d=google.com; s=arc-20160816; b=nz/SqX/g5wm0V+9M1tWXujf2M42Ry2I4w87UYnDf9a9VdMjraVEWbJZfNnjA0yrqb6 Z7DOdD5lI47PqZMN3vRe44hiODq2mMTtFyI5w+ML9xbOr/y1mnRur/iESbpx3VGkJXI6 xsOhEfMC4YFLPVZaXF/Oh1RBojWhdUXlYhoKVTeKjAo6V1bVMzSHCOVheAkwcz7H9uIH K7TquBx+fcl2EWg/Wi3a8CxsgOp+ah3ybhPzKX+4p+O6ogZ/5dOpQkiOUZ1TSQxkkfJz lElg1PsMHFC+3KVM0L0u9OfEVRiTfmI5Gikj7cxgTBRiiy48sZHEgmQlLUQlyy/VkXXg /x0Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=vi6da4/Db7htnChLx1vf7gUb+9xiUPQ+E45J3pdB96w=; fh=/h9QQkzJ8EboVkWg45aWwpaUro6WMavIVd2OhN45RtE=; b=DnVl4hzUpfb3FwCphIT/hXeyKGN09c2VYOniWS3V2VKR6zcC+ditX/hLZXaKwpvlXH quacgPO1/VhdW7J5K2IDCBjD6cmGMx/izq963OLRqW7IE9K1vAcwMKpq+gZJm04BxTlB PNk2FsuzGNV3jJmmLydDaabLtkujfQwSl7BxSAIn+9/rd7NXa8XfE1SOOGCNurUyRUBK Nf9h/SrpsiCnWun99DAGzwI0MzECIQaw8L+j6Y0grNbCZjzxh2KOiD8hmaPj0jHPtIpI ht6ZCvD3RPj/KqLb/eRye28YNuGuNQQlVIJRhRJEcgu+LJEQmq9nee2OAsPByrsVfnZk 162w==; dara=google.com ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=amikan@ilbers.de Received: from shymkent.ilbers.de (shymkent.ilbers.de. [85.214.156.166]) by gmr-mx.google.com with ESMTPS id 2adb3069b0e04-52ca27c60e5si77722e87.0.2024.06.14.04.34.38 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 14 Jun 2024 04:34:38 -0700 (PDT) Received-SPF: pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) client-ip=85.214.156.166; Received: from localhost.localdomain (r154-240-38-77-broadband.btv.lv [77.38.240.154] (may be forged)) (authenticated bits=0) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPSA id 45EBYZP0032043 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 14 Jun 2024 13:34:37 +0200 From: Anton Mikanovich To: isar-users@googlegroups.com Cc: Anton Mikanovich Subject: [PATCH v9 3/8] meta: Allow selecting bootstrap providers Date: Fri, 14 Jun 2024 14:34:20 +0300 Message-Id: <20240614113425.122722-4-amikan@ilbers.de> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240614113425.122722-1-amikan@ilbers.de> References: <20240614113425.122722-1-amikan@ilbers.de> MIME-Version: 1.0 X-Spam-Status: No, score=-4.6 required=5.0 tests=DKIMWL_WL_MED,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_EF,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,RCVD_IN_MSPIKE_H2,RCVD_IN_RP_CERTIFIED, RCVD_IN_RP_RNBL,RCVD_IN_RP_SAFE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on shymkent.ilbers.de X-Original-Sender: amikan@ilbers.de X-Original-Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=amikan@ilbers.de Precedence: list Mailing-list: list isar-users@googlegroups.com; contact isar-users+owners@googlegroups.com List-ID: X-Spam-Checked-In-Group: isar-users@googlegroups.com X-Google-Group-Id: 914930254986 List-Post: , List-Help: , List-Archive: , List-Unsubscribe: , X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= Both isar-bootstrap or isar-mmdebstrap can be used for initial rootfs preparation. Signed-off-by: Anton Mikanovich --- meta-isar/conf/local.conf.sample | 4 ++++ meta/classes/rootfs.bbclass | 2 +- meta/conf/bitbake.conf | 3 +++ meta/recipes-core/isar-bootstrap/isar-bootstrap-host.bb | 2 ++ meta/recipes-core/isar-bootstrap/isar-bootstrap-target.bb | 2 ++ meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap-host.bb | 2 ++ meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap-target.bb | 2 ++ 7 files changed, 16 insertions(+), 1 deletion(-) diff --git a/meta-isar/conf/local.conf.sample b/meta-isar/conf/local.conf.sample index 17455015..a7862352 100644 --- a/meta-isar/conf/local.conf.sample +++ b/meta-isar/conf/local.conf.sample @@ -221,3 +221,7 @@ USER_isar[flags] += "clear-text-password" # To know more details about this variable and how to set the value refer below # https://reproducible-builds.org/docs/source-date-epoch/ #SOURCE_DATE_EPOCH = + +# Uncomment this to use old isar-bootstrap provider for rootfs prepare +#PREFERRED_PROVIDER_bootstrap-host ?= "isar-bootstrap-host" +#PREFERRED_PROVIDER_bootstrap-target ?= "isar-bootstrap-target" diff --git a/meta/classes/rootfs.bbclass b/meta/classes/rootfs.bbclass index 498fbfd6..f684e969 100644 --- a/meta/classes/rootfs.bbclass +++ b/meta/classes/rootfs.bbclass @@ -183,7 +183,7 @@ rootfs_install_pkgs_install() { do_rootfs_install[root_cleandirs] = "${ROOTFSDIR}" do_rootfs_install[vardeps] += "${ROOTFS_CONFIGURE_COMMAND} ${ROOTFS_INSTALL_COMMAND}" do_rootfs_install[vardepsexclude] += "IMAGE_ROOTFS" -do_rootfs_install[depends] = "isar-bootstrap-${@'target' if d.getVar('ROOTFS_ARCH') == d.getVar('DISTRO_ARCH') else 'host'}:do_build" +do_rootfs_install[depends] = "bootstrap-${@'target' if d.getVar('ROOTFS_ARCH') == d.getVar('DISTRO_ARCH') else 'host'}:do_build" do_rootfs_install[recrdeptask] = "do_deploy_deb" do_rootfs_install[network] = "${TASK_USE_SUDO}" python do_rootfs_install() { diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf index 4cfa8b10..faaeede3 100644 --- a/meta/conf/bitbake.conf +++ b/meta/conf/bitbake.conf @@ -193,3 +193,6 @@ include conf/machine/${MACHINE}.conf include conf/distro/${DISTRO}.conf PATCHRESOLVE ?= "noop" + +PREFERRED_PROVIDER_bootstrap-host ??= "isar-mmdebstrap-host" +PREFERRED_PROVIDER_bootstrap-target ??= "isar-mmdebstrap-target" diff --git a/meta/recipes-core/isar-bootstrap/isar-bootstrap-host.bb b/meta/recipes-core/isar-bootstrap/isar-bootstrap-host.bb index 4f90fd01..12798488 100644 --- a/meta/recipes-core/isar-bootstrap/isar-bootstrap-host.bb +++ b/meta/recipes-core/isar-bootstrap/isar-bootstrap-host.bb @@ -9,6 +9,8 @@ Description = "Minimal host Debian root file system" DEPLOY_ISAR_BOOTSTRAP = "${DEPLOY_DIR_BOOTSTRAP}/${HOST_DISTRO}-host_${DISTRO}-${DISTRO_ARCH}" +PROVIDES += "bootstrap-host" + BOOTSTRAP_FOR_HOST = "1" require isar-bootstrap.inc diff --git a/meta/recipes-core/isar-bootstrap/isar-bootstrap-target.bb b/meta/recipes-core/isar-bootstrap/isar-bootstrap-target.bb index c66cb3b3..69e97b33 100644 --- a/meta/recipes-core/isar-bootstrap/isar-bootstrap-target.bb +++ b/meta/recipes-core/isar-bootstrap/isar-bootstrap-target.bb @@ -9,4 +9,6 @@ Description = "Minimal target Debian root file system" DEPLOY_ISAR_BOOTSTRAP = "${DEPLOY_DIR_BOOTSTRAP}/${DISTRO}-${DISTRO_ARCH}" +PROVIDES += "bootstrap-target" + require isar-bootstrap.inc diff --git a/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap-host.bb b/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap-host.bb index 66c8d11e..6de9c21a 100644 --- a/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap-host.bb +++ b/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap-host.bb @@ -9,6 +9,8 @@ Description = "Minimal host Debian root file system" DEPLOY_ISAR_BOOTSTRAP = "${DEPLOY_DIR_BOOTSTRAP}/${HOST_DISTRO}-host_${DISTRO}-${DISTRO_ARCH}" +PROVIDES += "bootstrap-host" + BOOTSTRAP_FOR_HOST = "1" require isar-mmdebstrap.inc diff --git a/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap-target.bb b/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap-target.bb index 84a89ff1..227ff04c 100644 --- a/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap-target.bb +++ b/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap-target.bb @@ -9,4 +9,6 @@ Description = "Minimal target Debian root file system" DEPLOY_ISAR_BOOTSTRAP = "${DEPLOY_DIR_BOOTSTRAP}/${DISTRO}-${DISTRO_ARCH}" +PROVIDES += "bootstrap-target" + require isar-mmdebstrap.inc From patchwork Fri Jun 14 11:34:21 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anton Mikanovich X-Patchwork-Id: 3607 Return-Path: Received: from shymkent.ilbers.de ([unix socket]) by shymkent (Cyrus 2.5.10-Debian-2.5.10-3+deb9u2) with LMTPA; Fri, 14 Jun 2024 13:34:48 +0200 X-Sieve: CMU Sieve 2.4 Received: from mail-wm1-f58.google.com (mail-wm1-f58.google.com [209.85.128.58]) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPS id 45EBYl9X032122 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Fri, 14 Jun 2024 13:34:47 +0200 Received: by mail-wm1-f58.google.com with SMTP id 5b1f17b1804b1-421f43ddc2asf13421175e9.1 for ; Fri, 14 Jun 2024 04:34:47 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1718364881; cv=pass; d=google.com; s=arc-20160816; b=IUKU9soapHrOD/pIwR8u+gR5PAriNgtFvuMdVt4G2UND7xjebT/StGBdaCTzzSS8ys QdeATJX2id6c4MtzJnS4F8HTRTItlFmgHGbqT2QrA/CBRwQMsw0le55o0t0JDzmb6CmT xq1TVPGCadJqDY0w9po64BXy7JvzzEkuIE50wkACFiUt1//bC8tsF8RnwjwOOgagNe5u U+YZOIIND0LcftSA+c1dh69/txkKv8W9Q654SiDw4qDDsFupwxvHc15+Egkb/ZtKNfmd dxmvqAlzkTJ1TtTElo7nXLXY7UPULMfAcwGaATNnQlu6BoWEYGvZGmKd72gVNJGb+xET 5wrQ== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:dkim-signature; bh=rZ2DTS3TAhPBJtepyK/W1kYXyjhcvMQWjAlDSrtIFOs=; fh=d9Aup7oew3q6T9KuTA82Px22YJYRFKa6Ffzde7e3GcQ=; b=x1s5GEW9rMMVUBvYzyE3mbDiYRAhl3kqHQY07Xm5PVWY9QOKD92fjEQ3HlYsepumnA 8SZQOJ/1CAHOAT8d4h5jfb/omqfIujyV1VHLVXCXIAKmVPO9rFTVgZWKLO0+HuDIHHyp Xh7f1pCUdXj1GJUCtT0joDW2tl9yrXpG73hvkPAuQujA3APNEaZRdc311zcsYA+ydPp/ pGjTiyCtMtQYmyP8jGpue44qZB3boq3FIncu/DkdWp6Uvy3+8OzxMwZdlueZZuofVecV F6O55Z+dDy41coMZ0KGv+u+Y09dq3tzviLPRefGBd6a5XRq9CzGAIvnGX7w5i/8TNO9T ruTA==; darn=isar-build.org ARC-Authentication-Results: i=2; gmr-mx.google.com; spf=pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=amikan@ilbers.de DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlegroups.com; s=20230601; t=1718364881; x=1718969681; darn=isar-build.org; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:x-original-authentication-results :x-original-sender:mime-version:references:in-reply-to:message-id :date:subject:cc:to:from:sender:from:to:cc:subject:date:message-id :reply-to; bh=rZ2DTS3TAhPBJtepyK/W1kYXyjhcvMQWjAlDSrtIFOs=; b=sYt3+350wFczp6D1blABdifKvm5ThahEWYlZi2nsR5zdTY3s6G/X+s2/xscCM9B/Tg YWWLljGxqIsCPizLq80XYUBLjrHM41ygWp4+GkEGZLKeeJTUbM+ItlMLqCJ9L7k8PziQ 9ugETBL8jmfdw4y+NKiCmU9uSVA9QG7172vlMDGYuHkatHK/6ic8J+HAuCkn9yT2feFe B3vubA1KxjOTSlmwL8uvxDc1tQlIu3aIHqpGlb6rulu2/LB8RJ97UMY+3Nfjy0UjjGK0 qUfORMmSoW4z4KF0U7/8PSxiwrdJgLaNnF+Tw9YuipcRbRLuptSO48qnvzRRJSjogzWp JXaQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1718364881; x=1718969681; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :x-spam-checked-in-group:list-id:mailing-list:precedence :x-original-authentication-results:x-original-sender:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :x-beenthere:x-gm-message-state:sender:from:to:cc:subject:date :message-id:reply-to; bh=rZ2DTS3TAhPBJtepyK/W1kYXyjhcvMQWjAlDSrtIFOs=; b=D7fF4HM7PfRJ8xovzMQjpkQuBMKYD09Hj9FqlAn2Gii5DloVBDQLUwZtLHBI2Qn2Id SZTV4/top35vfeWsqte5I3BEC4sO47zOh6CBO9+hT57Lc3ZZm+QMFeEjYQPd1Uc42DL6 EURHWuXtueyTPD2YKucgkNnR+QVqwPWA1Yt4n1rQzV5vnpm8D1nAWol+f9qB95uz0IJ+ hjf7RDgTif/Q6DQSRPxDwR8NLIIo2uXzbH9dyB8cRkaboS6KsfQEjtL+uNmm4xhis9AM AX7O5sGmKX2Rer2+SSW6yCsKyScnWy49iaURMQ03d/ATAJdFwqOn3tKrMC7i0KROq+lK djqA== Sender: isar-users@googlegroups.com X-Forwarded-Encrypted: i=2; AJvYcCXULu0gQiuMk6aHC5cHHNUYeuI+uM/MFOk/9kfsi1+xKcQ63gQC4xKTMFYPPu4ggfoCy78Ocbzea/3QHdmbczoSDHlRvhU= X-Gm-Message-State: AOJu0YzCJQhQytrSZBndHUx9148e6Y+SVNUq7gwcia6zDPQuLq0GcW29 7Sa4KOXAWEQEjEKALhSKXEuqIskgvHn9HpzKJMdoaQkaXSTaTlOw X-Google-Smtp-Source: AGHT+IFI/5vhAerEi8y+L+24QuGeX3qCcO9tioLvZP0xBQRbreo1ygTy6MR3XHIwdj9k3RZhLiF+dA== X-Received: by 2002:a05:600c:1990:b0:422:7996:3f37 with SMTP id 5b1f17b1804b1-42304844e18mr25009175e9.26.1718364880604; Fri, 14 Jun 2024 04:34:40 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a05:600c:1d2a:b0:422:1589:d81c with SMTP id 5b1f17b1804b1-422b8da8296ls8115845e9.2.-pod-prod-03-eu; Fri, 14 Jun 2024 04:34:39 -0700 (PDT) X-Received: by 2002:a05:600c:9a3:b0:423:6957:89af with SMTP id 5b1f17b1804b1-42369578b0cmr4629575e9.29.1718364878922; Fri, 14 Jun 2024 04:34:38 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1718364878; cv=none; d=google.com; s=arc-20160816; b=rUXUp8zyAGKMilhI61nlzVw5gUrQwQ6XOPJBMW0Hl4x77prB8Qeb6g7TxX0f2/50fS wnAuYpMjaIZYFjELdct0HMKvzO9YRcGHn90kdDppcLRFyTMaeMhyEIyWo+hXLtUuaHpC Lt+b+Ettzoh3iZ/evOTjhZK1dvZV9u+Bv3OQ3Ry/cWo6+PMIR7y3hrHWY4XTVPxHm+4R kTiX6mQVBl79XnBa12l+knATIo52fE3fWhNyBteSQsrLBkT9Tl3q0xQsg7y3cNGSgNYv X+zYSVR+noR6UlbVOl4IPO2/cGhn/jAZjZBo0UTBBr+iqubMsQX8AAecyh/jWTXYMopu CPjQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=EiJvC1m1Zca/TZCK2jpyyP8VFQiVs7/H9VYteCrkijU=; fh=/h9QQkzJ8EboVkWg45aWwpaUro6WMavIVd2OhN45RtE=; b=eZvjeMFHUa8q4bBhqAp+iVFqieOQskpPqNi/XCs6VV1k7b5+mb1J83cDNZlU+IL4+Y nJD8pnd9l+nTkrb0YaaTuR6daAlB3cGjCcIl5nwLeo8soBWj35sMEa/VVqCWqc3PXG4i aMzP6TvEg0ff3u/FIvcgDX228bMdd7GuL4c16SuzCZ3B819eSIuPhWVcF54wI6hVnKKP j5PS7scxrl7Hsab/vhc20/smrz+o/DLAp5NrNZ6MvaMIr4XOtLl0p4YD/G97MSHDH3+W htDFE32PNOLhPzxCxQE8edouoGla3jdz5yeAa6GtsHjPQUT3sXQv61lOFtZSoUtfmz9x rhtw==; dara=google.com ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=amikan@ilbers.de Received: from shymkent.ilbers.de (shymkent.ilbers.de. [85.214.156.166]) by gmr-mx.google.com with ESMTPS id 5b1f17b1804b1-42285317d3bsi4224045e9.1.2024.06.14.04.34.38 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 14 Jun 2024 04:34:38 -0700 (PDT) Received-SPF: pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) client-ip=85.214.156.166; Received: from localhost.localdomain (r154-240-38-77-broadband.btv.lv [77.38.240.154] (may be forged)) (authenticated bits=0) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPSA id 45EBYZP1032043 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 14 Jun 2024 13:34:38 +0200 From: Anton Mikanovich To: isar-users@googlegroups.com Cc: Anton Mikanovich Subject: [PATCH v9 4/8] testsuite: Allow variable bootstrap providers Date: Fri, 14 Jun 2024 14:34:21 +0300 Message-Id: <20240614113425.122722-5-amikan@ilbers.de> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240614113425.122722-1-amikan@ilbers.de> References: <20240614113425.122722-1-amikan@ilbers.de> MIME-Version: 1.0 X-Spam-Status: No, score=-4.6 required=5.0 tests=DKIMWL_WL_MED,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_EF,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,RCVD_IN_MSPIKE_H2,RCVD_IN_RP_CERTIFIED, RCVD_IN_RP_RNBL,RCVD_IN_RP_SAFE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on shymkent.ilbers.de X-Original-Sender: amikan@ilbers.de X-Original-Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=amikan@ilbers.de Precedence: list Mailing-list: list isar-users@googlegroups.com; contact isar-users+owners@googlegroups.com List-ID: X-Spam-Checked-In-Group: isar-users@googlegroups.com X-Google-Group-Id: 914930254986 List-Post: , List-Help: , List-Archive: , List-Unsubscribe: , X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= If we can use different recipes provide bootstrap, recipe name should be obtained from target environment. Signed-off-by: Anton Mikanovich --- testsuite/cibase.py | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/testsuite/cibase.py b/testsuite/cibase.py index 0f6997af..c6117b36 100755 --- a/testsuite/cibase.py +++ b/testsuite/cibase.py @@ -179,8 +179,9 @@ class CIBaseTest(CIBuilder): # Rebuild image self.move_in_build_dir('tmp', 'tmp_before_sstate') self.bitbake(image_target, **kwargs) + bootstrap_target_task = self.getVars('PREFERRED_PROVIDER_bootstrap-target', target=image_target) if not all([ - check_executed_tasks('isar-bootstrap-target', + check_executed_tasks(bootstrap_target_task, ['do_bootstrap_setscene', '!do_bootstrap']), check_executed_tasks('sbuild-chroot-target', ['do_rootfs_install_setscene', '!do_rootfs_install']), @@ -204,7 +205,7 @@ class CIBaseTest(CIBuilder): self.move_in_build_dir('tmp', 'tmp_middle_sstate') self.bitbake(package_target, **kwargs) if not all([ - check_executed_tasks('isar-bootstrap-target', + check_executed_tasks(bootstrap_target_task, ['do_bootstrap_setscene']), check_executed_tasks('sbuild-chroot-target', ['!do_sbuildchroot_deploy']), @@ -218,7 +219,7 @@ class CIBaseTest(CIBuilder): process.run(f'find {self.build_dir}/sstate-cache/ -name sstate:hello:* -delete') self.bitbake(image_target, **kwargs) if not all([ - check_executed_tasks('isar-bootstrap-target', + check_executed_tasks(bootstrap_target_task, ['do_bootstrap_setscene', '!do_bootstrap']), check_executed_tasks('sbuild-chroot-target', ['do_rootfs_install_setscene', '!do_rootfs_install']), From patchwork Fri Jun 14 11:34:22 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anton Mikanovich X-Patchwork-Id: 3612 Return-Path: Received: from shymkent.ilbers.de ([unix socket]) by shymkent (Cyrus 2.5.10-Debian-2.5.10-3+deb9u2) with LMTPA; Fri, 14 Jun 2024 13:34:51 +0200 X-Sieve: CMU Sieve 2.4 Received: from mail-lj1-f187.google.com (mail-lj1-f187.google.com [209.85.208.187]) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPS id 45EBYoDf032156 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Fri, 14 Jun 2024 13:34:50 +0200 Received: by mail-lj1-f187.google.com with SMTP id 38308e7fff4ca-2ebd962eb88sf2257311fa.3 for ; Fri, 14 Jun 2024 04:34:50 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1718364882; cv=pass; d=google.com; s=arc-20160816; b=sk7mzaUA8ChQO8lixnce9N/sWjJJ89hrOJri0E2BNO0W+R7OUzlYnDere9gdCsx2Ll BM5vEcSIqZOJHE36l3+XXwTdvrbDn3p/E4eHKQC7Ab6c/jjGzRqH82q4noH/XTQygPtL jNAGHqccVx5n/P0iNUD2JXLvRCP6DBEwtDgKhTnrhvPeK2Fon3Z1h4rqPpyiRDkr09nm OjfosdkjV4eh3qErmGNl6iFL3BTrCfx4PUUSNfUU6pHFFH86o77wid8geKj/vRObRkeb rW7aksFxaD9kz42fMo/6qgeiUhtRuTn7bKqVdqSVPpWnGeXp6pJYZ27ca8nCvJBUi+zb Wm8w== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:dkim-signature; bh=dKBk9d0jGzcoB8yi61SgQTXWfj7gpVPEcCokmdM+Obo=; fh=XLaVIgjAG4L9gSPDwHmINZqGyKNelhk/UTUaMTnNESI=; b=AK+y4ZCkkH59n4aDGyfV911kpHFGAnLIuoRqB75H80GIqVNpgjtMOstzci16vyLpzO isRp8WEVTRyLzfKVhMTIbyu06p5GdlHBpSMN6G4qc0kBZda6OO9MT0SeSB4rc7c0mOtQ EBfH4T/2Z/J3Q0aptLYzjmwI3wRQt0wviNjMjzHcpRndXSsGi7UiugER8I5M5ZyZEAye UwqFciaCJOKnUPVNkDmrl7UhmjXaIKoWascb725EmIrh9JMLh0nGSg4kuv1yptsH3cug rSzLF/FBuW+WMCOP9q3cjzRHncp4FNOJUWdXPFczIeCnVkS4oyAa7CYcMqWEHtcYTn4K b1Ew==; darn=isar-build.org ARC-Authentication-Results: i=2; gmr-mx.google.com; spf=pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=amikan@ilbers.de DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlegroups.com; s=20230601; t=1718364882; x=1718969682; darn=isar-build.org; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:x-original-authentication-results :x-original-sender:mime-version:references:in-reply-to:message-id :date:subject:cc:to:from:sender:from:to:cc:subject:date:message-id :reply-to; bh=dKBk9d0jGzcoB8yi61SgQTXWfj7gpVPEcCokmdM+Obo=; b=HUrQJHaVpNJR9JTGfpquR3CHQbHTAMYz+Nkk5BFTLtz/aQID2Ttxo6rvpXAFWa1nh2 JbdJ/J8l4OS3fEJZKPqyijyNX5NwUoMBH50ml4Q7uRLf5H8VRAnpIZQkggj0lCqgacyl SVzbqi7oweTEizYq6Gh8WhhnUmW6iMwpTXgvnMLSgfwilDudkZTyC5siH9rNlwW9nzVx g60dm/wS6o9DB15In1WfqydPsjJGZ5AyDFDEhmMW3IrBo6RGrqGOikH+jBCIxEe5shvt If17yuz78YZI7CK5GqJgEKAK58hTVmO9oW1GIU7aysK4Zo3xFPZL1goeptNTMRUfBc4/ 13wA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1718364882; x=1718969682; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :x-spam-checked-in-group:list-id:mailing-list:precedence :x-original-authentication-results:x-original-sender:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :x-beenthere:x-gm-message-state:sender:from:to:cc:subject:date :message-id:reply-to; bh=dKBk9d0jGzcoB8yi61SgQTXWfj7gpVPEcCokmdM+Obo=; b=ppWVqwQthjotEFHzPdRMGshbhvOZgQHzTCipC/EYR1+pxj1toUQYEyfuR9u2envj2i KMznvSrbLwx2nJ6E24MHh2jNzjDXSN45vrd76LKhmVHPyYlTVzh+ah9SlQaPeDybvicq 6TNda/hWxaPRp4Vo33C1oLHYYFiDBu78HZFLr+32TGboGYWYx+xX1Kis6DQ65WD/1cqN 4f5w9dXUYwOaUUz3SDGaLuI0kOGNCO815p+7VWpvBCEuGsibJq8lF2mxYQDjM0STy1OR p4LkSSIaREydbwQ+ML+zr5ZT3t1/6llbIF9mPNkdG9Lcg+TFJPBqj3J56W7rFtWWXEM+ XI0A== Sender: isar-users@googlegroups.com X-Forwarded-Encrypted: i=2; AJvYcCXT5yGX2xoFVCAzB4At0/HN+TGLD6vJlM9QMKrM4etOiXfBwrnEqo73URPDGRcU/OiNNbeF8OFWrUnMuFZ77xpnBEoGKqI= X-Gm-Message-State: AOJu0YzyIeHeaRpJ0QQjrV2DT2dtU2FfwbR0dMtGxlL5cdHNtRlgPW7V qNRYePxtmJce4QDscGDSe4Nmj7O7sQxCy1Uy5eWh0B+zTa/E105+ X-Google-Smtp-Source: AGHT+IG6BlABYcID3KsjkZCvYkSmB6+UCyN/ULXfoIS/TnXclIgk7gcjHmUPaY7Y5o0lSz1xPqbwFg== X-Received: by 2002:ac2:5ec9:0:b0:52b:bde6:2dba with SMTP id 2adb3069b0e04-52ca6ec43dfmr1261349e87.6.1718364881176; Fri, 14 Jun 2024 04:34:41 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a05:600c:4506:b0:422:27b5:8477 with SMTP id 5b1f17b1804b1-422b8ca9598ls9563645e9.2.-pod-prod-07-eu; Fri, 14 Jun 2024 04:34:39 -0700 (PDT) X-Received: by 2002:a05:600c:5251:b0:421:c211:a57e with SMTP id 5b1f17b1804b1-4230485a5c6mr21828295e9.35.1718364879259; Fri, 14 Jun 2024 04:34:39 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1718364879; cv=none; d=google.com; s=arc-20160816; b=g9evURf8i7NsAQPza7um3qp9G6B8DhYcW1/Kmg4qxYjwbsIKS8rdKY3LTM40gW1Pgg 2iG8GbBrqosxnYRHC56LbGv3OauGnrvk+a/asR22MZm2tXEix4PxWltGOnBSnPjSUSPc ECIyMxDxXrVZle4yk5Zcl5s2RXAz9Syq56kPis+Xc1rdBVwjf00+qj0w1feubJ6KdEoA eIjZMJwRWC/xxQMajvrsybk3qfLTgETizZKzgNouKD1ZIqhGAA7VNnxDznOHz3W5VL4x zdU+9B/Pg5q7qSFinC5BqHjTjXsw+o0PdDLeBYyrWR/+NaXQB3uFYpejkBtydGxhvDuZ LlZw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=3Lz6cPDKcV5Rgf80zj6VnfUs8knvXQN69JfYYVHa8yY=; fh=/h9QQkzJ8EboVkWg45aWwpaUro6WMavIVd2OhN45RtE=; b=uxsNNaRsVwaqQF1mDWqzIADnR2jz3LpPnaXQqKg3fkFhS7W0OOLwDwYMy34YkKnabg hkaKC48AXFkI2RnGS0Mv0dqKI004pX63BEjXvKqlJkN6KfRyPfrvD8LbbgaX2tHmu1P1 IcoWXKRZcnQyxDx2AbladzLxlofo+NiHhctgCW/TRf+f+2g2BuWVkCSDrz6WzSpq5Qr0 EXnVLFQdQcnTaOz67uKpCqHSeFsW5pgtZVLlfxEUiosl4gS6wKyrbL39YN97+C4cbBcy 1MqeAjtWhR3p3NJAsuYkZXY3mmmXHmkrcRrldWxbgSWwd6RVDXf18M6pGYx3g0OGaVDb 4ppA==; dara=google.com ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=amikan@ilbers.de Received: from shymkent.ilbers.de (shymkent.ilbers.de. [85.214.156.166]) by gmr-mx.google.com with ESMTPS id 5b1f17b1804b1-422874def6bsi1185675e9.2.2024.06.14.04.34.39 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 14 Jun 2024 04:34:39 -0700 (PDT) Received-SPF: pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) client-ip=85.214.156.166; Received: from localhost.localdomain (r154-240-38-77-broadband.btv.lv [77.38.240.154] (may be forged)) (authenticated bits=0) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPSA id 45EBYZP2032043 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 14 Jun 2024 13:34:38 +0200 From: Anton Mikanovich To: isar-users@googlegroups.com Cc: Anton Mikanovich Subject: [PATCH v9 5/8] mmdebstrap: Fix missing dpkg available Date: Fri, 14 Jun 2024 14:34:22 +0300 Message-Id: <20240614113425.122722-6-amikan@ilbers.de> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240614113425.122722-1-amikan@ilbers.de> References: <20240614113425.122722-1-amikan@ilbers.de> MIME-Version: 1.0 X-Spam-Status: No, score=-4.6 required=5.0 tests=DKIMWL_WL_MED,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_EF,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,RCVD_IN_MSPIKE_H2,RCVD_IN_RP_CERTIFIED, RCVD_IN_RP_RNBL,RCVD_IN_RP_SAFE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on shymkent.ilbers.de X-Original-Sender: amikan@ilbers.de X-Original-Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=amikan@ilbers.de Precedence: list Mailing-list: list isar-users@googlegroups.com; contact isar-users+owners@googlegroups.com List-ID: X-Spam-Checked-In-Group: isar-users@googlegroups.com X-Google-Group-Id: 914930254986 List-Post: , List-Help: , List-Archive: , List-Unsubscribe: , X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= In some distros /var/lib/dpkg/available is required for do_rootfs_install task. The fix for it is present in jessie-or-older mmdebstrap hook in Bookworm and later, but maybe-jessie-or-older wrapper do not handle distro version check correctly. That's why we need to check and apply hook manually or create /var/lib/dpkg/available by hand for older distros. Signed-off-by: Anton Mikanovich --- meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc b/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc index 72096ae2..4f56a0b8 100644 --- a/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc +++ b/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc @@ -17,6 +17,14 @@ DISTRO_BOOTSTRAP_BASE_PACKAGES:append:gnupg = ",gnupg" DISTRO_BOOTSTRAP_BASE_PACKAGES:append:https-support = ",ca-certificates" BOOTSTRAP_TMPDIR = "${WORKDIR}/tempdir" +# Fix for /var/lib/apt/available while maybe-jessie-or-older hook do not work +MM_HOOK_JESSIE = "/usr/share/mmdebstrap/hooks/jessie-or-older" +DPKG_HOOKS ?= "${@'--hook-dir='+d.getVar('MM_HOOK_JESSIE') \ + if os.path.isdir(d.getVar('MM_HOOK_JESSIE')) \ + else '--customize-hook=\'touch /var/lib/dpkg/available\''}" +MMHOOKS:ubuntu-focal ?= "${DPKG_HOOKS}" +MMHOOKS:debian-buster ?= "${DPKG_HOOKS}" + def get_distro_primary_source_entry(d): for source in generate_distro_sources(d): if source[0] == "deb": @@ -98,6 +106,7 @@ do_bootstrap() { sudo TMPDIR="${BOOTSTRAP_TMPDIR}" mmdebstrap $bootstrap_args \ $arch_param \ --mode=unshare \ + ${MMHOOKS} \ ${@get_distro_components_argument(d)} \ "${@get_distro_suite(d)}" \ "${WORKDIR}/rootfs.tar.zst" \ From patchwork Fri Jun 14 11:34:23 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anton Mikanovich X-Patchwork-Id: 3610 Return-Path: Received: from shymkent.ilbers.de ([unix socket]) by shymkent (Cyrus 2.5.10-Debian-2.5.10-3+deb9u2) with LMTPA; Fri, 14 Jun 2024 13:34:49 +0200 X-Sieve: CMU Sieve 2.4 Received: from mail-wm1-f63.google.com (mail-wm1-f63.google.com [209.85.128.63]) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPS id 45EBYmmp032137 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Fri, 14 Jun 2024 13:34:48 +0200 Received: by mail-wm1-f63.google.com with SMTP id 5b1f17b1804b1-4212a3ddb6bsf15912775e9.2 for ; Fri, 14 Jun 2024 04:34:48 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1718364882; cv=pass; d=google.com; s=arc-20160816; b=H2wIf5FVu+KwDt6ZYkUSIIsKt36Gwr23sPoYDAalCHYF2Q3AXSMo5oIhzSgGklVcfl vL0bxo+MR/gjk64hG/p/EvoktR0yIfionRW0AokMUN3xWdJxTpmIyjfr8LHTEq1kV1AO 03LSx51iX5YwInbd9KKTQ03MQjhJKd0o87AZ9joNdAcBsHUqTiBZuyVnPL5jazWQI1Qj 9Uc9tPbDkWf4wbldQg3zP6XY6ymdr0JP2Jy0a44XoMgIkZXRXxx6Xq282gVFHOodxatH 2pnKoylfOjG6seEQfe7sYKPBYwVxo/hAqiJjm8ue4Fk5YxssLIS/8xrogoOGpPpMTDHW oVeA== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:dkim-signature; bh=8vvEUbUFGA6XVsQULMLEFG/7vZ+XvihGOeoVSFHnHq0=; fh=1Yp4qXrPggWv8NOh+yrLCMDPguY6FFHMDzdrdgz/pDw=; b=mId454jG+mcer02yxJsqfMVB5uO8sK/hKHeDSkFcRNKjdNgXXeIDylvttj96hPXxqe jOMRdXQDo80B+r04Hvwx6dBgEfe8FttHhnH/CroCmEBAfJO4bKZqKMOnmbr2L9TSWEJ4 Z9+/Y/ESww/r78Nif8aDFTkPefqAup/vAY6WceW+9zhGAOMfKUebstz5v81nwFkEu17f /cwLp5nRA2XH+tvQMP5NBWlXq09J9d0Q7PtEtQRKL9/HTSv3EvDrT6Vd/HQNBtncU8On oC8pcjmKsHwn4xDnBUeWjCeszc6BiYoWrc4AzH+GTkMqtIhfnmmzy4oXgNXIeEKlnEfQ ts/Q==; darn=isar-build.org ARC-Authentication-Results: i=2; gmr-mx.google.com; spf=pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=amikan@ilbers.de DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlegroups.com; s=20230601; t=1718364882; x=1718969682; darn=isar-build.org; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:x-original-authentication-results :x-original-sender:mime-version:references:in-reply-to:message-id :date:subject:cc:to:from:sender:from:to:cc:subject:date:message-id :reply-to; bh=8vvEUbUFGA6XVsQULMLEFG/7vZ+XvihGOeoVSFHnHq0=; b=er9dJv3Midk0WdVRMn+ILnG3M0ub0ZO7e/e7i0dIKRyjZ2vhgpxJGN/AlpAR4JcfzO 33rL/I68oftBAwQloDJzfctKVqPLZ/9269eSpDa7+SzRVFA3SytlJWuzz93F2DJIVETi llhuCob/l2HMrEdDCt3a0Q2Tsr4LAUwEHZoxNhG7iR+hyKOHu4fYPPghFmg3GnsBvl7i 7Rj1/NiOiveMLkaDMVZmq1hFdngza7X+Vwuf/BkvwzZCNP2ZQTte1qgYxhXB09xKWSew e2F0h1JGBorZfjN5IqRTTbv63Lyfh/ZkmCC8fMkGF47llydrQyafoa+J8y4qdlEyWvxA K5Rg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1718364882; x=1718969682; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :x-spam-checked-in-group:list-id:mailing-list:precedence :x-original-authentication-results:x-original-sender:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :x-beenthere:x-gm-message-state:sender:from:to:cc:subject:date :message-id:reply-to; bh=8vvEUbUFGA6XVsQULMLEFG/7vZ+XvihGOeoVSFHnHq0=; b=A4pCKM78hlv3mxIaQeUhIWM8Xxpl0cCRHKTvHPUtMUoIvsXmdUTdXhLU8tlUjedwXp sCKZUsOjNt+hzQgAecMpkVUO3seMD/6yGDg/QjcALsB3aqFvLar53mN0MTXTyIH/HMDC j72cd8RdpKFES4LJonxzOePNtIh7ot5IoKlkEP2AaBXwK7nb/q+a0llPz5bvO7f2y4QZ mWu7K7S6jrh68EcavLPolb4xFW3sRpGegj0l7wyo+M0gZCQegI7BTxgSmUHXxGuwfkw7 aY85az/SEQlF3RSHTELdyFeb1BZNdAVw/YCKTAx0VmplCGN6rRftJlJ9Iv1qMGA0N7Up JYBA== Sender: isar-users@googlegroups.com X-Forwarded-Encrypted: i=2; AJvYcCW42H6h31nXS8P5Hb8TU2r8TrX9W9asVL65f77puLHRJRD7RMPTYoY9hEcXMLD4H1sxwhA91TgoMSbJo4e2R34R2MTI07w= X-Gm-Message-State: AOJu0YyWwslrcDXsU8r0xlWeL5ETaOP2yp9TBzytsXEp1tN1RXgV2/Aj dGg36t+6RaqLyIO0/kPN2iq1aeK/Z/4jg8InKrsGNUvRD5oQ+4o8 X-Google-Smtp-Source: AGHT+IF4YqozdR4NlhbRfSURz6buDWS4FicJpHeVTYYDBalheOdgYe1kg0ottPW5rAki47SGLCWslg== X-Received: by 2002:a05:600c:15d3:b0:422:720a:1be9 with SMTP id 5b1f17b1804b1-4230482fba4mr21179455e9.20.1718364881718; Fri, 14 Jun 2024 04:34:41 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a05:600c:3111:b0:422:1589:d806 with SMTP id 5b1f17b1804b1-422b8ca6900ls9557875e9.1.-pod-prod-07-eu; Fri, 14 Jun 2024 04:34:40 -0700 (PDT) X-Received: by 2002:a05:600c:1d8e:b0:422:5443:96aa with SMTP id 5b1f17b1804b1-42304827573mr22323055e9.15.1718364879861; Fri, 14 Jun 2024 04:34:39 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1718364879; cv=none; d=google.com; s=arc-20160816; b=aTVbU+asSGypG1tK4T1kinTjgOq/xiTNTJySF4qDtCpLJHn2rbDuy4rBSC+2f6eQOr XSoWQuI3FBHi72kptbhVNal8sY50r7gWqBw1A7DC4uduyv9oMVN8fu5vqL+j5DtKV7iy +QgHtQlxODAZSi4becotydsLpGCr45MfrNec2WfCV7dnph4PetEsll+yim/EjRhuRDHZ wDH1PyFfci4Pc1zDyve8kosgL6gUEeULX6SWOPL2aiuBUK/S4YC4EvtTpR4j6Ua5MsMb QCU1iLeGpvSx+pXC6Y33SeF+QC6JXpi9QIT60VkDWOf+X38SXNvySmQOXvY5kQmtRX8m oXnA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=r26ZZDuAPq7EDxfs9w7PznlG/tFWE7XmeH0N52DJOWg=; fh=/h9QQkzJ8EboVkWg45aWwpaUro6WMavIVd2OhN45RtE=; b=bQuJdh5p+MzT+K30Uv1IzpzIncebdr5tqdt5jyfawf1tqVRw/FkL72e8Ti22WocCvT JOK4XlZHyplxJFQkrayncMIKG1Q1ktN8KR48Q0viBict5ECiUObfw/Ns4Pnuko+Rv0bH 8WkMzbkRFp3DGCqxMmpvZ4NuOze0YxIpArPz7bLYUoT7wFs/gNhTPgzWt5e9WeLupCRo zmttb96LWDhg+4wCXoJ33jpZfroo/ptsccZnnc4Gh8o/vHDkcKeMMroQjlOXfAWBD8S3 vUyoqxwdbyso9WETGjwGS6Ga1+JMi7COOkYqhNNPfB1R1MVFf/sHYtprHreVvEtylOio jZYg==; dara=google.com ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=amikan@ilbers.de Received: from shymkent.ilbers.de (shymkent.ilbers.de. [85.214.156.166]) by gmr-mx.google.com with ESMTPS id 5b1f17b1804b1-42284b0a6c9si3142325e9.0.2024.06.14.04.34.39 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 14 Jun 2024 04:34:39 -0700 (PDT) Received-SPF: pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) client-ip=85.214.156.166; Received: from localhost.localdomain (r154-240-38-77-broadband.btv.lv [77.38.240.154] (may be forged)) (authenticated bits=0) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPSA id 45EBYZP3032043 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 14 Jun 2024 13:34:39 +0200 From: Anton Mikanovich To: isar-users@googlegroups.com Cc: Anton Mikanovich Subject: [PATCH v9 6/8] mmdebstrap: Move preparations to hooks Date: Fri, 14 Jun 2024 14:34:23 +0300 Message-Id: <20240614113425.122722-7-amikan@ilbers.de> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240614113425.122722-1-amikan@ilbers.de> References: <20240614113425.122722-1-amikan@ilbers.de> MIME-Version: 1.0 X-Spam-Status: No, score=-4.6 required=5.0 tests=DKIMWL_WL_MED,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_EF,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,RCVD_IN_MSPIKE_H2,RCVD_IN_RP_CERTIFIED, RCVD_IN_RP_RNBL,RCVD_IN_RP_SAFE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on shymkent.ilbers.de X-Original-Sender: amikan@ilbers.de X-Original-Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=amikan@ilbers.de Precedence: list Mailing-list: list isar-users@googlegroups.com; contact isar-users+owners@googlegroups.com List-ID: X-Spam-Checked-In-Group: isar-users@googlegroups.com X-Google-Group-Id: 914930254986 List-Post: , List-Help: , List-Archive: , List-Unsubscribe: , X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= Move all bootstrap rootfs prepare logic into mmdebstrap hooks. Also migrate from apt-key to gpg. Signed-off-by: Anton Mikanovich --- .../isar-mmdebstrap/isar-mmdebstrap.inc | 237 +++++++++--------- 1 file changed, 113 insertions(+), 124 deletions(-) diff --git a/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc b/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc index 4f56a0b8..c41dd3a1 100644 --- a/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc +++ b/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc @@ -13,8 +13,6 @@ FILESEXTRAPATHS:append = ":${LAYERDIR_core}/recipes-core/isar-bootstrap/files" ROOTFSDIR = "${WORKDIR}/rootfs" DISTRO_BOOTSTRAP_BASE_PACKAGES = "locales,apt,usrmerge" -DISTRO_BOOTSTRAP_BASE_PACKAGES:append:gnupg = ",gnupg" -DISTRO_BOOTSTRAP_BASE_PACKAGES:append:https-support = ",ca-certificates" BOOTSTRAP_TMPDIR = "${WORKDIR}/tempdir" # Fix for /var/lib/apt/available while maybe-jessie-or-older hook do not work @@ -25,53 +23,29 @@ DPKG_HOOKS ?= "${@'--hook-dir='+d.getVar('MM_HOOK_JESSIE') \ MMHOOKS:ubuntu-focal ?= "${DPKG_HOOKS}" MMHOOKS:debian-buster ?= "${DPKG_HOOKS}" -def get_distro_primary_source_entry(d): - for source in generate_distro_sources(d): - if source[0] == "deb": - return source[2:] - bb.fatal('Invalid apt sources list') - -def get_distro_have_https_source(d): - return any(source[2].startswith("https://") for source in generate_distro_sources(d)) - -def get_distro_needs_https_support(d): - if get_distro_have_https_source(d): - return "https-support" - else: - return "" - -OVERRIDES:append = ":${@get_distro_needs_https_support(d)}" - -def get_distro_needs_gpg_support(d): - if d.getVar("DISTRO_BOOTSTRAP_KEYS") or \ - d.getVar("THIRD_PARTY_APT_KEYS") or \ - d.getVar("BASE_REPO_KEY"): - return "gnupg" - else: - return "" - -OVERRIDES:append = ":${@get_distro_needs_gpg_support(d)}" - -APT_KEYS_DIR = "${WORKDIR}/aptkeys" DISTRO_BOOTSTRAP_KEYRING = "${WORKDIR}/distro-keyring.gpg" -do_generate_keyrings[cleandirs] = "${APT_KEYS_DIR}" -do_generate_keyrings[dirs] = "${DL_DIR}" +do_generate_keyrings[cleandirs] = "${WORKDIR}/trusted.gpg.d" +do_generate_keyrings[dirs] = "${DEBDIR}" do_generate_keyrings[vardeps] += "DISTRO_BOOTSTRAP_KEYS THIRD_PARTY_APT_KEYS" do_generate_keyrings[network] = "${TASK_USE_SUDO}" do_generate_keyrings() { + export GNUPGHOME="$(mktemp -td gpghomeXXXXXXXXXX)" if [ -n "${@d.getVar("THIRD_PARTY_APT_KEYFILES") or ""}" ]; then - chmod 777 "${APT_KEYS_DIR}" for keyfile in ${@d.getVar("THIRD_PARTY_APT_KEYFILES")}; do - cp "$keyfile" "${APT_KEYS_DIR}"/"$(basename "$keyfile")" + gpg --no-autostart --keyring "gnupg-ring:${DISTRO_BOOTSTRAP_KEYRING}" \ + --no-default-keyring --import $keyfile done fi if [ -n "${@d.getVar("DISTRO_BOOTSTRAP_KEYFILES") or ""}" ]; then for keyfile in ${@d.getVar("DISTRO_BOOTSTRAP_KEYFILES")}; do - sudo apt-key --keyring "${DISTRO_BOOTSTRAP_KEYRING}" add $keyfile - cp "$keyfile" "${APT_KEYS_DIR}"/"$(basename "$keyfile")" + gpg --no-autostart --keyring "gnupg-ring:${DISTRO_BOOTSTRAP_KEYRING}" \ + --no-default-keyring --import $keyfile done fi + if [ -r "${DISTRO_BOOTSTRAP_KEYRING}" ]; then + chmod o+r "${DISTRO_BOOTSTRAP_KEYRING}" + fi } addtask generate_keyrings before do_build after do_unpack @@ -80,10 +54,12 @@ do_bootstrap[vardeps] += " \ ISAR_ENABLE_COMPAT_ARCH \ ${DISTRO_VARS_PREFIX}DISTRO_APT_SOURCES \ " -do_bootstrap[dirs] = "${DEPLOY_DIR_BOOTSTRAP} ${BOOTSTRAP_TMPDIR}" +do_bootstrap[dirs] = "${DEPLOY_DIR_BOOTSTRAP} ${BOOTSTRAP_TMPDIR} ${WORKDIR}/trusted.gpg.d ${WORKDIR}/sources.list.d" do_bootstrap[depends] = "base-apt:do_cache isar-apt:do_cache_config" do_bootstrap[network] = "${TASK_USE_NETWORK_AND_SUDO}" +DEB_DL_LOCK ?= "${DEBDIR}/${BOOTSTRAP_BASE_DISTRO}-${BASE_DISTRO_CODENAME}.lock" + do_bootstrap() { if [ "${ISAR_ENABLE_COMPAT_ARCH}" = "1" ]; then if [ -z "${COMPAT_DISTRO_ARCH}" ]; then @@ -93,121 +69,134 @@ do_bootstrap() { bootstrap_args="--verbose --variant=minbase --include=${DISTRO_BOOTSTRAP_BASE_PACKAGES}" if [ -f "${DISTRO_BOOTSTRAP_KEYRING}" ]; then bootstrap_args="$bootstrap_args --keyring=${DISTRO_BOOTSTRAP_KEYRING}" + cp "${DISTRO_BOOTSTRAP_KEYRING}" "${WORKDIR}/trusted.gpg.d/" fi E="${@ isar_export_proxies(d)}" - export BOOTSTRAP_FOR_HOST - deb_dl_dir_import "${ROOTFSDIR}" "${BOOTSTRAP_BASE_DISTRO}-${BASE_DISTRO_CODENAME}" sudo rm -rf --one-file-system "${ROOTFSDIR}" mkdir -p "${ROOTFSDIR}" + if [ "${ISAR_USE_CACHED_BASE_REPO}" = "1" ]; then + base_apt_tmp="$(mktemp -d /tmp/isar-base-aptXXXXXXXXXX)" + bootstrap_list="${WORKDIR}/sources.list.d/base-apt.list" + line="copy://$base_apt_tmp/${BOOTSTRAP_BASE_DISTRO} ${BASE_DISTRO_CODENAME} main" + if [ -z "${BASE_REPO_KEY}" ]; then + line="[trusted=yes] ${line}" + fi + echo "deb ${line}" > "${WORKDIR}/sources.list.d/base-apt.list" + line="copy://$base_apt_tmp/${BASE_DISTRO} ${BASE_DISTRO_CODENAME} main" + if [ -z "${BASE_REPO_KEY}" ]; then + line="[trusted=yes] ${line}" + fi + echo "deb-src ${line}" >> "${WORKDIR}/sources.list.d/base-apt.list" + + # no need to sync /var/cache/apt/archives if base-apt used + syncin='echo skip sync-in' + syncout='echo skip sync-out' + extra_setup="mount --bind '${REPO_BASE_DIR}' $base_apt_tmp" + extra_extract="$syncout" + # save mmdebstrap tempdir for cleanup + extra_essential="mkdir -p \$1/$base_apt_tmp && \ + echo \$1 > ${WORKDIR}/mmtmpdir && \ + mount --bind '${REPO_BASE_DIR}' \$1/$base_apt_tmp" + # replace base-apt mount in tmp with /base-apt mount + extra_customize="sed -i \"s|copy://$base_apt_tmp|file:///base-apt|g\" \ + \$1/etc/apt/sources.list.d/*.list && \ + mkdir -p \$1/base-apt && \ + mount --bind '${REPO_BASE_DIR}' \$1/base-apt && \ + chroot \$1 apt-get update -y \ + -o APT::Update::Error-Mode=any && \ + umount \$1/base-apt && \ + umount \$1/$base_apt_tmp && rm ${WORKDIR}/mmtmpdir && \ + umount $base_apt_tmp && rm -rf --one-file-system $base_apt_tmp" + else + deb_dl_dir_import "${WORKDIR}/dl_dir" "${BOOTSTRAP_BASE_DISTRO}-${BASE_DISTRO_CODENAME}" + + bootstrap_list="${WORKDIR}/sources.list.d/bootstrap.list" + install -v -m644 "${APTSRCS}" \ + "${WORKDIR}/sources.list.d/bootstrap.list" + + syncin='flock -s ${DEB_DL_LOCK} cp -n --no-preserve=owner \ + "${WORKDIR}/dl_dir/var/cache/apt/archives/"*.deb \ + "$1/var/cache/apt/archives/" || true' + syncout='flock -s ${DEB_DL_LOCK} cp -n --no-preserve=owner \ + "$1/var/cache/apt/archives/"*.deb \ + "${WORKDIR}/dl_dir/var/cache/apt/archives/"' + extra_setup="$syncin" + extra_extract="$syncout" + # prefetch apt debs because mmdebstrap will clean them on next stage + extra_essential='apt-get install apt -y -d \ + -o Dir::State="$1/var/lib/apt" \ + -o Dir::Etc="$1/etc/apt" \ + -o Dir::Cache="$1/var/cache/apt" \ + -o Apt::Architecture="${BOOTSTRAP_DISTRO_ARCH}"' + extra_essential="$extra_essential && $syncout" + extra_customize="$syncout" + fi + + if [ ! -z "${SOURCE_DATE_EPOCH}" ]; then + export SOURCE_DATE_EPOCH="${SOURCE_DATE_EPOCH}" + fi + arch_param="--arch=${BOOTSTRAP_DISTRO_ARCH},${DISTRO_ARCH}" + if [ "${ISAR_ENABLE_COMPAT_ARCH}" = "1" ]; then + arch_param="$arch_param,${COMPAT_DISTRO_ARCH}" + fi + + # Cleanup mounts if fails + trap 'exit 1' INT HUP QUIT TERM ALRM USR1 + trap '[ -r "${WORKDIR}/mmtmpdir" ] && tmpdir=$(cat "${WORKDIR}/mmtmpdir") \ + && rm "${WORKDIR}/mmtmpdir"; \ + [ -d "$tmpdir" ] && mountpoint -q $tmpdir/$base_apt_tmp \ + && sudo umount $tmpdir/$base_apt_tmp; \ + [ -d "$tmpdir" ] && mountpoint -q $tmpdir/base-apt \ + && sudo umount $tmpdir/base-apt; \ + [ -d "$tmpdir" ] && sudo rm -rf --one-file-system $tmpdir; \ + [ -n "$base_apt_tmp" ] && mountpoint -q $base_apt_tmp \ + && sudo umount $base_apt_tmp \ + && rm -rf --one-file-system $base_apt_tmp' EXIT sudo TMPDIR="${BOOTSTRAP_TMPDIR}" mmdebstrap $bootstrap_args \ $arch_param \ --mode=unshare \ ${MMHOOKS} \ + --setup-hook='mkdir -p "$1/var/cache/apt/archives/"' \ + --setup-hook="$extra_setup" \ + --setup-hook='upload "${APTPREFS}" /etc/apt/preferences.d/bootstrap' \ + --setup-hook='upload "${APTSRCS_INIT}" /etc/apt/sources-list' \ + --setup-hook='upload "${WORKDIR}/locale" /etc/locale' \ + --setup-hook='mkdir -p "$1/etc/apt/trusted.gpg.d"' \ + --setup-hook='sync-in "${WORKDIR}/trusted.gpg.d" /etc/apt/trusted.gpg.d' \ + --setup-hook='install -v -m755 "${WORKDIR}/chroot-setup.sh" "$1/chroot-setup.sh"' \ + --extract-hook="$extra_extract" \ + --essential-hook="$extra_essential" \ + --customize-hook="$extra_customize" \ + --customize-hook='sed -i "/en_US.UTF-8 UTF-8/s/^#//g" "$1/etc/locale.gen"' \ + --customize-hook='chroot "$1" /usr/sbin/locale-gen' \ + --customize-hook='chroot "$1" /usr/bin/apt-get -y clean' \ + --skip=cleanup/apt \ + --skip=download/empty \ ${@get_distro_components_argument(d)} \ "${@get_distro_suite(d)}" \ "${WORKDIR}/rootfs.tar.zst" \ - "${@get_distro_source(d)}" + "$bootstrap_list" sudo -E -s <<'EOSUDO' set -e tar -xf "${WORKDIR}/rootfs.tar.zst" -C "${ROOTFSDIR}" --exclude="./dev/console" - # Install apt config - mkdir -p "${ROOTFSDIR}/etc/apt/preferences.d" - install -v -m644 "${APTPREFS}" \ - "${ROOTFSDIR}/etc/apt/preferences.d/bootstrap" - mkdir -p "${ROOTFSDIR}/etc/apt/sources.list.d" - if [ "${ISAR_USE_CACHED_BASE_REPO}" = "1" ]; then - line="file:///base-apt/${BOOTSTRAP_BASE_DISTRO} ${BASE_DISTRO_CODENAME} main" - if [ -z "${BASE_REPO_KEY}" ]; then - line="[trusted=yes] ${line}" - fi - echo "deb ${line}" > "${ROOTFSDIR}/etc/apt/sources.list.d/base-apt.list" - line="file:///base-apt/${BASE_DISTRO} ${BASE_DISTRO_CODENAME} main" - if [ -z "${BASE_REPO_KEY}" ]; then - line="[trusted=yes] ${line}" - fi - echo "deb-src ${line}" >> "${ROOTFSDIR}/etc/apt/sources.list.d/base-apt.list" - - mkdir -p ${ROOTFSDIR}/base-apt - mount --bind ${REPO_BASE_DIR} ${ROOTFSDIR}/base-apt - else - install -v -m644 "${APTSRCS}" \ - "${ROOTFSDIR}/etc/apt/sources.list.d/bootstrap.list" - fi - install -v -m644 "${APTSRCS_INIT}" "${ROOTFSDIR}/etc/apt/sources-list" - rm -f "${ROOTFSDIR}/etc/apt/sources.list" - rm -rf "${ROOTFSDIR}/var/lib/apt/lists/"* - find ${APT_KEYS_DIR}/ -type f | while read keyfile - do - MY_GPGHOME="$(chroot "${ROOTFSDIR}" mktemp -d /tmp/gpghomeXXXXXXXXXX)" - echo "Created temporary directory ${MY_GPGHOME} for gpg-agent" - export GNUPGHOME="${MY_GPGHOME}" - APT_KEY_APPEND="--homedir ${MY_GPGHOME}" - - kfn="$(basename $keyfile)" - cp $keyfile "${ROOTFSDIR}/tmp/$kfn" - chroot "${ROOTFSDIR}" /usr/bin/gpg-agent --daemon -- /usr/bin/apt-key \ - --keyring ${THIRD_PARTY_APT_KEYRING} ${APT_KEY_APPEND} add "/tmp/$kfn" - rm "${ROOTFSDIR}/tmp/$kfn" - - echo "Removing ${MY_GPGHOME}" - rm -rf "${ROOTFSDIR}${MY_GPGHOME}" - done - - # Set locale - install -v -m644 "${WORKDIR}/locale" "${ROOTFSDIR}/etc/locale" - - sed -i '/en_US.UTF-8 UTF-8/s/^#//g' "${ROOTFSDIR}/etc/locale.gen" - chroot "${ROOTFSDIR}" /usr/sbin/locale-gen - # setup chroot install -v -m755 "${WORKDIR}/chroot-setup.sh" "${ROOTFSDIR}/chroot-setup.sh" "${ROOTFSDIR}/chroot-setup.sh" "setup" "${ROOTFSDIR}" - # update APT - mount -o bind,private /dev ${ROOTFSDIR}/dev - mount --bind /dev/pts ${ROOTFSDIR}/dev/pts - mount -t tmpfs none "${ROOTFSDIR}/dev/shm" - mount -t proc none ${ROOTFSDIR}/proc - mount --rbind /sys ${ROOTFSDIR}/sys - mount --make-rslave ${ROOTFSDIR}/sys - - export DEBIAN_FRONTEND=noninteractive - - if [ "${BOOTSTRAP_FOR_HOST}" = "1" ]; then - chroot "${ROOTFSDIR}" /usr/bin/dpkg --add-architecture ${DISTRO_ARCH} - fi - - if [ "${ISAR_ENABLE_COMPAT_ARCH}" = "1" ]; then - chroot "${ROOTFSDIR}" /usr/bin/dpkg --add-architecture ${COMPAT_DISTRO_ARCH} - fi - - chroot "${ROOTFSDIR}" /usr/bin/apt-get update -y \ - -o APT::Update::Error-Mode=any - chroot "${ROOTFSDIR}" /usr/bin/apt-get install -y -f - chroot "${ROOTFSDIR}" /usr/bin/apt-get dist-upgrade -y \ - -o Debug::pkgProblemResolver=yes - - umount -l "${ROOTFSDIR}/dev/shm" - umount -l "${ROOTFSDIR}/dev/pts" - umount -l "${ROOTFSDIR}/dev" - umount -l "${ROOTFSDIR}/proc" - umount -l "${ROOTFSDIR}/sys" - umount -l "${ROOTFSDIR}/base-apt" || true - # Finalize bootstrap by setting the link in deploy ln -Tfsr "${ROOTFSDIR}" "${DEPLOY_ISAR_BOOTSTRAP}" EOSUDO - deb_dl_dir_export "${ROOTFSDIR}" "${BOOTSTRAP_BASE_DISTRO}-${BASE_DISTRO_CODENAME}" - - # Cleanup apt cache - sudo -Es chroot "${ROOTFSDIR}" /usr/bin/apt-get -y clean + if [ "${ISAR_USE_CACHED_BASE_REPO}" != "1" ]; then + deb_dl_dir_export "${WORKDIR}/dl_dir" "${BOOTSTRAP_BASE_DISTRO}-${BASE_DISTRO_CODENAME}" + sudo rm -rf --one-file-system "${WORKDIR}/dl_dir" + fi } addtask bootstrap before do_build after do_generate_keyrings From patchwork Fri Jun 14 11:34:24 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anton Mikanovich X-Patchwork-Id: 3613 Return-Path: Received: from shymkent.ilbers.de ([unix socket]) by shymkent (Cyrus 2.5.10-Debian-2.5.10-3+deb9u2) with LMTPA; Fri, 14 Jun 2024 13:47:17 +0200 X-Sieve: CMU Sieve 2.4 Received: from mail-lj1-f186.google.com (mail-lj1-f186.google.com [209.85.208.186]) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPS id 45EBlGFB032649 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Fri, 14 Jun 2024 13:47:17 +0200 Received: by mail-lj1-f186.google.com with SMTP id 38308e7fff4ca-2ebd9640ac2sf10512221fa.0 for ; Fri, 14 Jun 2024 04:47:17 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1718365631; cv=pass; d=google.com; s=arc-20160816; b=EVWxNDjZMX5j+puLjSBrWWcFe8H1uXguS416FZ6FFOWjl34PhFQhoxf0c09Z5jW/LJ Y9hzswKvur1M8ml+8Npz3Kwv2txC2rHlmH6bxzIrdnzEGXdMZ/w5b6zeVddgg7nahfqG Edg0u+IP0hs6V7NhdpXDONt+qhEtRoOl/mGMwq3LFPFM7/RI+172KJ/aanCb2kzf3pfa QlIBUFEtT4bP2C/aNI86FiVlVwRHNH7IzgyrLmGZTpjPk7NBTqAkDPt/v18l1dtflm5n IR5TEpKC2y/hGlVcmB4NKjhzsdOWxD+rL5jVq1WLCrJGh79X1UdzjhA4XyxiiMu/XyVl k/dw== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:dkim-signature; bh=mMATEjfZkMM54oyL95pFiHpX1HLTCHd2f9V+PfqCy1c=; fh=Xo4pE8Y7042oYt+0lxZCFWxN9PbcoW6ZNKotDDMLNd4=; b=WtqKAWUrI0QAYEIqqmAcuzWFneIElV27ODeI35uGHJQo7Patn6ZooJqkKaZgBMdkdO arlwzf9XPMRcMpCG8mKVOCw4O6uTwx6+K4vuPQ/bNPMer2ZspRJTROHNadrxKPZwfoGo +br/7LPtsvxz6jhacmXIasJKID92PpFrDnuaZdaTToPSyw2Tql/QtJoBnQOPBQyoC7i5 DDQR0Dm0vUncVyMKFzHsD1IA1iaI3vRe2xi1+XQ5cLYXryE8RXz1CdF5tgQ151Ach3hU ruPb9EXDTOLlcYN1TstiaNGa7QEXUsxwViDHWFz75zPMoxHUydsQwoZWiB5ygOIUaf0z vYjg==; darn=isar-build.org ARC-Authentication-Results: i=2; gmr-mx.google.com; spf=pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=amikan@ilbers.de DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlegroups.com; s=20230601; t=1718365631; x=1718970431; darn=isar-build.org; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:x-original-authentication-results :x-original-sender:mime-version:references:in-reply-to:message-id :date:subject:cc:to:from:sender:from:to:cc:subject:date:message-id :reply-to; bh=mMATEjfZkMM54oyL95pFiHpX1HLTCHd2f9V+PfqCy1c=; b=jGes1K1QuFuh/esNSGi/D5DGpnygPoNMyrEWZEFPmnDubIQFj3BVw/d439onjm5dmN 0ZNvnTsOAFcmTiUfN498PYQp4eOrpXpo/D1C9c+u54pj075tWFCmNXya2zYAkAkqcbvf /X85IolYNSXQ8bwejdft1Be5pSK8bHG60xAWvA4xq9ZKdUL8h9ZzboUKNNxWuCXXUpyD HJ9h0ohlpBCb2mp9FYQ+N+6ZZ4UlvxHdcfYeRKXrEONl2NYIMLqpnqDLYOoi5X1heJst OGzr2P1Jul6urfDn0rtQZFX83ywosul7HH+UWN73qS8anYF89avpD0bEAkKFxP8miq8p 8ZXg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1718365631; x=1718970431; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :x-spam-checked-in-group:list-id:mailing-list:precedence :x-original-authentication-results:x-original-sender:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :x-beenthere:x-gm-message-state:sender:from:to:cc:subject:date :message-id:reply-to; bh=mMATEjfZkMM54oyL95pFiHpX1HLTCHd2f9V+PfqCy1c=; b=oC3XF3u/56/mEKfmlSdeVZ/JYxXprlpFm9TjZvxWJ0qb2r3YQNCJ6YvpXC4SfmyVHy oNIdz10ybTAQCFKNgjmFZNINtxTQ2y4LdZ+q1vxObIVPBJlZ8O9NXyPKRidv5YIi83H+ JsA8ISpHfXLBdrd4rQy8z/TMD3ft6ku+5zJCcqB4+tktqzOfwX+b7TERcQvvK4mhALue wPjilGC2t3/FF/P+aXAKPQhKAs/HxCmryAvloMJaNsqRe06vdnpIE2ZXVoLdcsum3GmL ith0/539tGd4KJ8uUXylaNIgY2mAuQWwAOczdWUewuoMLZMBkyXahzk6fHqZVMq1zEZ+ NtJg== Sender: isar-users@googlegroups.com X-Forwarded-Encrypted: i=2; AJvYcCVDx9XbBrkgwd8MvdLabvG1cO2PGUBRHCzy5yRdIEufqJ3n0E8IKfQdW+xnmI0OJB3p/EUkssQNOTqSKoSe/q6OCHheyg8= X-Gm-Message-State: AOJu0YzEfJvAA/mqwFoOms1WdYxKSRBZJe80EUFoG5eeULq89G0CiCr9 RBwIlwpuwvSHH6kVG3dBW+Q3oubGP+w3hbd73KrcMHqTm/lkQCJo X-Google-Smtp-Source: AGHT+IHjRnBgesVUThfuHppu/GyVmW/If7NrOZYMbwDQo91/ImE13fxFAiWS3yFjQnbLaQPYx6uREA== X-Received: by 2002:a05:600c:5110:b0:421:7f07:92ce with SMTP id 5b1f17b1804b1-4230481bc6fmr25940395e9.3.1718364882034; Fri, 14 Jun 2024 04:34:42 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a1c:7708:0:b0:423:7d5:b537 with SMTP id 5b1f17b1804b1-42307d5b943ls1496685e9.0.-pod-prod-00-eu; Fri, 14 Jun 2024 04:34:40 -0700 (PDT) X-Received: by 2002:a05:600c:5492:b0:421:83fd:900f with SMTP id 5b1f17b1804b1-42304820ed0mr26647495e9.5.1718364880279; Fri, 14 Jun 2024 04:34:40 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1718364880; cv=none; d=google.com; s=arc-20160816; b=LmktkFzfQjluzWDOjL+ozUmyVE04C61RvXABg56nq88fc3v6HS2emauNAG2O97/A0l eA8ee57ffJb6SaArFWIlLaOVKuYCwdPKqIc6B5pviofQu8PuHibi4q1aBdin+TM3aUCY gaGIVCVf7SaKKjfcn2TQecTjNK7Js/1YSI0TiNT3vZ8KqVsfjSlXwCD6xQTxZy7hak2C WRKEwPEV5nXR2Zlz+omKwJaVXgyHj9+2X+/UnQff35gL9tU0PaQO9l1svGBnaiVXbAWY DAq5zun2exp8uCRhd1hz0jyYkkGprakR8AgBUy0X6BYf2OeDAKih13w2P8Bu6LsHuVDO Dujw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=vGr5siGIWeRHy+AKP7KSRi/M6TuppzFcwnyS/KQFw98=; fh=/h9QQkzJ8EboVkWg45aWwpaUro6WMavIVd2OhN45RtE=; b=eZCw7/pacLkyrliMoJUnICzl5nNx1vkoklzZ8VcmShOSHiwGMUZHtxZ34cfrPnvVNh hDV3G+LfgnOg/xZ582sxkkyx5Tr7p8lSH80qzdOCGrvt/zQjICZRcA745+e2ORSAwPrW uCnRKSdVBCJBSXpj/nG/31Il6ojG6ifKir9goE77Bh9QuG7Qzj0hpHKMbvvnte/Lezhk KgBYnIlwxosji0F9GN1O/e2msG33advBa/TEx/e2DXsZ3Xkvs3PvUtT8VSRND8XN7Nx1 z6mDRkLbSUPrDhyiEKBnjRHXEvmTGmhfInQzXpGNdFffsf6ZaEMJFnfds3gzMs9R551L M4iA==; dara=google.com ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=amikan@ilbers.de Received: from shymkent.ilbers.de (shymkent.ilbers.de. [85.214.156.166]) by gmr-mx.google.com with ESMTPS id 5b1f17b1804b1-42287f0b05bsi3013645e9.1.2024.06.14.04.34.40 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 14 Jun 2024 04:34:40 -0700 (PDT) Received-SPF: pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) client-ip=85.214.156.166; Received: from localhost.localdomain (r154-240-38-77-broadband.btv.lv [77.38.240.154] (may be forged)) (authenticated bits=0) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPSA id 45EBYZP4032043 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 14 Jun 2024 13:34:39 +0200 From: Anton Mikanovich To: isar-users@googlegroups.com Cc: Anton Mikanovich Subject: [PATCH v9 7/8] isar-bootstrap: Use tar output instead of directory Date: Fri, 14 Jun 2024 14:34:24 +0300 Message-Id: <20240614113425.122722-8-amikan@ilbers.de> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240614113425.122722-1-amikan@ilbers.de> References: <20240614113425.122722-1-amikan@ilbers.de> MIME-Version: 1.0 X-Spam-Status: No, score=-4.6 required=5.0 tests=DKIMWL_WL_MED,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_EF,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,RCVD_IN_MSPIKE_H2,RCVD_IN_RP_CERTIFIED, RCVD_IN_RP_RNBL,RCVD_IN_RP_SAFE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on shymkent.ilbers.de X-Original-Sender: amikan@ilbers.de X-Original-Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=amikan@ilbers.de Precedence: list Mailing-list: list isar-users@googlegroups.com; contact isar-users+owners@googlegroups.com List-ID: X-Spam-Checked-In-Group: isar-users@googlegroups.com X-Google-Group-Id: 914930254986 List-Post: , List-Help: , List-Archive: , List-Unsubscribe: , X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= Do not unpack tar with rootfs content inside bootstrap recipe. This change can allow to remove sudo usage during the bootstrap later. Using tar output requires moving chroot prepare step to the later unpack step. Signed-off-by: Anton Mikanovich --- meta/classes/rootfs.bbclass | 11 +++++---- .../isar-bootstrap/isar-bootstrap.inc | 22 +++++++++--------- .../isar-mmdebstrap/isar-mmdebstrap.inc | 23 ++++--------------- 3 files changed, 23 insertions(+), 33 deletions(-) diff --git a/meta/classes/rootfs.bbclass b/meta/classes/rootfs.bbclass index f684e969..a479b37a 100644 --- a/meta/classes/rootfs.bbclass +++ b/meta/classes/rootfs.bbclass @@ -73,12 +73,15 @@ rootfs_do_qemu() { fi } -BOOTSTRAP_SRC = "${DEPLOY_DIR_BOOTSTRAP}/${ROOTFS_DISTRO}-host_${DISTRO}-${DISTRO_ARCH}" -BOOTSTRAP_SRC:${ROOTFS_ARCH} = "${DEPLOY_DIR_BOOTSTRAP}/${ROOTFS_DISTRO}-${ROOTFS_ARCH}" +BOOTSTRAP_SRC = "${DEPLOY_DIR_BOOTSTRAP}/${ROOTFS_DISTRO}-host_${DISTRO}-${DISTRO_ARCH}.tar.zst" +BOOTSTRAP_SRC:${ROOTFS_ARCH} = "${DEPLOY_DIR_BOOTSTRAP}/${ROOTFS_DISTRO}-${ROOTFS_ARCH}.tar.zst" rootfs_prepare[weight] = "25" rootfs_prepare(){ - sudo cp -Trpfx --reflink=auto '${BOOTSTRAP_SRC}/' '${ROOTFSDIR}' + sudo tar -xf "${BOOTSTRAP_SRC}" -C "${ROOTFSDIR}" --exclude="./dev/console" + + # setup chroot + sudo "${ROOTFSDIR}/chroot-setup.sh" "setup" "${ROOTFSDIR}" } ROOTFS_CONFIGURE_COMMAND += "rootfs_configure_isar_apt" @@ -227,7 +230,7 @@ cache_deb_src() { # Note: ISAR updates the apt state information(apt-get update) only once during bootstrap and # relies on that through out the build. Copy that state information instead of apt-get update # which generates a new state from upstream. - sudo cp -Trpn --reflink=auto "${BOOTSTRAP_SRC}/var/lib/apt/lists/" "${ROOTFSDIR}/var/lib/apt/lists/" + sudo tar -xf "${BOOTSTRAP_SRC}" ./var/lib/apt/lists --one-top-level="${ROOTFSDIR}" deb_dl_dir_import ${ROOTFSDIR} ${ROOTFS_BASE_DISTRO}-${BASE_DISTRO_CODENAME} debsrc_download ${ROOTFSDIR} ${ROOTFS_BASE_DISTRO}-${BASE_DISTRO_CODENAME} diff --git a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc index 5c9eb5c8..efec42c6 100644 --- a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc +++ b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc @@ -162,7 +162,6 @@ do_bootstrap() { # setup chroot install -v -m755 "${WORKDIR}/chroot-setup.sh" "${ROOTFSDIR}/chroot-setup.sh" - "${ROOTFSDIR}/chroot-setup.sh" "setup" "${ROOTFSDIR}" # update APT mount -o bind,private /dev ${ROOTFSDIR}/dev @@ -195,8 +194,11 @@ do_bootstrap() { umount -l "${ROOTFSDIR}/sys" umount -l "${ROOTFSDIR}/base-apt" || true + # Compress rootfs for compatibility + tar --zstd -cf "${WORKDIR}/rootfs.tar.zst" -C "${ROOTFSDIR}" . + # Finalize debootstrap by setting the link in deploy - ln -Tfsr "${ROOTFSDIR}" "${DEPLOY_ISAR_BOOTSTRAP}" + ln -Tfsr "${WORKDIR}/rootfs.tar.zst" "${DEPLOY_ISAR_BOOTSTRAP}.tar.zst" EOSUDO deb_dl_dir_export "${ROOTFSDIR}" "${BOOTSTRAP_BASE_DISTRO}-${BASE_DISTRO_CODENAME}" @@ -212,19 +214,17 @@ SSTATEPOSTINSTFUNCS += "bootstrap_sstate_finalize" bootstrap_sstate_prepare() { # this runs in SSTATE_BUILDDIR, which will be deleted automatically - lopts="--one-file-system --exclude=var/cache/apt/archives" - sudo tar -C $(dirname "${ROOTFSDIR}") -cpSf bootstrap.tar $lopts $(basename "${ROOTFSDIR}") - sudo chown $(id -u):$(id -g) bootstrap.tar + sudo cp -a "${WORKDIR}/rootfs.tar.zst" ./bootstrap.tar.zst + sudo chown $(id -u):$(id -g) bootstrap.tar.zst } bootstrap_sstate_finalize() { # this runs in SSTATE_INSTDIR - # - after building the bootstrap, the tar won't be there, but we also don't need to unpack - # - after restoring from cache, there will be a tar which we unpack and then delete - if [ -f bootstrap.tar ]; then - sudo tar -C $(dirname "${ROOTFSDIR}") -xpf bootstrap.tar - sudo ln -Tfsr "${ROOTFSDIR}" "${DEPLOY_ISAR_BOOTSTRAP}" - rm bootstrap.tar + # we should restore symlinks after using tar + if [ -f bootstrap.tar.zst ]; then + mv bootstrap.tar.zst "${WORKDIR}/rootfs.tar.zst" + sudo ln -Tfsr "${WORKDIR}/rootfs.tar.zst" \ + "${DEPLOY_ISAR_BOOTSTRAP}.tar.zst" fi } diff --git a/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc b/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc index c41dd3a1..f1ad891b 100644 --- a/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc +++ b/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc @@ -11,7 +11,6 @@ inherit deb-dl-dir FILESEXTRAPATHS:append = ":${LAYERDIR_core}/recipes-core/isar-bootstrap/files" -ROOTFSDIR = "${WORKDIR}/rootfs" DISTRO_BOOTSTRAP_BASE_PACKAGES = "locales,apt,usrmerge" BOOTSTRAP_TMPDIR = "${WORKDIR}/tempdir" @@ -73,9 +72,6 @@ do_bootstrap() { fi E="${@ isar_export_proxies(d)}" - sudo rm -rf --one-file-system "${ROOTFSDIR}" - mkdir -p "${ROOTFSDIR}" - if [ "${ISAR_USE_CACHED_BASE_REPO}" = "1" ]; then base_apt_tmp="$(mktemp -d /tmp/isar-base-aptXXXXXXXXXX)" bootstrap_list="${WORKDIR}/sources.list.d/base-apt.list" @@ -181,18 +177,9 @@ do_bootstrap() { "${WORKDIR}/rootfs.tar.zst" \ "$bootstrap_list" - sudo -E -s <<'EOSUDO' - set -e - - tar -xf "${WORKDIR}/rootfs.tar.zst" -C "${ROOTFSDIR}" --exclude="./dev/console" - - # setup chroot - install -v -m755 "${WORKDIR}/chroot-setup.sh" "${ROOTFSDIR}/chroot-setup.sh" - "${ROOTFSDIR}/chroot-setup.sh" "setup" "${ROOTFSDIR}" + # Finalize bootstrap by setting the link in deploy + sudo ln -Tfsr "${WORKDIR}/rootfs.tar.zst" "${DEPLOY_ISAR_BOOTSTRAP}.tar.zst" - # Finalize bootstrap by setting the link in deploy - ln -Tfsr "${ROOTFSDIR}" "${DEPLOY_ISAR_BOOTSTRAP}" -EOSUDO if [ "${ISAR_USE_CACHED_BASE_REPO}" != "1" ]; then deb_dl_dir_export "${WORKDIR}/dl_dir" "${BOOTSTRAP_BASE_DISTRO}-${BASE_DISTRO_CODENAME}" sudo rm -rf --one-file-system "${WORKDIR}/dl_dir" @@ -206,7 +193,7 @@ SSTATEPOSTINSTFUNCS += "bootstrap_sstate_finalize" bootstrap_sstate_prepare() { # this runs in SSTATE_BUILDDIR, which will be deleted automatically - sudo cp -a "$(dirname "${ROOTFSDIR}")/rootfs.tar.zst" ./bootstrap.tar.zst + sudo cp -a "${WORKDIR}/rootfs.tar.zst" ./bootstrap.tar.zst sudo chown $(id -u):$(id -g) bootstrap.tar.zst } @@ -214,8 +201,8 @@ bootstrap_sstate_finalize() { # this runs in SSTATE_INSTDIR # we should restore symlinks after using tar if [ -f bootstrap.tar.zst ]; then - mv bootstrap.tar.zst "$(dirname "${ROOTFSDIR}")/rootfs.tar.zst" - sudo ln -Tfsr "$(dirname "${ROOTFSDIR}")/rootfs.tar.zst" \ + mv bootstrap.tar.zst "${WORKDIR}/rootfs.tar.zst" + sudo ln -Tfsr "${WORKDIR}/rootfs.tar.zst" \ "${DEPLOY_ISAR_BOOTSTRAP}.tar.zst" fi } From patchwork Fri Jun 14 11:34:25 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anton Mikanovich X-Patchwork-Id: 3611 Return-Path: Received: from shymkent.ilbers.de ([unix socket]) by shymkent (Cyrus 2.5.10-Debian-2.5.10-3+deb9u2) with LMTPA; Fri, 14 Jun 2024 13:34:50 +0200 X-Sieve: CMU Sieve 2.4 Received: from mail-lj1-f184.google.com (mail-lj1-f184.google.com [209.85.208.184]) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPS id 45EBYnV7032143 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Fri, 14 Jun 2024 13:34:49 +0200 Received: by mail-lj1-f184.google.com with SMTP id 38308e7fff4ca-2ec063ce179sf15969551fa.0 for ; Fri, 14 Jun 2024 04:34:49 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1718364884; cv=pass; d=google.com; s=arc-20160816; b=VLfL36BoNeM/fE2xSgHA7Mn66I2RlLrGuwsR2oie0X5WYscOysOo3ANnPP3bxFBetj p9zZtaeMpCxNhjrtGynF9qy5CDnkLiWJ8z1kV8vFx83+PoJ1kI8B/OgCWWePyrtObMBJ Fyyq7Q0krQNRdgEQUCk3Fo2gnLzTPtSIguKJ093o42MJnj0LSUa2B8GLo9EV48Qlgswm Xriqhr7X5ohKUXi4GBjRPvc9rjqbH+UmAd9QoD3VyknUb8QsUmaTc6GmnHNv3piSZ9OG e3O71cFf+JdTcJoOHc+/1yXYgMrvrTHbrPgZLls4f66AeNxOBMMWfKMrOvqzFIrDbkhd O9Ng== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:dkim-signature; bh=LXyjU/oqM2vxjCPnzHFGPYeBNqdErYN+2oRZsnozntE=; fh=hbhuJPGHkoTBR0pitoqu2FWCNzqwtznmONu2A2wYhFQ=; b=jlEW+64tenBxMAAdnHdU3hru6gfiWlTzOZfp7ZlO3bn/UdOgm++/j2yDdTHrPdirLM sWm/lmgflzjx+sJJ3DN21NctzBVkmTb+NGCz20IrE+v/N1qW+cfSb89P1SVSGXHw0gqt GjYq2A/JS6q8zm0xKbuxMs61+tLa6NcbBT7/djmZ4vccVUPT+3HAbVqreJYC4pDDqyKB bQJPTYLCyamMRpAGsAGbg7lp+fmCbGKx5hGvmbHJc7A2Uh5RGQO3aRSosp02qDhPFIss AurZLM5ohgcWdbeiYGKaN6mX3CXia4iYiCYd4y9lkEn8nE7n0RQzBKDn46Ly4Yq0JuG4 tQug==; darn=isar-build.org ARC-Authentication-Results: i=2; gmr-mx.google.com; spf=pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=amikan@ilbers.de DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlegroups.com; s=20230601; t=1718364884; x=1718969684; darn=isar-build.org; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:x-original-authentication-results :x-original-sender:mime-version:references:in-reply-to:message-id :date:subject:cc:to:from:sender:from:to:cc:subject:date:message-id :reply-to; bh=LXyjU/oqM2vxjCPnzHFGPYeBNqdErYN+2oRZsnozntE=; b=Dbe/QrfxeoFYP93+K4LPR/ZwlU4cdd668AH/Aqhj9yOnL1iTR44QGNVWilEzTEvTVy 6xuZpto5uxJ29ou4GcAvFlLJsy+G2EUWZ6fovCFzdmKS4Nnje3ad8tRsyhYhPVx+BfdV /E0vcbVkd6/P/n0kNmkEqtbMJSMaIwrRsJC5pdmkH2SB1BJOl5la5NUa8S2HEmInloRt DGkrh/4TWnaRhM18aZgjpIVsZMtJR4IaKX9OopGy8Vvc/ghohs8BEgDPCnFm4vBdtCU/ 6ympmQhVgX8gFc/bfx3oOa37K+OIh94jVlea2e+ArFNgB7Alc7rqe7RDUbL2Gtc1HtCA dLOg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1718364884; x=1718969684; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :x-spam-checked-in-group:list-id:mailing-list:precedence :x-original-authentication-results:x-original-sender:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :x-beenthere:x-gm-message-state:sender:from:to:cc:subject:date :message-id:reply-to; bh=LXyjU/oqM2vxjCPnzHFGPYeBNqdErYN+2oRZsnozntE=; b=P778sBGmfPnNwiFdc4eh0+5JCHeL4IitFQacG/ZQllGP/lHiu45hNeghE9OmAKwgZZ h+fCQ1IGnGt/EYFo3F2xkuv4gEqmRBJInssD96fmcUe8oot9kI0iCLx79O6l1Ld6iXuh dbaj2GQnkfNro/g830ZerxPxNRYUvIHH6W03RCSVrwzn2UzdQI4su75LDi08gIEvwf7l D5mO1/XLbXBjDEsjSg1K0tWJcMMwp6/Zy90Zdmsc33MrwcgyDNb/u3f1CMjO0VyWalhz OhBsEjj4xwZkTPUMs0X07IzeDY1UhVuibldu4w8dk59vsjv4Xiz/bgak5Uey5M1dI2+p Y/gQ== Sender: isar-users@googlegroups.com X-Forwarded-Encrypted: i=2; AJvYcCXKgPhNMHASsn8KR8JEjaSXbPKtLFiYg4tNfyY7RUxhGd/nJ6uSIIXu7i3Mz2HgV9X0NseOuv6y3d722CRdKA4lWhMVq+0= X-Gm-Message-State: AOJu0YzZZGn23B3o5jhLzJlb5RJDBl0nESEkgLY5+hycHRXYVbedsehj PmclJR8E7Ynf83eglva7Aq4rDd03B/CebeZcjwoL/usgnj696GFO X-Google-Smtp-Source: AGHT+IHZD81UcFRTlNv4zjZQI36JjV0vBJA7F7+3oawTaR5ZKsH3U/8INCWLGb1yJIFCamvJrUoNNg== X-Received: by 2002:a2e:96cd:0:b0:2ec:17a9:ce9c with SMTP id 38308e7fff4ca-2ec17a9d142mr11560241fa.47.1718364882776; Fri, 14 Jun 2024 04:34:42 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a2e:81d4:0:b0:2eb:e49e:d605 with SMTP id 38308e7fff4ca-2ec0284bfe6ls11543051fa.0.-pod-prod-05-eu; Fri, 14 Jun 2024 04:34:41 -0700 (PDT) X-Received: by 2002:a05:651c:8e:b0:2db:a9c9:4c5e with SMTP id 38308e7fff4ca-2ec0e482856mr19818061fa.21.1718364880759; Fri, 14 Jun 2024 04:34:40 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1718364880; cv=none; d=google.com; s=arc-20160816; b=dBMAu09GsclrthdzwULnPAycpOXTfS0cc1t2NKCg70miJkWOPYkTzb55xj/FEw176I GS1Crc7vXj3yM5PkpsNzub461Vn0eEJTGJlUBl0Sps/dftnqSbMU9HmMhCqt9swK9x4T HpUQG7BzewzBSiU4oCfepoyC1M5H9JGxI9DYoDzcxX9MlBNDLsVNaR3X/SYjhbtDj69M 4ec3kvntyRZvB0w/Bbbk51fLSByZvR3hrS+uxNVjQgkVAYeRG+HW/HsP1k4xtt3KE6on t//FpFpuzK720x+3Lj7oMEN4ELGrdnSpCD/tbw+nytDvyxeZ5fxSXiu2pGA4w8hpm0fo lWAw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=lvwF6WKv3P8yYcwuDcop6W4F//6PVxPP1jXGzzC+HQA=; fh=/h9QQkzJ8EboVkWg45aWwpaUro6WMavIVd2OhN45RtE=; b=DxPJ4+iAXpdYw1VRDOkPq0/Y4sypDyvpY/2E0RNiNyY1XtU5yxLXgq8otyMblXkEts Gi+UUb+EHp+1T5VGgmSBq7ZfsCk9aRqQZyQOW8FLzitpc5faedK0jyoLyRhc6avFcTOJ WVdep6VgwTfnu+9xuThkx26mgkkZpRfEHdeWPpBRWcbtlZWB7GudosXPTtP7nyWqbK5M y/UfT6B76DkvW/b+gYxoQxE6tjAqd3DQJGw73YZ1JU5FMlDuC51T7Chcr+y3bjsHwwOJ abUTIfaL9SuWlplwQXr+2KbAAawD7HLthrK9ggs9QFZGLyFg+15ohdXO51fatBF+7pyE AoLA==; dara=google.com ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=amikan@ilbers.de Received: from shymkent.ilbers.de (shymkent.ilbers.de. [85.214.156.166]) by gmr-mx.google.com with ESMTPS id 5b1f17b1804b1-422874def6bsi1185705e9.2.2024.06.14.04.34.40 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 14 Jun 2024 04:34:40 -0700 (PDT) Received-SPF: pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) client-ip=85.214.156.166; Received: from localhost.localdomain (r154-240-38-77-broadband.btv.lv [77.38.240.154] (may be forged)) (authenticated bits=0) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPSA id 45EBYZP5032043 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 14 Jun 2024 13:34:40 +0200 From: Anton Mikanovich To: isar-users@googlegroups.com Cc: Anton Mikanovich Subject: [PATCH v9 8/8] user_manual.md: Update boostrap related documentation Date: Fri, 14 Jun 2024 14:34:25 +0300 Message-Id: <20240614113425.122722-9-amikan@ilbers.de> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240614113425.122722-1-amikan@ilbers.de> References: <20240614113425.122722-1-amikan@ilbers.de> MIME-Version: 1.0 X-Spam-Status: No, score=-4.6 required=5.0 tests=DKIMWL_WL_MED,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_EF,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,RCVD_IN_MSPIKE_H2,RCVD_IN_RP_CERTIFIED, RCVD_IN_RP_RNBL,RCVD_IN_RP_SAFE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on shymkent.ilbers.de X-Original-Sender: amikan@ilbers.de X-Original-Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=amikan@ilbers.de Precedence: list Mailing-list: list isar-users@googlegroups.com; contact isar-users+owners@googlegroups.com List-ID: X-Spam-Checked-In-Group: isar-users@googlegroups.com X-Google-Group-Id: 914930254986 List-Post: , List-Help: , List-Archive: , List-Unsubscribe: , X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= Update documentation in user manual and recipes API changelog. Signed-off-by: Anton Mikanovich --- RECIPE-API-CHANGELOG.md | 13 +++++++++++++ doc/user_manual.md | 15 ++++++++++++--- 2 files changed, 25 insertions(+), 3 deletions(-) diff --git a/RECIPE-API-CHANGELOG.md b/RECIPE-API-CHANGELOG.md index 12ea93ec..589dae66 100644 --- a/RECIPE-API-CHANGELOG.md +++ b/RECIPE-API-CHANGELOG.md @@ -629,3 +629,16 @@ into kernel kbuild package. Only the "host" specific package is built automatically at cross builds. * Support emulated module build with cross-compiled kernel for linux-module + +### Default boostrap recipe changed to mmdebstrap + +New virtual packages bootstrap-host and bootstrap-target are introduced. +There are two providers of bootstrap-host/-target currently: + * isar-mmdebstrap: deafult one using mmdebstrap to prepare rootfs + * isar-bootstrap: previous bootstrap implementation left for compatibility + +So default bootstrap procedure is now performing with mmdebstrap. +Previous implementation still can be selected by setting in local.conf: + +PREFERRED_PROVIDER_bootstrap-host ?= "isar-bootstrap-host" +PREFERRED_PROVIDER_bootstrap-target ?= "isar-bootstrap-target" diff --git a/doc/user_manual.md b/doc/user_manual.md index b12c7692..d3b41ae8 100644 --- a/doc/user_manual.md +++ b/doc/user_manual.md @@ -66,7 +66,8 @@ The steps below describe how to build the images provided by default. ### Install Host Tools -The supported host system is >= buster. +The supported host system is >= buster for debootstrap and >= bullseye for +default mmdebstrap provider. Building `debian-trixie` requires host system >= bookworm. @@ -75,7 +76,9 @@ Install the following packages: apt install \ binfmt-support \ bzip2 \ - debootstrap \ + mmdebstrap \ + arch-test \ + apt-utils \ dpkg-dev \ gettext-base \ git \ @@ -93,6 +96,12 @@ apt install \ zstd ``` +If using isar-bootstrap provider, debootstrap should be installed instead of +mmdebstrap: +``` +apt install debootstrap +``` + If your host is >= buster, also install the following package. ``` apt install python3-distutils @@ -135,7 +144,7 @@ apt install qemu ### Setup Sudo -Isar requires `sudo` rights without password to work with `chroot` and `debootstrap`. To add them, use the following steps: +Isar requires `sudo` rights without password to work with `chroot`. To add them, use the following steps: ``` # visudo ```