From patchwork Thu May 18 09:47:11 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anton Mikanovich X-Patchwork-Id: 2811 Return-Path: Received: from shymkent.ilbers.de ([unix socket]) by shymkent (Cyrus 2.5.10-Debian-2.5.10-3+deb9u2) with LMTPA; Thu, 18 May 2023 11:47:38 +0200 X-Sieve: CMU Sieve 2.4 Received: from mail-qk1-f189.google.com (mail-qk1-f189.google.com [209.85.222.189]) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPS id 34I9lbWI020490 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 18 May 2023 11:47:38 +0200 Received: by mail-qk1-f189.google.com with SMTP id af79cd13be357-751409fae81sf230177285a.1; Thu, 18 May 2023 02:47:37 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1684403251; cv=pass; d=google.com; s=arc-20160816; b=wgQrDBWKR8GoZBKq4hD4F029aI/ffuwmwPjwPY7ODvvhB6pBNHpBH/F/iwXQTlWzxR liGbL/+u41tOQ5nXdkUTLJ/39tWV6+Or40Xlmj3db87W7n8fvAJZWMLtyRWZ3nP+hdh3 qdWgsO2HaZKbmtiVEusQsLUejGZx+Nz14LLETGCzbdLsnnbLIQ+03Zf2fVQZzXhkoebD sEDv48OYHYELdrGxIPzk1oNCBqWsDBPscmRaYb+9PBcToNy9eiQdKiPoufqC6YWPtOZ/ +9srInKz6Uje+hAaz54pGdyuysT+ThkAg9ReNah7C9gRLZS4WFGX1uqL2YbhUL85Ht3N VGYg== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:dkim-signature; bh=RcBU0dArOAZ0ok7Vse+fDJCvml/PIu0X4uxEb3QDTck=; b=ybRJCCEr4eDRTkKgyAFwreMga6ka5PYC0SLfcRDt9NxqctTd3fGF2+K4/vjXO7ogdY DyVF5yQ9QI6PplBI8I+2fh8R5EIu6mYo2tWJrN7CjhgdYiqJrds5q3R/z63CQnn+wnQz HGVMnLlwCFYW32G7rziQaSoK7LAi0af3xlVPOdGd3MMvCOwYq5NVUzvmMiCts2eEW1md GZGbnszTEznFIOXqSuCwYLOyvE7Fgckw8V1ZsJMk7WzosfEjNnONxBhVtWUcMdD1LKX2 LoYYtV3LD6GxppVyqWm8GBBHSTHyCuO8ULE2B8mVTTGi6o9AMtESX2yZEPZsvfzNAs06 vt2g== ARC-Authentication-Results: i=2; gmr-mx.google.com; spf=pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=amikan@ilbers.de DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlegroups.com; s=20221208; t=1684403251; x=1686995251; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:x-original-authentication-results :x-original-sender:mime-version:references:in-reply-to:message-id :date:subject:cc:to:from:sender:from:to:cc:subject:date:message-id :reply-to; bh=RcBU0dArOAZ0ok7Vse+fDJCvml/PIu0X4uxEb3QDTck=; b=cD4C3PaQGxFq3jalC4N0F7giHcp4u0oaDJDje0p17uNQ+WDIerw4YKfUP5QUuhScJb s34honhKCm9vOHrmAZa8dgROUECb5W12SphgPFI9UT28FzR4X+siOepIUj95v1jM6TXd ZRU9FNev5d5s8v9wVv4U2UEwHcCOoSQnf1Ga70YBnZ/KWbFMeAZsFsZ7rjMHb8KE6k5I hRwE3t18fOIQEh+4FgTFF85nUpDKJVDHUyw5KN+4gR9knB3qQwcQyji5bLMdsfkZtMfP yIkUDEgaMctnhoME9woTSGRWupL2Qufkv4Sx09PHVy2/4rB7/Xgp6yZyKmC+GzTi3+90 VLHg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684403251; x=1686995251; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :x-spam-checked-in-group:list-id:mailing-list:precedence :x-original-authentication-results:x-original-sender:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :x-beenthere:x-gm-message-state:sender:from:to:cc:subject:date :message-id:reply-to; bh=RcBU0dArOAZ0ok7Vse+fDJCvml/PIu0X4uxEb3QDTck=; b=XOLJUlF2kg+d1swbLufj4swBvaCHhxj2hofh7zGNWd/+dsP7/DjCiWcpcoXH+F3A31 KOxH3WdE57jYD+RBdPTDyNS+pEfOssOIt4gXRB5DfBRsaP5W9LAhSZvql8bA3cJTEIRG PS8XuGtMPPP8q7IgXs5oVXk/hkC7Qsw7dtqI9e66GGSLRIIleH8Sze16KlCHkMAOIOwk dZYKgqZxAWiKdLWmnXnytUcISylAJvHSNYZpnJQaSKOyLt9+WGwRacMAhn3OSL253Ki2 /lN9tuplaZIPg8Ae3W5ppMiWxEhNAj+mjEQTXr0Hu1eJ/Jhi4Gjds5UEFZSxQASWM3RW yYgQ== Sender: isar-users@googlegroups.com X-Gm-Message-State: AC+VfDzn/CIbXQMKIVadtUKVkfbKhnSLKy+KkJi+Tl094JaxyCPwmop/ RcIjz5J6KjFFmjBL48hygFW0sg== X-Google-Smtp-Source: ACHHUZ6dXnJj3dR5u9Rj2F14PsWs+u2MtVplJJrqq05x4X+wbcfx++TTsAt2bpLkwxtgHziA0ZZxHA== X-Received: by 2002:a05:620a:4686:b0:759:3ed1:a847 with SMTP id bq6-20020a05620a468600b007593ed1a847mr1046232qkb.9.1684403251612; Thu, 18 May 2023 02:47:31 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com Received: by 2002:ac8:4a02:0:b0:3ef:573b:68e1 with SMTP id x2-20020ac84a02000000b003ef573b68e1ls1252073qtq.0.-pod-prod-05-us; Thu, 18 May 2023 02:47:31 -0700 (PDT) X-Received: by 2002:a1f:3dd1:0:b0:456:e5d3:4161 with SMTP id k200-20020a1f3dd1000000b00456e5d34161mr250191vka.1.1684403251047; Thu, 18 May 2023 02:47:31 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684403251; cv=none; d=google.com; s=arc-20160816; b=r2fadtiAM/S6hROwUzx/rUMWFnwVZLUIwOr7SYDTL96cjaUgf+J27c8aKitCEOo3/G lDH3Tk3dlAU2SNVPghtJnj8ehLapaBrfCAjNJN7SUWLeuVIyQPvZtXav6936MjvnZrAk 8keuQV/cbSAzT7EgxFWQRvgzkchKzZcmy+xqsiwZE/3R1Nvwv/BIphT29dzzjLCW65sM icSK0POV7gUVHpShqHGE0F7OAAHumLHynrmiUll0q7AVZt6kCWYR6sekRoUwk0Cg7JFR NrFsZf/aOGuEmFuLTj6nEMarp5Ogw26JBuHzYKvfk11pQLsrT0e1fe89RbGGXmJv7Sfr u4FA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=2C9U9OzP/jFakvpFxylUz7Rq6ZTJctOzb4lyGL6n8p8=; b=yfO3kc8ZCOCAf9r0scWi/lvrP6tq/heVmswBJhHa7cRs7E/3ezp5bxqt5jaUdO1Opv 73vkZpTm3/wSXXm6nP6+a0ZgPlnYCQe65f/HtzteO0ZKmw1r0Ag4hvxlI9mRngSpMa5y TE29pYP3VP9zuAq2VPeUWVs/4dcVHrTuTmkDAuxzVIcJ2q8yMOSuKNYtTbdiOC8L7+HC sTyaTLrOYPxvAN7Yq62I+GQnkW9akDUDtnHIQ0lwNR01R+Q0d1V/TF1kqaD8hov4aJ3U tP9Ba51woseZUyQjxX5PIhbgQeE0yCw0N3xXfzVHhn5MF9aYLkhEosiJHZIYvDC1np5G vCug== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=amikan@ilbers.de Received: from shymkent.ilbers.de (shymkent.ilbers.de. [85.214.156.166]) by gmr-mx.google.com with ESMTPS id y197-20020a1f7dce000000b00456d8fcc97csi115928vkc.2.2023.05.18.02.47.29 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Thu, 18 May 2023 02:47:30 -0700 (PDT) Received-SPF: pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) client-ip=85.214.156.166; Received: from user-B660.. ([85.254.74.3]) (authenticated bits=0) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPSA id 34I9lM4R020451 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 18 May 2023 11:47:24 +0200 From: Anton Mikanovich To: isar-users@googlegroups.com Cc: Uladzimir Bely , Anton Mikanovich Subject: [PATCH v2 1/6] meta: Switch to mmdebstrap Date: Thu, 18 May 2023 12:47:11 +0300 Message-Id: <20230518094716.117770-2-amikan@ilbers.de> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230518094716.117770-1-amikan@ilbers.de> References: <20230518094716.117770-1-amikan@ilbers.de> MIME-Version: 1.0 X-Spam-Status: No, score=-0.9 required=5.0 tests=DKIMWL_WL_MED,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_EF,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,RCVD_IN_MSPIKE_H2,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on shymkent.ilbers.de X-Original-Sender: amikan@ilbers.de X-Original-Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=amikan@ilbers.de Precedence: list Mailing-list: list isar-users@googlegroups.com; contact isar-users+owners@googlegroups.com List-ID: X-Spam-Checked-In-Group: isar-users@googlegroups.com X-Google-Group-Id: 914930254986 List-Post: , List-Help: , List-Archive: , List-Unsubscribe: , X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= From: Uladzimir Bely Instead of debootstrap, use mmdebstrap alternative. Internally, it uses apt and allows to debootstrap the distro from multiple repositories. Signed-off-by: Uladzimir Bely Signed-off-by: Anton Mikanovich --- meta-isar/conf/distro/ubuntu-focal.conf | 4 -- .../isar-bootstrap/isar-bootstrap.inc | 57 ++++++++++--------- 2 files changed, 30 insertions(+), 31 deletions(-) diff --git a/meta-isar/conf/distro/ubuntu-focal.conf b/meta-isar/conf/distro/ubuntu-focal.conf index 0cb6958d..be8c951c 100644 --- a/meta-isar/conf/distro/ubuntu-focal.conf +++ b/meta-isar/conf/distro/ubuntu-focal.conf @@ -17,10 +17,6 @@ BOOTSTRAP_KEY = "file://${LAYERDIR_isar}/conf/distro/ubuntu.public.key;sha256sum DISTRO_BOOTSTRAP_KEYS += "${BOOTSTRAP_KEY}" HOST_DISTRO_BOOTSTRAP_KEYS += "${BOOTSTRAP_KEY}" - -# that is what debootstrap_1.0.118ubuntu1 does anyways -DISTRO_DEBOOTSTRAP_SCRIPT = "/usr/share/debootstrap/scripts/gutsy" - DISTRO_CONFIG_SCRIPT ?= "debian-configscript.sh" DISTRO_KERNELS ?= " \ diff --git a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc index b94ae0bd..f89c284e 100644 --- a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc +++ b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc @@ -15,7 +15,7 @@ PV = "1.0" BOOTSTRAP_FOR_HOST ?= "0" -DEBOOTSTRAP ?= "qemu-debootstrap" +DEBOOTSTRAP = "mmdebstrap" ROOTFSDIR = "${WORKDIR}/rootfs" APTPREFS = "${WORKDIR}/apt-preferences" APTSRCS = "${WORKDIR}/apt-sources" @@ -23,12 +23,14 @@ APTSRCS_INIT = "${WORKDIR}/apt-sources-init" DISTRO_BOOTSTRAP_KEYFILES = "" THIRD_PARTY_APT_KEYFILES = "" DEPLOY_ISAR_BOOTSTRAP ?= "" -DISTRO_BOOTSTRAP_BASE_PACKAGES = "locales" +DISTRO_BOOTSTRAP_BASE_PACKAGES = "locales,usrmerge" DISTRO_BOOTSTRAP_BASE_PACKAGES:append:gnupg = ",gnupg" DISTRO_BOOTSTRAP_BASE_PACKAGES:append:https-support = ",ca-certificates" DISTRO_VARS_PREFIX ?= "${@'HOST_' if d.getVar('BOOTSTRAP_FOR_HOST') == '1' else ''}" BOOTSTRAP_DISTRO = "${@d.getVar('HOST_DISTRO' if d.getVar('BOOTSTRAP_FOR_HOST') == '1' else 'DISTRO')}" BOOTSTRAP_BASE_DISTRO = "${@d.getVar('HOST_BASE_DISTRO' if d.getVar('BOOTSTRAP_FOR_HOST') == '1' else 'BASE_DISTRO')}" +BOOTSTRAP_DISTRO_ARCH = "${@d.getVar('HOST_ARCH' if d.getVar('BOOTSTRAP_FOR_HOST') == '1' else 'DISTRO_ARCH')}" +BOOTSTRAP_TMPDIR = "${WORKDIR}/tempdir" FILESEXTRAPATHS:append = ":${BBPATH}" inherit deb-dl-dir @@ -270,7 +272,7 @@ do_bootstrap[vardeps] += " \ ISAR_ENABLE_COMPAT_ARCH \ ${DISTRO_VARS_PREFIX}DISTRO_APT_SOURCES \ " -do_bootstrap[dirs] = "${DEPLOY_DIR_BOOTSTRAP}" +do_bootstrap[dirs] = "${DEPLOY_DIR_BOOTSTRAP} ${BOOTSTRAP_TMPDIR}" do_bootstrap[depends] = "base-apt:do_cache isar-apt:do_cache_config" do_bootstrap[network] = "${TASK_USE_NETWORK_AND_SUDO}" @@ -286,27 +288,31 @@ do_bootstrap() { if [ -f "${DISTRO_BOOTSTRAP_KEYRING}" ]; then debootstrap_args="$debootstrap_args --keyring=${DISTRO_BOOTSTRAP_KEYRING}" fi - if [ "${ISAR_USE_CACHED_BASE_REPO}" = "1" -a -z "${BASE_REPO_KEY}" ]; then - debootstrap_args="$debootstrap_args --no-check-gpg" - fi E="${@ isar_export_proxies(d)}" - export BOOTSTRAP_FOR_HOST debootstrap_args E + export BOOTSTRAP_FOR_HOST - sudo rm -rf --one-file-system "${ROOTFSDIR}" deb_dl_dir_import "${ROOTFSDIR}" "${BOOTSTRAP_BASE_DISTRO}-${BASE_DISTRO_CODENAME}" + sudo rm -rf --one-file-system "${ROOTFSDIR}" + mkdir -p "${ROOTFSDIR}" + + if [ ! -z "${SOURCE_DATE_EPOCH}" ]; then + export SOURCE_DATE_EPOCH="${SOURCE_DATE_EPOCH}" + fi + + arch_param="--arch=${BOOTSTRAP_DISTRO_ARCH},${DISTRO_ARCH}" + + sudo TMPDIR="${BOOTSTRAP_TMPDIR}" ${DEBOOTSTRAP} $debootstrap_args \ + $arch_param \ + --mode=unshare \ + ${@get_distro_components_argument(d)} \ + "${@get_distro_suite(d)}" \ + "${WORKDIR}/rootfs.tar.zst" \ + "${@get_distro_source(d)}" sudo -E -s <<'EOSUDO' set -e - if [ "${BOOTSTRAP_FOR_HOST}" = "0" ]; then - arch_param="--arch=${DISTRO_ARCH}" - fi - ${DEBOOTSTRAP} $debootstrap_args \ - $arch_param \ - ${@get_distro_components_argument(d)} \ - "${@get_distro_suite(d)}" \ - "${ROOTFSDIR}" \ - "${@get_distro_source(d)}" \ - ${DISTRO_DEBOOTSTRAP_SCRIPT} + + tar -xf "${WORKDIR}/rootfs.tar.zst" -C "${ROOTFSDIR}" --exclude="./dev/console" # Install apt config mkdir -p "${ROOTFSDIR}/etc/apt/preferences.d" @@ -409,19 +415,16 @@ SSTATEPOSTINSTFUNCS += "bootstrap_sstate_finalize" bootstrap_sstate_prepare() { # this runs in SSTATE_BUILDDIR, which will be deleted automatically - lopts="--one-file-system --exclude=var/cache/apt/archives" - sudo tar -C $(dirname "${ROOTFSDIR}") -cpSf bootstrap.tar $lopts $(basename "${ROOTFSDIR}") - sudo chown $(id -u):$(id -g) bootstrap.tar + sudo cp -a "$(dirname "${ROOTFSDIR}")/rootfs.tar.zst" ./bootstrap.tar.zst + sudo chown $(id -u):$(id -g) bootstrap.tar.zst } bootstrap_sstate_finalize() { # this runs in SSTATE_INSTDIR - # - after building the bootstrap, the tar won't be there, but we also don't need to unpack - # - after restoring from cache, there will be a tar which we unpack and then delete - if [ -f bootstrap.tar ]; then - sudo tar -C $(dirname "${ROOTFSDIR}") -xpf bootstrap.tar - sudo ln -Tfsr "${ROOTFSDIR}" "${DEPLOY_ISAR_BOOTSTRAP}" - rm bootstrap.tar + if [ -f bootstrap.tar.zst ]; then + mv bootstrap.tar.zst "$(dirname "${ROOTFSDIR}")/rootfs.tar.zst" + sudo ln -Tfsr "$(dirname "${ROOTFSDIR}")/rootfs.tar.zst" \ + "${DEPLOY_ISAR_BOOTSTRAP}.tar.zst" fi }