From patchwork Wed Sep 27 04:33:15 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Uladzimir Bely X-Patchwork-Id: 3137 Return-Path: Received: from shymkent.ilbers.de ([unix socket]) by shymkent (Cyrus 2.5.10-Debian-2.5.10-3+deb9u2) with LMTPA; Wed, 27 Sep 2023 06:33:34 +0200 X-Sieve: CMU Sieve 2.4 Received: from mail-lj1-f189.google.com (mail-lj1-f189.google.com [209.85.208.189]) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPS id 38R4XW5V022023 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Wed, 27 Sep 2023 06:33:33 +0200 Received: by mail-lj1-f189.google.com with SMTP id 38308e7fff4ca-2b6fdb8d0acsf28958881fa.0 for ; Tue, 26 Sep 2023 21:33:33 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1695789206; cv=pass; d=google.com; s=arc-20160816; b=GFhkqWNEh5eyg/HrAjvI2gorAqm4jh21qI2452EXW2DRTJcn6cnviPXAU6U3BvSLPf stC2Bh22qJx0vOyRga1z57iAkEA9XtwlwyLWxOk0DBQxeY9dcunMQhmdx+FiKCmYOasm BfSpa8j0murxWjiegPjc58a2FBkENtsiBDOF0XMnxvitnR7yoiznQFGFIFEdlUam7jQr afwBWqN7MpBExeC7FK5Vlxzic1vtE8hsmCB/bHLTXwApZo4y5O6dPnV+tVg7YHDtSYNs +P4l1ItSxYkf/7vYXYZPyF9damBq9ZYkfEwkf1zQ2HEvFTHQ0SMje0Iik3WbKRt63UgF rp9w== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:mime-version:references:in-reply-to :message-id:date:subject:to:from:sender:dkim-signature; bh=Wx0ooM/b8ieybBwjxE0VqPBaZvkIrPhoNW8quDOHKwU=; fh=7tclEdh7YbwSQowgJ6LNq720O7H5HTEaqj22NJWRE2E=; b=xvdKrohSO+joW6rPISSfD9C0uVRMiaJN9U6y88MLQIUgATzp0EUme1wPpK5WNXkpJT 6og384LrFMMK6knZLP3vjagJzYQ+coNrZyP9WQ3XbYkPAZqMKMBhRjUAjkTUUasYWSnD m2FRQhqWGiGCmuNvSZXfBLqmAVwrmV1r8QbJi/JBBUm6KAWkPuuwuJlMeEGqY+AUZftJ g2v4CvKRTedH05sQrS0//Ey1PgzP+Ahy5+/dewGB1sMhJPcdjWA3tpnEYMrIkrYJq3iE ybo1W1L1f2nMM40qpW85Xnf8NTQRPt/HgvjYB9TY8MbPSn53zt93LBi4YfcJbp74cIQV Xwig== ARC-Authentication-Results: i=2; gmr-mx.google.com; spf=pass (google.com: domain of ubely@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=ubely@ilbers.de DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlegroups.com; s=20230601; t=1695789206; x=1696394006; darn=isar-build.org; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:x-original-authentication-results :x-original-sender:mime-version:references:in-reply-to:message-id :date:subject:to:from:sender:from:to:cc:subject:date:message-id :reply-to; bh=Wx0ooM/b8ieybBwjxE0VqPBaZvkIrPhoNW8quDOHKwU=; b=p1qAlABZi87RvlKw3LQD/XzeIrRP0m6zrD2Qy+JJNqQqgFvR/PJrovclj4nOK9skl5 UqT0L/p1VQT9ssmUnnphJLB/kVCI5Z4aPqYjxsWAX/AOkYFLEuus9qSznvHOD0/xkWAS gOSRzhzAwRcNG0Y66pxgJB4AmMkF3zObcIKuZYH5bTa1yASEDPplEB32irBxXCxO7ZQR cnhm9hFZwSi9WFu1976r1EeuGakhk32qi1P9GjqAIkPsiBrjbTfeF8/4/rlwDCyIQuwc QA/kqL7rhHcSmwJ8ALVduVHmWxHejXd/cyqmyHFToOlg/DOh3rOa1oi1W2uIzGefSosl 6K6A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1695789206; x=1696394006; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :x-spam-checked-in-group:list-id:mailing-list:precedence :x-original-authentication-results:x-original-sender:mime-version :references:in-reply-to:message-id:date:subject:to:from:x-beenthere :x-gm-message-state:sender:from:to:cc:subject:date:message-id :reply-to; bh=Wx0ooM/b8ieybBwjxE0VqPBaZvkIrPhoNW8quDOHKwU=; b=mxcAbf285Bbf0XCIGu6qEuVEKxyxnQJTQ3M3LgVNSi3QbIpMVBMHLldUZ3w7emE5qf ufXfHbi+cGrjLbG5K8mEdkK8e+ZzUzwzFeyP+fbG+iJf24ETFZgY1g6bbsLlLypq1CHe 4SnYLfgAasHBoglMqwicVyrNhtXys2bBPyo+fccFEZOGeEhtszH8EDY1KWAld0/hqNps GhMjb30qF7D1VCCVhUAr8ZCBO00urnJu/a4G/UPArc2sqE74FHS1cvCIbq2WRJCMshxF c8BsyWS72tj1HbhJL0dJT13sWHpKNQsbtBuwo+YgGaXXM9Go74KFgEN38ibil6ZVjIBg fixA== Sender: isar-users@googlegroups.com X-Gm-Message-State: AOJu0YxfTNt9rTOo2RJtxOlGZF8nt/iacWV8dZ49qgcTaqmb2Hiuua4v QwAxbnpNRVwPgITWrMirEnI= X-Google-Smtp-Source: AGHT+IFWvYCxi4eTXHnFLi3QLfqyXHzEvqc2ftmsVE61TWpYfxHOdMVukciTFh7PTjyGucdP6RqCLw== X-Received: by 2002:ac2:548b:0:b0:501:b1c0:d545 with SMTP id t11-20020ac2548b000000b00501b1c0d545mr531580lfk.3.1695789204842; Tue, 26 Sep 2023 21:33:24 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a19:9112:0:b0:500:83e8:9d2c with SMTP id t18-20020a199112000000b0050083e89d2cls310293lfd.1.-pod-prod-04-eu; Tue, 26 Sep 2023 21:33:22 -0700 (PDT) X-Received: by 2002:a05:6512:1591:b0:503:5cd:998b with SMTP id bp17-20020a056512159100b0050305cd998bmr713004lfb.28.1695789202652; Tue, 26 Sep 2023 21:33:22 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1695789202; cv=none; d=google.com; s=arc-20160816; b=xpOo7kiaRCiGJ+XbpEGCPjGEHRClLpjKWhw7g7bBi3eXgT6ppPTPXRpyVaHfFMKXfK jWdx7PFC1NXcTx1P8j5m7rMcAOtsmPR+9B+T/USAlzF6htcgMVE6NlXfJWhSlRUSJFfw HeN+Whxybrsv5/5DKilAYtdIi0J8rjMfS64wJeNndAc6zQGevMljXYUPBZWoyP2xIthD hdG93VZPW265kJQwsSyRehATcftfQXwrJUbI6Jt/iw9msh+8Yh2eIN+b98SQA2TLcSp9 8HJ8NEDFvr8Mu+3bQbyeymw0/IQyObiYVIqsJOEgU/daGBwSRohMQcDVixx+yQDspGVg ZSuQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from; bh=2goDuKTbsNG99hOxMjCo9pY2xIRh9MR8EjRYdArj+0U=; fh=7tclEdh7YbwSQowgJ6LNq720O7H5HTEaqj22NJWRE2E=; b=JgEnxHe/mEG8ZFkaLvoaWu35T6PZJ8vur6xpmBZ2KIhj3QzJt2j3+zHpsaVZ77QeIu brMjPqV32mrO/gqe5MnfEvLHuw6jpPlF2zfjB4pS4MCMYnydln0C3NiEDTBvHOGI853O sQdPuR74GXeJx5UDn3L2Rn+GMkU/WOdQGbjnouBnA5yld6CRKJoyPJtvwuFL33XoZn4M a1xF0zkjFXxKCqupIFWNLdzMKaOaV48yALDyoduAjk95A2oGgnSStHcm9eJBN/zOswuX tGpfaWQFH3WHWIKpmOUX9sE2obj9bGjSYKKt8uOFhvAMv/EQEXeq4/xtAd1iUwWtpv9f wttQ== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of ubely@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=ubely@ilbers.de Received: from shymkent.ilbers.de (shymkent.ilbers.de. [85.214.156.166]) by gmr-mx.google.com with ESMTPS id b9-20020a056512070900b004fe3ba741c8si936154lfs.8.2023.09.26.21.33.22 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 26 Sep 2023 21:33:22 -0700 (PDT) Received-SPF: pass (google.com: domain of ubely@ilbers.de designates 85.214.156.166 as permitted sender) client-ip=85.214.156.166; Received: from baighyz.m.ilbers.de (host-80-81-17-52.static.customer.m-online.net [80.81.17.52]) (authenticated bits=0) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPSA id 38R4XKXl021959 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Wed, 27 Sep 2023 06:33:21 +0200 From: Uladzimir Bely To: isar-users@googlegroups.com Subject: [PATCH v4 1/6] meta: Switch to mmdebstrap Date: Wed, 27 Sep 2023 06:33:15 +0200 Message-Id: <20230927043320.6264-2-ubely@ilbers.de> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20230927043320.6264-1-ubely@ilbers.de> References: <20230927043320.6264-1-ubely@ilbers.de> MIME-Version: 1.0 X-Spam-Status: No, score=-0.9 required=5.0 tests=DKIMWL_WL_MED,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_EF,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,RCVD_IN_MSPIKE_H2,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on shymkent.ilbers.de X-Original-Sender: ubely@ilbers.de X-Original-Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of ubely@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=ubely@ilbers.de Precedence: list Mailing-list: list isar-users@googlegroups.com; contact isar-users+owners@googlegroups.com List-ID: X-Spam-Checked-In-Group: isar-users@googlegroups.com X-Google-Group-Id: 914930254986 List-Post: , List-Help: , List-Archive: , List-Unsubscribe: , X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= Instead of debootstrap, use mmdebstrap alternative. Internally, it uses apt and allows to debootstrap the distro from multiple repositories. Signed-off-by: Uladzimir Bely Signed-off-by: Anton Mikanovich --- meta-isar/conf/distro/ubuntu-common.inc | 4 -- .../isar-bootstrap/isar-bootstrap.inc | 57 ++++++++++--------- 2 files changed, 30 insertions(+), 31 deletions(-) diff --git a/meta-isar/conf/distro/ubuntu-common.inc b/meta-isar/conf/distro/ubuntu-common.inc index 6a990a89..afe96b73 100644 --- a/meta-isar/conf/distro/ubuntu-common.inc +++ b/meta-isar/conf/distro/ubuntu-common.inc @@ -18,10 +18,6 @@ HOST_DISTRO_BOOTSTRAP_KEYS += "${BOOTSTRAP_KEY}" # kernel package name is linux-image-generic in Ubuntu KERNEL_NAME = "generic" - -# that is what debootstrap_1.0.118ubuntu1 does anyways -DISTRO_DEBOOTSTRAP_SCRIPT = "/usr/share/debootstrap/scripts/gutsy" - DISTRO_CONFIG_SCRIPT ?= "debian-configscript.sh" DISTRO_KERNELS ?= " \ diff --git a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc index 3477c2fb..637018c6 100644 --- a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc +++ b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc @@ -15,7 +15,7 @@ PV = "1.0" BOOTSTRAP_FOR_HOST ?= "0" -DEBOOTSTRAP ?= "qemu-debootstrap" +DEBOOTSTRAP = "mmdebstrap" ROOTFSDIR = "${WORKDIR}/rootfs" APTPREFS = "${WORKDIR}/apt-preferences" APTSRCS = "${WORKDIR}/apt-sources" @@ -23,12 +23,14 @@ APTSRCS_INIT = "${WORKDIR}/apt-sources-init" DISTRO_BOOTSTRAP_KEYFILES = "" THIRD_PARTY_APT_KEYFILES = "" DEPLOY_ISAR_BOOTSTRAP ?= "" -DISTRO_BOOTSTRAP_BASE_PACKAGES = "locales" +DISTRO_BOOTSTRAP_BASE_PACKAGES = "locales,usrmerge" DISTRO_BOOTSTRAP_BASE_PACKAGES:append:gnupg = ",gnupg" DISTRO_BOOTSTRAP_BASE_PACKAGES:append:https-support = ",ca-certificates" DISTRO_VARS_PREFIX ?= "${@'HOST_' if d.getVar('BOOTSTRAP_FOR_HOST') == '1' else ''}" BOOTSTRAP_DISTRO = "${@d.getVar('HOST_DISTRO' if d.getVar('BOOTSTRAP_FOR_HOST') == '1' else 'DISTRO')}" BOOTSTRAP_BASE_DISTRO = "${@d.getVar('HOST_BASE_DISTRO' if d.getVar('BOOTSTRAP_FOR_HOST') == '1' else 'BASE_DISTRO')}" +BOOTSTRAP_DISTRO_ARCH = "${@d.getVar('HOST_ARCH' if d.getVar('BOOTSTRAP_FOR_HOST') == '1' else 'DISTRO_ARCH')}" +BOOTSTRAP_TMPDIR = "${WORKDIR}/tempdir" FILESEXTRAPATHS:append = ":${BBPATH}" inherit deb-dl-dir @@ -267,7 +269,7 @@ do_bootstrap[vardeps] += " \ ISAR_ENABLE_COMPAT_ARCH \ ${DISTRO_VARS_PREFIX}DISTRO_APT_SOURCES \ " -do_bootstrap[dirs] = "${DEPLOY_DIR_BOOTSTRAP}" +do_bootstrap[dirs] = "${DEPLOY_DIR_BOOTSTRAP} ${BOOTSTRAP_TMPDIR}" do_bootstrap[depends] = "base-apt:do_cache isar-apt:do_cache_config" do_bootstrap[network] = "${TASK_USE_NETWORK_AND_SUDO}" @@ -283,27 +285,31 @@ do_bootstrap() { if [ -f "${DISTRO_BOOTSTRAP_KEYRING}" ]; then debootstrap_args="$debootstrap_args --keyring=${DISTRO_BOOTSTRAP_KEYRING}" fi - if [ "${ISAR_USE_CACHED_BASE_REPO}" = "1" -a -z "${BASE_REPO_KEY}" ]; then - debootstrap_args="$debootstrap_args --no-check-gpg" - fi E="${@ isar_export_proxies(d)}" - export BOOTSTRAP_FOR_HOST debootstrap_args E + export BOOTSTRAP_FOR_HOST - sudo rm -rf --one-file-system "${ROOTFSDIR}" deb_dl_dir_import "${ROOTFSDIR}" "${BOOTSTRAP_BASE_DISTRO}-${BASE_DISTRO_CODENAME}" + sudo rm -rf --one-file-system "${ROOTFSDIR}" + mkdir -p "${ROOTFSDIR}" + + if [ ! -z "${SOURCE_DATE_EPOCH}" ]; then + export SOURCE_DATE_EPOCH="${SOURCE_DATE_EPOCH}" + fi + + arch_param="--arch=${BOOTSTRAP_DISTRO_ARCH},${DISTRO_ARCH}" + + sudo TMPDIR="${BOOTSTRAP_TMPDIR}" ${DEBOOTSTRAP} $debootstrap_args \ + $arch_param \ + --mode=unshare \ + ${@get_distro_components_argument(d)} \ + "${@get_distro_suite(d)}" \ + "${WORKDIR}/rootfs.tar.zst" \ + "${@get_distro_source(d)}" sudo -E -s <<'EOSUDO' set -e - if [ "${BOOTSTRAP_FOR_HOST}" = "0" ]; then - arch_param="--arch=${DISTRO_ARCH}" - fi - ${DEBOOTSTRAP} $debootstrap_args \ - $arch_param \ - ${@get_distro_components_argument(d)} \ - "${@get_distro_suite(d)}" \ - "${ROOTFSDIR}" \ - "${@get_distro_source(d)}" \ - ${DISTRO_DEBOOTSTRAP_SCRIPT} + + tar -xf "${WORKDIR}/rootfs.tar.zst" -C "${ROOTFSDIR}" --exclude="./dev/console" # Install apt config mkdir -p "${ROOTFSDIR}/etc/apt/preferences.d" @@ -406,19 +412,16 @@ SSTATEPOSTINSTFUNCS += "bootstrap_sstate_finalize" bootstrap_sstate_prepare() { # this runs in SSTATE_BUILDDIR, which will be deleted automatically - lopts="--one-file-system --exclude=var/cache/apt/archives" - sudo tar -C $(dirname "${ROOTFSDIR}") -cpSf bootstrap.tar $lopts $(basename "${ROOTFSDIR}") - sudo chown $(id -u):$(id -g) bootstrap.tar + sudo cp -a "$(dirname "${ROOTFSDIR}")/rootfs.tar.zst" ./bootstrap.tar.zst + sudo chown $(id -u):$(id -g) bootstrap.tar.zst } bootstrap_sstate_finalize() { # this runs in SSTATE_INSTDIR - # - after building the bootstrap, the tar won't be there, but we also don't need to unpack - # - after restoring from cache, there will be a tar which we unpack and then delete - if [ -f bootstrap.tar ]; then - sudo tar -C $(dirname "${ROOTFSDIR}") -xpf bootstrap.tar - sudo ln -Tfsr "${ROOTFSDIR}" "${DEPLOY_ISAR_BOOTSTRAP}" - rm bootstrap.tar + if [ -f bootstrap.tar.zst ]; then + mv bootstrap.tar.zst "$(dirname "${ROOTFSDIR}")/rootfs.tar.zst" + sudo ln -Tfsr "$(dirname "${ROOTFSDIR}")/rootfs.tar.zst" \ + "${DEPLOY_ISAR_BOOTSTRAP}.tar.zst" fi }