From patchwork Fri Jun 14 11:34:24 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anton Mikanovich X-Patchwork-Id: 3613 Return-Path: Received: from shymkent.ilbers.de ([unix socket]) by shymkent (Cyrus 2.5.10-Debian-2.5.10-3+deb9u2) with LMTPA; Fri, 14 Jun 2024 13:47:17 +0200 X-Sieve: CMU Sieve 2.4 Received: from mail-lj1-f186.google.com (mail-lj1-f186.google.com [209.85.208.186]) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPS id 45EBlGFB032649 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Fri, 14 Jun 2024 13:47:17 +0200 Received: by mail-lj1-f186.google.com with SMTP id 38308e7fff4ca-2ebd9640ac2sf10512221fa.0 for ; Fri, 14 Jun 2024 04:47:17 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1718365631; cv=pass; d=google.com; s=arc-20160816; b=EVWxNDjZMX5j+puLjSBrWWcFe8H1uXguS416FZ6FFOWjl34PhFQhoxf0c09Z5jW/LJ Y9hzswKvur1M8ml+8Npz3Kwv2txC2rHlmH6bxzIrdnzEGXdMZ/w5b6zeVddgg7nahfqG Edg0u+IP0hs6V7NhdpXDONt+qhEtRoOl/mGMwq3LFPFM7/RI+172KJ/aanCb2kzf3pfa QlIBUFEtT4bP2C/aNI86FiVlVwRHNH7IzgyrLmGZTpjPk7NBTqAkDPt/v18l1dtflm5n IR5TEpKC2y/hGlVcmB4NKjhzsdOWxD+rL5jVq1WLCrJGh79X1UdzjhA4XyxiiMu/XyVl k/dw== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:dkim-signature; bh=mMATEjfZkMM54oyL95pFiHpX1HLTCHd2f9V+PfqCy1c=; fh=Xo4pE8Y7042oYt+0lxZCFWxN9PbcoW6ZNKotDDMLNd4=; b=WtqKAWUrI0QAYEIqqmAcuzWFneIElV27ODeI35uGHJQo7Patn6ZooJqkKaZgBMdkdO arlwzf9XPMRcMpCG8mKVOCw4O6uTwx6+K4vuPQ/bNPMer2ZspRJTROHNadrxKPZwfoGo +br/7LPtsvxz6jhacmXIasJKID92PpFrDnuaZdaTToPSyw2Tql/QtJoBnQOPBQyoC7i5 DDQR0Dm0vUncVyMKFzHsD1IA1iaI3vRe2xi1+XQ5cLYXryE8RXz1CdF5tgQ151Ach3hU ruPb9EXDTOLlcYN1TstiaNGa7QEXUsxwViDHWFz75zPMoxHUydsQwoZWiB5ygOIUaf0z vYjg==; darn=isar-build.org ARC-Authentication-Results: i=2; gmr-mx.google.com; spf=pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=amikan@ilbers.de DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlegroups.com; s=20230601; t=1718365631; x=1718970431; darn=isar-build.org; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:x-original-authentication-results :x-original-sender:mime-version:references:in-reply-to:message-id :date:subject:cc:to:from:sender:from:to:cc:subject:date:message-id :reply-to; bh=mMATEjfZkMM54oyL95pFiHpX1HLTCHd2f9V+PfqCy1c=; b=jGes1K1QuFuh/esNSGi/D5DGpnygPoNMyrEWZEFPmnDubIQFj3BVw/d439onjm5dmN 0ZNvnTsOAFcmTiUfN498PYQp4eOrpXpo/D1C9c+u54pj075tWFCmNXya2zYAkAkqcbvf /X85IolYNSXQ8bwejdft1Be5pSK8bHG60xAWvA4xq9ZKdUL8h9ZzboUKNNxWuCXXUpyD HJ9h0ohlpBCb2mp9FYQ+N+6ZZ4UlvxHdcfYeRKXrEONl2NYIMLqpnqDLYOoi5X1heJst OGzr2P1Jul6urfDn0rtQZFX83ywosul7HH+UWN73qS8anYF89avpD0bEAkKFxP8miq8p 8ZXg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1718365631; x=1718970431; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :x-spam-checked-in-group:list-id:mailing-list:precedence :x-original-authentication-results:x-original-sender:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :x-beenthere:x-gm-message-state:sender:from:to:cc:subject:date :message-id:reply-to; bh=mMATEjfZkMM54oyL95pFiHpX1HLTCHd2f9V+PfqCy1c=; b=oC3XF3u/56/mEKfmlSdeVZ/JYxXprlpFm9TjZvxWJ0qb2r3YQNCJ6YvpXC4SfmyVHy oNIdz10ybTAQCFKNgjmFZNINtxTQ2y4LdZ+q1vxObIVPBJlZ8O9NXyPKRidv5YIi83H+ JsA8ISpHfXLBdrd4rQy8z/TMD3ft6ku+5zJCcqB4+tktqzOfwX+b7TERcQvvK4mhALue wPjilGC2t3/FF/P+aXAKPQhKAs/HxCmryAvloMJaNsqRe06vdnpIE2ZXVoLdcsum3GmL ith0/539tGd4KJ8uUXylaNIgY2mAuQWwAOczdWUewuoMLZMBkyXahzk6fHqZVMq1zEZ+ NtJg== Sender: isar-users@googlegroups.com X-Forwarded-Encrypted: i=2; AJvYcCVDx9XbBrkgwd8MvdLabvG1cO2PGUBRHCzy5yRdIEufqJ3n0E8IKfQdW+xnmI0OJB3p/EUkssQNOTqSKoSe/q6OCHheyg8= X-Gm-Message-State: AOJu0YzEfJvAA/mqwFoOms1WdYxKSRBZJe80EUFoG5eeULq89G0CiCr9 RBwIlwpuwvSHH6kVG3dBW+Q3oubGP+w3hbd73KrcMHqTm/lkQCJo X-Google-Smtp-Source: AGHT+IHjRnBgesVUThfuHppu/GyVmW/If7NrOZYMbwDQo91/ImE13fxFAiWS3yFjQnbLaQPYx6uREA== X-Received: by 2002:a05:600c:5110:b0:421:7f07:92ce with SMTP id 5b1f17b1804b1-4230481bc6fmr25940395e9.3.1718364882034; Fri, 14 Jun 2024 04:34:42 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a1c:7708:0:b0:423:7d5:b537 with SMTP id 5b1f17b1804b1-42307d5b943ls1496685e9.0.-pod-prod-00-eu; Fri, 14 Jun 2024 04:34:40 -0700 (PDT) X-Received: by 2002:a05:600c:5492:b0:421:83fd:900f with SMTP id 5b1f17b1804b1-42304820ed0mr26647495e9.5.1718364880279; Fri, 14 Jun 2024 04:34:40 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1718364880; cv=none; d=google.com; s=arc-20160816; b=LmktkFzfQjluzWDOjL+ozUmyVE04C61RvXABg56nq88fc3v6HS2emauNAG2O97/A0l eA8ee57ffJb6SaArFWIlLaOVKuYCwdPKqIc6B5pviofQu8PuHibi4q1aBdin+TM3aUCY gaGIVCVf7SaKKjfcn2TQecTjNK7Js/1YSI0TiNT3vZ8KqVsfjSlXwCD6xQTxZy7hak2C WRKEwPEV5nXR2Zlz+omKwJaVXgyHj9+2X+/UnQff35gL9tU0PaQO9l1svGBnaiVXbAWY DAq5zun2exp8uCRhd1hz0jyYkkGprakR8AgBUy0X6BYf2OeDAKih13w2P8Bu6LsHuVDO Dujw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=vGr5siGIWeRHy+AKP7KSRi/M6TuppzFcwnyS/KQFw98=; fh=/h9QQkzJ8EboVkWg45aWwpaUro6WMavIVd2OhN45RtE=; b=eZCw7/pacLkyrliMoJUnICzl5nNx1vkoklzZ8VcmShOSHiwGMUZHtxZ34cfrPnvVNh hDV3G+LfgnOg/xZ582sxkkyx5Tr7p8lSH80qzdOCGrvt/zQjICZRcA745+e2ORSAwPrW uCnRKSdVBCJBSXpj/nG/31Il6ojG6ifKir9goE77Bh9QuG7Qzj0hpHKMbvvnte/Lezhk KgBYnIlwxosji0F9GN1O/e2msG33advBa/TEx/e2DXsZ3Xkvs3PvUtT8VSRND8XN7Nx1 z6mDRkLbSUPrDhyiEKBnjRHXEvmTGmhfInQzXpGNdFffsf6ZaEMJFnfds3gzMs9R551L M4iA==; dara=google.com ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=amikan@ilbers.de Received: from shymkent.ilbers.de (shymkent.ilbers.de. [85.214.156.166]) by gmr-mx.google.com with ESMTPS id 5b1f17b1804b1-42287f0b05bsi3013645e9.1.2024.06.14.04.34.40 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 14 Jun 2024 04:34:40 -0700 (PDT) Received-SPF: pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) client-ip=85.214.156.166; Received: from localhost.localdomain (r154-240-38-77-broadband.btv.lv [77.38.240.154] (may be forged)) (authenticated bits=0) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPSA id 45EBYZP4032043 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 14 Jun 2024 13:34:39 +0200 From: Anton Mikanovich To: isar-users@googlegroups.com Cc: Anton Mikanovich Subject: [PATCH v9 7/8] isar-bootstrap: Use tar output instead of directory Date: Fri, 14 Jun 2024 14:34:24 +0300 Message-Id: <20240614113425.122722-8-amikan@ilbers.de> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240614113425.122722-1-amikan@ilbers.de> References: <20240614113425.122722-1-amikan@ilbers.de> MIME-Version: 1.0 X-Spam-Status: No, score=-4.6 required=5.0 tests=DKIMWL_WL_MED,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_EF,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,RCVD_IN_MSPIKE_H2,RCVD_IN_RP_CERTIFIED, RCVD_IN_RP_RNBL,RCVD_IN_RP_SAFE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on shymkent.ilbers.de X-Original-Sender: amikan@ilbers.de X-Original-Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=amikan@ilbers.de Precedence: list Mailing-list: list isar-users@googlegroups.com; contact isar-users+owners@googlegroups.com List-ID: X-Spam-Checked-In-Group: isar-users@googlegroups.com X-Google-Group-Id: 914930254986 List-Post: , List-Help: , List-Archive: , List-Unsubscribe: , X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= Do not unpack tar with rootfs content inside bootstrap recipe. This change can allow to remove sudo usage during the bootstrap later. Using tar output requires moving chroot prepare step to the later unpack step. Signed-off-by: Anton Mikanovich --- meta/classes/rootfs.bbclass | 11 +++++---- .../isar-bootstrap/isar-bootstrap.inc | 22 +++++++++--------- .../isar-mmdebstrap/isar-mmdebstrap.inc | 23 ++++--------------- 3 files changed, 23 insertions(+), 33 deletions(-) diff --git a/meta/classes/rootfs.bbclass b/meta/classes/rootfs.bbclass index f684e969..a479b37a 100644 --- a/meta/classes/rootfs.bbclass +++ b/meta/classes/rootfs.bbclass @@ -73,12 +73,15 @@ rootfs_do_qemu() { fi } -BOOTSTRAP_SRC = "${DEPLOY_DIR_BOOTSTRAP}/${ROOTFS_DISTRO}-host_${DISTRO}-${DISTRO_ARCH}" -BOOTSTRAP_SRC:${ROOTFS_ARCH} = "${DEPLOY_DIR_BOOTSTRAP}/${ROOTFS_DISTRO}-${ROOTFS_ARCH}" +BOOTSTRAP_SRC = "${DEPLOY_DIR_BOOTSTRAP}/${ROOTFS_DISTRO}-host_${DISTRO}-${DISTRO_ARCH}.tar.zst" +BOOTSTRAP_SRC:${ROOTFS_ARCH} = "${DEPLOY_DIR_BOOTSTRAP}/${ROOTFS_DISTRO}-${ROOTFS_ARCH}.tar.zst" rootfs_prepare[weight] = "25" rootfs_prepare(){ - sudo cp -Trpfx --reflink=auto '${BOOTSTRAP_SRC}/' '${ROOTFSDIR}' + sudo tar -xf "${BOOTSTRAP_SRC}" -C "${ROOTFSDIR}" --exclude="./dev/console" + + # setup chroot + sudo "${ROOTFSDIR}/chroot-setup.sh" "setup" "${ROOTFSDIR}" } ROOTFS_CONFIGURE_COMMAND += "rootfs_configure_isar_apt" @@ -227,7 +230,7 @@ cache_deb_src() { # Note: ISAR updates the apt state information(apt-get update) only once during bootstrap and # relies on that through out the build. Copy that state information instead of apt-get update # which generates a new state from upstream. - sudo cp -Trpn --reflink=auto "${BOOTSTRAP_SRC}/var/lib/apt/lists/" "${ROOTFSDIR}/var/lib/apt/lists/" + sudo tar -xf "${BOOTSTRAP_SRC}" ./var/lib/apt/lists --one-top-level="${ROOTFSDIR}" deb_dl_dir_import ${ROOTFSDIR} ${ROOTFS_BASE_DISTRO}-${BASE_DISTRO_CODENAME} debsrc_download ${ROOTFSDIR} ${ROOTFS_BASE_DISTRO}-${BASE_DISTRO_CODENAME} diff --git a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc index 5c9eb5c8..efec42c6 100644 --- a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc +++ b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc @@ -162,7 +162,6 @@ do_bootstrap() { # setup chroot install -v -m755 "${WORKDIR}/chroot-setup.sh" "${ROOTFSDIR}/chroot-setup.sh" - "${ROOTFSDIR}/chroot-setup.sh" "setup" "${ROOTFSDIR}" # update APT mount -o bind,private /dev ${ROOTFSDIR}/dev @@ -195,8 +194,11 @@ do_bootstrap() { umount -l "${ROOTFSDIR}/sys" umount -l "${ROOTFSDIR}/base-apt" || true + # Compress rootfs for compatibility + tar --zstd -cf "${WORKDIR}/rootfs.tar.zst" -C "${ROOTFSDIR}" . + # Finalize debootstrap by setting the link in deploy - ln -Tfsr "${ROOTFSDIR}" "${DEPLOY_ISAR_BOOTSTRAP}" + ln -Tfsr "${WORKDIR}/rootfs.tar.zst" "${DEPLOY_ISAR_BOOTSTRAP}.tar.zst" EOSUDO deb_dl_dir_export "${ROOTFSDIR}" "${BOOTSTRAP_BASE_DISTRO}-${BASE_DISTRO_CODENAME}" @@ -212,19 +214,17 @@ SSTATEPOSTINSTFUNCS += "bootstrap_sstate_finalize" bootstrap_sstate_prepare() { # this runs in SSTATE_BUILDDIR, which will be deleted automatically - lopts="--one-file-system --exclude=var/cache/apt/archives" - sudo tar -C $(dirname "${ROOTFSDIR}") -cpSf bootstrap.tar $lopts $(basename "${ROOTFSDIR}") - sudo chown $(id -u):$(id -g) bootstrap.tar + sudo cp -a "${WORKDIR}/rootfs.tar.zst" ./bootstrap.tar.zst + sudo chown $(id -u):$(id -g) bootstrap.tar.zst } bootstrap_sstate_finalize() { # this runs in SSTATE_INSTDIR - # - after building the bootstrap, the tar won't be there, but we also don't need to unpack - # - after restoring from cache, there will be a tar which we unpack and then delete - if [ -f bootstrap.tar ]; then - sudo tar -C $(dirname "${ROOTFSDIR}") -xpf bootstrap.tar - sudo ln -Tfsr "${ROOTFSDIR}" "${DEPLOY_ISAR_BOOTSTRAP}" - rm bootstrap.tar + # we should restore symlinks after using tar + if [ -f bootstrap.tar.zst ]; then + mv bootstrap.tar.zst "${WORKDIR}/rootfs.tar.zst" + sudo ln -Tfsr "${WORKDIR}/rootfs.tar.zst" \ + "${DEPLOY_ISAR_BOOTSTRAP}.tar.zst" fi } diff --git a/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc b/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc index c41dd3a1..f1ad891b 100644 --- a/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc +++ b/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc @@ -11,7 +11,6 @@ inherit deb-dl-dir FILESEXTRAPATHS:append = ":${LAYERDIR_core}/recipes-core/isar-bootstrap/files" -ROOTFSDIR = "${WORKDIR}/rootfs" DISTRO_BOOTSTRAP_BASE_PACKAGES = "locales,apt,usrmerge" BOOTSTRAP_TMPDIR = "${WORKDIR}/tempdir" @@ -73,9 +72,6 @@ do_bootstrap() { fi E="${@ isar_export_proxies(d)}" - sudo rm -rf --one-file-system "${ROOTFSDIR}" - mkdir -p "${ROOTFSDIR}" - if [ "${ISAR_USE_CACHED_BASE_REPO}" = "1" ]; then base_apt_tmp="$(mktemp -d /tmp/isar-base-aptXXXXXXXXXX)" bootstrap_list="${WORKDIR}/sources.list.d/base-apt.list" @@ -181,18 +177,9 @@ do_bootstrap() { "${WORKDIR}/rootfs.tar.zst" \ "$bootstrap_list" - sudo -E -s <<'EOSUDO' - set -e - - tar -xf "${WORKDIR}/rootfs.tar.zst" -C "${ROOTFSDIR}" --exclude="./dev/console" - - # setup chroot - install -v -m755 "${WORKDIR}/chroot-setup.sh" "${ROOTFSDIR}/chroot-setup.sh" - "${ROOTFSDIR}/chroot-setup.sh" "setup" "${ROOTFSDIR}" + # Finalize bootstrap by setting the link in deploy + sudo ln -Tfsr "${WORKDIR}/rootfs.tar.zst" "${DEPLOY_ISAR_BOOTSTRAP}.tar.zst" - # Finalize bootstrap by setting the link in deploy - ln -Tfsr "${ROOTFSDIR}" "${DEPLOY_ISAR_BOOTSTRAP}" -EOSUDO if [ "${ISAR_USE_CACHED_BASE_REPO}" != "1" ]; then deb_dl_dir_export "${WORKDIR}/dl_dir" "${BOOTSTRAP_BASE_DISTRO}-${BASE_DISTRO_CODENAME}" sudo rm -rf --one-file-system "${WORKDIR}/dl_dir" @@ -206,7 +193,7 @@ SSTATEPOSTINSTFUNCS += "bootstrap_sstate_finalize" bootstrap_sstate_prepare() { # this runs in SSTATE_BUILDDIR, which will be deleted automatically - sudo cp -a "$(dirname "${ROOTFSDIR}")/rootfs.tar.zst" ./bootstrap.tar.zst + sudo cp -a "${WORKDIR}/rootfs.tar.zst" ./bootstrap.tar.zst sudo chown $(id -u):$(id -g) bootstrap.tar.zst } @@ -214,8 +201,8 @@ bootstrap_sstate_finalize() { # this runs in SSTATE_INSTDIR # we should restore symlinks after using tar if [ -f bootstrap.tar.zst ]; then - mv bootstrap.tar.zst "$(dirname "${ROOTFSDIR}")/rootfs.tar.zst" - sudo ln -Tfsr "$(dirname "${ROOTFSDIR}")/rootfs.tar.zst" \ + mv bootstrap.tar.zst "${WORKDIR}/rootfs.tar.zst" + sudo ln -Tfsr "${WORKDIR}/rootfs.tar.zst" \ "${DEPLOY_ISAR_BOOTSTRAP}.tar.zst" fi }