From patchwork Fri Sep 4 07:52:47 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Aliaksei Karpovich X-Patchwork-Id: 5422 Return-Path: Received: from shymkent.ilbers.de ([unix socket]) by shymkent (Cyrus 2.5.10-Debian-2.5.10-3+deb9u2) with LMTPA; Fri, 04 Sep 2026 09:56:08 +0200 X-Sieve: CMU Sieve 2.4 Received: from mail-wm1-f58.google.com (mail-wm1-f58.google.com [209.85.128.58]) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPS id 6847u8FZ004327 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Fri, 4 Sep 2026 09:56:08 +0200 Received: by mail-wm1-f58.google.com with SMTP id 5b1f17b1804b1-490a767c7dcsf5248635e9.2 for ; Fri, 04 Sep 2026 00:56:08 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1788508558; cv=pass; d=google.com; s=arc-20260327; b=lk+c0ndfLNDX4B6QI81rYJVE7ir2aPIK0O7rK9WBm+WZ/07bcUsl2aEdtbmQ2Qz9Kb ujuafWbH2euhbx8rg2zF9uclf4SSe16NB4/c8YO+FTRPriHM66oU5VHWrQFOgIzTSeEz xJ8QPDK8VOIFiaTM9attioaFs8q+/Iv17y/UWQrLNCFOV+tSkQ+GDzVebR8PHjBKzHeg GFUHnjTG+VwEAR/XeZ2MG/Ao94mpa51SMq+ge2YvyzW72Q/IUjBK90BTTkEBxQPPTUSx zgtOVPvFV57AHzWn4WpCbpUnb55Cs3xydObJgEaohopZqNd8SIj7Jwc0NXuHaJy0Wf2y RbSQ== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20260327; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:dkim-signature; bh=oYZGu8CNF+UkMI7TSgHZgV9JufgkbvkgJDJaaa/+Wjo=; fh=reUdjcGzWHx943G31/HUdj4f35MoG86Oq3tLKk4PuKw=; b=GwLUM/kPAKOP/b6AMKsMnCUf/WL5/9olzLnjXrk92eFgFtDFY9cWIaig3W0OKITHpZ wcLUFbsS5tED6TodnIbGKjw7uUcypaUb2D1EnXgHvCVLuU1/HuWR8AmGRnvI+K2vIOpu b7jibLuL8+NZJgjRtaxjZ7HjlQm1mYhSSLqRnvevt6ev6EYh43TOusoGsQ5oxECjI9WV J8vgUPSDeJ2qWXDG7G7NJBhLpJlKrKGbJrSWeNPhlK+6lUzm6Is02601Rb0PVsHn6Ag4 wI0nflDn/h3BZ448w3XgMppEjPYn3Tf3EuvPKoNcEXlzX993oQIPV60cqQmPJcAh42w0 DNDg==; darn=isar-build.org ARC-Authentication-Results: i=2; gmr-mx.google.com; spf=pass (google.com: domain of akarpovich@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=akarpovich@ilbers.de DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlegroups.com; s=20251104; t=1788508558; x=1789113358; darn=isar-build.org; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:content-type :x-original-authentication-results:x-original-sender:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from:sender :from:to:cc:subject:date:message-id:reply-to:content-type; bh=oYZGu8CNF+UkMI7TSgHZgV9JufgkbvkgJDJaaa/+Wjo=; b=OmYOvXPprdERUljI3tDSOBScKQC8onQ3q40PCaIUoLLaUNu7KdVJS/xOWIbysoBeXF xUx3FrldJZUJACjWLQPlVkUBXuggrQaupsnr62eswc6BK+yPyzpCxNDPvjAjf0BVlP3Z Esv4t7FHEqVIjzEm0olJNAwuOkKHOhaqAuBzmtthZyT+KSC5KVuYshepY5g6ZKpyLMIB ywAGgckKMU4RGsqDhWiME0fviwT23529wUxTouu0nSCync8/k6OkOOJOthIIyv0nNxN+ WAYhpJAluumPbcBuKkW6n5Pp8bTiGSPy3olc+YAhgedoVscuQYrOnKDE1T7SV0D4nycA iSlQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1788508558; x=1789113358; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :x-spam-checked-in-group:list-id:mailing-list:precedence :content-type:x-original-authentication-results:x-original-sender :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:x-beenthere:x-gm-message-state:sender:from:to:cc:subject:date :message-id:reply-to:content-type; bh=oYZGu8CNF+UkMI7TSgHZgV9JufgkbvkgJDJaaa/+Wjo=; b=ErMhjxetPq6iXlIFeKUSORW8CbjDjIkhhNFFBpKgC1m7UU8lLtFgOnjdMpymuKIWPS zJp/r9UVTzf9jPFR9XHuuexVLwbI7aY3XxeN8xb1HGjnBGFYdren8hq4X6N7btnUm3ug t1sPCoFOe9owq11px5nLNOYnLnT4T1Ki3rqDFzq/xopd7gofExrz7dZVsIt7JBNg39Tv F1yihxz2APuVXwdNV9k9gbKRNR7IEI9iGfTr+UmxInnegOV3OhUW5+1OMLe6/abHfeqm SQPrK6eOEx6SS/CzE6J/Myi/r3qi13xDwBRV6v+VKCDkf77cSU2Wc+OpP5tI/ooHmhHf kxoQ== Sender: isar-users@googlegroups.com X-Forwarded-Encrypted: i=2; AKwUvBz1rQKPHXJ/JDRBP2Lm/TQgVb4D4iuNq7+9sUhl9ciGY9n9P1z/DBXoLSs+qwoH/c+kmuSr2Y0=@isar-build.org X-Gm-Message-State: AFuF++ldtHl7TbV9lC+O57vLg58typ48MoBBg9AFpZYXm0Zbn0fTeP4N wKzTe54DSquNUfh14it8QGPZw6WP8ARbfNNLEBqgbbmC7zfdByPTJuxN X-Received: by 2002:a05:600c:8716:b0:49a:a101:4157 with SMTP id 5b1f17b1804b1-49cf81f4924mr45843235e9.7.1788508557835; Fri, 04 Sep 2026 00:55:57 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com; h="ATskLdclBycwh5OFZyO2ds4yuet3M7zSQqrE7lPTG8aG8tIzSQ==" Received: by 2002:a05:600c:1f82:b0:49c:f166:cf1a with SMTP id 5b1f17b1804b1-49cf7b6db17ls5588315e9.2.-pod-prod-06-eu; Fri, 04 Sep 2026 00:55:55 -0700 (PDT) X-Received: by 2002:a05:600c:608a:b0:49c:fc6c:bdfd with SMTP id 5b1f17b1804b1-49cfc6cbfe7mr14067415e9.20.1788508555500; Fri, 04 Sep 2026 00:55:55 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1788508555; cv=none; d=google.com; s=arc-20260327; b=qrvQNvzGdwPZ+TaCrKcI7aIw/N6BQ8yAiJNp4ViBPvB3cQUfYZv/izVM7WR6VFz7k0 i6papOVtLTbaGEkY8HgqFgt2hmyZ8HcSKNlx5Kt5Z9uUoCZMfhMI89Ev7EkGGN06CR+4 A+dDAe90K96KP9Wt3ULQGtN6w3Nza2BgcycB6Hs+J8RBGVFU7TshjdmMCyUPSV5CUmth ZuoTxC5Ged2ncH+kCC7J1VLIuLvh5nHWQE5C4GWQ/CUeX63L4p+buE2ZgYXeSHcUYGu/ tJXZ34OKMvPlM07qPS+93q9lZY9+LWmmvLT3PyXh5d4mFNpn0O8u5Dgljd7H1df5oXZY lXrw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20260327; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=MIrzxeLR+T1X+QeRGxogTFaDmrhkI2/QpRguUHLC7fI=; fh=IIbR0JxonYgzdrHDdB9TmZniSh/o1z0HofhfbmLXp8s=; b=rifiW1DCEwjJN27inFVwkHZDdXU8q77KhyyoSbzNDrQ+rExFWHjLtDFt38iIeeHVur zJnXE2V5m1fFGaoL4n0LDdMa/kQdk/jS5fb3HS2haafw9Qx0UPd6mtyMx/rCvP2XF/Uo Y97AzfSIqzb9OZgVNtuQdNCxDgmzbUfx219O3wmCaEu2g2W+3VAKNcEh4BoAGyJ0BPW2 DTl4taDEKmDpHvNqpOdxbxEPsu2tdJsUslDMUTkB1zC31fhlrygqYdxmcJvqf5uEqR+I YMenXZgB3I0Z/46qQ3wJN9xFraRIz+ICau4XBp1soqVJPRPYNGd4HZUR2YixDhDyeBNZ YKmw==; dara=google.com ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of akarpovich@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=akarpovich@ilbers.de Received: from shymkent.ilbers.de (shymkent.ilbers.de. [85.214.156.166]) by gmr-mx.google.com with ESMTPS id 5b1f17b1804b1-49cf76eef24si378015e9.2.2026.09.04.00.55.55 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 04 Sep 2026 00:55:55 -0700 (PDT) Received-SPF: pass (google.com: domain of akarpovich@ilbers.de designates 85.214.156.166 as permitted sender) client-ip=85.214.156.166; Received: from akarpovich-Victus-by-HP-Laptop-16-e0xxx.promwad.corp (62.122.174.6.serverel.net [62.122.174.6] (may be forged)) (authenticated bits=0) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPSA id 6847tRSF004124 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 4 Sep 2026 09:55:53 +0200 From: Aliaksei Karpovich To: isar-users@googlegroups.com Cc: Uladzimir Bely , Aliaksei Karpovich Subject: [v10 05/12] base-apt: Predownload packages to base-apt before install Date: Fri, 4 Sep 2026 09:52:47 +0200 Message-ID: <20260904075515.25962-6-akarpovich@ilbers.de> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260904075515.25962-1-akarpovich@ilbers.de> References: <20260904075515.25962-1-akarpovich@ilbers.de> MIME-Version: 1.0 X-Spam-Status: No, score=-4.6 required=5.0 tests=DKIMWL_WL_MED,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_EF,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,RCVD_IN_DNSWL_BLOCKED,RCVD_IN_MSPIKE_H2, RCVD_IN_RP_CERTIFIED,RCVD_IN_RP_RNBL,RCVD_IN_RP_SAFE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on shymkent.ilbers.de X-Original-Sender: akarpovich@ilbers.de X-Original-Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of akarpovich@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=akarpovich@ilbers.de Precedence: list Mailing-list: list isar-users@googlegroups.com; contact isar-users+owners@googlegroups.com List-ID: X-Spam-Checked-In-Group: isar-users@googlegroups.com X-Google-Group-Id: 914930254986 List-Post: , List-Help: , List-Archive: , List-Unsubscribe: , X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= From: Uladzimir Bely This patch uses debrepo script to predownload packages to base-apt repository before they are installed in rootfs. Signed-off-by: Uladzimir Bely Signed-off-by: Aliaksei Karpovich --- meta/classes-recipe/crossvars.bbclass | 1 + meta/classes-recipe/dpkg-base.bbclass | 1 + meta/classes-recipe/dpkg.bbclass | 8 ++++ .../image-locales-extension.bbclass | 5 ++ .../image-tools-extension.bbclass | 13 ++++++ meta/classes-recipe/rootfs.bbclass | 5 +- meta/lib/aptsrc_fetcher.py | 46 +++++++++++++++---- .../sbom-chroot/sbom-chroot.bb | 2 + .../sbuild-chroot/sbuild-chroot-host.bb | 2 + 9 files changed, 74 insertions(+), 9 deletions(-) diff --git a/meta/classes-recipe/crossvars.bbclass b/meta/classes-recipe/crossvars.bbclass index 7cf2d660..6f070714 100644 --- a/meta/classes-recipe/crossvars.bbclass +++ b/meta/classes-recipe/crossvars.bbclass @@ -30,6 +30,7 @@ python __anonymous() { schroot_dir = d.getVar('SCHROOT_HOST_DIR', False) sbuild_dep = "sbuild-chroot-host" + flavor_suffix + ":do_build" sdk_toolchain = "crossbuild-essential-" + distro_arch + d.setVar('DEBREPO_WORKDIR', d.getVar('DEBREPO_HOST_DIR')) else: d.setVar('BUILD_ARCH', distro_arch) schroot_dir = d.getVar('SCHROOT_TARGET_DIR', False) diff --git a/meta/classes-recipe/dpkg-base.bbclass b/meta/classes-recipe/dpkg-base.bbclass index 4a81787e..91b17768 100644 --- a/meta/classes-recipe/dpkg-base.bbclass +++ b/meta/classes-recipe/dpkg-base.bbclass @@ -11,6 +11,7 @@ inherit terminal inherit repository inherit deb-dl-dir inherit essential +inherit debrepo # Local (WORKDIR-internal) dir where do_dpkg_build collects the built debs. # Its content is exported into the shared, sstate-tracked DEPLOY_DIR_DEB so diff --git a/meta/classes-recipe/dpkg.bbclass b/meta/classes-recipe/dpkg.bbclass index a42703d5..323202e4 100644 --- a/meta/classes-recipe/dpkg.bbclass +++ b/meta/classes-recipe/dpkg.bbclass @@ -117,6 +117,12 @@ dpkg_runbuild() { echo '$stalled_pkg_timeout = ${DPKG_BUILD_TIMEOUT};' >> ${SBUILD_CONFIG} DSC_FILE=$(find ${DEPLOY_DIR_SRC} -maxdepth 1 -name "${DEBIAN_SOURCE}_*.dsc" -print) + debrepo_parse_dscfile "${DSC_FILE}" + + locked_update_cmd=":" + if [ "${ISAR_PREFETCH_BASE_APT}" = "1" ]; then + locked_update_cmd="flock -x /base-apt/repo.lock -c 'apt-get -y update'" + fi # networking is automatically enabled on older versions of sbuild sbuild_network_option="" @@ -139,9 +145,11 @@ dpkg_runbuild() { --chroot-setup-commands="rm -f /var/log/dpkg.log" \ --chroot-setup-commands="mkdir -p ${deb_dir}" \ --chroot-setup-commands="find ${ext_deb_dir} -maxdepth 1 -name '*.deb' -exec ln -t ${deb_dir}/ -sf {} +" \ + --chroot-setup-commands="${locked_update_cmd}" \ --chroot-setup-commands="apt-get update -o Dir::Etc::SourceList=\"sources.list.d/isar-apt.list\" -o Dir::Etc::SourceParts=\"-\" -o APT::Get::List-Cleanup=\"0\"" \ --finished-build-commands="rm -f ${deb_dir}/sbuild-build-depends-*-dummy_*.deb" \ --finished-build-commands="find ${deb_dir} -maxdepth 1 -type f -name '*.deb' -print -exec cp ${CP_FLAGS} -t ${ext_deb_dir}/ {} +" \ + --finished-build-commands="mkdir -p ${ext_root}" \ ${@ '--finished-build-commands="cp /var/log/dpkg.log $ext_root/dpkg_partial.log"' if d.getVar('ISAR_CHROOT_MODE') == 'schroot' else '' } \ --build-path="" --build-dir=${WORKDIR} --dist="${DEBDISTRONAME}" ${DSC_FILE} diff --git a/meta/classes-recipe/image-locales-extension.bbclass b/meta/classes-recipe/image-locales-extension.bbclass index c03f34c0..32b5e19c 100644 --- a/meta/classes-recipe/image-locales-extension.bbclass +++ b/meta/classes-recipe/image-locales-extension.bbclass @@ -6,6 +6,8 @@ # This class extends the image.bbclass for setting locales and purging unneeded # ones. +inherit debrepo + LOCALE_GEN ?= "en_US.UTF-8 UTF-8\n\ en_US ISO-8859-1\n" LOCALE_DEFAULT ?= "en_US.UTF-8" @@ -29,6 +31,9 @@ ROOTFS_INSTALL_COMMAND_BEFORE_EXPORT += "image_install_localepurge_download" image_install_localepurge_download[weight] = "40" image_install_localepurge_download[network] = "${TASK_USE_NETWORK_AND_SUDO}" image_install_localepurge_download() { + debrepo_add_packages "${DEBREPO_WORKDIR}" "localepurge" + debrepo_update_apt_source_list "${ROOTFSDIR}" "base-apt" + run_privileged_heredoc <<'EOF' set -e ${@insert_isar_mounts(d, d.getVar('ROOTFSDIR'), d.getVar('ROOTFS_MOUNTS') if d.getVar('ISAR_CHROOT_MODE') == 'unshare' else '')} diff --git a/meta/classes-recipe/image-tools-extension.bbclass b/meta/classes-recipe/image-tools-extension.bbclass index 8f666444..229ce318 100644 --- a/meta/classes-recipe/image-tools-extension.bbclass +++ b/meta/classes-recipe/image-tools-extension.bbclass @@ -6,6 +6,11 @@ # This file extends the image.bbclass to supply tools for futher imager functions inherit sbuild +inherit debrepo + +python __anonymous() { + d.setVar('DEBREPO_WORKDIR', d.getVar('DEBREPO_TARGET_DIR')) +} IMAGER_INSTALL ??= "" IMAGER_BUILD_DEPS ??= "" @@ -60,12 +65,20 @@ imager_run_schroot() { echo "Installing imager deps: ${local_install}" distro="${BASE_DISTRO}-${BASE_DISTRO_CODENAME}" + debrepo_workdir=${DEBREPO_TARGET_DIR} if [ ${ISAR_CROSS_COMPILE} -eq 1 ]; then distro="${HOST_BASE_DISTRO}-${BASE_DISTRO_CODENAME}" + if [ ${HOST_ARCH} != ${DISTRO_ARCH} ]; then + debrepo_workdir=${DEBREPO_HOST_DIR} + fi fi E="${@ isar_export_proxies(d)}" deb_dl_dir_import ${schroot_dir} ${distro} + + debrepo_add_packages --isarapt "${debrepo_workdir}" "${local_install}" + debrepo_update_apt_source_list "${schroot_dir}" "base-apt" + ${SCRIPTSDIR}/lockrun.py -r -f "${REPO_ISAR_DIR}/isar.lock" -s <