From patchwork Fri Oct 2 07:01:05 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: akarpovich X-Patchwork-Id: 5514 Return-Path: Received: from shymkent.ilbers.de ([unix socket]) by shymkent (Cyrus 2.5.10-Debian-2.5.10-3+deb9u2) with LMTPA; Fri, 02 Oct 2026 09:05:07 +0200 X-Sieve: CMU Sieve 2.4 Received: from mail-ed1-f59.google.com (mail-ed1-f59.google.com [209.85.208.59]) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPS id 69275631003866 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Fri, 2 Oct 2026 09:05:06 +0200 Received: by mail-ed1-f59.google.com with SMTP id 4fb4d7f45d1cf-6ae14ecef5fsf2925204a12.0 for ; Fri, 02 Oct 2026 00:05:06 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1790924701; cv=pass; d=google.com; s=arc-20260327; b=OH7c99y9AsSvAJxCY/UThGd31YfQrTU32wNS3Y6DUgZPND3tVeo40LrzzGZWM+r+qy /CzRtZ5/pdZDcb8klObtrABWU+9BHH4HHHVVre2/48jqP0nsvSC9kJM6KC+28YJjFrdY Las4qZAWJdZJWeGCE5cJajpIhFN4mXJuMuDhLN4ZVdIwRmJZn6DHo6WP7FpisrwQSBUj bRZT5LP09n+YpBHPrU5EtwTdKPkyRoDCcFxBpDvM85vi0rqK9uyBpQAQHzNxzU8P4rEx qvgslT01eXb7rTmr9fBAKHxdFrU2NHiBfLJIcdR2DxPzepiSBCbFqb6HQ9zHLz3+is7d RiYw== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20260327; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:dkim-signature; bh=FpEt696zox+ITlWimkLnHzEIdSBWe6elJYgguk63Ql0=; fh=hma52+qoUiLod726vln0EWmnshkzvKwlST1gbiniW5M=; b=D0Jt/kYBJ6Rx/2Z+7/bsLP3+zAzHQOf7VAucIHmt49ylAhFjM6nzoJeNMs88klq22d e7OvRA5YMZF3nVrlatshHm2R5paav1rM9LhIm/m7E8R6ZpzKHhL/nU6S07+2AHg2kjtQ LB/LWUP3bkNGyTNxH6kHfEOPWwulvIMEBkA61KToDrwu56jyXAublrgM+CxXG9Lbxtz7 Il7w9a9lG3UaEqNILadj5Fsb8uGoTWsy1fHhPQ3sP2MNdrv0psCCod9LxHfzpC/bG+8H 1W/bg7b/H2LiaIEedwZXRewm+j6m2YRHlzZakfBT4H79Syi48vLNm9FqBhXpSKFg9aXe vcbA==; darn=isar-build.org ARC-Authentication-Results: i=2; gmr-mx.google.com; spf=pass (google.com: domain of akarpovich@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=akarpovich@ilbers.de DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlegroups.com; s=20251104; t=1790924701; x=1791529501; darn=isar-build.org; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:content-type :x-original-authentication-results:x-original-sender:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from:sender :from:to:cc:subject:date:message-id:reply-to:content-type; bh=FpEt696zox+ITlWimkLnHzEIdSBWe6elJYgguk63Ql0=; b=oTuOg+TBAZKZL1M5lPONEbcGQP2zFKEkd7z/dsd7ipDD6v2tKi4jdX99sMF0jJkCEt OH9uma8bUTTo6K+6cWcUbkM05moOdogaO7jYHJ5zI1KPlv7ZniVqYU3ACqKIJDPWZ5v7 guq0Mg/9wzMHCLSsThKRNI2efmgwU/LKgLEw35JFBz+0LPvl3bHu8DJv1ZifXjai05Er ssZyDKDN8a4flEB1ICWtVc6FAM96Bss0Jb+/+cxq4QP+HXyW9miYNkRs6yX6GWGNLaiJ QrzIRZAD2H6nU3q58ghd3190nIWoCHUps8y9uKUNsSql5rbsZd1XBVLEtlP8hn0DSRTd yeIg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20260707; t=1790924701; x=1791529501; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :x-spam-checked-in-group:list-id:mailing-list:precedence :content-type:x-original-authentication-results:x-original-sender :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:x-beenthere:x-gm-message-state:sender:from:to:cc:subject:date :message-id:reply-to:content-type; bh=FpEt696zox+ITlWimkLnHzEIdSBWe6elJYgguk63Ql0=; b=lTHgFyaoxslARUV9wkHFKivpAYq4c7vfosx4J/HBZ9P0prvR1e+Tl8UtAqZ/0X+pQ6 D7loD7nQ6ksWMG0A6o8m8VPX0BNBpiUYQZDqOJBURXzewuYY5o3H7KDfJQn8C1q2Joq/ Ig9zQ6PUHIXyjjizwWCZ558EAxna5aWL5neJHj48n+zNQ1ilrcGRqcYiN2dV7vFlIUy2 k9AIlwmouIuAPkPd7b2gKUSXqb/zEXpPItnr/COL7+XuCGL4oU6MwHNZ4UulH0TnoW8k VhI5Z0vwCaF3PD81NUSf1TDOa63R5iByQLex6kU9w1UD18Lz8gSVCnwcACF2rHms/sZQ 81MQ== Sender: isar-users@googlegroups.com X-Forwarded-Encrypted: i=2; AKwUvBx4KNQYASWC1Xuc+cxJyTSmqj2XHQ4ZWKq4yuW26o8Yu2g/AUuxCTqflxbUV7XytUFWZQfDP8Q=@isar-build.org X-Gm-Message-State: AFq9FYKE82ilOK4XZKebwYiv9i3UY4vxw+7ieNpane50Pn+IjGI4qgoN jB0Su6H+EcN2oRL1H9XhB2Y8QutGux1QVWbznBJtbCePrIwkk64QjAQq X-Received: by 2002:a05:6402:2790:b0:6ac:9a44:d404 with SMTP id 4fb4d7f45d1cf-6af9de99262mr1019184a12.0.1790924700514; Fri, 02 Oct 2026 00:05:00 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com; h="ATskLdcIstwvxsj898iwa6OpTFLHGMs3H87dyuQk4219OYmu/g==" Received: by 2002:a05:6402:24cc:b0:6aa:448f:b889 with SMTP id 4fb4d7f45d1cf-6af9dab6cd6ls521974a12.2.-pod-prod-05-eu; Fri, 02 Oct 2026 00:04:56 -0700 (PDT) X-Received: by 2002:a05:6402:3507:b0:6a7:ea54:38f with SMTP id 4fb4d7f45d1cf-6af9e318369mr1061654a12.32.1790924695778; Fri, 02 Oct 2026 00:04:55 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1790924695; cv=none; d=google.com; s=arc-20260327; b=P5h+h/qeykX8lI7M/fBY1XlsE3k1iW+EG19sAPg5DOyb9ovpFhlI5Psq9l8/U/gRM/ OLnrMfbydCmcchRuEHe+NTs5Q4BFlLKOhAEXOrjwACC1Phl2/wvQgxFPTF2xxlU7Wo+r xobPcwkP6X8QqeLZrJdRyNen0zU4IgSb7dtbaL/EMTEvEm7iCYOJz5T0YXBOw0LIPvDF 87BUT7vxMRtLljWTyVgNMZj/2/QzJH66Kt4IUBAcXIxjlMPcIH/InDbsCqjvS929kk2d ZP378G1MZFDHIGEOqX2ODwkQQo88V9KSdzbUgwDP2o9T0Y1xeHXB9YiBcQTkyA4VwGHe sMUA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20260327; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=DPK7XNfXmH1ik4kBjtPUHNR68d7Zw/Zr6vOXQ1ei14g=; fh=IIbR0JxonYgzdrHDdB9TmZniSh/o1z0HofhfbmLXp8s=; b=ioejjC6g9cCgcFZ07zl/R11/Ttb20cJZB/lDDr+EG2L+na4BEnIlQdpfqQM4OFy/pu agUKNS8I7zIdhUJAJ6nhhJUCCGYpA0HDCoYRGC1WaBLHPS9LOKk1QkrNpSjMmRiHysyB szkvrYV2r2tkTePBICdCs730f1bp8RU6iGdwcgZfvJmGsnjw0BkLiHt1y8xf52w+Gt4r IFL/TC6ftIYUKZNsrP4iPCu4nWdU/95gUBdykG4XQOb5XHX7P6NkWU6CicViKPbgqyVv tmzqc7v2G9/z9xYQ+T93+wt0ibLMguGG8SAtpQy+BYK+oaq/d62vPDokTKd8rBq77oW9 aRTw==; dara=google.com ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of akarpovich@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=akarpovich@ilbers.de Received: from shymkent.ilbers.de (shymkent.ilbers.de. [85.214.156.166]) by gmr-mx.google.com with ESMTPS id 4fb4d7f45d1cf-6af9e269409si29765a12.8.2026.10.02.00.04.55 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 02 Oct 2026 00:04:55 -0700 (PDT) Received-SPF: pass (google.com: domain of akarpovich@ilbers.de designates 85.214.156.166 as permitted sender) client-ip=85.214.156.166; Received: from laptop-0xxx (62.122.174.6.serverel.net [62.122.174.6] (may be forged)) (authenticated bits=0) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPSA id 69274a1h003799 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 2 Oct 2026 09:04:54 +0200 From: akarpovich To: isar-users@googlegroups.com Cc: Uladzimir Bely , Aliaksei Karpovich Subject: [v11 02/12] meta: Add debrepo bbclass handling base-apt prefetching Date: Fri, 2 Oct 2026 09:01:05 +0200 Message-ID: <20261002070335.26526-3-akarpovich@ilbers.de> X-Mailer: git-send-email 2.53.0 In-Reply-To: <20261002070335.26526-1-akarpovich@ilbers.de> References: <20261002070335.26526-1-akarpovich@ilbers.de> MIME-Version: 1.0 X-Spam-Status: No, score=-4.6 required=5.0 tests=DKIMWL_WL_MED,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_EF,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,RCVD_IN_DNSWL_BLOCKED,RCVD_IN_MSPIKE_H2, RCVD_IN_RP_CERTIFIED,RCVD_IN_RP_RNBL,RCVD_IN_RP_SAFE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on shymkent.ilbers.de X-Original-Sender: akarpovich@ilbers.de X-Original-Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of akarpovich@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=akarpovich@ilbers.de Precedence: list Mailing-list: list isar-users@googlegroups.com; contact isar-users+owners@googlegroups.com List-ID: X-Spam-Checked-In-Group: isar-users@googlegroups.com X-Google-Group-Id: 914930254986 List-Post: , List-Help: , List-Archive: , List-Unsubscribe: , X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= From: Uladzimir Bely This class uses 'scripts/debrepo' python script to prefetch given packages or sources to local base-apt repository. Signed-off-by: Uladzimir Bely Signed-off-by: Aliaksei Karpovich --- RECIPE-API-CHANGELOG.md | 10 ++++ doc/user_manual.md | 2 + meta/classes/debrepo.bbclass | 89 ++++++++++++++++++++++++++++++++++++ meta/conf/bitbake.conf | 5 ++ 4 files changed, 106 insertions(+) create mode 100644 meta/classes/debrepo.bbclass diff --git a/RECIPE-API-CHANGELOG.md b/RECIPE-API-CHANGELOG.md index abb67fa8..1951df08 100644 --- a/RECIPE-API-CHANGELOG.md +++ b/RECIPE-API-CHANGELOG.md @@ -1211,3 +1211,13 @@ ROOTFS_FEATURES:remove = "clean-apt-credentials" All `ROOTFS_POSTPROCESS_COMMAND` steps are now executed within the `do_rootfs_install` task to avoid idempotency issues on partial rebuilds. Task that previously had a dependency to `do_rootfs_postprocess` shall now be changed to run after `do_rootfs_install`. + +### "Prefetch" mode for base-apt + +Originally, `base-apt` repo is created only during second build when variable +ISAR_USE_CACHED_BASE_REPO is set. The repo is populated with every package that +took part in the first build and was cached in DL_DIR. + +New ISAR_PREFETCH_BASE_APT variable changes the way `base-apt` is populated. +Packages added to the repo before running any task that need them. Separate +`deb-repo` script is used for populating base-apt repo. diff --git a/doc/user_manual.md b/doc/user_manual.md index a134903d..6db4b8eb 100644 --- a/doc/user_manual.md +++ b/doc/user_manual.md @@ -86,12 +86,14 @@ apt install \ gettext-base \ git \ python3 \ + python3-apt \ quilt \ qemu-user-static \ reprepro \ sudo \ uidmap \ unzip \ + wget \ xz-utils \ git-buildpackage \ pristine-tar \ diff --git a/meta/classes/debrepo.bbclass b/meta/classes/debrepo.bbclass new file mode 100644 index 00000000..79a19144 --- /dev/null +++ b/meta/classes/debrepo.bbclass @@ -0,0 +1,89 @@ +# This software is a part of Isar. +# Copyright (C) 2026 ilbers GmbH +# +# SPDX-License-Identifier: MIT + +# Prefetch to base-apt repo by default +ISAR_PREFETCH_BASE_APT ??= "1" + +DEBREPO_WORKDIR ??= "${DEBREPO_TARGET_DIR}" + +debrepo_update_apt_source_list() { + [ "${ISAR_PREFETCH_BASE_APT}" != "1" ] && return + + chroot_dir=${1} + apt_list=${2} + + flock -x "${REPO_BASE_DIR}/repo.lock" -c " + sudo -E chroot ${chroot_dir} /usr/bin/apt-get update \ + -o Dir::Etc::SourceList=\"sources.list.d/${apt_list}.list\" \ + -o Dir::Etc::SourceParts=\"-\" \ + -o APT::Get::List-Cleanup=\"0\" + " +} + +debrepo_add_packages() { + [ "${ISAR_PREFETCH_BASE_APT}" != "1" ] && return + [ "${ISAR_USE_CACHED_BASE_REPO}" = "1" ] && return + + args="" + if [ "${1}" = "--srcmode" ]; then + args="${args} --srcmode" + shift + fi + + if [ "${1}" = "--isarapt" ]; then + args="${args} --extrarepo=${REPO_ISAR_DIR}/${DISTRO}" + shift + fi + + workdir="${1}" + args="${args} ${2}" + + if [ -n "${GNUPGHOME}" ]; then + export GNUPGHOME="${GNUPGHOME}" + else + export GNUPGHOME="${WORKDIR}/gpghome" + fi + + ${SCRIPTSDIR}/deb-repo \ + --workdir="${workdir}" \ + ${args} +} + +debrepo_parse_dscfile() { + [ "${ISAR_PREFETCH_BASE_APT}" != "1" ] && return + [ "${ISAR_USE_CACHED_BASE_REPO}" = "1" ] && return + + dscfile="${1}" + args="" + + if [ "${ISAR_CROSS_COMPILE}" = "1" ]; then + build_arch=${HOST_ARCH} + if [ "${PACKAGE_ARCH}" != "${build_arch}" ]; then + args="--crossbuild \ + crossbuild-essential-${PACKAGE_ARCH}:${build_arch} \ + dose-distcheck:${build_arch} \ + libc-dev:${PACKAGE_ARCH} \ + libstdc++-dev:${PACKAGE_ARCH} \ + " + fi + fi + + args="${args} --extrarepo=${WORKDIR}/isar-apt/${DISTRO}-${DISTRO_ARCH}/apt/${DISTRO}" + + if [ -n "${GNUPGHOME}" ]; then + export GNUPGHOME="${GNUPGHOME}" + else + export GNUPGHOME="${WORKDIR}/gpghome" + fi + + if [ -n "${DEB_BUILD_PROFILES}" ]; then + export DEB_BUILD_PROFILES="${DEB_BUILD_PROFILES}" + fi + + ${SCRIPTSDIR}/deb-repo \ + --workdir="${DEBREPO_WORKDIR}" \ + --dscfile="${dscfile}" \ + ${args} +} diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf index e69ec3bb..c8503ec6 100644 --- a/meta/conf/bitbake.conf +++ b/meta/conf/bitbake.conf @@ -67,6 +67,11 @@ SDKCHROOT_DIR = "${DEPLOY_DIR_SDKCHROOT}/${BPN}-${DISTRO}-${MACHINE}" CACHE = "${TMPDIR}/cache" KERNEL_FILE ?= "${@ 'vmlinux' if d.getVar('DISTRO_ARCH') in ['mipsel', 'riscv64', 'arm64'] else 'vmlinuz'}" +# debrepo config +DEBREPO_DIR = "${TOPDIR}/debrepo" +DEBREPO_HOST_DIR = "${DEBREPO_DIR}/${HOST_DISTRO}-${HOST_ARCH}_${DISTRO}-${DISTRO_ARCH}" +DEBREPO_TARGET_DIR = "${DEBREPO_DIR}/${DISTRO}-${DISTRO_ARCH}" + MACHINEOVERRIDES ?= "${MACHINE}" DISTROOVERRIDES ?= "${DISTRO}" OVERRIDES = "${PACKAGE_ARCH}:${MACHINEOVERRIDES}:${DISTROOVERRIDES}:${BASE_DISTRO_CODENAME}::${BASE_DISTRO}:${ISAR_CHROOT_MODE}:forcevariable"