From patchwork Fri Sep 4 07:52:43 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Aliaksei Karpovich X-Patchwork-Id: 5418 Return-Path: Received: from shymkent.ilbers.de ([unix socket]) by shymkent (Cyrus 2.5.10-Debian-2.5.10-3+deb9u2) with LMTPA; Fri, 04 Sep 2026 09:56:04 +0200 X-Sieve: CMU Sieve 2.4 Received: from mail-lf1-f60.google.com (mail-lf1-f60.google.com [209.85.167.60]) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPS id 6847tvdn004164 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Fri, 4 Sep 2026 09:55:57 +0200 Received: by mail-lf1-f60.google.com with SMTP id 2adb3069b0e04-5b2a9f6b0b2sf402366e87.2 for ; Fri, 04 Sep 2026 00:55:57 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1788508551; cv=pass; d=google.com; s=arc-20260327; b=ZEGU1S3wEPoeiu6+4hvWEpP8s9Xvh6Kfdo4JchGmUm5H0Ritt0oKX2zv+5ojgNpepW KpWkBZZIsiUWUcu0BwHXj+iwNL9HrRYnWidA+ZZXUW1sJQIfIwUvTXWPNv3InbV6zubV 8CyW5NmapIgAjKtFvjF5qDL5hvgLpKfILtiaSTuHWdJGBvt99sybwJ7RaZy0CSTDuNF+ Wxom5YvVMpFffEg2RHF583UOpux5EYdlLeLuAvOARu1zO5Lk0B5q1uy8uIBMT1cYrTdP xeAOs7YzfNrZ2c9NqP/Em62/tnltX4wSykbKgRUtnVBA8f8qcqS3FNQu2i7S4Zcgqd5h 2sQw== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20260327; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:dkim-signature; bh=7fAyHkSoAmhfswHvQAyG88O92Fakb0Mo6ze4NkHfKiQ=; fh=AnSiHhjNxwWOGUudTO1tooL/wWXMHorJuJ9gQ49vWJ8=; b=q7VWmZ7+PAQKlJWaUTTCv+omIPja1E606te1FxeO+MpA9LGecVPnBU7G5V6ec1UgT1 u69K1dLrfRCjLRVOl0PTgZCCabDopeOgmfYX9c0VIdF+5pb2pTy7r6948ZAlgYiGsP51 54YQSfzo+n86VkyEdFflR7u3QNf/FQCJGzmDajVYzew/OO1coaGq5aUxtOhuL6bwE0o5 kPV2f+ELdDIXYzxZEaQLnZ6/Czib2+2Fbod5WZbn7TToUGEE5hgTP2pnEjYOcPmkQigF r6I0DVR9KIf3tQq4TYBjE6r2N1UfJ2G47IlobMKg7m1N1utiGjyAsOlym4cK308bgZ8O 55eA==; darn=isar-build.org ARC-Authentication-Results: i=2; gmr-mx.google.com; spf=pass (google.com: domain of akarpovich@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=akarpovich@ilbers.de DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlegroups.com; s=20251104; t=1788508551; x=1789113351; darn=isar-build.org; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:content-type :x-original-authentication-results:x-original-sender:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from:sender :from:to:cc:subject:date:message-id:reply-to:content-type; bh=7fAyHkSoAmhfswHvQAyG88O92Fakb0Mo6ze4NkHfKiQ=; b=ydgBb0LAgNJKrk6p7k4Y9ld/4zNATIs9AuwvkYY+mwRWNIJkET68+sOrMakv+Olh/w M0aNrw/jxQMSjedMuJSO4uCa8ESggczLj8RFJt0QruKFcKFfCxWhUsHa9FZ2rAFEnuab Wm2kOnRvozeeKF5XP2cLReIRPfUYqakYHGPpYFslFe0B/G2/fmMm+pY8CQSaC5vNYBGT aRCs2Fgq7v2dGyaeYY1W6bOGtznIYjlGZ1aCL0LrQXOfnGWBYkmhD+rK+eAjDrlHOy6Z +X3Rm/rD+xgL7vkTVSmY3SqjkglxhHN11T/Ur5vqNXCQVQ6EBHVkqAx+tONhlchfab1m Ba6g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1788508551; x=1789113351; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :x-spam-checked-in-group:list-id:mailing-list:precedence :content-type:x-original-authentication-results:x-original-sender :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:x-beenthere:x-gm-message-state:sender:from:to:cc:subject:date :message-id:reply-to:content-type; bh=7fAyHkSoAmhfswHvQAyG88O92Fakb0Mo6ze4NkHfKiQ=; b=jDSZwlGVpBJjYfQaAOP0/UQcCgmCs9EJQ32bUC/Tbw0S73TpgOZosekZn/+T6oYgkf rmAjynNFTWiYr72M3641CboyLJzdakXRSxHCBaEPh5KqPo4Rw0Thly3Ivy5ZX4oQASpU QigcllYp5+15+bj8LEFlvJL5hBvDTW9ttKeNZMR8Rt4FHAlkLcIVvOjAv/njcpMFf5CZ WddMfiN7oGcsDxOEx3hkunewuWeUiK/aHQHYJLCJGnyYyL4VXme1GrsDppBWvZipRbcM qI0zJT4zuu0ECFXjfopoqaVsPO67e1PCdp/sOOP6IA0eTjay4kUuDKditwelzF/Ha7QG lCmQ== Sender: isar-users@googlegroups.com X-Forwarded-Encrypted: i=2; AKwUvBwZVJzoIEIjUz9EcYBsjKO+IjZH+lmi6Ts4dYE2mv7aojkkm/JxQ/xMFKuYCU7zpomkOkYfobo=@isar-build.org X-Gm-Message-State: AFuF++nR+cUPRnEebKJB0dUOADQWs/kFF+93/KbZfcqCBi5hMMWD22Hy RBL36RU3r/86ZRCZ7gtczWy0ktcX4iIUAZkSwCwj9IKo7i+FSTBTa/l7 X-Received: by 2002:a05:6512:15a9:b0:5b6:4ae:5b7f with SMTP id 2adb3069b0e04-5b617071b19mr903747e87.29.1788508550922; Fri, 04 Sep 2026 00:55:50 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com; h="ATskLdfiE8c5OXfT3A5aTklHbkGLpzCPMSY+KLnHpHx+k0SQMA==" Received: by 2002:a05:6512:834f:b0:5b2:a215:a2aa with SMTP id 2adb3069b0e04-5b615cb8f26ls189863e87.1.-pod-prod-06-eu; Fri, 04 Sep 2026 00:55:48 -0700 (PDT) X-Received: by 2002:a05:651c:221f:b0:3a3:42cf:c035 with SMTP id 38308e7fff4ca-3a371aa374cmr5259991fa.2.1788508548210; Fri, 04 Sep 2026 00:55:48 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1788508548; cv=none; d=google.com; s=arc-20260327; b=nN0owDftto32ZJn35Q7ATxZg1LfNh88wyL2Dwvjq9ZcrY9smi5IFZDoeqVboxKuMcs RJEBBkkyVTSg4RDCMq3Ind3waPDSJDpOGMPPPkfCBK02YtR6Ty2ZtnGzQIaU214L7B0G 4iSA+SXWoj1B37mwpbIM7AGIRFoTvTCa7sG6E2PR1O6oIoNXDsDcvJO8eGE2nxBxOGpY gC792JR0FJUYHme3tZKqD9O1+IF598vJpZQ9I+Pfh12hDk1wtm+33Td+xN5I3Rp7Mk1Z VeDFyWLdg0YJFkn/895VabqOSAIe5v3IC9xvlQUzhiRwiAPbJ38jw4mRhqiJts3vikZ6 p+Ww== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20260327; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=/JtL2fhoMYFIFDWwyKWVDfMR3QY79CvLv8MvqHX0kHY=; fh=8xvFQ9N6ENvazHnylkO2sbgzUV6bRnZQvNiaC5bb+x4=; b=hKXaffOFLq4JhseMM1A8EuY1pE5S5xNB5+BE/G8qYczfIQMdFYxI41baLiJSrFz1Dd WUZUF2zH07HqZ2VtlxSrGbvtSsD4jL1bin74GeiJDi/8wCyEIGqjxSR8Ri0+YwaNI12u Tf5ws26S/6JsLrk47f8gN/5XJYknuWBkjZcSoCJkIaNdI57lA4xUiZsC+mX4x3FeSsKu NEWsmpMBCu3RmUbNAD8lnZSiA/sKC91a1HRxCEfYZtA/azMV6ZCXSjQNnSZJirVsH8VA W9hdF6sPeOCt8vhVGUoRr6fQJZEnH4IQfvhqM7YyGoctxnEfhC9YmmlTmwZ4fn8J3rD3 t5JQ==; dara=google.com ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of akarpovich@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=akarpovich@ilbers.de Received: from shymkent.ilbers.de (shymkent.ilbers.de. [85.214.156.166]) by gmr-mx.google.com with ESMTPS id 38308e7fff4ca-3a3702d5ad8si437051fa.0.2026.09.04.00.55.48 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 04 Sep 2026 00:55:48 -0700 (PDT) Received-SPF: pass (google.com: domain of akarpovich@ilbers.de designates 85.214.156.166 as permitted sender) client-ip=85.214.156.166; Received: from akarpovich-Victus-by-HP-Laptop-16-e0xxx.promwad.corp (62.122.174.6.serverel.net [62.122.174.6] (may be forged)) (authenticated bits=0) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPSA id 6847tRSB004124 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 4 Sep 2026 09:55:45 +0200 From: Aliaksei Karpovich To: isar-users@googlegroups.com Cc: Uladzimir Bely , Aliaksei Karpovich , Aliaksei Laurynovich Subject: [v10 01/12] scripts: Add debrepo python script handling base-apt Date: Fri, 4 Sep 2026 09:52:43 +0200 Message-ID: <20260904075515.25962-2-akarpovich@ilbers.de> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260904075515.25962-1-akarpovich@ilbers.de> References: <20260904075515.25962-1-akarpovich@ilbers.de> MIME-Version: 1.0 X-Spam-Status: No, score=-4.6 required=5.0 tests=DKIMWL_WL_MED,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_EF,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,RCVD_IN_DNSWL_BLOCKED,RCVD_IN_MSPIKE_H2, RCVD_IN_RP_CERTIFIED,RCVD_IN_RP_RNBL,RCVD_IN_RP_SAFE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on shymkent.ilbers.de X-Original-Sender: akarpovich@ilbers.de X-Original-Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of akarpovich@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=akarpovich@ilbers.de Precedence: list Mailing-list: list isar-users@googlegroups.com; contact isar-users+owners@googlegroups.com List-ID: X-Spam-Checked-In-Group: isar-users@googlegroups.com X-Google-Group-Id: 914930254986 List-Post: , List-Help: , List-Archive: , List-Unsubscribe: , X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= From: Uladzimir Bely This is the main utility responsible for prefetching packages into local `base-apt` repo from external Debian mirrors. It uses python-apt module and requires some kind of minimal `rootfs` to work (let's call it "debrepo context"). Once initialized with `--init --workdir=`, it stores the initial configuration in `repo.opts` file inside the context and uses it at futher calls. In future, the logic `debrepo` script implements could be directly implemented inside bitbake classes. Signed-off-by: Uladzimir Bely Signed-off-by: Aliaksei Karpovich Signed-off-by: Aliaksei Laurynovich --- scripts/deb-repo | 610 +++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 610 insertions(+) create mode 100755 scripts/deb-repo diff --git a/scripts/deb-repo b/scripts/deb-repo new file mode 100755 index 00000000..23de0461 --- /dev/null +++ b/scripts/deb-repo @@ -0,0 +1,610 @@ +#!/usr/bin/env python3 + +""" +# This software is a part of Isar. +# Copyright (C) 2026 ilbers GmbH + +# deb-repo: build Debian-like repo using "python3-apt" library. + +When building the image, Isar downloads required Debian packages from external +mirrors. After build completed, it can pick all downloaded packages from DL_DIR +and build local 'base-apt' Debian-like repo from them. + +This tool allows to download packages and create local repo in advance. So, +Isar just uses this local repository and does not interact with external +mirrors. Such approach makes deb-dl import/export functionality redundant. + +Script `deb-repo` works in so-called "context" directory. It means some +rootfs-like directory with bare minimum of directories/files required for +python3-apt to work. + +Context directory path is passed with "--workdir " command-line option. +On context creating, all passed parameters are stored in the context directory +and picked every time the context is used again. + +1. Repo for building Debian system +``` +deb-repo --workdir=d12 --init locales gnupg +``` +Initialize the context in "d12" directory and create a repository in +"d12/repo/apt" directory sufficient to debootstrap default system +(e.g., debian-bookworm-amd64). Additionally, packages "locales" and "gnupg" +with all their dependencies will be available in this repo. + +``` +deb-repo --workdir=d12 docbook-to-man +``` +Adds "docbook-to-man" packages with its dependencies to earlier created repo. + +``` +deb-repo --workdir=d12 --srcmode docbook-to-man +``` +Downloads source package for "docbook-to-man" and adds it to the repo + +2. Repo for building Ubuntu system +``` +deb-repo --init --workdir=uf \ +--distro=ubuntu --codename=focal --arch=arm64 \ +--aptsrcsfile=/work/isar/meta-isar/conf/distro/ubuntu-focal-ports.list \ +--repodir=repo/apt --repodbdir=repo/db \ +--mirror=http://ports.ubuntu.com/ubuntu-ports \ +locales gnupg +``` +Initialize the context in "uf" directory and create a repository in "repo/apt" +directory sufficient deboostraup ubuntu-focal arm64 system. Mirror to use and +source list are specified by corresponding arguments. Packages "locales" and +" gnupg" with the dependencies will be also placed to the repo. + +``` +deb-repo --workdir=uf gnupg,locales +``` +Add "gnupg" and "locales" packages with their dependencies to earlier created +ubuntu repo. Other parameters (distro, codename, arch) are ommited since they +are picked from the context. + +3. Repo for cross-building Debian system +``` +deb-repo --init --workdir=d11 --codename=bullseye--arch=amd64 --crossarch=armhf +``` +Initialize the context in "d11" directory sufficient to deboostrap Debian +Bullseye (amd64) with foreign "armhf" architecture support + +``` +deb-repo --workdir=d11 gcc +``` +Add "gcc" package (amd64 version) to earlier created repo. + +``` +deb-repo --workdir=d11 --crossbuild gcc +``` +Add "gcc" package (armhf version) to earlier created repo. +""" + +import os +import sys +import fcntl + +import argparse +import shutil +import subprocess +import pickle +import urllib.parse + +import apt_pkg +import apt.progress.base + + +REPREPRO_TIMEOUT = 1200 + + +class DebRepo(object): + class DebRepoCtx(object): + def __init__(self, workdir): + self.distro = "debian" + self.codename = "bullseye" + self.arch = "amd64" + self.mirror = "http://deb.debian.org/debian" + + self.repodir = f"{workdir}/repo/apt" + self.repodbdir = f"{workdir}/repo/db" + + self.crossarch = self.arch + self.compatarch = None + self.keydir = "/etc/apt/trusted.gpg.d" + + def __init__(self, args): + self.workdir = os.path.abspath(args.workdir) + self.ctx = self.DebRepoCtx(self.workdir) + + self.cache = None + self.depcache = None + self.sr = None + self.extrarepo = None + + self.ctx_load() + self.ctx_update(args) + self.ctx_save() + + print( + f"ctx workdir: {self.workdir}\n" + f" distro: {self.ctx.distro}\n" + f" codename: {self.ctx.codename}\n" + f" arch: {self.ctx.arch}\n" + f" mirror: {self.ctx.mirror}\n" + f" repodir: {self.ctx.repodir}\n" + f" repodbdir: {self.ctx.repodbdir}\n" + f" crossarch: {self.ctx.crossarch}\n" + f" compatarch: {self.ctx.compatarch}\n" + f" keydir: {self.ctx.keydir}" + ) + + if args.extrarepo: + self.extrarepo = os.path.abspath(args.extrarepo) + + def ctx_load(self): + ctxfile = f"{self.workdir}/deb-repo.ctx" + + if os.path.isfile(ctxfile): + with open(ctxfile, 'rb') as f: + self.ctx = pickle.load(f) + + def ctx_save(self): + ctxfile = f"{self.workdir}/deb-repo.ctx" + + with open(ctxfile, 'wb') as f: + pickle.dump(self.ctx, f) + + def ctx_update(self, args): + if args.distro: + self.ctx.distro = args.distro + if args.codename: + self.ctx.codename = args.codename + if args.arch: + self.ctx.arch = args.arch + if args.mirror: + self.ctx.mirror = args.mirror + + if args.repodir: + self.ctx.repodir = os.path.abspath(args.repodir) + if args.repodbdir: + self.ctx.repodbdir = os.path.abspath(args.repodbdir) + + if args.crossarch: + self.ctx.crossarch = args.crossarch + if args.compatarch: + self.ctx.compatarch = args.compatarch + if args.keydir: + self.ctx.keydir = args.keydir + + def create_rootfs(self, aptsrcsfile): + os.makedirs(f"{self.workdir}/var/lib/dpkg", exist_ok=True) + with open(f"{self.workdir}/var/lib/dpkg/status", "w"): + pass + + os.makedirs(f"{self.workdir}/etc/apt/sources.list.d", exist_ok=True) + + srcfile = f"{self.workdir}/etc/apt/sources.list.d/bootstrap.list" + if aptsrcsfile and os.path.exists(aptsrcsfile): + shutil.copy(aptsrcsfile, srcfile) + else: + with open(srcfile, "w") as f: + repo = f"{self.ctx.mirror} {self.ctx.codename} main" + f.write(f"deb {repo}\n") + f.write(f"deb-src {repo}\n") + + dir_cache = f"../apt_cache/{self.ctx.distro}-{self.ctx.codename}" + os.makedirs(f"{self.workdir}/{dir_cache}/archives/partial", + exist_ok=True) + + os.makedirs(f"{self.workdir}/tmp", exist_ok=True) + + def create_repo_dist(self): + conf_dir = f"{self.ctx.repodir}/{self.ctx.distro}/conf" + os.makedirs(conf_dir, exist_ok=True) + if not os.path.exists(f"{conf_dir}/distributions"): + with open(f"{conf_dir}/distributions", "w") as f: + f.write(f"Codename: {self.ctx.codename}\n") + f.write( + "Architectures: " + "i386 armhf arm64 amd64 mipsel riscv64 source\n") + f.write("Components: main\n") + + def apt_config(self, init, crossbuild): + if not init and self.ctx.compatarch: + apt_pkg.config["APT::Architectures::"] = self.ctx.compatarch + + if not init and self.ctx.arch != self.ctx.crossarch: + apt_pkg.config["APT::Architectures::"] = self.ctx.crossarch + apt_pkg.config["APT::Architectures::"] = self.ctx.arch + + apt_pkg.config.set("APT::Architecture", self.ctx.arch) + + apt_pkg.config.set("Dir", self.workdir) + + dir_cache = f"../apt_cache/{self.ctx.distro}-{self.ctx.codename}" + apt_pkg.config.set("Dir::Cache", f"{self.workdir}/{dir_cache}") + apt_pkg.config.set("Dir::State::status", + f"{self.workdir}/var/lib/dpkg/status") + + apt_pkg.config.set("APT::Install-Recommends", "0") + apt_pkg.config.set("APT::Install-Suggests", "0") + + # Use host keys for authentification + apt_pkg.config.set("Dir::Etc::TrustedParts", self.ctx.keydir) + + # Allow using repositories without keys + apt_pkg.config.set("Acquire::AllowInsecureRepositories", "1") + + def mark_essential(self): + for pkg in self.cache.packages: + if pkg.architecture == self.ctx.arch: + if pkg.essential: + self.depcache.mark_install(pkg) + + def mark_by_prio(self, priority): + for pkg in self.cache.packages: + if pkg.architecture == self.ctx.arch: + ver = self.depcache.get_candidate_ver(pkg) + if ver and ver.priority <= priority: + self.depcache.mark_install(pkg) + + def mark_pkg(self, name, crossbuild): + pkgname = name + + if pkgname and (pkgname not in self.cache): + # Try for cross arch + if (pkgname, self.ctx.crossarch) in self.cache: + pkgname += f":{self.ctx.crossarch}" + + if pkgname not in self.cache: + print(f"Error: package '{name}' not found") + return False + + pkg = self.cache[pkgname] + + if (not crossbuild) or (':' in pkgname) or (not pkg.has_versions): + if (pkg.has_provides) and (not pkg.has_versions): + print("pkgname is virtual package, selecting best provide") + # Select first provide + pkg_provide = pkg.provides_list[0][2] + # Find better provide with higher version + for provide in pkg.provides_list: + if apt_pkg.version_compare(provide[2].ver_str, + pkg_provide.ver_str) > 0: + pkg_provide = provide[2] + self.depcache.mark_install(pkg_provide.parent_pkg) + else: + self.depcache.mark_install(pkg) + else: + version = pkg.version_list[0] + if version.arch == "all": + self.depcache.mark_install(pkg) + else: + if version.multi_arch == version.MULTI_ARCH_FOREIGN: + if (pkgname, self.ctx.arch) in self.cache: + nativepkg = self.cache[pkgname, self.ctx.arch] + self.depcache.mark_install(nativepkg) + else: + return False + else: + if (pkgname, self.ctx.crossarch) in self.cache: + crosspkg = self.cache[pkgname, self.ctx.crossarch] + self.depcache.mark_install(crosspkg) + else: + return False + + return True + + def mark_list(self, pkglist, crossbuild): + ret = True + if pkglist: + for pkgname in pkglist: + ret = ret and self.mark_pkg(pkgname, crossbuild) + + return ret + + def handle_deb(self, item): + fd = open(f"{self.ctx.repodir}/repo.lock", 'w') + fcntl.flock(fd, fcntl.LOCK_EX) + subprocess.run([ + "reprepro", + "--dbdir", f"{self.ctx.repodbdir}/{self.ctx.distro}", + "--outdir", f"{self.ctx.repodir}/{self.ctx.distro}", + "--confdir", f"{self.ctx.repodir}/{self.ctx.distro}/conf", + "-C", "main", + "includedeb", + self.ctx.codename, + item.destfile + ], timeout=REPREPRO_TIMEOUT) + fd.close() + + def handle_repo(self, fetcher): + dir_cache = f"../apt_cache/{self.ctx.distro}-{self.ctx.codename}" + fd = open(f"{self.workdir}/{dir_cache}.lock", "w") + fcntl.flock(fd, fcntl.LOCK_EX) + fetcher.run() + fd.close() + for item in fetcher.items: + if item.status == item.STAT_ERROR: + print("Some error ocured: '%s'" % item.error_text) + pass + else: + self.handle_deb(item) + + def get_filename(self, uri): + path = urllib.parse.urlparse(uri).path + unquoted_path = urllib.parse.unquote(path) + basename = os.path.basename(unquoted_path) + return basename + + def fetch_file(self, uri): + filename = self.get_filename(uri) + return subprocess.run([ + "wget", + "-H", + "--timeout=30", + "--tries=3", + "-nv", + uri, + "-O", + f"{self.workdir}/tmp/{filename}" + ], + stdout=subprocess.PIPE, + stderr=subprocess.PIPE) + + def handle_dsc(self, dsc_file): + fd = open(f"{self.ctx.repodir}/repo.lock", 'w') + fcntl.flock(fd, fcntl.LOCK_EX) + result = subprocess.run([ + "reprepro", + "--dbdir", f"{self.ctx.repodbdir}/{self.ctx.distro}", + "--outdir", f"{self.ctx.repodir}/{self.ctx.distro}", + "--confdir", f"{self.ctx.repodir}/{self.ctx.distro}/conf", + "-C", "main", + "-S", "-", "-P" "source", + "--delete", + "includedsc", + self.ctx.codename, + os.path.realpath(f"{self.workdir}/tmp/{dsc_file}") + ], timeout=REPREPRO_TIMEOUT) + fd.close() + return result.returncode == 0 + + def handle_src_list(self, pkgs): + ret = True + if pkgs: + local_repo = f"{self.ctx.repodir}/{self.ctx.distro}" + fetched_files = [] + for pkg in pkgs: + pkgname = pkg + pkgver = "" + if '=' in pkg: + pkgname = pkg.split("=")[0] + pkgver = pkg.split("=")[1] + + self.sr.restart() + while self.sr.lookup(pkgname): + if pkgver and pkgver != self.sr.version: + continue + + for sr_file in self.sr.files: + uri_file = self.sr.index.archive_uri(sr_file.path) + filename = os.path.basename(sr_file.path) + path = urllib.parse.urlparse(uri_file).path.lstrip('/') + # Strip the distro directory prefix from upstream path + # e.g. "debian/pool/main/h/hello/file.dsc" -> "pool/main/h/hello/file.dsc" + parts = path.split('/', 1) + local_path = parts[1] if len(parts) > 1 else path + local_file = f"{local_repo}/{local_path}" + print(uri_file) + if filename not in fetched_files: + if os.path.exists(local_file): + shutil.copy2(local_file, f"{self.workdir}/tmp/{filename}") + else: + result = self.fetch_file(uri_file) + if result.returncode != 0: + print(f"ERROR: cannot download {uri_file}") + print(f"ERROR: {result.stderr}") + fetched_files.append(filename) + if sr_file.type == 'dsc': + dsc_file = filename + + ret = self.handle_dsc(dsc_file) + break + return ret + + def apt_run(self, init, srcmode, pkgs, dscfile, crossbuild): + apt_pkg.init() + + extrarepo_list = f"{self.workdir}/etc/apt/sources.list.d/extrarepo.list" + if self.extrarepo: + extrarepo_list = f"{self.workdir}/etc/apt/sources.list.d/extrarepo.list" + with open(extrarepo_list, "w") as f: + distdir=os.path.join(self.extrarepo, "dists") + if os.path.isdir(distdir): + for dist in os.listdir(distdir): + repodir = os.path.join(distdir,dist) + if os.path.isdir(repodir): + for repo in os.listdir(repodir): + if os.path.isdir(os.path.join(repodir, repo)): + f.write(f"deb file://{self.extrarepo} " + f"{dist} {repo}\n") + + sources = apt_pkg.SourceList() + sources.read_main_list() + + progress = apt.progress.text.AcquireProgress() + + self.cache = apt_pkg.Cache() + if init: + self.cache.update(progress, sources) + self.cache = apt_pkg.Cache() + + self.sr = apt_pkg.SourceRecords() + + if self.extrarepo: + apt_pkg.config.set("Dir::Etc::SourceList", extrarepo_list) + apt_pkg.config.set("APT::Get::List-Cleanup", "0") + extrarepo_sources = apt_pkg.SourceList() + extrarepo_sources.read_main_list() + self.cache.update(progress, extrarepo_sources) + self.cache = apt_pkg.Cache() + os.remove(extrarepo_list) + + self.depcache = apt_pkg.DepCache(self.cache) + + ret = True + + if init: + self.mark_essential() + # 1(required), 2(important), 3(standard), 4(optional), 5(extra) + self.mark_by_prio(1) + + pkgs = list(filter(None, ','.join(pkgs).split(','))) + if srcmode: + ret = self.handle_src_list(set(pkgs)) + else: + ret = self.mark_list(pkgs, crossbuild) + + if dscfile: + fobj = open(dscfile, "r") + + try: + tagfile = apt_pkg.TagFile(fobj) + while tagfile.step() == 1: + deps = tagfile.section.get("Build-Depends", "") + # Remove extra commas and spaces - apt_pkg.parse_src_depends + # doesnt like lines like ", device-tree-compiler" + deps = ', '.join( + [s.strip() for s in deps.split(',') if s.strip()] + ) + print(f"parsed deps: {deps}") + for item in apt_pkg.parse_src_depends(deps, False): + pkgname = item[0][0] + self.mark_pkg(pkgname, crossbuild) + + finally: + fobj.close() + + if not ret: + sys.exit("Some of requested packages not found") + + if init or not srcmode: + fetcher = apt_pkg.Acquire(progress) + pm = apt_pkg.PackageManager(self.depcache) + + recs = apt_pkg.PackageRecords(self.cache) + pm.get_archives(fetcher, sources, recs) + + self.handle_repo(fetcher) + + +def parse_arguments(): + parser = argparse.ArgumentParser() + parser.add_argument( + "--init", + default=False, action="store_true", + help="initialize context in WORKDIR") + parser.add_argument( + "--workdir", + type=str, required=True, + help="work directory storing deb-repo context") + parser.add_argument( + "--aptsrcsfile", + type=str, metavar="PATH", + help="sources.list file to use when init") + parser.add_argument( + "--srcmode", + default=False, action="store_true", + help="add source packages instead of debs") + parser.add_argument( + "--repodir", + type=str, metavar="REPO", + help="repository directory") + parser.add_argument( + "--repodbdir", + type=str, metavar="REPODB", + help="repository database directory") + parser.add_argument( + "--extrarepo", + type=str, metavar="REPO", + help="extra repository to consider") + parser.add_argument( + "--mirror", + type=str, + help="use custom distro mirror") + parser.add_argument( + "--distro", + type=str, + help="select distro to use") + parser.add_argument( + "--codename", + type=str, + help="distro codename") + parser.add_argument( + "--arch", + type=str, + help="distro arch") + parser.add_argument( + "--compatarch", + type=str, metavar="ARCH", + help="compat arch to use") + parser.add_argument( + "--crossarch", + type=str, metavar="ARCH", + help="cross-build arch") + parser.add_argument( + "--keydir", + type=str, + help="directory with distro keys") + parser.add_argument( + "--no-check-gpg", + default=False, action="store_true", + help="allow insecure repositories") + parser.add_argument( + "--dscfile", + type=str, metavar="PATH", + help="Debian source file to parse") + parser.add_argument( + "--crossbuild", + default=False, action="store_true", + help="add packages with cross arch") + + parser.add_argument( + "packages", + nargs='*', type=str, + help="space- or comma-separated list of packages to add") + + args = parser.parse_args() + + return args + + +def main(): + args = parse_arguments() + + if not (args.init or args.packages or args.dscfile): + sys.exit("Nothing to do") + + workdir = os.path.abspath(args.workdir) + os.makedirs(workdir, exist_ok=True) + + with open(f"{workdir}/deb-repo.lock", "a") as file: + fcntl.flock(file.fileno(), fcntl.LOCK_EX) + + deb_repo = DebRepo(args) + + if args.init: + deb_repo.create_rootfs(args.aptsrcsfile) + deb_repo.create_repo_dist() + + deb_repo.apt_config(args.init, args.crossbuild) + deb_repo.apt_run(args.init, args.srcmode, args.packages, + args.dscfile, args.crossbuild) + + #Unlock deb-repo context + fcntl.flock(file.fileno(), fcntl.LOCK_UN) + + +if __name__ == "__main__": + main() From patchwork Fri Sep 4 07:52:44 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Aliaksei Karpovich X-Patchwork-Id: 5421 Return-Path: Received: from shymkent.ilbers.de ([unix socket]) by shymkent (Cyrus 2.5.10-Debian-2.5.10-3+deb9u2) with LMTPA; Fri, 04 Sep 2026 09:56:07 +0200 X-Sieve: CMU Sieve 2.4 Received: from mail-wm1-f61.google.com (mail-wm1-f61.google.com [209.85.128.61]) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPS id 6847tvLV004177 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Fri, 4 Sep 2026 09:56:03 +0200 Received: by mail-wm1-f61.google.com with SMTP id 5b1f17b1804b1-495689bfcc8sf5097525e9.1 for ; Fri, 04 Sep 2026 00:55:58 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1788508552; cv=pass; d=google.com; s=arc-20260327; b=HR1nvKbXOnjm+46sD9g2a8ahCdJsa7j7Ir9xTpsBtJEI4I45eiyqGQI9V7sYvAbhAb /1D9Y7dqhw4cwdw1v22oVxb19BY4fSdnt4KAipAtq0i0lR61/5Sy3NfUEp2shCliQABI 03ECOlCiMjW8wrYazbt0PxGIZL2dgfh2yIA9QY6rDSaJGbidVRqKNuLKoNkxQQbUlwdT 3kSLqgOONn5e8jWx/RTHVwpFPSTYM0oieF/44zsdEotcskUTrGszC5HxzcyVQKfkKPaR m396any7EYqwC08sqU9mQShL6qjHqu/TtEaPugqaF2ynbMIU1IF5gdMOGq3iI3c8nrnn ta/g== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20260327; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:dkim-signature; bh=6llZNBkutMZ69UhXXzDqAvHiJT8FO3zGBvozI8oATY0=; fh=in7aU7jtfadUpF602xcgYbOQB5SgT2suuwC0r82qaHg=; b=NjPkaWaGyXeLAU4Q3b/jBr68J5xESgs9g/rn/9526CBhPkekTSs0k7PEozDmYDYOMO Ox5A+GFrwZpJQzutiZnrpc/ahYxrvANL2VXowdjyXzu2lITUeeM7fMOhosCtOff3RJTc MppgUQ7S2pm1Eb22wzVZVcCe70wkS9H/cvZVT44rGyHTGg/4sIIkwj+D57gdZ1wr0TlH MaWuYZRqEkuiCmjSTbFoHTULODVlScigYEgzu+hG23bMZIVhlD3sS4gBc9IvyTEoklPZ Z+DNQWCDj3KPXdN8iFxAyb9zOMKSuho4rhZmJB4YB/sS4iQDDEpYnP+kzI3wARfnnTyd xgYg==; darn=isar-build.org ARC-Authentication-Results: i=2; gmr-mx.google.com; spf=pass (google.com: domain of akarpovich@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=akarpovich@ilbers.de DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlegroups.com; s=20251104; t=1788508552; x=1789113352; darn=isar-build.org; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:content-type :x-original-authentication-results:x-original-sender:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from:sender :from:to:cc:subject:date:message-id:reply-to:content-type; bh=6llZNBkutMZ69UhXXzDqAvHiJT8FO3zGBvozI8oATY0=; b=D2ehqAcCBXXdZo+7hWY3taA6cWgFJdTIHcsiKetukdFUS4LWCcU2RSW3eN29uz3BZe vhgkEXAG6qVBmIXmf+NP30VACh2RH+YvQPka3QW83g0vkWyqc9ySAcLHUpvwB8eo1vEM mHGUcJYVYS8CfZBp3ZyZ2Wj3yOGztVzJyWACySK5fkVM/9g4HDTJ6UtvD9bB9+YIvduV 2RJhAc6+mnTIuqPf7Gk6zPhqTVqxLYgySkQfyqoWug0wgDleA9TtYjZsz2mddAHxCF0o iYIIF0adibFJnDyRxkw4R1ubwkgzgyThWBTugpo7ho3eAYVBY3HiSrxN118oLn6YHREa p+Vg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1788508552; x=1789113352; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :x-spam-checked-in-group:list-id:mailing-list:precedence :content-type:x-original-authentication-results:x-original-sender :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:x-beenthere:x-gm-message-state:sender:from:to:cc:subject:date :message-id:reply-to:content-type; bh=6llZNBkutMZ69UhXXzDqAvHiJT8FO3zGBvozI8oATY0=; b=sSvcbrdzyqMQdUPolLymNrgLZVYUzF0Dg6cQLrFvKoyd1jeOlEQLN5kbFsyWir+ui6 qspLbEi4KcSfeAXRJzQpfoVegUIaql68ZupxPJsPG4n0hfbk40m8ykcGxyECw7IBhQkJ Q93s1sqrFlxxkrw2EWSk7tRvtScsJJcTuRfiP1VP7B6ovqMphudZa+OCZ8/OtM76h1I8 zwepBMbtny/7S7pyk0mzEmM5YJSC72Iom+VUJdFm3aHpdSfKYWXXhyDZ3Y4FUVe8iRJk sqgfr4RlfCELL7M0aa7JB6RlDgjULhd8/KVioIjz24rJSRogvBcmKoY+9H2pz7sAmUNE xA1A== Sender: isar-users@googlegroups.com X-Forwarded-Encrypted: i=2; AKwUvBxYx3dO4Xsq0ED8aUONR15dUa8KEStK920bOP0YEjZswy1S7R1oBhO4suIF2MuxhmW6HzOQ/k0=@isar-build.org X-Gm-Message-State: AFuF++nxSf3ahlOeg1Nwbe3Ww9mYUx9g+0jududKoVn7HL7yaNGfwoh4 VER7EPMJaJZw9Pe309JjVDmuUHa0zFnA1HsVrlA/6QGer2/C7LldRL4q X-Received: by 2002:a05:600c:3b1f:b0:49c:ee06:9c58 with SMTP id 5b1f17b1804b1-49cf821eff7mr37807335e9.4.1788508552385; Fri, 04 Sep 2026 00:55:52 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com; h="ATskLdfj2L38VVVope+Xm0uwkC960GE/dYfqaGCLdG1YigERIg==" Received: by 2002:a05:600c:3b93:b0:495:5f00:94cb with SMTP id 5b1f17b1804b1-49cf6887dcfls7239995e9.2.-pod-prod-03-eu; Fri, 04 Sep 2026 00:55:50 -0700 (PDT) X-Received: by 2002:a05:600c:3488:b0:49c:fa21:e745 with SMTP id 5b1f17b1804b1-49cfe764152mr1016275e9.27.1788508549867; Fri, 04 Sep 2026 00:55:49 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1788508549; cv=none; d=google.com; s=arc-20260327; b=Z9eb5/S5UgG+kkH0duHkFSvIHGV02TOVStOFvg4hDzNKZvmUkys1ORS+TwDMnQONcQ 91alcxneHDeNcKKQWlylKNQ/oRY3SoYpuBZvZJYkdYn2b68nUA9gxieSjrHv2djnB9sV 82VWHq6LuLveQ4LuN2dijFQw/4U0amwm6WH5+Z7i5bTdQAaXD1RfBMnl8v4zEPkgppyf M8pG7qZ790bV+YOe2QtOCbzFEldS+efheBt90c98Ia4SJS4XRXBoxsLBck+WhYh7GiwY WrhYvTr1zTmrqbZfmyegZyMasdMwOWyzDF/JxPTZsGggpjMTBYVbuSkHqzSjoFvlER/5 Tu+Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20260327; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=cohjnP7Ibv3Xi4YpM0Sm/IAJ4GxHx0S4U7vmVCC57uo=; fh=IIbR0JxonYgzdrHDdB9TmZniSh/o1z0HofhfbmLXp8s=; b=leUPCe8BMlMWcVRPegOqd2MLfTLj5qoRgHyTALHwRxbIxYSRIMxqkOIiWqCFUKKNRr yKHTSwzgEl/noocyyUnO44CRFnbygUsw6xBav2Mcj55ed1wvgrN9VFwpiM5xRCEk7wZV eVIdlRCWjJOvedPFSW+MrsTrwn8kmu8SIXyp2zwJtvz4XB+V/VL2rN+zihD4x+vGFFFl 0YH5PPWEsGC4YOjJgbAypsSIINrDWPX9cDd3WpfnRE6IADvCK8M6aUUTWJ8eEAimLRU6 q9NPWE+wKie5YRE++1WunN4KIA0z9XSMqWkTK/lQzvv0eHzeuZOegJiK+ySxTqX2Bd0x QnTQ==; dara=google.com ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of akarpovich@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=akarpovich@ilbers.de Received: from shymkent.ilbers.de (shymkent.ilbers.de. [85.214.156.166]) by gmr-mx.google.com with ESMTPS id 5b1f17b1804b1-49ce47b2d77si1233245e9.1.2026.09.04.00.55.49 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 04 Sep 2026 00:55:49 -0700 (PDT) Received-SPF: pass (google.com: domain of akarpovich@ilbers.de designates 85.214.156.166 as permitted sender) client-ip=85.214.156.166; Received: from akarpovich-Victus-by-HP-Laptop-16-e0xxx.promwad.corp (62.122.174.6.serverel.net [62.122.174.6] (may be forged)) (authenticated bits=0) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPSA id 6847tRSC004124 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 4 Sep 2026 09:55:48 +0200 From: Aliaksei Karpovich To: isar-users@googlegroups.com Cc: Uladzimir Bely , Aliaksei Karpovich Subject: [v10 02/12] meta: Add debrepo bbclass handling base-apt prefetching Date: Fri, 4 Sep 2026 09:52:44 +0200 Message-ID: <20260904075515.25962-3-akarpovich@ilbers.de> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260904075515.25962-1-akarpovich@ilbers.de> References: <20260904075515.25962-1-akarpovich@ilbers.de> MIME-Version: 1.0 X-Spam-Status: No, score=-4.6 required=5.0 tests=DKIMWL_WL_MED,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_EF,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,RCVD_IN_DNSWL_BLOCKED,RCVD_IN_MSPIKE_H2, RCVD_IN_RP_CERTIFIED,RCVD_IN_RP_RNBL,RCVD_IN_RP_SAFE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on shymkent.ilbers.de X-Original-Sender: akarpovich@ilbers.de X-Original-Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of akarpovich@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=akarpovich@ilbers.de Precedence: list Mailing-list: list isar-users@googlegroups.com; contact isar-users+owners@googlegroups.com List-ID: X-Spam-Checked-In-Group: isar-users@googlegroups.com X-Google-Group-Id: 914930254986 List-Post: , List-Help: , List-Archive: , List-Unsubscribe: , X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= From: Uladzimir Bely This class uses 'scripts/debrepo' python script to prefetch given packages or sources to local base-apt repository. Signed-off-by: Uladzimir Bely Signed-off-by: Aliaksei Karpovich --- RECIPE-API-CHANGELOG.md | 9 ++++ doc/user_manual.md | 1 + meta/classes/debrepo.bbclass | 89 ++++++++++++++++++++++++++++++++++++ meta/conf/bitbake.conf | 5 ++ 4 files changed, 104 insertions(+) create mode 100644 meta/classes/debrepo.bbclass diff --git a/RECIPE-API-CHANGELOG.md b/RECIPE-API-CHANGELOG.md index 2db05169..139fcc8b 100644 --- a/RECIPE-API-CHANGELOG.md +++ b/RECIPE-API-CHANGELOG.md @@ -1201,3 +1201,12 @@ To opt out, remove the feature explicitly: ``` ROOTFS_FEATURES:remove = "clean-apt-credentials" ``` +### "Prefetch" mode for base-apt + +Originally, `base-apt` repo is created only during second build when variable +ISAR_USE_CACHED_BASE_REPO is set. The repo is populated with every package that +took part in the first build and was cached in DL_DIR. + +New ISAR_PREFETCH_BASE_APT variable changes the way `base-apt` is populated. +Packages added to the repo before running any task that need them. Separate +`deb-repo` script is used for populating base-apt repo. diff --git a/doc/user_manual.md b/doc/user_manual.md index f3b1acb9..5cb942ad 100644 --- a/doc/user_manual.md +++ b/doc/user_manual.md @@ -86,6 +86,7 @@ apt install \ gettext-base \ git \ python3 \ + python3-apt \ quilt \ qemu-user-static \ reprepro \ diff --git a/meta/classes/debrepo.bbclass b/meta/classes/debrepo.bbclass new file mode 100644 index 00000000..79a19144 --- /dev/null +++ b/meta/classes/debrepo.bbclass @@ -0,0 +1,89 @@ +# This software is a part of Isar. +# Copyright (C) 2026 ilbers GmbH +# +# SPDX-License-Identifier: MIT + +# Prefetch to base-apt repo by default +ISAR_PREFETCH_BASE_APT ??= "1" + +DEBREPO_WORKDIR ??= "${DEBREPO_TARGET_DIR}" + +debrepo_update_apt_source_list() { + [ "${ISAR_PREFETCH_BASE_APT}" != "1" ] && return + + chroot_dir=${1} + apt_list=${2} + + flock -x "${REPO_BASE_DIR}/repo.lock" -c " + sudo -E chroot ${chroot_dir} /usr/bin/apt-get update \ + -o Dir::Etc::SourceList=\"sources.list.d/${apt_list}.list\" \ + -o Dir::Etc::SourceParts=\"-\" \ + -o APT::Get::List-Cleanup=\"0\" + " +} + +debrepo_add_packages() { + [ "${ISAR_PREFETCH_BASE_APT}" != "1" ] && return + [ "${ISAR_USE_CACHED_BASE_REPO}" = "1" ] && return + + args="" + if [ "${1}" = "--srcmode" ]; then + args="${args} --srcmode" + shift + fi + + if [ "${1}" = "--isarapt" ]; then + args="${args} --extrarepo=${REPO_ISAR_DIR}/${DISTRO}" + shift + fi + + workdir="${1}" + args="${args} ${2}" + + if [ -n "${GNUPGHOME}" ]; then + export GNUPGHOME="${GNUPGHOME}" + else + export GNUPGHOME="${WORKDIR}/gpghome" + fi + + ${SCRIPTSDIR}/deb-repo \ + --workdir="${workdir}" \ + ${args} +} + +debrepo_parse_dscfile() { + [ "${ISAR_PREFETCH_BASE_APT}" != "1" ] && return + [ "${ISAR_USE_CACHED_BASE_REPO}" = "1" ] && return + + dscfile="${1}" + args="" + + if [ "${ISAR_CROSS_COMPILE}" = "1" ]; then + build_arch=${HOST_ARCH} + if [ "${PACKAGE_ARCH}" != "${build_arch}" ]; then + args="--crossbuild \ + crossbuild-essential-${PACKAGE_ARCH}:${build_arch} \ + dose-distcheck:${build_arch} \ + libc-dev:${PACKAGE_ARCH} \ + libstdc++-dev:${PACKAGE_ARCH} \ + " + fi + fi + + args="${args} --extrarepo=${WORKDIR}/isar-apt/${DISTRO}-${DISTRO_ARCH}/apt/${DISTRO}" + + if [ -n "${GNUPGHOME}" ]; then + export GNUPGHOME="${GNUPGHOME}" + else + export GNUPGHOME="${WORKDIR}/gpghome" + fi + + if [ -n "${DEB_BUILD_PROFILES}" ]; then + export DEB_BUILD_PROFILES="${DEB_BUILD_PROFILES}" + fi + + ${SCRIPTSDIR}/deb-repo \ + --workdir="${DEBREPO_WORKDIR}" \ + --dscfile="${dscfile}" \ + ${args} +} diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf index e69ec3bb..c8503ec6 100644 --- a/meta/conf/bitbake.conf +++ b/meta/conf/bitbake.conf @@ -67,6 +67,11 @@ SDKCHROOT_DIR = "${DEPLOY_DIR_SDKCHROOT}/${BPN}-${DISTRO}-${MACHINE}" CACHE = "${TMPDIR}/cache" KERNEL_FILE ?= "${@ 'vmlinux' if d.getVar('DISTRO_ARCH') in ['mipsel', 'riscv64', 'arm64'] else 'vmlinuz'}" +# debrepo config +DEBREPO_DIR = "${TOPDIR}/debrepo" +DEBREPO_HOST_DIR = "${DEBREPO_DIR}/${HOST_DISTRO}-${HOST_ARCH}_${DISTRO}-${DISTRO_ARCH}" +DEBREPO_TARGET_DIR = "${DEBREPO_DIR}/${DISTRO}-${DISTRO_ARCH}" + MACHINEOVERRIDES ?= "${MACHINE}" DISTROOVERRIDES ?= "${DISTRO}" OVERRIDES = "${PACKAGE_ARCH}:${MACHINEOVERRIDES}:${DISTROOVERRIDES}:${BASE_DISTRO_CODENAME}::${BASE_DISTRO}:${ISAR_CHROOT_MODE}:forcevariable" From patchwork Fri Sep 4 07:52:45 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Aliaksei Karpovich X-Patchwork-Id: 5419 Return-Path: Received: from shymkent.ilbers.de ([unix socket]) by shymkent (Cyrus 2.5.10-Debian-2.5.10-3+deb9u2) with LMTPA; Fri, 04 Sep 2026 09:56:05 +0200 X-Sieve: CMU Sieve 2.4 Received: from mail-wm1-f61.google.com (mail-wm1-f61.google.com [209.85.128.61]) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPS id 6847txl5004183 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Fri, 4 Sep 2026 09:55:59 +0200 Received: by mail-wm1-f61.google.com with SMTP id 5b1f17b1804b1-49a1b4ea633sf7068495e9.0 for ; Fri, 04 Sep 2026 00:55:59 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1788508554; cv=pass; d=google.com; s=arc-20260327; b=e3dL67RoSO9wleLe5BLwPPfmGqjFGytf4Sd/+mBzGb6eBYarwtzIsPfGr5hGDBNv1u C+8k/n3KAI8Y/enBcon7N5WIofnhrF572Bf3zp2zBNtv59Vc7myGm+Dkr962YX1HfY22 oPwR6BI+9NVh9zJ0MmSDFHcYkttCU+BHkVYE5ghdYMYPQlNVyZnlgDFF8h+tnUbrDfMA Oh/JQ8CU50FWui2bS08hTt4icZh8G+iBLrIcAgwwfvE4XaMr/q8BxJeA3E5g3r9Xfh0H N8I65qpU38dL4d88ky3keGWHg+an5MEgsjVPDHXG29101aCxH+k+qf20/Vt0zBN5Pe+1 pADQ== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20260327; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:dkim-signature; bh=AqfTQy5XQL9YKUUJem01FRzNvJTdz7C728Xi+qMgJxk=; fh=VNQzvtSb2Ez3f6QJpWJqK5NifTFyA71R7PNj6VRLSV4=; b=rgA9z8eJ/Xao2e5T/JQcOaS+FzpNz+1ySWD1q7xubLLRvGFCQLl5x9wR8w1lHKdmVM diqbgi0CAoTqljKaP020ZD1Ui+B9QPNCh9wItwIDVOo3tDHXXC3ZDe2rMWHGeUxbqFav hniq+Or9kiSYrh/1kdIykN4EV4DuVjDOwaHDwqE41rNLNnoC3R64iLs/AHhp51kgJvSG m2JUp7Jenue0AXQ8iYnGP3JZ3uBn5xllro8vyusEq0v7VCzt1ZOBBjQUxUk3BGMp3O3/ 6TQjEu6mBZxpddLevNgcCQTimsvR5fw/U/YFwYH8Exitwzg7NFKVDp/SJpOG4QbNoi8g JR2g==; darn=isar-build.org ARC-Authentication-Results: i=2; gmr-mx.google.com; spf=pass (google.com: domain of akarpovich@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=akarpovich@ilbers.de DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlegroups.com; s=20251104; t=1788508554; x=1789113354; darn=isar-build.org; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:content-type :x-original-authentication-results:x-original-sender:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from:sender :from:to:cc:subject:date:message-id:reply-to:content-type; bh=AqfTQy5XQL9YKUUJem01FRzNvJTdz7C728Xi+qMgJxk=; b=OPpLCk9TWYZP6iytEie2J0FHx+jg1Gq42sUkCzYc4rDC53PETCVIPV5bLaXNhaP4V+ 0p3KAkQwzr5RG+DZeb+cpCy3I03l0yOHcVSSgK20+kK0JmasSd4czHxidm5nvTsaNo4B bJlthP3RobhSxGjazebXAGhLc31ELRKT0KmmxhdI4Y8PrbSneR3lMJUFfl4ctjk8m7GR YlSHbmziEtSXFbPec/tukyyHm6x5iXb6gbBl1YqpJW27rp6EUGtKNpoT5kMj/BHF/8u0 +ENvTxAQoSmLTP+uHwP8WLVS7Cg5Uva2nS7mhXJb/YH41Todmeq78Ji/Z/pUFcVb19uD 85ew== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1788508554; x=1789113354; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :x-spam-checked-in-group:list-id:mailing-list:precedence :content-type:x-original-authentication-results:x-original-sender :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:x-beenthere:x-gm-message-state:sender:from:to:cc:subject:date :message-id:reply-to:content-type; bh=AqfTQy5XQL9YKUUJem01FRzNvJTdz7C728Xi+qMgJxk=; b=oFzZAnuGD8n7Q+zKN9zoxhWPdQtvS7dVKlI9ecmW3SB+3vzBP/ArhX1gEOZxKDmiN6 Urhd/HkD/g0CmIFs/+JGIb/z8A0zRCMgMpBuYpTnNINGSkJUCVuNABva1J5vRhABtP1F MzkrlvB2IDHviNSG93fFewbmJYhTN8Z5ADo+10ti8Hw4FnPWHmT1OMv7CrSeY4+7yL8c pRp2YHjm6K3wPmaJLu7H6019qNkNN8D+tGDKi2VCplf4z3T8RhwIPzJM646c8DI6p4bv TcvyqPedTDGhYs3tSZmNud9SU8vEyZR7bnlQr+XqhaiH8R+joYBELnKIOgW5H1k3JjcB 9J7g== Sender: isar-users@googlegroups.com X-Forwarded-Encrypted: i=2; AKwUvByTbIZDBQtpE9DJJ1wnSrzFqoWqu9AC80DhsMR+Tt1XjtF+oR1LVEkuShpJfwVyUXahVLegUKc=@isar-build.org X-Gm-Message-State: AFuF++l20ThY22FH0Be7A2qC2dICm08amZ5AsTfz7k+NRHlyYIdWXKeC tqWluOdYiDFRgigT878DorjKgBKcHSmZ74pp0EzKKFv4gCAcY1WsL5+F X-Received: by 2002:a05:600c:1990:b0:49c:f512:2361 with SMTP id 5b1f17b1804b1-49cf82524a2mr43663205e9.14.1788508554076; Fri, 04 Sep 2026 00:55:54 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com; h="ATskLdcUj1i7fBAPjv/2kZ9gJ1LWfMCDy8CV9l00G4jS3oGAoA==" Received: by 2002:a05:600c:1c0e:b0:495:5dfd:ff7c with SMTP id 5b1f17b1804b1-49cf6877442ls8260775e9.1.-pod-prod-02-eu; Fri, 04 Sep 2026 00:55:52 -0700 (PDT) X-Received: by 2002:a05:600c:6306:b0:49c:f5c0:aa76 with SMTP id 5b1f17b1804b1-49cf81e366amr79919135e9.5.1788508551792; Fri, 04 Sep 2026 00:55:51 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1788508551; cv=none; d=google.com; s=arc-20260327; b=ZezfG+Prn3UQWznSaSI0lpN4mqvKlxW09d3PWPaPV7/GNwinAkANVRAqTl3X9A4xwi AUiUMimQkmY/t+MtRM6jxaary3X2F7TSb3xMh+Lsn6vPnthhWfkbT/GTi82Vd+OzJP9D uxOtHnOfyRPEyjFnb7Yvbxw9gPFJYRkSYOpZCgzIbiCmPUSjK3JlyoZ6X3IvW0edxLs5 kmhOnwzntGcHYpR7GlnJFfBod9i2sPrssdUC5NM2S5nwY1wVcj9L+OUf5XFvtBk5KSsH uNHF96lQ3yAM8m/4WlJu53wXwmdVjm9P3zycY7m2g+/ETZNBDATD/f2ZGa/intpGYj2M z7cA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20260327; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=G3ir034zB1T9z2HBCQOJ244dU3gGtKh1hG2Y5z66W0Q=; fh=8xvFQ9N6ENvazHnylkO2sbgzUV6bRnZQvNiaC5bb+x4=; b=kvfNXtujOfqWPBdMibdgKgpNasEOIV7Qwsd12tFvKjAAXw1TB2bXOBB8geoyOQASGx b87+35H4zQ5UT+mFmsV07NffsfYHgJphoyC2WXr+1GT3mjeKHJ52YRmV4SghYRqNbOQX e7HSU+vjD9jZq8SerP2FpI6vSl0xZyH/xtmARcvxx2/o0/1bhQGliRz+Qr/00pxVRNih E84ZA/h5gFm1m2GNHAQHLIkwWJCTzQ68v7sndh+VHXr1BXpOlDt+NeXwl4CHvDVMf92H H5+pP892TtjXzpvxCqbKHx7V694la9yql4l/UPRbJ8DnyEDzbjlnN8o+Q84+ux2MmTF1 zxIA==; dara=google.com ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of akarpovich@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=akarpovich@ilbers.de Received: from shymkent.ilbers.de (shymkent.ilbers.de. [85.214.156.166]) by gmr-mx.google.com with ESMTPS id ffacd0b85a97d-485885d1bc8si32138f8f.6.2026.09.04.00.55.51 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 04 Sep 2026 00:55:51 -0700 (PDT) Received-SPF: pass (google.com: domain of akarpovich@ilbers.de designates 85.214.156.166 as permitted sender) client-ip=85.214.156.166; Received: from akarpovich-Victus-by-HP-Laptop-16-e0xxx.promwad.corp (62.122.174.6.serverel.net [62.122.174.6] (may be forged)) (authenticated bits=0) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPSA id 6847tRSD004124 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 4 Sep 2026 09:55:49 +0200 From: Aliaksei Karpovich To: isar-users@googlegroups.com Cc: Uladzimir Bely , Aliaksei Karpovich , Aliaksei Laurynovich Subject: [v10 03/12] meta: Always use base-apt repo in local mode Date: Fri, 4 Sep 2026 09:52:45 +0200 Message-ID: <20260904075515.25962-4-akarpovich@ilbers.de> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260904075515.25962-1-akarpovich@ilbers.de> References: <20260904075515.25962-1-akarpovich@ilbers.de> MIME-Version: 1.0 X-Spam-Status: No, score=-4.6 required=5.0 tests=DKIMWL_WL_MED,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_EF,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,RCVD_IN_DNSWL_BLOCKED,RCVD_IN_MSPIKE_H2, RCVD_IN_RP_CERTIFIED,RCVD_IN_RP_RNBL,RCVD_IN_RP_SAFE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on shymkent.ilbers.de X-Original-Sender: akarpovich@ilbers.de X-Original-Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of akarpovich@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=akarpovich@ilbers.de Precedence: list Mailing-list: list isar-users@googlegroups.com; contact isar-users+owners@googlegroups.com List-ID: X-Spam-Checked-In-Group: isar-users@googlegroups.com X-Google-Group-Id: 914930254986 List-Post: , List-Help: , List-Archive: , List-Unsubscribe: , X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= From: Uladzimir Bely This means only local URLs in apt sources.list* are present during the build. Any installation of packages is done from local base-apt. So, base-apt should be always mounted in *_do_mounts since now. Signed-off-by: Uladzimir Bely Signed-off-by: Aliaksei Karpovich Signed-off-by: Aliaksei Laurynovich --- meta/classes-recipe/bootstrap.bbclass | 6 ++++-- meta/classes-recipe/rootfs.bbclass | 3 ++- meta/classes-recipe/sbuild.bbclass | 3 ++- .../isar-mmdebstrap/isar-mmdebstrap.inc | 17 ++++++++++----- meta/recipes-devtools/base-apt/base-apt.bb | 21 ++++++++++++------- 5 files changed, 34 insertions(+), 16 deletions(-) diff --git a/meta/classes-recipe/bootstrap.bbclass b/meta/classes-recipe/bootstrap.bbclass index c1a59fd2..f853a932 100644 --- a/meta/classes-recipe/bootstrap.bbclass +++ b/meta/classes-recipe/bootstrap.bbclass @@ -39,7 +39,8 @@ python () { # installation afterwards. However, bootstrap will include the key into # the rootfs automatically thus the right place is distro_bootstrap_keys. - if bb.utils.to_boolean(d.getVar('ISAR_USE_CACHED_BASE_REPO')): + if bb.utils.to_boolean(d.getVar('ISAR_PREFETCH_BASE_APT')) or \ + bb.utils.to_boolean(d.getVar('ISAR_USE_CACHED_BASE_REPO')) : own_pub_key = d.getVar("BASE_REPO_KEY") if own_pub_key: distro_bootstrap_keys += own_pub_key.split() @@ -121,7 +122,8 @@ def get_apt_source_mirror(d, aptsources_entry_list): # this is executed during parsing. No error checking possible use_snapshot = bb.utils.to_boolean(d.getVar('ISAR_USE_APT_SNAPSHOT')) snapshot_mirror = d.getVar('DISTRO_APT_SNAPSHOT_PREMIRROR') - if bb.utils.to_boolean(d.getVar('ISAR_USE_CACHED_BASE_REPO')): + if bb.utils.to_boolean(d.getVar('ISAR_PREFETCH_BASE_APT')) or \ + bb.utils.to_boolean(d.getVar('ISAR_USE_CACHED_BASE_REPO')) : premirrors = "\S* file://${REPO_BASE_DIR}/${BOOTSTRAP_BASE_DISTRO}\n" elif use_snapshot and snapshot_mirror: premirrors = snapshot_mirror diff --git a/meta/classes-recipe/rootfs.bbclass b/meta/classes-recipe/rootfs.bbclass index 8d394b05..99328b12 100644 --- a/meta/classes-recipe/rootfs.bbclass +++ b/meta/classes-recipe/rootfs.bbclass @@ -62,7 +62,8 @@ ROOTFS_MOUNTS ??= "${REPO_ISAR_DIR}/${DISTRO}:/isar-apt ${WORKDIR}:/isar-work" python () { mounts = d.getVar('ROOTFS_MOUNTS', False) - if bb.utils.to_boolean(d.getVar('ISAR_USE_CACHED_BASE_REPO')) and not ':/base-apt' in mounts: + if (bb.utils.to_boolean(d.getVar('ISAR_USE_CACHED_BASE_REPO')) or + bb.utils.to_boolean(d.getVar('ISAR_PREFETCH_BASE_APT'))) and not ':/base-apt' in mounts: base_apt = '{}:/base-apt'.format(d.getVar('REPO_BASE_DIR')) d.setVar('ROOTFS_MOUNTS', '{} {}'.format(mounts, base_apt)) } diff --git a/meta/classes-recipe/sbuild.bbclass b/meta/classes-recipe/sbuild.bbclass index 6db29251..29a8dd71 100644 --- a/meta/classes-recipe/sbuild.bbclass +++ b/meta/classes-recipe/sbuild.bbclass @@ -44,7 +44,8 @@ EOF cp -rf "${SCHROOT_CONF}/sbuild" "${SBUILD_CONF_DIR}" sbuild_fstab="${SBUILD_CONF_DIR}/fstab" - if [ "${ISAR_USE_CACHED_BASE_REPO}" = "1" ]; then + if [ "${ISAR_USE_CACHED_BASE_REPO}" = "1" ] || \ + [ "${ISAR_PREFETCH_BASE_APT}" = "1" ]; then fstab_baseapt="${REPO_BASE_DIR} /base-apt none rw,bind,private 0 0" grep -qxF "${fstab_baseapt}" ${sbuild_fstab} || echo "${fstab_baseapt}" >> ${sbuild_fstab} fi diff --git a/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc b/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc index 994da174..4d61e9be 100644 --- a/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc +++ b/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc @@ -153,16 +153,21 @@ do_bootstrap() { fi E="${@ isar_export_proxies(d)}" - if [ "${ISAR_USE_CACHED_BASE_REPO}" = "1" ]; then + if [ "${ISAR_PREFETCH_BASE_APT}" = "1" ] || \ + [ "${ISAR_USE_CACHED_BASE_REPO}" = "1" ]; then base_apt_tmp="$(mktemp -d /tmp/isar-base-aptXXXXXXXXXX)" bootstrap_list="${WORKDIR}/sources.list.d/base-apt.list" line="copy://$base_apt_tmp/${BOOTSTRAP_BASE_DISTRO} ${BASE_DISTRO_CODENAME} main" - if [ -z "${BASE_REPO_KEY}" ]; then + if [ -z "${BASE_REPO_KEY}" ] || \ + [ "${ISAR_PREFETCH_BASE_APT}" = "1" ]; then + line="[trusted=yes] ${line}" fi echo "deb ${line}" > "${WORKDIR}/sources.list.d/base-apt.list" line="copy://$base_apt_tmp/${BASE_DISTRO} ${BASE_DISTRO_CODENAME} main" - if [ -z "${BASE_REPO_KEY}" ]; then + if [ -z "${BASE_REPO_KEY}" ] || \ + [ "${ISAR_PREFETCH_BASE_APT}" = "1" ]; then + line="[trusted=yes] ${line}" fi echo "deb-src ${line}" >> "${WORKDIR}/sources.list.d/base-apt.list" @@ -183,9 +188,11 @@ do_bootstrap() { \$1/etc/apt/sources.list.d/*.list && \ mkdir -p \$1/base-apt && \ mount -o bind,private '${REPO_BASE_DIR}' \$1/base-apt && \ - chroot \$1 apt-get update -y \ + chroot \$1 mv /etc/apt/sources.list.d/0000base-apt.list \ + /etc/apt/sources.list.d/base-apt.list + flock -x ${REPO_BASE_DIR}/repo.lock -c \"chroot \$1 apt-get update -y \ -o APT::Update::Error-Mode=any \ - ${@'-o APT::Sandbox::User=root' if d.getVar('ISAR_CHROOT_MODE') == 'unshare' else ''} && \ + ${@'-o APT::Sandbox::User=root' if d.getVar('ISAR_CHROOT_MODE') == 'unshare' else ''} \" && \ chroot \$1 apt-get install -y dpkg && \ umount \$1/base-apt && \ umount \$1/$base_apt_tmp && \ diff --git a/meta/recipes-devtools/base-apt/base-apt.bb b/meta/recipes-devtools/base-apt/base-apt.bb index 06b1f6c8..05ef9741 100644 --- a/meta/recipes-devtools/base-apt/base-apt.bb +++ b/meta/recipes-devtools/base-apt/base-apt.bb @@ -59,9 +59,12 @@ repo() { "${BASE_DISTRO_CODENAME}" \ "${WORKDIR}/distributions.in" \ "${KEYFILES}" - populate_base_apt "${BASE_DISTRO}" - repo_sanity_test "${REPO_BASE_DIR}"/"${BASE_DISTRO}" \ - "${REPO_BASE_DB_DIR}"/"${BASE_DISTRO}" + if [ "${ISAR_USE_CACHED_BASE_REPO}" = "1" ] && \ + [ "${ISAR_PREFETCH_BASE_APT}" != "1" ]; then + populate_base_apt "${BASE_DISTRO}" + repo_sanity_test "${REPO_BASE_DIR}"/"${BASE_DISTRO}" \ + "${REPO_BASE_DB_DIR}"/"${BASE_DISTRO}" + fi if [ '${BASE_DISTRO}' != '${HOST_BASE_DISTRO}' ]; then repo_create "${REPO_BASE_DIR}"/"${HOST_BASE_DISTRO}" \ @@ -69,14 +72,18 @@ repo() { "${BASE_DISTRO_CODENAME}" \ "${WORKDIR}/distributions.in" \ "${KEYFILES}" - populate_base_apt "${HOST_BASE_DISTRO}" - repo_sanity_test "${REPO_BASE_DIR}"/"${HOST_BASE_DISTRO}" \ - "${REPO_BASE_DB_DIR}"/"${HOST_BASE_DISTRO}" + if [ "${ISAR_USE_CACHED_BASE_REPO}" = "1" ] && \ + [ "${ISAR_PREFETCH_BASE_APT}" != "1" ]; then + populate_base_apt "${HOST_BASE_DISTRO}" + repo_sanity_test "${REPO_BASE_DIR}"/"${HOST_BASE_DISTRO}" \ + "${REPO_BASE_DB_DIR}"/"${HOST_BASE_DISTRO}" + fi fi } python do_cache() { - if not bb.utils.to_boolean(d.getVar('ISAR_USE_CACHED_BASE_REPO')): + if not bb.utils.to_boolean(d.getVar('ISAR_PREFETCH_BASE_APT')) and \ + not bb.utils.to_boolean(d.getVar('ISAR_USE_CACHED_BASE_REPO')): return 0 for key in d.getVar('BASE_REPO_KEY').split(): From patchwork Fri Sep 4 07:52:46 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Aliaksei Karpovich X-Patchwork-Id: 5420 Return-Path: Received: from shymkent.ilbers.de ([unix socket]) by shymkent (Cyrus 2.5.10-Debian-2.5.10-3+deb9u2) with LMTPA; Fri, 04 Sep 2026 09:56:06 +0200 X-Sieve: CMU Sieve 2.4 Received: from mail-ed1-f58.google.com (mail-ed1-f58.google.com [209.85.208.58]) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPS id 6847u1Jn004198 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Fri, 4 Sep 2026 09:56:01 +0200 Received: by mail-ed1-f58.google.com with SMTP id 4fb4d7f45d1cf-6a5d46edd1esf658693a12.1 for ; Fri, 04 Sep 2026 00:56:01 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1788508556; cv=pass; d=google.com; s=arc-20260327; b=kJEfwzYS6JvZtmjLpJa09elr1whG/STzbMHZFTUCCFES+4kcNH8RnDeU0VrCdJVb0A Ocwq9HskhPnuiGDZ27UWMaV8OqlkHUFbnuMJzsYUzwDQJMy5IgS8oy2I2Ptj0/oqXEDC vb+OQv9bIniYcPp3LS5Z9Fgw6myc4AqxaFCEPybOI605f+a16xkPCYN7wvaaGQKhtc9Y 2XT/w2CzTVEPv8LP6JF9LCFeSPhbCug0BWF0Nb33WE7+9U7bJTBP3dKud2/dn2SuObdC 9J1VcZbBTRyIySzbOPLa1gae9DrZ/1Mff4FZhe7Xj5qs56AvfCcxnPzv/tRW4JJ8IE8V cVww== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20260327; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:dkim-signature; bh=NNojQUm75dnAtGjG2nccBfygNNXq1A9k9aJYVP52zJA=; fh=bCtn/v4S9sfQ3b2nVDbV0uRc1LQWp3LG30GLfMXm904=; b=YPQ7QWGaqHhBGKEXBiHPhpBg2h4kjoFNUDLPrLZAqyQ/0NQ47I1r3TsRejAQ8W1ijE /nRlhho9KNyDjGWW6oX1WvG8zhefoIcgnG0Lc8Xdf99A7deOi6mrFDmkZFOney87rlse 7IlWyV7bLc9etuol0wtXmO3lRpPjx1WLK7PyoqGI7+I7DCGRoQ9xNoDqWZAp2h+P2ZeB UE3TWXK5GeTdHY81arkuEgK91wjN0OLbr+bAwsDFWQC1P6N4ohV1GYlcdnr/GQmhsJIm IJe+ZgTDMrg+HR0K6LY2CuTR+8UgAb1VtMvOQeHxIUUiLyrfqkLtRyMXTIz0CLYglPv+ UyAQ==; darn=isar-build.org ARC-Authentication-Results: i=2; gmr-mx.google.com; spf=pass (google.com: domain of akarpovich@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=akarpovich@ilbers.de DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlegroups.com; s=20251104; t=1788508556; x=1789113356; darn=isar-build.org; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:content-type :x-original-authentication-results:x-original-sender:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from:sender :from:to:cc:subject:date:message-id:reply-to:content-type; bh=NNojQUm75dnAtGjG2nccBfygNNXq1A9k9aJYVP52zJA=; b=pbWLHDXhzC4PU9FOZwaBFiJZaziDlsySgqUm7GMBu3lzF+y0t7QuScMDNavNhvrDyO AxjeYYzxl73/6dMleXMtsBP1F4s/v22DkMx2wO3KiadKfxx+SWmozX4CQE/zKQte5cs1 Ajs4My6S8kif1bwH4miyCCClwWGnrGhFETAPgGTC2+KL4dGA66FskQ46blZdqgxA474t oODQha/4WpVrVPuLnGyaQTmkpcy/ojXIngYtsTcnP/IJJXOkXBCc4P+ALzQTYwBQDw2/ FWoa8wynUWzP2mOk8hr9xENfuFEMmJRVEGmPNf+F3GmIEYcxjL4R+QpN6TxBHDJWXYER QIZg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1788508556; x=1789113356; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :x-spam-checked-in-group:list-id:mailing-list:precedence :content-type:x-original-authentication-results:x-original-sender :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:x-beenthere:x-gm-message-state:sender:from:to:cc:subject:date :message-id:reply-to:content-type; bh=NNojQUm75dnAtGjG2nccBfygNNXq1A9k9aJYVP52zJA=; b=eJCajMV6jkU5TrcIz5CQX7bHYBha3aa9eNwui6L+z/Xkt2Hrssg6V95e/w/Wk+M3bB GfQlq2K1UXgEFlybeu3pwnsZ0doc53SHwqK5bPHyM9k7E2yfNl2fDemMTXlyMGpB9nhu +fE0kJ96XP13MEkVk8MQW81TWiTxdQ8sl+V1qKr5IKwx26DigbqXFDtsdKyOpp4TGQXH +K59FqdudHnINn1pxaYP7fbnXybcczefkin5Qq4cx+gvxtPKh6JipaKiGyrgMgsBGaZ9 hEjMe3cOsI27XA8L9bImNPJr8c0ycdhmXkCfpNK2hi4fgDLkDKc/nTG2g5IxfIwzQRI6 d9rA== Sender: isar-users@googlegroups.com X-Forwarded-Encrypted: i=2; AKwUvBzJYl68QFXNLHCsLtAahRYUc8Zze/B7D/u7VbcvO5PkEYBH7mcYjBkFlC1Ce+exTGAba/YBMlc=@isar-build.org X-Gm-Message-State: AFuF++kkaoVDXZQfbUH7NMNWOs7fk1phmumvYBu7AgDicBqXfo7KuJBg bqesuNr87ZaquUW4e5M+/PQ4W5NfOUR1e/riruLdlmge8CRU9VS8AMIU X-Received: by 2002:a05:6402:f0b:b0:6a7:ee54:f8a9 with SMTP id 4fb4d7f45d1cf-6a7ee55007dmr372373a12.45.1788508555903; Fri, 04 Sep 2026 00:55:55 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com; h="ATskLdcL8ZJA15CrJb7qJamwyuGu22LEIfxd6Pw+ie+uoWIxYw==" Received: by 2002:a05:6402:46c3:b0:6a3:8652:a348 with SMTP id 4fb4d7f45d1cf-6a7e5476b66ls935198a12.0.-pod-prod-07-eu; Fri, 04 Sep 2026 00:55:53 -0700 (PDT) X-Received: by 2002:a05:6402:324c:b0:6a7:ee54:f897 with SMTP id 4fb4d7f45d1cf-6a7ee55003emr411525a12.27.1788508553457; Fri, 04 Sep 2026 00:55:53 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1788508553; cv=none; d=google.com; s=arc-20260327; b=SDgoL7UtzzKosNBnUFj3G2lSBXYoXc0I2w+1zrZnIwFFRdwYIwCb+SqJCo2G/5bh7Z k5IlfHghGB348fPfmjjpZveXn/Ddb3eKoiwLIqq3XweGfVH/L5DuVBIhF5xAkllcBKEh 0JgoFMjcvrhdxNRlJSTxUxB+pBBmAJsuNyA+ErnR4Cja0W/M9dHJxyC5I/i4fjMyTPxJ 4n3ul1yCOlgmXCc6GggmMhMo2lgQy5oJMdFUMWDHNI7GBOLkv90eVStww/w+7uY4UJjA /Y+1DLKeDl1LzOQCSFOXxaEm7BRhOKhBHqQ/Tm2YddoAgJNV2Yo8z7oZ+f42NawZWGUv Ew5w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20260327; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=c7tUl8Ha9JUC7r7/gBbyym43wDKTBFWPjeZIpPemO9E=; fh=IIbR0JxonYgzdrHDdB9TmZniSh/o1z0HofhfbmLXp8s=; b=mEexQRmc/rOAqMwodK3T3KfgsrKy66pbGswlfDDe1W5ygLb/+JIzejClCRVahdM85d frgGC4KRhc23nHcgc9d8a6etLnp1XeZw4xA6vWpQ6vL+EJ1qjNw6AEE6cfrMrKlag35d qKxcDwDP7GpIQbtF6iO1YiDARh5DAX929Z8dbi2RfQaDaT5uPSzArepHUrf19kV8u49z wylvTQs144gQ/oqCZoacq3ltsEsH/gRMAzeuxVSh/lBh2v4D0fNMQcICHnVKhtPB982C 7/eqgtbisXL7Ou7RvFy8FneptFWz4zWnNK6fUdICgDba2xaNjXpwJ3oY8R6V1S71F1GD msUQ==; dara=google.com ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of akarpovich@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=akarpovich@ilbers.de Received: from shymkent.ilbers.de (shymkent.ilbers.de. [85.214.156.166]) by gmr-mx.google.com with ESMTPS id 4fb4d7f45d1cf-6a7e687c201si43092a12.6.2026.09.04.00.55.53 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 04 Sep 2026 00:55:53 -0700 (PDT) Received-SPF: pass (google.com: domain of akarpovich@ilbers.de designates 85.214.156.166 as permitted sender) client-ip=85.214.156.166; Received: from akarpovich-Victus-by-HP-Laptop-16-e0xxx.promwad.corp (62.122.174.6.serverel.net [62.122.174.6] (may be forged)) (authenticated bits=0) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPSA id 6847tRSE004124 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 4 Sep 2026 09:55:51 +0200 From: Aliaksei Karpovich To: isar-users@googlegroups.com Cc: Uladzimir Bely , Aliaksei Karpovich Subject: [v10 04/12] meta: Use cached base-apt repo to bootstrap Date: Fri, 4 Sep 2026 09:52:46 +0200 Message-ID: <20260904075515.25962-5-akarpovich@ilbers.de> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260904075515.25962-1-akarpovich@ilbers.de> References: <20260904075515.25962-1-akarpovich@ilbers.de> MIME-Version: 1.0 X-Spam-Status: No, score=-4.6 required=5.0 tests=DKIMWL_WL_MED,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_EF,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,RCVD_IN_DNSWL_BLOCKED,RCVD_IN_MSPIKE_H2, RCVD_IN_RP_CERTIFIED,RCVD_IN_RP_RNBL,RCVD_IN_RP_SAFE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on shymkent.ilbers.de X-Original-Sender: akarpovich@ilbers.de X-Original-Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of akarpovich@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=akarpovich@ilbers.de Precedence: list Mailing-list: list isar-users@googlegroups.com; contact isar-users+owners@googlegroups.com List-ID: X-Spam-Checked-In-Group: isar-users@googlegroups.com X-Google-Group-Id: 914930254986 List-Post: , List-Help: , List-Archive: , List-Unsubscribe: , X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= From: Uladzimir Bely This patch makes local base-apt repo to be created before bootstrap task. So, bootstrap is then done from it. The required packages are downloaded via python-apt and reprepro creates debian-like repository from .deb files. For debian targets host keyring is used while ubuntu/raspbian targets use keys specified by DISTRO_BOOTSTRAP_KEYS variable. The goal is have workable base-apt repo before first build completed. Signed-off-by: Uladzimir Bely Signed-off-by: Aliaksei Karpovich --- meta/classes-recipe/bootstrap.bbclass | 44 +++++++++++++++++++ .../isar-mmdebstrap/isar-mmdebstrap-host.bb | 2 + .../isar-mmdebstrap/isar-mmdebstrap.inc | 2 + 3 files changed, 48 insertions(+) diff --git a/meta/classes-recipe/bootstrap.bbclass b/meta/classes-recipe/bootstrap.bbclass index f853a932..10cb23be 100644 --- a/meta/classes-recipe/bootstrap.bbclass +++ b/meta/classes-recipe/bootstrap.bbclass @@ -16,6 +16,7 @@ BOOTSTRAP_FOR_HOST ?= "0" APTPREFS = "${WORKDIR}/apt-preferences" APTSRCS = "${WORKDIR}/apt-sources" +APTSRCS_INIT = "${WORKDIR}/apt-sources-init" DISTRO_BOOTSTRAP_KEYFILES = "" THIRD_PARTY_APT_KEYFILES = "" DISTRO_BOOTSTRAP_KEYS ?= "" @@ -223,8 +224,51 @@ python do_apt_config_prepare() { aggregate_files(d, apt_preferences_list, apt_preferences_out) apt_sources_out = d.getVar("APTSRCS") + apt_sources_init_out = d.getVar("APTSRCS_INIT") apt_sources_list = get_aptsources_list(d) + aggregate_files(d, apt_sources_list, apt_sources_init_out) aggregate_aptsources_list(d, apt_sources_list, apt_sources_out) } addtask apt_config_prepare before do_bootstrap after do_unpack + +inherit debrepo + +debrepo_bootstrap_prepare() { + [ "${ISAR_PREFETCH_BASE_APT}" != "1" ] && return + [ "${ISAR_USE_CACHED_BASE_REPO}" = "1" ] && return + + debrepo_args="" + if [ "${BASE_DISTRO}" != "debian" ]; then + if [ "${BASE_DISTRO}" != "raspbian" ] && [ "${BASE_DISTRO}" != "raspios" ] || [ "${BOOTSTRAP_FOR_HOST}" = "0" ]; then + debrepo_args="$debrepo_args --keydir=${WORKDIR}" + fi + else + if [ "${BASE_DISTRO_CODENAME}" = "sid" ]; then + debrepo_args="$debrepo_args --keydir=${WORKDIR}" + fi + fi + if [ "${ISAR_ENABLE_COMPAT_ARCH}" = "1" ]; then + debrepo_args="$debrepo_args --compatarch=${COMPAT_DISTRO_ARCH}" + fi + + if [ "${BOOTSTRAP_FOR_HOST}" = "1" ]; then + debrepo_args="$debrepo_args --crossarch=${DISTRO_ARCH}" + fi + + if [ -n "${GNUPGHOME}" ]; then + export GNUPGHOME="${GNUPGHOME}" + fi + + ${SCRIPTSDIR}/deb-repo --init \ + --workdir="${DEBREPO_WORKDIR}" \ + --aptsrcsfile="${APTSRCS_INIT}" \ + --repodir="${REPO_BASE_DIR}" \ + --repodbdir="${REPO_BASE_DB_DIR}" \ + --mirror="${@get_distro_source(d)}" \ + --arch="${BOOTSTRAP_DISTRO_ARCH}" \ + --distro="${BOOTSTRAP_BASE_DISTRO}" \ + --codename="${BASE_DISTRO_CODENAME}" \ + ${debrepo_args} \ + ${DISTRO_BOOTSTRAP_BASE_PACKAGES} +} diff --git a/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap-host.bb b/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap-host.bb index fa4b76a6..52563d33 100644 --- a/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap-host.bb +++ b/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap-host.bb @@ -17,3 +17,5 @@ require isar-mmdebstrap.inc HOST_DISTRO_BOOTSTRAP_KEYS ?= "" DISTRO_BOOTSTRAP_KEYS = "${HOST_DISTRO_BOOTSTRAP_KEYS}" + +DEBREPO_WORKDIR = "${DEBREPO_HOST_DIR}" diff --git a/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc b/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc index 4d61e9be..c452035f 100644 --- a/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc +++ b/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc @@ -147,6 +147,8 @@ do_bootstrap() { fi fi bootstrap_args="--verbose --variant=minbase --include=${@','.join(d.getVar('DISTRO_BOOTSTRAP_BASE_PACKAGES').split())}" + debrepo_bootstrap_prepare + if [ -f "${DISTRO_BOOTSTRAP_KEYRING}" ]; then bootstrap_args="$bootstrap_args --keyring=${DISTRO_BOOTSTRAP_KEYRING}" cp "${DISTRO_BOOTSTRAP_KEYRING}" "${WORKDIR}/trusted.gpg.d/" From patchwork Fri Sep 4 07:52:47 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Aliaksei Karpovich X-Patchwork-Id: 5422 Return-Path: Received: from shymkent.ilbers.de ([unix socket]) by shymkent (Cyrus 2.5.10-Debian-2.5.10-3+deb9u2) with LMTPA; Fri, 04 Sep 2026 09:56:08 +0200 X-Sieve: CMU Sieve 2.4 Received: from mail-wm1-f58.google.com (mail-wm1-f58.google.com [209.85.128.58]) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPS id 6847u8FZ004327 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Fri, 4 Sep 2026 09:56:08 +0200 Received: by mail-wm1-f58.google.com with SMTP id 5b1f17b1804b1-490a767c7dcsf5248635e9.2 for ; Fri, 04 Sep 2026 00:56:08 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1788508558; cv=pass; d=google.com; s=arc-20260327; b=lk+c0ndfLNDX4B6QI81rYJVE7ir2aPIK0O7rK9WBm+WZ/07bcUsl2aEdtbmQ2Qz9Kb ujuafWbH2euhbx8rg2zF9uclf4SSe16NB4/c8YO+FTRPriHM66oU5VHWrQFOgIzTSeEz xJ8QPDK8VOIFiaTM9attioaFs8q+/Iv17y/UWQrLNCFOV+tSkQ+GDzVebR8PHjBKzHeg GFUHnjTG+VwEAR/XeZ2MG/Ao94mpa51SMq+ge2YvyzW72Q/IUjBK90BTTkEBxQPPTUSx zgtOVPvFV57AHzWn4WpCbpUnb55Cs3xydObJgEaohopZqNd8SIj7Jwc0NXuHaJy0Wf2y RbSQ== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20260327; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:dkim-signature; bh=oYZGu8CNF+UkMI7TSgHZgV9JufgkbvkgJDJaaa/+Wjo=; fh=reUdjcGzWHx943G31/HUdj4f35MoG86Oq3tLKk4PuKw=; b=GwLUM/kPAKOP/b6AMKsMnCUf/WL5/9olzLnjXrk92eFgFtDFY9cWIaig3W0OKITHpZ wcLUFbsS5tED6TodnIbGKjw7uUcypaUb2D1EnXgHvCVLuU1/HuWR8AmGRnvI+K2vIOpu b7jibLuL8+NZJgjRtaxjZ7HjlQm1mYhSSLqRnvevt6ev6EYh43TOusoGsQ5oxECjI9WV J8vgUPSDeJ2qWXDG7G7NJBhLpJlKrKGbJrSWeNPhlK+6lUzm6Is02601Rb0PVsHn6Ag4 wI0nflDn/h3BZ448w3XgMppEjPYn3Tf3EuvPKoNcEXlzX993oQIPV60cqQmPJcAh42w0 DNDg==; darn=isar-build.org ARC-Authentication-Results: i=2; gmr-mx.google.com; spf=pass (google.com: domain of akarpovich@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=akarpovich@ilbers.de DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlegroups.com; s=20251104; t=1788508558; x=1789113358; darn=isar-build.org; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:content-type :x-original-authentication-results:x-original-sender:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from:sender :from:to:cc:subject:date:message-id:reply-to:content-type; bh=oYZGu8CNF+UkMI7TSgHZgV9JufgkbvkgJDJaaa/+Wjo=; b=OmYOvXPprdERUljI3tDSOBScKQC8onQ3q40PCaIUoLLaUNu7KdVJS/xOWIbysoBeXF xUx3FrldJZUJACjWLQPlVkUBXuggrQaupsnr62eswc6BK+yPyzpCxNDPvjAjf0BVlP3Z Esv4t7FHEqVIjzEm0olJNAwuOkKHOhaqAuBzmtthZyT+KSC5KVuYshepY5g6ZKpyLMIB ywAGgckKMU4RGsqDhWiME0fviwT23529wUxTouu0nSCync8/k6OkOOJOthIIyv0nNxN+ WAYhpJAluumPbcBuKkW6n5Pp8bTiGSPy3olc+YAhgedoVscuQYrOnKDE1T7SV0D4nycA iSlQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1788508558; x=1789113358; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :x-spam-checked-in-group:list-id:mailing-list:precedence :content-type:x-original-authentication-results:x-original-sender :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:x-beenthere:x-gm-message-state:sender:from:to:cc:subject:date :message-id:reply-to:content-type; bh=oYZGu8CNF+UkMI7TSgHZgV9JufgkbvkgJDJaaa/+Wjo=; b=ErMhjxetPq6iXlIFeKUSORW8CbjDjIkhhNFFBpKgC1m7UU8lLtFgOnjdMpymuKIWPS zJp/r9UVTzf9jPFR9XHuuexVLwbI7aY3XxeN8xb1HGjnBGFYdren8hq4X6N7btnUm3ug t1sPCoFOe9owq11px5nLNOYnLnT4T1Ki3rqDFzq/xopd7gofExrz7dZVsIt7JBNg39Tv F1yihxz2APuVXwdNV9k9gbKRNR7IEI9iGfTr+UmxInnegOV3OhUW5+1OMLe6/abHfeqm SQPrK6eOEx6SS/CzE6J/Myi/r3qi13xDwBRV6v+VKCDkf77cSU2Wc+OpP5tI/ooHmhHf kxoQ== Sender: isar-users@googlegroups.com X-Forwarded-Encrypted: i=2; AKwUvBz1rQKPHXJ/JDRBP2Lm/TQgVb4D4iuNq7+9sUhl9ciGY9n9P1z/DBXoLSs+qwoH/c+kmuSr2Y0=@isar-build.org X-Gm-Message-State: AFuF++ldtHl7TbV9lC+O57vLg58typ48MoBBg9AFpZYXm0Zbn0fTeP4N wKzTe54DSquNUfh14it8QGPZw6WP8ARbfNNLEBqgbbmC7zfdByPTJuxN X-Received: by 2002:a05:600c:8716:b0:49a:a101:4157 with SMTP id 5b1f17b1804b1-49cf81f4924mr45843235e9.7.1788508557835; Fri, 04 Sep 2026 00:55:57 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com; h="ATskLdclBycwh5OFZyO2ds4yuet3M7zSQqrE7lPTG8aG8tIzSQ==" Received: by 2002:a05:600c:1f82:b0:49c:f166:cf1a with SMTP id 5b1f17b1804b1-49cf7b6db17ls5588315e9.2.-pod-prod-06-eu; Fri, 04 Sep 2026 00:55:55 -0700 (PDT) X-Received: by 2002:a05:600c:608a:b0:49c:fc6c:bdfd with SMTP id 5b1f17b1804b1-49cfc6cbfe7mr14067415e9.20.1788508555500; Fri, 04 Sep 2026 00:55:55 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1788508555; cv=none; d=google.com; s=arc-20260327; b=qrvQNvzGdwPZ+TaCrKcI7aIw/N6BQ8yAiJNp4ViBPvB3cQUfYZv/izVM7WR6VFz7k0 i6papOVtLTbaGEkY8HgqFgt2hmyZ8HcSKNlx5Kt5Z9uUoCZMfhMI89Ev7EkGGN06CR+4 A+dDAe90K96KP9Wt3ULQGtN6w3Nza2BgcycB6Hs+J8RBGVFU7TshjdmMCyUPSV5CUmth ZuoTxC5Ged2ncH+kCC7J1VLIuLvh5nHWQE5C4GWQ/CUeX63L4p+buE2ZgYXeSHcUYGu/ tJXZ34OKMvPlM07qPS+93q9lZY9+LWmmvLT3PyXh5d4mFNpn0O8u5Dgljd7H1df5oXZY lXrw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20260327; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=MIrzxeLR+T1X+QeRGxogTFaDmrhkI2/QpRguUHLC7fI=; fh=IIbR0JxonYgzdrHDdB9TmZniSh/o1z0HofhfbmLXp8s=; b=rifiW1DCEwjJN27inFVwkHZDdXU8q77KhyyoSbzNDrQ+rExFWHjLtDFt38iIeeHVur zJnXE2V5m1fFGaoL4n0LDdMa/kQdk/jS5fb3HS2haafw9Qx0UPd6mtyMx/rCvP2XF/Uo Y97AzfSIqzb9OZgVNtuQdNCxDgmzbUfx219O3wmCaEu2g2W+3VAKNcEh4BoAGyJ0BPW2 DTl4taDEKmDpHvNqpOdxbxEPsu2tdJsUslDMUTkB1zC31fhlrygqYdxmcJvqf5uEqR+I YMenXZgB3I0Z/46qQ3wJN9xFraRIz+ICau4XBp1soqVJPRPYNGd4HZUR2YixDhDyeBNZ YKmw==; dara=google.com ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of akarpovich@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=akarpovich@ilbers.de Received: from shymkent.ilbers.de (shymkent.ilbers.de. [85.214.156.166]) by gmr-mx.google.com with ESMTPS id 5b1f17b1804b1-49cf76eef24si378015e9.2.2026.09.04.00.55.55 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 04 Sep 2026 00:55:55 -0700 (PDT) Received-SPF: pass (google.com: domain of akarpovich@ilbers.de designates 85.214.156.166 as permitted sender) client-ip=85.214.156.166; Received: from akarpovich-Victus-by-HP-Laptop-16-e0xxx.promwad.corp (62.122.174.6.serverel.net [62.122.174.6] (may be forged)) (authenticated bits=0) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPSA id 6847tRSF004124 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 4 Sep 2026 09:55:53 +0200 From: Aliaksei Karpovich To: isar-users@googlegroups.com Cc: Uladzimir Bely , Aliaksei Karpovich Subject: [v10 05/12] base-apt: Predownload packages to base-apt before install Date: Fri, 4 Sep 2026 09:52:47 +0200 Message-ID: <20260904075515.25962-6-akarpovich@ilbers.de> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260904075515.25962-1-akarpovich@ilbers.de> References: <20260904075515.25962-1-akarpovich@ilbers.de> MIME-Version: 1.0 X-Spam-Status: No, score=-4.6 required=5.0 tests=DKIMWL_WL_MED,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_EF,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,RCVD_IN_DNSWL_BLOCKED,RCVD_IN_MSPIKE_H2, RCVD_IN_RP_CERTIFIED,RCVD_IN_RP_RNBL,RCVD_IN_RP_SAFE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on shymkent.ilbers.de X-Original-Sender: akarpovich@ilbers.de X-Original-Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of akarpovich@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=akarpovich@ilbers.de Precedence: list Mailing-list: list isar-users@googlegroups.com; contact isar-users+owners@googlegroups.com List-ID: X-Spam-Checked-In-Group: isar-users@googlegroups.com X-Google-Group-Id: 914930254986 List-Post: , List-Help: , List-Archive: , List-Unsubscribe: , X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= From: Uladzimir Bely This patch uses debrepo script to predownload packages to base-apt repository before they are installed in rootfs. Signed-off-by: Uladzimir Bely Signed-off-by: Aliaksei Karpovich --- meta/classes-recipe/crossvars.bbclass | 1 + meta/classes-recipe/dpkg-base.bbclass | 1 + meta/classes-recipe/dpkg.bbclass | 8 ++++ .../image-locales-extension.bbclass | 5 ++ .../image-tools-extension.bbclass | 13 ++++++ meta/classes-recipe/rootfs.bbclass | 5 +- meta/lib/aptsrc_fetcher.py | 46 +++++++++++++++---- .../sbom-chroot/sbom-chroot.bb | 2 + .../sbuild-chroot/sbuild-chroot-host.bb | 2 + 9 files changed, 74 insertions(+), 9 deletions(-) diff --git a/meta/classes-recipe/crossvars.bbclass b/meta/classes-recipe/crossvars.bbclass index 7cf2d660..6f070714 100644 --- a/meta/classes-recipe/crossvars.bbclass +++ b/meta/classes-recipe/crossvars.bbclass @@ -30,6 +30,7 @@ python __anonymous() { schroot_dir = d.getVar('SCHROOT_HOST_DIR', False) sbuild_dep = "sbuild-chroot-host" + flavor_suffix + ":do_build" sdk_toolchain = "crossbuild-essential-" + distro_arch + d.setVar('DEBREPO_WORKDIR', d.getVar('DEBREPO_HOST_DIR')) else: d.setVar('BUILD_ARCH', distro_arch) schroot_dir = d.getVar('SCHROOT_TARGET_DIR', False) diff --git a/meta/classes-recipe/dpkg-base.bbclass b/meta/classes-recipe/dpkg-base.bbclass index 4a81787e..91b17768 100644 --- a/meta/classes-recipe/dpkg-base.bbclass +++ b/meta/classes-recipe/dpkg-base.bbclass @@ -11,6 +11,7 @@ inherit terminal inherit repository inherit deb-dl-dir inherit essential +inherit debrepo # Local (WORKDIR-internal) dir where do_dpkg_build collects the built debs. # Its content is exported into the shared, sstate-tracked DEPLOY_DIR_DEB so diff --git a/meta/classes-recipe/dpkg.bbclass b/meta/classes-recipe/dpkg.bbclass index a42703d5..323202e4 100644 --- a/meta/classes-recipe/dpkg.bbclass +++ b/meta/classes-recipe/dpkg.bbclass @@ -117,6 +117,12 @@ dpkg_runbuild() { echo '$stalled_pkg_timeout = ${DPKG_BUILD_TIMEOUT};' >> ${SBUILD_CONFIG} DSC_FILE=$(find ${DEPLOY_DIR_SRC} -maxdepth 1 -name "${DEBIAN_SOURCE}_*.dsc" -print) + debrepo_parse_dscfile "${DSC_FILE}" + + locked_update_cmd=":" + if [ "${ISAR_PREFETCH_BASE_APT}" = "1" ]; then + locked_update_cmd="flock -x /base-apt/repo.lock -c 'apt-get -y update'" + fi # networking is automatically enabled on older versions of sbuild sbuild_network_option="" @@ -139,9 +145,11 @@ dpkg_runbuild() { --chroot-setup-commands="rm -f /var/log/dpkg.log" \ --chroot-setup-commands="mkdir -p ${deb_dir}" \ --chroot-setup-commands="find ${ext_deb_dir} -maxdepth 1 -name '*.deb' -exec ln -t ${deb_dir}/ -sf {} +" \ + --chroot-setup-commands="${locked_update_cmd}" \ --chroot-setup-commands="apt-get update -o Dir::Etc::SourceList=\"sources.list.d/isar-apt.list\" -o Dir::Etc::SourceParts=\"-\" -o APT::Get::List-Cleanup=\"0\"" \ --finished-build-commands="rm -f ${deb_dir}/sbuild-build-depends-*-dummy_*.deb" \ --finished-build-commands="find ${deb_dir} -maxdepth 1 -type f -name '*.deb' -print -exec cp ${CP_FLAGS} -t ${ext_deb_dir}/ {} +" \ + --finished-build-commands="mkdir -p ${ext_root}" \ ${@ '--finished-build-commands="cp /var/log/dpkg.log $ext_root/dpkg_partial.log"' if d.getVar('ISAR_CHROOT_MODE') == 'schroot' else '' } \ --build-path="" --build-dir=${WORKDIR} --dist="${DEBDISTRONAME}" ${DSC_FILE} diff --git a/meta/classes-recipe/image-locales-extension.bbclass b/meta/classes-recipe/image-locales-extension.bbclass index c03f34c0..32b5e19c 100644 --- a/meta/classes-recipe/image-locales-extension.bbclass +++ b/meta/classes-recipe/image-locales-extension.bbclass @@ -6,6 +6,8 @@ # This class extends the image.bbclass for setting locales and purging unneeded # ones. +inherit debrepo + LOCALE_GEN ?= "en_US.UTF-8 UTF-8\n\ en_US ISO-8859-1\n" LOCALE_DEFAULT ?= "en_US.UTF-8" @@ -29,6 +31,9 @@ ROOTFS_INSTALL_COMMAND_BEFORE_EXPORT += "image_install_localepurge_download" image_install_localepurge_download[weight] = "40" image_install_localepurge_download[network] = "${TASK_USE_NETWORK_AND_SUDO}" image_install_localepurge_download() { + debrepo_add_packages "${DEBREPO_WORKDIR}" "localepurge" + debrepo_update_apt_source_list "${ROOTFSDIR}" "base-apt" + run_privileged_heredoc <<'EOF' set -e ${@insert_isar_mounts(d, d.getVar('ROOTFSDIR'), d.getVar('ROOTFS_MOUNTS') if d.getVar('ISAR_CHROOT_MODE') == 'unshare' else '')} diff --git a/meta/classes-recipe/image-tools-extension.bbclass b/meta/classes-recipe/image-tools-extension.bbclass index 8f666444..229ce318 100644 --- a/meta/classes-recipe/image-tools-extension.bbclass +++ b/meta/classes-recipe/image-tools-extension.bbclass @@ -6,6 +6,11 @@ # This file extends the image.bbclass to supply tools for futher imager functions inherit sbuild +inherit debrepo + +python __anonymous() { + d.setVar('DEBREPO_WORKDIR', d.getVar('DEBREPO_TARGET_DIR')) +} IMAGER_INSTALL ??= "" IMAGER_BUILD_DEPS ??= "" @@ -60,12 +65,20 @@ imager_run_schroot() { echo "Installing imager deps: ${local_install}" distro="${BASE_DISTRO}-${BASE_DISTRO_CODENAME}" + debrepo_workdir=${DEBREPO_TARGET_DIR} if [ ${ISAR_CROSS_COMPILE} -eq 1 ]; then distro="${HOST_BASE_DISTRO}-${BASE_DISTRO_CODENAME}" + if [ ${HOST_ARCH} != ${DISTRO_ARCH} ]; then + debrepo_workdir=${DEBREPO_HOST_DIR} + fi fi E="${@ isar_export_proxies(d)}" deb_dl_dir_import ${schroot_dir} ${distro} + + debrepo_add_packages --isarapt "${debrepo_workdir}" "${local_install}" + debrepo_update_apt_source_list "${schroot_dir}" "base-apt" + ${SCRIPTSDIR}/lockrun.py -r -f "${REPO_ISAR_DIR}/isar.lock" -s < X-Patchwork-Id: 5423 Return-Path: Received: from shymkent.ilbers.de ([unix socket]) by shymkent (Cyrus 2.5.10-Debian-2.5.10-3+deb9u2) with LMTPA; Fri, 04 Sep 2026 09:56:21 +0200 X-Sieve: CMU Sieve 2.4 Received: from mail-wm1-f58.google.com (mail-wm1-f58.google.com [209.85.128.58]) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPS id 6847uLUq004644 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Fri, 4 Sep 2026 09:56:21 +0200 Received: by mail-wm1-f58.google.com with SMTP id 5b1f17b1804b1-49cd74fc1aasf4076725e9.1 for ; Fri, 04 Sep 2026 00:56:21 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1788508576; cv=pass; d=google.com; s=arc-20260327; b=Y4AN6OEClNqa2j5i2/NeihpWf4GQCsur+dSITv5VQ+N5RwihOTb93XrBQ3IxUdML4t rA2lC+D1X6jb8UGoYf1LxNo2O5v53C8Oh9pMVOaVjvFGMJzNROGHH+lcUEjXB5MEC8LU ZxPU5jbNNvfDkv9u+8+DLuYikjL0cg3CSa4lYnF+n11YohgcYd7KQQr7qfpTQGuteYTz gmc/hm035zmfnkC4oqwhkjLvnQpxi/sl5LFwPaQvYAtyCxk7Ag/D3sFH5l5uMeRaWSK2 4ejj6SxklZZEhBTjzseoU664hUeissPm8TLaORk8f0oU8TP0jlsdrR8Gj8RRuudS32hl U7eA== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20260327; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:dkim-signature; bh=1VcGL/R6usAIkK4pd7nGxYokkqW17JE6X3WCSqxBLHA=; fh=L2bvKG9tDnQO/G1hn77TlJ5SV1BBfBF2zi0AaKiyQAc=; b=CwWmYbInxjE4b7PW2m+YVKgmDLovLst2uDfrmq52GJYtJYVdm0ZNfP/qz6FCtRFCQJ nKboKYNP+zM11AaVwbHrGU/kqxcVTzVFqcgp6rS/ii+hsGi2tJBkB8q5zbQCvCfwG2CN 9XjqAyZ0AMkL+hgTPQsecVQ/tZibfRFAldMStdcIY5Pr6y6zUoJReDf0MuiiCzigB/DN qQs9bmCHQ9O8AVabCYqsn06djQz1wOI2sZKGsSEtcGkBM9DNofacRgb5StoMS2K1hxrh Eer7G8ugZrmVJJHbKlrBUuiNeitCsklKRCiGWWviKwxTTsktTFS9kT29L9tvWgVBWF91 fzNA==; darn=isar-build.org ARC-Authentication-Results: i=2; gmr-mx.google.com; spf=pass (google.com: domain of akarpovich@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=akarpovich@ilbers.de DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlegroups.com; s=20251104; t=1788508576; x=1789113376; darn=isar-build.org; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:content-type :x-original-authentication-results:x-original-sender:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from:sender :from:to:cc:subject:date:message-id:reply-to:content-type; bh=1VcGL/R6usAIkK4pd7nGxYokkqW17JE6X3WCSqxBLHA=; b=JilH+S2xYeaYnC/eV2rzV+ajFGRjyiJ0nht+W4LAGsXVRin2SMBTkBa0F2pY1QnBKM Nm1blOqG7Wv7Jlv1Tf4HgEzXfadqWMcksCdzSwmwaIe6wBD1eVdADt13ltSepuXqraU8 LiOh9d9GU8Ucu5MT3TK8Jojar8Wx8oftv+IrD1Pw1sxIN6uewLJIWtUCIYQiTTgPnOK+ MmPrJygdDaous9OAOMEnzHocNjociHxN5wD66tKUyP3d43jC3Jf4qq3PbOETMLXPI1Gq eIRTPDVwV9qxPtBBu1y9yn0kHesbHb/oYaU1a12+J8FS58FJNIJv6cpq7dG33GYkv1X4 +mJA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1788508576; x=1789113376; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :x-spam-checked-in-group:list-id:mailing-list:precedence :content-type:x-original-authentication-results:x-original-sender :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:x-beenthere:x-gm-message-state:sender:from:to:cc:subject:date :message-id:reply-to:content-type; bh=1VcGL/R6usAIkK4pd7nGxYokkqW17JE6X3WCSqxBLHA=; b=F6pienCySOKM2sB9mG4EeUSA28f5yoL63J3Y+/WLIOZ08NOtLhfXpaEaTLHm+mrXs3 K9Ln/SsQlksIk131vm63Bdk7lp2fozyQccDHzk+vRoseRaW6IcKbP0ynHOQ8Hp3USUBi 0i3/Bv7db7/1hHX9uXwg7zwXfBJ2VA1bGjBBwZ3m8Nc9wFP1Bzt+mAOMAOeNXu4lJweb UjTyEiDXHk36gEPHByF3CEEEVOlgHDlVh/L4KgrdBuHiPbbMjDMdtAZJSbC8c5VWilKI HwvOnE/HX3TuYBIiGr2UPWgMeplswpwoVXAseTlmhgS3RC/0dpou0Ma8Dx9BVeRYJ2uL dhFw== Sender: isar-users@googlegroups.com X-Forwarded-Encrypted: i=2; AKwUvBxerzeHW/ZAd/ngbFaNaKwlTv0ERnqIiVe4U02yTh+oWQfu0mE6oFOkOar5Akx88VaLfZWVQhE=@isar-build.org X-Gm-Message-State: AFuF++m1qqDJtKa+LxM/464oE28TuC272/plaB67KlZSMOMDb1XA1VKE Y64AJ5wX6Zz3KHo/d9WmY9oXdDuC1lwOLJNrFEmmDp2+KUfHWjxQk0a9 X-Received: by 2002:a05:600c:6212:b0:49c:ed88:a782 with SMTP id 5b1f17b1804b1-49cf81fa7fbmr38946045e9.6.1788508575655; Fri, 04 Sep 2026 00:56:15 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com; h="ATskLdfmApVDjH55MpWN9HKTcfqMVDCm3W2M6SW9JN7e9tFkZA==" Received: by 2002:a05:600c:4f0d:b0:49c:f4da:1534 with SMTP id 5b1f17b1804b1-49cf6879450ls8712255e9.1.-pod-prod-04-eu; Fri, 04 Sep 2026 00:56:13 -0700 (PDT) X-Received: by 2002:a05:6000:2505:b0:485:8ad5:f965 with SMTP id ffacd0b85a97d-4858ad5ff97mr2345213f8f.2.1788508573480; Fri, 04 Sep 2026 00:56:13 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1788508573; cv=none; d=google.com; s=arc-20260327; b=EbsYDS+IacvPURAigNndmCau0iHmedmFmWvKLepC31DBaETnTKYUSR7ICJocC/JOXg A1ujQUPKIslhiWljQ3zlxWCmFdmxxbzlcV1PrYuBkAYwuggoa6c4YQ2+fOo8SrV4NIGS mPFVkTkjmAO+jmy/xV+JTIpquTOVZn53rkS7tteOyurzsqoFqJDLBRPE1um/Fg+t/viV qON5cDjHbFposrMFfHkF7bOGAHBkKj7wtaSQ04c037atn8AV7eqQdsKI26gQ6AJvoNMV Dq3VL4tjg2/J2oXRNJXX3EUysVbfhftz/PpRTP2oThxX+FWgUXU4UaWjjYiGcuy7CkTO X0kw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20260327; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=ktDVAHWQ+i3P8o/c+MKbAEM8KNhoe4y+/bu4Wo8SlQM=; fh=IIbR0JxonYgzdrHDdB9TmZniSh/o1z0HofhfbmLXp8s=; b=WUzp6HEM+MuwF65Aw01FjUkjLm96WzmwtvXumLiCg4iIzDR3uiFINX9Gzjkt3trVH9 6h1gxZLD/xzNLSc6+mF4YDbCJNgfK2X9Ge6fs4SOYjo1sFD2vKWHvcpTx9L3PoBFiFAC hc1JqlkKRSFANVR6CJzsI2YSWR4zZ5MiAOu/M0b45r+eqBGhQwC9I9FOpYDRJ3W1+nJn iubzcRjEKnZZ2ZMk8Dzf0PCXylfVlGBWgBl5x3L0AOp69OKLMvVt25KGULYc+XCRSdiq W30OJEQSrSZS3RJ7Hd4qZlgsP6gC0mP9MtW1EG7E34lIfFR/HP4HLkf0DFcbKRF7YExC 4xvw==; dara=google.com ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of akarpovich@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=akarpovich@ilbers.de Received: from shymkent.ilbers.de (shymkent.ilbers.de. [85.214.156.166]) by gmr-mx.google.com with ESMTPS id ffacd0b85a97d-485885b717dsi31034f8f.4.2026.09.04.00.56.13 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 04 Sep 2026 00:56:13 -0700 (PDT) Received-SPF: pass (google.com: domain of akarpovich@ilbers.de designates 85.214.156.166 as permitted sender) client-ip=85.214.156.166; Received: from akarpovich-Victus-by-HP-Laptop-16-e0xxx.promwad.corp (62.122.174.6.serverel.net [62.122.174.6] (may be forged)) (authenticated bits=0) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPSA id 6847tRSG004124 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 4 Sep 2026 09:55:55 +0200 From: Aliaksei Karpovich To: isar-users@googlegroups.com Cc: Uladzimir Bely , Aliaksei Karpovich Subject: [v10 06/12] meta: Add cache-deb-src functionality in base-apt mode Date: Fri, 4 Sep 2026 09:52:48 +0200 Message-ID: <20260904075515.25962-7-akarpovich@ilbers.de> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260904075515.25962-1-akarpovich@ilbers.de> References: <20260904075515.25962-1-akarpovich@ilbers.de> MIME-Version: 1.0 X-Spam-Status: No, score=-4.6 required=5.0 tests=DKIMWL_WL_MED,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_EF,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,RCVD_IN_DNSWL_BLOCKED,RCVD_IN_MSPIKE_H2, RCVD_IN_RP_CERTIFIED,RCVD_IN_RP_RNBL,RCVD_IN_RP_SAFE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on shymkent.ilbers.de X-Original-Sender: akarpovich@ilbers.de X-Original-Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of akarpovich@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=akarpovich@ilbers.de Precedence: list Mailing-list: list isar-users@googlegroups.com; contact isar-users+owners@googlegroups.com List-ID: X-Spam-Checked-In-Group: isar-users@googlegroups.com X-Google-Group-Id: 914930254986 List-Post: , List-Help: , List-Archive: , List-Unsubscribe: , X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= From: Uladzimir Bely Fill base-apt repo with source packages. Signed-off-by: Uladzimir Bely Signed-off-by: Aliaksei Karpovich --- meta/classes-recipe/deb-dl-dir.bbclass | 41 ++++++++++++++++++++++++++ meta/classes-recipe/rootfs.bbclass | 7 +++++ 2 files changed, 48 insertions(+) diff --git a/meta/classes-recipe/deb-dl-dir.bbclass b/meta/classes-recipe/deb-dl-dir.bbclass index 60e2fecc..7e9f89b1 100644 --- a/meta/classes-recipe/deb-dl-dir.bbclass +++ b/meta/classes-recipe/deb-dl-dir.bbclass @@ -27,6 +27,47 @@ debsrc_source_version_filter() { | sort -u } +debsrc_fill_base_apt() { + export rootfs="$1" + export rootfs_distro="$2" + mkdir -p "${DEBSRCDIR}"/"${rootfs_distro}" + + ( flock 9 + set -e + printenv | grep -q BB_VERBOSE_LOGS && set -x + + # We need temporary files for our lists of source packages + # trap exit of this sub-shell to remove them (this script may exit abruptly + # since "set -e" is used) + avail=$(mktemp) + wanted=$(mktemp) + trap "rm -f ${avail} ${wanted}" EXIT + + # List all packages known to apt + rootfs_cmd \ + --bind "${DEBSRCDIR}" "/deb-src" \ + --bind "${rootfs}" "${rootfs}" \ + -- \ + apt-cache dumpavail \ + | debsrc_source_version_filter > ${avail} + + # Use apt-ftparchive to scan all .deb files found in the download directory + # and get the pairs that we wish to download + apt-ftparchive --md5=no --sha1=no --sha256=no --sha512=no \ + -a "${DISTRO_ARCH}" packages \ + "${REPO_BASE_DIR}" \ + | debsrc_source_version_filter > ${wanted} + + # We now have two sorted lists: source packages we want and those known to + # apt. We will only consider source packages that may be found in both. + comm -12 ${wanted} ${avail} \ + | while read src version; do + debrepo_add_packages --srcmode "${DEBREPO_TARGET_DIR}" "${src}=${version}" + done + + ) 9>"${DEBSRCDIR}/${rootfs_distro}.lock" +} + debsrc_download() { export rootfs="$1" export rootfs_distro="$2" diff --git a/meta/classes-recipe/rootfs.bbclass b/meta/classes-recipe/rootfs.bbclass index d1f24d68..bdca9b42 100644 --- a/meta/classes-recipe/rootfs.bbclass +++ b/meta/classes-recipe/rootfs.bbclass @@ -489,6 +489,13 @@ do_cache_deb_src() { run_privileged tar -xf "${BOOTSTRAP_SRC}" ./var/lib/apt/lists --one-top-level="${ROOTFSDIR}" deb_dl_dir_import ${ROOTFSDIR} ${ROOTFS_BASE_DISTRO}-${BASE_DISTRO_CODENAME} + + debsrc_fill_base_apt ${ROOTFSDIR} ${ROOTFS_BASE_DISTRO}-${BASE_DISTRO_CODENAME} + + rootfs_do_mounts_priv + debrepo_update_apt_source_list "${ROOTFSDIR}" "base-apt" + rootfs_do_umounts_priv + debsrc_download ${ROOTFSDIR} ${ROOTFS_BASE_DISTRO}-${BASE_DISTRO_CODENAME} run_privileged rm -f "${ROOTFSDIR}"/etc/resolv.conf From patchwork Fri Sep 4 07:52:49 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Aliaksei Karpovich X-Patchwork-Id: 5427 Return-Path: Received: from shymkent.ilbers.de ([unix socket]) by shymkent (Cyrus 2.5.10-Debian-2.5.10-3+deb9u2) with LMTPA; Fri, 04 Sep 2026 09:56:39 +0200 X-Sieve: CMU Sieve 2.4 Received: from mail-wr1-f56.google.com (mail-wr1-f56.google.com [209.85.221.56]) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPS id 6847uMoT004660 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Fri, 4 Sep 2026 09:56:22 +0200 Received: by mail-wr1-f56.google.com with SMTP id ffacd0b85a97d-482e05af072sf565670f8f.3 for ; Fri, 04 Sep 2026 00:56:22 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1788508577; cv=pass; d=google.com; s=arc-20260327; b=pSvpPkiMwgvYj+4AMzLA7PU9dbXLJYQa5fBEhR44nq7lIik2XH88tsxQiEhDbPcD8d WAm1UftBb9G/akytS+u1ufIJcGQZZBBgPLkyy/meEEkVeRZgqGvsCd6y9NWoQvRixJ3Z zlJ7H6FwPdClHv3HEwmPMnXvJ3p/bdyuicBy+bq/Ku1Rnyf8AbP+LMmcbnze6DXTijLi wW4iq7V8lSvJjJwPvL6P8kGImOVIVyruw2Yxv9iPSng55GUPHqEEzypWLkpMHWGGpbK1 zj0dowA2RuJVz3Bm6l/64Q2x6DBCNbCr1ULdN78YSFprKIzLsqRYxqkoLf8NauPskHSX 6YnQ== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20260327; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:dkim-signature; bh=3EjbrRxULyXSodUzEwVr9OqAwxTs2MFvts4iqxDCqQI=; fh=OBoONMG5M6cMJiVblj5h1EvhqaBta+hsOWtm6pXr0qY=; b=fTYYiYH7dgGiMZdDDYIEpWgIkkw0Wntc2FiOr4dxU3Ylh1ns0Qahne9xM7D0alLY62 lDE1nEEPSxh5ni7NugseWazolwizuP8aMaKhFWsO7pJ8Nuj7yX9gXiYlHL+l8IijledD rsm1S14VlpRoiqkzmJZ6YnlTwV6YVyRDCdqPYdJQObhod4hZvTshhJObXxEd859PAUM7 z9t+16hUgld1CiBJ1cfwwNG7a+M1OVxvazKhDwyR6b3t4RnaRYHSd6IVC52IDezi9Z1D 9SkUjJLN8iu6SKDn71tAz+y+ePHEWTkmGeiUVym2wOuLi71oKOkWLYxSGxWREqW9uzUM YodQ==; darn=isar-build.org ARC-Authentication-Results: i=2; gmr-mx.google.com; spf=pass (google.com: domain of akarpovich@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=akarpovich@ilbers.de DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlegroups.com; s=20251104; t=1788508577; x=1789113377; darn=isar-build.org; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:content-type :x-original-authentication-results:x-original-sender:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from:sender :from:to:cc:subject:date:message-id:reply-to:content-type; bh=3EjbrRxULyXSodUzEwVr9OqAwxTs2MFvts4iqxDCqQI=; b=lSsvTtWU/8/1iEqeImh1pjmSO5m+ZfUOZmVksRdkTBQtcbv1z5f4Tkgf7zNaiKF3Rx aqbBU00U3H5K6ReEQjOmJuD5iv3cFctvpsJNQaiFloBF5iZRd4rRaCGsBUVtGRbIo81M cFl25p7uWq9SSqEc2eOAnpz8XCSs8wjK+7ViqDXPvPDKofjbQHzSToN0pkecL8dyXEXo BoUZ4P66tkNVlxGJW004bBI+uRj7Uw0Hix5m5qJHElWJS2rxt218MLWlKJ0F9rhPa2St pIDQRs0gg5IRdNRHUTXZUGGOpiTgDVBYL+tdbxRqvq3Pnt7fD/97KCwjd6SP1/dFTDo3 L7ow== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1788508577; x=1789113377; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :x-spam-checked-in-group:list-id:mailing-list:precedence :content-type:x-original-authentication-results:x-original-sender :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:x-beenthere:x-gm-message-state:sender:from:to:cc:subject:date :message-id:reply-to:content-type; bh=3EjbrRxULyXSodUzEwVr9OqAwxTs2MFvts4iqxDCqQI=; b=CYpUICuEXtOT11gip4TDY9SNlFJM5RuJXcFzrmDRFKZflt/bhn9XFLok0OHgNpDrtd eiPO+0ToYmaE10tWcs/hTpVqNQFiUEPn4y938kUCcWnpbyOZGuGf7AKo3sv4d7Akd0eW u8OMh5GNOk5iXsN1Hd3l8QR5f05Z9Cb4dpk0ulSkCjExZzur/TYUAbVRfSNjzuFbf/AE 5zYTRGDnFVk0ijBEPpbnBxxc3eslb44UYWb2oxsYJB71PFstKfsP31AddSgV+D9irAm/ QZda8vcROLYnK5ovQK6O7ttuN+p0YC//qRrM+0PtzFNTydpKrb/oLQ2F5dgFARPqSI0r +gvQ== Sender: isar-users@googlegroups.com X-Forwarded-Encrypted: i=2; AKwUvBxSqyB3Nmkq/12cXCcFeZPcl3TMHxvCwkOvbJJ9W8AbFMgfmyzYvkaifLd8l1dUoWbU1bZzHRc=@isar-build.org X-Gm-Message-State: AFuF++nRwz+9aH75cv34tkWnd+iUdSKApPiHNqUmrrHUWNkBuIjD19Qm semdplFKyvAhhMjr09M0hchgHXM+wmCslUnbwdqKZpGgtlJqs84AkkcL X-Received: by 2002:a05:6000:2c05:b0:485:8c17:9771 with SMTP id ffacd0b85a97d-4858c179a47mr1569056f8f.51.1788508576976; Fri, 04 Sep 2026 00:56:16 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com; h="ATskLdema9n6/tPJ+BNd65DZa7ql020ZL00TvIUtWGXgzcmAvw==" Received: by 2002:a05:6000:25e3:b0:481:5ae8:2abd with SMTP id ffacd0b85a97d-485864d8444ls597249f8f.1.-pod-prod-06-eu; Fri, 04 Sep 2026 00:56:15 -0700 (PDT) X-Received: by 2002:a05:6000:4b05:b0:485:8c17:9772 with SMTP id ffacd0b85a97d-4858c17998emr1381834f8f.52.1788508574707; Fri, 04 Sep 2026 00:56:14 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1788508574; cv=none; d=google.com; s=arc-20260327; b=RtKYmDHj2Y4YQdpyQpICk38c3QUhHdVu8qmG65UVmqXISAQ/OrIjjP+maLJh06AQbm D7bY6I7ivCu5Yj3EzdqgVIItEkReG3T1kfOdjRYRfjwaZXYiKKKksLv02tZO9ixE6jav YPYdMv1Ofa9KGwjEQABpiNGySi6CNsLQJhHTXtdF2SNxXrCh2RatTuYL6lfmd0Gl3vD1 n+zm0byr4tedgOeWlJ9twCYFyqgkoup/a8d9fMR16ug7npWNBuYaCojyEVw+lwrvgq9q tGud804j5vcDhYmPxYTn1khSBe1debBuhDtE8b0tYEUFJesdacXoTA6yA7QaMX7VhOc+ Ms4Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20260327; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=Pb17nx53D2NvJqqP5xnl4+2gn/g0GWgUhfme0UNqH8Q=; fh=2ALxNboLzDt503PhX56nFQ4pQzOkkVjVTCOSpqCio6o=; b=Ue7bCvxkmSHlMbyUvyfHfn3koClfeuP/Dlp20x/iGeKdSE6ZSQMJo/4/0oIHJhhlxz QihMOcMk2RYO9peHjUJA5MSMVCMoWRaTS6NVyutLQ3DsZPkVXzBboWQkNx517U4TP0/o FyzNm5jMdmdcowTuKo1ULK1hRAdhC52bElRgtdjGAp2cjSZSMWk3JBouo56XS2sTRwkC PQCk2E+Nmp5tUMeeImOigDy4d4L5Uy95HNnOu+M4UgV0vEaqR+V7D3kBb49KbVlRDJF0 NTdyGLWfrUY74au8m8ibXRGPQZUQoswBIrgrFgdrVQ7JQQxA8sYmlHVvsVOBpDoav6nb 3lUg==; dara=google.com ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of akarpovich@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=akarpovich@ilbers.de Received: from shymkent.ilbers.de (shymkent.ilbers.de. [85.214.156.166]) by gmr-mx.google.com with ESMTPS id ffacd0b85a97d-48587b1cb6asi29514f8f.0.2026.09.04.00.56.14 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 04 Sep 2026 00:56:14 -0700 (PDT) Received-SPF: pass (google.com: domain of akarpovich@ilbers.de designates 85.214.156.166 as permitted sender) client-ip=85.214.156.166; Received: from akarpovich-Victus-by-HP-Laptop-16-e0xxx.promwad.corp (62.122.174.6.serverel.net [62.122.174.6] (may be forged)) (authenticated bits=0) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPSA id 6847tRSH004124 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 4 Sep 2026 09:56:13 +0200 From: Aliaksei Karpovich To: isar-users@googlegroups.com Cc: Uladzimir Bely Subject: [v10 07/12] testsuite: Set ISAR_PREFETCH_BASE_APT by default Date: Fri, 4 Sep 2026 09:52:49 +0200 Message-ID: <20260904075515.25962-8-akarpovich@ilbers.de> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260904075515.25962-1-akarpovich@ilbers.de> References: <20260904075515.25962-1-akarpovich@ilbers.de> MIME-Version: 1.0 X-Spam-Status: No, score=-0.8 required=5.0 tests=DKIMWL_WL_MED,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_EF,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_PASS,T_SPF_HELO_TEMPERROR autolearn=unavailable autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on shymkent.ilbers.de X-Original-Sender: akarpovich@ilbers.de X-Original-Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of akarpovich@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=akarpovich@ilbers.de Precedence: list Mailing-list: list isar-users@googlegroups.com; contact isar-users+owners@googlegroups.com List-ID: X-Spam-Checked-In-Group: isar-users@googlegroups.com X-Google-Group-Id: 914930254986 List-Post: , List-Help: , List-Archive: , List-Unsubscribe: , X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= From: Uladzimir Bely This makes Isar use `base-apt` repo in different way. Any package installation is done from `base-apt` repo which is prepopulated from external mirrors. This behaviour is disabled by default for downstreams. To enable it, set the variable to "1", like isar does in local.conf.sample. In order to be able to run CI in old mode, allow CI read the option from the environment. Also, adjust some tests (like repro one) to make them work with ISAR_PREFETCH_BASE_APT set. Signed-off-by: Uladzimir Bely --- meta-test/conf/local.conf.sample | 3 +++ testsuite/cibase.py | 4 ++++ testsuite/cibuilder.py | 7 +++++++ 3 files changed, 14 insertions(+) diff --git a/meta-test/conf/local.conf.sample b/meta-test/conf/local.conf.sample index 2dbe28f1..74ee90b1 100644 --- a/meta-test/conf/local.conf.sample +++ b/meta-test/conf/local.conf.sample @@ -27,6 +27,9 @@ BB_DISKMON_DIRS = "\ MIRRORS ?= "git?://salsa\.debian\.org/debian/.* git://github.com/ilbers/BASENAME" MIRRORS += "https?://cdn\.kernel\.org/.* https://mirrors.edge.kernel.org/PATH" +# Use new base-apt behaviour +ISAR_PREFETCH_BASE_APT ?= "1" + # Users and groups USERS += "root" USER_root[password] ??= "$6$rounds=10000$RXeWrnFmkY$DtuS/OmsAS2cCEDo0BF5qQsizIrq6jPgXnwv3PHqREJeKd1sXdHX/ayQtuQWVDHe0KIO0/sVH8dvQm1KthF0d/" diff --git a/testsuite/cibase.py b/testsuite/cibase.py index 033b3b19..835593df 100755 --- a/testsuite/cibase.py +++ b/testsuite/cibase.py @@ -81,10 +81,14 @@ class CIBaseTest(CIBuilder): self.fail("GPG import failed") try: + self.move_in_build_dir('tmp', 'tmp_before_repro') self.bitbake(targets, **kwargs) repro_type = 'signed' if signed else 'unsigned' self.move_in_build_dir('tmp', f"tmp_middle_repro_{repro_type}") + + os.makedirs(f"{self.build_dir}/tmp/deploy/") + self.move_in_build_dir(f"tmp_middle_repro_{repro_type}/deploy/base-apt", 'tmp/deploy/base-apt') self.configure( targets=targets, gpg_pub_key=gpg_pub_key if signed else None, diff --git a/testsuite/cibuilder.py b/testsuite/cibuilder.py index 1d415084..ca75f2e1 100755 --- a/testsuite/cibuilder.py +++ b/testsuite/cibuilder.py @@ -174,6 +174,10 @@ class CIBuilder(Test): fail_on_cleanup = os.getenv('ISAR_FAIL_ON_CLEANUP') strlines = None if lines is None else '\\n'.join(lines) + + # get prefetch base apt mode from environment + prefetch_base_apt = os.getenv('ISAR_PREFETCH_BASE_APT') + self.log.info( f"===================================================\n" f"Configuring build_dir {self.build_dir}\n" @@ -197,6 +201,7 @@ class CIBuilder(Test): f" generate_sbom = {generate_sbom}\n" f" rootless = {rootless}\n" f" lines = {strlines}\n" + f" prefetch_base_apt = {prefetch_base_apt}\n" f"===================================================" ) @@ -305,6 +310,8 @@ class CIBuilder(Test): f.write('ISAR_ROOTLESS = "1"\n') if lines is not None: f.writelines((line + '\n' if not line.endswith('\n') else line) for line in lines) + if prefetch_base_apt == "0": + f.write('ISAR_PREFETCH_BASE_APT = "0"\n') # include ci_build.conf in local.conf with open(self.build_dir + '/conf/local.conf', 'r+') as f: From patchwork Fri Sep 4 07:52:50 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Aliaksei Karpovich X-Patchwork-Id: 5424 Return-Path: Received: from shymkent.ilbers.de ([unix socket]) by shymkent (Cyrus 2.5.10-Debian-2.5.10-3+deb9u2) with LMTPA; Fri, 04 Sep 2026 09:56:37 +0200 X-Sieve: CMU Sieve 2.4 Received: from mail-ed1-f58.google.com (mail-ed1-f58.google.com [209.85.208.58]) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPS id 6847uNPH004675 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Fri, 4 Sep 2026 09:56:23 +0200 Received: by mail-ed1-f58.google.com with SMTP id 4fb4d7f45d1cf-6a6634614cfsf628452a12.3 for ; Fri, 04 Sep 2026 00:56:23 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1788508578; cv=pass; d=google.com; s=arc-20260327; b=UZ1AI73Fc92chT8w98xs8/md3dVCCf2sH2RmCj5wf4JAykSOhNp5a+py67cOFpbMsU o/UXsDXbG5QcSRxVcwRBcR162WKlxutxNLOD0LXI819toKHKJb8JWXtKrMhYkk7Qo3eV LUDdnaH3vLxz66i1TFMfAaC1XEu946tPTqRlrD+tlwoJf08367Yza4ux8XEk1pY7b8Lg 7Rj2icwKVgQ2eTf/1QOrJmakgMjothK3vKzxbZn1jUCUVpliRpEe5p6CUkCKeMmhVyeg Cn/4GVrizEGYJtLT5lNgn+plmx+VrXuv8mSXQpQV1wky2I6+n226Wez7/7xsX2NFqXj1 1jMg== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20260327; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:dkim-signature; bh=Y14eFXZNRrl/5wQX5wWNEd1Gx93vcYk2/mJbv9QRL2U=; fh=lAgcP6fXDbnKglhRH9PbpbLvfxlsKWOOPC2mmOpcJ30=; b=KCOb6Uo8z7our+x+eBAd1SkfrcGwjURs0sdKbLLPEbi19Pduu5fHS+kl4SORg/mZzR t6uXlRWaqkdOtksgtGsj7mxL9mxdgdDZ2pGOivjtiZtv0GK7q3yENx/7JBZ67nhnxNfE hhtoUoRKAlNS/dy9nazzfFQ9EZDLhezKRZBRFpghpsOPGvef1/axy9o7ZpF23Z5YGK89 5Vndt3rJoKD3p7OE+goyNmfxFu5pjskr16K62DjGNgYf73BrtIO0kec46SEdYKcOgEVu yonWaU7/zt5M48UievkNYA+GyuoPKjSkHgHYb6r4ySyqStwI3t2NWjsQWFSocMMyCfsp Tssg==; darn=isar-build.org ARC-Authentication-Results: i=2; gmr-mx.google.com; spf=pass (google.com: domain of akarpovich@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=akarpovich@ilbers.de DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlegroups.com; s=20251104; t=1788508578; x=1789113378; darn=isar-build.org; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:content-type :x-original-authentication-results:x-original-sender:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from:sender :from:to:cc:subject:date:message-id:reply-to:content-type; bh=Y14eFXZNRrl/5wQX5wWNEd1Gx93vcYk2/mJbv9QRL2U=; b=mH3tEHUikzM2S9lmN55/PQ2XDl6947+dHjNpRFRD0XiiRZaohluWlpgmyYY9cf/QFv qVSv+ZAxawBIzqc9e3Pp9pAEoj+Wpv+1F63dH1cc55/+9txb2jjYVLBdByMH0xGMcpFR X7yG6AsQGOGxiwNdRT4RnRd3AgbgtkQHGua3vugbPAVMLDqAiGhl5XSNLvF/lRQK6Zj+ FQrKq+gFK9dTfUdTTv6FduFoWU53pI0t/JeyMDIFjfVvmqiy9+fpm9y3QryLK8lZBo5G XQYHi/PoUK8emVbWatcaSVYBPUFAM1NjR7O94HWnwiINR0bgyFBwv3a6+odNf/Mt/GeG IfEw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1788508578; x=1789113378; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :x-spam-checked-in-group:list-id:mailing-list:precedence :content-type:x-original-authentication-results:x-original-sender :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:x-beenthere:x-gm-message-state:sender:from:to:cc:subject:date :message-id:reply-to:content-type; bh=Y14eFXZNRrl/5wQX5wWNEd1Gx93vcYk2/mJbv9QRL2U=; b=mq+qMKITC6KgihtKUFypGpFZ9OG7uOHRCaobeS9TC733H4Y8WRc3o9IS51+PDYUoeT eQ6JZ2LMRVFxOo1m192I9PpDKFCGrOZFRQy/e+A4MIeupphRYaFbB2VcWyyiO1lvbbgt hneD+0C9goDOo/2FFJyXJ/AIbQMaY10T8l7qPMU+0AM54pQJs99UuxjLYnt8AaKhtG/n XiF/OAD1MMJJ8wFcWsNorhkM/eCwfaF3BmSWZ5Rf6QQpsyohi5yR1PM4c7GR7PdYXiA5 lCBFlfhfhJE7MlfN1jbvR/AcgkeLvrtE1QpQovR45IbUtVk5pBOi9KWuW8kaEJavfYAe K8qA== Sender: isar-users@googlegroups.com X-Forwarded-Encrypted: i=2; AKwUvBwlKWSovY332C5ruEnV5kQRpXNwZQRzpGdb08XAsLoRXJcqVw1JSWyilXZ8Dw8tCBCKiqeTF9U=@isar-build.org X-Gm-Message-State: AFuF++no0c7dO9eADtbxsaVkWcK6nNvAYPYVlT0BzP/MVkfFTBb0xxEm LtdEVDMbOHB2YfbOG4LRIuNfzckflbgnk3Bu6y+U7vtrcTAXHTL9Be3P X-Received: by 2002:a05:6402:3712:b0:6a6:7ed5:37ba with SMTP id 4fb4d7f45d1cf-6a7e8f99aa5mr1712160a12.22.1788508578313; Fri, 04 Sep 2026 00:56:18 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com; h="ATskLdf996yj5rMfaCdvUsXUGLR8a0b1ChmNNV+WZjlfBE0nqw==" Received: by 2002:a05:6402:4143:b0:6a7:ef6b:1bd0 with SMTP id 4fb4d7f45d1cf-6a7ef6b1daels198036a12.1.-pod-prod-08-eu; Fri, 04 Sep 2026 00:56:16 -0700 (PDT) X-Received: by 2002:a05:6402:f1b:b0:6a7:ee56:815d with SMTP id 4fb4d7f45d1cf-6a7ee56826amr358684a12.43.1788508576417; Fri, 04 Sep 2026 00:56:16 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1788508576; cv=none; d=google.com; s=arc-20260327; b=holBqNRlHJ/EDJi+tF4KW5fwUIEJ82ia5NMMCojb2H343o+vT+fZ7VyNO5YquTipfZ lBszSvI3upU0xkLu9byVRRcvfxVml5b7vMC4f9z7V26AyCGCf9kULfHMKK05ZNSvdyG6 phrrFIKqLzLucUXedBBd5yc6NmMNPSKk5MrkxrTDmjmo+hrmekVUHqPQnsxZdvD9jGEu Qd/DEtsFe+I2x+GK4yEzU8CC8UnVIYP60I+fsJ7C3epXbSU0UY1uLjAd7AlU9nKRyUUa YoqN9nrUJmyc91QBGsey1GMuSwYJFQD4xdY0REX68r6EzwNUNmLQqMb9dSuBQXYt+qGG vbPw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20260327; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=yXPPuCSu/IZUNh+pVd2HkNMbiQOUW1a6nZJ33oLrpOw=; fh=IIbR0JxonYgzdrHDdB9TmZniSh/o1z0HofhfbmLXp8s=; b=V41inNaZPzWgb5gWoRcZ0CewijrGz7vHfYoieEwuiFo5X4NDD5fWF5cX8DLkUguGfU B8aKKWK8cDZ+oFNZqkwJmW2UtyfdrqbL805qx11waGaNljwtsgT6sRBsrKPsmQb2v4WY ueOWeu9GX87qtXcxat9Ej4REM0OE4at9imCyU9IKpHYoYspwv9Mfm5XN7puhYsYaFr6m ytY/wMsNayw2qwlE0qXH4G5fMYbI36+99DiuF5wZWk1JN3RNr+1Mh6SMzlLEzXaU4ucp xt/Ulji1GsmZYUAmjPWyWIZgW1Hjb9p8/CGDmtgXCNjy13dCp5qEh4pttqFHJRRMc2cW f1dA==; dara=google.com ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of akarpovich@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=akarpovich@ilbers.de Received: from shymkent.ilbers.de (shymkent.ilbers.de. [85.214.156.166]) by gmr-mx.google.com with ESMTPS id 4fb4d7f45d1cf-6a7e6890207si41382a12.8.2026.09.04.00.56.16 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 04 Sep 2026 00:56:16 -0700 (PDT) Received-SPF: pass (google.com: domain of akarpovich@ilbers.de designates 85.214.156.166 as permitted sender) client-ip=85.214.156.166; Received: from akarpovich-Victus-by-HP-Laptop-16-e0xxx.promwad.corp (62.122.174.6.serverel.net [62.122.174.6] (may be forged)) (authenticated bits=0) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPSA id 6847tRSI004124 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 4 Sep 2026 09:56:14 +0200 From: Aliaksei Karpovich To: isar-users@googlegroups.com Cc: Uladzimir Bely , Aliaksei Karpovich Subject: [v10 08/12] Disable deb-dl-dir in base-apt prefetch mode Date: Fri, 4 Sep 2026 09:52:50 +0200 Message-ID: <20260904075515.25962-9-akarpovich@ilbers.de> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260904075515.25962-1-akarpovich@ilbers.de> References: <20260904075515.25962-1-akarpovich@ilbers.de> MIME-Version: 1.0 X-Spam-Status: No, score=-4.6 required=5.0 tests=DKIMWL_WL_MED,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_EF,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,RCVD_IN_DNSWL_BLOCKED,RCVD_IN_MSPIKE_H2, RCVD_IN_RP_CERTIFIED,RCVD_IN_RP_RNBL,RCVD_IN_RP_SAFE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on shymkent.ilbers.de X-Original-Sender: akarpovich@ilbers.de X-Original-Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of akarpovich@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=akarpovich@ilbers.de Precedence: list Mailing-list: list isar-users@googlegroups.com; contact isar-users+owners@googlegroups.com List-ID: X-Spam-Checked-In-Group: isar-users@googlegroups.com X-Google-Group-Id: 914930254986 List-Post: , List-Help: , List-Archive: , List-Unsubscribe: , X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= From: Uladzimir Bely Since all packages and source packages are placed to base-apt repo during build, there is no need to have them in one more place. Signed-off-by: Uladzimir Bely Signed-off-by: Aliaksei Karpovich --- meta/classes-recipe/deb-dl-dir.bbclass | 2 ++ meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc | 3 ++- 2 files changed, 4 insertions(+), 1 deletion(-) diff --git a/meta/classes-recipe/deb-dl-dir.bbclass b/meta/classes-recipe/deb-dl-dir.bbclass index 7e9f89b1..2fafd0a7 100644 --- a/meta/classes-recipe/deb-dl-dir.bbclass +++ b/meta/classes-recipe/deb-dl-dir.bbclass @@ -174,6 +174,7 @@ deb_dl_dir_import() { # nothing to copy if download directory does not exist just yet [ ! -d "${pc}" ] && return 0 + [ "${ISAR_PREFETCH_BASE_APT}" = "1" ] && return 0 # attempt to create hard-links for .deb files from downloads/ into # /var/cache/apt/archives/ so apt will only download packages we @@ -196,6 +197,7 @@ deb_dl_dir_export() { export rootfs="${1}" export owner=$(id -u):$(id -g) mkdir -p "${pc}" + [ "${ISAR_PREFETCH_BASE_APT}" = "1" ] && return 0 export isar_debs=$(${SCRIPTSDIR}/lockrun.py -r -f '${REPO_ISAR_DIR}/isar.lock' -c \ "find '${REPO_ISAR_DIR}/${DISTRO}' -name '*.deb' -print") diff --git a/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc b/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc index c452035f..94cc5fe3 100644 --- a/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc +++ b/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc @@ -274,7 +274,8 @@ do_bootstrap() { - \ "$bootstrap_list" > ${DEPLOYDIR}/${DEPLOY_ISAR_BOOTSTRAP}.tar.zst - if [ "${ISAR_USE_CACHED_BASE_REPO}" != "1" ]; then + if [ "${ISAR_USE_CACHED_BASE_REPO}" != "1" ] && \ + [ "${ISAR_PREFETCH_BASE_APT}" != "1" ]; then deb_dl_dir_export "${WORKDIR}/dl_dir" "${BOOTSTRAP_BASE_DISTRO}-${BASE_DISTRO_CODENAME}" run_privileged find ${WORKDIR}/dl_dir -maxdepth 1 -mindepth 1 -exec rm -rf --one-file-system "{}" \; rmdir ${WORKDIR}/dl_dir From patchwork Fri Sep 4 07:52:51 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Aliaksei Karpovich X-Patchwork-Id: 5425 Return-Path: Received: from shymkent.ilbers.de ([unix socket]) by shymkent (Cyrus 2.5.10-Debian-2.5.10-3+deb9u2) with LMTPA; Fri, 04 Sep 2026 09:56:38 +0200 X-Sieve: CMU Sieve 2.4 Received: from mail-wr1-f61.google.com (mail-wr1-f61.google.com [209.85.221.61]) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPS id 6847uPK5004683 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Fri, 4 Sep 2026 09:56:25 +0200 Received: by mail-wr1-f61.google.com with SMTP id ffacd0b85a97d-4837fc5c966sf58265f8f.3 for ; Fri, 04 Sep 2026 00:56:25 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1788508580; cv=pass; d=google.com; s=arc-20260327; b=PGFy4us0CZE6rK8Toa/afoRHhk3rk7TGfsw3yXqJhMVbLhQw0rebEocrh6qSW527Kw R8Y4CtouwSLNf6JDhDwSFOScBWsNvPSDFqDdPwNSvd7V43uXbQD51lIcdb5NTqMTBbnx pYZJ67SJNKjyEPNDkH2Ni19OgBSet1C8cG7Ni65OLXd4dd6wGCPxG7V2joS0BSdtMVa/ krHl7q/MM24IYtGvQrWjYQjAA/Wl8BPYxjWCdfiAyZAAZ41osm+2upXt1MeUNp5wjqCb 06BR/p4A3836FSazwrN7d6nFu7PtVYKHtgaIep34hWqeT964dRzSaI7oWg9ILa9ELhQa 9E0w== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20260327; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:dkim-signature; bh=RqZWZ9M+1nzIS2LjjZuiFXVDQt5eRn8TAWCGr1aYL44=; fh=8XQ6GRvmCBGoWzmaOZ25lrkzrZG/7jbwE7jdDbmEBrs=; b=r2pf6y47he+MbY/EgzeBhgYee5p+iERNmEe/gswTHcCo5zJppiILi7irxm3gYOgn65 tjZDMrIdRWywG7S+hs0FVYCea5TO/uMN7eoy3d+ewyOY/XKIY8RJ4/UVYAXwzOLcXtFw wmOSjl638749u9ITb90wDOSOpVCt3kB4mK91vsdngZ1r9pEIUdhlx9j8IXsBDtH5W/qP maS9SnlwGEZqG9Ilmu1TLjnzJSTvcFTMpiJCrxrpPlnmIMXoxbux+xVnSVdqkwtVDXG/ KW+1rdX+Y8AudNV5Bm9PKeRSb8ITC9MZRmswQ3MITNfNeiKkR1/XcuV5iZZ4jRTdMdxJ T0+w==; darn=isar-build.org ARC-Authentication-Results: i=2; gmr-mx.google.com; spf=pass (google.com: domain of akarpovich@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=akarpovich@ilbers.de DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlegroups.com; s=20251104; t=1788508580; x=1789113380; darn=isar-build.org; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:content-type :x-original-authentication-results:x-original-sender:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from:sender :from:to:cc:subject:date:message-id:reply-to:content-type; bh=RqZWZ9M+1nzIS2LjjZuiFXVDQt5eRn8TAWCGr1aYL44=; b=XsJLQRM6tK6h28GYvzLJLZJE0DnKJ+BklPxOG6ew/DBNTQm4pwT/LLc8KOekjUio54 Uf4lewHtfqpwNL5ax9w/RNbFKL7oihb4IctAf0qRXUVvZKb68S/WhST/su0RZwEFkdss CXt5z4WrIef3LDJy3L+JIzY14G3oKnBZQGFydJHI/HxXX4neq5TdihqqCFV+KiNCvg1x E0v/5fiM/F3/EVBgmlu5UOzvF7WLUtzLGrkheUleouCe84hY3BbjsQhVyTn6qUyDU8NF 13cddTUIWW/hKb41E5HgsU53VAi2z45TOmGJwFR+OaD4jDyn10b+bNaH7Y1jlDKdx8+x lkCQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1788508580; x=1789113380; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :x-spam-checked-in-group:list-id:mailing-list:precedence :content-type:x-original-authentication-results:x-original-sender :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:x-beenthere:x-gm-message-state:sender:from:to:cc:subject:date :message-id:reply-to:content-type; bh=RqZWZ9M+1nzIS2LjjZuiFXVDQt5eRn8TAWCGr1aYL44=; b=YIyLGyPKEugRUPGgOV6PHsaM0JzefraUxnng7V2NMuj29Q/BoR8m9NN3PDniTIB6xx W14V2HoNBfboEqDOk9f8K8D2Mubb6nu+BEI+ELHHV1mVvfSrFacuvEznSO1eHorNPNGP XgGrz+p4AApHj4XFXYF5u0hVwmn+Ptd7F3BlvtgnFnpKSB5wTqNEIsidctd0Kd67enXY dMqIOvtyFkdSduP9gNAMj+OH2XW6xhtm7uP2TJQ/82WpDEYE5sttN1LxiCNSh/AprEqs wnYn3uIX7NazZH70ypnWLzHGi4HGwwU04Rrj4xfdsmU6+JFhqPefgZnqN+wMo2vUM02s 1xlw== Sender: isar-users@googlegroups.com X-Forwarded-Encrypted: i=2; AKwUvBxw929/6Z8N+gUleuQ0KPV2Gj67FAhJqGHCjh1hiC9NQhKbwaNKiTb66V9vxw2x9H4fU0DRMC0=@isar-build.org X-Gm-Message-State: AFuF++mFt/U7xXvm3vgFR5gSztHHmGOm00ytdLxXqSjtvRYcxev0b8LB k0/yr25gOILHrE6SCELn3nRyek/DkxG533RmwO0xye3u5BuBXkhbqN4a X-Received: by 2002:a05:600c:34ce:b0:49b:910c:76fb with SMTP id 5b1f17b1804b1-49cf8268084mr28921255e9.2.1788508580233; Fri, 04 Sep 2026 00:56:20 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com; h="ATskLdd/+X+tF/YNukKTR2EBFjLh988sJD8rawqdXH5SzG+p7w==" Received: by 2002:a05:600c:4444:b0:495:482d:9805 with SMTP id 5b1f17b1804b1-49cf7b79eaels7156025e9.2.-pod-prod-07-eu; Fri, 04 Sep 2026 00:56:18 -0700 (PDT) X-Received: by 2002:a05:600c:46cf:b0:49c:fc6e:8cad with SMTP id 5b1f17b1804b1-49cfdb3a26amr6825845e9.17.1788508577919; Fri, 04 Sep 2026 00:56:17 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1788508577; cv=none; d=google.com; s=arc-20260327; b=b9pfVm8eANbkpAwU5nNL42Jy67eAa3XDxZbekRiLGzvrhNYXWfJYG2PO6hop+7ODuh YzsYVjtHqzbTC8/guOsLrgMNKuhVFE9rIIBVBc1d71qiUjUFI6RFOno4W2SNHzDiMTkd cU0F7/SoT93kUOB5cDGykdnb8/kbT33EVxBlgP+QnwNcoHyTQDYoCdBlwG7du96dQmHm utHJ9KTibB2374qQCZc9AST1UiP1tebBPHIbVwAc1mjcXqggpokFKC2RGsb9tT/9/EFr J6ispVCXfC2/gwF7BiVdDCU878UOz+rhYQW9O74skyWgvtC7L7l3FNPF7GOtqb7jnFfa GlpQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20260327; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=Nsv6KIh0VPKM1jN0krHJsGIu+7I3PGcYMTQpui9gMP0=; fh=2ALxNboLzDt503PhX56nFQ4pQzOkkVjVTCOSpqCio6o=; b=LmHTLvFY0ifo29Dhx6iucvuqymOGB2Q+4Iky9Uqk+tWS0dG/nQ2ixcInWVvjd3xy18 5ile+Df84e55iNMzwzezqAJsDrOB/uz2hIG5kNAwF+FQBbV8QB9o7mSTb3iaDHttA23f 8REohjXkN57CbnbB8tjCVBz6+LcW7XeMiwHg4SNEuwvQ4Pj4NmV4jPmDibMd1zNXjGvy tQpLwtXQWwrqIoMBeHjXe6UnQ/UrdlWd19n57J/SJHOqvZBU6cPMROEdAfTh5hI9xDUQ HN6JYMvV/ukdNtdfX1sYLn6oxlKHT17L8ZRozNbc9Qcyr5ZYu6qVluVHus6RZk9JiPhW EGKg==; dara=google.com ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of akarpovich@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=akarpovich@ilbers.de Received: from shymkent.ilbers.de (shymkent.ilbers.de. [85.214.156.166]) by gmr-mx.google.com with ESMTPS id 5b1f17b1804b1-49ce5520989si884975e9.2.2026.09.04.00.56.17 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 04 Sep 2026 00:56:17 -0700 (PDT) Received-SPF: pass (google.com: domain of akarpovich@ilbers.de designates 85.214.156.166 as permitted sender) client-ip=85.214.156.166; Received: from akarpovich-Victus-by-HP-Laptop-16-e0xxx.promwad.corp (62.122.174.6.serverel.net [62.122.174.6] (may be forged)) (authenticated bits=0) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPSA id 6847tRSJ004124 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 4 Sep 2026 09:56:16 +0200 From: Aliaksei Karpovich To: isar-users@googlegroups.com Cc: Uladzimir Bely Subject: [v10 09/12] kas: Add PREFETCH_BASE_APT config entry Date: Fri, 4 Sep 2026 09:52:51 +0200 Message-ID: <20260904075515.25962-10-akarpovich@ilbers.de> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260904075515.25962-1-akarpovich@ilbers.de> References: <20260904075515.25962-1-akarpovich@ilbers.de> MIME-Version: 1.0 X-Spam-Status: No, score=-4.6 required=5.0 tests=DKIMWL_WL_MED,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_EF,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,RCVD_IN_DNSWL_BLOCKED,RCVD_IN_MSPIKE_H2, RCVD_IN_RP_CERTIFIED,RCVD_IN_RP_RNBL,RCVD_IN_RP_SAFE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on shymkent.ilbers.de X-Original-Sender: akarpovich@ilbers.de X-Original-Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of akarpovich@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=akarpovich@ilbers.de Precedence: list Mailing-list: list isar-users@googlegroups.com; contact isar-users+owners@googlegroups.com List-ID: X-Spam-Checked-In-Group: isar-users@googlegroups.com X-Google-Group-Id: 914930254986 List-Post: , List-Help: , List-Archive: , List-Unsubscribe: , X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= From: Uladzimir Bely This option allows to set ISAR_PREFETCH_BASE_APT to "0" or "1" and choose between old and new base-apt behaviour. Docker image kas uses should have "python3-apt" preinstalled in order to have new functionality working. Signed-off-by: Uladzimir Bely --- kas/opt/Kconfig | 12 ++++++++++++ kas/opt/prefetch-base-apt.yaml | 9 +++++++++ 2 files changed, 21 insertions(+) create mode 100644 kas/opt/prefetch-base-apt.yaml diff --git a/kas/opt/Kconfig b/kas/opt/Kconfig index bc4ed997..316b755a 100644 --- a/kas/opt/Kconfig +++ b/kas/opt/Kconfig @@ -213,3 +213,15 @@ config KAS_INCLUDE_USE_DRACUT string default "kas/opt/dracut.yaml" depends on USE_DRACUT + +config PREFETCH_BASE_APT + bool "Prefetch base-apt repo" + default y + help + This makse Isar always take packages from base-apt repository where they + are prefetched by deb-repo script before requested. + +config KAS_INCLUDE_PREFETCH_BASE_APT + string + default "kas/opt/prefetch-base-apt.yaml" + depends on PREFETCH_BASE_APT diff --git a/kas/opt/prefetch-base-apt.yaml b/kas/opt/prefetch-base-apt.yaml new file mode 100644 index 00000000..0cbdb93f --- /dev/null +++ b/kas/opt/prefetch-base-apt.yaml @@ -0,0 +1,9 @@ +# This software is a part of Isar. +# Copyright (C) 2024 ilbers GmbH + +header: + version: 14 + +local_conf_header: + prefetch-base-apt: | + ISAR_PREFETCH_BASE_APT = "1" From patchwork Fri Sep 4 07:52:52 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Aliaksei Karpovich X-Patchwork-Id: 5426 Return-Path: Received: from shymkent.ilbers.de ([unix socket]) by shymkent (Cyrus 2.5.10-Debian-2.5.10-3+deb9u2) with LMTPA; Fri, 04 Sep 2026 09:56:39 +0200 X-Sieve: CMU Sieve 2.4 Received: from mail-wr1-f55.google.com (mail-wr1-f55.google.com [209.85.221.55]) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPS id 6847uRfc004753 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Fri, 4 Sep 2026 09:56:27 +0200 Received: by mail-wr1-f55.google.com with SMTP id ffacd0b85a97d-48437b84372sf74368f8f.0 for ; Fri, 04 Sep 2026 00:56:27 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1788508582; cv=pass; d=google.com; s=arc-20260327; b=r7OQwJL9ltJ5rn4J/YpBkKNrFY2ra664fOa79mKaQqgnLtNB+uk/hcdugt9EHxbhyy NFZoO2xakqW+RuXyr5617ZJidQuCAHrc1XPJcrdS3TFb8zhfs4y3kDxmqwEGbGwyaaVA tJ69U+tEUbssxV9v6eth1cE5ZolUEYrrENsEChbIIBTL+kbZDXFl4CbbWmtP7R1m2TRX mYM+RYxKH0h2+7Gd2ZaEPF+i7LdAFjV9NiYeKp71RF1jbnMgVfy9AjYIuh83PyyOZByn G+DxDVerDU8sE+cC0cPAvx0pBNHwe5sHYQ4RtclUn/csUGiryXW4xQQrO8ZhxlPcJPpK WkKQ== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20260327; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:dkim-signature; bh=yaOmBwZy74KFrGUgFHqZGQGa34E685G7RWvV21RyCZk=; fh=f8ap+Yx0/ZI2/f0NM2P8Yp9Vv6tjqOZz8UL5p7r/XMY=; b=YamL+cUa8CwqgSuxFCDGU+UmbihOBiObrouEVVvcGLX+KZ9UaNUjH8f9IxN4xLrU4a X6rflQNvqvnDTLOXwe29DZ8DTZzZp3ky0lq3cWi+B9cgNnsLpWCpQsgYGEXBc9CPldG/ 2CYfLHD8jt44dnEQ35BJeDYFAmEcx31WX+o1w4HbQQ3ADw8HzcR+pg1FkERufr+5y3y/ klsqxX2ci9vWGf9jitfMW03Gj8dot5bTx0nIl9B4kUfdfqOZC2G/Bx+naFXIR4gJbhFZ 5TrJVtHabg3LP1wtDhIgoHiiH28MKd4uU8KYxRpxX+AAGueZFOUS2WvS3A0cZ8OUgb4g oLHA==; darn=isar-build.org ARC-Authentication-Results: i=2; gmr-mx.google.com; spf=pass (google.com: domain of akarpovich@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=akarpovich@ilbers.de DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlegroups.com; s=20251104; t=1788508582; x=1789113382; darn=isar-build.org; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:content-type :x-original-authentication-results:x-original-sender:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from:sender :from:to:cc:subject:date:message-id:reply-to:content-type; bh=yaOmBwZy74KFrGUgFHqZGQGa34E685G7RWvV21RyCZk=; b=RKwkeSL1tpS36sSgki12RYStVNVN8lU3bt5j/MV6JcOKikiqsRYa/t5mppPAvw7lgO 1viXGHv2wshhnXN0vUNalsB/49MO1Tln3BEll/KE1Ol8YO1xXWZdW8Ft4RWzwG9qsa76 yGLSWf2sdk14aRUq+mMJUftapTs6qPvxfYyCs/IANvQ/Rp3oWl0XUmtKvKhEdZnKxAh6 6j5PURe9w9permerTBPkBwdN+7BDILTkT7L8fbamGlMnDw8Mp9v5iE5vF35QHrO0xCFS R6Q4HDBny0GhT6MFu9ks3ty0fP92ejoNvaA+qI4LB19g3lv40170wbjEFp5oc+ZmHAaD 7oRQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1788508582; x=1789113382; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :x-spam-checked-in-group:list-id:mailing-list:precedence :content-type:x-original-authentication-results:x-original-sender :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:x-beenthere:x-gm-message-state:sender:from:to:cc:subject:date :message-id:reply-to:content-type; bh=yaOmBwZy74KFrGUgFHqZGQGa34E685G7RWvV21RyCZk=; b=iSHNN3+p3gQn+fVNxxNCiGvUbO0ec1bNwDx4SVGTUgXpHIZJGBCQmYH2GRUK1yiqDZ u+3KtQYYsqUk0CWVZmpo1/xY/NPFtn9lb9Ivebk95B38DOzRetYV8gbHLshdtOy7qnVa MsswBg+8LctZi0bXnUbUOfb48xFdIfmsMy650SF2fJ9lU26DiKxMCpggkHgM3abTftcC alA44W4QHNdMrpO1Xx/RwgnbAU/47Ex3y+VEtKz20eD4hRBmeJUP4BwggPpNnV+8W15o UmU1OK5vvN3Yr+VuqLwjmTgdMXkJrw+PWdoQXvbU//EGkwwX/IJW0yMbK/pXXzG23cHL uSWg== Sender: isar-users@googlegroups.com X-Forwarded-Encrypted: i=2; AKwUvByHCoGDB3ZC6XM9nUct0efgF08lXhMPTPEuiDr1Ya4fsdMcYUpqI64H9rH9yMlCSONt3V0HeKY=@isar-build.org X-Gm-Message-State: AFuF++l5V/EjICdMGD5i0Ndfx6YHTt6NORq7JmtkC0cDSnAFRZq+HTzc zUPtu+mmi575q3qigUDCCGcj3bQu1ea7kbfpez/QzIyXP+zgP7YG7Tq8 X-Received: by 2002:a05:6000:41c7:b0:46d:ff8a:c8e6 with SMTP id ffacd0b85a97d-4858707c531mr3814238f8f.1.1788508581835; Fri, 04 Sep 2026 00:56:21 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com; h="ATskLdc1uwrbhtTsJtBrHuojcrM8Trms6jNxSFzA56BkJKMixA==" Received: by 2002:a05:6000:986:b0:484:3be3:54af with SMTP id ffacd0b85a97d-48586327065ls716992f8f.0.-pod-prod-06-eu; Fri, 04 Sep 2026 00:56:20 -0700 (PDT) X-Received: by 2002:adf:fa81:0:b0:484:3317:a19 with SMTP id ffacd0b85a97d-485872b896dmr5672424f8f.26.1788508579845; Fri, 04 Sep 2026 00:56:19 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1788508579; cv=none; d=google.com; s=arc-20260327; b=ivwogmtDJJrwpRu43d8j+6xR2dhKPog6ZLinnrW47AggQXkFWJMYaonQOOOMjV7ziD 8bU5xLB5ZTIH6QP4fe6DXagObdXFMX5wyC+gwR/BqnyXltRaKP5L/t5BhvdBfRrRhKr0 q2oxb5zTu8Kru+X4yxl8wNp8HsnFbhIv/xugLesVk6AzX4HjdUTmVSLYpYjnhcu/1zsD 2XkjCeXXoWm6TZegXLNmC2UEb5IGsNz6Gsq/EOvf7R7t7eny6W9XB+EkamiqL4aKMHKG ZQ0380gI7TgQSbhawy2Xl/RROAr/b0I1dMAOZwXXpPhqWwpT10AgOvZnzvsPh5ypq/VG G+Eg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20260327; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=6S1gGmQ1iqMBK+cvl2rU5/PoV+MJpfsbokv74/WFWKE=; fh=2ALxNboLzDt503PhX56nFQ4pQzOkkVjVTCOSpqCio6o=; b=fjdQd+K4dyKKO4nc/T4gdBtKl3fc69BSekuwfyDp1evzatCARgS6I+SiI/tYnyEMof Lhmz+sIEu/ln+dECMsdq6RNlz5wvFijt4Y0T8wL3ZeBeYCqGZglMV6sXHc/xQ2HVHJxU GzYZhVgfjaUEcIwICmAArw7F8u3BCUM7iVT7EncSVgwE83pgdkwPLjWZR4A4a3bn3TAU crmExSFtJssAqkfptQODrlHJOVMJQu+DBr+CNFDmKF2PqgUIquPq/bTicjIJ1bo7PGoy aLw9dNYiQ+fMIvSjDz2mAIemRIY96Ni3eo/pg2VsrE6b55WhmU3wcx/ZY/XvMciYkbHc 3H3g==; dara=google.com ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of akarpovich@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=akarpovich@ilbers.de Received: from shymkent.ilbers.de (shymkent.ilbers.de. [85.214.156.166]) by gmr-mx.google.com with ESMTPS id ffacd0b85a97d-485885dfa4bsi37530f8f.8.2026.09.04.00.56.19 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 04 Sep 2026 00:56:19 -0700 (PDT) Received-SPF: pass (google.com: domain of akarpovich@ilbers.de designates 85.214.156.166 as permitted sender) client-ip=85.214.156.166; Received: from akarpovich-Victus-by-HP-Laptop-16-e0xxx.promwad.corp (62.122.174.6.serverel.net [62.122.174.6] (may be forged)) (authenticated bits=0) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPSA id 6847tRSK004124 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 4 Sep 2026 09:56:17 +0200 From: Aliaksei Karpovich To: isar-users@googlegroups.com Cc: Uladzimir Bely Subject: [v10 10/12] ci_build.sh: Install python3-apt if not installed Date: Fri, 4 Sep 2026 09:52:52 +0200 Message-ID: <20260904075515.25962-11-akarpovich@ilbers.de> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260904075515.25962-1-akarpovich@ilbers.de> References: <20260904075515.25962-1-akarpovich@ilbers.de> MIME-Version: 1.0 X-Spam-Status: No, score=-4.6 required=5.0 tests=DKIMWL_WL_MED,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_EF,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,RCVD_IN_DNSWL_BLOCKED,RCVD_IN_MSPIKE_H2, RCVD_IN_RP_CERTIFIED,RCVD_IN_RP_RNBL,RCVD_IN_RP_SAFE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on shymkent.ilbers.de X-Original-Sender: akarpovich@ilbers.de X-Original-Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of akarpovich@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=akarpovich@ilbers.de Precedence: list Mailing-list: list isar-users@googlegroups.com; contact isar-users+owners@googlegroups.com List-ID: X-Spam-Checked-In-Group: isar-users@googlegroups.com X-Google-Group-Id: 914930254986 List-Post: , List-Help: , List-Archive: , List-Unsubscribe: , X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= From: Uladzimir Bely This is mostly related to gitlab CI that migth use an image without preinstalled python3-apt. Also, make system python packages available in virtualenv. Signed-off-by: Uladzimir Bely --- scripts/ci_build.sh | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/scripts/ci_build.sh b/scripts/ci_build.sh index 241ff8c8..71f62111 100755 --- a/scripts/ci_build.sh +++ b/scripts/ci_build.sh @@ -19,7 +19,7 @@ if ! command -v avocado > /dev/null; then sudo apt-get update -qq sudo apt-get install -y virtualenv rm -rf /tmp/avocado_venv - virtualenv --python python3 /tmp/avocado_venv + virtualenv --python python3 /tmp/avocado_venv --system-site-packages # shellcheck source=/dev/null source /tmp/avocado_venv/bin/activate pip install setuptools==81.0.0 @@ -138,6 +138,12 @@ if echo "$TAGS" | grep -Fqive "-startvm"; then fi fi +# install python3-apt +if [ ! -f /usr/share/doc/python3-apt/copyright ]; then + sudo apt-get update -qq + sudo apt-get install -y python3-apt +fi + # Provide working path mkdir -p .config/avocado cat < .config/avocado/avocado.conf From patchwork Fri Sep 4 07:52:53 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Aliaksei Karpovich X-Patchwork-Id: 5428 Return-Path: Received: from shymkent.ilbers.de ([unix socket]) by shymkent (Cyrus 2.5.10-Debian-2.5.10-3+deb9u2) with LMTPA; Fri, 04 Sep 2026 09:56:40 +0200 X-Sieve: CMU Sieve 2.4 Received: from mail-lf1-f60.google.com (mail-lf1-f60.google.com [209.85.167.60]) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPS id 6847uT3h004832 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Fri, 4 Sep 2026 09:56:30 +0200 Received: by mail-lf1-f60.google.com with SMTP id 2adb3069b0e04-5b5e279489dsf426742e87.3 for ; Fri, 04 Sep 2026 00:56:30 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1788508584; cv=pass; d=google.com; s=arc-20260327; b=IJOcdMn/LrhDiguE3yieI5bq2pFeVA4jUHEctME7jmIMRZI5qb3fuA8bceCl9mscZa hjtBLVCYhG+pZsMXO5uDh6c4IwbFvnSlSTrw5T1c49/yVVLKUNSSD01l2qWSXd3bNUKO GqnRn0vLNXtktAmh30mtXPqnm3AMPTmgIqtFph7ja1UIIs6jFh0P7wOtfaKXiXoAv8Nm 4DbvXWoN5ntOetlC7Wf21SgwdWOe3lB/3jmwMgq2eLp/g9bGHJFkdXjWv4lb+v/P0CuR qIRqTJpkJ0rg+z8RvQ1rcP2HhFs54qFwUim+NtyDodpfVsNaOmIiO6U4PndKSd1+7DoL mEOw== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20260327; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:dkim-signature; bh=LNGWt5K3a9kMYyMESppD79RSDI59RISdLWIOLBRzhYU=; fh=9E7H6zJZ8QTktG7YTGAuZIUQt8nLsuOn3o4phpzUaJQ=; b=TxoP3nrsT36GbxJy5kd0xvIlkaZZUtVhzv6MrL/CFigm6AueKCfPhZnr2Skdn2ZX0w J1fmrbM6ED2mzx7vybRGCubTy1cNdO3GSSSNcf1GrzN2eszfILz+P1JojrtW98VsZfu4 wuXS0nkNFdvbyyYrVgMtjxOhLSOGR8QlGtZb6fccefb/IMp+Yse/sUZtl9K23utUS0pQ LCFT1EQNagW6qNZppDH8+ZYssHlZOBowV+NtGBQpSX6fn57Zdqo8t8KD/xEx62PJAKG6 fv3disuzN7j0Lhkflzp6gtAIsUtJQkiJmBv7s8oBsJpbTrEVidaIIIC8q4XE5n0epUED yg0Q==; darn=isar-build.org ARC-Authentication-Results: i=2; gmr-mx.google.com; spf=pass (google.com: domain of akarpovich@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=akarpovich@ilbers.de DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlegroups.com; s=20251104; t=1788508584; x=1789113384; darn=isar-build.org; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:content-type :x-original-authentication-results:x-original-sender:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from:sender :from:to:cc:subject:date:message-id:reply-to:content-type; bh=LNGWt5K3a9kMYyMESppD79RSDI59RISdLWIOLBRzhYU=; b=iUJJZfjkBZkYkuNUEoBI9pypgNxYJH1Z9RhoVA6/xUM0vw7HYHswCGWApiYZmNUSLI 4NxCAa8uuSfdxZxuRzF8J/F0yzghuj9+qiSC6FP4QT7IdCjd0EoE4LgFmTpr5Z90dtoy W+nDxee0T7FMYME9RaLC1ZjWvS+K4OxEd3V7LJpCmbq91GX4znKqXrgNORfiWFrmPJZY 2zapFW7sDioS6RtBEhvbydkhsl5W016p1OBz1c2TTJ686r0PJGWDiPUYZ4en5mFA0dUb hdyZKxbSNXz7Kn7JwJmitddOViOa1HcN4Jx/Ch4T3d/YxyHW44vOVH9vO4T2UHW8DVtj exAA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1788508584; x=1789113384; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :x-spam-checked-in-group:list-id:mailing-list:precedence :content-type:x-original-authentication-results:x-original-sender :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:x-beenthere:x-gm-message-state:sender:from:to:cc:subject:date :message-id:reply-to:content-type; bh=LNGWt5K3a9kMYyMESppD79RSDI59RISdLWIOLBRzhYU=; b=Guk/Tuxm3pOfGXoJCN8rMzSXK9bHzMquha/5W1eXqfDE2/IhtDv7VcX0Nu2EKDIeOd nlcGvQm+aABTWResC6uQHCcrPIbBNpOs68hkdYfQth8n4CcVAnrOa/Z8SAAnpaa1IfB3 izuHM2HRlcqJTa9bdpTm1glJPfDyTgzLZ22IOnWF3AiGWPllluVv5zvdeUqpzCd6Qiih C+0/rmU3T5MKooaEZodXyM3ZZ9YRyjaDhJj6ALnI9hJ7jSKgamTwRgpGY+MSchoVGEkc 7linR071xvlamjJo8oZJma7y2zzu3QzNkmbIVvYgTvzdLxmhRKcaqEKpXW8VwGLAUXHn meOw== Sender: isar-users@googlegroups.com X-Forwarded-Encrypted: i=2; AKwUvBxin1sMQxdWnqOKFpfromW7N88jpyM9y1tnNB+Ey/fX6ujEmawji77Z/RRudq73/j3km7Si5M8=@isar-build.org X-Gm-Message-State: AFuF++kwldqlcwaTZe1S2btYKtL9bT7tsxV1BAlyU1gBxAA0m10bVdOJ J7lPt0+8V8kMPlUl1Nk1L8lxZwirHpcEiWZH6YdcL4SgfGIUK1pW6qln X-Received: by 2002:a05:6512:a8b:b0:5b3:e52:a005 with SMTP id 2adb3069b0e04-5b616f020b7mr816457e87.4.1788508584486; Fri, 04 Sep 2026 00:56:24 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com; h="ATskLdf/s25fogI3sLFQxcH4DCMkMGl32TJtVNMXVjlblaIV4g==" Received: by 2002:a05:651c:1690:b0:3a1:e61d:46c6 with SMTP id 38308e7fff4ca-3a372513376ls1825241fa.1.-pod-prod-06-eu; Fri, 04 Sep 2026 00:56:22 -0700 (PDT) X-Received: by 2002:a05:6512:3e17:b0:5b0:1516:8799 with SMTP id 2adb3069b0e04-5b6170524a4mr767024e87.17.1788508581984; Fri, 04 Sep 2026 00:56:21 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1788508581; cv=none; d=google.com; s=arc-20260327; b=EnY0JmOyg63wJ6MXiXoj01t3p5VXydpDT5KmVmDNDp+w6VvukatDuawzJptOH3xx60 U0LvKiu0bLyspk2IAwmFTlFh6xfgP6V/wmXxFhCJaVzEy9w81dl6FHkoC/DJTK1mge/r y7cmAGqPSogF7ppVhX2layDpEQDRgxfzhWftv2unE/yrs3jJI5ORHbKq0CMBlFhQrnZU 9Q6VTzgUre26D2O4WiN+ipGtnXrQz+ETMUcD36E8d5c/GUuWqvsy2iUMU6ieXeDgpfhT EsRvDHtMDZLQgXqDXHh7VeMfXPpQg0NKSER6dQAqasA3Dg7Dfujxblb+FXv4/+T0+6a+ 1/3w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20260327; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=fXV59MW+wfucag7p9lyuoIU9KCdjvsnT1Rffr5ejkLk=; fh=u+3kXEMzY4uVhDBiujW2QUo7NnpMLrGEgMQBYpCTRN4=; b=HOFgRm9/Dp/wbFeeK1tFE9Jmg83dvionDwIcSuMS5NnjhbfmeVSoLP4dwnbGQwW1kx 5VdjXgctM09UBCzWBK4NvDxh+CB3qCgF1sslASWI90PyvBSAnjvJHwaP+5r01zFj5OJv W1g2Cc5TcmrQfl40UqjULVBlQz6q2HLm8BTgaeHI3aBjMOd7+rpE+uEbuVNY2vVpHoIh IKZ66A17L4I3CVGAsSU104vH47WgCoB3JG4YBSFmaUuM25ACfZYMWn5LqIC0724UmMHq rjvzz2Q51URnlBoqvamjS/E/n8QcaHXRQ1r/5gncYct4pqYC/LFmidzvcf0fsIGjCcOL 0aVg==; dara=google.com ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of akarpovich@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=akarpovich@ilbers.de Received: from shymkent.ilbers.de (shymkent.ilbers.de. [85.214.156.166]) by gmr-mx.google.com with ESMTPS id 2adb3069b0e04-5b6166cb2cfsi35889e87.8.2026.09.04.00.56.21 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 04 Sep 2026 00:56:21 -0700 (PDT) Received-SPF: pass (google.com: domain of akarpovich@ilbers.de designates 85.214.156.166 as permitted sender) client-ip=85.214.156.166; Received: from akarpovich-Victus-by-HP-Laptop-16-e0xxx.promwad.corp (62.122.174.6.serverel.net [62.122.174.6] (may be forged)) (authenticated bits=0) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPSA id 6847tRSL004124 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 4 Sep 2026 09:56:20 +0200 From: Aliaksei Karpovich To: isar-users@googlegroups.com Cc: Aliaksei Karpovich Subject: [v10 11/12] debrepo: Add override for reprepro Date: Fri, 4 Sep 2026 09:52:53 +0200 Message-ID: <20260904075515.25962-12-akarpovich@ilbers.de> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260904075515.25962-1-akarpovich@ilbers.de> References: <20260904075515.25962-1-akarpovich@ilbers.de> MIME-Version: 1.0 X-Spam-Status: No, score=-4.6 required=5.0 tests=DKIMWL_WL_MED,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_EF,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,RCVD_IN_DNSWL_BLOCKED,RCVD_IN_MSPIKE_H2, RCVD_IN_RP_CERTIFIED,RCVD_IN_RP_RNBL,RCVD_IN_RP_SAFE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on shymkent.ilbers.de X-Original-Sender: akarpovich@ilbers.de X-Original-Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of akarpovich@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=akarpovich@ilbers.de Precedence: list Mailing-list: list isar-users@googlegroups.com; contact isar-users+owners@googlegroups.com List-ID: X-Spam-Checked-In-Group: isar-users@googlegroups.com X-Google-Group-Id: 914930254986 List-Post: , List-Help: , List-Archive: , List-Unsubscribe: , X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= It fixes an issue when bootstrap image in case of offline build is different than online. It happens because some packages has different Priority field because of override. F.e. libtext-wrapi18n-perl in trixie: - inside deb package: 'Priority: required' - inside Packages from deb repo: 'Priority: optional' Solution: download the override file from repo and pass it to reprepro. Signed-off-by: Aliaksei Karpovich --- scripts/deb-repo | 97 +++++++++++++++++++++++++++++++++++++++++++----- 1 file changed, 88 insertions(+), 9 deletions(-) diff --git a/scripts/deb-repo b/scripts/deb-repo index 23de0461..99913fb4 100755 --- a/scripts/deb-repo +++ b/scripts/deb-repo @@ -89,6 +89,8 @@ import shutil import subprocess import pickle import urllib.parse +import urllib.request +import gzip import apt_pkg import apt.progress.base @@ -96,8 +98,38 @@ import apt.progress.base REPREPRO_TIMEOUT = 1200 +def parse_aptsources_list_line(source_list_line): + import re + + s = source_list_line.strip() + + if not s or s.startswith("#"): + return None + + type, s = re.split(r"\s+", s, maxsplit=1) + if type not in ["deb", "deb-src"]: + return None + + options = "" + options_match = re.match(r"\[\s*(\S+=\S+(?=\s))*\s*(\S+=\S+)\s*\]\s+", s) + if options_match: + options = options_match.group(0).strip() + s = s[options_match.end():] + + source, s = re.split(r"\s+", s, maxsplit=1) + + if s.startswith("/"): + suite = "" + else: + suite, s = re.split(r"\s+", s, maxsplit=1) + + components = " ".join(s.split()) + + return [type, options, source, suite, components] class DebRepo(object): + BOOTSTRAP_LIST = "/etc/apt/sources.list.d/bootstrap.list" + class DebRepoCtx(object): def __init__(self, workdir): self.distro = "debian" @@ -181,9 +213,9 @@ class DebRepo(object): with open(f"{self.workdir}/var/lib/dpkg/status", "w"): pass - os.makedirs(f"{self.workdir}/etc/apt/sources.list.d", exist_ok=True) + srcfile = f"{self.workdir}{self.BOOTSTRAP_LIST}" + os.makedirs(os.path.dirname(srcfile), exist_ok=True) - srcfile = f"{self.workdir}/etc/apt/sources.list.d/bootstrap.list" if aptsrcsfile and os.path.exists(aptsrcsfile): shutil.copy(aptsrcsfile, srcfile) else: @@ -201,13 +233,60 @@ class DebRepo(object): def create_repo_dist(self): conf_dir = f"{self.ctx.repodir}/{self.ctx.distro}/conf" os.makedirs(conf_dir, exist_ok=True) - if not os.path.exists(f"{conf_dir}/distributions"): - with open(f"{conf_dir}/distributions", "w") as f: - f.write(f"Codename: {self.ctx.codename}\n") - f.write( - "Architectures: " - "i386 armhf arm64 amd64 mipsel riscv64 source\n") - f.write("Components: main\n") + override_fname = "override" + self.create_override(f"{conf_dir}/{override_fname}") + with open(f"{conf_dir}/distributions", "w") as f: + f.write(f"Codename: {self.ctx.codename}\n") + f.write( + "Architectures: " + "i386 armhf arm64 amd64 mipsel riscv64 source\n") + f.write("Components: main\n") + f.write(f"DebOverride: {override_fname}\n") + + def create_override(self, override_file): + deb_override_name = f"override.{self.ctx.codename}.main" + deb_override_file = f"{self.workdir}/tmp/{deb_override_name}" + if (not os.path.exists(deb_override_file) or + os.path.getsize(deb_override_file) < 1): + # Extract source URL + srcfile = f"{self.workdir}{self.BOOTSTRAP_LIST}" + with open(srcfile, "r") as f: + for line in f: + entry = parse_aptsources_list_line(line) + if entry: + source_url = entry[2] + # Download override file + self.fetch_file(f"{source_url}/indices/{deb_override_name}") + if (os.path.exists(deb_override_file) and + os.path.getsize(deb_override_file) > 1): + break + # Download packet override file + self.fetch_file(f"{source_url}/indices/{deb_override_name}.gz") + # Unpack override file + with gzip.open(f"{deb_override_file}.gz", "r") as f_in, open(deb_override_file, "wb") as f_out: + for line in f_in: + f_out.write(line) + break + + with open(override_file, "w") as f_out: + # Unpack deb override file + with open(deb_override_file, "r") as f: + # Convert deb override to reprepro override + for line in f: + fields = line.strip().split() + + if len(fields) < 3: + continue + + package = fields[0] + priority = fields[1] + section = fields[2] + + f_out.write(f"{package} Priority {priority}\n") + f_out.write(f"{package} Section {section}\n") + + if f_out.tell() < 1: + print("WARNING: override file is empty!") def apt_config(self, init, crossbuild): if not init and self.ctx.compatarch: From patchwork Fri Sep 4 07:52:54 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Aliaksei Karpovich X-Patchwork-Id: 5429 Return-Path: Received: from shymkent.ilbers.de ([unix socket]) by shymkent (Cyrus 2.5.10-Debian-2.5.10-3+deb9u2) with LMTPA; Fri, 04 Sep 2026 09:56:42 +0200 X-Sieve: CMU Sieve 2.4 Received: from mail-wm1-f60.google.com (mail-wm1-f60.google.com [209.85.128.60]) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPS id 6847ufmZ004952 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Fri, 4 Sep 2026 09:56:41 +0200 Received: by mail-wm1-f60.google.com with SMTP id 5b1f17b1804b1-49ccfad90f1sf4545315e9.0 for ; Fri, 04 Sep 2026 00:56:41 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1788508596; cv=pass; d=google.com; s=arc-20260327; b=oSmN11SQZna9KwHODAsOwPoUMsQO0Os1iHgKA9AiZREuLO4uZOQocV90ZKT08cVl6C VmYU+QtWe4KPyX4FboRcRCKyPcph87Wl2DFD9l0MVX3JHUVMlnP60hrEJm81p91vSyaD rsMSOnkbBUb6jkSeyzyFFmM4K0EHuFGiGNFToiR7CBqCCjNpoR6yyc3DFEEnD/XQZBoB eiWyM7+1DLnu0KSaVK13amZTu1Z0O8bax//YUjFL2CO1qqdTTiO1Mzw6G8p6nxKWTlDM 4y6sTGFriv6eqehaHHiqBxh7WX5UNot0R0Iln0/m7SToxhhwtzQfjSvT7POaR4WUhu78 Jmuw== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20260327; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:dkim-signature; bh=/edSDFywZR3SRRuQncjxFqw2HpKsV9XefW/miCYxk/E=; fh=ARlL1+3UrdAyxFOE/7Iy26yoIAkzdFtP9vy7fNfr8hU=; b=eXhaHyntSn3KHr0he3SLGTUmGG80aR14IPfPXN8PKtPUy9PVZxJVpbvNbnKjV/JuX0 wsCSQ2reuh0WQT9PlwpwxSkgSfg0lEP473+NLtJ6QTM28pmlNpVE0pcwYj0p06Q3i5oI /YjM5hZithnWhXH3Oi2RoMuTVikY/SDgPYldwJrwpcjx47nvvSvax4xbiO5yd3aW4Et5 3nlv/6JaTihtWNA6zHHT/3Cy4uB3EP1+MR13e3pYdyr/EPx3aJ7RVjg8sQcTmRdHWvVc bY/QUKIgkYpErVp/TPA5v4iKim52mh0u2MJVLesMwg+EFkykRqtyzesfqqU2CBHOdE1k U8lw==; darn=isar-build.org ARC-Authentication-Results: i=2; gmr-mx.google.com; spf=pass (google.com: domain of akarpovich@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=akarpovich@ilbers.de DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlegroups.com; s=20251104; t=1788508596; x=1789113396; darn=isar-build.org; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:content-type :x-original-authentication-results:x-original-sender:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from:sender :from:to:cc:subject:date:message-id:reply-to:content-type; bh=/edSDFywZR3SRRuQncjxFqw2HpKsV9XefW/miCYxk/E=; b=m0s6Iuget8Rgjxg4hYp5VQY9Zs8P6YNIc2jP+K7dTMc+MmoOpR/YjXtF9bLXCzIkb7 xjcpO/1c1EZDh4eurPWZzg44LgXva0RiIAgItjZCBdzMsZhDZsC830e/juYUS3Tcpqep lWZUKbT6Tn/V4Y6WVfIvlKirJOqisZnhqJUkNWoewsq1hzIqqcnZlhLLd3dFjuQo3UMf 2IRKlRuCFfL3wmVVrImfFZ7m0HuNGzwnmKMAhl9i+D4PSyQRs9WiGNrYz5fLxdqW81nd ubSSx72a9nZ57M/bxDkB93vwM5nBKd/it/c0Zy7HMMrnTEsqN4EswfZbIogJIzlq1SMZ vxUw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1788508596; x=1789113396; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :x-spam-checked-in-group:list-id:mailing-list:precedence :content-type:x-original-authentication-results:x-original-sender :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:x-beenthere:x-gm-message-state:sender:from:to:cc:subject:date :message-id:reply-to:content-type; bh=/edSDFywZR3SRRuQncjxFqw2HpKsV9XefW/miCYxk/E=; b=YCdvVXbpJbTX128kgmlVmeN7mT28oTkSEtQO97vY1E9ojduM6xwgZOU/oPCoTDDD6O FkDQt7O5onHBOiSXGpdznWHULQjKC5f4Hn+XQYPQwOxE+T8RKAy0CcMmdh/rXdwfN+Bn 6J8ZKCoOe05KAtavPYvU2+ZP0PHPf6QsZab20HuXJObHxS6/i2ZCOh/riDGOvQmDvuVm DsCY8yigqdeI7AKCK50bX1EK6+IcQ4KmWl/9f+NfGK/fHuPHh643ptiHwU7ReYg2xKYK KYDlqHVlz6seyRD6TUGXluPFsNgrKeqb5pqF38dopJSAHx3E636Mto8Zf5lUnaputEOT 6ilQ== Sender: isar-users@googlegroups.com X-Forwarded-Encrypted: i=2; AKwUvBxmbgyveUEg0ZY2b6kJ7eJR6ry9BmhjHyx54Vxx59rbGRz4/a54rCbbPFZE9VpxtD9NdZ1YvaM=@isar-build.org X-Gm-Message-State: AFuF++kBZmogzfMP3/ouIViRy4YBVlrsxxGmU3H6XhJI1aV2m48DpmIt 0szvJIA2qHTeh1QwraMrbZtGgi0C3aMNyfRM8O7W2VsPJAIbnWmAABh8 X-Received: by 2002:a05:600c:8716:b0:49a:a101:4157 with SMTP id 5b1f17b1804b1-49cf81f4924mr45865875e9.7.1788508595970; Fri, 04 Sep 2026 00:56:35 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com; h="ATskLdcsq5YbcIupFFDw36/XH47J9+B7e1BkPKAMptgLXrzClQ==" Received: by 2002:a05:600c:3b11:b0:497:feb0:f55c with SMTP id 5b1f17b1804b1-49cf7ab4985ls5563195e9.0.-pod-prod-06-eu; Fri, 04 Sep 2026 00:56:34 -0700 (PDT) X-Received: by 2002:a05:600c:37cc:b0:499:ad2e:f7bc with SMTP id 5b1f17b1804b1-49cf823f107mr52902325e9.10.1788508593991; Fri, 04 Sep 2026 00:56:33 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1788508593; cv=none; d=google.com; s=arc-20260327; b=q1SAPtOeMz3djDh9snBkrxr8zO9tN9fznNH2uadMWiB86RfuTR46tDFUuPGc1pkXIU qUu9wBF5Zjfj2b261U/sD3p/xzyBKPzSaVEZasNK+s8EVG5nrXFhMay0v733jIcV0bTm WfNZ87EH39b6F1j20nubpT28hLCNnpAvsycmxbeTQYJKncJIrgjU93YOp4ugiL8bIYgE oo1i/PvXZGm0WCEF3UCY0tKEptfaCJAgENQieBjnasg1AQP0eDuy8WJ7GjlIDgd27trb +tqTaXKjiOImqG0dG6z8/OZ2EPEmZ2MJz+mmRRdQhGvVBeSAs0/GATObnaCI6dxJakeo SMDg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20260327; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=wMtc3HrQ1iS4MO4ciJem8cciP5B2gyAEAwICcqQpyZU=; fh=u+3kXEMzY4uVhDBiujW2QUo7NnpMLrGEgMQBYpCTRN4=; b=CuvdfVO68Xp/oqZPwPWsTAeEAcyQ+qLDMPKQ0De7XwwclPAY1f2mwRUszU6j+vb4Oy pF2kcX1C0EsVl+dr6HldbLsuFZ9O28QTIeeHvYhufCBGIxMiK8Z3I2qYi9JGosLAr8oy +An65r3Nqv9pYGhq1YWRmrr90GQRIllIs9JbmOoYOIDP9r5wS7SRZC8ttG+DBSEcNLWk KZ/0sBf9WFjy3Y3jKxrsj2qODr71R3BPX1hJRBd521+107Oo8xP/ATThjOq4SoBSpbfB NAr5dbCgqSNf52CZAaRuvCg73BdeU3aIgBNf6gkzMxFTt2cm8TYGAars9dJv4WfTRsf9 K7PA==; dara=google.com ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of akarpovich@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=akarpovich@ilbers.de Received: from shymkent.ilbers.de (shymkent.ilbers.de. [85.214.156.166]) by gmr-mx.google.com with ESMTPS id ffacd0b85a97d-485885b717dsi31051f8f.4.2026.09.04.00.56.33 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 04 Sep 2026 00:56:33 -0700 (PDT) Received-SPF: pass (google.com: domain of akarpovich@ilbers.de designates 85.214.156.166 as permitted sender) client-ip=85.214.156.166; Received: from akarpovich-Victus-by-HP-Laptop-16-e0xxx.promwad.corp (62.122.174.6.serverel.net [62.122.174.6] (may be forged)) (authenticated bits=0) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPSA id 6847tRSM004124 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 4 Sep 2026 09:56:21 +0200 From: Aliaksei Karpovich To: isar-users@googlegroups.com Cc: Aliaksei Karpovich Subject: [v10 12/12] base-apt: Move bullseye-mipsel in separate dir Date: Fri, 4 Sep 2026 09:52:54 +0200 Message-ID: <20260904075515.25962-13-akarpovich@ilbers.de> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260904075515.25962-1-akarpovich@ilbers.de> References: <20260904075515.25962-1-akarpovich@ilbers.de> MIME-Version: 1.0 X-Spam-Status: No, score=-4.6 required=5.0 tests=DKIMWL_WL_MED,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_EF,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,RCVD_IN_DNSWL_BLOCKED,RCVD_IN_MSPIKE_H2, RCVD_IN_RP_CERTIFIED,RCVD_IN_RP_RNBL,RCVD_IN_RP_SAFE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on shymkent.ilbers.de X-Original-Sender: akarpovich@ilbers.de X-Original-Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of akarpovich@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=akarpovich@ilbers.de Precedence: list Mailing-list: list isar-users@googlegroups.com; contact isar-users+owners@googlegroups.com List-ID: X-Spam-Checked-In-Group: isar-users@googlegroups.com X-Google-Group-Id: 914930254986 List-Post: , List-Help: , List-Archive: , List-Unsubscribe: , X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= As the bullseye-mipsel has separate bullseye repo (archive) it leads to unresolving dependency (f.e. gnupg) in base-apt repo. It happens because mipsel satied on deb11u2 version (and dont't have secure update) then other move formward and have deb11u3 version. When the bullseye-mipsel is built in the same time with other arch targed (f.e. bullseye arm) the following dependency issue occurs: | The following packages have unmet dependencies: | gnupg : Depends: dirmngr (< 2.2.27-2+deb11u3.1~) but it is not going to be installed | Depends: dirmngr (>= 2.2.27-2+deb11u3) but it is not going to be installed | Depends: gnupg-utils (>= 2.2.27-2+deb11u3) but 2.2.27-2+deb11u2 is to be installed | Depends: gpg (>= 2.2.27-2+deb11u3) but 2.2.27-2+deb11u2 is to be installed | Depends: gpg-agent (>= 2.2.27-2+deb11u3) but 2.2.27-2+deb11u2 is to be installed It happens becasue (during arm build) gnupg_2.2.27-2+deb11u3_all.deb was fetched, but other dependencies (dirmngr, gnupg-utils) for mipsel have old deb11u2 version. Fix: separate bullseye-mipsel into separate repo in base-apt. Signed-off-by: Aliaksei Karpovich --- meta/conf/bitbake.conf | 5 +++-- meta/conf/distro/debian-bullseye.conf | 1 + 2 files changed, 4 insertions(+), 2 deletions(-) diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf index c8503ec6..97f200a0 100644 --- a/meta/conf/bitbake.conf +++ b/meta/conf/bitbake.conf @@ -99,8 +99,9 @@ REPO_ISAR_DIR = "${DEPLOY_DIR}/isar-apt/${DISTRO}/apt" REPO_ISAR_DB_DIR = "${DEPLOY_DIR}/isar-apt/${DISTRO}/db" # Base apt repository paths -REPO_BASE_DIR = "${DEPLOY_DIR}/base-apt/${DISTRO}/apt" -REPO_BASE_DB_DIR = "${DEPLOY_DIR}/base-apt/${DISTRO}/db" +REPO_BASE_DIR_NAME ?= "${DISTRO}" +REPO_BASE_DIR = "${DEPLOY_DIR}/base-apt/${REPO_BASE_DIR_NAME}/apt" +REPO_BASE_DB_DIR = "${DEPLOY_DIR}/base-apt/${REPO_BASE_DIR_NAME}/db" # Setup our default hash policy BB_SIGNATURE_HANDLER ?= "OEBasicHash" diff --git a/meta/conf/distro/debian-bullseye.conf b/meta/conf/distro/debian-bullseye.conf index c368dcaf..00d50df4 100644 --- a/meta/conf/distro/debian-bullseye.conf +++ b/meta/conf/distro/debian-bullseye.conf @@ -2,6 +2,7 @@ HOST_DISTRO_APT_SOURCES:mipsel ?= "conf/distro/${HOST_DISTRO}-mipsel.list" DISTRO_APT_SOURCES:mipsel ?= "conf/distro/${BASE_DISTRO}-${BASE_DISTRO_CODENAME}-mipsel.list" +REPO_BASE_DIR_NAME:mipsel = "${DISTRO}-mipsel" require debian-common.conf