From patchwork Fri Oct 2 07:01:04 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: akarpovich X-Patchwork-Id: 5517 Return-Path: Received: from shymkent.ilbers.de ([unix socket]) by shymkent (Cyrus 2.5.10-Debian-2.5.10-3+deb9u2) with LMTPA; Fri, 02 Oct 2026 09:05:09 +0200 X-Sieve: CMU Sieve 2.4 Received: from mail-lj1-f186.google.com (mail-lj1-f186.google.com [209.85.208.186]) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPS id 692757MA003900 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Fri, 2 Oct 2026 09:05:07 +0200 Received: by mail-lj1-f186.google.com with SMTP id 38308e7fff4ca-3a95fe1c97esf1180991fa.1 for ; Fri, 02 Oct 2026 00:05:07 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1790924700; cv=pass; d=google.com; s=arc-20260327; b=bKFpuIr6yeSxJJs2HQPmReUz08lZ2Kh3r5SB0Xxma9YH4tFn8TlmjqFnx1+WpYUjU/ 0hsKN4jzqEFJ3Ur4A1yIZQKjfysdvTfxY/WH0nJtBMM9wG+7VR481L3VzfTJ1rVgtmrL jr1WNNcERQ2sJQtPZ1g4J2wLAwqaNTKUTwP4PR4IIdCodrp8UDoVorwdyOZB8tq9DeSJ RR8QPLIsn8b9EB/OX/RbMQw/TWSS+hb7B+DY3/LAVhmjf0nb8sHXhor/jep4s2eyPA98 XT3LdkrF/Ij+a7fuHWl8n0yz4tY/xK6cHdxsLiKCEskMnDzsF4UwdMwGvD46eMhTZ9OF 2idg== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20260327; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:dkim-signature; bh=qV0JMDd3H2oAXAA8x4wOZe1ymCylLgK1gSYDfX9AD9g=; fh=QAYsispzbLS3abaHEVRC8eEEB69tHb+Ogsv3mCZ9JI0=; b=XJLDHkxH/f7LVUL9msGPDvJ+eRcg+gfslPZsNrsUw8y7fXtxgNvukA/c6glNeDQa87 N4tulwcGe6Z+q+yX0CS6UW70lMusnMpI/gTicX1xHh5koEAMHCt0KU6rvh/2m9yrl9PI ROY7rCgxpYhcvE79K4rJWVYSfS40xSMRuorz0zSouwI7W82n7h/TvlcsgkjvJvWnm5kR NpTusw0Lm2j7L/zZ9rgPo7DpXEtfNFUOVO/5sKWhan/hiZ/4U+F1D1heAx/FM0m4Vrp2 izCjvH0cPVL5Do94ZdFZ4Wark/5ICWzGpaCywx1EEVTCZFX1Y3ao/d2Zv+PRRGc0vJNx uBnQ==; darn=isar-build.org ARC-Authentication-Results: i=2; gmr-mx.google.com; spf=pass (google.com: domain of akarpovich@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=akarpovich@ilbers.de DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlegroups.com; s=20251104; t=1790924700; x=1791529500; darn=isar-build.org; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:content-type :x-original-authentication-results:x-original-sender:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from:sender :from:to:cc:subject:date:message-id:reply-to:content-type; bh=qV0JMDd3H2oAXAA8x4wOZe1ymCylLgK1gSYDfX9AD9g=; b=h4GGMaOOgr6lmyAP0oKF4N9s76HStAOokFzYLAA+5uUA6rx1IWxxHBtmHSpYbuQS9B 3ngzi/DqCckQO8crKNNtscj/r5CL+nDS15duia0IAqlY1q1STd6c5SQdlvYRZEwvprYI NY8yF+LEiHJDv+e6cQnUss0JUQn77mlXIeManIAsYtR9FpF7/Ct3TU+lr12n3neA0qLH zzMwuP6FdjMmxibc4OcbSat1gAeSSodOJfQsqa0Subk3SyNCsjjru57EBX7ORNsReisp 9QNorjdaor0EcfaWMdFoOTIZOoCH0bA/An2z3fC51LoFagl+U+0y5/qSlNvUwCKqJJuC 6VUQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20260707; t=1790924700; x=1791529500; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :x-spam-checked-in-group:list-id:mailing-list:precedence :content-type:x-original-authentication-results:x-original-sender :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:x-beenthere:x-gm-message-state:sender:from:to:cc:subject:date :message-id:reply-to:content-type; bh=qV0JMDd3H2oAXAA8x4wOZe1ymCylLgK1gSYDfX9AD9g=; b=szCyXm1XWLb2d+aQ5uMlmXvVb7x9s7L834CeGIsX4xQVDW2P4XhUkgRouVEovrl9Ff UoRZs4BTQc1OLCGyuP3d6h2baK2FHnuSxohsEbkVo2nu6v94whn189SPdsvGNEK6zjxL 1KzyWvJ/eW6br5SOMYrOtwM3cvhAQLHvPRUmjsLtZ90InT1pQYhj0p1SI6AFj/X9m/Ik U3rrtfJlvX4IR9JF3NFPTkF7y26jHuWNAX+qXyqAKcJQ4ojjaeaEZ4bswLVoC/m1h6LA HvO9CmsQGAyzG6ck5RmJLBnx8v9tGm2ZSMw8dftUMN5ToGmYXN8z8fyiF4V62Q1mYlEQ Wmeg== Sender: isar-users@googlegroups.com X-Forwarded-Encrypted: i=2; AKwUvBz/pr/hiOxU+xE01NfN+uSjWCn05Ep4qAtdRTQpMN1OOMeVNAGCLhJtWTA5J9dDR9VZfaXP/zk=@isar-build.org X-Gm-Message-State: AFq9FYLp6zrh7I++TeGdf4F91Aw35IhACC/jNKNL6tXN4Ev97hMLWDiz t4qXlRJtYRs19IgBvVqu5lTYJ8PTTazhDdrjwB+vH3X5+2fb3MkITAf4 X-Received: by 2002:a05:651c:15c3:20b0:3a7:72f6:9700 with SMTP id 38308e7fff4ca-3a8877bc819mr2789571fa.24.1790924700131; Fri, 02 Oct 2026 00:05:00 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com; h="ATskLddwE7Baa5jqH+0TwKygJGzPTBZujED4sNFxPZr1aFIGvg==" Received: by 2002:a2e:b5cc:0:b0:3a6:2ed5:81a6 with SMTP id 38308e7fff4ca-3a8a86335f8ls1039951fa.1.-pod-prod-07-eu; Fri, 02 Oct 2026 00:04:55 -0700 (PDT) X-Received: by 2002:a2e:be28:0:b0:3a7:733d:a303 with SMTP id 38308e7fff4ca-3a8877b9dc6mr4238221fa.22.1790924695353; Fri, 02 Oct 2026 00:04:55 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1790924695; cv=none; d=google.com; s=arc-20260327; b=MrX16x9+EGypW+8TN2C/LEZ/hu2Q7iSvysh1jquHMRl1ckLLGZq6ak/Bls1T8lJQYq o4C1D71n1gjEFuPW1jgjBss5DI3FZqhytzvAzTHaBYmHWaAdZhrpyK7Fij/0snikUEyl n0xmSMq9z88aD85hkBqCRMRsPqEPcBd2lr7OeWqOG+D/uuBIbo0i/IyUo36pwy/NFnnr 1/rko2gg8EYh3ts5khKI8trX9vVIydz5Kzk7qYekR0BI06oBW4gQHhK6wpDBc7gw8mut PPKzvSKYQMUcPapMX3p6ttDDdk7mB0RC0JcYRLyDgW1+Gdqb9v+jVtFbwrtdsYzsnPyX WJTA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20260327; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=3KCT9Sv+CdZ0s64PnShNsSXd3AAJOdfiEKyiskESHvA=; fh=8xvFQ9N6ENvazHnylkO2sbgzUV6bRnZQvNiaC5bb+x4=; b=DXs2ozufD0bdQBpB56szLr1nwo0XnzfZnuY0HRFMoo8b6QXYWB8jefUm7Yq8eHfo3Q b+blnPKmCEfxeZmuOCksGYkXUQnaARQqZYM1uvUV2mqhx4FUTLdDvIgMgj2behdu0aNQ bhrPlYYT+gDezlENb36TtMPY4PR4Y+dzi359qHsVpb0CJFGIxEJLT0c/9HL0BD63y+3G MxpJcYNSg1Yik56hrMBCX6LWNZthlrMgxH5JKJuc85fcPXqLV9XcT1/cU2L47bZ4vJBs DYJAZ+v13DRUbbyT95R9JrYu3bI9SicbSF8GA5MUW70rEasq55m8SPHJWjVJA+57rAly PiPQ==; dara=google.com ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of akarpovich@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=akarpovich@ilbers.de Received: from shymkent.ilbers.de (shymkent.ilbers.de. [85.214.156.166]) by gmr-mx.google.com with ESMTPS id 38308e7fff4ca-3a87e348be0si309521fa.9.2026.10.02.00.04.55 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 02 Oct 2026 00:04:55 -0700 (PDT) Received-SPF: pass (google.com: domain of akarpovich@ilbers.de designates 85.214.156.166 as permitted sender) client-ip=85.214.156.166; Received: from laptop-0xxx (62.122.174.6.serverel.net [62.122.174.6] (may be forged)) (authenticated bits=0) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPSA id 69274a1g003799 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 2 Oct 2026 09:04:53 +0200 From: akarpovich To: isar-users@googlegroups.com Cc: Uladzimir Bely , Aliaksei Karpovich , Aliaksei Laurynovich Subject: [v11 01/12] scripts: Add debrepo python script handling base-apt Date: Fri, 2 Oct 2026 09:01:04 +0200 Message-ID: <20261002070335.26526-2-akarpovich@ilbers.de> X-Mailer: git-send-email 2.53.0 In-Reply-To: <20261002070335.26526-1-akarpovich@ilbers.de> References: <20261002070335.26526-1-akarpovich@ilbers.de> MIME-Version: 1.0 X-Spam-Status: No, score=-4.6 required=5.0 tests=DKIMWL_WL_MED,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_EF,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,RCVD_IN_DNSWL_BLOCKED,RCVD_IN_MSPIKE_H2, RCVD_IN_RP_CERTIFIED,RCVD_IN_RP_RNBL,RCVD_IN_RP_SAFE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on shymkent.ilbers.de X-Original-Sender: akarpovich@ilbers.de X-Original-Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of akarpovich@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=akarpovich@ilbers.de Precedence: list Mailing-list: list isar-users@googlegroups.com; contact isar-users+owners@googlegroups.com List-ID: X-Spam-Checked-In-Group: isar-users@googlegroups.com X-Google-Group-Id: 914930254986 List-Post: , List-Help: , List-Archive: , List-Unsubscribe: , X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= From: Uladzimir Bely This is the main utility responsible for prefetching packages into local `base-apt` repo from external Debian mirrors. It uses python-apt module and requires some kind of minimal `rootfs` to work (let's call it "debrepo context"). Once initialized with `--init --workdir=`, it stores the initial configuration in `repo.opts` file inside the context and uses it at futher calls. In future, the logic `debrepo` script implements could be directly implemented inside bitbake classes. Signed-off-by: Uladzimir Bely Signed-off-by: Aliaksei Karpovich Signed-off-by: Aliaksei Laurynovich --- scripts/deb-repo | 610 +++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 610 insertions(+) create mode 100755 scripts/deb-repo diff --git a/scripts/deb-repo b/scripts/deb-repo new file mode 100755 index 00000000..23de0461 --- /dev/null +++ b/scripts/deb-repo @@ -0,0 +1,610 @@ +#!/usr/bin/env python3 + +""" +# This software is a part of Isar. +# Copyright (C) 2026 ilbers GmbH + +# deb-repo: build Debian-like repo using "python3-apt" library. + +When building the image, Isar downloads required Debian packages from external +mirrors. After build completed, it can pick all downloaded packages from DL_DIR +and build local 'base-apt' Debian-like repo from them. + +This tool allows to download packages and create local repo in advance. So, +Isar just uses this local repository and does not interact with external +mirrors. Such approach makes deb-dl import/export functionality redundant. + +Script `deb-repo` works in so-called "context" directory. It means some +rootfs-like directory with bare minimum of directories/files required for +python3-apt to work. + +Context directory path is passed with "--workdir " command-line option. +On context creating, all passed parameters are stored in the context directory +and picked every time the context is used again. + +1. Repo for building Debian system +``` +deb-repo --workdir=d12 --init locales gnupg +``` +Initialize the context in "d12" directory and create a repository in +"d12/repo/apt" directory sufficient to debootstrap default system +(e.g., debian-bookworm-amd64). Additionally, packages "locales" and "gnupg" +with all their dependencies will be available in this repo. + +``` +deb-repo --workdir=d12 docbook-to-man +``` +Adds "docbook-to-man" packages with its dependencies to earlier created repo. + +``` +deb-repo --workdir=d12 --srcmode docbook-to-man +``` +Downloads source package for "docbook-to-man" and adds it to the repo + +2. Repo for building Ubuntu system +``` +deb-repo --init --workdir=uf \ +--distro=ubuntu --codename=focal --arch=arm64 \ +--aptsrcsfile=/work/isar/meta-isar/conf/distro/ubuntu-focal-ports.list \ +--repodir=repo/apt --repodbdir=repo/db \ +--mirror=http://ports.ubuntu.com/ubuntu-ports \ +locales gnupg +``` +Initialize the context in "uf" directory and create a repository in "repo/apt" +directory sufficient deboostraup ubuntu-focal arm64 system. Mirror to use and +source list are specified by corresponding arguments. Packages "locales" and +" gnupg" with the dependencies will be also placed to the repo. + +``` +deb-repo --workdir=uf gnupg,locales +``` +Add "gnupg" and "locales" packages with their dependencies to earlier created +ubuntu repo. Other parameters (distro, codename, arch) are ommited since they +are picked from the context. + +3. Repo for cross-building Debian system +``` +deb-repo --init --workdir=d11 --codename=bullseye--arch=amd64 --crossarch=armhf +``` +Initialize the context in "d11" directory sufficient to deboostrap Debian +Bullseye (amd64) with foreign "armhf" architecture support + +``` +deb-repo --workdir=d11 gcc +``` +Add "gcc" package (amd64 version) to earlier created repo. + +``` +deb-repo --workdir=d11 --crossbuild gcc +``` +Add "gcc" package (armhf version) to earlier created repo. +""" + +import os +import sys +import fcntl + +import argparse +import shutil +import subprocess +import pickle +import urllib.parse + +import apt_pkg +import apt.progress.base + + +REPREPRO_TIMEOUT = 1200 + + +class DebRepo(object): + class DebRepoCtx(object): + def __init__(self, workdir): + self.distro = "debian" + self.codename = "bullseye" + self.arch = "amd64" + self.mirror = "http://deb.debian.org/debian" + + self.repodir = f"{workdir}/repo/apt" + self.repodbdir = f"{workdir}/repo/db" + + self.crossarch = self.arch + self.compatarch = None + self.keydir = "/etc/apt/trusted.gpg.d" + + def __init__(self, args): + self.workdir = os.path.abspath(args.workdir) + self.ctx = self.DebRepoCtx(self.workdir) + + self.cache = None + self.depcache = None + self.sr = None + self.extrarepo = None + + self.ctx_load() + self.ctx_update(args) + self.ctx_save() + + print( + f"ctx workdir: {self.workdir}\n" + f" distro: {self.ctx.distro}\n" + f" codename: {self.ctx.codename}\n" + f" arch: {self.ctx.arch}\n" + f" mirror: {self.ctx.mirror}\n" + f" repodir: {self.ctx.repodir}\n" + f" repodbdir: {self.ctx.repodbdir}\n" + f" crossarch: {self.ctx.crossarch}\n" + f" compatarch: {self.ctx.compatarch}\n" + f" keydir: {self.ctx.keydir}" + ) + + if args.extrarepo: + self.extrarepo = os.path.abspath(args.extrarepo) + + def ctx_load(self): + ctxfile = f"{self.workdir}/deb-repo.ctx" + + if os.path.isfile(ctxfile): + with open(ctxfile, 'rb') as f: + self.ctx = pickle.load(f) + + def ctx_save(self): + ctxfile = f"{self.workdir}/deb-repo.ctx" + + with open(ctxfile, 'wb') as f: + pickle.dump(self.ctx, f) + + def ctx_update(self, args): + if args.distro: + self.ctx.distro = args.distro + if args.codename: + self.ctx.codename = args.codename + if args.arch: + self.ctx.arch = args.arch + if args.mirror: + self.ctx.mirror = args.mirror + + if args.repodir: + self.ctx.repodir = os.path.abspath(args.repodir) + if args.repodbdir: + self.ctx.repodbdir = os.path.abspath(args.repodbdir) + + if args.crossarch: + self.ctx.crossarch = args.crossarch + if args.compatarch: + self.ctx.compatarch = args.compatarch + if args.keydir: + self.ctx.keydir = args.keydir + + def create_rootfs(self, aptsrcsfile): + os.makedirs(f"{self.workdir}/var/lib/dpkg", exist_ok=True) + with open(f"{self.workdir}/var/lib/dpkg/status", "w"): + pass + + os.makedirs(f"{self.workdir}/etc/apt/sources.list.d", exist_ok=True) + + srcfile = f"{self.workdir}/etc/apt/sources.list.d/bootstrap.list" + if aptsrcsfile and os.path.exists(aptsrcsfile): + shutil.copy(aptsrcsfile, srcfile) + else: + with open(srcfile, "w") as f: + repo = f"{self.ctx.mirror} {self.ctx.codename} main" + f.write(f"deb {repo}\n") + f.write(f"deb-src {repo}\n") + + dir_cache = f"../apt_cache/{self.ctx.distro}-{self.ctx.codename}" + os.makedirs(f"{self.workdir}/{dir_cache}/archives/partial", + exist_ok=True) + + os.makedirs(f"{self.workdir}/tmp", exist_ok=True) + + def create_repo_dist(self): + conf_dir = f"{self.ctx.repodir}/{self.ctx.distro}/conf" + os.makedirs(conf_dir, exist_ok=True) + if not os.path.exists(f"{conf_dir}/distributions"): + with open(f"{conf_dir}/distributions", "w") as f: + f.write(f"Codename: {self.ctx.codename}\n") + f.write( + "Architectures: " + "i386 armhf arm64 amd64 mipsel riscv64 source\n") + f.write("Components: main\n") + + def apt_config(self, init, crossbuild): + if not init and self.ctx.compatarch: + apt_pkg.config["APT::Architectures::"] = self.ctx.compatarch + + if not init and self.ctx.arch != self.ctx.crossarch: + apt_pkg.config["APT::Architectures::"] = self.ctx.crossarch + apt_pkg.config["APT::Architectures::"] = self.ctx.arch + + apt_pkg.config.set("APT::Architecture", self.ctx.arch) + + apt_pkg.config.set("Dir", self.workdir) + + dir_cache = f"../apt_cache/{self.ctx.distro}-{self.ctx.codename}" + apt_pkg.config.set("Dir::Cache", f"{self.workdir}/{dir_cache}") + apt_pkg.config.set("Dir::State::status", + f"{self.workdir}/var/lib/dpkg/status") + + apt_pkg.config.set("APT::Install-Recommends", "0") + apt_pkg.config.set("APT::Install-Suggests", "0") + + # Use host keys for authentification + apt_pkg.config.set("Dir::Etc::TrustedParts", self.ctx.keydir) + + # Allow using repositories without keys + apt_pkg.config.set("Acquire::AllowInsecureRepositories", "1") + + def mark_essential(self): + for pkg in self.cache.packages: + if pkg.architecture == self.ctx.arch: + if pkg.essential: + self.depcache.mark_install(pkg) + + def mark_by_prio(self, priority): + for pkg in self.cache.packages: + if pkg.architecture == self.ctx.arch: + ver = self.depcache.get_candidate_ver(pkg) + if ver and ver.priority <= priority: + self.depcache.mark_install(pkg) + + def mark_pkg(self, name, crossbuild): + pkgname = name + + if pkgname and (pkgname not in self.cache): + # Try for cross arch + if (pkgname, self.ctx.crossarch) in self.cache: + pkgname += f":{self.ctx.crossarch}" + + if pkgname not in self.cache: + print(f"Error: package '{name}' not found") + return False + + pkg = self.cache[pkgname] + + if (not crossbuild) or (':' in pkgname) or (not pkg.has_versions): + if (pkg.has_provides) and (not pkg.has_versions): + print("pkgname is virtual package, selecting best provide") + # Select first provide + pkg_provide = pkg.provides_list[0][2] + # Find better provide with higher version + for provide in pkg.provides_list: + if apt_pkg.version_compare(provide[2].ver_str, + pkg_provide.ver_str) > 0: + pkg_provide = provide[2] + self.depcache.mark_install(pkg_provide.parent_pkg) + else: + self.depcache.mark_install(pkg) + else: + version = pkg.version_list[0] + if version.arch == "all": + self.depcache.mark_install(pkg) + else: + if version.multi_arch == version.MULTI_ARCH_FOREIGN: + if (pkgname, self.ctx.arch) in self.cache: + nativepkg = self.cache[pkgname, self.ctx.arch] + self.depcache.mark_install(nativepkg) + else: + return False + else: + if (pkgname, self.ctx.crossarch) in self.cache: + crosspkg = self.cache[pkgname, self.ctx.crossarch] + self.depcache.mark_install(crosspkg) + else: + return False + + return True + + def mark_list(self, pkglist, crossbuild): + ret = True + if pkglist: + for pkgname in pkglist: + ret = ret and self.mark_pkg(pkgname, crossbuild) + + return ret + + def handle_deb(self, item): + fd = open(f"{self.ctx.repodir}/repo.lock", 'w') + fcntl.flock(fd, fcntl.LOCK_EX) + subprocess.run([ + "reprepro", + "--dbdir", f"{self.ctx.repodbdir}/{self.ctx.distro}", + "--outdir", f"{self.ctx.repodir}/{self.ctx.distro}", + "--confdir", f"{self.ctx.repodir}/{self.ctx.distro}/conf", + "-C", "main", + "includedeb", + self.ctx.codename, + item.destfile + ], timeout=REPREPRO_TIMEOUT) + fd.close() + + def handle_repo(self, fetcher): + dir_cache = f"../apt_cache/{self.ctx.distro}-{self.ctx.codename}" + fd = open(f"{self.workdir}/{dir_cache}.lock", "w") + fcntl.flock(fd, fcntl.LOCK_EX) + fetcher.run() + fd.close() + for item in fetcher.items: + if item.status == item.STAT_ERROR: + print("Some error ocured: '%s'" % item.error_text) + pass + else: + self.handle_deb(item) + + def get_filename(self, uri): + path = urllib.parse.urlparse(uri).path + unquoted_path = urllib.parse.unquote(path) + basename = os.path.basename(unquoted_path) + return basename + + def fetch_file(self, uri): + filename = self.get_filename(uri) + return subprocess.run([ + "wget", + "-H", + "--timeout=30", + "--tries=3", + "-nv", + uri, + "-O", + f"{self.workdir}/tmp/{filename}" + ], + stdout=subprocess.PIPE, + stderr=subprocess.PIPE) + + def handle_dsc(self, dsc_file): + fd = open(f"{self.ctx.repodir}/repo.lock", 'w') + fcntl.flock(fd, fcntl.LOCK_EX) + result = subprocess.run([ + "reprepro", + "--dbdir", f"{self.ctx.repodbdir}/{self.ctx.distro}", + "--outdir", f"{self.ctx.repodir}/{self.ctx.distro}", + "--confdir", f"{self.ctx.repodir}/{self.ctx.distro}/conf", + "-C", "main", + "-S", "-", "-P" "source", + "--delete", + "includedsc", + self.ctx.codename, + os.path.realpath(f"{self.workdir}/tmp/{dsc_file}") + ], timeout=REPREPRO_TIMEOUT) + fd.close() + return result.returncode == 0 + + def handle_src_list(self, pkgs): + ret = True + if pkgs: + local_repo = f"{self.ctx.repodir}/{self.ctx.distro}" + fetched_files = [] + for pkg in pkgs: + pkgname = pkg + pkgver = "" + if '=' in pkg: + pkgname = pkg.split("=")[0] + pkgver = pkg.split("=")[1] + + self.sr.restart() + while self.sr.lookup(pkgname): + if pkgver and pkgver != self.sr.version: + continue + + for sr_file in self.sr.files: + uri_file = self.sr.index.archive_uri(sr_file.path) + filename = os.path.basename(sr_file.path) + path = urllib.parse.urlparse(uri_file).path.lstrip('/') + # Strip the distro directory prefix from upstream path + # e.g. "debian/pool/main/h/hello/file.dsc" -> "pool/main/h/hello/file.dsc" + parts = path.split('/', 1) + local_path = parts[1] if len(parts) > 1 else path + local_file = f"{local_repo}/{local_path}" + print(uri_file) + if filename not in fetched_files: + if os.path.exists(local_file): + shutil.copy2(local_file, f"{self.workdir}/tmp/{filename}") + else: + result = self.fetch_file(uri_file) + if result.returncode != 0: + print(f"ERROR: cannot download {uri_file}") + print(f"ERROR: {result.stderr}") + fetched_files.append(filename) + if sr_file.type == 'dsc': + dsc_file = filename + + ret = self.handle_dsc(dsc_file) + break + return ret + + def apt_run(self, init, srcmode, pkgs, dscfile, crossbuild): + apt_pkg.init() + + extrarepo_list = f"{self.workdir}/etc/apt/sources.list.d/extrarepo.list" + if self.extrarepo: + extrarepo_list = f"{self.workdir}/etc/apt/sources.list.d/extrarepo.list" + with open(extrarepo_list, "w") as f: + distdir=os.path.join(self.extrarepo, "dists") + if os.path.isdir(distdir): + for dist in os.listdir(distdir): + repodir = os.path.join(distdir,dist) + if os.path.isdir(repodir): + for repo in os.listdir(repodir): + if os.path.isdir(os.path.join(repodir, repo)): + f.write(f"deb file://{self.extrarepo} " + f"{dist} {repo}\n") + + sources = apt_pkg.SourceList() + sources.read_main_list() + + progress = apt.progress.text.AcquireProgress() + + self.cache = apt_pkg.Cache() + if init: + self.cache.update(progress, sources) + self.cache = apt_pkg.Cache() + + self.sr = apt_pkg.SourceRecords() + + if self.extrarepo: + apt_pkg.config.set("Dir::Etc::SourceList", extrarepo_list) + apt_pkg.config.set("APT::Get::List-Cleanup", "0") + extrarepo_sources = apt_pkg.SourceList() + extrarepo_sources.read_main_list() + self.cache.update(progress, extrarepo_sources) + self.cache = apt_pkg.Cache() + os.remove(extrarepo_list) + + self.depcache = apt_pkg.DepCache(self.cache) + + ret = True + + if init: + self.mark_essential() + # 1(required), 2(important), 3(standard), 4(optional), 5(extra) + self.mark_by_prio(1) + + pkgs = list(filter(None, ','.join(pkgs).split(','))) + if srcmode: + ret = self.handle_src_list(set(pkgs)) + else: + ret = self.mark_list(pkgs, crossbuild) + + if dscfile: + fobj = open(dscfile, "r") + + try: + tagfile = apt_pkg.TagFile(fobj) + while tagfile.step() == 1: + deps = tagfile.section.get("Build-Depends", "") + # Remove extra commas and spaces - apt_pkg.parse_src_depends + # doesnt like lines like ", device-tree-compiler" + deps = ', '.join( + [s.strip() for s in deps.split(',') if s.strip()] + ) + print(f"parsed deps: {deps}") + for item in apt_pkg.parse_src_depends(deps, False): + pkgname = item[0][0] + self.mark_pkg(pkgname, crossbuild) + + finally: + fobj.close() + + if not ret: + sys.exit("Some of requested packages not found") + + if init or not srcmode: + fetcher = apt_pkg.Acquire(progress) + pm = apt_pkg.PackageManager(self.depcache) + + recs = apt_pkg.PackageRecords(self.cache) + pm.get_archives(fetcher, sources, recs) + + self.handle_repo(fetcher) + + +def parse_arguments(): + parser = argparse.ArgumentParser() + parser.add_argument( + "--init", + default=False, action="store_true", + help="initialize context in WORKDIR") + parser.add_argument( + "--workdir", + type=str, required=True, + help="work directory storing deb-repo context") + parser.add_argument( + "--aptsrcsfile", + type=str, metavar="PATH", + help="sources.list file to use when init") + parser.add_argument( + "--srcmode", + default=False, action="store_true", + help="add source packages instead of debs") + parser.add_argument( + "--repodir", + type=str, metavar="REPO", + help="repository directory") + parser.add_argument( + "--repodbdir", + type=str, metavar="REPODB", + help="repository database directory") + parser.add_argument( + "--extrarepo", + type=str, metavar="REPO", + help="extra repository to consider") + parser.add_argument( + "--mirror", + type=str, + help="use custom distro mirror") + parser.add_argument( + "--distro", + type=str, + help="select distro to use") + parser.add_argument( + "--codename", + type=str, + help="distro codename") + parser.add_argument( + "--arch", + type=str, + help="distro arch") + parser.add_argument( + "--compatarch", + type=str, metavar="ARCH", + help="compat arch to use") + parser.add_argument( + "--crossarch", + type=str, metavar="ARCH", + help="cross-build arch") + parser.add_argument( + "--keydir", + type=str, + help="directory with distro keys") + parser.add_argument( + "--no-check-gpg", + default=False, action="store_true", + help="allow insecure repositories") + parser.add_argument( + "--dscfile", + type=str, metavar="PATH", + help="Debian source file to parse") + parser.add_argument( + "--crossbuild", + default=False, action="store_true", + help="add packages with cross arch") + + parser.add_argument( + "packages", + nargs='*', type=str, + help="space- or comma-separated list of packages to add") + + args = parser.parse_args() + + return args + + +def main(): + args = parse_arguments() + + if not (args.init or args.packages or args.dscfile): + sys.exit("Nothing to do") + + workdir = os.path.abspath(args.workdir) + os.makedirs(workdir, exist_ok=True) + + with open(f"{workdir}/deb-repo.lock", "a") as file: + fcntl.flock(file.fileno(), fcntl.LOCK_EX) + + deb_repo = DebRepo(args) + + if args.init: + deb_repo.create_rootfs(args.aptsrcsfile) + deb_repo.create_repo_dist() + + deb_repo.apt_config(args.init, args.crossbuild) + deb_repo.apt_run(args.init, args.srcmode, args.packages, + args.dscfile, args.crossbuild) + + #Unlock deb-repo context + fcntl.flock(file.fileno(), fcntl.LOCK_UN) + + +if __name__ == "__main__": + main() From patchwork Fri Oct 2 07:01:05 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: akarpovich X-Patchwork-Id: 5514 Return-Path: Received: from shymkent.ilbers.de ([unix socket]) by shymkent (Cyrus 2.5.10-Debian-2.5.10-3+deb9u2) with LMTPA; Fri, 02 Oct 2026 09:05:07 +0200 X-Sieve: CMU Sieve 2.4 Received: from mail-ed1-f59.google.com (mail-ed1-f59.google.com [209.85.208.59]) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPS id 69275631003866 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Fri, 2 Oct 2026 09:05:06 +0200 Received: by mail-ed1-f59.google.com with SMTP id 4fb4d7f45d1cf-6ae14ecef5fsf2925204a12.0 for ; Fri, 02 Oct 2026 00:05:06 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1790924701; cv=pass; d=google.com; s=arc-20260327; b=OH7c99y9AsSvAJxCY/UThGd31YfQrTU32wNS3Y6DUgZPND3tVeo40LrzzGZWM+r+qy /CzRtZ5/pdZDcb8klObtrABWU+9BHH4HHHVVre2/48jqP0nsvSC9kJM6KC+28YJjFrdY Las4qZAWJdZJWeGCE5cJajpIhFN4mXJuMuDhLN4ZVdIwRmJZn6DHo6WP7FpisrwQSBUj bRZT5LP09n+YpBHPrU5EtwTdKPkyRoDCcFxBpDvM85vi0rqK9uyBpQAQHzNxzU8P4rEx qvgslT01eXb7rTmr9fBAKHxdFrU2NHiBfLJIcdR2DxPzepiSBCbFqb6HQ9zHLz3+is7d RiYw== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20260327; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:dkim-signature; bh=FpEt696zox+ITlWimkLnHzEIdSBWe6elJYgguk63Ql0=; fh=hma52+qoUiLod726vln0EWmnshkzvKwlST1gbiniW5M=; b=D0Jt/kYBJ6Rx/2Z+7/bsLP3+zAzHQOf7VAucIHmt49ylAhFjM6nzoJeNMs88klq22d e7OvRA5YMZF3nVrlatshHm2R5paav1rM9LhIm/m7E8R6ZpzKHhL/nU6S07+2AHg2kjtQ LB/LWUP3bkNGyTNxH6kHfEOPWwulvIMEBkA61KToDrwu56jyXAublrgM+CxXG9Lbxtz7 Il7w9a9lG3UaEqNILadj5Fsb8uGoTWsy1fHhPQ3sP2MNdrv0psCCod9LxHfzpC/bG+8H 1W/bg7b/H2LiaIEedwZXRewm+j6m2YRHlzZakfBT4H79Syi48vLNm9FqBhXpSKFg9aXe vcbA==; darn=isar-build.org ARC-Authentication-Results: i=2; gmr-mx.google.com; spf=pass (google.com: domain of akarpovich@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=akarpovich@ilbers.de DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlegroups.com; s=20251104; t=1790924701; x=1791529501; darn=isar-build.org; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:content-type :x-original-authentication-results:x-original-sender:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from:sender :from:to:cc:subject:date:message-id:reply-to:content-type; bh=FpEt696zox+ITlWimkLnHzEIdSBWe6elJYgguk63Ql0=; b=oTuOg+TBAZKZL1M5lPONEbcGQP2zFKEkd7z/dsd7ipDD6v2tKi4jdX99sMF0jJkCEt OH9uma8bUTTo6K+6cWcUbkM05moOdogaO7jYHJ5zI1KPlv7ZniVqYU3ACqKIJDPWZ5v7 guq0Mg/9wzMHCLSsThKRNI2efmgwU/LKgLEw35JFBz+0LPvl3bHu8DJv1ZifXjai05Er ssZyDKDN8a4flEB1ICWtVc6FAM96Bss0Jb+/+cxq4QP+HXyW9miYNkRs6yX6GWGNLaiJ QrzIRZAD2H6nU3q58ghd3190nIWoCHUps8y9uKUNsSql5rbsZd1XBVLEtlP8hn0DSRTd yeIg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20260707; t=1790924701; x=1791529501; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :x-spam-checked-in-group:list-id:mailing-list:precedence :content-type:x-original-authentication-results:x-original-sender :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:x-beenthere:x-gm-message-state:sender:from:to:cc:subject:date :message-id:reply-to:content-type; bh=FpEt696zox+ITlWimkLnHzEIdSBWe6elJYgguk63Ql0=; b=lTHgFyaoxslARUV9wkHFKivpAYq4c7vfosx4J/HBZ9P0prvR1e+Tl8UtAqZ/0X+pQ6 D7loD7nQ6ksWMG0A6o8m8VPX0BNBpiUYQZDqOJBURXzewuYY5o3H7KDfJQn8C1q2Joq/ Ig9zQ6PUHIXyjjizwWCZ558EAxna5aWL5neJHj48n+zNQ1ilrcGRqcYiN2dV7vFlIUy2 k9AIlwmouIuAPkPd7b2gKUSXqb/zEXpPItnr/COL7+XuCGL4oU6MwHNZ4UulH0TnoW8k VhI5Z0vwCaF3PD81NUSf1TDOa63R5iByQLex6kU9w1UD18Lz8gSVCnwcACF2rHms/sZQ 81MQ== Sender: isar-users@googlegroups.com X-Forwarded-Encrypted: i=2; AKwUvBx4KNQYASWC1Xuc+cxJyTSmqj2XHQ4ZWKq4yuW26o8Yu2g/AUuxCTqflxbUV7XytUFWZQfDP8Q=@isar-build.org X-Gm-Message-State: AFq9FYKE82ilOK4XZKebwYiv9i3UY4vxw+7ieNpane50Pn+IjGI4qgoN jB0Su6H+EcN2oRL1H9XhB2Y8QutGux1QVWbznBJtbCePrIwkk64QjAQq X-Received: by 2002:a05:6402:2790:b0:6ac:9a44:d404 with SMTP id 4fb4d7f45d1cf-6af9de99262mr1019184a12.0.1790924700514; Fri, 02 Oct 2026 00:05:00 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com; h="ATskLdcIstwvxsj898iwa6OpTFLHGMs3H87dyuQk4219OYmu/g==" Received: by 2002:a05:6402:24cc:b0:6aa:448f:b889 with SMTP id 4fb4d7f45d1cf-6af9dab6cd6ls521974a12.2.-pod-prod-05-eu; Fri, 02 Oct 2026 00:04:56 -0700 (PDT) X-Received: by 2002:a05:6402:3507:b0:6a7:ea54:38f with SMTP id 4fb4d7f45d1cf-6af9e318369mr1061654a12.32.1790924695778; Fri, 02 Oct 2026 00:04:55 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1790924695; cv=none; d=google.com; s=arc-20260327; b=P5h+h/qeykX8lI7M/fBY1XlsE3k1iW+EG19sAPg5DOyb9ovpFhlI5Psq9l8/U/gRM/ OLnrMfbydCmcchRuEHe+NTs5Q4BFlLKOhAEXOrjwACC1Phl2/wvQgxFPTF2xxlU7Wo+r xobPcwkP6X8QqeLZrJdRyNen0zU4IgSb7dtbaL/EMTEvEm7iCYOJz5T0YXBOw0LIPvDF 87BUT7vxMRtLljWTyVgNMZj/2/QzJH66Kt4IUBAcXIxjlMPcIH/InDbsCqjvS929kk2d ZP378G1MZFDHIGEOqX2ODwkQQo88V9KSdzbUgwDP2o9T0Y1xeHXB9YiBcQTkyA4VwGHe sMUA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20260327; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=DPK7XNfXmH1ik4kBjtPUHNR68d7Zw/Zr6vOXQ1ei14g=; fh=IIbR0JxonYgzdrHDdB9TmZniSh/o1z0HofhfbmLXp8s=; b=ioejjC6g9cCgcFZ07zl/R11/Ttb20cJZB/lDDr+EG2L+na4BEnIlQdpfqQM4OFy/pu agUKNS8I7zIdhUJAJ6nhhJUCCGYpA0HDCoYRGC1WaBLHPS9LOKk1QkrNpSjMmRiHysyB szkvrYV2r2tkTePBICdCs730f1bp8RU6iGdwcgZfvJmGsnjw0BkLiHt1y8xf52w+Gt4r IFL/TC6ftIYUKZNsrP4iPCu4nWdU/95gUBdykG4XQOb5XHX7P6NkWU6CicViKPbgqyVv tmzqc7v2G9/z9xYQ+T93+wt0ibLMguGG8SAtpQy+BYK+oaq/d62vPDokTKd8rBq77oW9 aRTw==; dara=google.com ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of akarpovich@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=akarpovich@ilbers.de Received: from shymkent.ilbers.de (shymkent.ilbers.de. [85.214.156.166]) by gmr-mx.google.com with ESMTPS id 4fb4d7f45d1cf-6af9e269409si29765a12.8.2026.10.02.00.04.55 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 02 Oct 2026 00:04:55 -0700 (PDT) Received-SPF: pass (google.com: domain of akarpovich@ilbers.de designates 85.214.156.166 as permitted sender) client-ip=85.214.156.166; Received: from laptop-0xxx (62.122.174.6.serverel.net [62.122.174.6] (may be forged)) (authenticated bits=0) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPSA id 69274a1h003799 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 2 Oct 2026 09:04:54 +0200 From: akarpovich To: isar-users@googlegroups.com Cc: Uladzimir Bely , Aliaksei Karpovich Subject: [v11 02/12] meta: Add debrepo bbclass handling base-apt prefetching Date: Fri, 2 Oct 2026 09:01:05 +0200 Message-ID: <20261002070335.26526-3-akarpovich@ilbers.de> X-Mailer: git-send-email 2.53.0 In-Reply-To: <20261002070335.26526-1-akarpovich@ilbers.de> References: <20261002070335.26526-1-akarpovich@ilbers.de> MIME-Version: 1.0 X-Spam-Status: No, score=-4.6 required=5.0 tests=DKIMWL_WL_MED,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_EF,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,RCVD_IN_DNSWL_BLOCKED,RCVD_IN_MSPIKE_H2, RCVD_IN_RP_CERTIFIED,RCVD_IN_RP_RNBL,RCVD_IN_RP_SAFE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on shymkent.ilbers.de X-Original-Sender: akarpovich@ilbers.de X-Original-Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of akarpovich@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=akarpovich@ilbers.de Precedence: list Mailing-list: list isar-users@googlegroups.com; contact isar-users+owners@googlegroups.com List-ID: X-Spam-Checked-In-Group: isar-users@googlegroups.com X-Google-Group-Id: 914930254986 List-Post: , List-Help: , List-Archive: , List-Unsubscribe: , X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= From: Uladzimir Bely This class uses 'scripts/debrepo' python script to prefetch given packages or sources to local base-apt repository. Signed-off-by: Uladzimir Bely Signed-off-by: Aliaksei Karpovich --- RECIPE-API-CHANGELOG.md | 10 ++++ doc/user_manual.md | 2 + meta/classes/debrepo.bbclass | 89 ++++++++++++++++++++++++++++++++++++ meta/conf/bitbake.conf | 5 ++ 4 files changed, 106 insertions(+) create mode 100644 meta/classes/debrepo.bbclass diff --git a/RECIPE-API-CHANGELOG.md b/RECIPE-API-CHANGELOG.md index abb67fa8..1951df08 100644 --- a/RECIPE-API-CHANGELOG.md +++ b/RECIPE-API-CHANGELOG.md @@ -1211,3 +1211,13 @@ ROOTFS_FEATURES:remove = "clean-apt-credentials" All `ROOTFS_POSTPROCESS_COMMAND` steps are now executed within the `do_rootfs_install` task to avoid idempotency issues on partial rebuilds. Task that previously had a dependency to `do_rootfs_postprocess` shall now be changed to run after `do_rootfs_install`. + +### "Prefetch" mode for base-apt + +Originally, `base-apt` repo is created only during second build when variable +ISAR_USE_CACHED_BASE_REPO is set. The repo is populated with every package that +took part in the first build and was cached in DL_DIR. + +New ISAR_PREFETCH_BASE_APT variable changes the way `base-apt` is populated. +Packages added to the repo before running any task that need them. Separate +`deb-repo` script is used for populating base-apt repo. diff --git a/doc/user_manual.md b/doc/user_manual.md index a134903d..6db4b8eb 100644 --- a/doc/user_manual.md +++ b/doc/user_manual.md @@ -86,12 +86,14 @@ apt install \ gettext-base \ git \ python3 \ + python3-apt \ quilt \ qemu-user-static \ reprepro \ sudo \ uidmap \ unzip \ + wget \ xz-utils \ git-buildpackage \ pristine-tar \ diff --git a/meta/classes/debrepo.bbclass b/meta/classes/debrepo.bbclass new file mode 100644 index 00000000..79a19144 --- /dev/null +++ b/meta/classes/debrepo.bbclass @@ -0,0 +1,89 @@ +# This software is a part of Isar. +# Copyright (C) 2026 ilbers GmbH +# +# SPDX-License-Identifier: MIT + +# Prefetch to base-apt repo by default +ISAR_PREFETCH_BASE_APT ??= "1" + +DEBREPO_WORKDIR ??= "${DEBREPO_TARGET_DIR}" + +debrepo_update_apt_source_list() { + [ "${ISAR_PREFETCH_BASE_APT}" != "1" ] && return + + chroot_dir=${1} + apt_list=${2} + + flock -x "${REPO_BASE_DIR}/repo.lock" -c " + sudo -E chroot ${chroot_dir} /usr/bin/apt-get update \ + -o Dir::Etc::SourceList=\"sources.list.d/${apt_list}.list\" \ + -o Dir::Etc::SourceParts=\"-\" \ + -o APT::Get::List-Cleanup=\"0\" + " +} + +debrepo_add_packages() { + [ "${ISAR_PREFETCH_BASE_APT}" != "1" ] && return + [ "${ISAR_USE_CACHED_BASE_REPO}" = "1" ] && return + + args="" + if [ "${1}" = "--srcmode" ]; then + args="${args} --srcmode" + shift + fi + + if [ "${1}" = "--isarapt" ]; then + args="${args} --extrarepo=${REPO_ISAR_DIR}/${DISTRO}" + shift + fi + + workdir="${1}" + args="${args} ${2}" + + if [ -n "${GNUPGHOME}" ]; then + export GNUPGHOME="${GNUPGHOME}" + else + export GNUPGHOME="${WORKDIR}/gpghome" + fi + + ${SCRIPTSDIR}/deb-repo \ + --workdir="${workdir}" \ + ${args} +} + +debrepo_parse_dscfile() { + [ "${ISAR_PREFETCH_BASE_APT}" != "1" ] && return + [ "${ISAR_USE_CACHED_BASE_REPO}" = "1" ] && return + + dscfile="${1}" + args="" + + if [ "${ISAR_CROSS_COMPILE}" = "1" ]; then + build_arch=${HOST_ARCH} + if [ "${PACKAGE_ARCH}" != "${build_arch}" ]; then + args="--crossbuild \ + crossbuild-essential-${PACKAGE_ARCH}:${build_arch} \ + dose-distcheck:${build_arch} \ + libc-dev:${PACKAGE_ARCH} \ + libstdc++-dev:${PACKAGE_ARCH} \ + " + fi + fi + + args="${args} --extrarepo=${WORKDIR}/isar-apt/${DISTRO}-${DISTRO_ARCH}/apt/${DISTRO}" + + if [ -n "${GNUPGHOME}" ]; then + export GNUPGHOME="${GNUPGHOME}" + else + export GNUPGHOME="${WORKDIR}/gpghome" + fi + + if [ -n "${DEB_BUILD_PROFILES}" ]; then + export DEB_BUILD_PROFILES="${DEB_BUILD_PROFILES}" + fi + + ${SCRIPTSDIR}/deb-repo \ + --workdir="${DEBREPO_WORKDIR}" \ + --dscfile="${dscfile}" \ + ${args} +} diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf index e69ec3bb..c8503ec6 100644 --- a/meta/conf/bitbake.conf +++ b/meta/conf/bitbake.conf @@ -67,6 +67,11 @@ SDKCHROOT_DIR = "${DEPLOY_DIR_SDKCHROOT}/${BPN}-${DISTRO}-${MACHINE}" CACHE = "${TMPDIR}/cache" KERNEL_FILE ?= "${@ 'vmlinux' if d.getVar('DISTRO_ARCH') in ['mipsel', 'riscv64', 'arm64'] else 'vmlinuz'}" +# debrepo config +DEBREPO_DIR = "${TOPDIR}/debrepo" +DEBREPO_HOST_DIR = "${DEBREPO_DIR}/${HOST_DISTRO}-${HOST_ARCH}_${DISTRO}-${DISTRO_ARCH}" +DEBREPO_TARGET_DIR = "${DEBREPO_DIR}/${DISTRO}-${DISTRO_ARCH}" + MACHINEOVERRIDES ?= "${MACHINE}" DISTROOVERRIDES ?= "${DISTRO}" OVERRIDES = "${PACKAGE_ARCH}:${MACHINEOVERRIDES}:${DISTROOVERRIDES}:${BASE_DISTRO_CODENAME}::${BASE_DISTRO}:${ISAR_CHROOT_MODE}:forcevariable" From patchwork Fri Oct 2 07:01:06 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: akarpovich X-Patchwork-Id: 5513 Return-Path: Received: from shymkent.ilbers.de ([unix socket]) by shymkent (Cyrus 2.5.10-Debian-2.5.10-3+deb9u2) with LMTPA; Fri, 02 Oct 2026 09:05:05 +0200 X-Sieve: CMU Sieve 2.4 Received: from mail-wm1-f55.google.com (mail-wm1-f55.google.com [209.85.128.55]) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPS id 692754RH003842 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Fri, 2 Oct 2026 09:05:04 +0200 Received: by mail-wm1-f55.google.com with SMTP id 5b1f17b1804b1-4a0228f4a1bsf9712255e9.1 for ; Fri, 02 Oct 2026 00:05:04 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1790924699; cv=pass; d=google.com; s=arc-20260327; b=qeuRmAZikFXvu4Ry51S9A0HfLl0jL5+Y+dq/RKqj0vu7Rc2Bva25acZF2ie3l46Kqd ADzsjprgEgpodAkjmf4zuDgfG+cQFX9Uigy5Hiusfm/o4QUp+SDVWBpwtAYy+EWDIAZJ NARSyyUBqLaBek0HT7xdsJnxzJKFZzeSswLKDz1nUNNhaI7DrmnejSozIBCiqIs5s2Q+ q2vhI1+KS+CUaNlMmeQofjRUczyL+RgdoRNKSi6KSe7VjR+N//lHbvQhprJun4WDyLmj eRXbwyZyJb9sqWfzLfqpa9jI+1oW8eYUmg/kbfd3l3vYjLi6xfKpeIYZ6y1zJFstBV0l 2Jew== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20260327; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:dkim-signature; bh=OV0vr+CFWjOG1RHQxC6X2ep2X64E/OK2BZ/Qne/wYa4=; fh=nIeSBR+1KqFziKnwzM+Aaaqyk0daFcEn0zMywM2qTGM=; b=n5bK635CbxDkSdsSbejixWhmmvyegNw0GpodhzYfTHv/mjLR5hlRExnKFK4+/OjfWY MOCoNYIa6cLOz/dd3Nxu1sIqYKOfoAe79xSkmExN2nSMRlJJlc+sC0xaijIRHSDP08YR fESiIukTE/LZCGVr50SgWD0BQ86nJ6n5KVcJcAi9CYjuiVCVtH2jkBXzWf2NGAjSbrs3 2ZshaZT0364oKOYdNUMjmZf9d8ZARxcHDqQUry4YU+5CuXzh6GWAHejAxDGjvp75qQGB I05ktXoYSY3h6Ag6d95ngNrMuAAoneGJ8b+g2fhbLf3vXov/Z1ZQA2nViOOXGL3WSvvq 02NQ==; darn=isar-build.org ARC-Authentication-Results: i=2; gmr-mx.google.com; spf=pass (google.com: domain of akarpovich@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=akarpovich@ilbers.de DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlegroups.com; s=20251104; t=1790924699; x=1791529499; darn=isar-build.org; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:content-type :x-original-authentication-results:x-original-sender:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from:sender :from:to:cc:subject:date:message-id:reply-to:content-type; bh=OV0vr+CFWjOG1RHQxC6X2ep2X64E/OK2BZ/Qne/wYa4=; b=hvsyJmRln8Q5UKwxqyME+t73SSUL7Eh9ysdZqVo8N5ib4iiNTTaHacWc9WdNVNZYTE 36SzjhgctIOBhFx+XIG6MV23/W+9cw9pJMuu9JbhRUcaJvaQESuOGPncatyvmJQ185O0 OFZqTMRABHsJ0yEDRyyQbL/uvkZXGmDv5Al3F0SxZYDzvMwDyCk71y+OcFMa+ic4TgEL nkKUX4cuHsMbp2CQ2C2Bsl5XNMyD5rps3r/HqLL9f1MLJUX9N9NqgwTi9lRe2OCr2Vsv mkpzqJ0btFPrhEONXh4dimnE1nZkMvWYaoTKpT08sdLNAby60uMrYgbz5pKW53CAwnGD 5jMg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20260707; t=1790924699; x=1791529499; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :x-spam-checked-in-group:list-id:mailing-list:precedence :content-type:x-original-authentication-results:x-original-sender :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:x-beenthere:x-gm-message-state:sender:from:to:cc:subject:date :message-id:reply-to:content-type; bh=OV0vr+CFWjOG1RHQxC6X2ep2X64E/OK2BZ/Qne/wYa4=; b=W7yzPO4CpaZ0UPe9cZ43qM29g5oWvSlfMfm+dVmXDE7GXvl5Y8sQFBKGBWEYy7lc2j obnkYGv5y3tFdeMwLsbK1f4CFQt3yzMjuQ9PZ326cUQimE9fcVaJVOR4fqo6cl6QvI5Q xpg0SBESLRZC1awv9o/pscBrpSs1lQQHTqHmLBNmZ0uYexdsJ1R36AaD8bK+UC8WB/u1 3CJvPjULdqqsxUOm/JPv4eKn/b7TivXBE9kUmPB/wepf23ad/fXTFdaoDru4qTbXBp0H nNpYmh3XykV0FSXMh8ryT/N3IDedOMhHIyLgt/BKfN6p8q9X8MUS4rOeplrg8FCvjU8d P/ZA== Sender: isar-users@googlegroups.com X-Forwarded-Encrypted: i=2; AKwUvBy3PmZmOJzvXG9B0zYRyNeXxZzwKDryY0Lmk2G5K+SYJ6Ps/HPQEwxKHiOXuhoefxv21z/TVdk=@isar-build.org X-Gm-Message-State: AFuF++krCtNtdcnJ2BB6kXTTrrQ0OlpXycySg3RLgXSf85IAV192Tdtb xJyIYRSmrMtfaZWaBRBGBk/i86kWAz3cIhYiaqgqL9YPr+pf6Bvq/zof X-Received: by 2002:a05:600c:4ec6:b0:49d:1cc7:5ec3 with SMTP id 5b1f17b1804b1-4a02756637cmr32240465e9.9.1790924699091; Fri, 02 Oct 2026 00:04:59 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com; h="ATskLdcyqWgdXGJPJoV5E4EViODVLhADolL1q+ZDsX70IQhVcg==" Received: by 2002:a05:600c:37cc:b0:49f:fd51:d81 with SMTP id 5b1f17b1804b1-4a028522102ls5255435e9.0.-pod-prod-02-eu; Fri, 02 Oct 2026 00:04:57 -0700 (PDT) X-Received: by 2002:a05:600c:993:b0:4a0:1fb9:2f51 with SMTP id 5b1f17b1804b1-4a027579559mr22255855e9.14.1790924696937; Fri, 02 Oct 2026 00:04:56 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1790924696; cv=none; d=google.com; s=arc-20260327; b=AMidUrO3AzAlPF9Ec4ikOnUsUO+/sXfVbNhUj1DKOItSHuU/dM7MkAH49+xKnNNEqP cdafq3vTx84EgEOgYVzhrdq1eLaBnC6a+Mh8fMnVg8eDBZGl7jRvD6ZwEncGsz6MRfHu 1y21TBTMU4rUJf6XH4JITrmoRa7m7n+gziPqDHQb3H5Zd8AqiB6vVSCn8Bt0o1mBkr99 AceMmuH35poAI7X+9f1A0vglDzoFQ15Kgh+HgBVGi3qXJKTqFrn1TZTvyeuODp+JmmgF Q+2aSteYWxo+b0+R2lU1tw2IGw9NvDKb6UbGjxt9//ZBusQYBmWz8zqd0tRnqGmQXuIs G4WA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20260327; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=mCdlnoF9LNspukmPnqCjY9Qrscd7la6o5+9YbL0E/cM=; fh=8xvFQ9N6ENvazHnylkO2sbgzUV6bRnZQvNiaC5bb+x4=; b=YrwPW9jYc75t3EhLjLghbXHa8YTLeoQOwV3O6jVZ7SKTLG+njJijLgIpTZiijyW73j WA3L/UqKbjpZEhJB/sYC0+aUH/yaTMC1bmyiVwo0XxsFV5ZS3PGvxi98/FS+XCTBbjUh YCDRV/xFwMFq57l+p0NOt0CpIbM93lOcHNbjr2Hx/NnbOQss4WIb3FsonsWCFtLdNFoE Xq4Pv7Ys+7SbaxSu8/+8imDvnt1z9Wxp1guinwwI3Mk7o2phEvYxglcIxyymUzjW5I+C NgZmVXtmB+rD7phryBy7yk3VSWodCOtaIvPdDWGEYHx6w00dXC5MdAfHgSIhQDbi/tW4 9iVw==; dara=google.com ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of akarpovich@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=akarpovich@ilbers.de Received: from shymkent.ilbers.de (shymkent.ilbers.de. [85.214.156.166]) by gmr-mx.google.com with ESMTPS id 5b1f17b1804b1-4a03ff09c09si158245e9.4.2026.10.02.00.04.56 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 02 Oct 2026 00:04:56 -0700 (PDT) Received-SPF: pass (google.com: domain of akarpovich@ilbers.de designates 85.214.156.166 as permitted sender) client-ip=85.214.156.166; Received: from laptop-0xxx (62.122.174.6.serverel.net [62.122.174.6] (may be forged)) (authenticated bits=0) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPSA id 69274a1i003799 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 2 Oct 2026 09:04:55 +0200 From: akarpovich To: isar-users@googlegroups.com Cc: Uladzimir Bely , Aliaksei Karpovich , Aliaksei Laurynovich Subject: [v11 03/12] meta: Always use base-apt repo in local mode Date: Fri, 2 Oct 2026 09:01:06 +0200 Message-ID: <20261002070335.26526-4-akarpovich@ilbers.de> X-Mailer: git-send-email 2.53.0 In-Reply-To: <20261002070335.26526-1-akarpovich@ilbers.de> References: <20261002070335.26526-1-akarpovich@ilbers.de> MIME-Version: 1.0 X-Spam-Status: No, score=-4.6 required=5.0 tests=DKIMWL_WL_MED,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_EF,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,RCVD_IN_DNSWL_BLOCKED,RCVD_IN_MSPIKE_H2, RCVD_IN_RP_CERTIFIED,RCVD_IN_RP_RNBL,RCVD_IN_RP_SAFE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on shymkent.ilbers.de X-Original-Sender: akarpovich@ilbers.de X-Original-Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of akarpovich@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=akarpovich@ilbers.de Precedence: list Mailing-list: list isar-users@googlegroups.com; contact isar-users+owners@googlegroups.com List-ID: X-Spam-Checked-In-Group: isar-users@googlegroups.com X-Google-Group-Id: 914930254986 List-Post: , List-Help: , List-Archive: , List-Unsubscribe: , X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= From: Uladzimir Bely This means only local URLs in apt sources.list* are present during the build. Any installation of packages is done from local base-apt. So, base-apt should be always mounted in *_do_mounts since now. Signed-off-by: Uladzimir Bely Signed-off-by: Aliaksei Karpovich Signed-off-by: Aliaksei Laurynovich --- meta/classes-recipe/bootstrap.bbclass | 6 ++++-- meta/classes-recipe/rootfs.bbclass | 3 ++- meta/classes-recipe/sbuild.bbclass | 3 ++- .../isar-mmdebstrap/isar-mmdebstrap.inc | 17 ++++++++++----- meta/recipes-devtools/base-apt/base-apt.bb | 21 ++++++++++++------- 5 files changed, 34 insertions(+), 16 deletions(-) diff --git a/meta/classes-recipe/bootstrap.bbclass b/meta/classes-recipe/bootstrap.bbclass index c1a59fd2..f853a932 100644 --- a/meta/classes-recipe/bootstrap.bbclass +++ b/meta/classes-recipe/bootstrap.bbclass @@ -39,7 +39,8 @@ python () { # installation afterwards. However, bootstrap will include the key into # the rootfs automatically thus the right place is distro_bootstrap_keys. - if bb.utils.to_boolean(d.getVar('ISAR_USE_CACHED_BASE_REPO')): + if bb.utils.to_boolean(d.getVar('ISAR_PREFETCH_BASE_APT')) or \ + bb.utils.to_boolean(d.getVar('ISAR_USE_CACHED_BASE_REPO')) : own_pub_key = d.getVar("BASE_REPO_KEY") if own_pub_key: distro_bootstrap_keys += own_pub_key.split() @@ -121,7 +122,8 @@ def get_apt_source_mirror(d, aptsources_entry_list): # this is executed during parsing. No error checking possible use_snapshot = bb.utils.to_boolean(d.getVar('ISAR_USE_APT_SNAPSHOT')) snapshot_mirror = d.getVar('DISTRO_APT_SNAPSHOT_PREMIRROR') - if bb.utils.to_boolean(d.getVar('ISAR_USE_CACHED_BASE_REPO')): + if bb.utils.to_boolean(d.getVar('ISAR_PREFETCH_BASE_APT')) or \ + bb.utils.to_boolean(d.getVar('ISAR_USE_CACHED_BASE_REPO')) : premirrors = "\S* file://${REPO_BASE_DIR}/${BOOTSTRAP_BASE_DISTRO}\n" elif use_snapshot and snapshot_mirror: premirrors = snapshot_mirror diff --git a/meta/classes-recipe/rootfs.bbclass b/meta/classes-recipe/rootfs.bbclass index bbaafa01..6aa3a103 100644 --- a/meta/classes-recipe/rootfs.bbclass +++ b/meta/classes-recipe/rootfs.bbclass @@ -75,7 +75,8 @@ ROOTFS_MOUNTS ??= "${REPO_ISAR_DIR}/${DISTRO}:/isar-apt ${WORKDIR}:/isar-work" python () { mounts = d.getVar('ROOTFS_MOUNTS', False) - if bb.utils.to_boolean(d.getVar('ISAR_USE_CACHED_BASE_REPO')) and not ':/base-apt' in mounts: + if (bb.utils.to_boolean(d.getVar('ISAR_USE_CACHED_BASE_REPO')) or + bb.utils.to_boolean(d.getVar('ISAR_PREFETCH_BASE_APT'))) and not ':/base-apt' in mounts: base_apt = '{}:/base-apt'.format(d.getVar('REPO_BASE_DIR')) d.setVar('ROOTFS_MOUNTS', '{} {}'.format(mounts, base_apt)) } diff --git a/meta/classes-recipe/sbuild.bbclass b/meta/classes-recipe/sbuild.bbclass index 6db29251..29a8dd71 100644 --- a/meta/classes-recipe/sbuild.bbclass +++ b/meta/classes-recipe/sbuild.bbclass @@ -44,7 +44,8 @@ EOF cp -rf "${SCHROOT_CONF}/sbuild" "${SBUILD_CONF_DIR}" sbuild_fstab="${SBUILD_CONF_DIR}/fstab" - if [ "${ISAR_USE_CACHED_BASE_REPO}" = "1" ]; then + if [ "${ISAR_USE_CACHED_BASE_REPO}" = "1" ] || \ + [ "${ISAR_PREFETCH_BASE_APT}" = "1" ]; then fstab_baseapt="${REPO_BASE_DIR} /base-apt none rw,bind,private 0 0" grep -qxF "${fstab_baseapt}" ${sbuild_fstab} || echo "${fstab_baseapt}" >> ${sbuild_fstab} fi diff --git a/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc b/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc index 6135da4f..9423ee72 100644 --- a/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc +++ b/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc @@ -157,16 +157,21 @@ do_bootstrap() { fi E="${@ isar_export_proxies(d)}" - if [ "${ISAR_USE_CACHED_BASE_REPO}" = "1" ]; then + if [ "${ISAR_PREFETCH_BASE_APT}" = "1" ] || \ + [ "${ISAR_USE_CACHED_BASE_REPO}" = "1" ]; then base_apt_tmp="$(mktemp -d /tmp/isar-base-aptXXXXXXXXXX)" bootstrap_list="${WORKDIR}/sources.list.d/base-apt.list" line="copy://$base_apt_tmp/${BOOTSTRAP_BASE_DISTRO} ${BASE_DISTRO_CODENAME} main" - if [ -z "${BASE_REPO_KEY}" ]; then + if [ -z "${BASE_REPO_KEY}" ] || \ + [ "${ISAR_PREFETCH_BASE_APT}" = "1" ]; then + line="[trusted=yes] ${line}" fi echo "deb ${line}" > "${WORKDIR}/sources.list.d/base-apt.list" line="copy://$base_apt_tmp/${BASE_DISTRO} ${BASE_DISTRO_CODENAME} main" - if [ -z "${BASE_REPO_KEY}" ]; then + if [ -z "${BASE_REPO_KEY}" ] || \ + [ "${ISAR_PREFETCH_BASE_APT}" = "1" ]; then + line="[trusted=yes] ${line}" fi echo "deb-src ${line}" >> "${WORKDIR}/sources.list.d/base-apt.list" @@ -187,9 +192,11 @@ do_bootstrap() { \$1/etc/apt/sources.list.d/*.list && \ mkdir -p \$1/base-apt && \ mount -o bind,private '${REPO_BASE_DIR}' \$1/base-apt && \ - chroot \$1 apt-get update -y \ + chroot \$1 mv /etc/apt/sources.list.d/0000base-apt.list \ + /etc/apt/sources.list.d/base-apt.list + flock -x ${REPO_BASE_DIR}/repo.lock -c \"chroot \$1 apt-get update -y \ -o APT::Update::Error-Mode=any \ - ${@'-o APT::Sandbox::User=root' if d.getVar('ISAR_CHROOT_MODE') == 'unshare' else ''} && \ + ${@'-o APT::Sandbox::User=root' if d.getVar('ISAR_CHROOT_MODE') == 'unshare' else ''} \" && \ chroot \$1 apt-get install -y dpkg && \ umount \$1/base-apt && \ umount \$1/$base_apt_tmp && \ diff --git a/meta/recipes-devtools/base-apt/base-apt.bb b/meta/recipes-devtools/base-apt/base-apt.bb index 06b1f6c8..05ef9741 100644 --- a/meta/recipes-devtools/base-apt/base-apt.bb +++ b/meta/recipes-devtools/base-apt/base-apt.bb @@ -59,9 +59,12 @@ repo() { "${BASE_DISTRO_CODENAME}" \ "${WORKDIR}/distributions.in" \ "${KEYFILES}" - populate_base_apt "${BASE_DISTRO}" - repo_sanity_test "${REPO_BASE_DIR}"/"${BASE_DISTRO}" \ - "${REPO_BASE_DB_DIR}"/"${BASE_DISTRO}" + if [ "${ISAR_USE_CACHED_BASE_REPO}" = "1" ] && \ + [ "${ISAR_PREFETCH_BASE_APT}" != "1" ]; then + populate_base_apt "${BASE_DISTRO}" + repo_sanity_test "${REPO_BASE_DIR}"/"${BASE_DISTRO}" \ + "${REPO_BASE_DB_DIR}"/"${BASE_DISTRO}" + fi if [ '${BASE_DISTRO}' != '${HOST_BASE_DISTRO}' ]; then repo_create "${REPO_BASE_DIR}"/"${HOST_BASE_DISTRO}" \ @@ -69,14 +72,18 @@ repo() { "${BASE_DISTRO_CODENAME}" \ "${WORKDIR}/distributions.in" \ "${KEYFILES}" - populate_base_apt "${HOST_BASE_DISTRO}" - repo_sanity_test "${REPO_BASE_DIR}"/"${HOST_BASE_DISTRO}" \ - "${REPO_BASE_DB_DIR}"/"${HOST_BASE_DISTRO}" + if [ "${ISAR_USE_CACHED_BASE_REPO}" = "1" ] && \ + [ "${ISAR_PREFETCH_BASE_APT}" != "1" ]; then + populate_base_apt "${HOST_BASE_DISTRO}" + repo_sanity_test "${REPO_BASE_DIR}"/"${HOST_BASE_DISTRO}" \ + "${REPO_BASE_DB_DIR}"/"${HOST_BASE_DISTRO}" + fi fi } python do_cache() { - if not bb.utils.to_boolean(d.getVar('ISAR_USE_CACHED_BASE_REPO')): + if not bb.utils.to_boolean(d.getVar('ISAR_PREFETCH_BASE_APT')) and \ + not bb.utils.to_boolean(d.getVar('ISAR_USE_CACHED_BASE_REPO')): return 0 for key in d.getVar('BASE_REPO_KEY').split(): From patchwork Fri Oct 2 07:01:07 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: akarpovich X-Patchwork-Id: 5515 Return-Path: Received: from shymkent.ilbers.de ([unix socket]) by shymkent (Cyrus 2.5.10-Debian-2.5.10-3+deb9u2) with LMTPA; Fri, 02 Oct 2026 09:05:08 +0200 X-Sieve: CMU Sieve 2.4 Received: from mail-lj1-f184.google.com (mail-lj1-f184.google.com [209.85.208.184]) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPS id 692756fa003873 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Fri, 2 Oct 2026 09:05:06 +0200 Received: by mail-lj1-f184.google.com with SMTP id 38308e7fff4ca-3a7fd50de8dsf6943861fa.1 for ; Fri, 02 Oct 2026 00:05:06 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1790924701; cv=pass; d=google.com; s=arc-20260327; b=PBWJoDo2an2LyV+1zEmiStMdxfbrU5ZeLa/bIPTRClYiGTSamlSYc+cIv7vEMGrvOA HTPioKhxBO4Y1OgqGYKbcjmT81aXGi0FCoK32/2CRLyRp6weYIn/swg3rRvtQHs4Z8GF jV7oJQc39qjIkpPXQgpvt/G5oK+YCuG8kmT2SY+C+G1hNwOUParQUUSJRT4SUtLAY6T+ hvGjrdRuHa/AUi1sp7LuhR5B8wV4cO24dqExxLwjFhTz64z18Wltb2xo+E2ufYP11EOS 49aoCYGQP7VEk7EcTs2/Q408PF4itiuKNkOLgguiwtpN/vKp9PT3tibnGIVNcWVg5vMp zkTg== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20260327; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:dkim-signature; bh=iwQf0aUDJla2BrBRF9q8VTbfCrkoHSujW9JZmx9X4/0=; fh=0KtSdw0s/0z2Hf1TAC/USyys1Qq3K0jbN2K+2i8x4kc=; b=LVXJFJaoXZGsjpZHRvVf85nTSpe1ntwYtNppXucM5EmRJiuKYTr3c7XwsD/inFF5fk 7ivbbMbC6Rc4zj2yrwt1ftdcm0OpYXHPldxsDViGJ7SeFvUSHzjT6Ki5M2Pwa0FJzXTU NAuwmZ4lsfWJcPGXEWi+NYps6lBnM+YqpYNa/XTmDJGqBv5kfRVFSnoch1X6R3e4By0w RHMmd1zOrzb1eHFLiWg+BG8ffjS7qYklLRUMMHgAXEMn+QcVkqNKDgqd23A8qMqnoUEW INbAvQM54FglFG8JL0CVvNFjIdIdV9UoNZ50gtEOxOOza7TbnMxlMVDzzN1YWg6Ux0GF df7Q==; darn=isar-build.org ARC-Authentication-Results: i=2; gmr-mx.google.com; spf=pass (google.com: domain of akarpovich@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=akarpovich@ilbers.de DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlegroups.com; s=20251104; t=1790924701; x=1791529501; darn=isar-build.org; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:content-type :x-original-authentication-results:x-original-sender:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from:sender :from:to:cc:subject:date:message-id:reply-to:content-type; bh=iwQf0aUDJla2BrBRF9q8VTbfCrkoHSujW9JZmx9X4/0=; b=XJD1TC3TpM9hsESMRWixvBa9Y8VTPHieudk6m9kHYFNkSvYYa/y8NrJNOgGl6x4OLQ p2Jz12jyMliVkvvPGrVSPoOACPOPVXfCeDDmum5jTCQ2bshGCBUCvJxur73T1SLH4DhI z/NkjXAORyz6dMNRV5HyvACJitq1XVZyDGRR4S10dzV/uDL36agyoFBwnwgEHjzrxWT8 2fT9umgo/ugigJFKf1tLduXG0ULtZnhsmmSkSfS1nS/PmDY7cmyF+jxYPkOe9aDlo9W1 34JSb0BptuFsPzSkCOuj7qp1E9d3Y6UCf2fkUbTLNB1kytefxBvLwbKGe9J/9/JjxtVo bFeQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20260707; t=1790924701; x=1791529501; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :x-spam-checked-in-group:list-id:mailing-list:precedence :content-type:x-original-authentication-results:x-original-sender :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:x-beenthere:x-gm-message-state:sender:from:to:cc:subject:date :message-id:reply-to:content-type; bh=iwQf0aUDJla2BrBRF9q8VTbfCrkoHSujW9JZmx9X4/0=; b=0xWp1ABlak80UQxGW3w7/p1us5wHPzb64cy1b4KhFaTOgFjAzi/5StWhD6vQ5oMO60 6n40VVZjzCRZdOBIpURJui70No4nAZ2GvK0bcZTuWmUSEnIfu8RBRvt3TOW8S0ppG5GX zFPVLIdGOQtoA3vzETsoOrIdAIxbDeXxJ7I6cauls/3Ipb0mFIMJh2HE7L46ooL8WnYn uxtFY1Np07Uj+PTtd+mUDp/1Pqx3sG7faf/+fPYGv0mXcuORLdqhFVIlkOeBcBxqTZ6o wloKH/wQYRMo/MKO/ycCNGcQZmuyVrrifmDqdo3se7/YbsCpzYjS+fLtR/zAKIsp/AOs Gwpw== Sender: isar-users@googlegroups.com X-Forwarded-Encrypted: i=2; AKwUvBxRDzR8kuQow/DGG2u+e16B3iBGOSjAomQnfzey3r8zfUSBMRVOuQs4sfJLxFHExB26SUTQGlc=@isar-build.org X-Gm-Message-State: AFq9FYJ2CMZfxf22U9T5P59LHmqLUnJ0aBeqClFOjXXewrX63rIlWF60 MmkzZGAfJSdXF4AkFL/uOnvLsrpMDg7pB0BMrmdcB3raeCXpudl6LWhi X-Received: by 2002:a05:651c:2121:b0:3a8:7b84:be9b with SMTP id 38308e7fff4ca-3a8877b73a8mr4024701fa.5.1790924700642; Fri, 02 Oct 2026 00:05:00 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com; h="ATskLdcwpLNaLWm1BOA+fxG/DJ7SHIrBSdIPpJRhv4SlY/MyUA==" Received: by 2002:a05:651c:2343:20b0:3a2:c29b:9103 with SMTP id 38308e7fff4ca-3a8a8659059ls614541fa.1.-pod-prod-01-eu; Fri, 02 Oct 2026 00:04:58 -0700 (PDT) X-Received: by 2002:a2e:a58d:0:b0:3a9:6ebd:5ad3 with SMTP id 38308e7fff4ca-3a96ebd5d9dmr990801fa.6.1790924698158; Fri, 02 Oct 2026 00:04:58 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1790924698; cv=none; d=google.com; s=arc-20260327; b=UAz/JFYZ6+SWexMN0FUHT26OUIyLCAxZaunP46nk3x1Y1IKRg6OLvjEs7BkA4z7XEw w/ynUybx+5XaNaMunGnd1sT1sLbmrfZ1bbNBz2guoN1WiibJzL9yk1AdJP0oG7nDOf1C PP+HOGkDhdD2SvyHvv5yk8VIUk1DewPBlyiJlpM2W45LxUkP+BXL4SHHl5+Kir2nxipd cNd+td7RnB9bw6c9yzP7j3cUc1qquZW7Pjjtslo2J1HbYRhCwPeZtv2qlizy0nM+81fF 7P6XkwlZ93Os2MmaBBgrIlK8k8m+2Z1LGmH8u5ASCDt/ngEEkQ6iJWklqBCtnmHBybwm ehvw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20260327; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=q8x5tKfd44tXVgXzz/BPxTximXki/DZXYrr5OBDTACA=; fh=IIbR0JxonYgzdrHDdB9TmZniSh/o1z0HofhfbmLXp8s=; b=UhCHmZB+FGl/yXSa5eyRGdoAdJ5UCDBn0Tucp3bx9VwjhazMgVd2nx4cWCDeVUkC+o Zcw4mhhlWMSLOdbzxlyXzxnk9GGU9Hjd3Yhl1kF+5ENWCoOrZ+q7iYlXpG/33oxcEqkI HhJgHBP80Cw6spgghd5CMo5uHnUW4bFUczUBXq8x8vU1utbPeV3IgOYSkzp1HcTBt/CJ RBwp5qXCNWP12q18wwZTJCmtKHoy4YTqOR/3LjtD1VXLyPbe94fD2eek+sYrQEhTG+5o zTTvWESLArjXc8Latvzfkx6CerxsX4KF0FjZzlaRZMBvcnkrOy2KsJFh1mIGbzl31E00 aFqg==; dara=google.com ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of akarpovich@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=akarpovich@ilbers.de Received: from shymkent.ilbers.de (shymkent.ilbers.de. [85.214.156.166]) by gmr-mx.google.com with ESMTPS id 38308e7fff4ca-3a87e348be0si309551fa.9.2026.10.02.00.04.58 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 02 Oct 2026 00:04:58 -0700 (PDT) Received-SPF: pass (google.com: domain of akarpovich@ilbers.de designates 85.214.156.166 as permitted sender) client-ip=85.214.156.166; Received: from laptop-0xxx (62.122.174.6.serverel.net [62.122.174.6] (may be forged)) (authenticated bits=0) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPSA id 69274a1j003799 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 2 Oct 2026 09:04:56 +0200 From: akarpovich To: isar-users@googlegroups.com Cc: Uladzimir Bely , Aliaksei Karpovich Subject: [v11 04/12] meta: Use cached base-apt repo to bootstrap Date: Fri, 2 Oct 2026 09:01:07 +0200 Message-ID: <20261002070335.26526-5-akarpovich@ilbers.de> X-Mailer: git-send-email 2.53.0 In-Reply-To: <20261002070335.26526-1-akarpovich@ilbers.de> References: <20261002070335.26526-1-akarpovich@ilbers.de> MIME-Version: 1.0 X-Spam-Status: No, score=-4.6 required=5.0 tests=DKIMWL_WL_MED,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_EF,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,RCVD_IN_DNSWL_BLOCKED,RCVD_IN_MSPIKE_H2, RCVD_IN_RP_CERTIFIED,RCVD_IN_RP_RNBL,RCVD_IN_RP_SAFE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on shymkent.ilbers.de X-Original-Sender: akarpovich@ilbers.de X-Original-Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of akarpovich@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=akarpovich@ilbers.de Precedence: list Mailing-list: list isar-users@googlegroups.com; contact isar-users+owners@googlegroups.com List-ID: X-Spam-Checked-In-Group: isar-users@googlegroups.com X-Google-Group-Id: 914930254986 List-Post: , List-Help: , List-Archive: , List-Unsubscribe: , X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= From: Uladzimir Bely This patch makes local base-apt repo to be created before bootstrap task. So, bootstrap is then done from it. The required packages are downloaded via python-apt and reprepro creates debian-like repository from .deb files. For debian targets host keyring is used while ubuntu/raspbian targets use keys specified by DISTRO_BOOTSTRAP_KEYS variable. The goal is have workable base-apt repo before first build completed. Signed-off-by: Uladzimir Bely Signed-off-by: Aliaksei Karpovich --- meta/classes-recipe/bootstrap.bbclass | 44 +++++++++++++++++++ .../isar-mmdebstrap/isar-mmdebstrap-host.bb | 2 + .../isar-mmdebstrap/isar-mmdebstrap.inc | 2 + 3 files changed, 48 insertions(+) diff --git a/meta/classes-recipe/bootstrap.bbclass b/meta/classes-recipe/bootstrap.bbclass index f853a932..10cb23be 100644 --- a/meta/classes-recipe/bootstrap.bbclass +++ b/meta/classes-recipe/bootstrap.bbclass @@ -16,6 +16,7 @@ BOOTSTRAP_FOR_HOST ?= "0" APTPREFS = "${WORKDIR}/apt-preferences" APTSRCS = "${WORKDIR}/apt-sources" +APTSRCS_INIT = "${WORKDIR}/apt-sources-init" DISTRO_BOOTSTRAP_KEYFILES = "" THIRD_PARTY_APT_KEYFILES = "" DISTRO_BOOTSTRAP_KEYS ?= "" @@ -223,8 +224,51 @@ python do_apt_config_prepare() { aggregate_files(d, apt_preferences_list, apt_preferences_out) apt_sources_out = d.getVar("APTSRCS") + apt_sources_init_out = d.getVar("APTSRCS_INIT") apt_sources_list = get_aptsources_list(d) + aggregate_files(d, apt_sources_list, apt_sources_init_out) aggregate_aptsources_list(d, apt_sources_list, apt_sources_out) } addtask apt_config_prepare before do_bootstrap after do_unpack + +inherit debrepo + +debrepo_bootstrap_prepare() { + [ "${ISAR_PREFETCH_BASE_APT}" != "1" ] && return + [ "${ISAR_USE_CACHED_BASE_REPO}" = "1" ] && return + + debrepo_args="" + if [ "${BASE_DISTRO}" != "debian" ]; then + if [ "${BASE_DISTRO}" != "raspbian" ] && [ "${BASE_DISTRO}" != "raspios" ] || [ "${BOOTSTRAP_FOR_HOST}" = "0" ]; then + debrepo_args="$debrepo_args --keydir=${WORKDIR}" + fi + else + if [ "${BASE_DISTRO_CODENAME}" = "sid" ]; then + debrepo_args="$debrepo_args --keydir=${WORKDIR}" + fi + fi + if [ "${ISAR_ENABLE_COMPAT_ARCH}" = "1" ]; then + debrepo_args="$debrepo_args --compatarch=${COMPAT_DISTRO_ARCH}" + fi + + if [ "${BOOTSTRAP_FOR_HOST}" = "1" ]; then + debrepo_args="$debrepo_args --crossarch=${DISTRO_ARCH}" + fi + + if [ -n "${GNUPGHOME}" ]; then + export GNUPGHOME="${GNUPGHOME}" + fi + + ${SCRIPTSDIR}/deb-repo --init \ + --workdir="${DEBREPO_WORKDIR}" \ + --aptsrcsfile="${APTSRCS_INIT}" \ + --repodir="${REPO_BASE_DIR}" \ + --repodbdir="${REPO_BASE_DB_DIR}" \ + --mirror="${@get_distro_source(d)}" \ + --arch="${BOOTSTRAP_DISTRO_ARCH}" \ + --distro="${BOOTSTRAP_BASE_DISTRO}" \ + --codename="${BASE_DISTRO_CODENAME}" \ + ${debrepo_args} \ + ${DISTRO_BOOTSTRAP_BASE_PACKAGES} +} diff --git a/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap-host.bb b/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap-host.bb index fa4b76a6..52563d33 100644 --- a/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap-host.bb +++ b/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap-host.bb @@ -17,3 +17,5 @@ require isar-mmdebstrap.inc HOST_DISTRO_BOOTSTRAP_KEYS ?= "" DISTRO_BOOTSTRAP_KEYS = "${HOST_DISTRO_BOOTSTRAP_KEYS}" + +DEBREPO_WORKDIR = "${DEBREPO_HOST_DIR}" diff --git a/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc b/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc index 9423ee72..01ec56e7 100644 --- a/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc +++ b/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc @@ -151,6 +151,8 @@ do_bootstrap() { fi fi bootstrap_args="--verbose --variant=minbase --include=${@','.join(d.getVar('DISTRO_BOOTSTRAP_BASE_PACKAGES').split())}" + debrepo_bootstrap_prepare + if [ -f "${DISTRO_BOOTSTRAP_KEYRING}" ]; then bootstrap_args="$bootstrap_args --keyring=${DISTRO_BOOTSTRAP_KEYRING}" cp "${DISTRO_BOOTSTRAP_KEYRING}" "${WORKDIR}/trusted.gpg.d/" From patchwork Fri Oct 2 07:01:08 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: akarpovich X-Patchwork-Id: 5516 Return-Path: Received: from shymkent.ilbers.de ([unix socket]) by shymkent (Cyrus 2.5.10-Debian-2.5.10-3+deb9u2) with LMTPA; Fri, 02 Oct 2026 09:05:08 +0200 X-Sieve: CMU Sieve 2.4 Received: from mail-wr1-f63.google.com (mail-wr1-f63.google.com [209.85.221.63]) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPS id 692757oo003902 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Fri, 2 Oct 2026 09:05:07 +0200 Received: by mail-wr1-f63.google.com with SMTP id ffacd0b85a97d-48af3e230b9sf3065841f8f.0 for ; Fri, 02 Oct 2026 00:05:07 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1790924701; cv=pass; d=google.com; s=arc-20260327; b=m9phLZy+Vvq6tJg081am7z4AcJEOXpomgCXX/1cnuevEtl8g2+/5ynslN+Jj0D642x sz+TM6tctVm7/q0aEaWdm5VEMYermnRzEfLcdvYBHf5tqvhYxAxNuDGH0vMKPwaX7o1v 8jjdLKY9HxqCvxvthg5V+3moNkUdvxKAspA7Tf76C4iqycRuDVYalI0DwP2hKKEIxRM7 LuGSIN/lZwAUFkiO0qlAYR5PuvTOuW1Rnvwge77Gq9mha3d6p5k0E9+hX1gWPUEg4kXm 3rdWIozILfuDaV/46gnT8wjbQceE7ym4x2//+ycReUo39HoJrW5XjxP1hwKd6mZ6h8T0 MuLg== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20260327; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:dkim-signature; bh=5vdPDeHWM3XeLEwY+dGYudICQjX4GmUBfABmkkXIaQA=; fh=Ae7G+iR0KqtPkY7V6uaNln866BWqyQjwtHAFVqC7R4U=; b=swH18Bb5gDpkTmlrmcOYlXwlOWv/7YM7cFOaENMy/KkNQ2DnhJu8ifXwFNOK33ZDAl P6GA2y7X3bdA9FtyV4StuJ9CVaBlN62nhhhD5XY4GT1XlMekLFkRTO+irycsHhEtDDgF Is7h0A92V2msW9zWd8Mnr1QV0gczpW8K2nDVLVvj5XnG/EdCK8NOTKlSyTdgsGX40ngB H7iVRhy43Niv1LpQEW5Qkw09viZKp7Cw7mIlkVJIFktnuKStJGyzX1Wrc7jQL3SHQw4g Tmb1imMOzO3B7wNV7BmJ3kg/SkvN+0eW7vQ9OlajeiUJCs7HkHEopK6QSk7WrwotnY3W rasQ==; darn=isar-build.org ARC-Authentication-Results: i=2; gmr-mx.google.com; spf=pass (google.com: domain of akarpovich@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=akarpovich@ilbers.de DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlegroups.com; s=20251104; t=1790924701; x=1791529501; darn=isar-build.org; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:content-type :x-original-authentication-results:x-original-sender:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from:sender :from:to:cc:subject:date:message-id:reply-to:content-type; bh=5vdPDeHWM3XeLEwY+dGYudICQjX4GmUBfABmkkXIaQA=; b=OIniVlfXUhio13cFzFqpTz1fTpZIgPFcOVT4USoHWVKssxLlIUNE/J3OkMnv4bWo7J ZA4r9QyjKWgqXpRDZY2l+5SebrvZnvP+zVqsNldFSfKTPC/7BbYSRm+ofVS4HtHG2PbJ qUcWynKHPL8Nl6kE9/r3C9lLI9hhpUH0dgXk3OFZ0j1sexfoku8m5U3a8pDPNKtpHez2 6f62mMsonaTjnU+PR6H9g4hQsINhfuXbpYBj3yvA38lY8TP6s5luEYYFqOQYJFEOnm62 T55DCad7Bar5KVEEaBOEEK7xJpEhZZUPtDt1KDOZS68h6k+gHl2MzZ5BPd6NjMmqh14g 7fwg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20260707; t=1790924701; x=1791529501; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :x-spam-checked-in-group:list-id:mailing-list:precedence :content-type:x-original-authentication-results:x-original-sender :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:x-beenthere:x-gm-message-state:sender:from:to:cc:subject:date :message-id:reply-to:content-type; bh=5vdPDeHWM3XeLEwY+dGYudICQjX4GmUBfABmkkXIaQA=; b=cKAaI1IFWp3W6bkpAe6DiMDYLnuRXGbMWrJuq1TBS33MCx6XP/xDiIcy43uCkduTuN WZBTFQrhNyqBjSqQx78lA/u+riXdfd2WnfgGYjLTvzzM+rtTXwaTnYL0xTF6bcIchpSS UQHvbJHVTCR90HMQGPhWCB9d5P3vR5oN1fDFULtUKf52JV+vpapZrEGRrxGsqg+2P8WN agczjdMyZloujC8jbwFF8nTFNIIPSSgE4uF6+DVe1ksHvx0rednLyyaQ3LD4ZUTkXq/O S1cXjfBlmU/OhWxzo3RXeZ1uhAEfkQIBuU58fTpUfmDyf75x11WSMJKYWW1bGEeZs5Ri nQ8w== Sender: isar-users@googlegroups.com X-Forwarded-Encrypted: i=2; AKwUvBzqBApVyCtIBOsuV0wKVmzae5lIpTa7HdYmqJ0NoEs/NTJyY4TWvfcM6bGAl144bamLjAw3lCw=@isar-build.org X-Gm-Message-State: AFq9FYJeKQVBLNOqwO+Fehx1mJa9EDd9wNxa3DCxJl0QmenUldbY7LNx o9tMDV1cAUgtqSzXnkj9iPC9qzGOFOGQEGlKkr1IssDISbNJol7UbYXP X-Received: by 2002:a5d:6f05:0:b0:48b:6bb:6d3e with SMTP id ffacd0b85a97d-48b1274aa1bmr3225588f8f.15.1790924701566; Fri, 02 Oct 2026 00:05:01 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com; h="ATskLddPbeJQstuqCWnSB7lMLpCIJ+y/nNv5Q4OuyWKHuJz5VQ==" Received: by 2002:a05:6000:3c7:b0:487:a89:5d6f with SMTP id ffacd0b85a97d-48b12db6fe1ls403280f8f.0.-pod-prod-07-eu; Fri, 02 Oct 2026 00:04:59 -0700 (PDT) X-Received: by 2002:a05:6000:60a:b0:48a:fe3b:2d99 with SMTP id ffacd0b85a97d-48b1276809fmr3337294f8f.47.1790924699134; Fri, 02 Oct 2026 00:04:59 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1790924699; cv=none; d=google.com; s=arc-20260327; b=kbXZONJ3LAZysv8W1jGF48pwcddkOx2lQfRFQNbM7Gdtu/8dEJ1/P6CMUdaQ47U+M/ yTLoPRhQ/Cp+62rpsGpIlQXdFvQfgHmRX6alyYWho2sJYaGCC3cX7ztVYHyrKLCFXejL W6dVsDaJjb7tybe2ZHrWqTOFhnKmFQQYqrcNKsSxSVNoJEkHHs01kMEiEnIIEcSiMKkr s8nsGDT6M8QOHcxfu6GB1BFyTKi4nt1QErF7Vei94fkkU0y8FzsvmLrUgvC6LqAZ0D9L 0xxp3/3K7skWssfJVJ4VQ3rlMCppjmy5pOQZOF0+8p1gr3fgC1eXhiHPIRvkQrcgxWAa M3Dg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20260327; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=F/U5Br7k5DYetkXwwRQEBxedyLpqCKFKMh/vf6PJs8M=; fh=IIbR0JxonYgzdrHDdB9TmZniSh/o1z0HofhfbmLXp8s=; b=kr7kymvJ/+FtNUwEPk31g+a8mD4n5e5eI+J0Jn7XQ5T6FQMUyQ3MHFYh9xHrLoecUY CxHwErjIFsA2asHvkRz179FX9XPp2GRpR/xTCdNx1ocCSb8RJ9qSzwNRk2E/N8QmpE4g QpGfly+sxB+cmW7hHwDQYM82SVnv7cGBO0yPSZjCGhPGN+OHqTWiuJskxRvmpAhbmu+1 VK0KmoY6oWK8GxX73DAcVfs+U7yxyDWBNeqAwyM0mJXWLYX9N29FcI/xXSQDr1jSTq/I SRmmILUqumwPWthYq64Hh1XHvtnGBxTdZMwdeRqQUXRYaD6NJEu7mwFoqOkd+y60jqtJ jegA==; dara=google.com ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of akarpovich@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=akarpovich@ilbers.de Received: from shymkent.ilbers.de (shymkent.ilbers.de. [85.214.156.166]) by gmr-mx.google.com with ESMTPS id ffacd0b85a97d-48b5b30d315si21602f8f.9.2026.10.02.00.04.59 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 02 Oct 2026 00:04:59 -0700 (PDT) Received-SPF: pass (google.com: domain of akarpovich@ilbers.de designates 85.214.156.166 as permitted sender) client-ip=85.214.156.166; Received: from laptop-0xxx (62.122.174.6.serverel.net [62.122.174.6] (may be forged)) (authenticated bits=0) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPSA id 69274a1k003799 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 2 Oct 2026 09:04:57 +0200 From: akarpovich To: isar-users@googlegroups.com Cc: Uladzimir Bely , Aliaksei Karpovich Subject: [v11 05/12] base-apt: Predownload packages to base-apt before install Date: Fri, 2 Oct 2026 09:01:08 +0200 Message-ID: <20261002070335.26526-6-akarpovich@ilbers.de> X-Mailer: git-send-email 2.53.0 In-Reply-To: <20261002070335.26526-1-akarpovich@ilbers.de> References: <20261002070335.26526-1-akarpovich@ilbers.de> MIME-Version: 1.0 X-Spam-Status: No, score=-4.6 required=5.0 tests=DKIMWL_WL_MED,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_EF,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,RCVD_IN_DNSWL_BLOCKED,RCVD_IN_MSPIKE_H2, RCVD_IN_RP_CERTIFIED,RCVD_IN_RP_RNBL,RCVD_IN_RP_SAFE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on shymkent.ilbers.de X-Original-Sender: akarpovich@ilbers.de X-Original-Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of akarpovich@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=akarpovich@ilbers.de Precedence: list Mailing-list: list isar-users@googlegroups.com; contact isar-users+owners@googlegroups.com List-ID: X-Spam-Checked-In-Group: isar-users@googlegroups.com X-Google-Group-Id: 914930254986 List-Post: , List-Help: , List-Archive: , List-Unsubscribe: , X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= From: Uladzimir Bely This patch uses debrepo script to predownload packages to base-apt repository before they are installed in rootfs. Signed-off-by: Uladzimir Bely Signed-off-by: Aliaksei Karpovich --- meta/classes-recipe/crossvars.bbclass | 1 + meta/classes-recipe/dpkg-base.bbclass | 1 + meta/classes-recipe/dpkg.bbclass | 8 + .../image-locales-extension.bbclass.orig | 159 ++++++++++++++++++ .../image-tools-extension.bbclass | 13 ++ meta/classes-recipe/rootfs.bbclass | 5 +- meta/lib/aptsrc_fetcher.py | 46 ++++- .../sbom-chroot/sbom-chroot.bb | 2 + .../sbuild-chroot/sbuild-chroot-host.bb | 2 + 9 files changed, 228 insertions(+), 9 deletions(-) create mode 100644 meta/classes-recipe/image-locales-extension.bbclass.orig diff --git a/meta/classes-recipe/crossvars.bbclass b/meta/classes-recipe/crossvars.bbclass index 7cf2d660..6f070714 100644 --- a/meta/classes-recipe/crossvars.bbclass +++ b/meta/classes-recipe/crossvars.bbclass @@ -30,6 +30,7 @@ python __anonymous() { schroot_dir = d.getVar('SCHROOT_HOST_DIR', False) sbuild_dep = "sbuild-chroot-host" + flavor_suffix + ":do_build" sdk_toolchain = "crossbuild-essential-" + distro_arch + d.setVar('DEBREPO_WORKDIR', d.getVar('DEBREPO_HOST_DIR')) else: d.setVar('BUILD_ARCH', distro_arch) schroot_dir = d.getVar('SCHROOT_TARGET_DIR', False) diff --git a/meta/classes-recipe/dpkg-base.bbclass b/meta/classes-recipe/dpkg-base.bbclass index fbb952bd..4d268eff 100644 --- a/meta/classes-recipe/dpkg-base.bbclass +++ b/meta/classes-recipe/dpkg-base.bbclass @@ -11,6 +11,7 @@ inherit terminal inherit repository inherit deb-dl-dir inherit essential +inherit debrepo # Local (WORKDIR-internal) dir where do_dpkg_build collects the built debs. # Its content is exported into the shared, sstate-tracked DEPLOY_DIR_DEB so diff --git a/meta/classes-recipe/dpkg.bbclass b/meta/classes-recipe/dpkg.bbclass index 35408495..fe98655b 100644 --- a/meta/classes-recipe/dpkg.bbclass +++ b/meta/classes-recipe/dpkg.bbclass @@ -120,6 +120,12 @@ dpkg_runbuild() { echo '$stalled_pkg_timeout = ${DPKG_BUILD_TIMEOUT};' >> ${SBUILD_CONFIG} DSC_FILE=$(find ${DEPLOY_DIR_SRC} -maxdepth 1 -name "${DEBIAN_SOURCE}_*.dsc" -print) + debrepo_parse_dscfile "${DSC_FILE}" + + locked_update_cmd=":" + if [ "${ISAR_PREFETCH_BASE_APT}" = "1" ]; then + locked_update_cmd="flock -x /base-apt/repo.lock -c 'apt-get -y update'" + fi # networking is automatically enabled on older versions of sbuild sbuild_network_option="" @@ -143,9 +149,11 @@ dpkg_runbuild() { --chroot-setup-commands="rm -f /var/log/dpkg.log" \ --chroot-setup-commands="mkdir -p ${deb_dir}" \ --chroot-setup-commands="find ${ext_deb_dir} -maxdepth 1 -name '*.deb' -exec ln -t ${deb_dir}/ -sf {} +" \ + --chroot-setup-commands="${locked_update_cmd}" \ --chroot-setup-commands="apt-get update -o Dir::Etc::SourceList=\"sources.list.d/isar-apt.list\" -o Dir::Etc::SourceParts=\"-\" -o APT::Get::List-Cleanup=\"0\"" \ --finished-build-commands="rm -f ${deb_dir}/sbuild-build-depends-*-dummy_*.deb" \ --finished-build-commands="find ${deb_dir} -maxdepth 1 -type f -name '*.deb' -print -exec cp ${CP_FLAGS} -t ${ext_deb_dir}/ {} +" \ + --finished-build-commands="mkdir -p ${ext_root}" \ ${@ '--finished-build-commands="cp /var/log/dpkg.log $ext_root/dpkg_partial.log"' if d.getVar('ISAR_CHROOT_MODE') == 'schroot' else '' } \ --build-path="" --build-dir=${WORKDIR} --dist="${DEBDISTRONAME}" ${DSC_FILE} diff --git a/meta/classes-recipe/image-locales-extension.bbclass.orig b/meta/classes-recipe/image-locales-extension.bbclass.orig new file mode 100644 index 00000000..c54923cc --- /dev/null +++ b/meta/classes-recipe/image-locales-extension.bbclass.orig @@ -0,0 +1,159 @@ +# This software is a part of Isar. +# Copyright (C) Siemens AG, 2019 +# +# SPDX-License-Identifier: MIT +# +# This class extends the image.bbclass for setting locales and purging unneeded +# ones. + +inherit debrepo + +LOCALE_GEN ?= "en_US.UTF-8 UTF-8\n\ + en_US ISO-8859-1\n" +LOCALE_DEFAULT ?= "en_US.UTF-8" +# set locale used during package installation, which is visible to +# maintainer scripts (e.g. used in console-setup) +export LANG = "${LOCALE_DEFAULT}" +export LANGUAGE = "${LOCALE_DEFAULT}" +export LC_ALL = "${LOCALE_DEFAULT}" + +def get_locale_gen(d, sep='\n'): + locale_gen = d.getVar("LOCALE_GEN") or "" + return sep.join(sorted(set(i.strip() + for i in locale_gen.split('\\n') + if i.strip()))) + +def get_locale_dirs(d): + # Directory names below /usr/share/locale that shall be kept, derived from + # the language and language_territory parts of LOCALE_GEN. + locale_gen = d.getVar("LOCALE_GEN") or "" + return sorted(set(i + for j in locale_gen.split('\\n') + if j.strip() + for i in (j.split()[0].split("_")[0], + j.split()[0].split(".")[0]))) + +<<<<<<< HEAD +def get_locale_path_include(d): + lines = [] + for name in get_locale_dirs(d): + lines.append("path-include=/usr/share/locale/%s" % name) + lines.append("path-include=/usr/share/locale/%s/*" % name) + return "\n".join(lines) + +# Configure dpkg to only unpack the requested locales, so unneeded locale files +# are never installed. This runs before any package is installed into the image. +# Locale files that the bootstrap already installed are cleaned up here as well. +ROOTFS_CONFIGURE_COMMAND += "image_configure_locale_filter" +image_configure_locale_filter[weight] = "5" +image_configure_locale_filter() { + cat<<__EOF__ > ${WORKDIR}/locale.dpkg-filter +path-exclude=/usr/share/locale/* +path-include=/usr/share/locale/locale.alias +${@get_locale_path_include(d)} +__EOF__ + + run_privileged_heredoc <<'EOSUDO' + set -e + + mkdir -p '${ROOTFSDIR}/etc/dpkg/dpkg.cfg.d' + cat '${WORKDIR}/locale.dpkg-filter' \ + > '${ROOTFSDIR}/etc/dpkg/dpkg.cfg.d/50isar-locales' + + # Drop locale files the bootstrap installed that are not requested. + if [ -d '${ROOTFSDIR}/usr/share/locale' ]; then + keep=' locale.alias ${@' '.join(get_locale_dirs(d))} ' + for entry in '${ROOTFSDIR}'/usr/share/locale/*; do + [ -e "$entry" ] || continue + name=$(basename "$entry") + case "$keep" in + *" $name "*) ;; + *) rm -rf "$entry" ;; + esac + done + fi +EOSUDO +======= +ROOTFS_INSTALL_COMMAND_BEFORE_EXPORT += "image_install_localepurge_download" +image_install_localepurge_download[weight] = "40" +image_install_localepurge_download[network] = "${TASK_USE_NETWORK_AND_SUDO}" +image_install_localepurge_download() { + debrepo_add_packages "${DEBREPO_WORKDIR}" "localepurge" + debrepo_update_apt_source_list "${ROOTFSDIR}" "base-apt" + + run_privileged_heredoc <<'EOF' + set -e + ${@insert_isar_mounts(d, d.getVar('ROOTFSDIR'), d.getVar('ROOTFS_MOUNTS') if d.getVar('ISAR_CHROOT_MODE') == 'unshare' else '')} + chroot ${ROOTFSDIR} \ + /usr/bin/apt-get ${ROOTFS_APT_ARGS} -o Debug::NoLocking=1 --download-only localepurge +EOF +>>>>>>> 3308a3c4 (base-apt: Predownload packages to base-apt before install) +} + +# Preseed the debconf selection and generate the requested locales up front, +# before any package is installed, so package maintainer scripts find working +# locales. +ROOTFS_CONFIGURE_COMMAND += "image_configure_locale_debconf" +image_configure_locale_debconf[weight] = "5" +image_configure_locale_debconf() { + cat<<__EOF__ > ${WORKDIR}/locale.debconf +locales locales/locales_to_be_generated multiselect ${@get_locale_gen(d, ', ')} +locales locales/default_environment_locale select ${LOCALE_DEFAULT} +__EOF__ + cat<<__EOF__ > ${WORKDIR}/locale.gen +${@get_locale_gen(d)} +__EOF__ + cat<<__EOF__ > ${WORKDIR}/locale.default +LANG=${LOCALE_DEFAULT} +__EOF__ + + run_privileged_heredoc <<'EOSUDO' + set -e + + ${@insert_isar_mounts(d, d.getVar('ROOTFSDIR'), '')} + + cat '${WORKDIR}/locale.default' > '${ROOTFSDIR}/etc/default/locale' + cat '${WORKDIR}/locale.debconf' > '${ROOTFSDIR}/tmp/locale.debconf' + + # Enable the requested locales by uncommenting them in /etc/locale.gen + while read -r locale; do + [ -n "$locale" ] || continue + sed -i "/$locale/s/^# *//" '${ROOTFSDIR}/etc/locale.gen' + done < '${WORKDIR}/locale.gen' + + chroot '${ROOTFSDIR}' /bin/sh <<'EOSH' + set -e + + debconf-set-selections /tmp/locale.debconf + rm -f /tmp/locale.debconf +EOSH +EOSUDO +} + +# The systemd locale.conf symlink can only be created once systemd is installed. +ROOTFS_INSTALL_COMMAND += "image_configure_locales" +image_configure_locales[weight] = "100" +image_configure_locales() { + run_privileged_heredoc <<'EOSUDO' + set -e + + ${@insert_isar_mounts(d, d.getVar('ROOTFSDIR'), '')} + + chroot '${ROOTFSDIR}' /bin/sh <<'EOSH' + set -e + + SYSTEMD_VERSION=$(dpkg-query \ + --showformat='${source:Upstream-Version}' \ + --show systemd || echo "0" ) + + if dpkg --compare-versions "$SYSTEMD_VERSION" "ge" "251"; then + if dpkg --compare-versions "$SYSTEMD_VERSION" "lt" "253"; then + ln -s /etc/default/locale /etc/locale.conf + fi + fi + + echo 'reconfigure locales' + dpkg-reconfigure -f noninteractive locales +EOSH +EOSUDO +} diff --git a/meta/classes-recipe/image-tools-extension.bbclass b/meta/classes-recipe/image-tools-extension.bbclass index 8f666444..229ce318 100644 --- a/meta/classes-recipe/image-tools-extension.bbclass +++ b/meta/classes-recipe/image-tools-extension.bbclass @@ -6,6 +6,11 @@ # This file extends the image.bbclass to supply tools for futher imager functions inherit sbuild +inherit debrepo + +python __anonymous() { + d.setVar('DEBREPO_WORKDIR', d.getVar('DEBREPO_TARGET_DIR')) +} IMAGER_INSTALL ??= "" IMAGER_BUILD_DEPS ??= "" @@ -60,12 +65,20 @@ imager_run_schroot() { echo "Installing imager deps: ${local_install}" distro="${BASE_DISTRO}-${BASE_DISTRO_CODENAME}" + debrepo_workdir=${DEBREPO_TARGET_DIR} if [ ${ISAR_CROSS_COMPILE} -eq 1 ]; then distro="${HOST_BASE_DISTRO}-${BASE_DISTRO_CODENAME}" + if [ ${HOST_ARCH} != ${DISTRO_ARCH} ]; then + debrepo_workdir=${DEBREPO_HOST_DIR} + fi fi E="${@ isar_export_proxies(d)}" deb_dl_dir_import ${schroot_dir} ${distro} + + debrepo_add_packages --isarapt "${debrepo_workdir}" "${local_install}" + debrepo_update_apt_source_list "${schroot_dir}" "base-apt" + ${SCRIPTSDIR}/lockrun.py -r -f "${REPO_ISAR_DIR}/isar.lock" -s < X-Patchwork-Id: 5518 Return-Path: Received: from shymkent.ilbers.de ([unix socket]) by shymkent (Cyrus 2.5.10-Debian-2.5.10-3+deb9u2) with LMTPA; Fri, 02 Oct 2026 09:05:10 +0200 X-Sieve: CMU Sieve 2.4 Received: from mail-wm1-f56.google.com (mail-wm1-f56.google.com [209.85.128.56]) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPS id 692758KL003918 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Fri, 2 Oct 2026 09:05:08 +0200 Received: by mail-wm1-f56.google.com with SMTP id 5b1f17b1804b1-4a0284121c6sf3620845e9.3 for ; Fri, 02 Oct 2026 00:05:08 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1790924702; cv=pass; d=google.com; s=arc-20260327; b=PN9FVd23dYjW0zFTYvvOGzmY8znCCpjB8Nm8iBNKsmliJqhRbMo4BKkpyTZYhYLtuX l2ngH9VsULjs58g77njUNeVOtwwlB0Ca7qBBjs5C22xQnLTO6RfG4lfM1GJGrXNyezDr 3yEQKrLAJTofvKHOMfin2h6Wu+xgi5FLJ4NzUSR7AWuz0Fc45v1aR6nx02L+lG31TICD BHJsKLgHGfVziMRc2IwGMRU7+fe3BeQBdkF8Ir9BPdhvpZrl9sQYB1x0JWOIpWahH7sC z9wd7lKM3sgmljfY/7tr4ZgIEZq6qQJgcREXqiyCVeGOlCTqlOlT7rQOsiwNqtjeIGG9 Bm/A== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20260327; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:dkim-signature; bh=L5XYnjhRoESvMqpG2C2Wd8lOiikmr509gViOsqMrKXE=; fh=nMol+soWvhxQTpJu8ihYdO+bz/Kk1Rt+f3rc2aJsT5w=; b=gzwNRVAxziMy9ewcHOOZMrMCDODGcTgmZ105kGELU1WG9pb3BZtZCcqQJNmQcAHIvN cyZuqaZxd4n2vwgZdMb4G7h69TV8HUmrS1QLOfnIBnv8Y38zp74FldREsQ490N1Hhuhs PqfXE59tr4xZLXoaixeGLrnbpkUcbKdld9U+dOFznNgLFGzVdeGr9AP8YcCwglxstHcw d+UFvtSLLTsmuGNF7ianCFv2Fg69Pfho6lIIZW5kk5J5eb1Bx/5M4T9s6Or8QOMezU+a TADsvHlB99LlbC7lUcJrr4EQCx55gSEoqtHn9kN94onPb43NTzlIT2j9TF9fipthFY3s bxBA==; darn=isar-build.org ARC-Authentication-Results: i=2; gmr-mx.google.com; spf=pass (google.com: domain of akarpovich@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=akarpovich@ilbers.de DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlegroups.com; s=20251104; t=1790924702; x=1791529502; darn=isar-build.org; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:content-type :x-original-authentication-results:x-original-sender:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from:sender :from:to:cc:subject:date:message-id:reply-to:content-type; bh=L5XYnjhRoESvMqpG2C2Wd8lOiikmr509gViOsqMrKXE=; b=p3D39Lh5Lcoe3ZrFbh1uL8TpiGBmsY5+g0dSEaMBMOm1lM0zfRKOi9XiP4zh7CFiS6 eALubtN1fL7UK2Ut/V/Y7ua4jmZtbehbxSertKj+TCYpspEMtpnVrgq+wqsaaklUGkMu zPXf40cq0UbrKTbahiMuMaWhgX4kOrODeZmWS5s/jdtGshXSG+7V0j4JeJdT2c9/hNgC YxWg3kUoNotCHWOIce2oX5wE+wNfsvWFa0J3V31HcrC4E+uO3f2i45BtL58TT7zUdJpu DvYb3VMtnXhi3+G2/h+S/4IuPdhT5CcD0CD23c0yDP66r0OaDINUPlFpdDz9/I+8o5iZ 3IHA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20260707; t=1790924702; x=1791529502; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :x-spam-checked-in-group:list-id:mailing-list:precedence :content-type:x-original-authentication-results:x-original-sender :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:x-beenthere:x-gm-message-state:sender:from:to:cc:subject:date :message-id:reply-to:content-type; bh=L5XYnjhRoESvMqpG2C2Wd8lOiikmr509gViOsqMrKXE=; b=OQAcCH99WEc+Z5mhEfCBNUPYcIHrB6fWDQwAF0QNbAoDohqqwnNgzGztNxK9wY7mop RZu6fNM5cUZTKoUMmqCKaRtcb4ago521rRPOXGWxK0HDgc/FXiymO/NlK0c54GbYM4QU PEDfdQevpHzG9HmEyUNbW74zqQ7hJghOd8Jk4FddNCS1Ijk/gtgGpP8WdNVsyi5dJVrr w5KHKZUaAxMitheMtkITLdeeiiJ9bLIgfzb8mBZv/7IcECgZHHWod+XJAlgnkQ0B9m5H pQv/GiH2pLiQs0kugu6NeYaJeMCVCqpVcSLjOfJBIq1Qu2/V39mdoNwCNg7BtjjN5Y42 Mc9Q== Sender: isar-users@googlegroups.com X-Forwarded-Encrypted: i=2; AKwUvBxkCYhDMJki0R4mJ91SSgESu/s4bFYobAkm5kjN9GHMstAWz6oTAyzDSse5n/xTh3fjbdL2P60=@isar-build.org X-Gm-Message-State: AFuF++mRZC4UB3gc97tToUaXySe4iBSa++VVQB34mZDzRHvtB2fj2otk 01QjVobKRbK2sIkB57yU9VoXkx/OQDk48p//JYvFuaDI4GdAnT8biYhG X-Received: by 2002:a05:600c:3e15:b0:49f:ff32:8044 with SMTP id 5b1f17b1804b1-4a0276a1f1dmr28711835e9.7.1790924701989; Fri, 02 Oct 2026 00:05:01 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com; h="ATskLdfqZdH96c0QJoNvChWeQLxmZ4ouakGmoEfEkyeUNnMhyA==" Received: by 2002:a05:600c:4f44:b0:4a0:14ac:2d52 with SMTP id 5b1f17b1804b1-4a0285d570fls5882575e9.1.-pod-prod-08-eu; Fri, 02 Oct 2026 00:05:00 -0700 (PDT) X-Received: by 2002:a05:600c:3b1d:b0:4a0:2423:946 with SMTP id 5b1f17b1804b1-4a0276b5d6bmr32014805e9.15.1790924699970; Fri, 02 Oct 2026 00:04:59 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1790924699; cv=none; d=google.com; s=arc-20260327; b=KrE1AemJ1MBR2rtn4wK0rWYTSHAKe53Anl58SAoW+Hg3uGHnnGYree4j5DWnUw/7iP nhZrkF2Ld6WTxqaIWNtke72z1UvVTz5L8iPYkVPV8MmMpSvOmnwrs/fcoH/iOQsWm8Wm iLxcgxnCchp2HU7QlOGQfL6xwpR/KcgpWDZc23XY6T6XaKXGddd+ztQY6XiZx9Zjg1WO buNm9rB9+Ia9hJe7inYmdvT/ZQK89V0YKfvHkZrsj01CwlMO1zfIZAg2IhWZ3S9OtzYE vM3cSEYUZyN7fd4bvD60cHrVp+wNAjiFZr7hEsgkwy164wacj5FLfy0vy196VCpyMZGW aU6w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20260327; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=33o/eN1mkQ939/qbSuPt6BmZTb7KPFmfxlF9Zx3JHAw=; fh=IIbR0JxonYgzdrHDdB9TmZniSh/o1z0HofhfbmLXp8s=; b=EpLutULA3KCyWqBKwxNR7d5nNIv9T5Tf/sk1LehvdJKX7T59cxte9OuoFgMjaueXbX C0lvNkbtGSCGaHe64U4jEi4g1S8prCwnqxfVoj3vEt839bNigq1kz3Cvjwz6d1oojIq/ pr01pYVM3HMIbqcz7vJKk84OxKtaPN3z2iqmBGPdMMtb0B3L/huNyecBYMliTovXkmnZ 8FaMK1Azl2dCzG4Fskave0HNvm1aUmUh3XlMiMAh31VL9O946wzisdGyODFP2FrqmZUa k/vxyifXs4HBQtq8LDD9sR9a4L7RLMg1iRd4wY+mL/vbjV5tpgTPPBvmKBkCu8T4IEf0 12zA==; dara=google.com ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of akarpovich@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=akarpovich@ilbers.de Received: from shymkent.ilbers.de (shymkent.ilbers.de. [85.214.156.166]) by gmr-mx.google.com with ESMTPS id 5b1f17b1804b1-4a03fef0c14si162245e9.3.2026.10.02.00.04.59 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 02 Oct 2026 00:04:59 -0700 (PDT) Received-SPF: pass (google.com: domain of akarpovich@ilbers.de designates 85.214.156.166 as permitted sender) client-ip=85.214.156.166; Received: from laptop-0xxx (62.122.174.6.serverel.net [62.122.174.6] (may be forged)) (authenticated bits=0) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPSA id 69274a1l003799 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 2 Oct 2026 09:04:58 +0200 From: akarpovich To: isar-users@googlegroups.com Cc: Uladzimir Bely , Aliaksei Karpovich Subject: [v11 06/12] meta: Add cache-deb-src functionality in base-apt mode Date: Fri, 2 Oct 2026 09:01:09 +0200 Message-ID: <20261002070335.26526-7-akarpovich@ilbers.de> X-Mailer: git-send-email 2.53.0 In-Reply-To: <20261002070335.26526-1-akarpovich@ilbers.de> References: <20261002070335.26526-1-akarpovich@ilbers.de> MIME-Version: 1.0 X-Spam-Status: No, score=-4.6 required=5.0 tests=DKIMWL_WL_MED,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_EF,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,RCVD_IN_DNSWL_BLOCKED,RCVD_IN_MSPIKE_H2, RCVD_IN_RP_CERTIFIED,RCVD_IN_RP_RNBL,RCVD_IN_RP_SAFE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on shymkent.ilbers.de X-Original-Sender: akarpovich@ilbers.de X-Original-Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of akarpovich@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=akarpovich@ilbers.de Precedence: list Mailing-list: list isar-users@googlegroups.com; contact isar-users+owners@googlegroups.com List-ID: X-Spam-Checked-In-Group: isar-users@googlegroups.com X-Google-Group-Id: 914930254986 List-Post: , List-Help: , List-Archive: , List-Unsubscribe: , X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= From: Uladzimir Bely Fill base-apt repo with source packages. Signed-off-by: Uladzimir Bely Signed-off-by: Aliaksei Karpovich --- meta/classes-recipe/deb-dl-dir.bbclass | 41 ++++++++++++++++++++++++++ meta/classes-recipe/rootfs.bbclass | 7 +++++ 2 files changed, 48 insertions(+) diff --git a/meta/classes-recipe/deb-dl-dir.bbclass b/meta/classes-recipe/deb-dl-dir.bbclass index 60e2fecc..7e9f89b1 100644 --- a/meta/classes-recipe/deb-dl-dir.bbclass +++ b/meta/classes-recipe/deb-dl-dir.bbclass @@ -27,6 +27,47 @@ debsrc_source_version_filter() { | sort -u } +debsrc_fill_base_apt() { + export rootfs="$1" + export rootfs_distro="$2" + mkdir -p "${DEBSRCDIR}"/"${rootfs_distro}" + + ( flock 9 + set -e + printenv | grep -q BB_VERBOSE_LOGS && set -x + + # We need temporary files for our lists of source packages + # trap exit of this sub-shell to remove them (this script may exit abruptly + # since "set -e" is used) + avail=$(mktemp) + wanted=$(mktemp) + trap "rm -f ${avail} ${wanted}" EXIT + + # List all packages known to apt + rootfs_cmd \ + --bind "${DEBSRCDIR}" "/deb-src" \ + --bind "${rootfs}" "${rootfs}" \ + -- \ + apt-cache dumpavail \ + | debsrc_source_version_filter > ${avail} + + # Use apt-ftparchive to scan all .deb files found in the download directory + # and get the pairs that we wish to download + apt-ftparchive --md5=no --sha1=no --sha256=no --sha512=no \ + -a "${DISTRO_ARCH}" packages \ + "${REPO_BASE_DIR}" \ + | debsrc_source_version_filter > ${wanted} + + # We now have two sorted lists: source packages we want and those known to + # apt. We will only consider source packages that may be found in both. + comm -12 ${wanted} ${avail} \ + | while read src version; do + debrepo_add_packages --srcmode "${DEBREPO_TARGET_DIR}" "${src}=${version}" + done + + ) 9>"${DEBSRCDIR}/${rootfs_distro}.lock" +} + debsrc_download() { export rootfs="$1" export rootfs_distro="$2" diff --git a/meta/classes-recipe/rootfs.bbclass b/meta/classes-recipe/rootfs.bbclass index 61d2bdcb..ba56678a 100644 --- a/meta/classes-recipe/rootfs.bbclass +++ b/meta/classes-recipe/rootfs.bbclass @@ -515,6 +515,13 @@ do_cache_deb_src() { run_privileged tar -xf "${BOOTSTRAP_SRC}" ./var/lib/apt/lists --one-top-level="${ROOTFSDIR}" deb_dl_dir_import ${ROOTFSDIR} ${ROOTFS_BASE_DISTRO}-${BASE_DISTRO_CODENAME} + + debsrc_fill_base_apt ${ROOTFSDIR} ${ROOTFS_BASE_DISTRO}-${BASE_DISTRO_CODENAME} + + rootfs_do_mounts_priv + debrepo_update_apt_source_list "${ROOTFSDIR}" "base-apt" + rootfs_do_umounts_priv + debsrc_download ${ROOTFSDIR} ${ROOTFS_BASE_DISTRO}-${BASE_DISTRO_CODENAME} run_privileged rm -f "${ROOTFSDIR}"/etc/resolv.conf From patchwork Fri Oct 2 07:01:10 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: akarpovich X-Patchwork-Id: 5520 Return-Path: Received: from shymkent.ilbers.de ([unix socket]) by shymkent (Cyrus 2.5.10-Debian-2.5.10-3+deb9u2) with LMTPA; Fri, 02 Oct 2026 09:05:12 +0200 X-Sieve: CMU Sieve 2.4 Received: from mail-wr1-f59.google.com (mail-wr1-f59.google.com [209.85.221.59]) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPS id 69275ANx004018 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Fri, 2 Oct 2026 09:05:10 +0200 Received: by mail-wr1-f59.google.com with SMTP id ffacd0b85a97d-48704504d61sf2440836f8f.3 for ; Fri, 02 Oct 2026 00:05:10 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1790924703; cv=pass; d=google.com; s=arc-20260327; b=LO5gv5H2Qp0gf2fy4w1d9Ah4Dt23bntnH7b3kqF5M2FuAq66/CeRaz43L/JRuYYLZV th1JRiyQej8zTIHFAemne5f/n8XjLic3z/gfkTumN0R4rKa0fqjEwdsvVExHX4+HxoLM X4PFCE/dUq5otd5dg+sF+/xIFpoOUSBM2unslbaHU4gmy2u++aEnTtO91mnhTAjRc0ic Syr1r9dpaQI/JDxYTOPYkm14cUzHDUxSwWaaxRUdPFAmH3xEOYtz9k9SVZRQCbLq0sWV PD5+vYJwRNory9Xqocdth1Qv/CD9DeIOJzPlyi2Ebd2g6qL/YNFNUvzaFByPfxcCNKS3 wGNw== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20260327; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:dkim-signature; bh=tCJn0zy6e8q6oPejWu0nWrmb9rYsUfSfcdiFDQS+F6M=; fh=uqMiGDXN4NkclaWWzwb/qvXpmHw5Mn7XPBCfZM+07S4=; b=ctwCoFRQo/D8Nn33V3wmcsURfRY2GktzYMgPmq2dI9lh5iv3yXDA5TsBy84DjFQjJd qfkyXEOSEtgxbzlfB5a+ItSKJCb3tR42krKWKhSyUIsxUkReu54yJsFkDOFPqb22FTRK KJZz8nQ7xl2dVHRAO1YlivbHWQdXNHWgKb6ASxC07UO1yLqI54YqPq+czA08QwWuiQ8j +OZK6FS5pvQoYPqfbnCCSjPw0uwjRIfn4vwClzh4MxGLhCGZJl4AqYLp9RfLiCtITzdN y00XrulZO3BMqRs/Gj1dTG5Tp18F1kgzoeX8LyBCfPRoUjf+KN2NWWB/LW7jV4tyf6Bu 8IWA==; darn=isar-build.org ARC-Authentication-Results: i=2; gmr-mx.google.com; spf=pass (google.com: domain of akarpovich@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=akarpovich@ilbers.de DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlegroups.com; s=20251104; t=1790924703; x=1791529503; darn=isar-build.org; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:content-type :x-original-authentication-results:x-original-sender:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from:sender :from:to:cc:subject:date:message-id:reply-to:content-type; bh=tCJn0zy6e8q6oPejWu0nWrmb9rYsUfSfcdiFDQS+F6M=; b=rjnncl6DRUmE6VcqAG4mga6uUFCdUP/Z54NrY2oNuhWL8SHSERt800eBrJDpryvRH0 5aBsd41R3VNBrFKr++29CqquV687hV3dI/ow49NdAKJOMiLlwVPuBun0zg0oFjE5mGVN rQejXODYU+fKANyayBgD4523cFeRlmy+KqAPX2socH5qOO0zO2MZ11KNU3LI7WZHzLux pnrVIjwazjq+mFc7xLTL9CFTQme8sdjdr8C6qFfuqWiQ/yvCBkeMGA4QzTqiMkZrzkJy ROwkLaYwaYAOIlLeCzJ6Mixh9zPVgHDBKXu28QsZsNH59MZMbpnrMXrcEXAj2FHUF5lX vyLA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20260707; t=1790924703; x=1791529503; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :x-spam-checked-in-group:list-id:mailing-list:precedence :content-type:x-original-authentication-results:x-original-sender :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:x-beenthere:x-gm-message-state:sender:from:to:cc:subject:date :message-id:reply-to:content-type; bh=tCJn0zy6e8q6oPejWu0nWrmb9rYsUfSfcdiFDQS+F6M=; b=BfkF90gwfNqwHhrHzQXFCxla/njGsWKKr4GfWUDg1vkCseBRfmxuaXA+NcMQknHaXa +bAXf3txRf3Ol/4g8AWVUQqw8jiNCsGcc6eyABx2rv5ac4Q9AqFtEOHgxoBbJhLiZHph spgwUnnvW9ABBv33u+lmTVkClXz67cLi8uU/OqthtCBcJHpi2JcMkengztNd1epLirAG PK0YQwnRSXfqY0N5/tKt9cdY2kcDIB1X12UQYandHEWF5aA7qLNdjtCPaCZCCU4LHPoO q7pVsPYl/LikW0JtllAivG/D9nYMyZQ7tTJITDnspqqxtZr7M1Ib06Tnt5szhNeevZ1k pYPw== Sender: isar-users@googlegroups.com X-Forwarded-Encrypted: i=2; AKwUvBwil0hYKld+07TfBitY0wpEBTm/v8qoHMo86ZanM/qhXBjHpnHPGNGeB7xNYUUmf7NC2eUClKQ=@isar-build.org X-Gm-Message-State: AFuF++kkKD5OyItk7DfXWWQiHEqz5AFBonxrkJBbl1rLDtAk7IJ5g2OQ gux2Pm1unSLQIS4wh9PIUegLwP/l6idN3oCcWzb8pw0+faU6T6Sbo3pv X-Received: by 2002:a05:600c:1546:b0:4a0:1c43:3c04 with SMTP id 5b1f17b1804b1-4a02756e5edmr31789885e9.13.1790924702792; Fri, 02 Oct 2026 00:05:02 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com; h="ATskLdfWty0j0XmsuPczBBD7xVEVPXdSjVGJSL4kSnkjDW11EQ==" Received: by 2002:a05:600c:4594:b0:499:8364:19be with SMTP id 5b1f17b1804b1-4a02861bf4cls3335295e9.2.-pod-prod-04-eu; Fri, 02 Oct 2026 00:05:01 -0700 (PDT) X-Received: by 2002:a05:600c:1546:b0:4a0:1c43:3c04 with SMTP id 5b1f17b1804b1-4a02756e5edmr31787985e9.13.1790924700815; Fri, 02 Oct 2026 00:05:00 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1790924700; cv=none; d=google.com; s=arc-20260327; b=TRvifkvdg9LU4PNLox8WP7FEaBk/nB8qbcwscB61fSO7pcSBgs8uvYAWbcgAtC+1e0 tQbcBQ38f3u8BPCUeUhh8u3njsG655ze1C3tYIM0bNI8VZah903cO91TMZGEkpU8pZ9r F/PAZELMiGGRoIzn5gCTJaYhVhkM7Q4DfsPZ5VcTN5Eqt2ZrS39ZHKgtg9IBoggnUCzo W4fgQCUUzrskI1P3bJFWQsSSW+kw9uD44GBz6pGD4vqCH+1VUrlLlPaP9OfGz3a1SMx3 31fFmf9+u7aZZNEEbJ57rbncconGhmbY9Q6a1l8biJt3G+UvEZ2tUvl6Fo+1UuZMjkn1 sO3g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20260327; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=sIIVCfY0gPIsPacWQnGKL+djwi3sXvz36GQCSeK5yX4=; fh=2ALxNboLzDt503PhX56nFQ4pQzOkkVjVTCOSpqCio6o=; b=mOs/LqyaQJ9T4+o6BNb/UugWovm/QS1UgpZsjjSciEgPc7pdwI4pUsbcT5rqtDNHBK eTN97ZCr3I5eGaGBk+Id99ovFyVK5lgYdcvz2lCxg6+YRgHBPXNcN1mhCGZLIchvqcEw /Kv5WZfUSd9FoC5op/oQA+edmbHpaV/NVQZ8Xae5ucWbmMhJcB06mJfcdwqcMP54oNeE xLMnQ+2k6PFTThmVTjnVgn1VGRhzA1rZhMVtk6WFYdHpy0MPTA+vnSdJo6j6uT/F1xeB aaJsoWARv+a0CxoxorQxduCb8T9AgbMmUnXWXI3t/LB7G8dGOsYGcevGToP6UsuUYNQS EMxQ==; dara=google.com ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of akarpovich@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=akarpovich@ilbers.de Received: from shymkent.ilbers.de (shymkent.ilbers.de. [85.214.156.166]) by gmr-mx.google.com with ESMTPS id 5b1f17b1804b1-4a0280af5easi227365e9.3.2026.10.02.00.05.00 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 02 Oct 2026 00:05:00 -0700 (PDT) Received-SPF: pass (google.com: domain of akarpovich@ilbers.de designates 85.214.156.166 as permitted sender) client-ip=85.214.156.166; Received: from laptop-0xxx (62.122.174.6.serverel.net [62.122.174.6] (may be forged)) (authenticated bits=0) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPSA id 69274a1m003799 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 2 Oct 2026 09:04:59 +0200 From: akarpovich To: isar-users@googlegroups.com Cc: Uladzimir Bely Subject: [v11 07/12] testsuite: Set ISAR_PREFETCH_BASE_APT by default Date: Fri, 2 Oct 2026 09:01:10 +0200 Message-ID: <20261002070335.26526-8-akarpovich@ilbers.de> X-Mailer: git-send-email 2.53.0 In-Reply-To: <20261002070335.26526-1-akarpovich@ilbers.de> References: <20261002070335.26526-1-akarpovich@ilbers.de> MIME-Version: 1.0 X-Spam-Status: No, score=-4.6 required=5.0 tests=DKIMWL_WL_MED,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_EF,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,RCVD_IN_DNSWL_BLOCKED,RCVD_IN_MSPIKE_H2, RCVD_IN_RP_CERTIFIED,RCVD_IN_RP_RNBL,RCVD_IN_RP_SAFE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on shymkent.ilbers.de X-Original-Sender: akarpovich@ilbers.de X-Original-Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of akarpovich@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=akarpovich@ilbers.de Precedence: list Mailing-list: list isar-users@googlegroups.com; contact isar-users+owners@googlegroups.com List-ID: X-Spam-Checked-In-Group: isar-users@googlegroups.com X-Google-Group-Id: 914930254986 List-Post: , List-Help: , List-Archive: , List-Unsubscribe: , X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= From: Uladzimir Bely This makes Isar use `base-apt` repo in different way. Any package installation is done from `base-apt` repo which is prepopulated from external mirrors. This behaviour is disabled by default for downstreams. To enable it, set the variable to "1", like isar does in local.conf.sample. In order to be able to run CI in old mode, allow CI read the option from the environment. Also, adjust some tests (like repro one) to make them work with ISAR_PREFETCH_BASE_APT set. Signed-off-by: Uladzimir Bely --- meta-test/conf/local.conf.sample | 3 +++ testsuite/cibase.py | 4 ++++ testsuite/cibuilder.py | 7 +++++++ 3 files changed, 14 insertions(+) diff --git a/meta-test/conf/local.conf.sample b/meta-test/conf/local.conf.sample index 2dbe28f1..74ee90b1 100644 --- a/meta-test/conf/local.conf.sample +++ b/meta-test/conf/local.conf.sample @@ -27,6 +27,9 @@ BB_DISKMON_DIRS = "\ MIRRORS ?= "git?://salsa\.debian\.org/debian/.* git://github.com/ilbers/BASENAME" MIRRORS += "https?://cdn\.kernel\.org/.* https://mirrors.edge.kernel.org/PATH" +# Use new base-apt behaviour +ISAR_PREFETCH_BASE_APT ?= "1" + # Users and groups USERS += "root" USER_root[password] ??= "$6$rounds=10000$RXeWrnFmkY$DtuS/OmsAS2cCEDo0BF5qQsizIrq6jPgXnwv3PHqREJeKd1sXdHX/ayQtuQWVDHe0KIO0/sVH8dvQm1KthF0d/" diff --git a/testsuite/cibase.py b/testsuite/cibase.py index 2cf8448b..cff22282 100755 --- a/testsuite/cibase.py +++ b/testsuite/cibase.py @@ -153,10 +153,14 @@ class CIBaseTest(CIBuilder): self.fail("GPG import failed") try: + self.move_in_build_dir('tmp', 'tmp_before_repro') self.bitbake(targets, **kwargs) repro_type = 'signed' if signed else 'unsigned' self.move_in_build_dir('tmp', f"tmp_middle_repro_{repro_type}") + + os.makedirs(f"{self.build_dir}/tmp/deploy/") + self.move_in_build_dir(f"tmp_middle_repro_{repro_type}/deploy/base-apt", 'tmp/deploy/base-apt') self.configure( targets=targets, gpg_pub_key=gpg_pub_key if signed else None, diff --git a/testsuite/cibuilder.py b/testsuite/cibuilder.py index 1d415084..ca75f2e1 100755 --- a/testsuite/cibuilder.py +++ b/testsuite/cibuilder.py @@ -174,6 +174,10 @@ class CIBuilder(Test): fail_on_cleanup = os.getenv('ISAR_FAIL_ON_CLEANUP') strlines = None if lines is None else '\\n'.join(lines) + + # get prefetch base apt mode from environment + prefetch_base_apt = os.getenv('ISAR_PREFETCH_BASE_APT') + self.log.info( f"===================================================\n" f"Configuring build_dir {self.build_dir}\n" @@ -197,6 +201,7 @@ class CIBuilder(Test): f" generate_sbom = {generate_sbom}\n" f" rootless = {rootless}\n" f" lines = {strlines}\n" + f" prefetch_base_apt = {prefetch_base_apt}\n" f"===================================================" ) @@ -305,6 +310,8 @@ class CIBuilder(Test): f.write('ISAR_ROOTLESS = "1"\n') if lines is not None: f.writelines((line + '\n' if not line.endswith('\n') else line) for line in lines) + if prefetch_base_apt == "0": + f.write('ISAR_PREFETCH_BASE_APT = "0"\n') # include ci_build.conf in local.conf with open(self.build_dir + '/conf/local.conf', 'r+') as f: From patchwork Fri Oct 2 07:01:11 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: akarpovich X-Patchwork-Id: 5519 Return-Path: Received: from shymkent.ilbers.de ([unix socket]) by shymkent (Cyrus 2.5.10-Debian-2.5.10-3+deb9u2) with LMTPA; Fri, 02 Oct 2026 09:05:10 +0200 X-Sieve: CMU Sieve 2.4 Received: from mail-ed1-f56.google.com (mail-ed1-f56.google.com [209.85.208.56]) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPS id 692758B8003927 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Fri, 2 Oct 2026 09:05:08 +0200 Received: by mail-ed1-f56.google.com with SMTP id 4fb4d7f45d1cf-6aaa4535b3bsf8335587a12.3 for ; Fri, 02 Oct 2026 00:05:08 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1790924703; cv=pass; d=google.com; s=arc-20260327; b=YAZqkKx2gBdl56as7lr9c52X5Ohod02wGSuxbTvGyE20bPJcVK0zyWRmicDo7hhmKS WCi4+yQS0C0uMtY65izLVrRgVnaC3yBqowAUrHQzc9OZdSbTLcLT8jDKoc9nNY3GMyeM F6mo6l8eLqwc0OHLJ/BgjY4v4nLUP8OOzz2XFImPuxtopusuhPzc72SU9qEk3Ob4BSw4 OftwQv7aN9zGegpmTbdiJ0Ux73lQh1RkDI3wVrvVVWK0F9BtR7huLb0aNPfrOweFMBtm 2Q/71ppOkrH2UO+xS6TEVmikaOrFYHCO7/n/OgB0S2fZAL9bB/MW4kaJx+ANCBlosOqf 4U7Q== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20260327; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:dkim-signature; bh=VNXeDRzEegrHty6NcUrfbaL+5K8eeKERzvWadehKikw=; fh=9I+JMRXeEeKhOBOf9CTuXmRBS3uJ3DYAL8Tmhkw+aVQ=; b=T+Gs6S0DybHW7Or3GuNZAcYDz1J3D9qav4jVWDjGhL3fgzkqvivQtDribRxp63S9ew hPUjcjFABZQanm8huHoKb4oubOYzhRMPNvqLjXxVRbCxo9zDAVQXunukcQtxX+9WzElN /V6f5lGmq4sI9bqxhi9wZFNHAASpDWd28ktMZP86GjJfAZqd+z+7usudc+CgGD10nEqK bbgLm9b9dUcN2iyvToVTQn73BKgDn4jX/V2GjJPc4SWkbqzCw/eFhXDL8ViIM25AFy7x ikgeHXRUZkGgnIBiYR1Bj8aWi9fjLnCtb4wS0W4K8cVEjdoh6RuNs5teu4F7mEMsOkne /rPw==; darn=isar-build.org ARC-Authentication-Results: i=2; gmr-mx.google.com; spf=pass (google.com: domain of akarpovich@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=akarpovich@ilbers.de DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlegroups.com; s=20251104; t=1790924703; x=1791529503; darn=isar-build.org; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:content-type :x-original-authentication-results:x-original-sender:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from:sender :from:to:cc:subject:date:message-id:reply-to:content-type; bh=VNXeDRzEegrHty6NcUrfbaL+5K8eeKERzvWadehKikw=; b=Ll+tB0smgXMUHbSJ/05UxI43z2Ys/BzRH0Y0YEp93iRPcFKCI9T3eM2Yj4pVUWe3Ks pXLTHvD2tEs+fLVOL1tLlRRBoYuYuPCockHjvVJUoFzL2v47Di6c3Ynoadfbl5wdJXbF 1cZDIP7e8L8RJJoKr/gnjR0T4NDBUiKkWwfWRz42dkH30sddSaoRc7U6vDGb0Nr0rhJc SjUZXYHkUD9nB8mHnfhaHD67NlyRchJIG+6HrCcr+AO8Nrhr2Y+w1fajVd06jvG0GrB4 64/74tA4whCmM9hXrh2TMKU3VSzTMptU3+b9tjSVtDOgHWh8Gc59ZC5ClHuHIeWSDHI0 cJGQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20260707; t=1790924703; x=1791529503; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :x-spam-checked-in-group:list-id:mailing-list:precedence :content-type:x-original-authentication-results:x-original-sender :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:x-beenthere:x-gm-message-state:sender:from:to:cc:subject:date :message-id:reply-to:content-type; bh=VNXeDRzEegrHty6NcUrfbaL+5K8eeKERzvWadehKikw=; b=XWsXT9Z4oggtVYFZDzfBMeMf5kiK59iy1r7zCnbHnm/U0ZzpV6+B1ocxuavHrdE0wR q4Y/RMvenj6yQKWsB/qNGDsEzHGUTl9DEmLetgEK/On5WRGj5Au40hx0n1xyAbkmlX3J fiEOh7zzghLx/GHmdT3LOndCVywQLX2c9m0jWxMFJ/gvAlTozWo0qY2gV381YaCpq4CE uhQnj9zQPX0c2bEAesqe3TMHZJZSnTp37C9exXvwn0utjQ/2LPZKyXsterD2ozzmPoeK M49X/sTeayqYK0nU9p8GCqa5TITy6bsEij484kOfQAmBZUhzSOCxUZcMHAxySnR4M/26 w0WQ== Sender: isar-users@googlegroups.com X-Forwarded-Encrypted: i=2; AKwUvBxXwUsaJ84DJcOm31aeJJJEuud6H6/H+3WOt5lBv3J/elKQBjmxzwcCGUci5NmQCN/vGraZU9I=@isar-build.org X-Gm-Message-State: AFq9FYJ/osxN7G0FDhLbbe9oS4RyhLEu7iRxUqwLLKvwJTwXXRjSvw7a IDS+K0bOx2excNtKlLndEnRlf8k3pBLFb6IjmYzq978tnqbTfOVuHhJX X-Received: by 2002:a05:6402:388d:b0:6a7:e514:1d85 with SMTP id 4fb4d7f45d1cf-6af9e2dbda2mr1084566a12.13.1790924703514; Fri, 02 Oct 2026 00:05:03 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com; h="ATskLdeIhBg9p4LpF+I5jtZaT0hDZ+GfWmWjyO9SEiHDAlvfrA==" Received: by 2002:a05:6402:52c5:b0:6aa:c8a7:f5ea with SMTP id 4fb4d7f45d1cf-6af9d773a83ls20552a12.0.-pod-prod-05-eu; Fri, 02 Oct 2026 00:05:02 -0700 (PDT) X-Received: by 2002:a05:6402:e0a:b0:6ac:66c5:c318 with SMTP id 4fb4d7f45d1cf-6af9e3369a3mr1020545a12.41.1790924701699; Fri, 02 Oct 2026 00:05:01 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1790924701; cv=none; d=google.com; s=arc-20260327; b=G7nWzdLMtnGbGEBsavn/Xx/6Za8+27Z4dzccfhPDWukxZAGQjgaKjP5VGbFkSsZwYI NVQgSMZhns1hGmmu7YgG9jHBUbpbK6kCGGA4fuR2uU6y48SGZhSxNOWrTD4AS+zlVral gYVYwtFAIiNN+AsYX2ngli8npHVYbYVbSTGB83lPm4pJf96iZ9GAfrBO2U2Qxpvpf89M 60LSGhTgj8HHkyjcKCuCkY8c0qD/n9RTYUi2N/wHRiFuDQEYeBSGrbivnYVV0PeIDEw9 5j9QDRQMtBoTPRgtTSMo3+wWSTn03BraPqLaMjtBMLJUlehCUvHixL2lKjWgn5RjvcVk O9vg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20260327; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=Xo0TqgDvzxvyhBCJCxv4w+Krzs0SD+pI6+LIB7ogdBY=; fh=IIbR0JxonYgzdrHDdB9TmZniSh/o1z0HofhfbmLXp8s=; b=W4/UKgvqcQbsNB7J7p8yHyeSeG7xLRaGBrKbTChFSKKhU0LNi85tCZM9s6HKWTuQw2 eZcKW0dodv8EH4XZmPTFyp2WybcUrR8Rw+jCngFJbPsnTUzD4oBjNKrBQmiEhpdM/n2W sq4tkm9nfo7sNVjjoNcSilIgu++FQ5uiaCFsMQ4xkJjkf54Mi1oiBpQYQIwv5rPC16zT Y9bogVpG/eHPZFKMa2xNqz4ga+OKtx4QqBTexecDDbGvoH/pJL8Xk2WUTHPKPWsoAjrS Ci5yhWqfBINFxs3Qodmzw7QvfybPnzfY8dVwsaqjV9LpkU/6jDb0TMJIXtmbxAwmz767 hTNQ==; dara=google.com ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of akarpovich@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=akarpovich@ilbers.de Received: from shymkent.ilbers.de (shymkent.ilbers.de. [85.214.156.166]) by gmr-mx.google.com with ESMTPS id 4fb4d7f45d1cf-6af9e266641si20845a12.7.2026.10.02.00.05.01 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 02 Oct 2026 00:05:01 -0700 (PDT) Received-SPF: pass (google.com: domain of akarpovich@ilbers.de designates 85.214.156.166 as permitted sender) client-ip=85.214.156.166; Received: from laptop-0xxx (62.122.174.6.serverel.net [62.122.174.6] (may be forged)) (authenticated bits=0) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPSA id 69274a1n003799 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 2 Oct 2026 09:05:00 +0200 From: akarpovich To: isar-users@googlegroups.com Cc: Uladzimir Bely , Aliaksei Karpovich Subject: [v11 08/12] Disable deb-dl-dir in base-apt prefetch mode Date: Fri, 2 Oct 2026 09:01:11 +0200 Message-ID: <20261002070335.26526-9-akarpovich@ilbers.de> X-Mailer: git-send-email 2.53.0 In-Reply-To: <20261002070335.26526-1-akarpovich@ilbers.de> References: <20261002070335.26526-1-akarpovich@ilbers.de> MIME-Version: 1.0 X-Spam-Status: No, score=-4.6 required=5.0 tests=DKIMWL_WL_MED,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_EF,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,RCVD_IN_DNSWL_BLOCKED,RCVD_IN_MSPIKE_H2, RCVD_IN_RP_CERTIFIED,RCVD_IN_RP_RNBL,RCVD_IN_RP_SAFE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on shymkent.ilbers.de X-Original-Sender: akarpovich@ilbers.de X-Original-Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of akarpovich@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=akarpovich@ilbers.de Precedence: list Mailing-list: list isar-users@googlegroups.com; contact isar-users+owners@googlegroups.com List-ID: X-Spam-Checked-In-Group: isar-users@googlegroups.com X-Google-Group-Id: 914930254986 List-Post: , List-Help: , List-Archive: , List-Unsubscribe: , X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= From: Uladzimir Bely Since all packages and source packages are placed to base-apt repo during build, there is no need to have them in one more place. Signed-off-by: Uladzimir Bely Signed-off-by: Aliaksei Karpovich --- meta/classes-recipe/deb-dl-dir.bbclass | 2 ++ meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc | 3 ++- 2 files changed, 4 insertions(+), 1 deletion(-) diff --git a/meta/classes-recipe/deb-dl-dir.bbclass b/meta/classes-recipe/deb-dl-dir.bbclass index 7e9f89b1..2fafd0a7 100644 --- a/meta/classes-recipe/deb-dl-dir.bbclass +++ b/meta/classes-recipe/deb-dl-dir.bbclass @@ -174,6 +174,7 @@ deb_dl_dir_import() { # nothing to copy if download directory does not exist just yet [ ! -d "${pc}" ] && return 0 + [ "${ISAR_PREFETCH_BASE_APT}" = "1" ] && return 0 # attempt to create hard-links for .deb files from downloads/ into # /var/cache/apt/archives/ so apt will only download packages we @@ -196,6 +197,7 @@ deb_dl_dir_export() { export rootfs="${1}" export owner=$(id -u):$(id -g) mkdir -p "${pc}" + [ "${ISAR_PREFETCH_BASE_APT}" = "1" ] && return 0 export isar_debs=$(${SCRIPTSDIR}/lockrun.py -r -f '${REPO_ISAR_DIR}/isar.lock' -c \ "find '${REPO_ISAR_DIR}/${DISTRO}' -name '*.deb' -print") diff --git a/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc b/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc index 01ec56e7..2004dafb 100644 --- a/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc +++ b/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc @@ -281,7 +281,8 @@ do_bootstrap() { - \ "$bootstrap_list" > ${DEPLOYDIR}/${DEPLOY_ISAR_BOOTSTRAP}.tar.zst - if [ "${ISAR_USE_CACHED_BASE_REPO}" != "1" ]; then + if [ "${ISAR_USE_CACHED_BASE_REPO}" != "1" ] && \ + [ "${ISAR_PREFETCH_BASE_APT}" != "1" ]; then deb_dl_dir_export "${WORKDIR}/dl_dir" "${BOOTSTRAP_BASE_DISTRO}-${BASE_DISTRO_CODENAME}" run_privileged find ${WORKDIR}/dl_dir -maxdepth 1 -mindepth 1 -exec rm -rf --one-file-system "{}" \; rmdir ${WORKDIR}/dl_dir From patchwork Fri Oct 2 07:01:12 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: akarpovich X-Patchwork-Id: 5521 Return-Path: Received: from shymkent.ilbers.de ([unix socket]) by shymkent (Cyrus 2.5.10-Debian-2.5.10-3+deb9u2) with LMTPA; Fri, 02 Oct 2026 09:05:12 +0200 X-Sieve: CMU Sieve 2.4 Received: from mail-lj1-f184.google.com (mail-lj1-f184.google.com [209.85.208.184]) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPS id 69275AJP004020 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Fri, 2 Oct 2026 09:05:10 +0200 Received: by mail-lj1-f184.google.com with SMTP id 38308e7fff4ca-3a7845604d1sf10660831fa.1 for ; Fri, 02 Oct 2026 00:05:10 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1790924705; cv=pass; d=google.com; s=arc-20260327; b=TNSTXXSqJPjjSw+YJLoMkmgasuTNPNMauw8xYXPwy49NbH0UXyBI0T1sS7FGVyOB2d Vb5ygL1GPuSObWgxKX0n41Kh891TQILgMQBiHrBYCdhnYWCD5UTX7RW1bfz8+fVNdsw3 uG4IRUfvy7myu2BdpqDtyRtYh1equRfS2Cnd7wevjiZad6o3htdyJ6UROVhbp44Yf1VI 1I39evVNNLJuhK4H0BvxLcCs/0hY93JJU83As6FEjhLDRN+VkKiEuUE+p9znBNsTDkb7 pXaRGmhOUxsByBI0jYJ/cLgOvAjKZL7xMs+4RaYnfWaXKxfdNmL/lgr5cP4pcESMVlnp Es0Q== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20260327; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:dkim-signature; bh=OVCnL6TMVEKbVih9lSekpaIWmicCHQjqwOWVBW4/41I=; fh=jPZYfH+WTQVeSYvBiyFyyUFPM0yoeEdQhRRhpm6bE/I=; b=cbDYpZ0rW6oJoM1qZCIJXavtlgqrgBbY/115bfdJBnY/aTo0ammUQRw/f2qnEOKVC/ L7JJI4rwuGpDbKabBR+eMq+5AOyxxIYvtXMwiR6WnjcRFnPAiGpIjimjLg/IcHL8aIq7 CmAnWmQ86pNXnP5rBVMGNK/fy/unZgkQReyVwGM6x74+bd/ngIauWg91f7sf1AuK+06/ OAQSFRUPIjSk76poQtXXAgn6vBzncPQfRi1IB8jjvwXsd9boJwgfD8gcAMUxjuNeEPr3 pIK8fvR7cegUqn/1yLSGhCglxgE5FKL9iF8Cx+gL53wAKTvkQxTi/NhkxQTQWCdDtQWi fgLA==; darn=isar-build.org ARC-Authentication-Results: i=2; gmr-mx.google.com; spf=pass (google.com: domain of akarpovich@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=akarpovich@ilbers.de DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlegroups.com; s=20251104; t=1790924705; x=1791529505; darn=isar-build.org; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:content-type :x-original-authentication-results:x-original-sender:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from:sender :from:to:cc:subject:date:message-id:reply-to:content-type; bh=OVCnL6TMVEKbVih9lSekpaIWmicCHQjqwOWVBW4/41I=; b=x1FCT/Mdvim6NFChE1vX14edFPofr9SCkeyuToheoMWRIVTnlhU9IAqiaEHQXD4CGX 3hJAjqYhxTjNyaUhuq2uSIm7//8XKQVTgUEFX9e3/z8C7g23d9QiOeeNMoND+61+/IFF fIvHoSkHSRBnzUgcoMebpF4+r/BOPu0hySq0IIGiKY/CSNgYykILz4kKAikZyEreIYIP ZUetzJy2tT7wCRVZJIhFXRySc4Y+uPRKgWd/dofc45kxd7k+XnJ0nS/mszHr7vsMk/8k Gr3yUCqem7B93XgLo1CcqcLsmu8B6rs/iWQ+CaTdo05F3EAWznSRsHEwaRh5NLL44shL eO4w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20260707; t=1790924705; x=1791529505; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :x-spam-checked-in-group:list-id:mailing-list:precedence :content-type:x-original-authentication-results:x-original-sender :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:x-beenthere:x-gm-message-state:sender:from:to:cc:subject:date :message-id:reply-to:content-type; bh=OVCnL6TMVEKbVih9lSekpaIWmicCHQjqwOWVBW4/41I=; b=QWkZTEiH3bJh6LRefLnfzk/WtzVC+57vlfZWFM80xnTMSWZVhRwb2R59dXiRZHMKdj 9TjDHIiPdJJtrtd32yLUfejLLWWqC8Hh9ISuUm1euf/75oP984YGAgRWTai+XGNyzjQY wgx/LRprClCJzF2WKxG1RB+GgcJXfryAh/GdUQ8KUIZ6bHroub5kQAVm+BAH5IrEWSCa odO9P2T0b7F8eLyCxIQ9BlNCiLf9cVsV6Zuu7lzvjq5teT9DVx+DD3danSQ7GpqrdC28 BIbaRj9Am5wVdzm+ha5Hjx6mbS0ZlhFYO8bU8b6aPRmekBx7K9kX1FdJyPPcuIKoUjkH mUOA== Sender: isar-users@googlegroups.com X-Forwarded-Encrypted: i=2; AKwUvBy+GnXBpfsRDx9TA6I3Ga2T2n4v0vJ4rRVtccrdP6WXRNFeuqdJN9c+THqwLEBT0Mp9fYJ7Vik=@isar-build.org X-Gm-Message-State: AFq9FYIYSRD0DR2vRtlfUkp4XfcrsNo0n4/M+7Pem0AHvqkcgM10L/pY kDS3tb0ZOvumlxnC4OrkWAzSJdSJVWQjYukRvoE07ak7DKv6ZB+1dFQJ X-Received: by 2002:a2e:a986:0:b0:3a8:7db9:3b19 with SMTP id 38308e7fff4ca-3a88765a8f9mr4101221fa.9.1790924705053; Fri, 02 Oct 2026 00:05:05 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com; h="ATskLde+hx/+N8m5Jxa47HokXjbk5Tz491HlQTnFdlL4KcP46Q==" Received: by 2002:a2e:88cc:0:b0:3a5:9af0:f2a5 with SMTP id 38308e7fff4ca-3a8a86335edls1886711fa.1.-pod-prod-02-eu; Fri, 02 Oct 2026 00:05:03 -0700 (PDT) X-Received: by 2002:a2e:bea6:0:b0:3a7:5ff2:576 with SMTP id 38308e7fff4ca-3a87ed41ebfmr4308991fa.1.1790924702648; Fri, 02 Oct 2026 00:05:02 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1790924702; cv=none; d=google.com; s=arc-20260327; b=Ow5+9yYVUa0MqFr/9JV9HiwbKHZiz7lAa8gVj76imVNll500rYBqERj62riCcZiadF TMXne7tmKs43rVVsLB0vRV0uf4I2h7n3aT5dJ+brk7glFafh2QnORPKoHINC8ZtBGUNb zlpe5uv4NiiskXDHREELkf4Iw92jnyyKP0HQW5g0kHBN2xqKk2B3PwOH1nl9fVxiXYUn I9zkEeD+dvSYrGhVpRTAaUZTdr7gnqaPOKEfxQcORjoH+B1ouIdcmfE6W7VP166BIZq7 sSTDPVTXZVimhe0SZeK+y0N5D7lwj5IFZKs1YHLr2Isp/ySAsemzXizivMvfbBdtCsps TQ6Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20260327; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=6jgUoTyoKd+Ylgr7JQl/ljKplh5BC6L4IxxErMKgZws=; fh=2ALxNboLzDt503PhX56nFQ4pQzOkkVjVTCOSpqCio6o=; b=BR3vqnqx4c3QNP4CcGYZU8ZU699+w0OvgNXPZ8PPXgebY70O1euW4ESU+kH3HuPx8I csMf1DxFF02WimiC/TQ3iH9mADXbaMWlp08H4TnJs8Iwe0iBem/78fF6m2Zfjfxt3ATH 0uLNfCIHxJoePv1hvoOkUGpDEcA9U1mU2W/JIhe1mYLyctJlTJFL5YSKHd1K1YvaF+6D mCB7mtWjg6Ij6mXbiHjHsxZcCEhgkZVTAUGKZkuf3rJC+fQFqYxiDXuSotHwps/SdMMS JXq37yxHRSKGAPLj+CGvrkfcBS7UPqnkszqfN6/US0eLpHF/WHOMX5R+q97dALnBv2MI DRhg==; dara=google.com ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of akarpovich@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=akarpovich@ilbers.de Received: from shymkent.ilbers.de (shymkent.ilbers.de. [85.214.156.166]) by gmr-mx.google.com with ESMTPS id 38308e7fff4ca-3a8a87e1e56si177351fa.5.2026.10.02.00.05.02 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 02 Oct 2026 00:05:02 -0700 (PDT) Received-SPF: pass (google.com: domain of akarpovich@ilbers.de designates 85.214.156.166 as permitted sender) client-ip=85.214.156.166; Received: from laptop-0xxx (62.122.174.6.serverel.net [62.122.174.6] (may be forged)) (authenticated bits=0) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPSA id 69274a1o003799 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 2 Oct 2026 09:05:01 +0200 From: akarpovich To: isar-users@googlegroups.com Cc: Uladzimir Bely Subject: [v11 09/12] kas: Add PREFETCH_BASE_APT config entry Date: Fri, 2 Oct 2026 09:01:12 +0200 Message-ID: <20261002070335.26526-10-akarpovich@ilbers.de> X-Mailer: git-send-email 2.53.0 In-Reply-To: <20261002070335.26526-1-akarpovich@ilbers.de> References: <20261002070335.26526-1-akarpovich@ilbers.de> MIME-Version: 1.0 X-Spam-Status: No, score=-4.6 required=5.0 tests=DKIMWL_WL_MED,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_EF,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,RCVD_IN_DNSWL_BLOCKED,RCVD_IN_MSPIKE_H2, RCVD_IN_RP_CERTIFIED,RCVD_IN_RP_RNBL,RCVD_IN_RP_SAFE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on shymkent.ilbers.de X-Original-Sender: akarpovich@ilbers.de X-Original-Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of akarpovich@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=akarpovich@ilbers.de Precedence: list Mailing-list: list isar-users@googlegroups.com; contact isar-users+owners@googlegroups.com List-ID: X-Spam-Checked-In-Group: isar-users@googlegroups.com X-Google-Group-Id: 914930254986 List-Post: , List-Help: , List-Archive: , List-Unsubscribe: , X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= From: Uladzimir Bely This option allows to set ISAR_PREFETCH_BASE_APT to "0" or "1" and choose between old and new base-apt behaviour. Docker image kas uses should have "python3-apt" preinstalled in order to have new functionality working. Signed-off-by: Uladzimir Bely --- kas/opt/Kconfig | 12 ++++++++++++ kas/opt/prefetch-base-apt.yaml | 9 +++++++++ 2 files changed, 21 insertions(+) create mode 100644 kas/opt/prefetch-base-apt.yaml diff --git a/kas/opt/Kconfig b/kas/opt/Kconfig index bc4ed997..316b755a 100644 --- a/kas/opt/Kconfig +++ b/kas/opt/Kconfig @@ -213,3 +213,15 @@ config KAS_INCLUDE_USE_DRACUT string default "kas/opt/dracut.yaml" depends on USE_DRACUT + +config PREFETCH_BASE_APT + bool "Prefetch base-apt repo" + default y + help + This makse Isar always take packages from base-apt repository where they + are prefetched by deb-repo script before requested. + +config KAS_INCLUDE_PREFETCH_BASE_APT + string + default "kas/opt/prefetch-base-apt.yaml" + depends on PREFETCH_BASE_APT diff --git a/kas/opt/prefetch-base-apt.yaml b/kas/opt/prefetch-base-apt.yaml new file mode 100644 index 00000000..0cbdb93f --- /dev/null +++ b/kas/opt/prefetch-base-apt.yaml @@ -0,0 +1,9 @@ +# This software is a part of Isar. +# Copyright (C) 2024 ilbers GmbH + +header: + version: 14 + +local_conf_header: + prefetch-base-apt: | + ISAR_PREFETCH_BASE_APT = "1" From patchwork Fri Oct 2 07:01:13 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: akarpovich X-Patchwork-Id: 5522 Return-Path: Received: from shymkent.ilbers.de ([unix socket]) by shymkent (Cyrus 2.5.10-Debian-2.5.10-3+deb9u2) with LMTPA; Fri, 02 Oct 2026 09:05:13 +0200 X-Sieve: CMU Sieve 2.4 Received: from mail-ed1-f63.google.com (mail-ed1-f63.google.com [209.85.208.63]) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPS id 69275Aw3004062 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Fri, 2 Oct 2026 09:05:11 +0200 Received: by mail-ed1-f63.google.com with SMTP id 4fb4d7f45d1cf-6afa187d5c3sf285695a12.2 for ; Fri, 02 Oct 2026 00:05:11 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1790924705; cv=pass; d=google.com; s=arc-20260327; b=r7VeDfifuBoEaN5smIRncHLhpfbuXRn5u4kjdb0/ji3+A9a7ZiNnsPcC4nbcaQdaqK kldjW5z223w3udda/cobDUrntt9TRETHSNRg+tRvp2jyQRe6f3otUxPBbKVieoN/TjZw qwNTLesRH6FFPljnQDxE1r2duOCsAScOOutOUduEdQi9B+E8pSBC8KJlEf9bsogqWPam 18sjZkdROoWD9559TvzvtYU2KEQCZISYgOB9QTX6+ETaIuZZkI6eMv8SWoW9nNn8Bw2o v7o3Zc2jbuceATWz9jx/PXrN6PfhxtG/2+SFrBGmOO7z6hFmOpMmj5Zrd8J9yidfpHr+ WkKw== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20260327; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:dkim-signature; bh=ni6py7xtvaJX13Ji8eiQL/Ywbc5EUHDgmhZ0V7quwfA=; fh=j4mH36m8t395k29dJbyK/C89ATKgEBiRYsszoEOBBHY=; b=jY8pzqNtjNSz51kO289LAZ5W7tgreVIiNx6pPHFdM95nYKHZtWWrZHLqjLe5vgVn9D oBeqYAfgrJadaya3KKXCahY9zdTSrZPFvj4nKGKzE2zHSnkERYAsr7onCkXoIKQlszYo Z0Su0LtsRWq51SeoIRtk2/Vlubt9ZLLxzg66TvFEzNgssiFSjFSrpR1ItbsMoTZnMtIH RSQsG8a7nvqfepo4qwydCC6gbjYxrJYRjiq9/VhsLPg3tOVNWWtyJ4re2wUR4IJ5dfwZ 8QID82maBJEHDVoLatsYNhSyZdDpEWQ6f35Urafe8iXqaVL4ARVQuDFT3BK4dela2veZ cmKw==; darn=isar-build.org ARC-Authentication-Results: i=2; gmr-mx.google.com; spf=pass (google.com: domain of akarpovich@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=akarpovich@ilbers.de DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlegroups.com; s=20251104; t=1790924705; x=1791529505; darn=isar-build.org; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:content-type :x-original-authentication-results:x-original-sender:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from:sender :from:to:cc:subject:date:message-id:reply-to:content-type; bh=ni6py7xtvaJX13Ji8eiQL/Ywbc5EUHDgmhZ0V7quwfA=; b=mX+3Fqqm9jmV7zka4CHi88Gyv4sKbV5yEkU/xTY59nlSJXMggBAK9HAa9h5bYx2LQJ vy0UahEJf+kXaAe7A+/aE5zGOi8DTxwdKnMB+OY+uvPjBpKCmawNlbtA58tMkcx30FCB 5Cyu3t5tDVDWUKkWRHxl52j8W8lh82sFnFT/sSA6aZSleF1nQMq6dB8HjVds4o1seiLm 2El9LIjUo9wYRekuYgziQ0vVqv4Ke7ICH8gsT4MdWKIko5OU0ScJKq8jc7c0ZmMkKSvD zlmt9WRLQ3/bMt74OFCKITMkZaI2cURMhf6/vdh8WVSD8MiuZA2LAdARUMcfg9fOvDcv /l/Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20260707; t=1790924705; x=1791529505; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :x-spam-checked-in-group:list-id:mailing-list:precedence :content-type:x-original-authentication-results:x-original-sender :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:x-beenthere:x-gm-message-state:sender:from:to:cc:subject:date :message-id:reply-to:content-type; bh=ni6py7xtvaJX13Ji8eiQL/Ywbc5EUHDgmhZ0V7quwfA=; b=DtTthF6qSweODt2JdBdH8tVAsWlOK9lZnaVbtV7NsmcBmYce1VYCmMycI8LyqJg9Q0 XNmk+SLM1SzkxIk3dvKVOAIvo99Jsrp7T3MFO9RQHhWP1mdtvwQg16p2pMvpNHDRerr8 zvkyzfK8rkCSuIY6FVI67Gte5OirkNxH3pQCDye7Yg9GDgoTlEsHUqYPjQQRtc0h+QxA 3fpBrfd+bYgx2RytQtywrg33aIC5pArLPFCTtcdDeTzsRh6039ncXKvNTxyg5p6jMcRr H0XlrJ4cvZO5N9UNM40uOjNzAiqgRTvSgIuri2gDobjg5IAz9FdpnkwSGAvZru4X9Fui vI3g== Sender: isar-users@googlegroups.com X-Forwarded-Encrypted: i=2; AKwUvBzA6m2jG7jm0iIu62pcWoT2ywKM07WW6aDKCroie7ncbgKOBkSkqYdX9Gul1IITDIdQ6fw+39M=@isar-build.org X-Gm-Message-State: AFq9FYKWeQIpApHwE797nNa0DuQdOrMCAuV+8uJTcNeLGb6v5lQ62Q4s yS/xahIRXZCFPQxFDuL40RpRk0H26+KVnVBVPoI2MIv33QF74zgSZLiZ X-Received: by 2002:a05:6402:44d1:b0:6a9:850d:bb38 with SMTP id 4fb4d7f45d1cf-6af9e2e3be7mr820966a12.10.1790924705333; Fri, 02 Oct 2026 00:05:05 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com; h="ATskLdeROrUu68B/N/yDOTO8eorcFLZsVJEGy80tBKuC0R9DEw==" Received: by 2002:a05:6402:3111:b0:6aa:ca1c:540a with SMTP id 4fb4d7f45d1cf-6af9d8acad2ls571593a12.1.-pod-prod-07-eu; Fri, 02 Oct 2026 00:05:03 -0700 (PDT) X-Received: by 2002:a17:907:6e87:b0:c2e:3124:e9c0 with SMTP id a640c23a62f3a-c2e4aac5d08mr138593666b.17.1790924703329; Fri, 02 Oct 2026 00:05:03 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1790924703; cv=none; d=google.com; s=arc-20260327; b=suUzIwJDWp1FRZ9QMPJWuCFkRrs7LEapY2ak/u57VApt4U93vS37ws75jeMZ9sMwGi 3sm53tpBncxGig0wHvXPDM1bctMMZxcaP8z1Nx8WJJvg7/2eqnrV4mid03P27BVwTbAm Vn36GKSTy23iO9TMvsuO13pzEbvtsOIHgNi/p0Nspco8GL6+wgPgLKzw3JKI8w0JA99j nUBoGKbtXi1ygV8KVY0OBry4C07g6FfuWMAY5RxpRkO83oyXXbNKWOSNyftwrpGkvos6 Rhd4hLAUx9W/aAl0ukIfo8mpUCyurh6SIetE5ki0FRV1SVSyCHFh8AsMB8KVHddpJ+8y wynQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20260327; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=PjNepT01DSLZ1RI83NbCFLjc5ydikwGDzNdqAc59JuM=; fh=2ALxNboLzDt503PhX56nFQ4pQzOkkVjVTCOSpqCio6o=; b=NyBq6DgPYHZwehZX+bqpIKgKIbUZHblr8TL9lvRSy6sWNYpTjMOs16eHq0kBOMCOOs fJAEpQ7siKRcOfeD4Q90UU2JicgtFind5rIDbrCIPoZEJI2eWSvHMEH3kXJlZqgblo2M jSrcCQjMISZztdmW4Xe+mEo3CWHHgBICkelZ3eLm0O7kW3Dcbd/zqBhH68mEjo0SQKXD Q9rB77mgh6Shj3U85jLiRfml1kK+mZX1WN7pckEG5mGO+AG/pmbreLhWzyDPdwIme/AT zQz/ne+NnBfNdhzpksQ3fiup4rpuss8cvrlWKwQm4SAaQKKdthNLhBDlzID7LiHQEEtq eIzQ==; dara=google.com ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of akarpovich@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=akarpovich@ilbers.de Received: from shymkent.ilbers.de (shymkent.ilbers.de. [85.214.156.166]) by gmr-mx.google.com with ESMTPS id a640c23a62f3a-c2e4ca7c573si2580466b.2.2026.10.02.00.05.03 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 02 Oct 2026 00:05:03 -0700 (PDT) Received-SPF: pass (google.com: domain of akarpovich@ilbers.de designates 85.214.156.166 as permitted sender) client-ip=85.214.156.166; Received: from laptop-0xxx (62.122.174.6.serverel.net [62.122.174.6] (may be forged)) (authenticated bits=0) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPSA id 69274a1p003799 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 2 Oct 2026 09:05:02 +0200 From: akarpovich To: isar-users@googlegroups.com Cc: Uladzimir Bely Subject: [v11 10/12] ci_build.sh: Install python3-apt if not installed Date: Fri, 2 Oct 2026 09:01:13 +0200 Message-ID: <20261002070335.26526-11-akarpovich@ilbers.de> X-Mailer: git-send-email 2.53.0 In-Reply-To: <20261002070335.26526-1-akarpovich@ilbers.de> References: <20261002070335.26526-1-akarpovich@ilbers.de> MIME-Version: 1.0 X-Spam-Status: No, score=-4.6 required=5.0 tests=DKIMWL_WL_MED,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_EF,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,RCVD_IN_DNSWL_BLOCKED,RCVD_IN_MSPIKE_H2, RCVD_IN_RP_CERTIFIED,RCVD_IN_RP_RNBL,RCVD_IN_RP_SAFE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on shymkent.ilbers.de X-Original-Sender: akarpovich@ilbers.de X-Original-Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of akarpovich@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=akarpovich@ilbers.de Precedence: list Mailing-list: list isar-users@googlegroups.com; contact isar-users+owners@googlegroups.com List-ID: X-Spam-Checked-In-Group: isar-users@googlegroups.com X-Google-Group-Id: 914930254986 List-Post: , List-Help: , List-Archive: , List-Unsubscribe: , X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= From: Uladzimir Bely This is mostly related to gitlab CI that migth use an image without preinstalled python3-apt. Also, make system python packages available in virtualenv. Signed-off-by: Uladzimir Bely --- scripts/ci_build.sh | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/scripts/ci_build.sh b/scripts/ci_build.sh index 241ff8c8..71f62111 100755 --- a/scripts/ci_build.sh +++ b/scripts/ci_build.sh @@ -19,7 +19,7 @@ if ! command -v avocado > /dev/null; then sudo apt-get update -qq sudo apt-get install -y virtualenv rm -rf /tmp/avocado_venv - virtualenv --python python3 /tmp/avocado_venv + virtualenv --python python3 /tmp/avocado_venv --system-site-packages # shellcheck source=/dev/null source /tmp/avocado_venv/bin/activate pip install setuptools==81.0.0 @@ -138,6 +138,12 @@ if echo "$TAGS" | grep -Fqive "-startvm"; then fi fi +# install python3-apt +if [ ! -f /usr/share/doc/python3-apt/copyright ]; then + sudo apt-get update -qq + sudo apt-get install -y python3-apt +fi + # Provide working path mkdir -p .config/avocado cat < .config/avocado/avocado.conf From patchwork Fri Oct 2 07:01:14 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: akarpovich X-Patchwork-Id: 5523 Return-Path: Received: from shymkent.ilbers.de ([unix socket]) by shymkent (Cyrus 2.5.10-Debian-2.5.10-3+deb9u2) with LMTPA; Fri, 02 Oct 2026 09:05:13 +0200 X-Sieve: CMU Sieve 2.4 Received: from mail-lj1-f189.google.com (mail-lj1-f189.google.com [209.85.208.189]) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPS id 69275CrI004202 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Fri, 2 Oct 2026 09:05:12 +0200 Received: by mail-lj1-f189.google.com with SMTP id 38308e7fff4ca-3a46cc236d2sf4451931fa.2 for ; Fri, 02 Oct 2026 00:05:12 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1790924707; cv=pass; d=google.com; s=arc-20260327; b=sHoOuOPI/V+PwpzMDW56JNXECCm0RVfIxZ9CtBJeRu3f7Y7UvUry5Fky0RTKxEDgCz fiBCj+ejGdH3vOBDDcgEgeaTe89U/udKQFNwGXl4nY6QqM9RLKqrDBNFfng96a0aIzKy HNn7EmigkamKIN1UumOozeZHzQlOx5VHbQDp/UtvTMYHolODsLvz/v/W7GnTf0xrEiZf hudN0jOchDgjlo08qJWMLsw4KA6RE7sMWTADRMoGhh7jojCl7gb612pWxj3X5Oje0neP qUPvAzG627cBEX8CKzrA2fkPA96FZ4bVmC+VvRE4HkivRERuzecBmxknYeqv5a4/00ZN 6qFw== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20260327; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:dkim-signature; bh=yIpwCR/ijoFVO9gCHFrI2puMvUZbZUDOi1NV5qsBTVc=; fh=ZyMp8RiiSDF2yp9QM11dUIgh86S6zj5gd3K8//2OdOc=; b=pWnRpKrJXOdGQhA6kx+0eB0GRBymcjsRb60WaD1Sb9XhsXY+VS3r4cvIMW6XoUbSIZ VRUf8FA37l8/jj+4tN0DLHTyvToop8THYKcv1glfZlMvYuCthkqxlm43EaI56nw1VXgB 6mMH8rUNP0KLeF4JoVx6O91WTR6kxtHBGWxr4zN6V4NdalglI0TP9l2jhhshemX7aDzA Do2Q5Vls6h2mSN7rftyjDJTFkqhTtZov0kToKK8cmjhkiH8Gm6gMsgWKvHSr+RQ5DrvA n/JLrgEmtAFqWnkg5rAxNWhS0fe4aE/ZDREzc9UJ67ylzTNAyPs3SO4KR76ARJWRtO3R dDQg==; darn=isar-build.org ARC-Authentication-Results: i=2; gmr-mx.google.com; spf=pass (google.com: domain of akarpovich@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=akarpovich@ilbers.de DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlegroups.com; s=20251104; t=1790924707; x=1791529507; darn=isar-build.org; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:content-type :x-original-authentication-results:x-original-sender:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from:sender :from:to:cc:subject:date:message-id:reply-to:content-type; bh=yIpwCR/ijoFVO9gCHFrI2puMvUZbZUDOi1NV5qsBTVc=; b=CTH4WG4WbeyIclJ1jYWRq2jQU7bLmnVaVqp6VEodINoT6hMEDSYfcmuBkIgARsKPkD Y73zej4AOv8QjIO9H+B1C1hEVd6/1xo8BCWHaaWDXggXwjLteMX3JJI+ASdgW1VeAwYa EynzcQawGW4gx6xNU5EdOwx3MauMYXnLRa100MmAZzwCx5pgp2nENMF0fLv56FHgQefy z++ag3sR3gFYVEnfMgjllo7nbLYJSUehGUpIi610wEJqpT9hMmq5UTPOTSm+V7rFCux6 hXq9zQuzBG1+vbxtDgIpN/5TIShdisa6iYRptzL4wbzp7/cjrC9v/TLAFsJ/jf31J2mh pnwQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20260707; t=1790924707; x=1791529507; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :x-spam-checked-in-group:list-id:mailing-list:precedence :content-type:x-original-authentication-results:x-original-sender :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:x-beenthere:x-gm-message-state:sender:from:to:cc:subject:date :message-id:reply-to:content-type; bh=yIpwCR/ijoFVO9gCHFrI2puMvUZbZUDOi1NV5qsBTVc=; b=fqOB8NNf1WMPs3+MGP3EjtGdINBqq0Cc26Rs48XOEV7O5ip9Joh7ZX8UhP1lZrPCf5 u3KxOeQQxq6bCvuuAqroPIRqJsFxJqYuorxwCz39Kyh0EHc4MPdabi0lC8BDEtVNkY+G gxBgfttTHx5jMCio1IEHmTb6iYhCSAbnkm+U1HmuA5sEbS3xIQo9XfR+nUMg9C/XSGGw NT/whGK4awmFnJFLgo1n6DQN9udyTT+6Z1raxOUCm2a4WXqe5tJ9omdeBiCcfgCP1ucv lfd613rx6mykxRjNShv0+AV/0YWReUPYk9ALNhGC2mzHBXji0vnwysbk5oZbe/uLN+uu g8yg== Sender: isar-users@googlegroups.com X-Forwarded-Encrypted: i=2; AKwUvBz/nX7BrPlH02ckM28iydJKjGXkMAJIyUe9dIcpWHg0s510NBiYbuJYTXra0jO1VDCcQuw4Tr8=@isar-build.org X-Gm-Message-State: AFq9FYLWHqN8Mdn74iLduAjosNI1MXmcI35IjAU/Y9lpmBrStYY3g0OW Af0T0fzPEuoKwX8GWFZRvdF793ISLzzSI1pJhHhIOb5FaBKtlLsHB9qm X-Received: by 2002:a05:651c:1b84:b0:3a6:629e:447c with SMTP id 38308e7fff4ca-3a8827de31amr3189791fa.1.1790924706865; Fri, 02 Oct 2026 00:05:06 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com; h="ATskLdfS9remxdMpfolQ+SsUihCTw8FH9BcfXVL9+KngO/qb6w==" Received: by 2002:a2e:9dd7:0:b0:3a6:4926:b36c with SMTP id 38308e7fff4ca-3a8a31cfb9fls1276041fa.0.-pod-prod-09-eu; Fri, 02 Oct 2026 00:05:04 -0700 (PDT) X-Received: by 2002:a2e:b88b:0:b0:3a7:69bd:80ad with SMTP id 38308e7fff4ca-3a8877e829fmr3863981fa.6.1790924704431; Fri, 02 Oct 2026 00:05:04 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1790924704; cv=none; d=google.com; s=arc-20260327; b=Udng/A3n+91xN0/wNWFEGQgaPV/Py/SLGDlL0uFwTWRsOVvevQPwJus7KVHKS1j46g GtDyM0Hgk1yRbnNgiUCBgWXBJHVI5G2HLmy66n01NGhDlbiqbjozOLaAmj3qShPww5Jl uy3yA+ArSu7Ucq9NVZViJs3C/WDFuwtrxMtbv4D30jKOpZMu55cj+dI1YxWnqrNWObDU TsdR/zp9DBcKxYjX4b795S6ySKOUtJwdEi5oGTRzF9Rvi1v0lBBxex2AC6B0h+AEhVOW l8EZIIuwDiP72iO46c0WbYwrOUSXtv7yu7Br/FHUs/oD99dRNfnuk6+h3FDzvSCbC9tB 1Wrw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20260327; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=uoNsvHCWo0EqXpjLKWpWuF6AtgPN7r4tHaHy/IoTbAI=; fh=u+3kXEMzY4uVhDBiujW2QUo7NnpMLrGEgMQBYpCTRN4=; b=BhvIZklSe76V/3PqC0f1tKXGdBIwwNrOAFZb9+R7Xw1aMo3OtSg/lX2c0TDts9vciJ 9TWQyqx9aX8aG5E6r3t336xGj4Eg2uoFo4g/1ZsSVGsF3M11sk3HtpAV2dwVivA7n44O MRdZrkK+rn4yBh6WpF8w90zSRdO8V2XFzKwOrt2okiYOiiSYSEnUaUVwUo2ldskyTMuA oLEmSlribOSGZJ5luo9ThGspyD+/PZxV1HQF18ulNqAMHC90dm46zuvHAIq6MVXeHy3i 550GDGsuLnuVFd8YsBcKJo50UEtTMPeb6z1rBCwfrNcokR7y1i5vxO3rTdm6+n5jh3bB FAgw==; dara=google.com ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of akarpovich@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=akarpovich@ilbers.de Received: from shymkent.ilbers.de (shymkent.ilbers.de. [85.214.156.166]) by gmr-mx.google.com with ESMTPS id 38308e7fff4ca-3a87e339376si264021fa.8.2026.10.02.00.05.04 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 02 Oct 2026 00:05:04 -0700 (PDT) Received-SPF: pass (google.com: domain of akarpovich@ilbers.de designates 85.214.156.166 as permitted sender) client-ip=85.214.156.166; Received: from laptop-0xxx (62.122.174.6.serverel.net [62.122.174.6] (may be forged)) (authenticated bits=0) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPSA id 69274a1q003799 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 2 Oct 2026 09:05:03 +0200 From: akarpovich To: isar-users@googlegroups.com Cc: Aliaksei Karpovich Subject: [v11 11/12] debrepo: Add override for reprepro Date: Fri, 2 Oct 2026 09:01:14 +0200 Message-ID: <20261002070335.26526-12-akarpovich@ilbers.de> X-Mailer: git-send-email 2.53.0 In-Reply-To: <20261002070335.26526-1-akarpovich@ilbers.de> References: <20261002070335.26526-1-akarpovich@ilbers.de> MIME-Version: 1.0 X-Spam-Status: No, score=-4.6 required=5.0 tests=DKIMWL_WL_MED,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_EF,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,RCVD_IN_DNSWL_BLOCKED,RCVD_IN_MSPIKE_H2, RCVD_IN_RP_CERTIFIED,RCVD_IN_RP_RNBL,RCVD_IN_RP_SAFE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on shymkent.ilbers.de X-Original-Sender: akarpovich@ilbers.de X-Original-Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of akarpovich@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=akarpovich@ilbers.de Precedence: list Mailing-list: list isar-users@googlegroups.com; contact isar-users+owners@googlegroups.com List-ID: X-Spam-Checked-In-Group: isar-users@googlegroups.com X-Google-Group-Id: 914930254986 List-Post: , List-Help: , List-Archive: , List-Unsubscribe: , X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= From: Aliaksei Karpovich It fixes an issue when bootstrap image in case of offline build is different than online. It happens because some packages has different Priority field because of override. F.e. libtext-wrapi18n-perl in trixie: - inside deb package: 'Priority: required' - inside Packages from deb repo: 'Priority: optional' Solution: download the override file from repo and pass it to reprepro. Signed-off-by: Aliaksei Karpovich --- scripts/deb-repo | 97 +++++++++++++++++++++++++++++++++++++++++++----- 1 file changed, 88 insertions(+), 9 deletions(-) diff --git a/scripts/deb-repo b/scripts/deb-repo index 23de0461..99913fb4 100755 --- a/scripts/deb-repo +++ b/scripts/deb-repo @@ -89,6 +89,8 @@ import shutil import subprocess import pickle import urllib.parse +import urllib.request +import gzip import apt_pkg import apt.progress.base @@ -96,8 +98,38 @@ import apt.progress.base REPREPRO_TIMEOUT = 1200 +def parse_aptsources_list_line(source_list_line): + import re + + s = source_list_line.strip() + + if not s or s.startswith("#"): + return None + + type, s = re.split(r"\s+", s, maxsplit=1) + if type not in ["deb", "deb-src"]: + return None + + options = "" + options_match = re.match(r"\[\s*(\S+=\S+(?=\s))*\s*(\S+=\S+)\s*\]\s+", s) + if options_match: + options = options_match.group(0).strip() + s = s[options_match.end():] + + source, s = re.split(r"\s+", s, maxsplit=1) + + if s.startswith("/"): + suite = "" + else: + suite, s = re.split(r"\s+", s, maxsplit=1) + + components = " ".join(s.split()) + + return [type, options, source, suite, components] class DebRepo(object): + BOOTSTRAP_LIST = "/etc/apt/sources.list.d/bootstrap.list" + class DebRepoCtx(object): def __init__(self, workdir): self.distro = "debian" @@ -181,9 +213,9 @@ class DebRepo(object): with open(f"{self.workdir}/var/lib/dpkg/status", "w"): pass - os.makedirs(f"{self.workdir}/etc/apt/sources.list.d", exist_ok=True) + srcfile = f"{self.workdir}{self.BOOTSTRAP_LIST}" + os.makedirs(os.path.dirname(srcfile), exist_ok=True) - srcfile = f"{self.workdir}/etc/apt/sources.list.d/bootstrap.list" if aptsrcsfile and os.path.exists(aptsrcsfile): shutil.copy(aptsrcsfile, srcfile) else: @@ -201,13 +233,60 @@ class DebRepo(object): def create_repo_dist(self): conf_dir = f"{self.ctx.repodir}/{self.ctx.distro}/conf" os.makedirs(conf_dir, exist_ok=True) - if not os.path.exists(f"{conf_dir}/distributions"): - with open(f"{conf_dir}/distributions", "w") as f: - f.write(f"Codename: {self.ctx.codename}\n") - f.write( - "Architectures: " - "i386 armhf arm64 amd64 mipsel riscv64 source\n") - f.write("Components: main\n") + override_fname = "override" + self.create_override(f"{conf_dir}/{override_fname}") + with open(f"{conf_dir}/distributions", "w") as f: + f.write(f"Codename: {self.ctx.codename}\n") + f.write( + "Architectures: " + "i386 armhf arm64 amd64 mipsel riscv64 source\n") + f.write("Components: main\n") + f.write(f"DebOverride: {override_fname}\n") + + def create_override(self, override_file): + deb_override_name = f"override.{self.ctx.codename}.main" + deb_override_file = f"{self.workdir}/tmp/{deb_override_name}" + if (not os.path.exists(deb_override_file) or + os.path.getsize(deb_override_file) < 1): + # Extract source URL + srcfile = f"{self.workdir}{self.BOOTSTRAP_LIST}" + with open(srcfile, "r") as f: + for line in f: + entry = parse_aptsources_list_line(line) + if entry: + source_url = entry[2] + # Download override file + self.fetch_file(f"{source_url}/indices/{deb_override_name}") + if (os.path.exists(deb_override_file) and + os.path.getsize(deb_override_file) > 1): + break + # Download packet override file + self.fetch_file(f"{source_url}/indices/{deb_override_name}.gz") + # Unpack override file + with gzip.open(f"{deb_override_file}.gz", "r") as f_in, open(deb_override_file, "wb") as f_out: + for line in f_in: + f_out.write(line) + break + + with open(override_file, "w") as f_out: + # Unpack deb override file + with open(deb_override_file, "r") as f: + # Convert deb override to reprepro override + for line in f: + fields = line.strip().split() + + if len(fields) < 3: + continue + + package = fields[0] + priority = fields[1] + section = fields[2] + + f_out.write(f"{package} Priority {priority}\n") + f_out.write(f"{package} Section {section}\n") + + if f_out.tell() < 1: + print("WARNING: override file is empty!") def apt_config(self, init, crossbuild): if not init and self.ctx.compatarch: From patchwork Fri Oct 2 07:01:15 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: akarpovich X-Patchwork-Id: 5524 Return-Path: Received: from shymkent.ilbers.de ([unix socket]) by shymkent (Cyrus 2.5.10-Debian-2.5.10-3+deb9u2) with LMTPA; Fri, 02 Oct 2026 09:05:14 +0200 X-Sieve: CMU Sieve 2.4 Received: from mail-lj1-f184.google.com (mail-lj1-f184.google.com [209.85.208.184]) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPS id 69275DJI004315 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Fri, 2 Oct 2026 09:05:13 +0200 Received: by mail-lj1-f184.google.com with SMTP id 38308e7fff4ca-3a650355e31sf32269031fa.2 for ; Fri, 02 Oct 2026 00:05:13 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1790924708; cv=pass; d=google.com; s=arc-20260327; b=kyZ2Gr+jFmmhXBBBc44RdVFqTu2OHAM5oYghBqHBI9i0oQnpeqpw4Qv5kgHwy3Fyvw VS5qojflZS9pwe73xf8lBgdpoAnq5EDsnc8PW0cBSOQTow6muPbAiA6YGHjOiic3k2FB t+adCAJl5eu1S5dpnofl6+LZN647bIoftV0gdk+zZ4py41ZYTvCV2Vy/4oy2OEPmyIdg x5+6d/Jv9W4336kq+ZGUO4bSsaX/ioDZYdfr9X5yEBNika4EO08/Fdyv3J8BKnzQnpPY 52pKQ8XyOAwcylRwHJDyoQ2olEcAeCLorh3azuzPYmvPhu09rOcJZcplxRGeF3Z8ijkU cifA== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20260327; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:dkim-signature; bh=H23WhCvncPJcJgz8V3YVtoXDJCDUpG/4PTiywi3EQ9E=; fh=XOzhN684iYRFU+umgz39suHheGxHJzse9U8aXypijiE=; b=e+LVfsuS+Sdh3cxgSngWLamaxoVpp2jxpjoqHw2HPPAMJXlqL8r0M1ueRUWy6iLO/A rb+6On0fnaml5uYi0afCzLZ0r1VOfkx8Ql3U7dR3oi3UJ49Z8GLhr9JRfxkpZCw9ms5U QeuLVNUJ6cfD2QDiDidPeOKlSaRWICDop/9ADx/xA2MJs9p8y7jAyPY1B7pFygkG0jYO JuAIXtlzlcNqYc0dylOsSmu7SZvHUMa4B1gstmB1DmeLkfcFNu5G2dUjEQXgwq/F4X0W Ovk81hyXjKOkCSOBLOfiUukfIlB/v1KBsneY2VR7Un/VxopCyInHE4Ge/CmlF5pmZ/c1 vrHQ==; darn=isar-build.org ARC-Authentication-Results: i=2; gmr-mx.google.com; spf=pass (google.com: domain of akarpovich@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=akarpovich@ilbers.de DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlegroups.com; s=20251104; t=1790924708; x=1791529508; darn=isar-build.org; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:content-type :x-original-authentication-results:x-original-sender:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from:sender :from:to:cc:subject:date:message-id:reply-to:content-type; bh=H23WhCvncPJcJgz8V3YVtoXDJCDUpG/4PTiywi3EQ9E=; b=GS5NaVPnt07UbGXRtNo8MeUteI8rh0XjXiQ8AgGjmX7Khq1i58FKKBmdupc7fEr/97 8eDcEiPDSX8mbyMRzVjsx03oVBjCq79mYRIAmkkBjHCV35mbE0z4QdI5JDVLe4RRZT1m Zynb0uRDDZnR8GLfmkf6dsEZGhfpxmW7s6yR7kEvsmonrl2k8YcSGqgZPvQeigqbFINy h/GbrIbdR76gpj/NKxH3dapLwlEqyibi5j0EHPvrMUFTljjvSBhO9IXQdvVsgiIH54// U+/PYcHQVm4WykY8yCq2gAyGUk4riVt4B0vem8vgwE65YDkSRvKN5WT4dH87BlLVOtRc R/qw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20260707; t=1790924708; x=1791529508; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :x-spam-checked-in-group:list-id:mailing-list:precedence :content-type:x-original-authentication-results:x-original-sender :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:x-beenthere:x-gm-message-state:sender:from:to:cc:subject:date :message-id:reply-to:content-type; bh=H23WhCvncPJcJgz8V3YVtoXDJCDUpG/4PTiywi3EQ9E=; b=f90C8kAyi83bIZGKAh3mOe4jpz6M44Fz10Z5BjS8bAWbx0CC2Z9iHaoaQM/7r1WkDi yS81fdERy4dOFHtgbU3Yfshb8r+pg4JZPsxS41NvLZq9A/NBl3yyVfz6SWiWIszwxrsU Miy1FWb7RnSOF4Xc7qgZcXzZdSvC/HNLi/c7YDydt32qC1te4ppMR6lHIKr33A/186Mz mEaJpdM+C2qASz1VTWQnVJ2x0OusJJFlD/ssTiwA/LTypLLDE06PpBTVR0ZbSD2y6lcw wnCjyGXcIIVHmzw/g64+yr+bOO3UxbBixUzHbfxy31UanYl+F5MsTyEFQxy2hcVwjv/N DYag== Sender: isar-users@googlegroups.com X-Forwarded-Encrypted: i=2; AKwUvBy/ZY59a84gGFUJ0x6uTGhVjLm510a2N/m+etZsDxtaHApNXYtHFdqxO60Tp2k7ZokPgQlv9zY=@isar-build.org X-Gm-Message-State: AFq9FYKpiw7pHypALKIvG2Pyijoj8kOwRNjEHjNpio3Wq+Xn+yOy7Wr+ LWureWbsd0pTm1bWZmPu8GZ4q4i/ZFkFkEeYDJetPuLhawf/zxd4yoEp X-Received: by 2002:a2e:a54c:0:b0:3a6:5b2b:c50e with SMTP id 38308e7fff4ca-3a88767d003mr3785801fa.4.1790924707811; Fri, 02 Oct 2026 00:05:07 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com; h="ATskLddKsjM6BYQejhJExq2DBIILUvydhtjU7XhnzI3Hsu+LTQ==" Received: by 2002:a2e:969a:0:b0:3a7:826e:ed20 with SMTP id 38308e7fff4ca-3a8a31d3cbals1443511fa.0.-pod-prod-03-eu; Fri, 02 Oct 2026 00:05:05 -0700 (PDT) X-Received: by 2002:a2e:a54c:0:b0:3a6:5b2b:c50e with SMTP id 38308e7fff4ca-3a88767d003mr3785411fa.4.1790924705354; Fri, 02 Oct 2026 00:05:05 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1790924705; cv=none; d=google.com; s=arc-20260327; b=TKGrA/mqq4FOUrSxdSWU+1K4X81zGl56C/kzmAh8EW5BIpiv7fKt5KrRmPjHlhagwm 3W+blulPnP4hzS8s2j0FlQvjVhoPuGSr0AAm0ox6+Pf8xwiuywTltd9gQo2lyS/bslis BUNBf0DkJ4MFeSxC/RXP2iQ1DCVQf0YNzx++Jah9GbLH+lB63VDzJbR7nHqRAWqTj0Se gloh3cp98TXtON9S9mz8y00bZvE5ut89tXcz87KhLzNGGyIHzHQ/nzqvJanKXcmBYNZs qwtmEd7U4dcuwOh9g8CP/4CVr5CvyGSh0X+d8l+41Jr2LLg28ophYksDkPeS3+2hfd6b NtoA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20260327; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=HuOZ5PN6Wpekt2ytHYMwznohMtmHTtqQOSRnVAs3Brc=; fh=u+3kXEMzY4uVhDBiujW2QUo7NnpMLrGEgMQBYpCTRN4=; b=DNJbrQdB7nO/2NHsCEFkbn4NDsS9i5gXFN4zBDa/bg1deYpPd8Gz9FludTY2ZVMNCX s6lrArylTYMdUEvd2DT1EYoZtLArg5z+2xW1x4NTr5/gpjS59Tzkpzrwz56tiBGr6VDA MAU9DcypK1Q8WB9CIwR/q/KMKS/MPMA13OI0pV+2J/UFczvr+y/U7dqWO16ndD22KNW0 S1O6/UgOHepMYnatOsghYDQeVah4TME6jSkWSZMSMomv+2J45bmgqtVLz/QN4L/0KbNh uUO52mUwYwkHhWQ/AwDJFWwIqRAaREAs9r6zjFfe7HG64gqmTPD5ps492k0oS0fQKtsT NEOg==; dara=google.com ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of akarpovich@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=akarpovich@ilbers.de Received: from shymkent.ilbers.de (shymkent.ilbers.de. [85.214.156.166]) by gmr-mx.google.com with ESMTPS id 38308e7fff4ca-3a87e3232fasi275811fa.7.2026.10.02.00.05.05 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 02 Oct 2026 00:05:05 -0700 (PDT) Received-SPF: pass (google.com: domain of akarpovich@ilbers.de designates 85.214.156.166 as permitted sender) client-ip=85.214.156.166; Received: from laptop-0xxx (62.122.174.6.serverel.net [62.122.174.6] (may be forged)) (authenticated bits=0) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPSA id 69274a1r003799 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 2 Oct 2026 09:05:03 +0200 From: akarpovich To: isar-users@googlegroups.com Cc: Aliaksei Karpovich Subject: [v11 12/12] base-apt: Move bullseye-mipsel in separate dir Date: Fri, 2 Oct 2026 09:01:15 +0200 Message-ID: <20261002070335.26526-13-akarpovich@ilbers.de> X-Mailer: git-send-email 2.53.0 In-Reply-To: <20261002070335.26526-1-akarpovich@ilbers.de> References: <20261002070335.26526-1-akarpovich@ilbers.de> MIME-Version: 1.0 X-Spam-Status: No, score=-4.6 required=5.0 tests=DKIMWL_WL_MED,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_EF,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,RCVD_IN_DNSWL_BLOCKED,RCVD_IN_MSPIKE_H2, RCVD_IN_RP_CERTIFIED,RCVD_IN_RP_RNBL,RCVD_IN_RP_SAFE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on shymkent.ilbers.de X-Original-Sender: akarpovich@ilbers.de X-Original-Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of akarpovich@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=akarpovich@ilbers.de Precedence: list Mailing-list: list isar-users@googlegroups.com; contact isar-users+owners@googlegroups.com List-ID: X-Spam-Checked-In-Group: isar-users@googlegroups.com X-Google-Group-Id: 914930254986 List-Post: , List-Help: , List-Archive: , List-Unsubscribe: , X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= From: Aliaksei Karpovich As the bullseye-mipsel has separate bullseye repo (archive) it leads to unresolving dependency (f.e. gnupg) in base-apt repo. It happens because mipsel satied on deb11u2 version (and dont't have secure update) then other move formward and have deb11u3 version. When the bullseye-mipsel is built in the same time with other arch targed (f.e. bullseye arm) the following dependency issue occurs: | The following packages have unmet dependencies: | gnupg : Depends: dirmngr (< 2.2.27-2+deb11u3.1~) but it is not going to be installed | Depends: dirmngr (>= 2.2.27-2+deb11u3) but it is not going to be installed | Depends: gnupg-utils (>= 2.2.27-2+deb11u3) but 2.2.27-2+deb11u2 is to be installed | Depends: gpg (>= 2.2.27-2+deb11u3) but 2.2.27-2+deb11u2 is to be installed | Depends: gpg-agent (>= 2.2.27-2+deb11u3) but 2.2.27-2+deb11u2 is to be installed It happens becasue (during arm build) gnupg_2.2.27-2+deb11u3_all.deb was fetched, but other dependencies (dirmngr, gnupg-utils) for mipsel have old deb11u2 version. Fix: separate bullseye-mipsel into separate repo in base-apt. Signed-off-by: Aliaksei Karpovich --- meta/conf/bitbake.conf | 5 +++-- meta/conf/distro/debian-bullseye.conf | 1 + 2 files changed, 4 insertions(+), 2 deletions(-) diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf index c8503ec6..97f200a0 100644 --- a/meta/conf/bitbake.conf +++ b/meta/conf/bitbake.conf @@ -99,8 +99,9 @@ REPO_ISAR_DIR = "${DEPLOY_DIR}/isar-apt/${DISTRO}/apt" REPO_ISAR_DB_DIR = "${DEPLOY_DIR}/isar-apt/${DISTRO}/db" # Base apt repository paths -REPO_BASE_DIR = "${DEPLOY_DIR}/base-apt/${DISTRO}/apt" -REPO_BASE_DB_DIR = "${DEPLOY_DIR}/base-apt/${DISTRO}/db" +REPO_BASE_DIR_NAME ?= "${DISTRO}" +REPO_BASE_DIR = "${DEPLOY_DIR}/base-apt/${REPO_BASE_DIR_NAME}/apt" +REPO_BASE_DB_DIR = "${DEPLOY_DIR}/base-apt/${REPO_BASE_DIR_NAME}/db" # Setup our default hash policy BB_SIGNATURE_HANDLER ?= "OEBasicHash" diff --git a/meta/conf/distro/debian-bullseye.conf b/meta/conf/distro/debian-bullseye.conf index c368dcaf..00d50df4 100644 --- a/meta/conf/distro/debian-bullseye.conf +++ b/meta/conf/distro/debian-bullseye.conf @@ -2,6 +2,7 @@ HOST_DISTRO_APT_SOURCES:mipsel ?= "conf/distro/${HOST_DISTRO}-mipsel.list" DISTRO_APT_SOURCES:mipsel ?= "conf/distro/${BASE_DISTRO}-${BASE_DISTRO_CODENAME}-mipsel.list" +REPO_BASE_DIR_NAME:mipsel = "${DISTRO}-mipsel" require debian-common.conf