From patchwork Sat Jan 27 07:42:53 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Uladzimir Bely X-Patchwork-Id: 3320 Return-Path: Received: from shymkent.ilbers.de ([unix socket]) by shymkent (Cyrus 2.5.10-Debian-2.5.10-3+deb9u2) with LMTPA; Sat, 27 Jan 2024 08:43:05 +0100 X-Sieve: CMU Sieve 2.4 Received: from mail-wm1-f61.google.com (mail-wm1-f61.google.com [209.85.128.61]) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPS id 40R7h4T5022948 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Sat, 27 Jan 2024 08:43:04 +0100 Received: by mail-wm1-f61.google.com with SMTP id 5b1f17b1804b1-40eb6ce1ddasf14574415e9.1 for ; Fri, 26 Jan 2024 23:43:04 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1706341378; cv=pass; d=google.com; s=arc-20160816; b=UXeKibaNvdqkF2ZPFq77SPAZAPx4t0+csidmehpfXqPf6EAgHugZjrxrV2MD7ZwDq5 Gq88JweU7hxYk6oG4m7QGZsCc896McUeL6/cLpo7AB4cE+DY5TUKNCtIKF5PHk3gjQY4 Re13aSusehTPcFFG1WgrUVWo+0tdtFl8JF109YRdvJgTYqqlK9AY3wmgmZmsLSPZ7/yu kv33tiEdlN69548Y2z4yiLNBrRP4jIX8PAlP8/p4YCcjAzEBc6RwUDwZEU1M2vovKo58 qRw4n45HP4K+Rg95IJX3ux+kAeGDLtXTDdzidHAm5VgiP6Eoiigvx99wa4r33ebMJnJY TVvg== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:mime-version:message-id:date :subject:to:from:sender:dkim-signature; bh=/U1LxQ74ODJLB8Uw1NLCRy91HnKy/6H1P2rE8EIv3SM=; fh=7tclEdh7YbwSQowgJ6LNq720O7H5HTEaqj22NJWRE2E=; b=pT0Y2k+Ob+Xr54/ucfQndLkfQ6VjUlpvB/xSQcEQCG2eqya2bjL0jJh7PkqP28wP4C Gt39gaSrqe57+YwnnvJ0nnbn+L9jdfXWDLi5FE7B+qdXLHAL08dNmZ3OpCOjGV3ndZZC xMYICrlxFjx/LcXwFcx3PYWXdQbPejpX/HiS8m4A8VpcKw7a/XKljuVukTJBsw1V9u2B B8X+LqU2xJQknod0ILKzE1GWJNTqOMVWHQMQCDkE8mcwVtxd4razKnHKkGqhBE5g8o4A FeG3xSrsqhaxjdX14eU1Dg1T8fCbl5iwE2B0EJrgUavzFWMQ6GQZD6pqAUTO1RObBIvA XATQ== ARC-Authentication-Results: i=2; gmr-mx.google.com; spf=pass (google.com: domain of ubely@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=ubely@ilbers.de DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlegroups.com; s=20230601; t=1706341378; x=1706946178; darn=isar-build.org; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:x-original-authentication-results :x-original-sender:mime-version:message-id:date:subject:to:from :sender:from:to:cc:subject:date:message-id:reply-to; bh=/U1LxQ74ODJLB8Uw1NLCRy91HnKy/6H1P2rE8EIv3SM=; b=Nu97b0YMQiX1MS1KDglCcwiqR1X7wCln5pGapFZUndXVEbmr5Q/p/s8WotgPf3RqXY eQIH3A3sjh9mOtFXRKGAEDcc2p2jebCMyLkb+DSvbx4DiEGH1IyV5Bdr5d5XDGcnkuFf EPC7c9TyWYVDu7zYxXiWmK57iohvWY8QWmu2PTM1GvYWsFHAH3jJmlapVA3yvFecvyjM wERPY5nxb8PrbsOFbVvAPOh+g43w8uZUNFk5t7eAVVWWHLewFRr9tkADixXNfUHV2Q35 MfWGTQYilPh4CnXYnLf6Sq1grXYfWktJRC/YkH7N3RwEaWisMv6xJhzVEg4W8DVJ38mM cFkg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1706341378; x=1706946178; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :x-spam-checked-in-group:list-id:mailing-list:precedence :x-original-authentication-results:x-original-sender:mime-version :message-id:date:subject:to:from:x-beenthere:x-gm-message-state :sender:from:to:cc:subject:date:message-id:reply-to; bh=/U1LxQ74ODJLB8Uw1NLCRy91HnKy/6H1P2rE8EIv3SM=; b=on464JFWVhzuVt1uwFe4EondIBf9mMO1Xs05H0RHFk5gbt2qXUxIv3RRTu6W4PsBcb zHl+2zFc4xaHmOmhRlXOE4PaRoCXTSUr5+qU6RLoo/yhpEQzC7OfMQpbXZOTa5RCAnpv 1jS2HyYbR9YWIjaFvdcWScIPpO0k3sf0wU5n3q2Ff2eaqpmfZxBpOsuOjvYIPwzNMrWE se2irpKdQANUApZYxGv7QubANal95wfY6CaKIB9LeYSZGi0n8XxYPtoQY6kcyDVFZomC sxdjSBuvlhte6FI8ylTESwJEHWVk4nYyZDdV/dxAYYj6HUNfA9+3Ftmtrkx1ngfRMQMN sZaA== Sender: isar-users@googlegroups.com X-Gm-Message-State: AOJu0Yz22pZVwikfw1RrIsIpR5OLA4d3VsNeKRuH4TsJFv6n9O2gSlwA f2RRiccBUwi3zs5jp5BhJeHglyLKQxCWxJF/wryvzMoz+JkwrFGb X-Google-Smtp-Source: AGHT+IE2Ax1VUYkJ4fzes9eE9EwNfP5JttzgBvvUzhAiLrM4wNeLYsTPUCU8uYlaB2DhqMxB714FDg== X-Received: by 2002:adf:f281:0:b0:337:c0f2:7cc8 with SMTP id k1-20020adff281000000b00337c0f27cc8mr456404wro.260.1706341377825; Fri, 26 Jan 2024 23:42:57 -0800 (PST) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a05:651c:1991:b0:2cc:defb:48b0 with SMTP id bx17-20020a05651c199100b002ccdefb48b0ls577603ljb.0.-pod-prod-08-eu; Fri, 26 Jan 2024 23:42:55 -0800 (PST) X-Received: by 2002:a2e:bc83:0:b0:2cf:48c9:1554 with SMTP id h3-20020a2ebc83000000b002cf48c91554mr780291ljf.0.1706341375070; Fri, 26 Jan 2024 23:42:55 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1706341375; cv=none; d=google.com; s=arc-20160816; b=zjJrrHpr9TDPOHd+L6rfwvzUPj23S9/Cc8LzZafpSlFSKE+REIHqfby/GgH/imdsR3 uIv/0BHHEt29ktVLjojZySG2z1q/QMgTXtB+Utq3i+1N7ZJ2tk13NtoqpDC48j4qtLrn i3Xxg1g90Eg/tp8rjdMjmdOGYYtK67lR4erLTdBEhQBl8E/cSGsWMNWf2AEzgerEntti CqWLbsfH0/X016s6qCb+6efQYOj6+q0QYNhaihy9ioEP35UM/m/Gm+CT8LS/4bwahuEC R7DzYfodSCDWDFvh4VHKBVk6JXQnHM8+myW3LbgFI8rtwBH/C7yRSVlt20WNRf/JhBsj nOCQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:message-id:date:subject:to :from; bh=mvOMseGLX5wrwvLFkCQm8mrPME33Yf9oW52dF4Cci58=; fh=7tclEdh7YbwSQowgJ6LNq720O7H5HTEaqj22NJWRE2E=; b=JgmpH7XLhFl9Q/PzNG0/9SlhnUP4xduzmlYHFzWuS/l7E1kIoHeTexe5zpmpSgRofw F5hmtUP7OmFVhh3HruurgVkt778ClbuhYIBYNAJ2BHFE6Lp1HjIVjGgjjoQTeQrkruD3 HM/5b48ieBhWjKJRCx6ToZZ4viUkmLfM6igQkH56D1S9/2r3bTqjzTYN69/qWPhUOJQ7 T6CAZhU817/fqlEr0s3qgUfpJX6MYUiNB0ikqjHhnFEjJVSayVWkfc8o3NHSnxpwHloz tkBxzxEjNe6iZFbTYehK4uEBzg3AQ0s0TTYMMTsvKmU4n5ssG6fRUrVlVHWCCIMFZqta MfAw== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of ubely@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=ubely@ilbers.de Received: from shymkent.ilbers.de (shymkent.ilbers.de. [85.214.156.166]) by gmr-mx.google.com with ESMTPS id v6-20020a2e9906000000b002ccc27fab8csi96964lji.7.2024.01.26.23.42.54 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 26 Jan 2024 23:42:54 -0800 (PST) Received-SPF: pass (google.com: domain of ubely@ilbers.de designates 85.214.156.166 as permitted sender) client-ip=85.214.156.166; Received: from azat.m.ilbers.de (host-80-81-17-52.static.customer.m-online.net [80.81.17.52]) (authenticated bits=0) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPSA id 40R7grj9022934 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Sat, 27 Jan 2024 08:42:53 +0100 From: Uladzimir Bely To: isar-users@googlegroups.com Subject: [PATCH v7] dpkg: Restore support for replacing pre-installed packages in sbuild-chroot Date: Sat, 27 Jan 2024 08:42:53 +0100 Message-Id: <20240127074253.24570-1-ubely@ilbers.de> X-Mailer: git-send-email 2.20.1 MIME-Version: 1.0 X-Spam-Status: No, score=-0.9 required=5.0 tests=DKIMWL_WL_MED,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_EF,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,RCVD_IN_MSPIKE_H2,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on shymkent.ilbers.de X-Original-Sender: ubely@ilbers.de X-Original-Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of ubely@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=ubely@ilbers.de Precedence: list Mailing-list: list isar-users@googlegroups.com; contact isar-users+owners@googlegroups.com List-ID: X-Spam-Checked-In-Group: isar-users@googlegroups.com X-Google-Group-Id: 914930254986 List-Post: , List-Help: , List-Archive: , List-Unsubscribe: , X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= From: Jan Kiszka During the migration to sbuild, support for using self-built packages in the build environment was lost if those were already part of the sbuild-chroot. This restores it by adding --apt-distupgrade to the sbuild call. But that is not enough because sbuild will only upgrade packages from already configured sources, not those specified via --extra-repository. We therefore have to switch back to configuring isar-apt during sbuild-chroot creation. As rootfs_configure_isar_apt configures the isar repo under /isar-apt, we bind-mount the one in /home/builder/${PN} to that folder. Another difference is that we now need to run apt-get update explicitly, but only for isar-apt. For apt fetching, we neither need nor want isar-apt to be available. Rebuilding self-generated apt packages is generally pointless as the needs are better addressed in the generating recipe itself. Exposing isar-apt to the fetch may furthermore lead to fetching a previously built source package of the same recipe, rather than pulling the external version. And because the sbuild-chroots are now left behind with isar-apt configured, the configuration in imager_run can be removed (credits to Srinuvasan Arjunan). Signed-off-by: Jan Kiszka Reviewed-by: Uladzimir Bely --- Changes in v7: Merged v6 with fix disabling isar-apt for do_apt_fetch meta/classes/dpkg-base.bbclass | 37 ++++++++++++------- meta/classes/dpkg.bbclass | 5 ++- meta/classes/image-tools-extension.bbclass | 13 ------- .../sbuild-chroot/sbuild-chroot.inc | 2 - 4 files changed, 27 insertions(+), 30 deletions(-) diff --git a/meta/classes/dpkg-base.bbclass b/meta/classes/dpkg-base.bbclass index 95ca3db8..86466b57 100644 --- a/meta/classes/dpkg-base.bbclass +++ b/meta/classes/dpkg-base.bbclass @@ -19,8 +19,6 @@ DEPENDS:append:riscv64 = "${@' crossbuild-essential-riscv64' if d.getVar('ISAR_C DEB_BUILD_PROFILES ?= "" DEB_BUILD_OPTIONS ?= "" -ISAR_APT_REPO ?= "deb [trusted=yes] file:///home/builder/${PN}/isar-apt/${DISTRO}-${DISTRO_ARCH}/apt/${DISTRO} ${DEBDISTRONAME} main" - python do_adjust_git() { import subprocess @@ -109,16 +107,24 @@ do_apt_fetch() { E="${@ isar_export_proxies(d)}" schroot_create_configs + session_id=$(schroot -q -b -c ${SBUILD_CHROOT}) + echo "Started session: ${session_id}" + schroot_cleanup() { + schroot -q -f -e -c ${session_id} > /dev/null 2>&1 schroot_delete_configs } trap 'exit 1' INT HUP QUIT TERM ALRM USR1 trap 'schroot_cleanup' EXIT + schroot -r -c ${session_id} -d / -u root -- \ + rm /etc/apt/sources.list.d/isar-apt.list /etc/apt/preferences.d/isar-apt for uri in "${SRC_APT}"; do - schroot -d / -c ${SBUILD_CHROOT} -- \ + schroot -r -c ${session_id} -d / -- \ sh -c 'mkdir -p /downloads/deb-src/"$1"/"$2" && cd /downloads/deb-src/"$1"/"$2" && apt-get -y --download-only --only-source source "$2"' my_script "${BASE_DISTRO}-${BASE_DISTRO_CODENAME}" "${uri}" done + + schroot -e -c ${session_id} schroot_delete_configs } @@ -221,8 +227,17 @@ def isar_export_build_settings(d): os.environ['DEB_BUILD_OPTIONS'] = isar_deb_build_options(d) os.environ['DEB_BUILD_PROFILES'] = isar_deb_build_profiles(d) +dpkg_schroot_create_configs() { + schroot_create_configs + sudo -s <<'EOSUDO' + sbuild_fstab="${SBUILD_CONF_DIR}/fstab" + fstab_isarapt="${WORKDIR}/isar-apt/${DISTRO}-${DISTRO_ARCH}/apt/${DISTRO} /isar-apt none rw,bind 0 0" + grep -qxF "${fstab_isarapt}" ${sbuild_fstab} || echo "${fstab_isarapt}" >> ${sbuild_fstab} +EOSUDO +} + python do_dpkg_build() { - bb.build.exec_func('schroot_create_configs', d) + bb.build.exec_func('dpkg_schroot_create_configs', d) try: bb.build.exec_func("dpkg_runbuild", d) finally: @@ -288,7 +303,7 @@ do_deploy_deb[lockfiles] = "${REPO_ISAR_DIR}/isar.lock" do_deploy_deb[dirs] = "${S}" python do_devshell() { - bb.build.exec_func('schroot_create_configs', d) + bb.build.exec_func('dpkg_schroot_create_configs', d) isar_export_proxies(d) isar_export_ccache(d) @@ -297,11 +312,9 @@ python do_devshell() { bb.build.exec_func('schroot_configure_ccache', d) schroot = d.getVar('SBUILD_CHROOT') - isar_apt = d.getVar('ISAR_APT_REPO') pkg_arch = d.getVar('PACKAGE_ARCH') build_arch = d.getVar('BUILD_ARCH') pp_pps = os.path.join(d.getVar('PP'), d.getVar('PPS')) - debdistroname = d.getVar('DEBDISTRONAME') install_deps = ":" if d.getVar('BB_CURRENTTASK') == "devshell_nodeps" else f"mk-build-deps -i \ --host-arch {pkg_arch} --build-arch {build_arch} \ @@ -310,15 +323,13 @@ python do_devshell() { termcmd = "schroot -d / -c {0} -u root -- sh -c ' \ cd {1}; \ - echo {2} > /etc/apt/sources.list.d/isar_apt.list; \ - echo \"Package: *\nPin: release n={3}\nPin-Priority: 1000\" > /etc/apt/preferences.d/isar-apt; \ - echo \"APT::Get::allow-downgrades 1;\" > /etc/apt/apt.conf.d/50isar-apt; \ - apt-get -y -q update; \ - {4}; \ + apt-get -y -q update -o Dir::Etc::SourceList=\"sources.list.d/isar-apt.list\" -o Dir::Etc::SourceParts=\"-\" -o APT::Get::List-Cleanup=\"0\"; \ + apt-get -y upgrade; \ + {2}; \ export PATH=$PATH_PREPEND:$PATH; \ $SHELL -i \ '" - oe_terminal(termcmd.format(schroot, pp_pps, isar_apt, debdistroname, install_deps), "Isar devshell", d) + oe_terminal(termcmd.format(schroot, pp_pps, install_deps), "Isar devshell", d) bb.build.exec_func('schroot_delete_configs', d) } diff --git a/meta/classes/dpkg.bbclass b/meta/classes/dpkg.bbclass index c699a84d..d1666f78 100644 --- a/meta/classes/dpkg.bbclass +++ b/meta/classes/dpkg.bbclass @@ -110,16 +110,17 @@ dpkg_runbuild() { DEB_SOURCE_NAME=$(dpkg-parsechangelog --show-field Source --file ${WORKDIR}/${PPS}/debian/changelog) DSC_FILE=$(find ${WORKDIR} -name "${DEB_SOURCE_NAME}*.dsc" -maxdepth 1 -print) - sbuild -A -n -c ${SBUILD_CHROOT} --extra-repository="${ISAR_APT_REPO}" \ + sbuild -A -n -c ${SBUILD_CHROOT} \ --host=${PACKAGE_ARCH} --build=${BUILD_ARCH} ${profiles} \ --no-run-lintian --no-run-piuparts --no-run-autopkgtest --resolve-alternatives \ --bd-uninstallable-explainer=apt \ - --no-apt-update \ + --no-apt-update --apt-distupgrade \ --chroot-setup-commands="echo \"Package: *\nPin: release n=${DEBDISTRONAME}\nPin-Priority: 1000\" > /etc/apt/preferences.d/isar-apt" \ --chroot-setup-commands="echo \"APT::Get::allow-downgrades 1;\" > /etc/apt/apt.conf.d/50isar-apt" \ --chroot-setup-commands="rm -f /var/log/dpkg.log" \ --chroot-setup-commands="mkdir -p ${deb_dir}" \ --chroot-setup-commands="ln -sf ${ext_deb_dir}/*.deb -t ${deb_dir}/" \ + --chroot-setup-commands="apt-get update -o Dir::Etc::SourceList=\"sources.list.d/isar-apt.list\" -o Dir::Etc::SourceParts=\"-\" -o APT::Get::List-Cleanup=\"0\"" \ --finished-build-commands="rm -f ${deb_dir}/sbuild-build-depends-main-dummy_*.deb" \ --finished-build-commands="[ -z \"\$(find ${deb_dir} -maxdepth 1 -name '*.deb' -print -quit)\" ] || cp ${CP_FLAGS} ${deb_dir}/*.deb -t ${ext_deb_dir}/" \ --finished-build-commands="cp /var/log/dpkg.log ${ext_root}/dpkg_partial.log" \ diff --git a/meta/classes/image-tools-extension.bbclass b/meta/classes/image-tools-extension.bbclass index 58799fcf..46bdf78b 100644 --- a/meta/classes/image-tools-extension.bbclass +++ b/meta/classes/image-tools-extension.bbclass @@ -44,19 +44,6 @@ imager_run() { distro="${HOST_BASE_DISTRO}-${BASE_DISTRO_CODENAME}" fi - # prepare isar-apt - schroot -r -c ${session_id} -d / -u root -- sh -c " \ - mkdir -p '/etc/apt/sources.list.d' - echo 'deb [trusted=yes] file:///isar-apt ${DEBDISTRONAME} main' > \ - '/etc/apt/sources.list.d/isar-apt.list' - - mkdir -p '/etc/apt/preferences.d' - cat << EOF > '/etc/apt/preferences.d/isar-apt' -Package: * -Pin: release n=${DEBDISTRONAME} -Pin-Priority: 1000 -EOF" - E="${@ isar_export_proxies(d)}" deb_dl_dir_import ${schroot_dir} ${distro} schroot -r -c ${session_id} -d / -u root -- sh -c " \ diff --git a/meta/recipes-devtools/sbuild-chroot/sbuild-chroot.inc b/meta/recipes-devtools/sbuild-chroot/sbuild-chroot.inc index 39ced80b..2f07de82 100644 --- a/meta/recipes-devtools/sbuild-chroot/sbuild-chroot.inc +++ b/meta/recipes-devtools/sbuild-chroot/sbuild-chroot.inc @@ -56,8 +56,6 @@ SBUILD_CHROOT_DIR = "${WORKDIR}/rootfs" ROOTFSDIR = "${SBUILD_CHROOT_DIR}" ROOTFS_PACKAGES = "${SBUILD_CHROOT_PREINSTALL}" -# We don't need /etc/apt/sources.list.d/isar-apt.list' while it's handled by sbuild -ROOTFS_CONFIGURE_COMMAND:remove = "rootfs_configure_isar_apt" ROOTFS_POSTPROCESS_COMMAND:remove = "rootfs_cleanup_isar_apt" # Do not cleanup base-apt