From patchwork Wed Nov 6 08:21:15 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anton Mikanovich X-Patchwork-Id: 3900 Return-Path: Received: from shymkent.ilbers.de ([unix socket]) by shymkent (Cyrus 2.5.10-Debian-2.5.10-3+deb9u2) with LMTPA; Wed, 06 Nov 2024 09:21:41 +0100 X-Sieve: CMU Sieve 2.4 Received: from mail-lf1-f62.google.com (mail-lf1-f62.google.com [209.85.167.62]) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPS id 4A68Ld4Q017098 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Wed, 6 Nov 2024 09:21:40 +0100 Received: by mail-lf1-f62.google.com with SMTP id 2adb3069b0e04-539fbbadca7sf6077410e87.3 for ; Wed, 06 Nov 2024 00:21:40 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1730881293; cv=pass; d=google.com; s=arc-20240605; b=JnII8UflU0W5bzCEQBk8dYwBlDKhwh2av9SnoZnHMBIUFxYa+U/NZaUYbbh4qateuf P67Tey1SPhyI9GxjwcUuAepASgFTbKA3GB/zN77tyYR3/2hEVTFzZc0EoVxBYpyHixuc qjAS5Olfpi2CUmhaUwl00mdNewklZ3QgLAbBU7kiZCHg67WyxxEpYN1EXqKnNnBW9qdb U43duN9Pj8EtXp7QGwSfrapZpa1+Yv+u+g8o0RUmwtOqK3bcoiguSizXCYTQHNPILK9O cV/ZZPL2fL2Ph6GZRNE/MGrPLh39LHoebcwwmQ6NS4/zmy6wCiA4srL++2llAbxRHWZH Qmxg== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20240605; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:dkim-signature; bh=51vSoTxZEcpv/ff8baGrTTAQ/e/M5ucuALuXAsu+Jhs=; fh=WwEAQWaYJFdaw/vUOgvVmj1DMd7QDrvBJw+9be7emRA=; b=aS+ZUdeF2m3ApPYGoOQe6AvEuStj7qoQ+ILOgTBLbXBkEQk8khvN//trqgTkUlx4sk wqQNIDW65YFU8xZPHQOcZtQ2yS62J5vLq/83MxLdil2JuECzLsaeTZGUJfgvafc1AgEP zGOn1ga8vcxyOMuXFBA6+cppzN4roGpt22wtziI2oSCjafFVft08cGsIpxFqpt9ZNSNf 74wyHFLsLunGtzEQBOazuf2GTLzk/krsJPIrKKHvAw68O9UQln3ZxDIMAIR08QzJeC3V RQAO0pxVcts2OY+Y26dzXigsDVq4A+zrS6IBwacOosK+ppT8Dgo1EZpcE3tgEXqKem1X 8ZoA==; darn=isar-build.org ARC-Authentication-Results: i=2; gmr-mx.google.com; spf=pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=amikan@ilbers.de DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlegroups.com; s=20230601; t=1730881293; x=1731486093; darn=isar-build.org; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:x-original-authentication-results :x-original-sender:mime-version:references:in-reply-to:message-id :date:subject:cc:to:from:sender:from:to:cc:subject:date:message-id :reply-to; bh=51vSoTxZEcpv/ff8baGrTTAQ/e/M5ucuALuXAsu+Jhs=; b=BX66mNx4wSaW4l9y64X6Bex7QX8lr0aRgMZYXKvT1GrdR336EKHpUOAgtn67C0B0ky y2zU/wL5Zoxw+46E+4/zpDP0tcnEJAGxR2uAiDqQekzfBK6+eogYdrIFWeoVeJgIqjV7 s5CX+IMsVB1jjPI+dqTqu3AKzs42KtfejqkjVuvAQzfOJ9/DrboMfx85tJUsly50wDnZ DJmhqPmQ5LqyGVZaHevu3DFtdivJxtanyzV43Ae8WITQ/+8BYm2QPJ81G7eNI+1LsKnd 4u/LEs5XldVmhXVbHjcKCAwjDu8RjAoR/pwUmJPFE/gBC/kh5XUGOWG+xzbHkXE0k3Am P1WA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1730881293; x=1731486093; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :x-spam-checked-in-group:list-id:mailing-list:precedence :x-original-authentication-results:x-original-sender:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :x-beenthere:x-gm-message-state:sender:from:to:cc:subject:date :message-id:reply-to; bh=51vSoTxZEcpv/ff8baGrTTAQ/e/M5ucuALuXAsu+Jhs=; b=iA/JkQD9/xCNEMpD2tAh5P1Yp7fe4TwZLstt6OMXTApHXE9SwPYDK61fHaz8sohH2F G6bmr8gxEeX7SBWasia3ZscUUkkbBY/oJmqOdIpHtK29zqG1NFlBRx4kf17QhsbHLlCU GPKJ3rWwPVMVsqpFXPUbaujFoQob3NnPNESS6cmThlN8YSNx9/Bscv0fJDAIzNGJHvln Ki8vKM6AzHPdaTPyplZPPChBPPCtQtvP3ejAdix2bbUsTRO4lBN9TxmEWA9wILanM88J RPe+UruP1Ji5njXs7mbrf7L3+2I2BHI5Ia+5BWEhXI3wzDDRpp5bSqXcIndptDe7r+OR UyjA== Sender: isar-users@googlegroups.com X-Forwarded-Encrypted: i=2; AJvYcCXUbUXmr+//D/WcldE15LnxtCvnh+CMVcKTlcVNZSeCkO9w3DYTnv+bgFI3LOvomp9jfzXUJNk=@isar-build.org X-Gm-Message-State: AOJu0YwqqK63YoaLH8ZvfRAPDQOAVMW8CG/FgcSc4U3JxeyO4LgxVCMl I832HapWt/F+y7GNjDvHcVMAPpuj+fpQQjHd5wgx6Sl8Pys8/Eb/ X-Google-Smtp-Source: AGHT+IF9RezwmqgHPKB7pCw0Tuw638UtFGxjjhHgl1cI+yPNbAxhe8JSEBJo/1FgDEdN6miPTBN9/Q== X-Received: by 2002:a05:6512:124b:b0:53a:3a:f4f4 with SMTP id 2adb3069b0e04-53c79e4c400mr10080792e87.31.1730881292865; Wed, 06 Nov 2024 00:21:32 -0800 (PST) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a05:6512:3993:b0:539:fc31:fc7b with SMTP id 2adb3069b0e04-53c791f9e43ls336278e87.0.-pod-prod-02-eu; Wed, 06 Nov 2024 00:21:31 -0800 (PST) X-Received: by 2002:ac2:4e14:0:b0:539:d9e2:9d15 with SMTP id 2adb3069b0e04-53c79e4b459mr11103958e87.29.1730881290629; Wed, 06 Nov 2024 00:21:30 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1730881290; cv=none; d=google.com; s=arc-20240605; b=CjL531GqpgmSxA6lBiD1liEQE8UDDjdA64lkI+xyHXG7SypFGwTKOJKHcljWO1RLEn tVf/M1DOb92FlE92HO0sewVMsyY2rV/a6p6XfKwiCDb1lgCIXcgSn+mN+YhvYTY/RfFQ EAU4sNnZRrJJiOn4RgTdrXHhu4lIlZB0mPLSlHkiOdwMgS4gX1z/MQ4QIvVsMX5P94w6 8khkDr5weUMUyGjQq79VdKB5BJyx5Ze4QHY/uLcV+x/1BMmbAZHOhNQugDb537Qd2wlD Xnz82e/ClyduJjlnF26gVUbmJyzK+dlUVTCIr1HjsD4iHxlmGdVgf3NFL2MHgMNVoDBL XmtA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20240605; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=EcO0BBUZurkMFuUMYbrT06ViPUO76krUApGcQep/SoY=; fh=/h9QQkzJ8EboVkWg45aWwpaUro6WMavIVd2OhN45RtE=; b=hc2ydAZ7+nWJz9FrZsnZESDINrogEE1coiagGm/4ojShVmDa6KDEVipUjvsy4uhPuY Dpap/hdFbkGGVJIGtAza2LIUo49rBxv4ZtG5w1R9PWdejv84DTo6ML5YkWyIrPMg7Qrq IBiEQ3KFwDe6bJjInUxP8IpQ5FElemgWhOfdgZrBRLbI0xBkFJp89VzMl/ceo/tDVGx/ 8jrvwQTXnHnD/zRrnGdWWr2+a3jT0a92eTLQGtOBqdOypdMUg44alAZ71V7IDhHX7HBi /HQW4b2tKZQcjQEcuYg/SUtj8ZvN69fHgeV4CYq7B3weXmOaIOWQATNGz3vzw5y25KpS 8aHQ==; dara=google.com ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=amikan@ilbers.de Received: from shymkent.ilbers.de (shymkent.ilbers.de. [85.214.156.166]) by gmr-mx.google.com with ESMTPS id 2adb3069b0e04-53c7bccc719si508517e87.10.2024.11.06.00.21.30 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Wed, 06 Nov 2024 00:21:30 -0800 (PST) Received-SPF: pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) client-ip=85.214.156.166; Received: from user-B660.promwad.corp ([159.148.83.114]) (authenticated bits=0) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPSA id 4A68LOjv016956 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 6 Nov 2024 09:21:29 +0100 From: Anton Mikanovich To: isar-users@googlegroups.com Cc: Anton Mikanovich Subject: [PATCH v11 6/8] mmdebstrap: Move preparations to hooks Date: Wed, 6 Nov 2024 10:21:15 +0200 Message-Id: <20241106082117.1089554-7-amikan@ilbers.de> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20241106082117.1089554-1-amikan@ilbers.de> References: <20241106082117.1089554-1-amikan@ilbers.de> MIME-Version: 1.0 X-Spam-Status: No, score=-4.6 required=5.0 tests=DKIMWL_WL_MED,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_EF,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,RCVD_IN_DNSWL_BLOCKED,RCVD_IN_MSPIKE_H2, RCVD_IN_RP_CERTIFIED,RCVD_IN_RP_RNBL,RCVD_IN_RP_SAFE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on shymkent.ilbers.de X-Original-Sender: amikan@ilbers.de X-Original-Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=amikan@ilbers.de Precedence: list Mailing-list: list isar-users@googlegroups.com; contact isar-users+owners@googlegroups.com List-ID: X-Spam-Checked-In-Group: isar-users@googlegroups.com X-Google-Group-Id: 914930254986 List-Post: , List-Help: , List-Archive: , List-Unsubscribe: , X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= Move all bootstrap rootfs prepare logic into mmdebstrap hooks. Also migrate from apt-key to gpg. Signed-off-by: Anton Mikanovich --- .../isar-mmdebstrap/isar-mmdebstrap.inc | 244 ++++++++---------- 1 file changed, 114 insertions(+), 130 deletions(-) diff --git a/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc b/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc index e1c3832d..4128ad80 100644 --- a/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc +++ b/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc @@ -13,8 +13,6 @@ FILESEXTRAPATHS:append = ":${LAYERDIR_core}/recipes-core/isar-bootstrap/files" ROOTFSDIR = "${WORKDIR}/rootfs" DISTRO_BOOTSTRAP_BASE_PACKAGES = "locales,apt,usrmerge" -DISTRO_BOOTSTRAP_BASE_PACKAGES:append:gnupg = ",gnupg" -DISTRO_BOOTSTRAP_BASE_PACKAGES:append:https-support = ",ca-certificates" BOOTSTRAP_TMPDIR = "${WORKDIR}/tempdir" # Fix for /var/lib/apt/available while maybe-jessie-or-older hook do not work @@ -25,53 +23,29 @@ DPKG_HOOKS ?= "${@'--hook-dir='+d.getVar('MM_HOOK_JESSIE') \ MMHOOKS:ubuntu-focal ?= "${DPKG_HOOKS}" MMHOOKS:debian-buster ?= "${DPKG_HOOKS}" -def get_distro_primary_source_entry(d): - for source in generate_distro_sources(d): - if source[0] == "deb": - return source[2:] - bb.fatal('Invalid apt sources list') - -def get_distro_have_https_source(d): - return any(source[2].startswith("https://") for source in generate_distro_sources(d)) - -def get_distro_needs_https_support(d): - if get_distro_have_https_source(d): - return "https-support" - else: - return "" - -OVERRIDES:append = ":${@get_distro_needs_https_support(d)}" - -def get_distro_needs_gpg_support(d): - if d.getVar("DISTRO_BOOTSTRAP_KEYS") or \ - d.getVar("THIRD_PARTY_APT_KEYS") or \ - d.getVar("BASE_REPO_KEY"): - return "gnupg" - else: - return "" - -OVERRIDES:append = ":${@get_distro_needs_gpg_support(d)}" - -APT_KEYS_DIR = "${WORKDIR}/aptkeys" DISTRO_BOOTSTRAP_KEYRING = "${WORKDIR}/distro-keyring.gpg" -do_generate_keyrings[cleandirs] = "${APT_KEYS_DIR}" -do_generate_keyrings[dirs] = "${DL_DIR}" +do_generate_keyrings[cleandirs] = "${WORKDIR}/trusted.gpg.d" +do_generate_keyrings[dirs] = "${DEBDIR}" do_generate_keyrings[vardeps] += "DISTRO_BOOTSTRAP_KEYS THIRD_PARTY_APT_KEYS" do_generate_keyrings[network] = "${TASK_USE_SUDO}" do_generate_keyrings() { + export GNUPGHOME="$(mktemp -td gpghomeXXXXXXXXXX)" if [ -n "${@d.getVar("THIRD_PARTY_APT_KEYFILES") or ""}" ]; then - chmod 777 "${APT_KEYS_DIR}" for keyfile in ${@d.getVar("THIRD_PARTY_APT_KEYFILES")}; do - cp "$keyfile" "${APT_KEYS_DIR}"/"$(basename "$keyfile")" + gpg --no-autostart --keyring "gnupg-ring:${DISTRO_BOOTSTRAP_KEYRING}" \ + --no-default-keyring --import $keyfile done fi if [ -n "${@d.getVar("DISTRO_BOOTSTRAP_KEYFILES") or ""}" ]; then for keyfile in ${@d.getVar("DISTRO_BOOTSTRAP_KEYFILES")}; do - sudo apt-key --keyring "${DISTRO_BOOTSTRAP_KEYRING}" add $keyfile - cp "$keyfile" "${APT_KEYS_DIR}"/"$(basename "$keyfile")" + gpg --no-autostart --keyring "gnupg-ring:${DISTRO_BOOTSTRAP_KEYRING}" \ + --no-default-keyring --import $keyfile done fi + if [ -r "${DISTRO_BOOTSTRAP_KEYRING}" ]; then + chmod o+r "${DISTRO_BOOTSTRAP_KEYRING}" + fi } addtask generate_keyrings before do_build after do_unpack @@ -80,10 +54,12 @@ do_bootstrap[vardeps] += " \ ISAR_ENABLE_COMPAT_ARCH \ ${DISTRO_VARS_PREFIX}DISTRO_APT_SOURCES \ " -do_bootstrap[dirs] = "${DEPLOY_DIR_BOOTSTRAP} ${BOOTSTRAP_TMPDIR}" +do_bootstrap[dirs] = "${DEPLOY_DIR_BOOTSTRAP} ${BOOTSTRAP_TMPDIR} ${WORKDIR}/trusted.gpg.d ${WORKDIR}/sources.list.d" do_bootstrap[depends] = "base-apt:do_cache isar-apt:do_cache_config" do_bootstrap[network] = "${TASK_USE_NETWORK_AND_SUDO}" +DEB_DL_LOCK ?= "${DEBDIR}/${BOOTSTRAP_BASE_DISTRO}-${BASE_DISTRO_CODENAME}.lock" + do_bootstrap() { if [ "${ISAR_ENABLE_COMPAT_ARCH}" = "1" ]; then if [ -z "${COMPAT_DISTRO_ARCH}" ]; then @@ -93,126 +69,134 @@ do_bootstrap() { bootstrap_args="--verbose --variant=minbase --include=${DISTRO_BOOTSTRAP_BASE_PACKAGES}" if [ -f "${DISTRO_BOOTSTRAP_KEYRING}" ]; then bootstrap_args="$bootstrap_args --keyring=${DISTRO_BOOTSTRAP_KEYRING}" + cp "${DISTRO_BOOTSTRAP_KEYRING}" "${WORKDIR}/trusted.gpg.d/" fi E="${@ isar_export_proxies(d)}" - export BOOTSTRAP_FOR_HOST - deb_dl_dir_import "${ROOTFSDIR}" "${BOOTSTRAP_BASE_DISTRO}-${BASE_DISTRO_CODENAME}" sudo rm -rf --one-file-system "${ROOTFSDIR}" mkdir -p "${ROOTFSDIR}" + if [ "${ISAR_USE_CACHED_BASE_REPO}" = "1" ]; then + base_apt_tmp="$(mktemp -d /tmp/isar-base-aptXXXXXXXXXX)" + bootstrap_list="${WORKDIR}/sources.list.d/base-apt.list" + line="copy://$base_apt_tmp/${BOOTSTRAP_BASE_DISTRO} ${BASE_DISTRO_CODENAME} main" + if [ -z "${BASE_REPO_KEY}" ]; then + line="[trusted=yes] ${line}" + fi + echo "deb ${line}" > "${WORKDIR}/sources.list.d/base-apt.list" + line="copy://$base_apt_tmp/${BASE_DISTRO} ${BASE_DISTRO_CODENAME} main" + if [ -z "${BASE_REPO_KEY}" ]; then + line="[trusted=yes] ${line}" + fi + echo "deb-src ${line}" >> "${WORKDIR}/sources.list.d/base-apt.list" + + # no need to sync /var/cache/apt/archives if base-apt used + syncin='echo skip sync-in' + syncout='echo skip sync-out' + extra_setup="mount --bind '${REPO_BASE_DIR}' $base_apt_tmp" + extra_extract="$syncout" + # save mmdebstrap tempdir for cleanup + extra_essential="mkdir -p \$1/$base_apt_tmp && \ + echo \$1 > ${WORKDIR}/mmtmpdir && \ + mount -o bind,private '${REPO_BASE_DIR}' \$1/$base_apt_tmp" + # replace base-apt mount in tmp with /base-apt mount + extra_customize="sed -i \"s|copy://$base_apt_tmp|file:///base-apt|g\" \ + \$1/etc/apt/sources.list.d/*.list && \ + mkdir -p \$1/base-apt && \ + mount -o bind,private '${REPO_BASE_DIR}' \$1/base-apt && \ + chroot \$1 apt-get update -y \ + -o APT::Update::Error-Mode=any && \ + chroot \$1 apt-get install -y dpkg && \ + umount \$1/base-apt && \ + umount \$1/$base_apt_tmp && rm ${WORKDIR}/mmtmpdir && \ + umount $base_apt_tmp && rm -rf --one-file-system $base_apt_tmp" + else + deb_dl_dir_import "${WORKDIR}/dl_dir" "${BOOTSTRAP_BASE_DISTRO}-${BASE_DISTRO_CODENAME}" + + bootstrap_list="${WORKDIR}/sources.list.d/bootstrap.list" + install -v -m644 "${APTSRCS}" \ + "${WORKDIR}/sources.list.d/bootstrap.list" + + syncin='flock -s ${DEB_DL_LOCK} cp -n --no-preserve=owner \ + "${WORKDIR}/dl_dir/var/cache/apt/archives/"*.deb \ + "$1/var/cache/apt/archives/" || true' + syncout='flock -s ${DEB_DL_LOCK} cp -n --no-preserve=owner \ + "$1/var/cache/apt/archives/"*.deb \ + "${WORKDIR}/dl_dir/var/cache/apt/archives/"' + extra_setup="$syncin" + extra_extract="$syncout" + # prefetch apt debs because mmdebstrap will clean them on next stage + extra_essential='apt-get install apt -y -d \ + -o Dir::State="$1/var/lib/apt" \ + -o Dir::Etc="$1/etc/apt" \ + -o Dir::Cache="$1/var/cache/apt" \ + -o Apt::Architecture="${BOOTSTRAP_DISTRO_ARCH}"' + extra_essential="$extra_essential && $syncout" + extra_customize="$syncout" + fi + + if [ ! -z "${SOURCE_DATE_EPOCH}" ]; then + export SOURCE_DATE_EPOCH="${SOURCE_DATE_EPOCH}" + fi + arch_param="--arch=${BOOTSTRAP_DISTRO_ARCH},${DISTRO_ARCH}" + if [ "${ISAR_ENABLE_COMPAT_ARCH}" = "1" ]; then + arch_param="$arch_param,${COMPAT_DISTRO_ARCH}" + fi + + # Cleanup mounts if fails + trap 'exit 1' INT HUP QUIT TERM ALRM USR1 + trap '[ -r "${WORKDIR}/mmtmpdir" ] && tmpdir=$(cat "${WORKDIR}/mmtmpdir") \ + && rm "${WORKDIR}/mmtmpdir"; \ + [ -d "$tmpdir" ] && mountpoint -q $tmpdir/$base_apt_tmp \ + && sudo umount $tmpdir/$base_apt_tmp; \ + [ -d "$tmpdir" ] && mountpoint -q $tmpdir/base-apt \ + && sudo umount $tmpdir/base-apt; \ + [ -d "$tmpdir" ] && sudo rm -rf --one-file-system $tmpdir; \ + [ -n "$base_apt_tmp" ] && mountpoint -q $base_apt_tmp \ + && sudo umount $base_apt_tmp \ + && rm -rf --one-file-system $base_apt_tmp' EXIT sudo TMPDIR="${BOOTSTRAP_TMPDIR}" mmdebstrap $bootstrap_args \ $arch_param \ --mode=unshare \ ${MMHOOKS} \ + --setup-hook='mkdir -p "$1/var/cache/apt/archives/"' \ + --setup-hook="$extra_setup" \ + --setup-hook='upload "${APTPREFS}" /etc/apt/preferences.d/bootstrap' \ + --setup-hook='upload "${APTSRCS_INIT}" /etc/apt/sources-list' \ + --setup-hook='upload "${WORKDIR}/locale" /etc/locale' \ + --setup-hook='mkdir -p "$1/etc/apt/trusted.gpg.d"' \ + --setup-hook='sync-in "${WORKDIR}/trusted.gpg.d" /etc/apt/trusted.gpg.d' \ + --setup-hook='install -v -m755 "${WORKDIR}/chroot-setup.sh" "$1/chroot-setup.sh"' \ + --extract-hook="$extra_extract" \ + --essential-hook="$extra_essential" \ + --customize-hook="$extra_customize" \ + --customize-hook='sed -i "/en_US.UTF-8 UTF-8/s/^#//g" "$1/etc/locale.gen"' \ + --customize-hook='chroot "$1" /usr/sbin/locale-gen' \ + --customize-hook='chroot "$1" /usr/bin/apt-get -y clean' \ + --skip=cleanup/apt \ + --skip=download/empty \ ${@get_distro_components_argument(d)} \ "${@get_distro_suite(d)}" \ "${WORKDIR}/rootfs.tar.zst" \ - "${@get_distro_source(d)}" + "$bootstrap_list" sudo -E -s <<'EOSUDO' set -e tar -xf "${WORKDIR}/rootfs.tar.zst" -C "${ROOTFSDIR}" --exclude="./dev/console" - # Install apt config - mkdir -p "${ROOTFSDIR}/etc/apt/preferences.d" - install -v -m644 "${APTPREFS}" \ - "${ROOTFSDIR}/etc/apt/preferences.d/bootstrap" - mkdir -p "${ROOTFSDIR}/etc/apt/sources.list.d" - if [ "${ISAR_USE_CACHED_BASE_REPO}" = "1" ]; then - line="file:///base-apt/${BOOTSTRAP_BASE_DISTRO} ${BASE_DISTRO_CODENAME} main" - if [ -z "${BASE_REPO_KEY}" ]; then - line="[trusted=yes] ${line}" - fi - echo "deb ${line}" > "${ROOTFSDIR}/etc/apt/sources.list.d/base-apt.list" - line="file:///base-apt/${BASE_DISTRO} ${BASE_DISTRO_CODENAME} main" - if [ -z "${BASE_REPO_KEY}" ]; then - line="[trusted=yes] ${line}" - fi - echo "deb-src ${line}" >> "${ROOTFSDIR}/etc/apt/sources.list.d/base-apt.list" - - mkdir -p ${ROOTFSDIR}/base-apt - mount -o bind,private ${REPO_BASE_DIR} ${ROOTFSDIR}/base-apt - else - install -v -m644 "${APTSRCS}" \ - "${ROOTFSDIR}/etc/apt/sources.list.d/bootstrap.list" - fi - install -v -m644 "${APTSRCS_INIT}" "${ROOTFSDIR}/etc/apt/sources-list" - rm -f "${ROOTFSDIR}/etc/apt/sources.list" - rm -rf "${ROOTFSDIR}/var/lib/apt/lists/"* - find ${APT_KEYS_DIR}/ -type f | while read keyfile - do - MY_GPGHOME="$(chroot "${ROOTFSDIR}" mktemp -d /tmp/gpghomeXXXXXXXXXX)" - echo "Created temporary directory ${MY_GPGHOME} for gpg-agent" - export GNUPGHOME="${MY_GPGHOME}" - APT_KEY_APPEND="--homedir ${MY_GPGHOME}" - - kfn="$(basename $keyfile)" - cp $keyfile "${ROOTFSDIR}/tmp/$kfn" - chroot "${ROOTFSDIR}" /usr/bin/gpg-agent --daemon -- /usr/bin/apt-key \ - --keyring ${THIRD_PARTY_APT_KEYRING} ${APT_KEY_APPEND} add "/tmp/$kfn" - rm "${ROOTFSDIR}/tmp/$kfn" - - echo "Removing ${MY_GPGHOME}" - rm -rf "${ROOTFSDIR}${MY_GPGHOME}" - done - - # Set locale - install -v -m644 "${WORKDIR}/locale" "${ROOTFSDIR}/etc/locale" - - sed -i '/en_US.UTF-8 UTF-8/s/^#//g' "${ROOTFSDIR}/etc/locale.gen" - chroot "${ROOTFSDIR}" /usr/sbin/locale-gen - - # update APT - mount -o bind,private /dev ${ROOTFSDIR}/dev - mount -o bind,private /dev/pts ${ROOTFSDIR}/dev/pts - mount -t tmpfs none "${ROOTFSDIR}/dev/shm" - mount -t proc none ${ROOTFSDIR}/proc - mount -o bind,private /sys ${ROOTFSDIR}/sys - mount --make-rslave ${ROOTFSDIR}/sys - - export DEBIAN_FRONTEND=noninteractive - - if [ "${BOOTSTRAP_FOR_HOST}" = "1" ]; then - chroot "${ROOTFSDIR}" /usr/bin/dpkg --add-architecture ${DISTRO_ARCH} - fi - - if [ "${ISAR_ENABLE_COMPAT_ARCH}" = "1" ]; then - chroot "${ROOTFSDIR}" /usr/bin/dpkg --add-architecture ${COMPAT_DISTRO_ARCH} - fi - - chroot "${ROOTFSDIR}" /usr/bin/apt-get update -y \ - -o APT::Update::Error-Mode=any - - chroot "${ROOTFSDIR}" /usr/bin/apt-get install -y dpkg - # setup chroot - install -v -m755 "${WORKDIR}/chroot-setup.sh" "${ROOTFSDIR}/chroot-setup.sh" "${ROOTFSDIR}/chroot-setup.sh" "setup" "${ROOTFSDIR}" - chroot "${ROOTFSDIR}" /usr/bin/apt-get install -y -f - chroot "${ROOTFSDIR}" /usr/bin/apt-get dist-upgrade -y \ - -o Debug::pkgProblemResolver=yes - - umount "${ROOTFSDIR}/dev/shm" - umount "${ROOTFSDIR}/dev/pts" - umount "${ROOTFSDIR}/dev" - umount "${ROOTFSDIR}/proc" - umount "${ROOTFSDIR}/sys" - if mountpoint -q "${ROOTFSDIR}/base-apt"; then - umount "${ROOTFSDIR}/base-apt" - fi - # Finalize bootstrap by setting the link in deploy ln -Tfsr "${ROOTFSDIR}" "${DEPLOY_ISAR_BOOTSTRAP}" EOSUDO - deb_dl_dir_export "${ROOTFSDIR}" "${BOOTSTRAP_BASE_DISTRO}-${BASE_DISTRO_CODENAME}" - - # Cleanup apt cache - sudo -Es chroot "${ROOTFSDIR}" /usr/bin/apt-get -y clean + if [ "${ISAR_USE_CACHED_BASE_REPO}" != "1" ]; then + deb_dl_dir_export "${WORKDIR}/dl_dir" "${BOOTSTRAP_BASE_DISTRO}-${BASE_DISTRO_CODENAME}" + sudo rm -rf --one-file-system "${WORKDIR}/dl_dir" + fi } addtask bootstrap before do_build after do_generate_keyrings