@@ -838,3 +838,11 @@ instead of
```
ROOTFS_FEATURE += "no-generate-initrd"
```
+
+### Remove isar-bootstrap support
+
+The default bootstrap procedure is performed with isar-mmdebstrap.
+The alternative with isar-bootstrap is outdated and untested. It is therefore
+removed.
+A custom implementation still can be selected by setting
+PREFERRED_PROVIDER_bootstrap-{host,target} in local.conf.
@@ -68,7 +68,7 @@ Also called sbuild chroot rootfs, because of the usage by sbuild tool.
One of the key aspect of Debian philosophy claims the fact, that everything in
Debian should be built within Debian environment. To follow this rule, Isar
introduces the new component - schroot rootfs. Schroot rootfs is typical Debian
-filesystem that is created using standard Debian tools: debootstrap/mmdebstrap, apt-get.
+filesystem that is created using standard Debian tools: mmdebstrap, apt-get.
The source of packages can be either official Debian repositories or custom
repositories created by user. It could be one of the following type:
@@ -80,8 +80,7 @@ repositories created by user. It could be one of the following type:
Schroot rootfs lifecycle can be described as following:
- Schroot rootfs starts out with a minimal base system that was generated by
- the isar-bootstrap-{host,target}/isar-mmdebstrap-{host,target} recipe that uses
- debootstrap/mmdebstrap respectively and apt-get.
+ the isar-mmdebstrap-{host,target} recipe that uses mmdebstrap and apt-get.
- Common development packages needed for packages building are installed on
top of this rootfs.
@@ -95,7 +94,7 @@ architecture.
Target filesystem lifecycle can be described as following:
- Target filesystem starts, like the schroot rootfs, with the same minimal
- base system generated by the isar-bootstrap recipe.
+ base system generated by the isar-mmdebstrap recipe.
- According to the list of custom packages in bitbake recipes, the initial
filesystem will be populated by successfully built packages from locally
@@ -107,7 +106,7 @@ Target filesystem lifecycle can be described as following:
Whole Isar build process can be split into the following steps:
- - Generation of a minimal base system by isar-bootstrap/isar-mmdebstrap using debootstrap/mmdebstrap and
+ - Generation of a minimal base system by isar-mmdebstrap using mmdebstrap and
apt for each configuration requested by the user (Debian distro, machine and
architecture).
@@ -125,27 +124,26 @@ All these steps are described in details below.
## 3.2 Minimal base system
-A minimal base system is generated by the isar-bootstrap/isar-mmdebstrap recipe found here:
-`meta/recipes-core/isar-bootstrap/isar-bootstrap-[host,target].bb` /
+A minimal base system is generated by the isar-mmdebstrap recipe found here:
`meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap-[host,target].bb`
This recipe implements a few tasks that are executed in this order:
1. `do_generate_keyring`: Generates a new gpg keyring from the gpg keys
specified in the `APTKEYFILES` variable. This keyring is later used by
- debootstrap/mmdebstrap and apt to verify the apt repositories.
+ mmdebstrap and apt to verify the apt repositories.
2. `do_apt_config_prepare`: Generates aggregated apt preferences and source
files based on the `DISTRO_APT_PREFERENCES` and `DISTRO_APT_SOURCES`
variables. That are later deployed to the minimal base system to be used by
`apt` itself.
-3. `do_bootstrap`: Bootstraps a minimal debian base system using debootstrap/mmdebstrap.
+3. `do_bootstrap`: Bootstraps a minimal debian base system using mmdebstrap.
## 3.3 Initial Schroot Rootfs Generation
As mentioned above, initial schroot rootfs is generated using the minimal base
-system generated by isar-bootstrap. The bitbake recipe which is responsible for
+system generated by isar-mmdebstrap. The bitbake recipe which is responsible for
schroot rootfs can be found here:
`meta/recipes-devtools/sbuild-chroot/sbuild-chroot-{host,target}.bb`
@@ -66,8 +66,7 @@ The steps below describe how to build the images provided by default.
### Install Host Tools
-The supported host system is >= buster for debootstrap and >= bullseye for
-default mmdebstrap provider.
+The supported host system is >= bullseye for default mmdebstrap provider.
Building `debian-trixie` requires host system >= bookworm.
@@ -97,12 +96,6 @@ apt install \
zstd
```
-If using isar-bootstrap provider, debootstrap should be installed instead of
-mmdebstrap:
-```
-apt install debootstrap
-```
-
If your host is bullseye or bookworm, also install the following package.
```
apt install python3-distutils
@@ -1245,7 +1238,7 @@ The debug artifacts will be placed in `${CCACHE_DIR}/debug`.
Isar supports caching of bitbake task artifacts using the sstate-cache
feature known from OpenEmbedded. Isar caches
- * the Debian bootstrap (`isar-bootstrap` recipe)
+ * the Debian bootstrap (`isar-mmdebstrap` recipe)
* Debian packages (built with the `dpkg` or `dpkg-raw` classes)
* root file systems (schroot and image rootfs)
@@ -14,7 +14,7 @@ config IMAGE_BASE
bool "Base image"
help
This is the basic Isar image with bare minimum of preinstalled packages.
- It's technically close do the debootstrapped system with packages
+ It's technically close do the mmdebstrapped system with packages
required to boot up the system.
config IMAGE_DEBUG
@@ -23,9 +23,6 @@ DISTRO_BOOTSTRAP_BASE_PACKAGES:append:jammy = " usrmerge"
# kernel package name is linux-image-generic in Ubuntu
KERNEL_NAME = "generic"
-# that is what debootstrap_1.0.118ubuntu1 does anyways
-DISTRO_DEBOOTSTRAP_SCRIPT = "/usr/share/debootstrap/scripts/gutsy"
-
DISTRO_CONFIG_SCRIPT ?= "debian-configscript.sh"
DISTRO_KERNELS ?= " \
@@ -231,7 +231,3 @@ USER_isar[flags] += "clear-text-password"
# To know more details about this variable and how to set the value refer below
# https://reproducible-builds.org/docs/source-date-epoch/
#SOURCE_DATE_EPOCH =
-
-# Uncomment this to use old isar-bootstrap provider for rootfs prepare
-#PREFERRED_PROVIDER_bootstrap-host ?= "isar-bootstrap-host"
-#PREFERRED_PROVIDER_bootstrap-target ?= "isar-bootstrap-target"
@@ -419,14 +419,6 @@ do_rootfs_finalize() {
-maxdepth 1 -name 'qemu-*-static' -type f -delete
fi
- # needed only for debootstrap, mmdebstrap leave a 0000bootstrap.list behind
- if [ -e "${ROOTFSDIR}/etc/apt/sources-list" ] && \
- [ -d "${ROOTFSDIR}/etc/apt/sources.list.d" ] && \
- [ -z "$(find ${ROOTFSDIR}/etc/apt/sources.list.d -mindepth 1)" ]; then
- mv "${ROOTFSDIR}/etc/apt/sources-list" \
- "${ROOTFSDIR}/etc/apt/sources.list.d/bootstrap.list"
- fi
-
if [ -n "${IMAGE_LISTS}" ]; then
find "${ROOTFSDIR}/etc/apt/sources.list.d/" ! -type d -exec rm -f {} \;
for l in ${IMAGE_LISTS}; do
@@ -95,7 +95,6 @@ PACKAGE_ARCH ?= "${DISTRO_ARCH}"
# Isar apt repository paths
REPO_ISAR_DIR = "${DEPLOY_DIR}/isar-apt/${DISTRO}-${DISTRO_ARCH}/apt"
REPO_ISAR_DB_DIR = "${DEPLOY_DIR}/isar-apt/${DISTRO}-${DISTRO_ARCH}/db"
-THIRD_PARTY_APT_KEYRING = "/etc/apt/trusted.gpg.d/third_party.gpg"
# Base apt repository paths
REPO_BASE_DIR = "${DEPLOY_DIR}/base-apt/${DISTRO}/apt"
deleted file mode 100644
@@ -1,19 +0,0 @@
-# Minimal host Debian root file system
-#
-# This software is a part of ISAR.
-# Copyright (c) Siemens AG, 2018
-#
-# SPDX-License-Identifier: MIT
-
-Description = "Minimal host Debian root file system"
-
-DEPLOY_ISAR_BOOTSTRAP = "${DEPLOY_DIR_BOOTSTRAP}/${HOST_DISTRO}-host_${DISTRO}-${DISTRO_ARCH}"
-
-PROVIDES += "bootstrap-host"
-
-BOOTSTRAP_FOR_HOST = "1"
-
-require isar-bootstrap.inc
-
-HOST_DISTRO_BOOTSTRAP_KEYS ?= ""
-DISTRO_BOOTSTRAP_KEYS = "${HOST_DISTRO_BOOTSTRAP_KEYS}"
deleted file mode 100644
@@ -1,14 +0,0 @@
-# Minimal target Debian root file system
-#
-# This software is a part of ISAR.
-# Copyright (c) Siemens AG, 2018
-#
-# SPDX-License-Identifier: MIT
-
-Description = "Minimal target Debian root file system"
-
-DEPLOY_ISAR_BOOTSTRAP = "${DEPLOY_DIR_BOOTSTRAP}/${DISTRO}-${DISTRO_ARCH}"
-
-PROVIDES += "bootstrap-target"
-
-require isar-bootstrap.inc
deleted file mode 100644
@@ -1,242 +0,0 @@
-# Minimal debian root file system
-#
-# This software is a part of ISAR.
-# Copyright (c) Siemens AG, 2018-2021
-#
-# SPDX-License-Identifier: MIT
-
-inherit bootstrap
-
-LICENSE = "gpl-2.0"
-LIC_FILES_CHKSUM = "file://${LAYERDIR_core}/licenses/COPYING.GPLv2;md5=751419260aa954499f7abaabaa882bbe"
-PV = "1.0"
-
-DEBOOTSTRAP ?= "qemu-debootstrap"
-ROOTFSDIR = "${WORKDIR}/rootfs"
-DISTRO_BOOTSTRAP_BASE_PACKAGES += "locales"
-DISTRO_BOOTSTRAP_BASE_PACKAGES:append:gnupg = " gnupg"
-DISTRO_BOOTSTRAP_BASE_PACKAGES:append:https-support = " ca-certificates"
-
-inherit deb-dl-dir
-
-def get_distro_have_https_source(d):
- return any(source[2].startswith("https://") for source in generate_distro_sources(d))
-
-def get_distro_needs_https_support(d):
- if get_distro_have_https_source(d):
- return "https-support"
- else:
- return ""
-
-OVERRIDES:append = ":${@get_distro_needs_https_support(d)}"
-
-def get_distro_needs_gpg_support(d):
- if d.getVar("DISTRO_BOOTSTRAP_KEYS") or \
- d.getVar("THIRD_PARTY_APT_KEYS") or \
- d.getVar("BASE_REPO_KEY"):
- return "gnupg"
- else:
- return ""
-
-OVERRIDES:append = ":${@get_distro_needs_gpg_support(d)}"
-
-APT_KEYS_DIR = "${WORKDIR}/aptkeys"
-DISTRO_BOOTSTRAP_KEYRING = "${WORKDIR}/distro-keyring.gpg"
-
-do_generate_keyrings[cleandirs] = "${APT_KEYS_DIR}"
-do_generate_keyrings[dirs] = "${DL_DIR}"
-do_generate_keyrings[vardeps] += "DISTRO_BOOTSTRAP_KEYS THIRD_PARTY_APT_KEYS"
-do_generate_keyrings[network] = "${TASK_USE_SUDO}"
-do_generate_keyrings() {
- if [ -n "${@d.getVar("THIRD_PARTY_APT_KEYFILES") or ""}" ]; then
- chmod 777 "${APT_KEYS_DIR}"
- for keyfile in ${@d.getVar("THIRD_PARTY_APT_KEYFILES")}; do
- cp "$keyfile" "${APT_KEYS_DIR}"/"$(basename "$keyfile")"
- done
- fi
- if [ -n "${@d.getVar("DISTRO_BOOTSTRAP_KEYFILES") or ""}" ]; then
- for keyfile in ${@d.getVar("DISTRO_BOOTSTRAP_KEYFILES")}; do
- sudo apt-key --keyring "${DISTRO_BOOTSTRAP_KEYRING}" add $keyfile
- cp "$keyfile" "${APT_KEYS_DIR}"/"$(basename "$keyfile")"
- done
- fi
-}
-addtask generate_keyrings before do_build after do_unpack
-
-def get_host_release():
- import platform
- rel = platform.release()
- return rel
-
-do_bootstrap[vardeps] += " \
- DISTRO_APT_PREMIRRORS \
- ISAR_ENABLE_COMPAT_ARCH \
- ${DISTRO_VARS_PREFIX}DISTRO_APT_SOURCES \
- "
-do_bootstrap[dirs] = "${DEPLOY_DIR_BOOTSTRAP}"
-do_bootstrap[depends] = "base-apt:do_cache isar-apt:do_cache_config"
-do_bootstrap[network] = "${TASK_USE_NETWORK_AND_SUDO}"
-
-inherit compat
-
-do_bootstrap[vardeps] += "DISTRO_BOOTSTRAP_BASE_PACKAGES"
-do_bootstrap() {
- if [ "${ISAR_ENABLE_COMPAT_ARCH}" = "1" ]; then
- if [ -z "${COMPAT_DISTRO_ARCH}" ]; then
- bbfatal "${DISTRO_ARCH} does not have a compat arch"
- fi
- fi
- debootstrap_args="--verbose --variant=minbase --include=${@','.join(d.getVar('DISTRO_BOOTSTRAP_BASE_PACKAGES').split())}"
- if [ -f "${DISTRO_BOOTSTRAP_KEYRING}" ]; then
- debootstrap_args="$debootstrap_args --keyring=${DISTRO_BOOTSTRAP_KEYRING}"
- fi
- if [ "${ISAR_USE_CACHED_BASE_REPO}" = "1" -a -z "${BASE_REPO_KEY}" ]; then
- debootstrap_args="$debootstrap_args --no-check-gpg"
- fi
- E="${@ isar_export_proxies(d)}"
- export BOOTSTRAP_FOR_HOST debootstrap_args E
-
- sudo rm -rf --one-file-system "${ROOTFSDIR}"
- deb_dl_dir_import "${ROOTFSDIR}" "${BOOTSTRAP_BASE_DISTRO}-${BASE_DISTRO_CODENAME}"
-
- sudo -E -s <<'EOSUDO'
- set -e
- if [ "${BOOTSTRAP_FOR_HOST}" = "0" ]; then
- arch_param="--arch=${DISTRO_ARCH}"
- fi
- ${DEBOOTSTRAP} $debootstrap_args \
- $arch_param \
- ${@get_distro_components_argument(d)} \
- "${@get_distro_suite(d)}" \
- "${ROOTFSDIR}" \
- "${@get_distro_source(d)}" \
- ${DISTRO_DEBOOTSTRAP_SCRIPT}
-
- # Install apt config
- mkdir -p "${ROOTFSDIR}/etc/apt/preferences.d"
- install -v -m644 "${APTPREFS}" \
- "${ROOTFSDIR}/etc/apt/preferences.d/bootstrap"
- mkdir -p "${ROOTFSDIR}/etc/apt/sources.list.d"
- if [ "${ISAR_USE_CACHED_BASE_REPO}" = "1" ]; then
- line="file:///base-apt/${BOOTSTRAP_BASE_DISTRO} ${BASE_DISTRO_CODENAME} main"
- if [ -z "${BASE_REPO_KEY}" ]; then
- line="[trusted=yes] ${line}"
- fi
- echo "deb ${line}" > "${ROOTFSDIR}/etc/apt/sources.list.d/base-apt.list"
- line="file:///base-apt/${BASE_DISTRO} ${BASE_DISTRO_CODENAME} main"
- if [ -z "${BASE_REPO_KEY}" ]; then
- line="[trusted=yes] ${line}"
- fi
- echo "deb-src ${line}" >> "${ROOTFSDIR}/etc/apt/sources.list.d/base-apt.list"
-
- mkdir -p ${ROOTFSDIR}/base-apt
- mount -o bind,private "${REPO_BASE_DIR}" "${ROOTFSDIR}/base-apt"
- else
- install -v -m644 "${APTSRCS}" \
- "${ROOTFSDIR}/etc/apt/sources.list.d/bootstrap.list"
- fi
- install -v -m644 "${APTSRCS_INIT}" "${ROOTFSDIR}/etc/apt/sources-list"
- rm -f "${ROOTFSDIR}/etc/apt/sources.list"
- rm -rf "${ROOTFSDIR}/var/lib/apt/lists/"*
- find ${APT_KEYS_DIR}/ -type f | while read keyfile
- do
- MY_GPGHOME="$(chroot "${ROOTFSDIR}" mktemp -d /tmp/gpghomeXXXXXXXXXX)"
- echo "Created temporary directory ${MY_GPGHOME} for gpg-agent"
- export GNUPGHOME="${MY_GPGHOME}"
- APT_KEY_APPEND="--homedir ${MY_GPGHOME}"
-
- kfn="$(basename $keyfile)"
- cp $keyfile "${ROOTFSDIR}/tmp/$kfn"
- chroot "${ROOTFSDIR}" /usr/bin/gpg-agent --daemon -- /usr/bin/apt-key \
- --keyring ${THIRD_PARTY_APT_KEYRING} ${APT_KEY_APPEND} add "/tmp/$kfn"
- rm "${ROOTFSDIR}/tmp/$kfn"
-
- echo "Removing ${MY_GPGHOME}"
- rm -rf "${ROOTFSDIR}${MY_GPGHOME}"
- done
-
- # Set locale
- install -v -m644 "${WORKDIR}/locale" "${ROOTFSDIR}/etc/locale"
-
- sed -i '/en_US.UTF-8 UTF-8/s/^#//g' "${ROOTFSDIR}/etc/locale.gen"
- chroot "${ROOTFSDIR}" /usr/sbin/locale-gen
-
- # update APT
- mount -o bind,private /dev ${ROOTFSDIR}/dev
- mount -o bind,private /dev/pts "${ROOTFSDIR}/dev/pts"
- mount -t tmpfs none "${ROOTFSDIR}/dev/shm"
- mount -t proc none ${ROOTFSDIR}/proc
- mount -o bind,private /sys "${ROOTFSDIR}/sys"
- mount --make-rslave ${ROOTFSDIR}/sys
-
- export DEBIAN_FRONTEND=noninteractive
-
- if [ "${BOOTSTRAP_FOR_HOST}" = "1" ]; then
- chroot "${ROOTFSDIR}" /usr/bin/dpkg --add-architecture ${DISTRO_ARCH}
- fi
-
- if [ "${ISAR_ENABLE_COMPAT_ARCH}" = "1" ]; then
- chroot "${ROOTFSDIR}" /usr/bin/dpkg --add-architecture ${COMPAT_DISTRO_ARCH}
- fi
-
- chroot "${ROOTFSDIR}" /usr/bin/apt-get update -y \
- -o APT::Update::Error-Mode=any
-
- chroot "${ROOTFSDIR}" /usr/bin/apt-get install -y dpkg
-
- # prepare setup chroot script
- install -v -m755 "${WORKDIR}/chroot-setup.sh" "${ROOTFSDIR}/chroot-setup.sh"
-
- chroot "${ROOTFSDIR}" /usr/bin/apt-get install -y -f
- chroot "${ROOTFSDIR}" /usr/bin/apt-get dist-upgrade -y \
- -o Debug::pkgProblemResolver=yes
-
- umount "${ROOTFSDIR}/dev/shm"
- umount "${ROOTFSDIR}/dev/pts"
- umount "${ROOTFSDIR}/dev"
- umount "${ROOTFSDIR}/proc"
- umount "${ROOTFSDIR}/sys"
- mountpoint -q "${ROOTFSDIR}/base-apt" && \
- umount "${ROOTFSDIR}/base-apt"
-
- # Compress rootfs for compatibility
- lopts="--one-file-system --exclude=var/cache/apt/archives"
- tar --zstd -cf "${WORKDIR}/rootfs.tar.zst" $lopts -C "${ROOTFSDIR}" .
-
- # Finalize debootstrap by setting the link in deploy
- ln -Tfsr "${WORKDIR}/rootfs.tar.zst" "${DEPLOY_ISAR_BOOTSTRAP}.tar.zst"
-EOSUDO
- deb_dl_dir_export "${ROOTFSDIR}" "${BOOTSTRAP_BASE_DISTRO}-${BASE_DISTRO_CODENAME}"
-
- # Cleanup apt cache
- sudo -Es chroot "${ROOTFSDIR}" /usr/bin/apt-get -y clean
-}
-
-addtask bootstrap before do_build after do_generate_keyrings
-
-SSTATETASKS += "do_bootstrap"
-SSTATECREATEFUNCS += "bootstrap_sstate_prepare"
-SSTATEPOSTINSTFUNCS += "bootstrap_sstate_finalize"
-
-bootstrap_sstate_prepare() {
- # this runs in SSTATE_BUILDDIR, which will be deleted automatically
- sudo cp -a "${WORKDIR}/rootfs.tar.zst" ./bootstrap.tar.zst
- sudo chown $(id -u):$(id -g) bootstrap.tar.zst
-}
-
-bootstrap_sstate_finalize() {
- # this runs in SSTATE_INSTDIR
- # we should restore symlinks after using tar
- if [ -f bootstrap.tar.zst ]; then
- mv bootstrap.tar.zst "${WORKDIR}/rootfs.tar.zst"
- sudo ln -Tfsr "${WORKDIR}/rootfs.tar.zst" \
- "${DEPLOY_ISAR_BOOTSTRAP}.tar.zst"
- fi
-}
-
-python do_bootstrap_setscene() {
- sstate_setscene(d)
-}
-
-addtask do_bootstrap_setscene
-do_bootstrap_setscene[dirs] = "${DEPLOY_DIR_BOOTSTRAP}"
similarity index 100%
rename from meta/recipes-core/isar-bootstrap/files/chroot-setup.sh
rename to meta/recipes-core/isar-mmdebstrap/files/chroot-setup.sh
similarity index 100%
rename from meta/recipes-core/isar-bootstrap/files/locale
rename to meta/recipes-core/isar-mmdebstrap/files/locale
@@ -9,8 +9,6 @@ inherit bootstrap
inherit compat
inherit deb-dl-dir
-FILESEXTRAPATHS:append = ":${LAYERDIR_core}/recipes-core/isar-bootstrap/files"
-
DISTRO_BOOTSTRAP_BASE_PACKAGES += "locales apt"
DISTRO_BOOTSTRAP_BASE_PACKAGES:append:https-support = " ca-certificates"
The minimal base system for the initial schroot rootfs at build is generated by default by isar-mmdebstrap. The old-school alternative isar-bootstrap is outdated, untested by the isar testsuite and is therefore removed. The preferred provider of bootstrap-[host,target] can still be overwritten. Signed-off-by: Clara Kowalsky <clara.kowalsky@siemens.com> --- RECIPE-API-CHANGELOG.md | 8 + doc/technical_overview.md | 18 +- doc/user_manual.md | 11 +- kas/image/Kconfig | 2 +- meta-isar/conf/distro/ubuntu-common.inc | 3 - meta-isar/conf/local.conf.sample | 4 - meta/classes/image.bbclass | 8 - meta/conf/bitbake.conf | 1 - .../isar-bootstrap/isar-bootstrap-host.bb | 19 -- .../isar-bootstrap/isar-bootstrap-target.bb | 14 - .../isar-bootstrap/isar-bootstrap.inc | 242 ------------------ .../files/chroot-setup.sh | 0 .../files/locale | 0 .../isar-mmdebstrap/isar-mmdebstrap.inc | 2 - 14 files changed, 19 insertions(+), 313 deletions(-) delete mode 100644 meta/recipes-core/isar-bootstrap/isar-bootstrap-host.bb delete mode 100644 meta/recipes-core/isar-bootstrap/isar-bootstrap-target.bb delete mode 100644 meta/recipes-core/isar-bootstrap/isar-bootstrap.inc rename meta/recipes-core/{isar-bootstrap => isar-mmdebstrap}/files/chroot-setup.sh (100%) rename meta/recipes-core/{isar-bootstrap => isar-mmdebstrap}/files/locale (100%)