| Message ID | 20210329155640.62445-2-silvano.cirujano-cuesta@siemens.com |
|---|---|
| State | Accepted, archived |
| Headers | show |
| Series | Add support for containerized root filesystems | expand |
On 29.03.21 17:56, [ext] Silvano Cirujano Cuesta wrote: > This class can be used to create container images which root filesystem > is that generated by the do_rootfs task. > > Containerized root filesystems have following possible use-cases: > - Using ISAR as a container image builder. > - Simplify distribution of runtime rootfs (binaries, libraries, > configurations, ...) for application development or testing. > - Distributing SDKs. > > Signed-off-by: Silvano Cirujano Cuesta <silvano.cirujano-cuesta@siemens.com> > --- > .../classes/image-container-extension.bbclass | 82 +++++++++++++++++++ > meta/classes/image.bbclass | 1 + > 2 files changed, 83 insertions(+) > create mode 100644 meta/classes/image-container-extension.bbclass > > diff --git a/meta/classes/image-container-extension.bbclass b/meta/classes/image-container-extension.bbclass > new file mode 100644 > index 0000000..f693627 > --- /dev/null > +++ b/meta/classes/image-container-extension.bbclass > @@ -0,0 +1,82 @@ > +# This software is a part of ISAR. > +# Copyright (C) Siemens AG, 2021 > +# > +# SPDX-License-Identifier: MIT > +# > +# This class extends the image.bbclass for containerizing the root filesystem. > + > +CONTAINER_FORMATS ?= "docker-archive" > +IMAGE_INSTALL += "isar-exclude-docs isar-disable-apt-cache" > + Quirin tried latest Isar on meta-iot2050, and that now fails during setup of openjdk: Setting up openjdk-11-jre-headless:arm64 (11.0.9.1+1-1~deb10u2) ... update-alternatives: using /usr/lib/jvm/java-11-openjdk-arm64/bin/rmid to provide /usr/bin/rmid (rmid) in auto mode update-alternatives: error: error creating symbolic link '/usr/share/man/man1/rmid.1.gz.dpkg-tmp': No such file or directory While watching the installation, I happened to see isar-exclude-docs suddenly being installed while not being selected explicitly, specifically not for the target image. I bet it's coming in via this class and its unconditional extension of the IMAGE_INSTALL list. That leads to (at least) two questions: - Why do we have isar-exclude-docs here, and also isar-disable-apt-cache? - Does isar-exclude-docs have some issue that prevents the usage together with openjdk, or are those simply incompatible by design? In any case, we have a regression. Jan > +containerize_rootfs() { > + local cmd="/bin/dash" > + local empty_tag="empty" > + local full_tag="latest" > + local oci_img_dir="${WORKDIR}/oci-image" > + local rootfs="$1" > + local rootfs_id="$2" > + local container_formats="$3" > + > + # prepare OCI container image skeleton > + bbdebug 1 "prepare OCI container image skeleton" > + rm -rf "${oci_img_dir}" > + sudo umoci init --layout "${oci_img_dir}" > + sudo umoci new --image "${oci_img_dir}:${empty_tag}" > + sudo umoci config --image "${oci_img_dir}:${empty_tag}" \ > + --config.cmd="${cmd}" > + sudo umoci unpack --image "${oci_img_dir}:${empty_tag}" \ > + "${oci_img_dir}_unpacked" > + > + # add root filesystem as the flesh of the skeleton > + sudo cp -a "${rootfs}"/* "${oci_img_dir}_unpacked/rootfs/" > + # clean-up temporary files > + sudo find "${oci_img_dir}_unpacked/rootfs/tmp" -mindepth 1 -delete > + > + # pack container image > + bbdebug 1 "pack container image" > + sudo umoci repack --image "${oci_img_dir}:${full_tag}" \ > + "${oci_img_dir}_unpacked" > + sudo umoci remove --image "${oci_img_dir}:${empty_tag}" > + sudo rm -rf "${oci_img_dir}_unpacked" > + > + # no root needed anymore > + sudo chown --recursive $(id -u):$(id -g) "${oci_img_dir}" > + > + # convert the OCI container image to the desired format > + image_name="isar-${rootfs_id}" > + for image_type in ${CONTAINER_FORMATS} ; do > + image_archive="${DEPLOY_DIR_IMAGE}/${rootfs_id}-${image_type}.tar" > + bbdebug 1 "Creating container image type: ${image_type}" > + case "${image_type}" in > + "docker-archive" | "oci-archive") > + if [ "${image_type}" = "oci-archive" ] ; then > + target="${image_type}:${image_archive}:latest" > + else > + target="${image_type}:${image_archive}:${image_name}:latest" > + fi > + rm -f "${image_archive}" "${image_archive}.xz" > + bbdebug 2 "Converting OCI image to ${image_type}" > + skopeo --insecure-policy copy \ > + "oci:${oci_img_dir}:${full_tag}" "${target}" > + bbdebug 2 "Compressing image" > + xz -T0 "${image_archive}" > + ;; > + "oci") > + tar --create --xz --directory "${oci_img_dir}" \ > + --file "${image_archive}.xz" . > + ;; > + "docker-daemon" | "containers-storage") > + if [ -f /.dockerenv ] || [ -f /run/.containerenv ] ; then > + die "Adding the container image to a container runtime (${image_type}) not supported if running from a container (e.g. 'kas-container')" > + fi > + skopeo --insecure-policy copy \ > + "oci:${oci_img_dir}:${full_tag}" \ > + "${image_type}:${image_name}:latest" > + ;; > + *) > + die "Unsupported format for containerize_rootfs: ${image_type}" > + ;; > + esac > + done > +} > + > diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass > index eddc444..ec93cab 100644 > --- a/meta/classes/image.bbclass > +++ b/meta/classes/image.bbclass > @@ -76,6 +76,7 @@ inherit image-tools-extension > inherit image-postproc-extension > inherit image-locales-extension > inherit image-account-extension > +inherit image-container-extension > > # Extra space for rootfs in MB > ROOTFS_EXTRA ?= "64" >
On 08/04/2021 19:55, Jan Kiszka wrote: > On 29.03.21 17:56, [ext] Silvano Cirujano Cuesta wrote: >> This class can be used to create container images which root >> filesystem is that generated by the do_rootfs task. >> >> Containerized root filesystems have following possible use-cases: - >> Using ISAR as a container image builder. - Simplify distribution of >> runtime rootfs (binaries, libraries, configurations, ...) for >> application development or testing. - Distributing SDKs. >> >> Signed-off-by: Silvano Cirujano Cuesta >> <silvano.cirujano-cuesta@siemens.com> --- >> .../classes/image-container-extension.bbclass | 82 >> +++++++++++++++++++ meta/classes/image.bbclass | >> 1 + 2 files changed, 83 insertions(+) create mode 100644 >> meta/classes/image-container-extension.bbclass >> >> diff --git a/meta/classes/image-container-extension.bbclass >> b/meta/classes/image-container-extension.bbclass new file mode >> 100644 index 0000000..f693627 --- /dev/null +++ >> b/meta/classes/image-container-extension.bbclass @@ -0,0 +1,82 @@ >> +# This software is a part of ISAR. +# Copyright (C) Siemens AG, >> 2021 +# +# SPDX-License-Identifier: MIT +# +# This class extends >> the image.bbclass for containerizing the root filesystem. + >> +CONTAINER_FORMATS ?= "docker-archive" +IMAGE_INSTALL += >> "isar-exclude-docs isar-disable-apt-cache" + > Quirin tried latest Isar on meta-iot2050, and that now fails during > setup of openjdk: > > Setting up openjdk-11-jre-headless:arm64 (11.0.9.1+1-1~deb10u2) ... > > > > update-alternatives: using > /usr/lib/jvm/java-11-openjdk-arm64/bin/rmid to provide /usr/bin/rmid > (rmid) in auto mode > > > update-alternatives: error: error creating symbolic link > '/usr/share/man/man1/rmid.1.gz.dpkg-tmp': No such file or directory > > > > While watching the installation, I happened to see isar-exclude-docs > suddenly being installed while not being selected explicitly, > specifically not for the target image. I bet it's coming in via this > class and its unconditional extension of the IMAGE_INSTALL list. You're right, it was a late addition and I didn't realized that I was unconditionally adding it to image.bbclass. My fault, I'll provide a fix. > That leads to (at least) two questions: - Why do we have > isar-exclude-docs here, and also isar-disable-apt-cache? This exclusion was only meant for container images, you don't want to have manpages and APT caches in a container image. As mentioned above, it shouldn't be an unconditional addition to the IMAGE_INSTALL list, but only for container images. > - Does isar-exclude-docs have some issue that prevents the usage > together with openjdk, or are those simply incompatible by design? In any case if someone wants to get rid of manpages for very small devices, the recipe isar-exclude-docs should help on that. The same applies to isar-disable-apt-cache. My error just triggered a hidden issue: the incompatibility between the implementation of the isar-exclude-docs recipe and other components (for example, openjdk). IMO this issue should get fix too (apart from the unconditional addition of the recipe). > In any case, we have a regression. Yes, we do. > Jan Silvano >> +containerize_rootfs() { + local cmd="/bin/dash" + local >> empty_tag="empty" + local full_tag="latest" + local >> oci_img_dir="${WORKDIR}/oci-image" + local rootfs="$1" + >> local rootfs_id="$2" + local container_formats="$3" + + # >> prepare OCI container image skeleton + bbdebug 1 "prepare OCI >> container image skeleton" + rm -rf "${oci_img_dir}" + sudo >> umoci init --layout "${oci_img_dir}" + sudo umoci new --image >> "${oci_img_dir}:${empty_tag}" + sudo umoci config --image >> "${oci_img_dir}:${empty_tag}" \ + --config.cmd="${cmd}" + >> sudo umoci unpack --image "${oci_img_dir}:${empty_tag}" \ + >> "${oci_img_dir}_unpacked" + + # add root filesystem as the flesh >> of the skeleton + sudo cp -a "${rootfs}"/* >> "${oci_img_dir}_unpacked/rootfs/" + # clean-up temporary files + >> sudo find "${oci_img_dir}_unpacked/rootfs/tmp" -mindepth 1 -delete >> + + # pack container image + bbdebug 1 "pack container >> image" + sudo umoci repack --image "${oci_img_dir}:${full_tag}" >> \ + "${oci_img_dir}_unpacked" + sudo umoci remove --image >> "${oci_img_dir}:${empty_tag}" + sudo rm -rf >> "${oci_img_dir}_unpacked" + + # no root needed anymore + sudo >> chown --recursive $(id -u):$(id -g) "${oci_img_dir}" + + # >> convert the OCI container image to the desired format + >> image_name="isar-${rootfs_id}" + for image_type in >> ${CONTAINER_FORMATS} ; do + >> image_archive="${DEPLOY_DIR_IMAGE}/${rootfs_id}-${image_type}.tar" >> + bbdebug 1 "Creating container image type: ${image_type}" + >> case "${image_type}" in + "docker-archive" | >> "oci-archive") + if [ "${image_type}" = >> "oci-archive" ] ; then + >> target="${image_type}:${image_archive}:latest" + >> else + >> target="${image_type}:${image_archive}:${image_name}:latest" + >> fi + rm -f "${image_archive}" "${image_archive}.xz" >> + bbdebug 2 "Converting OCI image to ${image_type}" >> + skopeo --insecure-policy copy \ + >> "oci:${oci_img_dir}:${full_tag}" "${target}" + >> bbdebug 2 "Compressing image" + xz -T0 >> "${image_archive}" + ;; + "oci") + >> tar --create --xz --directory "${oci_img_dir}" \ + >> --file "${image_archive}.xz" . + ;; + >> "docker-daemon" | "containers-storage") + if [ -f >> /.dockerenv ] || [ -f /run/.containerenv ] ; then + >> die "Adding the container image to a container runtime >> (${image_type}) not supported if running from a container (e.g. >> 'kas-container')" + fi + skopeo >> --insecure-policy copy \ + >> "oci:${oci_img_dir}:${full_tag}" \ + >> "${image_type}:${image_name}:latest" + ;; + >> *) + die "Unsupported format for >> containerize_rootfs: ${image_type}" + ;; + >> esac + done +} + diff --git a/meta/classes/image.bbclass >> b/meta/classes/image.bbclass index eddc444..ec93cab 100644 --- >> a/meta/classes/image.bbclass +++ b/meta/classes/image.bbclass @@ >> -76,6 +76,7 @@ inherit image-tools-extension inherit >> image-postproc-extension inherit image-locales-extension inherit >> image-account-extension +inherit image-container-extension >> >> # Extra space for rootfs in MB ROOTFS_EXTRA ?= "64" >> Silvano Cirujano Cuesta
Apparently my mail client mangled my previous reply... :-/ Thanks to Jan for making me aware of it. This one should be readable. On 08/04/2021 19:55, Jan Kiszka wrote: > On 29.03.21 17:56, [ext] Silvano Cirujano Cuesta wrote: >> This class can be used to create container images which root filesystem >> is that generated by the do_rootfs task. >> >> Containerized root filesystems have following possible use-cases: >> - Using ISAR as a container image builder. >> - Simplify distribution of runtime rootfs (binaries, libraries, >> configurations, ...) for application development or testing. >> - Distributing SDKs. >> >> Signed-off-by: Silvano Cirujano Cuesta <silvano.cirujano-cuesta@siemens.com> >> --- >> .../classes/image-container-extension.bbclass | 82 +++++++++++++++++++ >> meta/classes/image.bbclass | 1 + >> 2 files changed, 83 insertions(+) >> create mode 100644 meta/classes/image-container-extension.bbclass >> >> diff --git a/meta/classes/image-container-extension.bbclass b/meta/classes/image-container-extension.bbclass >> new file mode 100644 >> index 0000000..f693627 >> --- /dev/null >> +++ b/meta/classes/image-container-extension.bbclass >> @@ -0,0 +1,82 @@ >> +# This software is a part of ISAR. >> +# Copyright (C) Siemens AG, 2021 >> +# >> +# SPDX-License-Identifier: MIT >> +# >> +# This class extends the image.bbclass for containerizing the root filesystem. >> + >> +CONTAINER_FORMATS ?= "docker-archive" >> +IMAGE_INSTALL += "isar-exclude-docs isar-disable-apt-cache" >> + > > Quirin tried latest Isar on meta-iot2050, and that now fails during > setup of openjdk: > > Setting up openjdk-11-jre-headless:arm64 (11.0.9.1+1-1~deb10u2) ... > > > > update-alternatives: using /usr/lib/jvm/java-11-openjdk-arm64/bin/rmid > to provide /usr/bin/rmid (rmid) in auto mode > > > update-alternatives: error: error creating symbolic link > '/usr/share/man/man1/rmid.1.gz.dpkg-tmp': No such file or directory > > > > While watching the installation, I happened to see isar-exclude-docs > suddenly being installed while not being selected explicitly, > specifically not for the target image. I bet it's coming in via this > class and its unconditional extension of the IMAGE_INSTALL list. You're right, it was a late addition and I didn't realized that I was unconditionally adding it to image.bbclass. This exclusion was only meant for container images, you don't want to have manpages and APT caches in a container image. My fault, I'll provide a fix. > > That leads to (at least) two questions: > - Why do we have isar-exclude-docs here, and also > isar-disable-apt-cache? > - Does isar-exclude-docs have some issue that prevents the usage > together with openjdk, or are those simply incompatible by design? IMO if someone wants to get rid of manpages for very small devices, the recipe isar-exclude-docs should help on that. The same applies to isar-disable-apt-cache. My error just triggered a hidden issue: the incompatibility between the implementation of the isar-exclude-docs recipe and other components (for example, openjdk). IMO this issue should get fix too (apart from the unconditional addition of the recipe). Unless such a configuration is unsupported, but I think that's the scope of the other thread that you opened ;-) > > In any case, we have a regression. Yes, we do. Silvano > > Jan > >> +containerize_rootfs() { >> + local cmd="/bin/dash" >> + local empty_tag="empty" >> + local full_tag="latest" >> + local oci_img_dir="${WORKDIR}/oci-image" >> + local rootfs="$1" >> + local rootfs_id="$2" >> + local container_formats="$3" >> + >> + # prepare OCI container image skeleton >> + bbdebug 1 "prepare OCI container image skeleton" >> + rm -rf "${oci_img_dir}" >> + sudo umoci init --layout "${oci_img_dir}" >> + sudo umoci new --image "${oci_img_dir}:${empty_tag}" >> + sudo umoci config --image "${oci_img_dir}:${empty_tag}" \ >> + --config.cmd="${cmd}" >> + sudo umoci unpack --image "${oci_img_dir}:${empty_tag}" \ >> + "${oci_img_dir}_unpacked" >> + >> + # add root filesystem as the flesh of the skeleton >> + sudo cp -a "${rootfs}"/* "${oci_img_dir}_unpacked/rootfs/" >> + # clean-up temporary files >> + sudo find "${oci_img_dir}_unpacked/rootfs/tmp" -mindepth 1 -delete >> + >> + # pack container image >> + bbdebug 1 "pack container image" >> + sudo umoci repack --image "${oci_img_dir}:${full_tag}" \ >> + "${oci_img_dir}_unpacked" >> + sudo umoci remove --image "${oci_img_dir}:${empty_tag}" >> + sudo rm -rf "${oci_img_dir}_unpacked" >> + >> + # no root needed anymore >> + sudo chown --recursive $(id -u):$(id -g) "${oci_img_dir}" >> + >> + # convert the OCI container image to the desired format >> + image_name="isar-${rootfs_id}" >> + for image_type in ${CONTAINER_FORMATS} ; do >> + image_archive="${DEPLOY_DIR_IMAGE}/${rootfs_id}-${image_type}.tar" >> + bbdebug 1 "Creating container image type: ${image_type}" >> + case "${image_type}" in >> + "docker-archive" | "oci-archive") >> + if [ "${image_type}" = "oci-archive" ] ; then >> + target="${image_type}:${image_archive}:latest" >> + else >> + target="${image_type}:${image_archive}:${image_name}:latest" >> + fi >> + rm -f "${image_archive}" "${image_archive}.xz" >> + bbdebug 2 "Converting OCI image to ${image_type}" >> + skopeo --insecure-policy copy \ >> + "oci:${oci_img_dir}:${full_tag}" "${target}" >> + bbdebug 2 "Compressing image" >> + xz -T0 "${image_archive}" >> + ;; >> + "oci") >> + tar --create --xz --directory "${oci_img_dir}" \ >> + --file "${image_archive}.xz" . >> + ;; >> + "docker-daemon" | "containers-storage") >> + if [ -f /.dockerenv ] || [ -f /run/.containerenv ] ; then >> + die "Adding the container image to a container runtime (${image_type}) not supported if running from a container (e.g. 'kas-container')" >> + fi >> + skopeo --insecure-policy copy \ >> + "oci:${oci_img_dir}:${full_tag}" \ >> + "${image_type}:${image_name}:latest" >> + ;; >> + *) >> + die "Unsupported format for containerize_rootfs: ${image_type}" >> + ;; >> + esac >> + done >> +} >> + >> diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass >> index eddc444..ec93cab 100644 >> --- a/meta/classes/image.bbclass >> +++ b/meta/classes/image.bbclass >> @@ -76,6 +76,7 @@ inherit image-tools-extension >> inherit image-postproc-extension >> inherit image-locales-extension >> inherit image-account-extension >> +inherit image-container-extension >> >> # Extra space for rootfs in MB >> ROOTFS_EXTRA ?= "64" >> >
On Mon, 2021-03-29 at 17:56 +0200, [ext] Silvano Cirujano Cuesta wrote: > This class can be used to create container images which root filesystem > is that generated by the do_rootfs task. > > Containerized root filesystems have following possible use-cases: > - Using ISAR as a container image builder. > - Simplify distribution of runtime rootfs (binaries, libraries, > configurations, ...) for application development or testing. > - Distributing SDKs. > > Signed-off-by: Silvano Cirujano Cuesta <silvano.cirujano-cuesta@siemens.com> > --- > .../classes/image-container-extension.bbclass | 82 +++++++++++++++++++ > meta/classes/image.bbclass | 1 + > 2 files changed, 83 insertions(+) > create mode 100644 meta/classes/image-container-extension.bbclass > > diff --git a/meta/classes/image-container-extension.bbclass b/meta/classes/image-container-extension.bbclass > new file mode 100644 > index 0000000..f693627 > --- /dev/null > +++ b/meta/classes/image-container-extension.bbclass > @@ -0,0 +1,82 @@ > +# This software is a part of ISAR. > +# Copyright (C) Siemens AG, 2021 > +# > +# SPDX-License-Identifier: MIT > +# > +# This class extends the image.bbclass for containerizing the root filesystem. > + > +CONTAINER_FORMATS ?= "docker-archive" > +IMAGE_INSTALL += "isar-exclude-docs isar-disable-apt-cache" > + > +containerize_rootfs() { > + local cmd="/bin/dash" > + local empty_tag="empty" > + local full_tag="latest" > + local oci_img_dir="${WORKDIR}/oci-image" > + local rootfs="$1" > + local rootfs_id="$2" > + local container_formats="$3" > + > + # prepare OCI container image skeleton > + bbdebug 1 "prepare OCI container image skeleton" > + rm -rf "${oci_img_dir}" > + sudo umoci init --layout "${oci_img_dir}" Sorry for replying to this quite old thread but the timestamp of the commit actually delivered this patch as root cause: Who makes sure umoci is available? We updated to recent ISAR-next and got a build failure. umoci not found. Sounds like a missing IMAGER_INSTALL += "umoci" or similar. Florian > + sudo umoci new --image "${oci_img_dir}:${empty_tag}" > + sudo umoci config --image "${oci_img_dir}:${empty_tag}" \ > + --config.cmd="${cmd}" > + sudo umoci unpack --image "${oci_img_dir}:${empty_tag}" \ > + "${oci_img_dir}_unpacked" > + > + # add root filesystem as the flesh of the skeleton > + sudo cp -a "${rootfs}"/* "${oci_img_dir}_unpacked/rootfs/" > + # clean-up temporary files > + sudo find "${oci_img_dir}_unpacked/rootfs/tmp" -mindepth 1 -delete > + > + # pack container image > + bbdebug 1 "pack container image" > + sudo umoci repack --image "${oci_img_dir}:${full_tag}" \ > + "${oci_img_dir}_unpacked" > + sudo umoci remove --image "${oci_img_dir}:${empty_tag}" > + sudo rm -rf "${oci_img_dir}_unpacked" > + > + # no root needed anymore > + sudo chown --recursive $(id -u):$(id -g) "${oci_img_dir}" > + > + # convert the OCI container image to the desired format > + image_name="isar-${rootfs_id}" > + for image_type in ${CONTAINER_FORMATS} ; do > + image_archive="${DEPLOY_DIR_IMAGE}/${rootfs_id}-${image_type}.tar" > + bbdebug 1 "Creating container image type: ${image_type}" > + case "${image_type}" in > + "docker-archive" | "oci-archive") > + if [ "${image_type}" = "oci-archive" ] ; then > + target="${image_type}:${image_archive}:latest" > + else > + target="${image_type}:${image_archive}:${image_name}:latest" > + fi > + rm -f "${image_archive}" "${image_archive}.xz" > + bbdebug 2 "Converting OCI image to ${image_type}" > + skopeo --insecure-policy copy \ > + "oci:${oci_img_dir}:${full_tag}" "${target}" > + bbdebug 2 "Compressing image" > + xz -T0 "${image_archive}" > + ;; > + "oci") > + tar --create --xz --directory "${oci_img_dir}" \ > + --file "${image_archive}.xz" . > + ;; > + "docker-daemon" | "containers-storage") > + if [ -f /.dockerenv ] || [ -f /run/.containerenv ] ; then > + die "Adding the container image to a container runtime (${image_type}) not supported if running from a container (e.g. 'kas-container')" > + fi > + skopeo --insecure-policy copy \ > + "oci:${oci_img_dir}:${full_tag}" \ > + "${image_type}:${image_name}:latest" > + ;; > + *) > + die "Unsupported format for containerize_rootfs: ${image_type}" > + ;; > + esac > + done > +} > + > diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass > index eddc444..ec93cab 100644 > --- a/meta/classes/image.bbclass > +++ b/meta/classes/image.bbclass > @@ -76,6 +76,7 @@ inherit image-tools-extension > inherit image-postproc-extension > inherit image-locales-extension > inherit image-account-extension > +inherit image-container-extension > > # Extra space for rootfs in MB > ROOTFS_EXTRA ?= "64" > -- > 2.30.2 >
On 19.08.21 22:43, Bezdeka, Florian wrote: > On Mon, 2021-03-29 at 17:56 +0200, [ext] Silvano Cirujano Cuesta wrote: >> This class can be used to create container images which root filesystem >> is that generated by the do_rootfs task. >> >> Containerized root filesystems have following possible use-cases: >> - Using ISAR as a container image builder. >> - Simplify distribution of runtime rootfs (binaries, libraries, >> configurations, ...) for application development or testing. >> - Distributing SDKs. >> >> Signed-off-by: Silvano Cirujano Cuesta <silvano.cirujano-cuesta@siemens.com> >> --- >> .../classes/image-container-extension.bbclass | 82 +++++++++++++++++++ >> meta/classes/image.bbclass | 1 + >> 2 files changed, 83 insertions(+) >> create mode 100644 meta/classes/image-container-extension.bbclass >> >> diff --git a/meta/classes/image-container-extension.bbclass b/meta/classes/image-container-extension.bbclass >> new file mode 100644 >> index 0000000..f693627 >> --- /dev/null >> +++ b/meta/classes/image-container-extension.bbclass >> @@ -0,0 +1,82 @@ >> +# This software is a part of ISAR. >> +# Copyright (C) Siemens AG, 2021 >> +# >> +# SPDX-License-Identifier: MIT >> +# >> +# This class extends the image.bbclass for containerizing the root filesystem. >> + >> +CONTAINER_FORMATS ?= "docker-archive" >> +IMAGE_INSTALL += "isar-exclude-docs isar-disable-apt-cache" >> + >> +containerize_rootfs() { >> + local cmd="/bin/dash" >> + local empty_tag="empty" >> + local full_tag="latest" >> + local oci_img_dir="${WORKDIR}/oci-image" >> + local rootfs="$1" >> + local rootfs_id="$2" >> + local container_formats="$3" >> + >> + # prepare OCI container image skeleton >> + bbdebug 1 "prepare OCI container image skeleton" >> + rm -rf "${oci_img_dir}" >> + sudo umoci init --layout "${oci_img_dir}" > > Sorry for replying to this quite old thread but the timestamp of the > commit actually delivered this patch as root cause: > > Who makes sure umoci is available? > The build environment you need to prepare for Isar, either a manually installed Debian or a container like kas-isar. Jan
On 20/08/2021 08:54, Jan Kiszka wrote: > On 19.08.21 22:43, Bezdeka, Florian wrote: >> On Mon, 2021-03-29 at 17:56 +0200, [ext] Silvano Cirujano Cuesta wrote: >>> This class can be used to create container images which root filesystem >>> is that generated by the do_rootfs task. >>> >>> Containerized root filesystems have following possible use-cases: >>> - Using ISAR as a container image builder. >>> - Simplify distribution of runtime rootfs (binaries, libraries, >>> configurations, ...) for application development or testing. >>> - Distributing SDKs. >>> >>> Signed-off-by: Silvano Cirujano Cuesta <silvano.cirujano-cuesta@siemens.com> >>> --- >>> .../classes/image-container-extension.bbclass | 82 +++++++++++++++++++ >>> meta/classes/image.bbclass | 1 + >>> 2 files changed, 83 insertions(+) >>> create mode 100644 meta/classes/image-container-extension.bbclass >>> >>> diff --git a/meta/classes/image-container-extension.bbclass b/meta/classes/image-container-extension.bbclass >>> new file mode 100644 >>> index 0000000..f693627 >>> --- /dev/null >>> +++ b/meta/classes/image-container-extension.bbclass >>> @@ -0,0 +1,82 @@ >>> +# This software is a part of ISAR. >>> +# Copyright (C) Siemens AG, 2021 >>> +# >>> +# SPDX-License-Identifier: MIT >>> +# >>> +# This class extends the image.bbclass for containerizing the root filesystem. >>> + >>> +CONTAINER_FORMATS ?= "docker-archive" >>> +IMAGE_INSTALL += "isar-exclude-docs isar-disable-apt-cache" >>> + >>> +containerize_rootfs() { >>> + local cmd="/bin/dash" >>> + local empty_tag="empty" >>> + local full_tag="latest" >>> + local oci_img_dir="${WORKDIR}/oci-image" >>> + local rootfs="$1" >>> + local rootfs_id="$2" >>> + local container_formats="$3" >>> + >>> + # prepare OCI container image skeleton >>> + bbdebug 1 "prepare OCI container image skeleton" >>> + rm -rf "${oci_img_dir}" >>> + sudo umoci init --layout "${oci_img_dir}" >> >> Sorry for replying to this quite old thread but the timestamp of the >> commit actually delivered this patch as root cause: >> >> Who makes sure umoci is available? >> > > The build environment you need to prepare for Isar, either a manually > installed Debian or a container like kas-isar. > > Jan > Let me add that the needed packages are officially available for the freshly released Debian Stable/Bullseye/11. Manually backporting them to Buster might work though if stricktly needed...
diff --git a/meta/classes/image-container-extension.bbclass b/meta/classes/image-container-extension.bbclass new file mode 100644 index 0000000..f693627 --- /dev/null +++ b/meta/classes/image-container-extension.bbclass @@ -0,0 +1,82 @@ +# This software is a part of ISAR. +# Copyright (C) Siemens AG, 2021 +# +# SPDX-License-Identifier: MIT +# +# This class extends the image.bbclass for containerizing the root filesystem. + +CONTAINER_FORMATS ?= "docker-archive" +IMAGE_INSTALL += "isar-exclude-docs isar-disable-apt-cache" + +containerize_rootfs() { + local cmd="/bin/dash" + local empty_tag="empty" + local full_tag="latest" + local oci_img_dir="${WORKDIR}/oci-image" + local rootfs="$1" + local rootfs_id="$2" + local container_formats="$3" + + # prepare OCI container image skeleton + bbdebug 1 "prepare OCI container image skeleton" + rm -rf "${oci_img_dir}" + sudo umoci init --layout "${oci_img_dir}" + sudo umoci new --image "${oci_img_dir}:${empty_tag}" + sudo umoci config --image "${oci_img_dir}:${empty_tag}" \ + --config.cmd="${cmd}" + sudo umoci unpack --image "${oci_img_dir}:${empty_tag}" \ + "${oci_img_dir}_unpacked" + + # add root filesystem as the flesh of the skeleton + sudo cp -a "${rootfs}"/* "${oci_img_dir}_unpacked/rootfs/" + # clean-up temporary files + sudo find "${oci_img_dir}_unpacked/rootfs/tmp" -mindepth 1 -delete + + # pack container image + bbdebug 1 "pack container image" + sudo umoci repack --image "${oci_img_dir}:${full_tag}" \ + "${oci_img_dir}_unpacked" + sudo umoci remove --image "${oci_img_dir}:${empty_tag}" + sudo rm -rf "${oci_img_dir}_unpacked" + + # no root needed anymore + sudo chown --recursive $(id -u):$(id -g) "${oci_img_dir}" + + # convert the OCI container image to the desired format + image_name="isar-${rootfs_id}" + for image_type in ${CONTAINER_FORMATS} ; do + image_archive="${DEPLOY_DIR_IMAGE}/${rootfs_id}-${image_type}.tar" + bbdebug 1 "Creating container image type: ${image_type}" + case "${image_type}" in + "docker-archive" | "oci-archive") + if [ "${image_type}" = "oci-archive" ] ; then + target="${image_type}:${image_archive}:latest" + else + target="${image_type}:${image_archive}:${image_name}:latest" + fi + rm -f "${image_archive}" "${image_archive}.xz" + bbdebug 2 "Converting OCI image to ${image_type}" + skopeo --insecure-policy copy \ + "oci:${oci_img_dir}:${full_tag}" "${target}" + bbdebug 2 "Compressing image" + xz -T0 "${image_archive}" + ;; + "oci") + tar --create --xz --directory "${oci_img_dir}" \ + --file "${image_archive}.xz" . + ;; + "docker-daemon" | "containers-storage") + if [ -f /.dockerenv ] || [ -f /run/.containerenv ] ; then + die "Adding the container image to a container runtime (${image_type}) not supported if running from a container (e.g. 'kas-container')" + fi + skopeo --insecure-policy copy \ + "oci:${oci_img_dir}:${full_tag}" \ + "${image_type}:${image_name}:latest" + ;; + *) + die "Unsupported format for containerize_rootfs: ${image_type}" + ;; + esac + done +} + diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass index eddc444..ec93cab 100644 --- a/meta/classes/image.bbclass +++ b/meta/classes/image.bbclass @@ -76,6 +76,7 @@ inherit image-tools-extension inherit image-postproc-extension inherit image-locales-extension inherit image-account-extension +inherit image-container-extension # Extra space for rootfs in MB ROOTFS_EXTRA ?= "64"
This class can be used to create container images which root filesystem is that generated by the do_rootfs task. Containerized root filesystems have following possible use-cases: - Using ISAR as a container image builder. - Simplify distribution of runtime rootfs (binaries, libraries, configurations, ...) for application development or testing. - Distributing SDKs. Signed-off-by: Silvano Cirujano Cuesta <silvano.cirujano-cuesta@siemens.com> --- .../classes/image-container-extension.bbclass | 82 +++++++++++++++++++ meta/classes/image.bbclass | 1 + 2 files changed, 83 insertions(+) create mode 100644 meta/classes/image-container-extension.bbclass