Message ID | 2e1537004e7ad48f33cd3d3daf2b8f7c3294d31a.1721038111.git.jan.kiszka@siemens.com |
---|---|
State | Superseded, archived |
Headers | show |
Series | Introduce container fetcher and pre-loader | expand |
On Mon, 2024-07-15 at 12:08 +0200, Jan Kiszka wrote: > From: Jan Kiszka <jan.kiszka@siemens.com> > > Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com> > --- > doc/user_manual.md | 60 ++++++++++++++++++++++++++++++++++++++++++++++ > 1 file changed, 60 insertions(+) > > diff --git a/doc/user_manual.md b/doc/user_manual.md > index 776ae52c..733b2b30 100644 > --- a/doc/user_manual.md > +++ b/doc/user_manual.md > @@ -1519,3 +1519,63 @@ SBUILD_CHROOT_PREINSTALL_EXTRA += "<base packages>" > > Then, in the dpkg recipe of your package, simply set `SBUILD_FLAVOR = "<your flavor>"`. > To install additional packages into the sbuild chroot, add them to > `SBUILD_CHROOT_PREINSTALL_EXTRA`. > + > +## Pre-install container images > + > +If an isar-generated image shall provides a container runtime, it may also be If an isar-generated image shall provide a container runtime... > +desirable to pre-install container images to avoid having to download them on > +first boot or because they may not be accessible outside of the build > +environment. Isar supports this scenario via two services, a container fetcher > +and a container loader. > + > +### Bitbake fetcher for containers > + > +The bitbake fetching protocol "docker://" allows to download pre-built images > +from container registries. The URL consists of the image path, followed by > +a recommened digests in the form `digest=sha256:<sha256sum>` and an optional a recommended... > +tag in the form `tag=<tag>`. A digest is preferred over a tag to identify the > +an when fetching as it also allows to validate its integrity. If tag is not ... to indentify the ?an? when fetching... > +specified, `latest` is used as tag name. > + > +When specifying a multi-arch image, the fetcher will download the images for > +all available architectures. If this is not desired, directly specify the > +digest of the desired architecture manifest instead of that the manifest list. > + > +The fetched container image are stored in a directory in the `WORKDIR` of the > +requesting recipe. When a multi-arch image was specified, only the image > +matching `PACKAGE_ARCH` will be stored. The name of the image directory is > +derived from the container image name, replacing all `/` with `.`. > + > +### Container loader helpers > + > +To create a Debian package which can carry container images and load them into > +local storage of docker or podman, there is a set of helpers available. To use > +them into an own recipe, add > +`require recipes-support/container-loader/docker-loader.inc` when using docker > +and `require recipes-support/container-loader/podman-loader.inc` when using > +podman. The loader will try to transfer the packaged image into the container > +runtime storage on boot, but only if no container image of the same name and > +tag is present already. > + > +Unless `CONTAINER_DELETE_AFTER_LOAD` is set to `1`, the source container images > +remain by default available and may be used again for loading the storage after > +it may have been emptied later on (factory reset). > + > +Source container images may either be fetched as binaries from a registry, see > +above, or built via isar as well. > + > +### Example > + > +This creates debian package with will download, package and then load the > +`debian:bookworm-20240701-slim` container image into the docker container > +storage. The package will depend on `docker.io`, thus ensure that that basic > +runtime services are installed on the target as well. The packaged image will > +be deleted from the target device's rootfs after successful import. > + > +``` > +require recipes-support/container-loader/docker-loader.inc > + > +CONTAINER_DELETE_AFTER_LOAD = "1" > + > +SRC_URI += > "docker://debian;digest=sha256:f528891ab1aa484bf7233dbcc84f3c806c3e427571d75510a9d74bb5ec535b33;ta > g=bookworm-20240701-slim" > +``` Benedikt
On 16.07.24 14:56, Niedermayr, Benedikt (T CED OES-DE) wrote: > On Mon, 2024-07-15 at 12:08 +0200, Jan Kiszka wrote: >> From: Jan Kiszka <jan.kiszka@siemens.com> >> >> Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com> >> --- >> doc/user_manual.md | 60 ++++++++++++++++++++++++++++++++++++++++++++++ >> 1 file changed, 60 insertions(+) >> >> diff --git a/doc/user_manual.md b/doc/user_manual.md >> index 776ae52c..733b2b30 100644 >> --- a/doc/user_manual.md >> +++ b/doc/user_manual.md >> @@ -1519,3 +1519,63 @@ SBUILD_CHROOT_PREINSTALL_EXTRA += "<base packages>" >> >> Then, in the dpkg recipe of your package, simply set `SBUILD_FLAVOR = "<your flavor>"`. >> To install additional packages into the sbuild chroot, add them to >> `SBUILD_CHROOT_PREINSTALL_EXTRA`. >> + >> +## Pre-install container images >> + >> +If an isar-generated image shall provides a container runtime, it may also be > > If an isar-generated image shall provide a container runtime... > >> +desirable to pre-install container images to avoid having to download them on >> +first boot or because they may not be accessible outside of the build >> +environment. Isar supports this scenario via two services, a container fetcher >> +and a container loader. >> + >> +### Bitbake fetcher for containers >> + >> +The bitbake fetching protocol "docker://" allows to download pre-built images >> +from container registries. The URL consists of the image path, followed by >> +a recommened digests in the form `digest=sha256:<sha256sum>` and an optional > a recommended... > >> +tag in the form `tag=<tag>`. A digest is preferred over a tag to identify the >> +an when fetching as it also allows to validate its integrity. If tag is not > ... to indentify the ?an? when fetching... > >> +specified, `latest` is used as tag name. >> + >> +When specifying a multi-arch image, the fetcher will download the images for >> +all available architectures. If this is not desired, directly specify the >> +digest of the desired architecture manifest instead of that the manifest list. >> + >> +The fetched container image are stored in a directory in the `WORKDIR` of the >> +requesting recipe. When a multi-arch image was specified, only the image >> +matching `PACKAGE_ARCH` will be stored. The name of the image directory is >> +derived from the container image name, replacing all `/` with `.`. >> + >> +### Container loader helpers >> + >> +To create a Debian package which can carry container images and load them into >> +local storage of docker or podman, there is a set of helpers available. To use >> +them into an own recipe, add >> +`require recipes-support/container-loader/docker-loader.inc` when using docker >> +and `require recipes-support/container-loader/podman-loader.inc` when using >> +podman. The loader will try to transfer the packaged image into the container >> +runtime storage on boot, but only if no container image of the same name and >> +tag is present already. >> + >> +Unless `CONTAINER_DELETE_AFTER_LOAD` is set to `1`, the source container images >> +remain by default available and may be used again for loading the storage after >> +it may have been emptied later on (factory reset). >> + >> +Source container images may either be fetched as binaries from a registry, see >> +above, or built via isar as well. >> + >> +### Example >> + >> +This creates debian package with will download, package and then load the >> +`debian:bookworm-20240701-slim` container image into the docker container >> +storage. The package will depend on `docker.io`, thus ensure that that basic >> +runtime services are installed on the target as well. The packaged image will >> +be deleted from the target device's rootfs after successful import. >> + >> +``` >> +require recipes-support/container-loader/docker-loader.inc >> + >> +CONTAINER_DELETE_AFTER_LOAD = "1" >> + >> +SRC_URI += >> "docker://debian;digest=sha256:f528891ab1aa484bf7233dbcc84f3c806c3e427571d75510a9d74bb5ec535b33;ta >> g=bookworm-20240701-slim" >> +``` > > Benedikt > Thanks for actually reading it :). Update will follow. Jan
'Jan Kiszka' via isar-users <isar-users@googlegroups.com>, Montag, 15. Juli 2024 12:09: > Subject: [PATCH v2 5/5] doc: Describe how to use the container fetcher and loader > > From: Jan Kiszka <jan.kiszka@siemens.com> > > Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com> > --- > doc/user_manual.md | 60 > ++++++++++++++++++++++++++++++++++++++++++++++ > 1 file changed, 60 insertions(+) > > diff --git a/doc/user_manual.md b/doc/user_manual.md > index 776ae52c..733b2b30 100644 > --- a/doc/user_manual.md > +++ b/doc/user_manual.md > @@ -1519,3 +1519,63 @@ SBUILD_CHROOT_PREINSTALL_EXTRA += "<base > packages>" > > Then, in the dpkg recipe of your package, simply set `SBUILD_FLAVOR = "<your > flavor>"`. > To install additional packages into the sbuild chroot, add them to > `SBUILD_CHROOT_PREINSTALL_EXTRA`. > + > +## Pre-install container images > + > +If an isar-generated image shall provides a container runtime, it may also be > +desirable to pre-install container images to avoid having to download them on > +first boot or because they may not be accessible outside of the build > +environment. Isar supports this scenario via two services, a container fetcher > +and a container loader. > + > +### Bitbake fetcher for containers > + > +The bitbake fetching protocol "docker://" allows to download pre-built images > +from container registries. The URL consists of the image path, followed by > +a recommened digests in the form `digest=sha256:<sha256sum>` and an optional recommended *digest > +tag in the form `tag=<tag>`. A digest is preferred over a tag to identify the > +an when fetching as it also allows to validate its integrity. If tag is not > +specified, `latest` is used as tag name. > + > +When specifying a multi-arch image, the fetcher will download the images for > +all available architectures. If this is not desired, directly specify the > +digest of the desired architecture manifest instead of that the manifest list. > + > +The fetched container image are stored in a directory in the `WORKDIR` of the *images > +requesting recipe. When a multi-arch image was specified, only the image > +matching `PACKAGE_ARCH` will be stored. The name of the image directory is > +derived from the container image name, replacing all `/` with `.`. > + > +### Container loader helpers > + > +To create a Debian package which can carry container images and load them into > +local storage of docker or podman, there is a set of helpers available. To use > +them into an own recipe, add them *in an own > +`require recipes-support/container-loader/docker-loader.inc` when using docker > +and `require recipes-support/container-loader/podman-loader.inc` when using > +podman. The loader will try to transfer the packaged image into the container > +runtime storage on boot, but only if no container image of the same name and > +tag is present already. > + > +Unless `CONTAINER_DELETE_AFTER_LOAD` is set to `1`, the source container > images > +remain by default available and may be used again for loading the storage after > +it may have been emptied later on (factory reset). > + > +Source container images may either be fetched as binaries from a registry, see > +above, or built via isar as well. > + > +### Example > + > +This creates debian package with will download, package and then load the *which will download > +`debian:bookworm-20240701-slim` container image into the docker container > +storage. The package will depend on `docker.io`, thus ensure that that basic *ensuring? Adriaan > +runtime services are installed on the target as well. The packaged image will > +be deleted from the target device's rootfs after successful import. > + > +``` > +require recipes-support/container-loader/docker-loader.inc > + > +CONTAINER_DELETE_AFTER_LOAD = "1" > + > +SRC_URI += > "docker://debian;digest=sha256:f528891ab1aa484bf7233dbcc84f3c806c3e427571d75 > 510a9d74bb5ec535b33;tag=bookworm-20240701-slim" > +``` > -- > 2.43.0 > > -- > You received this message because you are subscribed to the Google Groups "isar- > users" group. > To unsubscribe from this group and stop receiving emails from it, send an email to > isar-users+unsubscribe@googlegroups.com. > To view this discussion on the web visit https://groups.google.com/d/msgid/isar- > users/2e1537004e7ad48f33cd3d3daf2b8f7c3294d31a.1721038111.git.jan.kiszka%40s > iemens.com.
diff --git a/doc/user_manual.md b/doc/user_manual.md index 776ae52c..733b2b30 100644 --- a/doc/user_manual.md +++ b/doc/user_manual.md @@ -1519,3 +1519,63 @@ SBUILD_CHROOT_PREINSTALL_EXTRA += "<base packages>" Then, in the dpkg recipe of your package, simply set `SBUILD_FLAVOR = "<your flavor>"`. To install additional packages into the sbuild chroot, add them to `SBUILD_CHROOT_PREINSTALL_EXTRA`. + +## Pre-install container images + +If an isar-generated image shall provides a container runtime, it may also be +desirable to pre-install container images to avoid having to download them on +first boot or because they may not be accessible outside of the build +environment. Isar supports this scenario via two services, a container fetcher +and a container loader. + +### Bitbake fetcher for containers + +The bitbake fetching protocol "docker://" allows to download pre-built images +from container registries. The URL consists of the image path, followed by +a recommened digests in the form `digest=sha256:<sha256sum>` and an optional +tag in the form `tag=<tag>`. A digest is preferred over a tag to identify the +an when fetching as it also allows to validate its integrity. If tag is not +specified, `latest` is used as tag name. + +When specifying a multi-arch image, the fetcher will download the images for +all available architectures. If this is not desired, directly specify the +digest of the desired architecture manifest instead of that the manifest list. + +The fetched container image are stored in a directory in the `WORKDIR` of the +requesting recipe. When a multi-arch image was specified, only the image +matching `PACKAGE_ARCH` will be stored. The name of the image directory is +derived from the container image name, replacing all `/` with `.`. + +### Container loader helpers + +To create a Debian package which can carry container images and load them into +local storage of docker or podman, there is a set of helpers available. To use +them into an own recipe, add +`require recipes-support/container-loader/docker-loader.inc` when using docker +and `require recipes-support/container-loader/podman-loader.inc` when using +podman. The loader will try to transfer the packaged image into the container +runtime storage on boot, but only if no container image of the same name and +tag is present already. + +Unless `CONTAINER_DELETE_AFTER_LOAD` is set to `1`, the source container images +remain by default available and may be used again for loading the storage after +it may have been emptied later on (factory reset). + +Source container images may either be fetched as binaries from a registry, see +above, or built via isar as well. + +### Example + +This creates debian package with will download, package and then load the +`debian:bookworm-20240701-slim` container image into the docker container +storage. The package will depend on `docker.io`, thus ensure that that basic +runtime services are installed on the target as well. The packaged image will +be deleted from the target device's rootfs after successful import. + +``` +require recipes-support/container-loader/docker-loader.inc + +CONTAINER_DELETE_AFTER_LOAD = "1" + +SRC_URI += "docker://debian;digest=sha256:f528891ab1aa484bf7233dbcc84f3c806c3e427571d75510a9d74bb5ec535b33;tag=bookworm-20240701-slim" +```