[v2,5/5] doc: Describe how to use the container fetcher and loader

From: Jan Kiszka <jan.kiszka@siemens.com>

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
---
 doc/user_manual.md | 60 ++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 60 insertions(+)

On Mon, 2024-07-15 at 12:08 +0200, Jan Kiszka wrote:
> From: Jan Kiszka <jan.kiszka@siemens.com>
> 
> Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
> ---
>  doc/user_manual.md | 60 ++++++++++++++++++++++++++++++++++++++++++++++
>  1 file changed, 60 insertions(+)
> 
> diff --git a/doc/user_manual.md b/doc/user_manual.md
> index 776ae52c..733b2b30 100644
> --- a/doc/user_manual.md
> +++ b/doc/user_manual.md
> @@ -1519,3 +1519,63 @@ SBUILD_CHROOT_PREINSTALL_EXTRA += "<base packages>"
>  
>  Then, in the dpkg recipe of your package, simply set `SBUILD_FLAVOR = "<your flavor>"`.
>  To install additional packages into the sbuild chroot, add them to
> `SBUILD_CHROOT_PREINSTALL_EXTRA`.
> +
> +## Pre-install container images
> +
> +If an isar-generated image shall provides a container runtime, it may also be

If an isar-generated image shall provide a container runtime...

> +desirable to pre-install container images to avoid having to download them on
> +first boot or because they may not be accessible outside of the build
> +environment. Isar supports this scenario via two services, a container fetcher
> +and a container loader.
> +
> +### Bitbake fetcher for containers
> +
> +The bitbake fetching protocol "docker://" allows to download pre-built images
> +from container registries. The URL consists of the image path, followed by
> +a recommened digests in the form `digest=sha256:<sha256sum>` and an optional
a recommended...

> +tag in the form `tag=<tag>`. A digest is preferred over a tag to identify the
> +an when fetching as it also allows to validate its integrity. If tag is not
... to indentify the ?an? when fetching...

> +specified, `latest` is used as tag name.
> +
> +When specifying a multi-arch image, the fetcher will download the images for
> +all available architectures. If this is not desired, directly specify the
> +digest of the desired architecture manifest instead of that the manifest list.
> +
> +The fetched container image are stored in a directory in the `WORKDIR` of the
> +requesting recipe. When a multi-arch image was specified, only the image
> +matching `PACKAGE_ARCH` will be stored. The name of the image directory is
> +derived from the container image name, replacing all `/` with `.`.
> +
> +### Container loader helpers
> +
> +To create a Debian package which can carry container images and load them into
> +local storage of docker or podman, there is a set of helpers available. To use
> +them into an own recipe, add
> +`require recipes-support/container-loader/docker-loader.inc` when using docker
> +and `require recipes-support/container-loader/podman-loader.inc` when using
> +podman. The loader will try to transfer the packaged image into the container
> +runtime storage on boot, but only if no container image of the same name and
> +tag is present already.
> +
> +Unless `CONTAINER_DELETE_AFTER_LOAD` is set to `1`, the source container images
> +remain by default available and may be used again for loading the storage after
> +it may have been emptied later on (factory reset).
> +
> +Source container images may either be fetched as binaries from a registry, see
> +above, or built via isar as well.
> +
> +### Example
> +
> +This creates debian package with will download, package and then load the
> +`debian:bookworm-20240701-slim` container image into the docker container
> +storage. The package will depend on `docker.io`, thus ensure that that basic
> +runtime services are installed on the target as well. The packaged image will
> +be deleted from the target device's rootfs after successful import.
> +
> +```
> +require recipes-support/container-loader/docker-loader.inc
> +
> +CONTAINER_DELETE_AFTER_LOAD = "1"
> +
> +SRC_URI +=
> "docker://debian;digest=sha256:f528891ab1aa484bf7233dbcc84f3c806c3e427571d75510a9d74bb5ec535b33;ta
> g=bookworm-20240701-slim"
> +```

Benedikt

On 16.07.24 14:56, Niedermayr, Benedikt (T CED OES-DE) wrote:
> On Mon, 2024-07-15 at 12:08 +0200, Jan Kiszka wrote:
>> From: Jan Kiszka <jan.kiszka@siemens.com>
>>
>> Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
>> ---
>>  doc/user_manual.md | 60 ++++++++++++++++++++++++++++++++++++++++++++++
>>  1 file changed, 60 insertions(+)
>>
>> diff --git a/doc/user_manual.md b/doc/user_manual.md
>> index 776ae52c..733b2b30 100644
>> --- a/doc/user_manual.md
>> +++ b/doc/user_manual.md
>> @@ -1519,3 +1519,63 @@ SBUILD_CHROOT_PREINSTALL_EXTRA += "<base packages>"
>>  
>>  Then, in the dpkg recipe of your package, simply set `SBUILD_FLAVOR = "<your flavor>"`.
>>  To install additional packages into the sbuild chroot, add them to
>> `SBUILD_CHROOT_PREINSTALL_EXTRA`.
>> +
>> +## Pre-install container images
>> +
>> +If an isar-generated image shall provides a container runtime, it may also be
> 
> If an isar-generated image shall provide a container runtime...
> 
>> +desirable to pre-install container images to avoid having to download them on
>> +first boot or because they may not be accessible outside of the build
>> +environment. Isar supports this scenario via two services, a container fetcher
>> +and a container loader.
>> +
>> +### Bitbake fetcher for containers
>> +
>> +The bitbake fetching protocol "docker://" allows to download pre-built images
>> +from container registries. The URL consists of the image path, followed by
>> +a recommened digests in the form `digest=sha256:<sha256sum>` and an optional
> a recommended...
> 
>> +tag in the form `tag=<tag>`. A digest is preferred over a tag to identify the
>> +an when fetching as it also allows to validate its integrity. If tag is not
> ... to indentify the ?an? when fetching...
> 
>> +specified, `latest` is used as tag name.
>> +
>> +When specifying a multi-arch image, the fetcher will download the images for
>> +all available architectures. If this is not desired, directly specify the
>> +digest of the desired architecture manifest instead of that the manifest list.
>> +
>> +The fetched container image are stored in a directory in the `WORKDIR` of the
>> +requesting recipe. When a multi-arch image was specified, only the image
>> +matching `PACKAGE_ARCH` will be stored. The name of the image directory is
>> +derived from the container image name, replacing all `/` with `.`.
>> +
>> +### Container loader helpers
>> +
>> +To create a Debian package which can carry container images and load them into
>> +local storage of docker or podman, there is a set of helpers available. To use
>> +them into an own recipe, add
>> +`require recipes-support/container-loader/docker-loader.inc` when using docker
>> +and `require recipes-support/container-loader/podman-loader.inc` when using
>> +podman. The loader will try to transfer the packaged image into the container
>> +runtime storage on boot, but only if no container image of the same name and
>> +tag is present already.
>> +
>> +Unless `CONTAINER_DELETE_AFTER_LOAD` is set to `1`, the source container images
>> +remain by default available and may be used again for loading the storage after
>> +it may have been emptied later on (factory reset).
>> +
>> +Source container images may either be fetched as binaries from a registry, see
>> +above, or built via isar as well.
>> +
>> +### Example
>> +
>> +This creates debian package with will download, package and then load the
>> +`debian:bookworm-20240701-slim` container image into the docker container
>> +storage. The package will depend on `docker.io`, thus ensure that that basic
>> +runtime services are installed on the target as well. The packaged image will
>> +be deleted from the target device's rootfs after successful import.
>> +
>> +```
>> +require recipes-support/container-loader/docker-loader.inc
>> +
>> +CONTAINER_DELETE_AFTER_LOAD = "1"
>> +
>> +SRC_URI +=
>> "docker://debian;digest=sha256:f528891ab1aa484bf7233dbcc84f3c806c3e427571d75510a9d74bb5ec535b33;ta
>> g=bookworm-20240701-slim"
>> +```
> 
> Benedikt
> 

Thanks for actually reading it :). Update will follow.

Jan

'Jan Kiszka' via isar-users <isar-users@googlegroups.com>, Montag, 15. Juli 2024 12:09:
> Subject: [PATCH v2 5/5] doc: Describe how to use the container fetcher and loader
> 
> From: Jan Kiszka <jan.kiszka@siemens.com>
> 
> Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
> ---
>  doc/user_manual.md | 60
> ++++++++++++++++++++++++++++++++++++++++++++++
>  1 file changed, 60 insertions(+)
> 
> diff --git a/doc/user_manual.md b/doc/user_manual.md
> index 776ae52c..733b2b30 100644
> --- a/doc/user_manual.md
> +++ b/doc/user_manual.md
> @@ -1519,3 +1519,63 @@ SBUILD_CHROOT_PREINSTALL_EXTRA += "<base
> packages>"
> 
>  Then, in the dpkg recipe of your package, simply set `SBUILD_FLAVOR = "<your
> flavor>"`.
>  To install additional packages into the sbuild chroot, add them to
> `SBUILD_CHROOT_PREINSTALL_EXTRA`.
> +
> +## Pre-install container images
> +
> +If an isar-generated image shall provides a container runtime, it may also be
> +desirable to pre-install container images to avoid having to download them on
> +first boot or because they may not be accessible outside of the build
> +environment. Isar supports this scenario via two services, a container fetcher
> +and a container loader.
> +
> +### Bitbake fetcher for containers
> +
> +The bitbake fetching protocol "docker://" allows to download pre-built images
> +from container registries. The URL consists of the image path, followed by
> +a recommened digests in the form `digest=sha256:<sha256sum>` and an optional

recommended *digest

> +tag in the form `tag=<tag>`. A digest is preferred over a tag to identify the
> +an when fetching as it also allows to validate its integrity. If tag is not
> +specified, `latest` is used as tag name.
> +
> +When specifying a multi-arch image, the fetcher will download the images for
> +all available architectures. If this is not desired, directly specify the
> +digest of the desired architecture manifest instead of that the manifest list.
> +
> +The fetched container image are stored in a directory in the `WORKDIR` of the

*images

> +requesting recipe. When a multi-arch image was specified, only the image
> +matching `PACKAGE_ARCH` will be stored. The name of the image directory is
> +derived from the container image name, replacing all `/` with `.`.
> +
> +### Container loader helpers
> +
> +To create a Debian package which can carry container images and load them into
> +local storage of docker or podman, there is a set of helpers available. To use
> +them into an own recipe, add

them *in an own

> +`require recipes-support/container-loader/docker-loader.inc` when using docker
> +and `require recipes-support/container-loader/podman-loader.inc` when using
> +podman. The loader will try to transfer the packaged image into the container
> +runtime storage on boot, but only if no container image of the same name and
> +tag is present already.
> +
> +Unless `CONTAINER_DELETE_AFTER_LOAD` is set to `1`, the source container
> images
> +remain by default available and may be used again for loading the storage after
> +it may have been emptied later on (factory reset).
> +
> +Source container images may either be fetched as binaries from a registry, see
> +above, or built via isar as well.
> +
> +### Example
> +
> +This creates debian package with will download, package and then load the

*which will download

> +`debian:bookworm-20240701-slim` container image into the docker container
> +storage. The package will depend on `docker.io`, thus ensure that that basic

*ensuring?

Adriaan

> +runtime services are installed on the target as well. The packaged image will
> +be deleted from the target device's rootfs after successful import.
> +
> +```
> +require recipes-support/container-loader/docker-loader.inc
> +
> +CONTAINER_DELETE_AFTER_LOAD = "1"
> +
> +SRC_URI +=
> "docker://debian;digest=sha256:f528891ab1aa484bf7233dbcc84f3c806c3e427571d75
> 510a9d74bb5ec535b33;tag=bookworm-20240701-slim"
> +```
> --
> 2.43.0
> 
> --
> You received this message because you are subscribed to the Google Groups "isar-
> users" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to
> isar-users+unsubscribe@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/isar-
> users/2e1537004e7ad48f33cd3d3daf2b8f7c3294d31a.1721038111.git.jan.kiszka%40s
> iemens.com.