image-account-extension: configure adduser UID/GID pools

For users and groups with an explicit uid/gid set, generate adduser pool
files so that maintainer scripts calling adduser/addgroup during package
installation will reserve the expected IDs.

A new 'reserve-only' flag allows entries to exist solely for pool
reservation without being explicitly created during image postprocessing.

Work-around: /etc/adduser.conf is pre-created with UID_POOL/GID_POOL
directives and --force-confold is passed to dpkg so that our version is
kept when the adduser package is installed. This is needed because
adduser does not support loading configuration fragments from a .d
directory or from environment variables. We want to discuss this!
Do we want to create images from a template richer than bootstrap
so adduser could be pre-installed and possibly its configuration
already patched to use UID_POOL / GID_POOL?

Signed-off-by: Cedric Hombourger <cedric.hombourger@siemens.com>
---
 doc/user_manual.md                            |  44 +++++--
 .../image-account-extension.bbclass           | 113 +++++++++++++++++-
 2 files changed, 145 insertions(+), 12 deletions(-)

On 21.05.26 18:21, 'Cedric Hombourger' via isar-users wrote:
> For users and groups with an explicit uid/gid set, generate adduser pool
> files so that maintainer scripts calling adduser/addgroup during package
> installation will reserve the expected IDs.
> 
> A new 'reserve-only' flag allows entries to exist solely for pool
> reservation without being explicitly created during image postprocessing.
> 
> Work-around: /etc/adduser.conf is pre-created with UID_POOL/GID_POOL
> directives and --force-confold is passed to dpkg so that our version is
> kept when the adduser package is installed. This is needed because
> adduser does not support loading configuration fragments from a .d
> directory or from environment variables. We want to discuss this!
> Do we want to create images from a template richer than bootstrap
> so adduser could be pre-installed and possibly its configuration
> already patched to use UID_POOL / GID_POOL?
> 

This is a valuable starting point for (public) discussions, thanks!

Internally, we already thought about this, also considering to add a way
for deriving this preceeding of the UIDs/GIDs from a "version 1" run of
a build so that you do not have to collect and encode all the data
manually. This would also emulate a normal Debian system lifecycle:
Install a blank version, then add or upgrade packages, thus,
users/groups while maintaining the assignments of the initial installation.

Jan

> Signed-off-by: Cedric Hombourger <cedric.hombourger@siemens.com>
> ---
>  doc/user_manual.md                            |  44 +++++--
>  .../image-account-extension.bbclass           | 113 +++++++++++++++++-
>  2 files changed, 145 insertions(+), 12 deletions(-)
> 
> diff --git a/doc/user_manual.md b/doc/user_manual.md
> index 69e8dfef..3bd2e767 100644
> --- a/doc/user_manual.md
> +++ b/doc/user_manual.md
> @@ -737,7 +737,8 @@ The `GROUP_<groupname>` variable contains the settings of a group named `groupna
>  
>   - `gid` - The numeric group id.
>   - `flags` - A list of additional flags of the group. Those are the currently recognized flags:
> -   - `system` - The group is created using the `--system` parameter.
> +    - `system` - The group is created using the `--system` parameter.
> +    - `reserve-only` - The group is not explicitly created during image postprocessing. Instead, its `gid` is reserved in the adduser GID pool so that packages creating this group via maintainer scripts will use the specified ID.
>  
>  The `USERS` and `USER:<username>` variable works similar to the `GROUPS` and `GROUP:<groupname>` variable. The difference are the accepted flags of the `USER:<username>` variable. It accepts the following flags:
>  
> @@ -750,13 +751,14 @@ The `USERS` and `USER:<username>` variable works similar to the `GROUPS` and `GR
>   - `home` - This changes the default home directory of the user with `usermod --move-home`. Only takes effect when used together with the `create-home` flag.
>   - `shell` - This users login shell
>   - `groups` - A space separated list of groups this user is a member of.
> - - `flags` - A list of additional flags of the user:
> -   - `no-create-home` - `useradd` will be called with `-M` to prevent creation of the users home directory.
> -   - `create-home` - `useradd` will be called with `-m` to force creation of the users home directory.
> -   - `system` - `useradd` will be called with `--system`.
> -   - `allow-empty-password` - Even if the `password` flag is empty, it will still be set. This results in a login without password.
> -   - `clear-text-password` - The `password` flag of the given user contains a clear-text password and not an encrypted version of it.
> -   - `force-passwd-change` - Force the user to change to password on first login.
> +  - `flags` - A list of additional flags of the user:
> +    - `no-create-home` - `useradd` will be called with `-M` to prevent creation of the users home directory.
> +    - `create-home` - `useradd` will be called with `-m` to force creation of the users home directory.
> +    - `system` - `useradd` will be called with `--system`.
> +    - `allow-empty-password` - Even if the `password` flag is empty, it will still be set. This results in a login without password.
> +    - `clear-text-password` - The `password` flag of the given user contains a clear-text password and not an encrypted version of it.
> +    - `force-passwd-change` - Force the user to change to password on first login.
> +    - `reserve-only` - The user is not explicitly created during image postprocessing. Instead, its `uid` is reserved in the adduser UID pool so that packages creating this user via maintainer scripts will use the specified ID.
>  
>  #### Example
>  
> @@ -779,6 +781,32 @@ USER_root[flags] = "create-home system force-passwd-change"
>  
>  Some examples can be also found in `meta-isar/conf/local.conf.sample`.
>  
> +#### UID/GID pool reservation
> +
> +When a user or group entry has an explicit `uid` or `gid` set, it is added to
> +the adduser UID/GID pool. This ensures that packages creating users or groups
> +via their maintainer scripts (e.g. `adduser` or `addgroup`) will allocate the
> +specified IDs. Combined with the `reserve-only` flag, this allows reserving IDs
> +without explicitly creating the accounts:
> +
> +```
> +USERS += "tss"
> +USER_tss[uid] = "666"
> +USER_tss[flags] = "reserve-only"
> +
> +GROUPS += "tss"
> +GROUP_tss[gid] = "666"
> +GROUP_tss[flags] = "reserve-only"
> +
> +GROUPS += "docker"
> +GROUP_docker[gid] = "1234"
> +GROUP_docker[flags] = "reserve-only"
> +```
> +
> +In this example, when `tpm2-abrmd` or `docker.io` are installed, their
> +maintainer scripts will create the `tss` and `docker` accounts using the
> +reserved IDs rather than dynamically allocated ones.
> +
>  #### Home directory contents prefilling
>  
>  To cover all users simply use `/etc/skel`. Files in there will be available in every home directory under correct permissions.
> diff --git a/meta/classes-recipe/image-account-extension.bbclass b/meta/classes-recipe/image-account-extension.bbclass
> index e874f3c7..7dfcd8e0 100644
> --- a/meta/classes-recipe/image-account-extension.bbclass
> +++ b/meta/classes-recipe/image-account-extension.bbclass
> @@ -14,16 +14,18 @@ python() {
>      for entry in (d.getVar("GROUPS") or "").split():
>          group_entry = "GROUP_{}".format(entry)
>          d.appendVarFlag("image_postprocess_accounts", "vardeps", " {}".format(group_entry))
> +        d.appendVarFlag("image_configure_adduser_pools", "vardeps", " {}".format(group_entry))
>          d.appendVarFlag("do_rootfs_install", "vardeps", " {}".format(group_entry))
>  
>      for entry in (d.getVar("USERS") or "").split():
>          user_entry = "USER_{}".format(entry)
>          d.appendVarFlag("image_postprocess_accounts", "vardeps", " {}".format(user_entry))
> +        d.appendVarFlag("image_configure_adduser_pools", "vardeps", " {}".format(user_entry))
>          d.appendVarFlag("do_rootfs_install", "vardeps", " {}".format(user_entry))
>  }
>  do_rootfs_install[vardeps] += "GROUPS USERS"
>  
> -def image_create_groups(d: "DataSmart") -> None:
> +def image_create_groups(d):
>      """Creates the groups defined in the ``GROUPS`` bitbake variable.
>  
>      Args:
> @@ -40,6 +42,10 @@ def image_create_groups(d: "DataSmart") -> None:
>          args = []
>          group_entry = "GROUP_{}".format(entry)
>  
> +        flags = (d.getVarFlag(group_entry, "flags") or "").split()
> +        if "reserve-only" in flags:
> +            continue
> +
>          with open("{}/etc/group".format(rootfsdir), "r") as group_file:
>              exists = any(line.startswith("{}:".format(entry)) for line in group_file)
>  
> @@ -59,7 +65,7 @@ def image_create_groups(d: "DataSmart") -> None:
>              bb.process.run([*chroot, "/usr/sbin/groupadd", *args, entry])
>  
>  
> -def image_create_users(d: "DataSmart") -> None:
> +def image_create_users(d):
>      """Creates the users defined in the ``USERS`` bitbake variable.
>  
>      Args:
> @@ -78,6 +84,10 @@ def image_create_users(d: "DataSmart") -> None:
>          args = []
>          user_entry = "USER_{}".format(entry)
>  
> +        flags = (d.getVarFlag(user_entry, "flags") or "").split()
> +        if "reserve-only" in flags:
> +            continue
> +
>          with open("{}/etc/passwd".format(rootfsdir), "r") as passwd_file:
>              exists = any(line.startswith("{}:".format(entry)) for line in passwd_file)
>  
> @@ -99,8 +109,6 @@ def image_create_users(d: "DataSmart") -> None:
>              args.append("--groups")
>              args.append(','.join(groups))
>  
> -        flags = (d.getVarFlag(user_entry, "flags") or "").split()
> -
>          if exists:
>              add_user_option("--home", "home")
>              if d.getVarFlag(user_entry, "home") or "":
> @@ -143,6 +151,103 @@ def image_create_users(d: "DataSmart") -> None:
>              bb.process.run([*chroot, "/usr/bin/passwd", "--expire", entry])
>  
>  
> +def configure_adduser_pools(d):
> +    """Configures adduser UID/GID pools for users and groups with explicit IDs.
> +
> +    Creates pool files and a minimal /etc/adduser.conf with UID_POOL/GID_POOL
> +    directives before package installation.
> +
> +    Args:
> +        d (DataSmart): The bitbake datastore.
> +
> +    Returns:
> +        None
> +    """
> +    import os
> +    import tempfile
> +
> +    rootfsdir = d.getVar("ROOTFSDIR")
> +    adduser_conf = "{}/etc/adduser.conf".format(rootfsdir)
> +    uid_pool_path = "/etc/adduser-uid.pool"
> +    gid_pool_path = "/etc/adduser-gid.pool"
> +
> +    uid_pool_entries = []
> +    seen_users = set()
> +    for entry in (d.getVar("USERS") or "").split():
> +        if entry in seen_users:
> +            continue
> +        seen_users.add(entry)
> +        user_entry = "USER_{}".format(entry)
> +        uid = d.getVarFlag(user_entry, "uid") or ""
> +        if uid:
> +            uid_pool_entries.append("{}:{}".format(entry, uid))
> +
> +    gid_pool_entries = []
> +    seen_groups = set()
> +    for entry in (d.getVar("GROUPS") or "").split():
> +        if entry in seen_groups:
> +            continue
> +        seen_groups.add(entry)
> +        group_entry = "GROUP_{}".format(entry)
> +        gid = d.getVarFlag(group_entry, "gid") or ""
> +        if gid:
> +            gid_pool_entries.append("{}:{}".format(entry, gid))
> +
> +    if not uid_pool_entries and not gid_pool_entries:
> +        return
> +
> +    if uid_pool_entries:
> +        with tempfile.NamedTemporaryFile(mode="w", delete=False) as f:
> +            f.write("\n".join(uid_pool_entries) + "\n")
> +            tmp = f.name
> +        bb.process.run(
> +            ["sudo", "cp", tmp, "{}{}".format(rootfsdir, uid_pool_path)])
> +        bb.process.run(
> +            ["sudo", "chmod", "644", "{}{}".format(rootfsdir, uid_pool_path)])
> +        os.unlink(tmp)
> +
> +    if gid_pool_entries:
> +        with tempfile.NamedTemporaryFile(mode="w", delete=False) as f:
> +            f.write("\n".join(gid_pool_entries) + "\n")
> +            tmp = f.name
> +        bb.process.run(
> +            ["sudo", "cp", tmp, "{}{}".format(rootfsdir, gid_pool_path)])
> +        bb.process.run(
> +            ["sudo", "chmod", "644", "{}{}".format(rootfsdir, gid_pool_path)])
> +        os.unlink(tmp)
> +
> +    # Create /etc/adduser.conf with the upstream default content plus pool
> +    # directives. We use --force-confold during package installation so that
> +    # dpkg keeps this version when the adduser package is installed.
> +    conf_lines = []
> +    conf_lines.append("# /etc/adduser.conf: `adduser' configuration.")
> +    conf_lines.append("# See adduser(8) and adduser.conf(5) for full documentation.")
> +    conf_lines.append("")
> +    if uid_pool_entries:
> +        conf_lines.append("UID_POOL={}".format(uid_pool_path))
> +    if gid_pool_entries:
> +        conf_lines.append("GID_POOL={}".format(gid_pool_path))
> +
> +    with tempfile.NamedTemporaryFile(mode="w", delete=False) as f:
> +        f.write("\n".join(conf_lines) + "\n")
> +        tmp = f.name
> +    bb.process.run(["sudo", "cp", tmp, adduser_conf])
> +    bb.process.run(["sudo", "chmod", "644", adduser_conf])
> +    os.unlink(tmp)
> +
> +
> +# Work-around: pre-create /etc/adduser.conf with pool directives and use
> +# --force-confold so dpkg keeps our version when the adduser package is
> +# installed. This is needed because adduser does not support loading
> +# configuration from /etc/adduser.conf.d/ or from environment variables.
> +ROOTFS_APT_ARGS += "-o DPkg::Options::=--force-confold"
> +
> +ROOTFS_CONFIGURE_COMMAND += "image_configure_adduser_pools"
> +image_configure_adduser_pools[vardeps] += "USERS GROUPS"
> +python image_configure_adduser_pools() {
> +    configure_adduser_pools(d)
> +}
> +
>  ROOTFS_POSTPROCESS_COMMAND += "image_postprocess_accounts"
>  image_postprocess_accounts[vardeps] += "USERS GROUPS"
>  python image_postprocess_accounts() {