[RFC,v3,1/1] test: provide ci_shell as a convenience wrapper

From: Cedric Hombourger' via isar-users <isar-users@googlegroups.com>

Ease use of the container environment that may be used to run
tests from this project by adding a `scripts/ci_shell` script
to invoke `kas-container` for you. The script uses the existing
kas/isar.yml kas file to use your existing checkout of the Isar
code: this makes it easier to test your work-in-progress changes.

Lastly, `ci_setup.sh` will produce less output (e.g. when
installing dependencies) and will print a few hints to remind
users with useful commands.

[Felix]
- fix pyhashtables cleanup
- speedup installation by skipping recommends
- add more helper commands
- always run with sstate cache
- run everything in tmux to access logs while test is running
- provide pycache path to speedup python startup
- add builtin (memory only) squid cache
- use host network instead of container network (allows use of federated proxies)

Signed-off-by: Cedric Hombourger <cedric.hombourger@siemens.com>
Signed-off-by: Felix Moessbauer <felix.moessbauer@siemens.com>
---
@Cedric: This is RFC is based on your patchset and tries to further
simplify and speedup the testsuite invocation. Feel free to integrate
whatever you think makes sense. With your patch and my additions,
I finally found a way to efficiently execute single tests. However,
there is still much room for improvement in the testsuite itself.

Best regards,
Felix

 CONTRIBUTING.md     | 52 ++++++++++----------------------------------
 scripts/ci_setup.sh | 53 ++++++++++++++++++++++++++++++++++++++++-----
 scripts/ci_shell    | 36 ++++++++++++++++++++++++++++++
 3 files changed, 95 insertions(+), 46 deletions(-)
 create mode 100755 scripts/ci_shell

On 12/12/25 16:01, 'Felix Moessbauer' via isar-users wrote:
> From: Cedric Hombourger' via isar-users <isar-users@googlegroups.com>
>
> Ease use of the container environment that may be used to run
> tests from this project by adding a `scripts/ci_shell` script
> to invoke `kas-container` for you. The script uses the existing
> kas/isar.yml kas file to use your existing checkout of the Isar
> code: this makes it easier to test your work-in-progress changes.
>
> Lastly, `ci_setup.sh` will produce less output (e.g. when
> installing dependencies) and will print a few hints to remind
> users with useful commands.
>
> [Felix]
> - fix pyhashtables cleanup
> - speedup installation by skipping recommends
> - add more helper commands
> - always run with sstate cache
> - run everything in tmux to access logs while test is running
> - provide pycache path to speedup python startup
> - add builtin (memory only) squid cache
> - use host network instead of container network (allows use of federated proxies)
>
> Signed-off-by: Cedric Hombourger <cedric.hombourger@siemens.com>
> Signed-off-by: Felix Moessbauer <felix.moessbauer@siemens.com>
> ---
> @Cedric: This is RFC is based on your patchset and tries to further
> simplify and speedup the testsuite invocation. Feel free to integrate
> whatever you think makes sense. With your patch and my additions,
> I finally found a way to efficiently execute single tests. However,
> there is still much room for improvement in the testsuite itself.
>
> Best regards,
> Felix
>
>   CONTRIBUTING.md     | 52 ++++++++++----------------------------------
>   scripts/ci_setup.sh | 53 ++++++++++++++++++++++++++++++++++++++++-----
>   scripts/ci_shell    | 36 ++++++++++++++++++++++++++++++
>   3 files changed, 95 insertions(+), 46 deletions(-)
>   create mode 100755 scripts/ci_shell
>
> diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md
> index 07621232..6970477b 100644
> --- a/CONTRIBUTING.md
> +++ b/CONTRIBUTING.md
> @@ -81,54 +81,26 @@ Plan merges to `master` so that both fit the two-week window; short extensions s
>        by launching CI tests scripts. The procedure is described below:
>   
>       ```
> -    git clone https://github.com/siemens/kas
> -    cat > kas.yml <<EOF
> -    header:
> -      version: 14
> -    build_system: isar
> -    defaults:
> -      repos:
> -        patches:
> -          repo: isar
> -    repos:
> -      isar:
> -        url: "http://github.com:/ilbers/isar"
> -        branch: next
> -        layers:
> -          meta:
> -          meta-isar:
> -    EOF
> -    kas/kas-container shell --command /work/isar/scripts/ci_setup.sh kas.yml
> +    ./scripts/ci_shell
>       ```
>   
> -    In kas shell:
> +    This will spawn a shell where you may just run:
>   
>       ```
> -    cd /work/isar/testsuite
> -    avocado run citest.py -t dev --max-parallel-tasks=1
> +    trun citest.py -t dev --max-parallel-tasks=1
>       ```
>   
> -    Your git-formatpatches may be listed in the `kas.yml` file as illustrated below:
> +    Note: (`trun` is an alias for `avocado run`).
>   
> -    ```
> -    ...
> -    repos:
> -      isar:
> -        url: "http://github.com:/ilbers/isar"
> -        branch: next
> -	patches:
> -          0001:
> -            path: /work/0001-my-contribution-to-isar.patch
> -        layers:
> -          meta:
> -          meta-isar:
> -    ```
> -
> -    Perform the above steps from a clean directory for your CI run to be as close as
> -    possible to the environment that our project maintainers will be using. That
> -    directory would contain: *.patch isar/ kas/ kas.yml
> +    The container has `/work` bind-mounted to your local copy of the Isar code
> +    base (and therefore with your changes). Make sure to track upstream changes
> +    and regularly rebase your tree: your patches shall be sent against the latest
> +    HEAD available when submitting. Keep an eye on changes while your series are
> +    under review: you may need to send a refreshed version if contributions from
> +    others were accepted and touch the same areas (make sure to re-run the test
> +    suite after rebasing).
>   
> -    Be also mindful of community-provided resources such as deb.debian.org or
> +    Be mindful of community-provided resources such as deb.debian.org or
>       snapshot.debian.org and consider using a caching proxy in your setup to
>       reduce traffic as much as possible.
>   
> diff --git a/scripts/ci_setup.sh b/scripts/ci_setup.sh
> index f373d1c0..bbca4ab8 100755
> --- a/scripts/ci_setup.sh
> +++ b/scripts/ci_setup.sh
> @@ -5,28 +5,69 @@
>   # Copyright (c) Siemens AG, 2025
>   # SPDX-License-Identifier: MIT
>   
> +# start a squid http proxy in the container
> +with_squid="1"
> +
>   gpg_key=/etc/apt/trusted.gpg.d/debian-isar.gpg
>   [ -f "${gpg_key}" ] || {
>       wget -q http://deb.isar-build.org/debian-isar.key -O- \
>       | gpg --dearmor \
> -    | sudo dd of="${gpg_key}"
> +    | sudo dd of="${gpg_key}" status=none
>   }
>   
>   list=/etc/apt/sources.list.d/10-isar_build.list
>   [ -f "${list}" ] || {
>       echo "deb [signed-by=/etc/apt/trusted.gpg.d/debian-isar.gpg] \
>           http://deb.isar-build.org/debian-isar bookworm-isar main" \
> -    | sudo tee /etc/apt/sources.list.d/10-isar_build.list
> +    | sudo tee /etc/apt/sources.list.d/10-isar_build.list >/dev/null
>   }
>   
> -tools="avocado qemu-system-aarch64 qemu-system-arm qemu-system-i386 qemu-system-x86_64"
> +echo "🔍 Checking dependencies..."
> +tools="avocado qemu-system-aarch64 qemu-system-arm qemu-system-i386 qemu-system-x86_64 htop"
> +if [ "${with_squid}" = "1" ]; then
> +    tools="${tools} squid"
> +    squid_pkg="squid"
> +    start_squid="sudo /usr/sbin/squid;"
> +    echo "export http_proxy=http://localhost:3128" \
> +    | sudo tee -a /etc/profile.d/squid.sh >/dev/null
> +fi
> +
>   need_install=0
>   for tool in ${tools}; do
>       which "${tool}" || need_install=1
>   done
>   [ "${need_install}" = "0" ] || {
> -    sudo apt-get update
> -    sudo apt-get install -y avocado qemu-system-arm qemu-system-x86
> +    echo "⏳ Installing missing dependencies..."
> +    (
> +        sudo apt-get update
> +        sudo apt-get install -y --no-install-recommends \
> +            avocado qemu-system-arm qemu-system-x86 ovmf htop $squid_pkg
> +    )
>   }
>   
> -exec /container-entrypoint ${*}
> +(
> +    cat <<'    EOF'
> +    alias tlist="avocado list"
> +    alias trun="avocado run -p sstate=1 --max-parallel-tasks=1 --failfast"
> +    alias trun-fg="avocado --show avocado.test run -p sstate=1 --max-parallel-tasks=1 --failfast"
> +    alias tquit="tmux kill-session -t isar_ci"
> +    alias thelp="cat ~/.trun-help"
> +    export PYTHONPYCACHEPREFIX=/work/build/pycache
> +    mkdir -p ${PYTHONPYCACHEPREFIX}
> +    cd /work/testsuite
> +    EOF
> +) | sudo tee -a /etc/profile.d/ci.sh >/dev/null
> +
> +cat <<'EOF' > ~/.trun-help
> +
> +💡 Use "tlist citest.py" to list existing test cases
> +💡 Use "trun citest.py:TestClass.test_method" to run a specific test
> +💡 Use "trun-fg <test> to run a specific test and emit the logs
> +💡 Use "tquit" to leave the testing environment
> +💡 Use "thelp" to show this help message
> +
> +✅ Test environment ready for use!
> +
> +EOF
> +
> +exec /container-entrypoint tmux new-session -A -s isar_ci "$start_squid cat ~/.trun-help && bash -l"
> diff --git a/scripts/ci_shell b/scripts/ci_shell
> new file mode 100755
> index 00000000..917d1272
> --- /dev/null
> +++ b/scripts/ci_shell
> @@ -0,0 +1,36 @@
> +#!/bin/bash
> +# A simple wrapper to run a container to run CI tests
> +#
> +# Cedric Hombourger <cedric.hombourger@siemens.com>
> +# Copyright (c) Siemens AG, 2025
> +# SPDX-License-Identifier: MIT
> +#
> +# Usage: ci_shell [kas-container-version]
> +
> +self=${0}
> +topdir=$(readlink -f $(dirname ${self})/..)
> +
> +export KAS_BUILD_DIR=$(mktemp -d)
> +
> +if [[ -n "$1" ]]; then
> +    export KAS_IMAGE_VERSION="$1"
> +fi
> +
> +drop_pyhashtables() {
> +    # Remove possibly outdated pyhashtables.py file that is generated on-the-fly.
> +    # The file is owned by the (bitbake) calling user, hence it's lifetime needs
> +    # to be bound to the container's lifetime.
> +    rm -f bitbake/lib/bb/pysh/pyshtables.py
> +}
> +cleanup() {
> +    rm -rf --one-file-system ${KAS_BUILD_DIR}
> +    drop_pyhashtables
> +}
> +trap "cleanup" EXIT
> +
> +cd ${topdir}
> +drop_pyhashtables
> +kas/kas-container \
> +    --runtime-args "--network=host" \
> +    shell --command "rm -rf /work/build/conf && /work/scripts/ci_setup.sh" \
> +    kas/isar.yaml
I would suggest to add some instructions on using avocado, like the ones 
mentioned in [1], but that can be a separate patch.
[1] 
https://lists.isar-build.org/isar-users/c3db73d0-5753-41cd-a300-21bec09a0c96@ilbers.de

Zhihang

On 12.12.25 16:01, Felix Moessbauer wrote:
> From: Cedric Hombourger' via isar-users <isar-users@googlegroups.com>
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^
(found by https://github.com/jan-kiszka/copypatch because signed-off
differs)

Jan

On 12.12.25 16:01, Felix Moessbauer wrote:
> From: Cedric Hombourger' via isar-users <isar-users@googlegroups.com>
> 
> Ease use of the container environment that may be used to run
> tests from this project by adding a `scripts/ci_shell` script
> to invoke `kas-container` for you. The script uses the existing
> kas/isar.yml kas file to use your existing checkout of the Isar
> code: this makes it easier to test your work-in-progress changes.
> 
> Lastly, `ci_setup.sh` will produce less output (e.g. when
> installing dependencies) and will print a few hints to remind
> users with useful commands.
> 
> [Felix]
> - fix pyhashtables cleanup
> - speedup installation by skipping recommends
> - add more helper commands
> - always run with sstate cache
> - run everything in tmux to access logs while test is running

This is a nice feature - if you like it. I hate it and would strongly
recommend to make this opt-in.

Jan

> - provide pycache path to speedup python startup
> - add builtin (memory only) squid cache
> - use host network instead of container network (allows use of federated proxies)
> 
> Signed-off-by: Cedric Hombourger <cedric.hombourger@siemens.com>
> Signed-off-by: Felix Moessbauer <felix.moessbauer@siemens.com>
> ---
> @Cedric: This is RFC is based on your patchset and tries to further
> simplify and speedup the testsuite invocation. Feel free to integrate
> whatever you think makes sense. With your patch and my additions,
> I finally found a way to efficiently execute single tests. However,
> there is still much room for improvement in the testsuite itself.
> 
> Best regards,
> Felix
> 
>  CONTRIBUTING.md     | 52 ++++++++++----------------------------------
>  scripts/ci_setup.sh | 53 ++++++++++++++++++++++++++++++++++++++++-----
>  scripts/ci_shell    | 36 ++++++++++++++++++++++++++++++
>  3 files changed, 95 insertions(+), 46 deletions(-)
>  create mode 100755 scripts/ci_shell
> 
> diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md
> index 07621232..6970477b 100644
> --- a/CONTRIBUTING.md
> +++ b/CONTRIBUTING.md
> @@ -81,54 +81,26 @@ Plan merges to `master` so that both fit the two-week window; short extensions s
>       by launching CI tests scripts. The procedure is described below:
>  
>      ```
> -    git clone https://github.com/siemens/kas
> -    cat > kas.yml <<EOF
> -    header:
> -      version: 14
> -    build_system: isar
> -    defaults:
> -      repos:
> -        patches:
> -          repo: isar
> -    repos:
> -      isar:
> -        url: "http://github.com:/ilbers/isar"
> -        branch: next
> -        layers:
> -          meta:
> -          meta-isar:
> -    EOF
> -    kas/kas-container shell --command /work/isar/scripts/ci_setup.sh kas.yml
> +    ./scripts/ci_shell
>      ```
>  
> -    In kas shell:
> +    This will spawn a shell where you may just run:
>  
>      ```
> -    cd /work/isar/testsuite
> -    avocado run citest.py -t dev --max-parallel-tasks=1
> +    trun citest.py -t dev --max-parallel-tasks=1
>      ```
>  
> -    Your git-formatpatches may be listed in the `kas.yml` file as illustrated below:
> +    Note: (`trun` is an alias for `avocado run`).
>  
> -    ```
> -    ...
> -    repos:
> -      isar:
> -        url: "http://github.com:/ilbers/isar"
> -        branch: next
> -	patches:
> -          0001:
> -            path: /work/0001-my-contribution-to-isar.patch
> -        layers:
> -          meta:
> -          meta-isar:
> -    ```
> -
> -    Perform the above steps from a clean directory for your CI run to be as close as
> -    possible to the environment that our project maintainers will be using. That
> -    directory would contain: *.patch isar/ kas/ kas.yml
> +    The container has `/work` bind-mounted to your local copy of the Isar code
> +    base (and therefore with your changes). Make sure to track upstream changes
> +    and regularly rebase your tree: your patches shall be sent against the latest
> +    HEAD available when submitting. Keep an eye on changes while your series are
> +    under review: you may need to send a refreshed version if contributions from
> +    others were accepted and touch the same areas (make sure to re-run the test
> +    suite after rebasing).
>  
> -    Be also mindful of community-provided resources such as deb.debian.org or
> +    Be mindful of community-provided resources such as deb.debian.org or
>      snapshot.debian.org and consider using a caching proxy in your setup to
>      reduce traffic as much as possible.
>  
> diff --git a/scripts/ci_setup.sh b/scripts/ci_setup.sh
> index f373d1c0..bbca4ab8 100755
> --- a/scripts/ci_setup.sh
> +++ b/scripts/ci_setup.sh
> @@ -5,28 +5,69 @@
>  # Copyright (c) Siemens AG, 2025
>  # SPDX-License-Identifier: MIT
>  
> +# start a squid http proxy in the container
> +with_squid="1"
> +
>  gpg_key=/etc/apt/trusted.gpg.d/debian-isar.gpg
>  [ -f "${gpg_key}" ] || {
>      wget -q http://deb.isar-build.org/debian-isar.key -O- \
>      | gpg --dearmor \
> -    | sudo dd of="${gpg_key}"
> +    | sudo dd of="${gpg_key}" status=none
>  }
>  
>  list=/etc/apt/sources.list.d/10-isar_build.list
>  [ -f "${list}" ] || {
>      echo "deb [signed-by=/etc/apt/trusted.gpg.d/debian-isar.gpg] \
>          http://deb.isar-build.org/debian-isar bookworm-isar main" \
> -    | sudo tee /etc/apt/sources.list.d/10-isar_build.list
> +    | sudo tee /etc/apt/sources.list.d/10-isar_build.list >/dev/null
>  }
>  
> -tools="avocado qemu-system-aarch64 qemu-system-arm qemu-system-i386 qemu-system-x86_64"
> +echo "🔍 Checking dependencies..."
> +tools="avocado qemu-system-aarch64 qemu-system-arm qemu-system-i386 qemu-system-x86_64 htop"
> +if [ "${with_squid}" = "1" ]; then
> +    tools="${tools} squid"
> +    squid_pkg="squid"
> +    start_squid="sudo /usr/sbin/squid;"
> +    echo "export http_proxy=http://localhost:3128" \
> +    | sudo tee -a /etc/profile.d/squid.sh >/dev/null
> +fi
> +
>  need_install=0
>  for tool in ${tools}; do
>      which "${tool}" || need_install=1
>  done
>  [ "${need_install}" = "0" ] || {
> -    sudo apt-get update
> -    sudo apt-get install -y avocado qemu-system-arm qemu-system-x86
> +    echo "⏳ Installing missing dependencies..."
> +    (
> +        sudo apt-get update
> +        sudo apt-get install -y --no-install-recommends \
> +            avocado qemu-system-arm qemu-system-x86 ovmf htop $squid_pkg
> +    )
>  }
>  
> -exec /container-entrypoint ${*}
> +(
> +    cat <<'    EOF'
> +    alias tlist="avocado list"
> +    alias trun="avocado run -p sstate=1 --max-parallel-tasks=1 --failfast"
> +    alias trun-fg="avocado --show avocado.test run -p sstate=1 --max-parallel-tasks=1 --failfast"
> +    alias tquit="tmux kill-session -t isar_ci"
> +    alias thelp="cat ~/.trun-help"
> +    export PYTHONPYCACHEPREFIX=/work/build/pycache
> +    mkdir -p ${PYTHONPYCACHEPREFIX}
> +    cd /work/testsuite
> +    EOF
> +) | sudo tee -a /etc/profile.d/ci.sh >/dev/null
> +
> +cat <<'EOF' > ~/.trun-help
> +
> +💡 Use "tlist citest.py" to list existing test cases
> +💡 Use "trun citest.py:TestClass.test_method" to run a specific test
> +💡 Use "trun-fg <test> to run a specific test and emit the logs
> +💡 Use "tquit" to leave the testing environment
> +💡 Use "thelp" to show this help message
> +
> +✅ Test environment ready for use!
> +
> +EOF
> +
> +exec /container-entrypoint tmux new-session -A -s isar_ci "$start_squid cat ~/.trun-help && bash -l"
> diff --git a/scripts/ci_shell b/scripts/ci_shell
> new file mode 100755
> index 00000000..917d1272
> --- /dev/null
> +++ b/scripts/ci_shell
> @@ -0,0 +1,36 @@
> +#!/bin/bash
> +# A simple wrapper to run a container to run CI tests
> +#
> +# Cedric Hombourger <cedric.hombourger@siemens.com>
> +# Copyright (c) Siemens AG, 2025
> +# SPDX-License-Identifier: MIT
> +#
> +# Usage: ci_shell [kas-container-version]
> +
> +self=${0}
> +topdir=$(readlink -f $(dirname ${self})/..)
> +
> +export KAS_BUILD_DIR=$(mktemp -d)
> +
> +if [[ -n "$1" ]]; then
> +    export KAS_IMAGE_VERSION="$1"
> +fi
> +
> +drop_pyhashtables() {
> +    # Remove possibly outdated pyhashtables.py file that is generated on-the-fly.
> +    # The file is owned by the (bitbake) calling user, hence it's lifetime needs
> +    # to be bound to the container's lifetime.
> +    rm -f bitbake/lib/bb/pysh/pyshtables.py
> +}
> +cleanup() {
> +    rm -rf --one-file-system ${KAS_BUILD_DIR}
> +    drop_pyhashtables
> +}
> +trap "cleanup" EXIT
> +
> +cd ${topdir}
> +drop_pyhashtables
> +kas/kas-container \
> +    --runtime-args "--network=host" \
> +    shell --command "rm -rf /work/build/conf && /work/scripts/ci_setup.sh" \
> +    kas/isar.yaml

On Fri, 2025-12-12 at 17:23 +0100, Jan Kiszka wrote:
> On 12.12.25 16:01, Felix Moessbauer wrote:
> > From: Cedric Hombourger' via isar-users <isar-users@googlegroups.com>
>                            ^^^^^^^^^^^^^^^^^^^^^^^^^^
> (found by https://github.com/jan-kiszka/copypatch because signed-off
> differs)

Good catch. I'm wondering why the author association got lost. But this
patch is anyways not for merging but merely a way to express feedback
regarding what the ci wrapper should offer.

Felix

On 12.12.25 16:01, Felix Moessbauer wrote:
> +cat <<'EOF' > ~/.trun-help
> +
> +💡 Use "tlist citest.py" to list existing test cases
> +💡 Use "trun citest.py:TestClass.test_method" to run a specific test

In our context, it is almost always "citest.py", no? Then please enhance
the aliases, making it a default so that we do not have to type this all
the time, e.g. "tlist [citest.py]" and "trun
[citest.py:]TestClass.test_method".

> +💡 Use "trun-fg <test> to run a specific test and emit the logs
> +💡 Use "tquit" to leave the testing environment
> +💡 Use "thelp" to show this help message
> +

Jan

On Fri, 2025-12-12 at 17:34 +0100, Jan Kiszka wrote:
> On 12.12.25 16:01, Felix Moessbauer wrote:
> > +cat <<'EOF' > ~/.trun-help
> > +
> > +💡 Use "tlist citest.py" to list existing test cases
> > +💡 Use "trun citest.py:TestClass.test_method" to run a specific test
> 
> In our context, it is almost always "citest.py", no? Then please enhance

We also have repro-build-test.py and hopefully more testcase specific
files in the future. However, I'm wondering if there is any value in
letting the user select the test file. Otherwise we could also add a
python module that includes all downstream tests - which is then used
by the trun alias.

Felix