| Message ID | 20251218073853.41121-3-felix.moessbauer@siemens.com |
|---|---|
| State | Under Review |
| Headers | show |
| Series | Enable SState cache in GitLab CI | expand |
On Thu, 2025-12-18 at 08:38 +0100, Felix Moessbauer wrote: > The gitlab CI jobs should finish in a timely manner to give quick > feedback to the developer. By running the tests with sstate cache, > simple changes require far less CI time. > > We prepare the gitlab ci job description to pick up the sstate cache > configuration from the environment and enable the caching for all jobs > (we always enable it, even if running without sstate cache, as then the > cache is simply thrown away later on). We further introduce a (manual) > info task to show the state of the cache, as well as a cleanup task to > drain the cache. > > Signed-off-by: Felix Moessbauer <felix.moessbauer@siemens.com> > --- > .gitlab-ci.yml | 34 ++++++++++++++++++++++++++++------ > 1 file changed, 28 insertions(+), 6 deletions(-) > > diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml > index 1a7abcb8..9bd8ff07 100644 > --- a/.gitlab-ci.yml > +++ b/.gitlab-ci.yml > @@ -12,6 +12,14 @@ variables: > - export ftp_proxy=$FTP_PROXY > - export no_proxy=$NO_PROXY > - export DISTRO_APT_PREMIRRORS=$DISTRO_APT_PREMIRRORS > + after_script: > + - | > + if [[ -n "${SSTATE_LOCATION}" ]] && [[ -d "sstate-cache" ]]; then > + echo "=== Upload sstate artifacts to ${SSTATE_LOCATION} ===" > + ./scripts/isar-sstate upload "sstate-cache" "${SSTATE_LOCATION}" > + ./scripts/isar-sstate info -v "${SSTATE_LOCATION}" > + fi We need to increase the timeout of the after_script, as otherwise not all artifacts might be uploaded on larger runs like "-t fast". We need to add [1] variables: RUNNER_AFTER_SCRIPT_TIMEOUT: 15m I'll add it in case a v2 is needed. Otherwise it can also be added later one, when more evidence around resonable timeouts are gathered. [1] https://docs.gitlab.com/ci/runners/configure_runners/#set-script-and-after_script-timeouts Felix
diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index 1a7abcb8..9bd8ff07 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -12,6 +12,14 @@ variables: - export ftp_proxy=$FTP_PROXY - export no_proxy=$NO_PROXY - export DISTRO_APT_PREMIRRORS=$DISTRO_APT_PREMIRRORS + after_script: + - | + if [[ -n "${SSTATE_LOCATION}" ]] && [[ -d "sstate-cache" ]]; then + echo "=== Upload sstate artifacts to ${SSTATE_LOCATION} ===" + ./scripts/isar-sstate upload "sstate-cache" "${SSTATE_LOCATION}" + ./scripts/isar-sstate info -v "${SSTATE_LOCATION}" + fi + artifacts: name: "logs-$CI_JOB_ID" paths: @@ -48,7 +56,7 @@ dev-ci: - *use-default-image - if: $TESTSUITE == 'dev' || $CI_PIPELINE_SOURCE != 'schedule' script: - - scripts/ci_build.sh -T dev + - scripts/ci_build.sh --sstate 1 -T dev fast-ci: <<: *common-build @@ -56,7 +64,7 @@ fast-ci: - *use-default-image - if: $TESTSUITE == 'fast' script: - - scripts/ci_build.sh -T fast + - scripts/ci_build.sh --sstate 1 -T fast full-ci: <<: *common-build @@ -68,7 +76,7 @@ full-ci: - PREVIOUS_SHA="$(cat .CI_COMMIT_SHA || true)" - if [ "$CI_COMMIT_SHA" != "$PREVIOUS_SHA" ]; then echo "$CI_COMMIT_SHA" > .CI_COMMIT_SHA; - scripts/ci_build.sh -T full; + scripts/ci_build.sh --sstate 1 -T full; fi cache: key: "$CI_COMMIT_REF_SLUG" @@ -83,7 +91,7 @@ dev-ci-isar: - *use-docker-isar-image - if: $TESTSUITE == 'dev' script: - - scripts/ci_build.sh -T dev + - scripts/ci_build.sh --sstate 1 -T dev fast-ci-isar: <<: *docker-isar @@ -92,7 +100,7 @@ fast-ci-isar: - *use-docker-isar-image - if: $TESTSUITE == 'fast' script: - - scripts/ci_build.sh -T fast + - scripts/ci_build.sh --sstate 1 -T fast full-ci-isar: <<: *docker-isar @@ -101,4 +109,18 @@ full-ci-isar: - *use-docker-isar-image - if: $TESTSUITE == 'full' script: - - scripts/ci_build.sh -T full + - scripts/ci_build.sh --sstate 1 -T full + +sstate-cache-info: + stage: build + when: manual + script: + - ./scripts/isar-sstate info -v "${SSTATE_LOCATION}" + +sstate-cache-clean: + stage: build + when: manual + variables: + SSTATE_MAX_AGE: "0d" + script: + - ./scripts/isar-sstate clean "${SSTATE_LOCATION}" --max-age "${SSTATE_MAX_AGE}"
The gitlab CI jobs should finish in a timely manner to give quick feedback to the developer. By running the tests with sstate cache, simple changes require far less CI time. We prepare the gitlab ci job description to pick up the sstate cache configuration from the environment and enable the caching for all jobs (we always enable it, even if running without sstate cache, as then the cache is simply thrown away later on). We further introduce a (manual) info task to show the state of the cache, as well as a cleanup task to drain the cache. Signed-off-by: Felix Moessbauer <felix.moessbauer@siemens.com> --- .gitlab-ci.yml | 34 ++++++++++++++++++++++++++++------ 1 file changed, 28 insertions(+), 6 deletions(-)