Message ID | 20221120075336.16006-1-venkata.pyla@toshiba-tsip.com |
---|---|
State | Superseded, archived |
Headers | show |
Series | scripts/repro-test.sh: script to test reproducibility of Isar image | expand |
20.11.2022 10:53, venkata.pyla@toshiba-tsip.com wrote: > From: venkata pyla <venkata.pyla@toshiba-tsip.com> > > This script helps to identify reproducible problems in isar image, it > uses 'diffoscope'[1] tool to check the difference between two > consecutive builds and copy the differeneces in html format. > > [1] https://diffoscope.org/ > > Signed-off-by: venkata pyla <venkata.pyla@toshiba-tsip.com> Hello, any chance this can be integrated into our testsuite?
>-----Original Message----- >From: isar-users@googlegroups.com <isar-users@googlegroups.com> On Behalf >Of Anton Mikanovich >Sent: 22 November 2022 12:58 >To: pyla venkata(TSIP TMIEC ODG Porting) <Venkata.Pyla@toshiba- >tsip.com>; isar-users@googlegroups.com >Cc: jan.kiszka@siemens.com; henning.schild@siemens.com; hayashi kazuhiro(林 >和宏 □SWC◯ACT) <kazuhiro3.hayashi@toshiba.co.jp>; dinesh kumar(TS >IP TMIEC ODG Porting) <dinesh.kumar@toshiba-tsip.com> >Subject: Re: [PATCH] scripts/repro-test.sh: script to test reproducibility of Isar >image > >20.11.2022 10:53, venkata.pyla@toshiba-tsip.com wrote: >> From: venkata pyla <venkata.pyla@toshiba-tsip.com> >> >> This script helps to identify reproducible problems in isar image, it >> uses 'diffoscope'[1] tool to check the difference between two >> consecutive builds and copy the differeneces in html format. >> >> [1] https://diffoscope.org/ >> >> Signed-off-by: venkata pyla <venkata.pyla@toshiba-tsip.com> > >Hello, any chance this can be integrated into our testsuite? Hi, Thanks for considering, I have not used avocado framework much, so it will take some more time for me to implement this test in the testsuite format. Can we use this bash script test case currently and later upgrade to testsuite format? > >-- >You received this message because you are subscribed to the Google Groups >"isar-users" group. >To unsubscribe from this group and stop receiving emails from it, send an email >to isar-users+unsubscribe@googlegroups.com. >To view this discussion on the web visit >https://groups.google.com/d/msgid/isar-users/18251c8e-6732-5f0e-dcf4- >1e34fbe804ef%40ilbers.de.
19.12.2022 17:41, venkata.pyla@toshiba-tsip.com wrote: > From: venkata pyla <venkata.pyla@toshiba-tsip.com> > > Hi, > > The below patch set will add the test for verifying the rootfs images > are reproducible, it will help to identify the reproducible problems > and also can verify the fixes that solves the problem. > > Please check if this can add to the isar testsuite and eventually helps > to verify the reproducible problems in isar. > > Looking forward for your feedback. > > Thanks, > Venkata. > > venkata pyla (2): > cibuilder.py: Add source_date_epoch to build configuration > repro-build-test.py: Test to check images are reproducible > > testsuite/cibuilder.py | 5 ++- > testsuite/repro-build-test.py | 68 +++++++++++++++++++++++++++++++++++ > 2 files changed, 72 insertions(+), 1 deletion(-) > create mode 100755 testsuite/repro-build-test.py > Applied to next, thanks.
diff --git a/scripts/repro-test.sh b/scripts/repro-test.sh new file mode 100755 index 0000000..0509f32 --- /dev/null +++ b/scripts/repro-test.sh @@ -0,0 +1,105 @@ +#!/bin/bash + +set -e + +DISTRO="debian-bullseye" +MACHINE="qemuamd64" +DISTRO_ARCH="amd64" +IMAGE_TYPE="isar-image-base" + +clean(){ + if [ $? -eq 0 ]; then + [ -d $tmpdir ] && sudo rm -rf $tmpdir + else + echo "Keep build artifacts at $tmpdir" + fi +} + +build(){ + build_id="$1" + echo "Build name: $build_id" + mkdir "$tmpdir/build" + + cd "$isar_root_dir" + . isar-init-build-env "$tmpdir/build" && bitbake mc:${MACHINE}-${DISTRO#*-}:${IMAGE_TYPE} + + mv "$tmpdir/build" "$tmpdir/$build_id" +} + +repro_check(){ + image1="$tmpdir/$1/tmp/deploy/images/${MACHINE}/${IMAGE_TYPE}-${DISTRO}-${MACHINE}.tar.gz" + image2="$tmpdir/$2/tmp/deploy/images/${MACHINE}/${IMAGE_TYPE}-${DISTRO}-${MACHINE}.tar.gz" + + # Compare two build artifacts + if diffoscope --html-dir "$tmpdir/diffoscope_html_output" "$image1" "$image2"; then + echo "[Success] The build artifacts are reproduicible" + else + echo "[Fail] The build artifacts are not matching" + fi +} + +usage() { + echo "This script verifies the reproducibility of the Isar images." + echo + echo "Usage:" + echo " $0 [params]" + echo + echo "Parameters:" + echo " -d, --distro DISTRO set DISTRO variable for the builds (e.g.: debian-buster, debian-bullseye)" + echo " -m, --machine MACHINE set MACHINE variable for the builds (e.g.: qemuamd64, qemuarm64, qemuarmhf)" + echo " -a, --arch DISTRO_ARCH set DISTRO_ARCH variable for the builds (e.g.: amd64, arm64, armhf)" + echo " -h, --help" + echo +} + +while [ "$1" != "" ]; do + case $1 in + -d | --distro ) + shift + DISTRO="$1" + ;; + -m | --mahcine ) + shift + MACHINE="$1" + ;; + -a | --arch ) + shift + DISTRO_ARCH="$1" + ;; + -h | --help ) + usage + exit + ;; + * ) + usage + exit 1 + esac + shift +done + +if ! command -v diffoscope &> /dev/null +then + echo "diffoscope could not be found" + exit +fi + +cd "$(dirname "$0")/.." +isar_root_dir="$(pwd)" + +echo "Distro: $DISTRO" +echo "MACHINE: $MACHINE" +echo "DISTRO_ARCH: $DISTRO_ARCH" + +# export the variables for the build +export DISTRO=${DISTRO} +export MACHINE=${MACHINE} +export DISTRO_ARCH=${DISTRO_ARCH} + +# Temorary directory for keeping build artifacts +tmpdir=$(mktemp --directory --tmpdir /home/seshu/temp/isar.repro.test.XXXXXXXXXX) +trap clean INT TERM EXIT + +build build1 +build build2 + +repro_check build1 build2