diff mbox series

scripts/repro-test.sh: script to test reproducibility of Isar image

Message ID 20221120075336.16006-1-venkata.pyla@toshiba-tsip.com
State Superseded, archived
Headers show
Series scripts/repro-test.sh: script to test reproducibility of Isar image | expand

Commit Message

venkata.pyla@toshiba-tsip.com Nov. 20, 2022, 7:53 a.m. UTC 
From: venkata pyla <venkata.pyla@toshiba-tsip.com>

This script helps to identify reproducible problems in isar image, it
uses 'diffoscope'[1] tool to check the difference between two
consecutive builds and copy the differeneces in html format.

[1] https://diffoscope.org/

Signed-off-by: venkata pyla <venkata.pyla@toshiba-tsip.com>
---
 scripts/repro-test.sh | 105 ++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 105 insertions(+)
 create mode 100755 scripts/repro-test.sh

Comments

Anton Mikanovich Nov. 22, 2022, 7:28 a.m. UTC | #1 
20.11.2022 10:53, venkata.pyla@toshiba-tsip.com wrote:
> From: venkata pyla <venkata.pyla@toshiba-tsip.com>
>
> This script helps to identify reproducible problems in isar image, it
> uses 'diffoscope'[1] tool to check the difference between two
> consecutive builds and copy the differeneces in html format.
>
> [1] https://diffoscope.org/
>
> Signed-off-by: venkata pyla <venkata.pyla@toshiba-tsip.com>

Hello, any chance this can be integrated into our testsuite?
venkata.pyla@toshiba-tsip.com Nov. 22, 2022, 8:49 a.m. UTC | #2 
>-----Original Message-----
>From: isar-users@googlegroups.com <isar-users@googlegroups.com> On Behalf
>Of Anton Mikanovich
>Sent: 22 November 2022 12:58
>To: pyla venkata(ＴＳＩＰ TMIEC ODG Porting) <Venkata.Pyla@toshiba-
>tsip.com>; isar-users@googlegroups.com
>Cc: jan.kiszka@siemens.com; henning.schild@siemens.com; hayashi kazuhiro(林
>和宏 □ＳＷＣ◯ＡＣＴ) <kazuhiro3.hayashi@toshiba.co.jp>; dinesh kumar(ＴＳ
>ＩＰ TMIEC ODG Porting) <dinesh.kumar@toshiba-tsip.com>
>Subject: Re: [PATCH] scripts/repro-test.sh: script to test reproducibility of Isar
>image
>
>20.11.2022 10:53, venkata.pyla@toshiba-tsip.com wrote:
>> From: venkata pyla <venkata.pyla@toshiba-tsip.com>
>>
>> This script helps to identify reproducible problems in isar image, it
>> uses 'diffoscope'[1] tool to check the difference between two
>> consecutive builds and copy the differeneces in html format.
>>
>> [1] https://diffoscope.org/
>>
>> Signed-off-by: venkata pyla <venkata.pyla@toshiba-tsip.com>
>
>Hello, any chance this can be integrated into our testsuite?

Hi,
Thanks for considering, I have not used avocado framework much, so it will take some more time for me to implement this test in the testsuite format.

Can we use this bash script test case currently and later upgrade to testsuite format?

>
>--
>You received this message because you are subscribed to the Google Groups
>"isar-users" group.
>To unsubscribe from this group and stop receiving emails from it, send an email
>to isar-users+unsubscribe@googlegroups.com.
>To view this discussion on the web visit
>https://groups.google.com/d/msgid/isar-users/18251c8e-6732-5f0e-dcf4-
>1e34fbe804ef%40ilbers.de.
Anton Mikanovich Dec. 28, 2022, 8:40 a.m. UTC | #3 
19.12.2022 17:41, venkata.pyla@toshiba-tsip.com wrote:
> From: venkata pyla <venkata.pyla@toshiba-tsip.com>
>
> Hi,
>
> The below patch set will add the test for verifying the rootfs images
> are reproducible, it will help to identify the reproducible problems
> and also can verify the fixes that solves the problem.
>
> Please check if this can add to the isar testsuite and eventually helps
> to verify the reproducible problems in isar.
>
> Looking forward for your feedback.
>
> Thanks,
> Venkata.
>
> venkata pyla (2):
>    cibuilder.py: Add source_date_epoch to build configuration
>    repro-build-test.py: Test to check images are reproducible
>
>   testsuite/cibuilder.py        |  5 ++-
>   testsuite/repro-build-test.py | 68 +++++++++++++++++++++++++++++++++++
>   2 files changed, 72 insertions(+), 1 deletion(-)
>   create mode 100755 testsuite/repro-build-test.py
>
Applied to next, thanks.
diff mbox series

Patch

diff --git a/scripts/repro-test.sh b/scripts/repro-test.sh
new file mode 100755
index 0000000..0509f32
--- /dev/null
+++ b/scripts/repro-test.sh
@@ -0,0 +1,105 @@ 
+#!/bin/bash
+
+set -e
+
+DISTRO="debian-bullseye"
+MACHINE="qemuamd64"
+DISTRO_ARCH="amd64"
+IMAGE_TYPE="isar-image-base"
+
+clean(){
+        if [ $? -eq 0 ]; then
+		[ -d $tmpdir ] && sudo rm -rf $tmpdir
+        else
+		echo "Keep build artifacts at $tmpdir"
+        fi
+}
+
+build(){
+        build_id="$1"
+        echo "Build name: $build_id"
+        mkdir "$tmpdir/build"
+
+        cd "$isar_root_dir"
+        . isar-init-build-env "$tmpdir/build" && bitbake mc:${MACHINE}-${DISTRO#*-}:${IMAGE_TYPE}
+
+        mv "$tmpdir/build" "$tmpdir/$build_id"
+}
+
+repro_check(){
+        image1="$tmpdir/$1/tmp/deploy/images/${MACHINE}/${IMAGE_TYPE}-${DISTRO}-${MACHINE}.tar.gz"
+        image2="$tmpdir/$2/tmp/deploy/images/${MACHINE}/${IMAGE_TYPE}-${DISTRO}-${MACHINE}.tar.gz"
+
+        # Compare two build artifacts
+        if diffoscope --html-dir "$tmpdir/diffoscope_html_output" "$image1" "$image2"; then
+		echo "[Success] The build artifacts are reproduicible"
+	else
+		echo "[Fail] The build artifacts are not matching"
+	fi
+}
+
+usage() {
+	echo "This script verifies the reproducibility of the Isar images."
+	echo
+	echo "Usage:"
+	echo "       $0 [params]"
+	echo
+	echo "Parameters:"
+	echo "	-d, --distro DISTRO 		set DISTRO variable for the builds (e.g.: debian-buster, debian-bullseye)"
+	echo "	-m, --machine MACHINE		set MACHINE variable for the builds (e.g.: qemuamd64, qemuarm64, qemuarmhf)"
+	echo "	-a, --arch DISTRO_ARCH		set DISTRO_ARCH variable for the builds (e.g.: amd64, arm64, armhf)"
+	echo "	-h, --help"
+	echo
+}
+
+while [ "$1" != "" ]; do
+        case $1 in
+		-d | --distro )
+			shift
+			DISTRO="$1"
+                        ;;
+                -m | --mahcine )
+                        shift
+                        MACHINE="$1"
+                        ;;
+                -a | --arch )
+                        shift
+                        DISTRO_ARCH="$1"
+                        ;;
+                -h | --help )
+                        usage
+                        exit
+                        ;;
+                * )
+                        usage
+                        exit 1
+        esac
+        shift
+done
+
+if ! command -v diffoscope &> /dev/null
+then
+    echo "diffoscope could not be found"
+    exit
+fi
+
+cd "$(dirname "$0")/.."
+isar_root_dir="$(pwd)"
+
+echo "Distro: $DISTRO"
+echo "MACHINE: $MACHINE"
+echo "DISTRO_ARCH: $DISTRO_ARCH"
+
+# export the variables for the build
+export DISTRO=${DISTRO}
+export MACHINE=${MACHINE}
+export DISTRO_ARCH=${DISTRO_ARCH}
+
+# Temorary directory for keeping build artifacts
+tmpdir=$(mktemp --directory --tmpdir /home/seshu/temp/isar.repro.test.XXXXXXXXXX)
+trap clean INT TERM EXIT
+
+build build1
+build build2
+
+repro_check build1 build2