From patchwork Fri Jun 21 15:08:10 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anton Mikanovich X-Patchwork-Id: 3625 Return-Path: Received: from shymkent.ilbers.de ([unix socket]) by shymkent (Cyrus 2.5.10-Debian-2.5.10-3+deb9u2) with LMTPA; Fri, 21 Jun 2024 17:08:36 +0200 X-Sieve: CMU Sieve 2.4 Received: from mail-lj1-f187.google.com (mail-lj1-f187.google.com [209.85.208.187]) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPS id 45LF8YG9006194 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Fri, 21 Jun 2024 17:08:35 +0200 Received: by mail-lj1-f187.google.com with SMTP id 38308e7fff4ca-2ec507c1b59sf6745021fa.3 for ; Fri, 21 Jun 2024 08:08:35 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1718982509; cv=pass; d=google.com; s=arc-20160816; b=gJu3pzd7EEkWlaYsQ81U0H0Crxn9OXlgQRQHO0e1+oV8UqrGtpItQXQlSg7Lvzh+xq X69GVtNfjm7j6EFENsNxxcBFv9db0pGLv/srkZ4biE/FKJsKdjPhY7atDG24W1686DJI guhPC/agzQlfOdi8owqVEEmv798UeU9b3kCUJ8HE1ZJuVGMfRLE7p30JLyDIlZuc5chX eUEtKCXeJFef4U0/dT4sc9YGNyEnniKCWjwSsYqOh+jCfx8uM5P5psk5oxkgmq5Jx6mZ AvkvIME/c7szGkybXo1qJamiSQmJx5Ap96q5PuCKs11hbpZpgJkZUiItiz2WZy32otkM ObJg== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:dkim-signature; bh=mGkOGeTuHlVCue8ajvAaTuTY1puIfrMbISsMQf1cP0I=; fh=lyZGSrwmVyv8siVc2vUnU0xCXoRs42zYdJq7dsL6TMI=; b=BvRV9ktqAz9g/ulC+mNj7qf8gpByrW9OhIEPytQ9eBE7FUoQVwgKfItj5DENZyLmq/ vGxOuUtnFAakSH16FSJB8px7J3tV81scmH4f60O8JfOH+9XrLiPe3jMtKaQ5Yt3hY6D0 cgABWaRm/xJ+dIK8CzmUeyts2MGvJuOIp9f6CZ8acuK0d34rl9WF2GRDz6ddaNQUcGUS QdrBlFhlrvvasTKo/hXJnbAuO67l3qV2Vf0JP8NFfOsE13EHb7e8uhTQNH6IjozfJWpn rsN4LLcYe7duzzsu4ESbvgi86k5me51BnjC+AXW8ag9wwvbZGUWcrev16Uz0ac8hlGff zAbQ==; darn=isar-build.org ARC-Authentication-Results: i=2; gmr-mx.google.com; spf=pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=amikan@ilbers.de DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlegroups.com; s=20230601; t=1718982509; x=1719587309; darn=isar-build.org; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:x-original-authentication-results :x-original-sender:mime-version:references:in-reply-to:message-id :date:subject:cc:to:from:sender:from:to:cc:subject:date:message-id :reply-to; bh=mGkOGeTuHlVCue8ajvAaTuTY1puIfrMbISsMQf1cP0I=; b=uGyFwRsCDFFdQJf1Rk7pTHC2Xh/R3qy9v9Ar99gf3V4R1I42ITRV9EWt7MpAEQUb9/ uUeQZRyXd92LGfEpidJlf155MGYh7QrAED5Vl8PvjvsIqRbsUf+RAKlN90SrfMbdAkDB LiOfoCQXaxtrqOYsTOt/J5KQd7NghNtqzD1Oxshpkr4YO15j9xFS/qz5fdIJsorx6pGM jfbXimcn80pEck1OaY8P7PJZvag4mZGmGPE/ISwp9fcOI1z3CuIf5WsTxR6byaYmrXw5 MRqJuf/moDNYQQ/i+rxQgcDy+Up9iOmT+15KKiesL07Fix/7L25CHL0yg0q4hTLQAIpI cQqQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1718982509; x=1719587309; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :x-spam-checked-in-group:list-id:mailing-list:precedence :x-original-authentication-results:x-original-sender:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :x-beenthere:x-gm-message-state:sender:from:to:cc:subject:date :message-id:reply-to; bh=mGkOGeTuHlVCue8ajvAaTuTY1puIfrMbISsMQf1cP0I=; b=NrQp/RLzFoAubznUmlPornt+GrIVgVyQg8+szibSYD0hgNHwyxaLJKdJ61GOo05rg4 +9X5kq4UQ8FhEUMzmHUl6cEpZrJtxkLV1XYTEuaCV1gQEm3bhviOIRLLjRRBwTyOrSht kpjM3uEyN0VKuIf2wQJo8XbP1i+BrmXwjHKxnV3OzT4TT+NlVB3HfxP0DGrurIH7ishp zZWfJhDctClblo/uCFebdHyBJSth3NcO8mvYu+gaYKoT5RTJp7KNIkQLkCwV1WKe4Kzi bKqlWkorIfupT15DAEbVXluBXjA3RjNCl8ysI79vNETF0152L5HxU986qYO++7pmLNhC Zq9Q== Sender: isar-users@googlegroups.com X-Forwarded-Encrypted: i=2; AJvYcCUT99RYxLr+bXe2+6/lr83BSdDfMJSuoSVpwEszVk8wNmyZyBgkikv4lXKvF/ICXrXLZc5s8Z/DsSgF844W0KTy8v3rcxE= X-Gm-Message-State: AOJu0Yyc4365iKA6cZ9MGT/cUESciMYaKAMFQbNUbRpqQ93UqCjlDPR8 qsnZJO3B1qy2zNqlLVeiwHcNKvMHnG7Ht8ZiVkqi2KMaLGiHWqFq X-Google-Smtp-Source: AGHT+IHT+Np8YIuZVexhDCaPipNdWbg07HAQR72mzVsjrgQPN6f/4tj2i5NS7m3wqz1925n2D4SEkA== X-Received: by 2002:a2e:b60f:0:b0:2ec:4f0c:36f9 with SMTP id 38308e7fff4ca-2ec4f0c38ccmr17297661fa.36.1718982508680; Fri, 21 Jun 2024 08:08:28 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a2e:9306:0:b0:2ec:4470:4e2b with SMTP id 38308e7fff4ca-2ec44704f95ls10607981fa.1.-pod-prod-05-eu; Fri, 21 Jun 2024 08:08:27 -0700 (PDT) X-Received: by 2002:a2e:a548:0:b0:2ec:40ab:694b with SMTP id 38308e7fff4ca-2ec40ab6a03mr70861591fa.47.1718982506667; Fri, 21 Jun 2024 08:08:26 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1718982506; cv=none; d=google.com; s=arc-20160816; b=LszWDLujei/x9COMGu+OBzgd2WPQEroGbKQynuSXo0H+SeNn/pmcZ16cQuXiUrBj5b 9LMOaiQQpc8Fycn3f8JcjAx+rW8bUvIcI0Cj2sDR0z4bZmtKysdBDRqLOTYlVN5588i6 JainqCrD56lvnvHAyMRH3tQ3XLDR5h/2ApkHN8tDvL3zywYJ/M0y+8Y9zXudtUXsrZtl G+aqvIhd3FaxjheeQK9U4ztaCfTn+Vnno2AKIFbw/83DhGxoqfFR/7O+3ls6zbEAjmDB yk4/cnoFkI9uqVdiyGhwisDbxRq+ii3zkohzp5clcdueEcqnBvc7IkdixQtuVcEu5G6D Ma/A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=bevBQJalmm++54d3Hasad1tvftGzZGzU/bCvhO1+YUU=; fh=z3rP6Lb9kCHMNktWueEpSrG6TTw5jLe7zgHgRhRlVvY=; b=Y/5opPc5P905kL5VG88yUCWCM6Lag6ydx4gVyMIzKaiDN9WRXK21ETrHWvfZVIHHky qn3WaHL/s7/aYakUyheU8IVIj7WPnT/Vu5EjQ2CLk2KZlEUVvGBaSUwmIS0IBlGl0eys 9G4pTfXz6/YLBKSDmWls3V4cCNCHlGFeS+N6swTzl9cJ/T9X7C986y1czmHKEnIyG4hb r3lkBS+XLkCWv8btWX2S/kYpJ/rT2vwngMdcwhB6v6rYt921VSootbb1aNSzCMJ1MCuT Wz41jqteyQ0BLJdMrIPkwkrWuHg7BB1W7uecr/Wmv6hZU4CFWBCc7eOOAZjcGyZhXWEc qzkQ==; dara=google.com ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=amikan@ilbers.de Received: from shymkent.ilbers.de (shymkent.ilbers.de. [85.214.156.166]) by gmr-mx.google.com with ESMTPS id 5b1f17b1804b1-42478c9bbdesi1253275e9.1.2024.06.21.08.08.26 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 21 Jun 2024 08:08:26 -0700 (PDT) Received-SPF: pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) client-ip=85.214.156.166; Received: from user-B660.promwad.corp ([159.148.83.114]) (authenticated bits=0) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPSA id 45LF8ObC006158 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 21 Jun 2024 17:08:25 +0200 From: Anton Mikanovich To: isar-users@googlegroups.com Cc: Ilia Skochilov , Anton Mikanovich Subject: [PATCH v3 1/5] start_vm: Switch to python version Date: Fri, 21 Jun 2024 18:08:10 +0300 Message-Id: <20240621150814.189288-2-amikan@ilbers.de> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240621150814.189288-1-amikan@ilbers.de> References: <20240621150814.189288-1-amikan@ilbers.de> MIME-Version: 1.0 X-Spam-Status: No, score=-4.6 required=5.0 tests=DKIMWL_WL_MED,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_EF,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,RCVD_IN_MSPIKE_H2,RCVD_IN_RP_CERTIFIED, RCVD_IN_RP_RNBL,RCVD_IN_RP_SAFE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on shymkent.ilbers.de X-Original-Sender: amikan@ilbers.de X-Original-Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=amikan@ilbers.de Precedence: list Mailing-list: list isar-users@googlegroups.com; contact isar-users+owners@googlegroups.com List-ID: X-Spam-Checked-In-Group: isar-users@googlegroups.com X-Google-Group-Id: 914930254986 List-Post: , List-Help: , List-Archive: , List-Unsubscribe: , X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= From: Ilia Skochilov Remove the shell version of start_vm. Use the python version of start_vm for both testsuite and command line. Signed-off-by: Ilia Skochilov Signed-off-by: Anton Mikanovich --- scripts/start_vm | 161 +++-------------------------------------- testsuite/README.md | 4 +- testsuite/cibuilder.py | 2 +- testsuite/start_vm.py | 7 +- 4 files changed, 19 insertions(+), 155 deletions(-) diff --git a/scripts/start_vm b/scripts/start_vm index 42899df9..cc80bce5 100755 --- a/scripts/start_vm +++ b/scripts/start_vm @@ -1,156 +1,17 @@ -#!/bin/sh +#!/usr/bin/env python3 # -# This software is a part of ISAR. -# Copyright (C) 2015-2017 ilbers GmbH +# Helper script to start QEMU with Isar image +# Copyright (c) 2024, ilbers GmbH -set -e +import os +import sys -ES_OK=0 -ES_BUG=3 +sys.path.append(os.path.join(os.path.dirname(__file__), '..', 'testsuite')) -# Convert bitbake assignments to shell ones -# a = b a=b -# a ?= b a=b -# TODO: Use bitbake to parse vars correctly (overriding in local.conf, etc.) -bb2sh() { - sed 's/[[:space:]]*?*=[[:space:]]*/=/' -} +from start_vm import parse_args, start_qemu -start_qemu() { - root=`echo $QEMU_DISK_ARGS \ - | sed 's,##ROOTFS_IMAGE##,'$IMAGE_DIR/$ROOTFS_IMAGE','` - [ -n "$KARGS$EXTRA_KARGS" ] && OPT_KERNEL_ARGS="-append $KARGS$EXTRA_KARGS" - local had_x - echo $- | grep -q x && had_x=1 || had_x=0 - [ $had_x -eq 0 ] && set -x - qemu-system-$QEMU_ARCH \ - -m 1024M \ - -M $QEMU_MACHINE \ - $QCPU \ - -nographic \ - $EXTRA_ARGS \ - $OPT_KERNEL_ARGS \ - $root - [ $had_x -eq 0 ] && set +x -} +if __name__ == "__main__": + args = parse_args() -show_help() { - echo "This script runs ISAR image in QEMU emulator." - echo - echo "Usage:" - echo " $0 [params] [BUILD_DIR]" - echo "BUILD_DIR is your ISAR build folder. If not set, current folder" - echo "is used." - echo - echo "Parameters:" - echo " -a, --arch ARCH set isar machine architecture." - echo " Supported: arm, i386, amd64, arm64, mipsel, riscv64." - echo " -b, --build BUILD set path to build directory." - echo " -d, --distro DISTRO set isar Debian distribution." - echo " Supported: buster, bullseye, bookworm" - echo " -o, --out FILE Route QEMU console output to" - echo " specified file." - echo " -p, --pid FILE Store QEMU pid to file." - echo " -s, --secureboot Enable secureboot with default MS keys." - echo " --help display this message and exit." - echo - echo "Exit status:" - echo " 0 if OK," - echo " 3 if invalid parameters are passed." -} - -# Set default values, that can be overwritten from command line -ARCH=arm -DISTRO=bookworm -BUILD_DIR=$PWD - -# Parse command line to get user configuration -while [ $# -gt 0 ] -do - key="$1" - - case $key in - -h|--help) - show_help - exit 0 - ;; - -a|--arch) - ARCH=$2 - shift - ;; - -b|--build) - BUILD_DIR=$2 - shift - ;; - -d|--distro) - DISTRO=$2 - shift - ;; - -o|--out) - EXTRA_ARGS="$EXTRA_ARGS -serial file:$2" - shift - ;; - -p|--pid) - EXTRA_ARGS="$EXTRA_ARGS -pidfile $2" - shift - ;; - -s|--secureboot) - OVMF_VARS_ORIG="/usr/share/OVMF/OVMF_VARS_4M.ms.fd" - OVMF_VARS="$(basename "${OVMF_VARS_ORIG}")" - cp "${OVMF_VARS_ORIG}" "${OVMF_VARS}" - EXTRA_ARGS="$EXTRA_ARGS -drive if=pflash,format=raw,unit=1,file=${OVMF_VARS}" - ;; - *) - echo "error: invalid parameter '$key', please try '--help' to get list of supported parameters" - exit $ES_BUG - ;; - esac - - shift -done - -eval $(bitbake -e mc:qemu$ARCH-$DISTRO:isar-image-base | grep "^DEPLOY_DIR_IMAGE=") -readonly IMAGE_DIR=$DEPLOY_DIR_IMAGE - -readonly ISARROOT="$(dirname "$0")"/.. - -readonly MACHINE_CONF=$ISARROOT/meta-isar/conf/machine/qemu$ARCH.conf -eval "$(egrep 'MACHINE_SERIAL|QEMU_' $MACHINE_CONF |bb2sh)" - -eval $(bitbake -e mc:qemu$ARCH-$DISTRO:isar-image-base | grep "\(^IMAGE_FSTYPES=\|^IMAGE_FULLNAME=\)") -# Take first image type for VM if there are several defined -IMAGE_FSTYPES=$(echo "$IMAGE_FSTYPES" | awk '{print $1}') -case "$IMAGE_FSTYPES" in - ext4) - readonly ROOTFS_IMAGE=$IMAGE_FULLNAME.ext4 - - eval $(bitbake -e mc:qemu$ARCH-$DISTRO:isar-image-base | grep "^KERNEL_IMAGE=") - eval $(bitbake -e mc:qemu$ARCH-$DISTRO:isar-image-base | grep "^INITRD_DEPLOY_FILE=") - QKERNEL=$IMAGE_DIR/${KERNEL_IMAGE} - QINITRD=/dev/null - [ -n "$INITRD_DEPLOY_FILE" ] && QINITRD=$IMAGE_DIR/${INITRD_DEPLOY_FILE} - if [ "$ARCH" = "riscv64" ]; then - EXTRA_ARGS="$EXTRA_ARGS -device loader,file=$QKERNEL,addr=0x80200000" - QKERNEL="/usr/lib/riscv64-linux-gnu/opensbi/qemu/virt/fw_jump.elf" - fi - EXTRA_ARGS="$EXTRA_ARGS \ - -kernel $QKERNEL \ - -initrd $QINITRD" - KARGS="console=$MACHINE_SERIAL root=/dev/$QEMU_ROOTFS_DEV rw $QEMU_KARGS" - ;; - wic) - readonly ROOTFS_IMAGE=$IMAGE_FULLNAME.wic - EXTRA_ARGS="$EXTRA_ARGS -snapshot" - ;; - *) - echo "IMAGE_FSTYPES \"$IMAGE_FSTYPES\" not supported" - exit 1 - ;; -esac - -QCPU= -[ -n "$QEMU_CPU" ] && QCPU="-cpu $QEMU_CPU" - -start_qemu - -exit $ES_OK + start_qemu(args.arch, args.build, args.distro, args.image, args.out, + args.pid, args.pcbios) diff --git a/testsuite/README.md b/testsuite/README.md index 913767fc..cfcfb1bf 100644 --- a/testsuite/README.md +++ b/testsuite/README.md @@ -57,11 +57,11 @@ $ avocado run ../testsuite/citest.py -t startvm,full ## Manual running -There is a tool `start_vm.py` which is the replacement for the bash script in +There is a tool start_vm which is the replacement for the bash script in `isar/scripts` directory. It can be used to run image previously built: ``` -./start_vm.py -a amd64 -b /build -d bullseye -i isar-image-base +start_vm -a amd64 -b /build -d bullseye -i isar-image-base ``` # Tests for running commands under qemu images diff --git a/testsuite/cibuilder.py b/testsuite/cibuilder.py index 227d0a96..12c48180 100755 --- a/testsuite/cibuilder.py +++ b/testsuite/cibuilder.py @@ -529,7 +529,7 @@ BBPATH .= ":${LAYERDIR}"\ 'DISTRO', \ target=multiconfig) - # only the first type will be tested in start_vm.py + # only the first type will be tested in start_vm if image_fstypes.split()[0] == 'wic': if wks_file: # ubuntu is less verbose so we do not see the message diff --git a/testsuite/start_vm.py b/testsuite/start_vm.py index ef0dfbcb..2c07b816 100755 --- a/testsuite/start_vm.py +++ b/testsuite/start_vm.py @@ -98,7 +98,7 @@ def start_qemu(arch, build, distro, image, out, pid, enforce_pcbios): print(cmdline) p1 = subprocess.call('exec ' + ' '.join(cmdline), shell=True) -if __name__ == "__main__": +def parse_args(): parser = argparse.ArgumentParser() parser.add_argument('-a', '--arch', choices=['arm', 'arm64', 'amd64', 'i386', 'mipsel'], help='set isar machine architecture.', default='arm') parser.add_argument('-b', '--build', help='set path to build directory.', default=os.getcwd()) @@ -107,6 +107,9 @@ if __name__ == "__main__": parser.add_argument('-o', '--out', help='Route QEMU console output to specified file.') parser.add_argument('-p', '--pid', help='Store QEMU pid to specified file.') parser.add_argument('--pcbios', action="store_true", help='remove any bios options to enforce use of pc bios') - args = parser.parse_args() + return parser.parse_args() + +if __name__ == "__main__": + args = parse_args() start_qemu(args.arch, args.build, args.distro, args.image, args.out, args.pid, args.pcbios) From patchwork Fri Jun 21 15:08:11 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anton Mikanovich X-Patchwork-Id: 3626 Return-Path: Received: from shymkent.ilbers.de ([unix socket]) by shymkent (Cyrus 2.5.10-Debian-2.5.10-3+deb9u2) with LMTPA; Fri, 21 Jun 2024 17:08:36 +0200 X-Sieve: CMU Sieve 2.4 Received: from mail-lj1-f189.google.com (mail-lj1-f189.google.com [209.85.208.189]) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPS id 45LF8ZKs006196 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Fri, 21 Jun 2024 17:08:35 +0200 Received: by mail-lj1-f189.google.com with SMTP id 38308e7fff4ca-2ec466c1748sf14460781fa.1 for ; Fri, 21 Jun 2024 08:08:35 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1718982509; cv=pass; d=google.com; s=arc-20160816; b=wD56LVR4ciIrIwkXQoFOGwwccbuNJbjREGpPY8nPVweBt5hdIZe3RbZxVE3iQs9/tG JyCZZYJf7T6FotlrDCiM8rTSDw/Aft9LT+vg7FoZOl4ygezKH7ARPQxPvsYijFhygmgM OSVDK2wpRYrLTHMI8AhfizCxeoOL5nYp7W6RCwt2hr0EYhEEz/5a0LoYecYKI8ms+Fk3 QMBv/WwOGwFfJ2coz2qWQqxc8UcWqbJZMr/RYFZUzMuH0D8R1DofHC8UgI4dpu4R4osD 8GVX6XZiLdy495Y81o2eunUTrZ7aVPz2Dg7Xdiu45BG4t8RqraqZ71DPfYej5XY9b4zP ynqA== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:sender:dkim-signature; bh=ffjvPC72uHUzZlkQc4o3O9QDVWF2pGUkAYV0f5+278A=; fh=+Rg1jTKH8GANFtS28OJowJh2fqG1mqJ0z56I3BmPPlg=; b=IXIpPqpJfsdcbuUeaX93GgPrkiG0GDl5qMZuiLudy0Ri6jl1OqO6G9Sn5ngHGAnTGj DVrmijjJERUpb3EDfGC2YCDcGR6OOWT+jdfSlLUVSRgkuSYNld/PQeUMVoV48qkG4CMP sVvfFTlc7dnWiwMLw/xCF4WKTS0uBQZ0+P4ICMJZcGjeiTomBoh2Ho0gBcmG+leAOS/3 6Iw97A8HkDy25WEi3cqARQGi/LjbBiWwqyJ52xu15xW2QJdDMmGTSc2BJfJGozsWYl24 71fFd+cz1XiSyuLhAbQIyAyGKUEFydHwxYb29Jwrez7Cwp1gvFnvvW1xtl6S0xOh6r4D Ea2w==; darn=isar-build.org ARC-Authentication-Results: i=2; gmr-mx.google.com; spf=pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=amikan@ilbers.de DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlegroups.com; s=20230601; t=1718982509; x=1719587309; darn=isar-build.org; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:x-original-authentication-results :x-original-sender:content-transfer-encoding:mime-version:references :in-reply-to:message-id:date:subject:cc:to:from:sender:from:to:cc :subject:date:message-id:reply-to; bh=ffjvPC72uHUzZlkQc4o3O9QDVWF2pGUkAYV0f5+278A=; b=rX/ayjwJsCX2Zx2c/ON5WkipaucNq1MmkQFpY71tGIViwU8I0gOl3aOUcUngLn21e5 5meZbJbWUDMch5W99ndfEhQQQoSDkVoslR3WoEA4QQNr4rDUxrKEm7l7lUE9Us7irdH4 Ofof5kU9DMsfXWz3KcyNkFClPgBSnHFn5vtVgdTQgBnWJQWdDBavkQkvCpVEKIMpkG75 68WjTa3RZQDQxzsHFg9zPFK0QXiCl6b8DiaO7TuU0xV9SUcW8v+gHSs/Q6xasRzwPhR2 Qy4dTlj/lHOhaxDrjyUBAlcDC3hQxwLevvqimQLY11MFlXVoh/Rhzpb/+rDY5GEgYPxy VXuw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1718982509; x=1719587309; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :x-spam-checked-in-group:list-id:mailing-list:precedence :x-original-authentication-results:x-original-sender :content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-beenthere:x-gm-message-state :sender:from:to:cc:subject:date:message-id:reply-to; bh=ffjvPC72uHUzZlkQc4o3O9QDVWF2pGUkAYV0f5+278A=; b=XwwtffX5SIFWu64Mo+0bfc50BKCQhnX12V1voXEtoKHDX5I6lAZxXOhhLVg2LDQSI0 ABybdULzaetfGciZ9Jo57UFeoUeEN4vlTPn9NqQX4Fg+25d+7PuUMuco9AJeYj2VfPv8 FSHFKIq+drppgRAXXnDS3peKh9VkzFsmEAVECSPWkio3+aMJMceXCUKlIy5ecOKleVyC TYsCsD3g7qP/etcS60SGGL4a22Ik+9mhu1LcJW6jem2RYuoSGqs/TnYCDGiQ5GVTbLVR udapH602B6u41MG/eZ/xgTKcTSX3ZbIGlHLda8aoNCYW2/CoiVqFV571YY9vJQZ8gXCV D0rA== Sender: isar-users@googlegroups.com X-Forwarded-Encrypted: i=2; AJvYcCXyFMuaieNO2uIB6t0bZ+NedSGHF34r1ztQNtoscIqasrpj6NPmmWziNEFkVdar+qH2YW9cDN+/vNInc9FpyDFreRTFpus= X-Gm-Message-State: AOJu0YzaIma+XswYw2fIy731F77+3EULgMVaJF2Fd1TSvOoBDkqBQRkV wsJZgR/5VSQ6yqaFmjE8pH31y2Q9InhAA4mj9pfBXr76IdIV4I0B X-Google-Smtp-Source: AGHT+IERx8ysu5vglgDfJktenmH9yD2L5p8kkDZTvKmOIA6YbaZSQTptEu7I5WDetJqU2qrDFbZThA== X-Received: by 2002:a2e:9e4c:0:b0:2ea:e592:5bf9 with SMTP id 38308e7fff4ca-2ec3ceb6b5amr53763491fa.11.1718982509263; Fri, 21 Jun 2024 08:08:29 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a05:651c:21c:b0:2eb:fe83:2b09 with SMTP id 38308e7fff4ca-2ec44377817ls10012051fa.1.-pod-prod-04-eu; Fri, 21 Jun 2024 08:08:27 -0700 (PDT) X-Received: by 2002:a2e:9dd4:0:b0:2ec:1ad3:fb0a with SMTP id 38308e7fff4ca-2ec3cfff3afmr53252521fa.43.1718982507130; Fri, 21 Jun 2024 08:08:27 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1718982507; cv=none; d=google.com; s=arc-20160816; b=AY6kwAx8k7yS1a9vbmt5jEwAkp18KYuDk2/qf2NGMhxXQrJPHD32qMZiH1bOjNyWPf PpkNQe1QmNzdhBwQv8sdR40K76vzSaw5j7U+TN57QsBHxi9ZgRXWk3DLpZblxaQ97Ue9 3lOaD4VT67EGCYGC87mITXEtf+8NABwXiaalyB8EHlfcdGlP7/PJtXJuX/LxTRmuBnNA OU3f5twwkA+qzxoQJdZze/hL3tXImZdYKlbNNGdmeFUpHo8bQgd0VVFJfMK4Wkma+Wpw n0nkt0qEs3++YNTLvi1LYuByIobhbNNOd7wXPYxjGA9+QvjsH3dOVj9R2okB+BUqt8q5 n7XA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=rLtBOlbrqAcftBjpFbc85TzRJOdShW3rqncFAksxL9Q=; fh=/h9QQkzJ8EboVkWg45aWwpaUro6WMavIVd2OhN45RtE=; b=PCDJc1cgzf+315gU3dklwzwSTIsSxaE2/8elY09GxocaQSwSO7GUHjqRMNNaHkTzGw 9PvKHQaYNhKVl1m704xXDZ6rp4xZ9FqH4jZYgwwW9Ag6utufYO2Z06gZSttqWdgXWcov ahh6fxvqwzeoil5ynmJWoR1LvwhpyFLAUxLgjZaegY+uaLqKiEdB1Sq8N+Il/sIJn5Of pGyxVbKxOShk4k0ym7XccJd/B46WIqQJR2nxoxcBYqcZ7FURuw0Vy0KVO7ZBTpVupx1O 9BZwmDmxE7+oxf23gYed6zBbxAeJM7Zv/t92L4OM7R86JEKFu8951IWzsXYnArdoDdPh sAuw==; dara=google.com ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=amikan@ilbers.de Received: from shymkent.ilbers.de (shymkent.ilbers.de. [85.214.156.166]) by gmr-mx.google.com with ESMTPS id 38308e7fff4ca-2ec4d75859esi419381fa.3.2024.06.21.08.08.26 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 21 Jun 2024 08:08:27 -0700 (PDT) Received-SPF: pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) client-ip=85.214.156.166; Received: from user-B660.promwad.corp ([159.148.83.114]) (authenticated bits=0) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPSA id 45LF8ObD006158 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 21 Jun 2024 17:08:26 +0200 From: Anton Mikanovich To: isar-users@googlegroups.com Cc: Anton Mikanovich Subject: [PATCH v3 2/5] start_vm: Add secureboot support Date: Fri, 21 Jun 2024 18:08:11 +0300 Message-Id: <20240621150814.189288-3-amikan@ilbers.de> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240621150814.189288-1-amikan@ilbers.de> References: <20240621150814.189288-1-amikan@ilbers.de> MIME-Version: 1.0 X-Spam-Status: No, score=-4.6 required=5.0 tests=DKIMWL_WL_MED,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_EF,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,RCVD_IN_MSPIKE_H2,RCVD_IN_RP_CERTIFIED, RCVD_IN_RP_RNBL,RCVD_IN_RP_SAFE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on shymkent.ilbers.de X-Original-Sender: amikan@ilbers.de X-Original-Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=amikan@ilbers.de Precedence: list Mailing-list: list isar-users@googlegroups.com; contact isar-users+owners@googlegroups.com List-ID: X-Spam-Checked-In-Group: isar-users@googlegroups.com X-Google-Group-Id: 914930254986 List-Post: , List-Help: , List-Archive: , List-Unsubscribe: , X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= Add sb_copy_vars()/sb_cleanup() API for creation of OVMF variables file copy in case it was declared in QEMU_DISK_ARGS value. If OVMF_VARS_4M.ms.fd is already exists it will be reused, otherwise created copy will be deleted after QEMU exit. sb_copy_vars() returns True if cleanup needed. Signed-off-by: Anton Mikanovich --- meta-isar/conf/machine/qemuamd64-sb.conf | 2 +- testsuite/cibuilder.py | 15 ++++++--- testsuite/start_vm.py | 41 ++++++++++++++++++++++-- 3 files changed, 50 insertions(+), 8 deletions(-) diff --git a/meta-isar/conf/machine/qemuamd64-sb.conf b/meta-isar/conf/machine/qemuamd64-sb.conf index 2bec553b..9ad5f8d6 100644 --- a/meta-isar/conf/machine/qemuamd64-sb.conf +++ b/meta-isar/conf/machine/qemuamd64-sb.conf @@ -18,4 +18,4 @@ IMAGER_INSTALL:wic += "${GRUB_DEBIAN_SB_MOK}" IMAGE_PREINSTALL += "mokutil" # overwrite qemu disk args for signed boot -QEMU_DISK_ARGS = "-drive file=##ROOTFS_IMAGE##,format=raw -global driver=cfi.pflash01,property=secure,value=on -drive if=pflash,format=raw,unit=0,file=/usr/share/OVMF/OVMF_CODE_4M.ms.fd,readonly=on" +QEMU_DISK_ARGS = "-drive file=##ROOTFS_IMAGE##,format=raw -global driver=cfi.pflash01,property=secure,value=on -drive if=pflash,format=raw,unit=0,file=/usr/share/OVMF/OVMF_CODE_4M.ms.fd,readonly=on -drive if=pflash,format=raw,unit=1,file=OVMF_VARS_4M.ms.fd" diff --git a/testsuite/cibuilder.py b/testsuite/cibuilder.py index 12c48180..87d71387 100755 --- a/testsuite/cibuilder.py +++ b/testsuite/cibuilder.py @@ -477,6 +477,8 @@ BBPATH .= ":${LAYERDIR}"\ boot_log, None, enforce_pcbios) cmdline.insert(1, '-nographic') + need_sb_cleanup = start_vm.sb_copy_vars(cmdline) + self.log.info('QEMU boot line:\n' + ' '.join(cmdline)) self.log.info('QEMU boot log:\n' + boot_log) @@ -485,7 +487,7 @@ BBPATH .= ":${LAYERDIR}"\ universal_newlines=True) self.log.info("Started VM with pid %s" % (p1.pid)) - return p1, cmdline, boot_log + return p1, cmdline, boot_log, need_sb_cleanup def vm_wait_boot(self, p1, timeout): @@ -564,6 +566,9 @@ BBPATH .= ":${LAYERDIR}"\ pid = self.vm_dict[vm][0] os.kill(pid, signal.SIGKILL) + if self.vm_dict[vm][3]: + start_vm.sb_cleanup() + del(self.vm_dict[vm]) self.vm_dump_dict(vm) @@ -600,7 +605,7 @@ BBPATH .= ":${LAYERDIR}"\ stderr = "" if vm in self.vm_dict: - pid, cmdline, boot_log = self.vm_dict[vm] + pid, cmdline, boot_log, need_sb_cleanup = self.vm_dict[vm] # Check that corresponding process exists proc = subprocess.run("ps -o cmd= %d" % (pid), shell=True, text=True, @@ -612,8 +617,10 @@ BBPATH .= ":${LAYERDIR}"\ if run_qemu: self.log.info("No qemu-system process for `%s` found, run new VM" % (vm)) - p1, cmdline, boot_log = self.vm_turn_on(arch, distro, image, enforce_pcbios) - self.vm_dict[vm] = p1.pid, cmdline, boot_log + p1, cmdline, boot_log, \ + need_sb_cleanup = self.vm_turn_on(arch, distro, image, + enforce_pcbios) + self.vm_dict[vm] = p1.pid, cmdline, boot_log, need_sb_cleanup self.vm_dump_dict(vm) rc = self.vm_wait_boot(p1, timeout) diff --git a/testsuite/start_vm.py b/testsuite/start_vm.py index 2c07b816..0e0d7000 100755 --- a/testsuite/start_vm.py +++ b/testsuite/start_vm.py @@ -1,15 +1,18 @@ #!/usr/bin/env python3 # # Helper script to start QEMU with Isar image -# Copyright (c) 2019, ilbers GmbH +# Copyright (c) 2019-2024, ilbers GmbH import argparse import os import socket import subprocess import sys +import shutil import time +OVMF_VARS_PATH = '/usr/share/OVMF/OVMF_VARS_4M.ms.fd' + def get_bitbake_env(arch, distro, image): multiconfig = 'mc:qemu' + arch + '-' + distro + ':' + image output = subprocess.check_output(['bitbake', '-e', str(multiconfig)]) @@ -91,16 +94,48 @@ def format_qemu_cmdline(arch, build, distro, image, out, pid, enforce_pcbios=Fal return cmd + +def sb_copy_vars(cmdline): + ovmf_vars_filename = os.path.basename(OVMF_VARS_PATH) + + for param in cmdline: + if ovmf_vars_filename in param: + if os.path.exists(ovmf_vars_filename): + break + if not os.path.exists(OVMF_VARS_PATH): + print(f'{OVMF_VARS_PATH} required but not found!', + file=sys.stderr) + break + shutil.copy(OVMF_VARS_PATH, ovmf_vars_filename) + return True + + return False + + +def sb_cleanup(): + os.remove(os.path.basename(OVMF_VARS_PATH)) + + def start_qemu(arch, build, distro, image, out, pid, enforce_pcbios): cmdline = format_qemu_cmdline(arch, build, distro, image, out, pid, enforce_pcbios) cmdline.insert(1, '-nographic') + need_cleanup = sb_copy_vars(cmdline) + print(cmdline) - p1 = subprocess.call('exec ' + ' '.join(cmdline), shell=True) + + try: + subprocess.call('exec ' + ' '.join(cmdline), shell=True) + finally: + if need_cleanup: + sb_cleanup() + def parse_args(): parser = argparse.ArgumentParser() - parser.add_argument('-a', '--arch', choices=['arm', 'arm64', 'amd64', 'i386', 'mipsel'], help='set isar machine architecture.', default='arm') + arch_names = ['arm', 'arm64', 'amd64', 'amd64-sb', 'i386', 'mipsel'] + parser.add_argument('-a', '--arch', choices=arch_names, + help='set isar machine architecture.', default='arm') parser.add_argument('-b', '--build', help='set path to build directory.', default=os.getcwd()) parser.add_argument('-d', '--distro', choices=['buster', 'bullseye', 'bookworm', 'trixie', 'focal', 'jammy'], help='set isar Debian distribution.', default='bookworm') parser.add_argument('-i', '--image', help='set image name.', default='isar-image-base') From patchwork Fri Jun 21 15:08:12 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anton Mikanovich X-Patchwork-Id: 3627 Return-Path: Received: from shymkent.ilbers.de ([unix socket]) by shymkent (Cyrus 2.5.10-Debian-2.5.10-3+deb9u2) with LMTPA; Fri, 21 Jun 2024 17:08:37 +0200 X-Sieve: CMU Sieve 2.4 Received: from mail-lj1-f191.google.com (mail-lj1-f191.google.com [209.85.208.191]) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPS id 45LF8ZWB006208 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Fri, 21 Jun 2024 17:08:35 +0200 Received: by mail-lj1-f191.google.com with SMTP id 38308e7fff4ca-2ec266e43b4sf20241031fa.2 for ; Fri, 21 Jun 2024 08:08:35 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1718982510; cv=pass; d=google.com; s=arc-20160816; b=aQEGrjnxANQtCQOdtK4V2IeNW+00+72w+qsq/tEvDfi1FVRgzamaGRRG4onc4UgCWx ViyR9SFwuI5tvXJtJmYi+BjwZ7hnfpB2hbz6AwAMfxKClANcZzA0luNnv7wqWkJoLdzX kP3tIwC2IVUE0109YJbVOr5Qbci4TBalOF6YOoBx5vORpH3PBa0lVDkjj49QBQEapvj4 67tvLWzBpqD7PjhFD18yBt7t8jjuLhdKzr7AaF1cTkYb77rU/r1KPxDjbDNC2nASt8aJ m4z4J8Hm6AupF9SqRAym/9We+JYkxZuXyK85RE3kBbp41/CwcQI/X+hF/HR+dky+BK8d M0Fw== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:dkim-signature; bh=MsLqxAf0YUT+74pquErGUgo4UD3jNcWonVfWl3Pdz1w=; fh=RC8mkgqgo6MKS0gT552yB1wp6wHFplHWXcAfpXHD1RE=; b=GA0yB6/yNJ/BKQdIr6HSKlmPan+4DIqfPjJtlfbfNnhFPf4YxMyGoxcFMd5/433K2c 0w4DNq1zCzUj22OJMbcVPW/CtuVXwtu8UE75IUpcOoEDUBEX61xna+YIR6gnNoYx1rPg o23uJ63nEhysJszq9js2AhpNI5TtEQrZ+m1C9WeY02QvmbQGGrEzQqgKB+n8qxVUdJIq 0LSBYsDz4QkKFLyDBMCcG/Y/xfZ0vrE7tkpSUTOo3jhyEeYfmqG3V4SPZzGxRFjpLsKY ahkbhCqnBTLltr9cE+9quwgUxp7Be0rjSKFvTqBpshiuPKVhTdgRHeic7+/I+UT3+7yi eUXg==; darn=isar-build.org ARC-Authentication-Results: i=2; gmr-mx.google.com; spf=pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=amikan@ilbers.de DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlegroups.com; s=20230601; t=1718982510; x=1719587310; darn=isar-build.org; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:x-original-authentication-results :x-original-sender:mime-version:references:in-reply-to:message-id :date:subject:cc:to:from:sender:from:to:cc:subject:date:message-id :reply-to; bh=MsLqxAf0YUT+74pquErGUgo4UD3jNcWonVfWl3Pdz1w=; b=HHCjxfqGqH+Kd0TQMvZ55+2Bpwr7CZIbS17j3LJa0/Otekv3KOjow71PIwLS41x2Y3 wpmzRzOyCHLe8MlLGyHj76v3Qv00oi1TbDoejqH/U0VeX7Hx+QtxJKTGP/3ChWqexPPe wM3tqJXZCdkG+ipxm/PhE+2yKvwaYJlFU2WkYU8ZNUrLIeaGE86gBnYkgaRQGgwQppix DRYubgyTd7PEb7pXwjjM/llw+lkFCuPBVGpGqTADIm3XqnmjhHIDHXD99CKvdOv1R//N +XqKBMFYSjkHgoiocLgkhKF7hjQ2Mj71dispLKJKiar/IWPYGCe5yGTjcaHdXKmyc6pk OOaQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1718982510; x=1719587310; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :x-spam-checked-in-group:list-id:mailing-list:precedence :x-original-authentication-results:x-original-sender:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :x-beenthere:x-gm-message-state:sender:from:to:cc:subject:date :message-id:reply-to; bh=MsLqxAf0YUT+74pquErGUgo4UD3jNcWonVfWl3Pdz1w=; b=abIGzoKRBuCbADNZ4TPKoyIyKsg7RlOn0e1lbwkduJYmTJxQzPM/cyLgc22x7DFWBN ajaEfYkYsxCyMt4gtcLnzu4OEuyXkJjvFX7PghRvCghbl8fUOVD6vL/B5AzoYFyKCu/F uzpju7hOO35i+jmKLD6h8W9vPeFtM9TyxsRCxHLYEHXb5RcWntX23Ue4K550gK5dSYl7 CSeBpsB8FGlt41zOGyiyAu7E6++NSTm8Pl3PYQrRo5YPkMjG3xxilAHa9WLi9UtqdYWd jHgHmKuti13VXsfyp/WsEpEmeAC9COTjeiDWaJgwAunPCCIzrVtNlA3nee3R/eJ9mgAD WkLA== Sender: isar-users@googlegroups.com X-Forwarded-Encrypted: i=2; AJvYcCU0CV3xppUF2TmzE/EdRfEYAA3JqrQi2E53idJB0DDdZjgYgowtsFxMi4d/l1PAKfgKmOj6jjUkTAsXrw9rJzGgD2IGWnI= X-Gm-Message-State: AOJu0YzOTpUrW6A+bM5n9ytQ++9lbzWeEzAbUu9auznK+p7Vry1C3uo/ mqYOz5gmOTfcLuJQkJ8olJPyWV996Jb2PPi51bNpeUb2X99wsobF X-Google-Smtp-Source: AGHT+IGz5Nl4c4ED7jyBj3Si534seDYH06w4bHHF2c100CQBjxQ08tLIcq0MyKWjX6zvhI5buXf4uQ== X-Received: by 2002:ac2:538c:0:b0:52c:dba6:b4c8 with SMTP id 2adb3069b0e04-52cdba6b563mr601891e87.13.1718982509612; Fri, 21 Jun 2024 08:08:29 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a19:430d:0:b0:52c:a0a5:c6a0 with SMTP id 2adb3069b0e04-52ccf0a227fls972798e87.2.-pod-prod-08-eu; Fri, 21 Jun 2024 08:08:27 -0700 (PDT) X-Received: by 2002:ac2:5968:0:b0:52c:da39:87a7 with SMTP id 2adb3069b0e04-52cda398bcfmr1001430e87.41.1718982507519; Fri, 21 Jun 2024 08:08:27 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1718982507; cv=none; d=google.com; s=arc-20160816; b=YBnmfLHNsBQedSSGShsxOKucCFSEsc6A0NlnjIWwJOM23e5lK7zS3CLaPR3eGzZyzv A0CT+oTcCpQy2/UX6JLAGYs/Uw9NVmrkCI2Z4nkIY0MXNZ5DU1lQDI3VG6Ndh3IBmVDY K6QV69gDZj60tsTX/nzzuqVp14OIi4/HQ4eUZciHDMSy3AH417WJz0mSWT3GowPrflzV wJuidwM7mrLcvajfWii7Zq9L4UjO1baUFLRhmJi5T0j8ui77rRJL7DfAuu1sRI8dSVhu 4zpxlLqJj9kCOW7C9O1CB1vJpO98K/AgaHf5Rqbc7vkPbR1+eWGGNCd4pHKJuXbrKTgM oHVQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=iLMagJZvpLbiHY5OQYbhzhmnD94wl/7GMgu8HpDDuPg=; fh=z3rP6Lb9kCHMNktWueEpSrG6TTw5jLe7zgHgRhRlVvY=; b=jo5Ox3SHmTy/a2N2tfonmTKdSmMW7JO5uygK7o1l5KcZj21/9MI+M8o8zNsyig8kAI SigyGHp2S5YG6CRHYOV8d8mnsJPyippPOWdyB9YlfcZc454HQ9ckzdzvrE3Uqf97Nxr0 Bf0SY6yMDmG3R0KlVE59N+3glPZTQzU2chm/ojUvruVC5w2WagvJzv6r4D2q5mpFHu2k fJzJRyO8aVfS7MaB3/xqUSqyjaV5gyHjAUqX9Fwyz9Q+ZhzPd7fYsjUS7WOPd7o2Jbxu GYoHpZBCUMI7btUdjBItxjdCJzDDbAJfu6XzTQaKdNXm0+caAg1mmlSbN/B2876/VNLw G7MQ==; dara=google.com ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=amikan@ilbers.de Received: from shymkent.ilbers.de (shymkent.ilbers.de. [85.214.156.166]) by gmr-mx.google.com with ESMTPS id 2adb3069b0e04-52cd63bdd9bsi39211e87.5.2024.06.21.08.08.27 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 21 Jun 2024 08:08:27 -0700 (PDT) Received-SPF: pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) client-ip=85.214.156.166; Received: from user-B660.promwad.corp ([159.148.83.114]) (authenticated bits=0) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPSA id 45LF8ObE006158 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 21 Jun 2024 17:08:26 +0200 From: Anton Mikanovich To: isar-users@googlegroups.com Cc: Ilia Skochilov , Anton Mikanovich Subject: [PATCH v3 3/5] scripts: Remove vm_smoke_test Date: Fri, 21 Jun 2024 18:08:12 +0300 Message-Id: <20240621150814.189288-4-amikan@ilbers.de> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240621150814.189288-1-amikan@ilbers.de> References: <20240621150814.189288-1-amikan@ilbers.de> MIME-Version: 1.0 X-Spam-Status: No, score=-4.6 required=5.0 tests=DKIMWL_WL_MED,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_EF,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,RCVD_IN_MSPIKE_H2,RCVD_IN_RP_CERTIFIED, RCVD_IN_RP_RNBL,RCVD_IN_RP_SAFE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on shymkent.ilbers.de X-Original-Sender: amikan@ilbers.de X-Original-Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=amikan@ilbers.de Precedence: list Mailing-list: list isar-users@googlegroups.com; contact isar-users+owners@googlegroups.com List-ID: X-Spam-Checked-In-Group: isar-users@googlegroups.com X-Google-Group-Id: 914930254986 List-Post: , List-Help: , List-Archive: , List-Unsubscribe: , X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= From: Ilia Skochilov Remove vm_smoke_test, its functionality is now covered in the dev testsuite. Remove mentioning of vm_smoke_test in documentation. Add instructions about how to run the dev testsuite via avocado framework in a kas container. Signed-off-by: Ilia Skochilov Signed-off-by: Anton Mikanovich --- CONTRIBUTING.md | 50 +++++++++++--------- scripts/vm_smoke_test | 106 ------------------------------------------ 2 files changed, 28 insertions(+), 128 deletions(-) delete mode 100755 scripts/vm_smoke_test diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 9ad6bf3b..f932417b 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -80,28 +80,34 @@ Plan merges to `master` so that both fit the two-week window; short extensions s * It's highly suggested to test your patchset before submitting it to the mailing by launching CI tests scripts. The procedure is described below: - First, run "fast" CI -``` - scripts/ci_build.sh -q -f - ... - source isar-init-build-env - scripts/vm_smoke_test -f -``` - Currently "fast" CI launches - * parallel cross build of QEMU arm/arm64/amd64 Debian stretch and Raspberry Pi 1 Raspbian stretch targets - * cross build of one of the supported boards which includes compilation of Linux kernel/U-Boot for it - * Launches login prompt check tests for built QEMU targets - - Second, run standard CI -``` - scripts/ci_build.sh -q - ... - source isar-init-build-env - scripts/vm_smoke_test -q -``` - Currently standard CI launches - * parallel native build of QEMU arm/arm64/i386/amd64 Debian stretch/buster and Raspberry Pi 1 Raspbian stretch targets - * Launches login prompt check tests for built QEMU targets + git clone https://github.com/siemens/kas + cat > kas.yml < .config/avocado/avocado.conf -[datadir.paths] -base_dir = $BUILD_DIR/ -test_dir = $BUILD_DIR/tests -data_dir = $BUILD_DIR/data -logs_dir = $BUILD_DIR/job-results -EOF -export VIRTUAL_ENV="./" - -if avocado $VERBOSE run "$TESTSUITE_DIR/citest.py" -t $TAGS,startvm \ - --test-runner=runner --disable-sysinfo \ - -p build_dir="$BUILD_DIR" -p time_to_wait=$TIMEOUT; then - RET=$ES_OK -fi - -exit $RET From patchwork Fri Jun 21 15:08:13 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anton Mikanovich X-Patchwork-Id: 3628 Return-Path: Received: from shymkent.ilbers.de ([unix socket]) by shymkent (Cyrus 2.5.10-Debian-2.5.10-3+deb9u2) with LMTPA; Fri, 21 Jun 2024 17:08:38 +0200 X-Sieve: CMU Sieve 2.4 Received: from mail-lf1-f58.google.com (mail-lf1-f58.google.com [209.85.167.58]) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPS id 45LF8aWl006227 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Fri, 21 Jun 2024 17:08:37 +0200 Received: by mail-lf1-f58.google.com with SMTP id 2adb3069b0e04-52c805e6f38sf1708166e87.0 for ; Fri, 21 Jun 2024 08:08:37 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1718982510; cv=pass; d=google.com; s=arc-20160816; b=0ME5Msra8K8L2pot3YUwESv2LPZ1ip2hkM/E2G8QTjKf2gxxKojKVtWDU7K7IFy73X xLTJoN4p9nUMIDrUglblpVrSpe2hRPmtle9JlK/zktsq62zjIR4eMazW8mbHeb2SwduH sgaCXH1bJLCxD7Boz8K9Rdht65YVF/vIOMUlF6UWX2S0TUceCQ8LGLKwdd4gJbrIDWdK z6X7UAe5wgsQpaPQAV3Wdlyfz/wk6u+KQRMqyasxeZwssCLjxLDPH+i+Y/Mcj3Lrhsk3 x4y7xsH8mfRuRWFeVCdHDL6sTGRI+/C1stL1yJ9a3sluv3c8OZE4CRrKc/0yCuC10TZN 1glQ== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:dkim-signature; bh=LNivfEa5oZS35Fux4/cRKVMpln8H1gjip2LbM+Ih97s=; fh=FZGH+NqNtSsECqCQoJNaq8cRES7+ZiCtaWwkFBikIg0=; b=jcIUEOPx3i6QhrfNkjZrpHAa3gNh6b0GnLKQPw+bqRuzerZXH5JLcyBQNSaVg7Oifh SB090NYpxDNJV9y5R6IcBUsDtYDnTMie/6/I4LubingEl0lU7fWFjEqNdOm9+MaAg4TY 3gsq9UaOPVkANgvEuIV7AukMvnabhTIke56YgxvZVdAfqF86R1SjQrQBoYUAXyesFq9e Z8f9PgSjFX5emsSQUoA+BP+qQXz3En5Ni+TxG6YnamIk77teZFu+Edvy7yadF/QPrmS8 Q24TmjN3MFNDQw+NAFa/11Zz1HYK8cZm0Crtxscw9yeyk1+8Szgcxv58HOouVPGrrXMy 5mMA==; darn=isar-build.org ARC-Authentication-Results: i=2; gmr-mx.google.com; spf=pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=amikan@ilbers.de DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlegroups.com; s=20230601; t=1718982510; x=1719587310; darn=isar-build.org; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:x-original-authentication-results :x-original-sender:mime-version:references:in-reply-to:message-id :date:subject:cc:to:from:sender:from:to:cc:subject:date:message-id :reply-to; bh=LNivfEa5oZS35Fux4/cRKVMpln8H1gjip2LbM+Ih97s=; b=ioDgsD3A+JoqgyaVevjXzm/kivZpKufmfJgRt82upjGJMN98fQf8JZXem6/TyJb3WM SkzRYXeawbEWSwbQ7h1Nd5n2lfsgJopBkG8EKDzyvJ7qXyCOum11HtV7EOTzvUGepoQT VwXb188FpmfKWd/z4z8Wzp1ufxRq/cOFwB4elPQwqTtroAQ/9qsgiEwxPc0F70JwB575 TIK/xG9frdfYncQwBMfEfASYMf2xiK7WGI/1tVCGOIVXZaghvf8PCABeQ6di6E92IvXI d8trlQHx0IjToLicWiDASjFJufX/pseydaAjq6YhdyfDmDz1FnmyM9cEmJod3e0afCYQ SGWA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1718982510; x=1719587310; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :x-spam-checked-in-group:list-id:mailing-list:precedence :x-original-authentication-results:x-original-sender:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :x-beenthere:x-gm-message-state:sender:from:to:cc:subject:date :message-id:reply-to; bh=LNivfEa5oZS35Fux4/cRKVMpln8H1gjip2LbM+Ih97s=; b=a4y+bI8JfVpWkLB3it7Z+MRfxqlr02Cv7Mpv31Q2vN+tuJ/go/J5J9gtdDJeQB5FjZ T+iu4eUM76xeKWNnC3TI+/4V4v0cqzNzXdXRnpGw1KQxYVelIoqDsWZtWpTdep6j/zHn xevKr0EPgzGPbMPEvBKZTJZBzT5LTKpSIRSPTFqRZG90zY9E/djibeZQ4/6TWxTxY94B 2ZmgAeTVZuCSLFoXtxU4TsM0efgxTkdhuMhTNJ7gZWjwiJaLFFMuwd3SpXoJoPoAKzvo iQXNHrk6rtmdtM6TEnd6KvqfKaGdm/F2mXHQrXnuAwSJTELxJMSdFimRmigPT8kQQd/m LcZA== Sender: isar-users@googlegroups.com X-Forwarded-Encrypted: i=2; AJvYcCXw45sH9sxcrvgVnc02Gkt0Q+Xyr/dWPYwwsgSj8oBlGi4mCd7O54nB4EAc6FCsk6ozKT6FoXuZUAXpHOR6stKyAznd0gQ= X-Gm-Message-State: AOJu0YxbZD7iuZiC3m3p9w4abJQDi15g4eijSm+6lOdRhPDMgkgpp6H9 +BMF38RTSRdacAZN383qh4OjYbq0h1IiukFE5NEBQ24Q15MY0qy9 X-Google-Smtp-Source: AGHT+IEJGt+xQD2aqARxww1okWe/vOkDT4KGXyIvQXmQAkFxqkUDGXO+QVQ3Qxebo8/SOri0eav0lA== X-Received: by 2002:a19:644c:0:b0:52c:dbe6:f5f9 with SMTP id 2adb3069b0e04-52cdbe6f808mr399648e87.12.1718982509647; Fri, 21 Jun 2024 08:08:29 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a05:6512:2206:b0:52c:a105:90ee with SMTP id 2adb3069b0e04-52ccf08ea71ls164306e87.1.-pod-prod-01-eu; Fri, 21 Jun 2024 08:08:28 -0700 (PDT) X-Received: by 2002:ac2:489b:0:b0:52c:d56f:b2cc with SMTP id 2adb3069b0e04-52cd56fb3bfmr1421759e87.58.1718982507620; Fri, 21 Jun 2024 08:08:27 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1718982507; cv=none; d=google.com; s=arc-20160816; b=YlEFlGHUzXH0cxRe+jEQZ0I21onVVkku/dSSTJTao8Uamxy1U5bERY+F+ujv5BTgR6 fq7VnLwyQ0vOuR6XAO65CE9r1TQIQNxeoEXCpSKMjwG/Uk+cZQ1PdgehGwhy6vrrqPwO eI89orGc4bcAEo7Soyl6cGWN/Skbqpj27WXM4fShrIy1HIt6ZNzylHh4R91m3vEsCkrt BjajtNkJSzRUJk3Ah6MnNJ7AFEA0vorJSciQZ4tjkyeHYLaWCWBNWt7vWzLOyo4+fDuS kj4m2DQdoNzFVAKLqazsR2BvSt0SFaVr/qj5ERwg1V8svy/LUDoF8WCO4ceaasazRJmk PwIw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=GQDMB7shaswAgAwX4lgZ4QgmvgefyZhHCIVp2Od8fxk=; fh=/h9QQkzJ8EboVkWg45aWwpaUro6WMavIVd2OhN45RtE=; b=YH1dPrB/GdbJ08MXNr38UvxAsbHHFMlUO7RPsBCeJSpFHHoN8KAD0v/NlyAs83rKRz LbwWtZ04Jkp+fJLZi50KZKlGjv4yyONfeqwH4/X5xq2s83uO93eKnAnIAkSY5SHFEW+g xOSKZqzAVGXV0VIh/kquJU+JFjzuMb71kkzC3JN5OY/QZnmDGPkLAxZ5mhY5ffUhH7Kq aZA3mHGgltKENKzdiqF5mPGs+ooGfuY0tETkvP5AB3RC57E4xYJPJfk8UTKdLz4gaCTR kDdD2hYMKP797gBIPFFkA59bGfHx8A9dLvzViWvs+9emikyYPf1ubh5zUy19bmJA9STm V1CA==; dara=google.com ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=amikan@ilbers.de Received: from shymkent.ilbers.de (shymkent.ilbers.de. [85.214.156.166]) by gmr-mx.google.com with ESMTPS id 5b1f17b1804b1-4247194de3bsi6211355e9.0.2024.06.21.08.08.27 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 21 Jun 2024 08:08:27 -0700 (PDT) Received-SPF: pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) client-ip=85.214.156.166; Received: from user-B660.promwad.corp ([159.148.83.114]) (authenticated bits=0) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPSA id 45LF8ObF006158 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 21 Jun 2024 17:08:26 +0200 From: Anton Mikanovich To: isar-users@googlegroups.com Cc: Anton Mikanovich Subject: [PATCH v3 4/5] CI: Allow external usage for some APIs Date: Fri, 21 Jun 2024 18:08:13 +0300 Message-Id: <20240621150814.189288-5-amikan@ilbers.de> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240621150814.189288-1-amikan@ilbers.de> References: <20240621150814.189288-1-amikan@ilbers.de> MIME-Version: 1.0 X-Spam-Status: No, score=-4.6 required=5.0 tests=DKIMWL_WL_MED,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_EF,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,RCVD_IN_MSPIKE_H2,RCVD_IN_RP_CERTIFIED, RCVD_IN_RP_RNBL,RCVD_IN_RP_SAFE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on shymkent.ilbers.de X-Original-Sender: amikan@ilbers.de X-Original-Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=amikan@ilbers.de Precedence: list Mailing-list: list isar-users@googlegroups.com; contact isar-users+owners@googlegroups.com List-ID: X-Spam-Checked-In-Group: isar-users@googlegroups.com X-Google-Group-Id: 914930254986 List-Post: , List-Help: , List-Archive: , List-Unsubscribe: , X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= Provide utilities class for external usage to be imported from outside of Avocado framework. Signed-off-by: Anton Mikanovich --- testsuite/cibase.py | 8 +++-- testsuite/cibuilder.py | 52 ++++----------------------------- testsuite/citest.py | 4 ++- testsuite/repro-build-test.py | 4 ++- testsuite/utils.py | 55 +++++++++++++++++++++++++++++++++++ 5 files changed, 72 insertions(+), 51 deletions(-) create mode 100755 testsuite/utils.py diff --git a/testsuite/cibase.py b/testsuite/cibase.py index 0f6997af..fd571c29 100755 --- a/testsuite/cibase.py +++ b/testsuite/cibase.py @@ -8,6 +8,8 @@ import tempfile import time from cibuilder import CIBuilder, isar_root +from utils import CIUtils + from avocado.utils import process class CIBaseTest(CIBuilder): @@ -238,13 +240,13 @@ class CIBaseTest(CIBuilder): for target in targets: sfiles[target] = dict() package = target.rsplit(':', 1)[-1] - isar_apt = self.getVars('REPO_ISAR_DB_DIR', target=target) + isar_apt = CIUtils.getVars('REPO_ISAR_DB_DIR', target=target) fpath = f'{package}/{package}*.tar.gz' targz = set(glob.glob(f'{isar_apt}/../apt/*/pool/*/*/{fpath}')) if len(targz) < 1: self.fail('No source packages found') - for filename in targz: - sfiles[target][filename] = self.get_tar_content(filename) + for fname in targz: + sfiles[target][fname] = CIUtils.get_tar_content(fname) return sfiles self.configure(**kwargs) diff --git a/testsuite/cibuilder.py b/testsuite/cibuilder.py index 87d71387..a20e88f9 100755 --- a/testsuite/cibuilder.py +++ b/testsuite/cibuilder.py @@ -9,11 +9,11 @@ import shutil import signal import subprocess import sys -import tarfile import time import tempfile import start_vm +from utils import CIUtils from avocado import Test from avocado.utils import path @@ -22,7 +22,6 @@ from avocado.utils import process sys.path.insert(0, os.path.dirname(os.path.realpath(__file__)) + '/../bitbake/lib') import bb -import bb.tinfoil DEF_VM_TO_SEC = 600 @@ -253,38 +252,6 @@ class CIBuilder(Test): except FileNotFoundError: self.log.warn(path + backup_prefix + ' not exist') - def getVars(self, *vars, target=None): - self.check_init() - def fixStream(stream): - # fix stream objects to emulate _io.TextIOWrapper - stream.isatty = lambda: False - stream.fileno = lambda: False - stream.encoding = sys.getdefaultencoding() - - sl = target is not None - fixStream(sys.stdout) - fixStream(sys.stderr) - - # wait until previous bitbake will be finished - lockfile = os.path.join(self.build_dir, 'bitbake.lock') - checks = 0 - while os.path.exists(lockfile) and checks < 5: - time.sleep(1) - checks += 1 - - with bb.tinfoil.Tinfoil(setup_logging=sl) as tinfoil: - values = () - if target: - tinfoil.prepare(quiet=2) - d = tinfoil.parse_recipe(target) - for var in vars: - values = values + (d.getVar(var, True) or 'None',) - else: - tinfoil.prepare(config_only=True, quiet=2) - for var in vars: - values = values + (tinfoil.config_data.getVar(var, True) or 'None',) - return values if len(values) > 1 else values[0] - def create_tmp_layer(self): tmp_layer_dir = os.path.join(isar_root, 'meta-tmp') @@ -314,13 +281,6 @@ BBPATH .= ":${LAYERDIR}"\ bb.utils.edit_bblayers_conf(bblayersconf_file, None, tmp_layer_dir) bb.utils.prunedir(tmp_layer_dir) - def get_tar_content(self, filename): - try: - tar = tarfile.open(filename) - return tar.getnames() - except Exception: - return [] - def get_ssh_cmd_prefix(self, user, host, port, priv_key): cmd_prefix = 'ssh -o ConnectTimeout=5 -o StrictHostKeyChecking=no '\ '-p %s -o IdentityFile=%s %s@%s ' \ @@ -525,11 +485,11 @@ BBPATH .= ":${LAYERDIR}"\ module_output = b'Just an example' resize_output = None image_fstypes, \ - wks_file, \ - bbdistro = self.getVars('IMAGE_FSTYPES', \ - 'WKS_FILE', \ - 'DISTRO', \ - target=multiconfig) + wks_file, \ + bbdistro = CIUtils.getVars('IMAGE_FSTYPES', + 'WKS_FILE', + 'DISTRO', + target=multiconfig) # only the first type will be tested in start_vm if image_fstypes.split()[0] == 'wic': diff --git a/testsuite/citest.py b/testsuite/citest.py index 42d44f6a..b84ae0e1 100755 --- a/testsuite/citest.py +++ b/testsuite/citest.py @@ -5,6 +5,7 @@ import os from avocado import skipUnless from avocado.utils import path from cibase import CIBaseTest +from utils import CIUtils UMOCI_AVAILABLE = True SKOPEO_AVAILABLE = True @@ -17,6 +18,7 @@ try: except path.CmdNotFoundError: SKOPEO_AVAILABLE = False + class DevTest(CIBaseTest): """ @@ -46,7 +48,7 @@ class DevTest(CIBaseTest): def test_dev_rebuild(self): self.init() - layerdir_core = self.getVars('LAYERDIR_core') + layerdir_core = CIUtils.getVars('LAYERDIR_core') dpkgbase_file = layerdir_core + '/classes/dpkg-base.bbclass' diff --git a/testsuite/repro-build-test.py b/testsuite/repro-build-test.py index 843f3bc6..04e4ddc7 100755 --- a/testsuite/repro-build-test.py +++ b/testsuite/repro-build-test.py @@ -1,6 +1,7 @@ #!/usr/bin/env python3 from cibuilder import CIBuilder +from utils import CIUtils from avocado.utils import process @@ -29,7 +30,8 @@ class ReproBuild(CIBuilder): def get_image_path(self, target_name): image_dir = "tmp/deploy/images" - machine, image_name = self.getVars("MACHINE", "IMAGE_FULLNAME", target=target_name) + machine, image_name = CIUtils.getVars('MACHINE', 'IMAGE_FULLNAME', + target=target_name) return f"{image_dir}/{machine}/{image_name}.tar.gz" def build_repro_image( diff --git a/testsuite/utils.py b/testsuite/utils.py new file mode 100755 index 00000000..ffcb5cc7 --- /dev/null +++ b/testsuite/utils.py @@ -0,0 +1,55 @@ +#!/usr/bin/env python3 + +import os +import sys +import tarfile +import time + +sys.path.append(os.path.join(os.path.dirname(__file__), '../bitbake/lib')) + +import bb +import bb.tinfoil + + +class CIUtils(): + @staticmethod + def getVars(*vars, target=None): + def fixStream(stream): + # fix stream objects to emulate _io.TextIOWrapper + stream.isatty = lambda: False + stream.fileno = lambda: False + stream.encoding = sys.getdefaultencoding() + + sl = target is not None + if not hasattr(sys.stdout, 'isatty'): + fixStream(sys.stdout) + if not hasattr(sys.stderr, 'isatty'): + fixStream(sys.stderr) + + # wait until previous bitbake will be finished + lockfile = os.path.join(os.getcwd(), 'bitbake.lock') + checks = 0 + while os.path.exists(lockfile) and checks < 5: + time.sleep(1) + checks += 1 + + with bb.tinfoil.Tinfoil(setup_logging=sl) as tinfoil: + values = () + if target: + tinfoil.prepare(quiet=2) + d = tinfoil.parse_recipe(target) + for var in vars: + values = values + (d.getVar(var, True) or '',) + else: + tinfoil.prepare(config_only=True, quiet=2) + for var in vars: + values = values + (tinfoil.config_data.getVar(var) or '',) + return values if len(values) > 1 else values[0] + + @staticmethod + def get_tar_content(filename): + try: + tar = tarfile.open(filename) + return tar.getnames() + except Exception: + return [] From patchwork Fri Jun 21 15:08:14 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anton Mikanovich X-Patchwork-Id: 3629 Return-Path: Received: from shymkent.ilbers.de ([unix socket]) by shymkent (Cyrus 2.5.10-Debian-2.5.10-3+deb9u2) with LMTPA; Fri, 21 Jun 2024 17:08:38 +0200 X-Sieve: CMU Sieve 2.4 Received: from mail-lf1-f60.google.com (mail-lf1-f60.google.com [209.85.167.60]) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPS id 45LF8bFU006232 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Fri, 21 Jun 2024 17:08:38 +0200 Received: by mail-lf1-f60.google.com with SMTP id 2adb3069b0e04-52c97141df8sf1544509e87.3 for ; Fri, 21 Jun 2024 08:08:38 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1718982510; cv=pass; d=google.com; s=arc-20160816; b=Q8+QNGLVW+PmRbRjJGk1dx1Hn5gddaiU/IMeMzeMIvjMqlxqq4EYZ3jq9iViOQmg5J RzHn6R7ra8gqpW1LEkmKzKn/JaH7897CoxjFJIbjOTmo19Kn4Gj6jusgPiC/znI0UDxV iHsrnL3Qgq15ARZzpC4e9cxja4a9nXIE03kfrTNPejGTVwF3OZLWz9XyCSsbpFnOejs5 d2bIdeQP1jYRYJ9QMWl16QErpd13H33w24JnB/r1Fh/4YojUpbuPGDUVsY41DXrf8vKT Wybk0V6e0penRgFZa4lRV/zIZ0VMrT7Fq3/9G9V2ahCUAJGFAqMWDRKUlxTYl+goqe4X OMFw== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:dkim-signature; bh=6ExFoqc9ZZAohyVLdMAWVpnWG+Xl15NZ180aj5ZdeR8=; fh=O0oa2/3SVcYItKgUHYoLMmdS9Tu6a+j6RHOnGYxKf9E=; b=QT5bHCNjVDHTgaNw7iT2kdU0OqUE418uqIE17LmCaEIBmE/KtONWl7PW/3o3jAdvuF zDo7SDq/APg+Ri8VlrNVtNABrSaFF2/VTD+jtycxbNyP/36Trd/WX1A6c25SvgdUvTAM WRPF6/+gml4YD7Ggq0I1ESgyW6OoTdKiNrD9JwU+r3G6LASqNXT64shGYFuuBRY/HssU 54WJE+8CXFGCm2KtcNIjsZReWY0aENMHWrOtkZ5LXn/uzdcemSz3D1JQZEa5/BABMNu2 7UwcKOfPbPzjBgEjBNjKPPgHuyiudjAp6Jw9sg8n9qenp0PYRJ0jU4ZxnvzatHEwIhRX pbsw==; darn=isar-build.org ARC-Authentication-Results: i=2; gmr-mx.google.com; spf=pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=amikan@ilbers.de DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlegroups.com; s=20230601; t=1718982510; x=1719587310; darn=isar-build.org; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:x-original-authentication-results :x-original-sender:mime-version:references:in-reply-to:message-id :date:subject:cc:to:from:sender:from:to:cc:subject:date:message-id :reply-to; bh=6ExFoqc9ZZAohyVLdMAWVpnWG+Xl15NZ180aj5ZdeR8=; b=ukjIcql6+5CChiZm0z1q0EUleLxEslFqF/cXl6tFnMWsLkuTJmF5LqQuKHg/4WUdbV 4MiXvk7Hi1w9L/b/LQtrat0jgAWZCtqOdseq+7zSrs6DsCAdAGo6nhk1TzGMrXfww7va 09vJAWvc+2ppXHHji88mi3nAXsE8kTGlh3Xpfro5DtN0kcjsua/4/ALWpWLq9ZQ6bxk+ ppHuoyTprEJfsbCqFr6YDle+xqzkcLdHa18U9oCpr4bBfe54VprM7M3fk7cHD3QhFIt4 EZ+IeIlsZs2ucpofEF2cB7YE8BSt1HosRVXOLVTS9u9CQ++wrUvyn1S2qVnAfvG6du+T UtAQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1718982510; x=1719587310; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :x-spam-checked-in-group:list-id:mailing-list:precedence :x-original-authentication-results:x-original-sender:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :x-beenthere:x-gm-message-state:sender:from:to:cc:subject:date :message-id:reply-to; bh=6ExFoqc9ZZAohyVLdMAWVpnWG+Xl15NZ180aj5ZdeR8=; b=eMK4PdO6AhyJeZAIRDSz7C5rJkEtYFWbJ0Kn+xLFrD02z2w8vNLfU114Sr3k/aCIlM rDROlwVdHmwWbuuu617lg4a8HCUd5YSHZQRWi8XFhHN3vuBMtVbx6fETpLFvDFZXE7bh ZTEGJtVk9mfsYxDkVbhG8zXKDnPkDfpJr5cH16UQCSeY0ZoJmpyp6ZgE/FNEW6DUm0OV pUMRwDJ2YVjvYh/BZYSZYWBkjIBhgShQbL/6IdtLNdL97CcQGjE8dZo5Hbm5BN2v6E5H gQsyKlEj7K5W+iKTbtaV97ZGDT7ghMI8hBzJVbhE+F7YYwSUu45cvfLjmWR6eE4ZSbQP Aozw== Sender: isar-users@googlegroups.com X-Forwarded-Encrypted: i=2; AJvYcCVXl33XLA9+ITbfrkyff8Hxwjd4OgqNFpu8R+vu9zDCKzotMUYAFehUB3na84Bu67n9jGeWger3apgrJboAzieQbXj98bM= X-Gm-Message-State: AOJu0YxbvhuglsyvnEh6+EUmPc5ECYUfLlx5yuU1Lw9iQglVCAhodjWu MAwuv4lgQp3pXH67jpou9brZh5N10yZKD5ibZP9EZUy28Dcl2PpU X-Google-Smtp-Source: AGHT+IH0HslV2jcAZsFeJ+irTyCpTmKIFP39IJBzLMaG0tI1V8+QJAuwB0L1DT8s25KFavhYpMDeMA== X-Received: by 2002:ac2:5ecb:0:b0:52c:9383:4c16 with SMTP id 2adb3069b0e04-52ccaa5d4b2mr5090809e87.22.1718982510400; Fri, 21 Jun 2024 08:08:30 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com Received: by 2002:ac2:53bb:0:b0:52c:d248:3eba with SMTP id 2adb3069b0e04-52cd2483edbls584573e87.2.-pod-prod-07-eu; Fri, 21 Jun 2024 08:08:28 -0700 (PDT) X-Received: by 2002:ac2:5dc8:0:b0:52c:dc57:868b with SMTP id 2adb3069b0e04-52cdc57871emr276820e87.13.1718982507992; Fri, 21 Jun 2024 08:08:27 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1718982507; cv=none; d=google.com; s=arc-20160816; b=i2Rq6lk4Gb+ZJTDfNKYiZGt0lju3+849V7NwhTLG2RbHxrKf0srN3u1N8jes3YMuKj y86nnllKI3J3/83SCkO7n697mgwlsMC/g7i3KZSmPKJ/z/Fgo/+hoXtYSNUrMEzabyq8 lQy+CY7Xvt1hKnawAscyCoZF+WXaqmK/QXAPJWpJFdFZsQ6xGq6XYFXvaU8dByxQ0CdK z3UJLFYbkX1g+4PBsWZAcPXZkPg1BAOMPvUWg9eAOCuxmi4M2aIazlg/BAXOQMy7QK4o BbwpOLwSujoZG2XaDCxvXVtkWEi724sIqC/MuDZSF1LTnvp1VcvfpW0NoHS/zSUlPlcR 7ZDA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=ok1UBy81qlkPE8j8qElbuQiJiBsAP0Zkl6cdH7pTMbU=; fh=/h9QQkzJ8EboVkWg45aWwpaUro6WMavIVd2OhN45RtE=; b=jZZhImWUEoJKA72uCWYlyTdR5PbbYWqnnbZwvmCqUhj3btAs/nvseje14tNuKqmM5C OCG+RfU4kB/jIQQF+jQOppSYmwGaZuAKdsU+cCCgg5fm+thb/+5ESrA4yY9OMG7ecEvt MB0Yp23gFrP9cmS9z4paTZRtKOtLPRDbOOP8/uzOjUA3PJ1EjXSgZMS4d9K83x5j/TS1 ANkPMUDRb+r63EEVa+elSPvor9WD370q0QqLVHfGh1Ub6a1wMWfF+sBaWLOhi+WY+6dJ o/+dJ04GZuFGNI3p8xmtKS+e8NP7QhMggU+r5pmDaUd+Bu63tevyR5hJxKXRbAIAQ4Rz rzGg==; dara=google.com ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=amikan@ilbers.de Received: from shymkent.ilbers.de (shymkent.ilbers.de. [85.214.156.166]) by gmr-mx.google.com with ESMTPS id 5b1f17b1804b1-42478c9bbdesi1253295e9.1.2024.06.21.08.08.27 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 21 Jun 2024 08:08:27 -0700 (PDT) Received-SPF: pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) client-ip=85.214.156.166; Received: from user-B660.promwad.corp ([159.148.83.114]) (authenticated bits=0) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPSA id 45LF8ObG006158 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 21 Jun 2024 17:08:27 +0200 From: Anton Mikanovich To: isar-users@googlegroups.com Cc: Anton Mikanovich Subject: [PATCH v3 5/5] start_vm: Reuse getVars API Date: Fri, 21 Jun 2024 18:08:14 +0300 Message-Id: <20240621150814.189288-6-amikan@ilbers.de> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240621150814.189288-1-amikan@ilbers.de> References: <20240621150814.189288-1-amikan@ilbers.de> MIME-Version: 1.0 X-Spam-Status: No, score=-4.6 required=5.0 tests=DKIMWL_WL_MED,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_EF,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,RCVD_IN_MSPIKE_H2,RCVD_IN_RP_CERTIFIED, RCVD_IN_RP_RNBL,RCVD_IN_RP_SAFE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on shymkent.ilbers.de X-Original-Sender: amikan@ilbers.de X-Original-Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=amikan@ilbers.de Precedence: list Mailing-list: list isar-users@googlegroups.com; contact isar-users+owners@googlegroups.com List-ID: X-Spam-Checked-In-Group: isar-users@googlegroups.com X-Google-Group-Id: 914930254986 List-Post: , List-Help: , List-Archive: , List-Unsubscribe: , X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= We already have an API to get Bitbake variable values without manual handling of bitbake execution. This will speedup und simplify start_vm. Signed-off-by: Anton Mikanovich --- testsuite/start_vm.py | 50 ++++++++++++++++++++++--------------------- 1 file changed, 26 insertions(+), 24 deletions(-) diff --git a/testsuite/start_vm.py b/testsuite/start_vm.py index 0e0d7000..d6e04049 100755 --- a/testsuite/start_vm.py +++ b/testsuite/start_vm.py @@ -11,43 +11,50 @@ import sys import shutil import time -OVMF_VARS_PATH = '/usr/share/OVMF/OVMF_VARS_4M.ms.fd' - -def get_bitbake_env(arch, distro, image): - multiconfig = 'mc:qemu' + arch + '-' + distro + ':' + image - output = subprocess.check_output(['bitbake', '-e', str(multiconfig)]) - return output +from utils import CIUtils -def get_bitbake_var(output, var): - ret = '' - for line in output.splitlines(): - if line.startswith(var + '='): - ret = line.split('"')[1] - return ret +OVMF_VARS_PATH = '/usr/share/OVMF/OVMF_VARS_4M.ms.fd' def format_qemu_cmdline(arch, build, distro, image, out, pid, enforce_pcbios=False): - bb_output = get_bitbake_env(arch, distro, image).decode() + multiconfig = f'mc:qemu{arch}-{distro}:{image}' + + image_fstypes, \ + deploy_dir_image, \ + kernel_image, \ + initrd_image, \ + serial, \ + root_dev, \ + qemu_arch, \ + qemu_machine, \ + qemu_cpu, \ + qemu_disk_args = CIUtils.getVars('IMAGE_FSTYPES', + 'DEPLOY_DIR_IMAGE', + 'KERNEL_IMAGE', + 'INITRD_DEPLOY_FILE', + 'MACHINE_SERIAL', + 'QEMU_ROOTFS_DEV', + 'QEMU_ARCH', + 'QEMU_MACHINE', + 'QEMU_CPU', + 'QEMU_DISK_ARGS', + target=multiconfig) extra_args = '' cpu = [''] - image_type = get_bitbake_var(bb_output, 'IMAGE_FSTYPES').split()[0] - deploy_dir_image = get_bitbake_var(bb_output, 'DEPLOY_DIR_IMAGE') + image_type = image_fstypes.split()[0] base = 'ubuntu' if distro in ['jammy', 'focal'] else 'debian' rootfs_image = image + '-' + base + '-' + distro + '-qemu' + arch + '.' + image_type if image_type == 'ext4': - kernel_image = deploy_dir_image + '/' + get_bitbake_var(bb_output, 'KERNEL_IMAGE') - initrd_image = get_bitbake_var(bb_output, 'INITRD_DEPLOY_FILE') + kernel_image = deploy_dir_image + '/' + kernel_image if not initrd_image: initrd_image = '/dev/null' else: initrd_image = deploy_dir_image + '/' + initrd_image - serial = get_bitbake_var(bb_output, 'MACHINE_SERIAL') - root_dev = get_bitbake_var(bb_output, 'QEMU_ROOTFS_DEV') kargs = ['-append', '"console=' + serial + ' root=/dev/' + root_dev + ' rw"'] extra_args = ['-kernel', kernel_image, '-initrd', initrd_image] @@ -57,11 +64,6 @@ def format_qemu_cmdline(arch, build, distro, image, out, pid, enforce_pcbios=Fal else: raise ValueError('Invalid image type: ' + str(image_type)) - qemu_arch = get_bitbake_var(bb_output, 'QEMU_ARCH') - qemu_machine = get_bitbake_var(bb_output, 'QEMU_MACHINE') - qemu_cpu = get_bitbake_var(bb_output, 'QEMU_CPU') - qemu_disk_args = get_bitbake_var(bb_output, 'QEMU_DISK_ARGS') - if out: extra_args.extend(['-chardev','stdio,id=ch0,logfile=' + out]) extra_args.extend(['-serial','chardev:ch0'])