From patchwork Mon May 12 08:32:37 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anton Mikanovich X-Patchwork-Id: 4207 Return-Path: Received: from shymkent.ilbers.de ([unix socket]) by shymkent (Cyrus 2.5.10-Debian-2.5.10-3+deb9u2) with LMTPA; Mon, 12 May 2025 10:32:58 +0200 X-Sieve: CMU Sieve 2.4 Received: from mail-wm1-f64.google.com (mail-wm1-f64.google.com [209.85.128.64]) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPS id 54C8WvUQ018202 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Mon, 12 May 2025 10:32:57 +0200 Received: by mail-wm1-f64.google.com with SMTP id 5b1f17b1804b1-43f405810b4sf20545935e9.1 for ; Mon, 12 May 2025 01:32:57 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1747038771; cv=pass; d=google.com; s=arc-20240605; b=N27jAXsZw7X6w1baazzua1NVJcSU7/ZbmWPbxIhVmS4yB4iVQdknIBjHUkiLl8u5X/ AADcPiU96ehfR0Fbna+kioO3SgSkdsubTu+zd4Gw5dakzRLw3B/C4qoy7VE0NpqSj5Ey RQW6wX2l4FlVEonP9YHx2ON/9W3ynzsN+GOUtCbEnd+7w1+gnJLjUyU+kAh9Yw+qzLgG l17ZsWId5kGmCDkGaEeR8ncTLHX4CfjvHcUz+fxOk939FHdhNaDSnForoBEKt5N22+7X 92TQO45xnCFAQtYCf24Am/7Q09fKHS+0RqGPO9IhX3IRRZFNraUqbIU33bhTHrnFX1aS fQxA== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20240605; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:dkim-signature; bh=/AUQlwz9dnhxx1sErww4/tLteQK3noEAT+rYtuAorwg=; fh=T3oP0egzZ+Vzi9SmbKq2mVOqeydutNlnqUm0HIFU2So=; b=g6Irm3h4/ALmddsCLbV3jyNlJSA6ew6y7DhYcCwkWX72F80Ex4Spug9gcQfiFVNOMs x3vQD9yBZGRmLU1t8CRoaNUeZ0BLxHHrMzDyegUzjgwm5RkV9I94Yz+ASwZQsi6Ffc/m OMq+OemnjrDAPvFiUkGR0xsrciqxKtKHDGoV51D9jyDEaGXZAtOaz81zQABZ4ZcsUzei z++9aGT5Gx+AslUmgPDqbJ1p4SgJIIq2dfn3DC//lrM1gAF6C1Cia9UbzkU2DxKVquzD 4hFpU6K831Yi8dkOHAZ4t9R2i7CPoh9Vv/8w6oDhDvljbV+JfKg4RE5wlLFClQ1qOPKa JKMA==; darn=isar-build.org ARC-Authentication-Results: i=2; gmr-mx.google.com; spf=pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=amikan@ilbers.de DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlegroups.com; s=20230601; t=1747038771; x=1747643571; darn=isar-build.org; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:x-original-authentication-results :x-original-sender:mime-version:references:in-reply-to:message-id :date:subject:cc:to:from:sender:from:to:cc:subject:date:message-id :reply-to; bh=/AUQlwz9dnhxx1sErww4/tLteQK3noEAT+rYtuAorwg=; b=MtPeXjGS4O7OHV5ERAl1lZuNU4ki12Nax/ZoUdHLLFlYDIgslZxP94eV7whv/UmbhG SzZNdCPSJupKVsnT0dNkwESs/7lOdC/hTrAkRQExYg1/W8Js5Fb8YT/5bAgrtLFOS3MF 1yOpD6HCOTK7bxgcK0w2ZrSZ5CTLnms9xk2mjlRRPm69vA1mJI+W9rFAs1aWNKQMD3qF TGEnEExTfmn2Y90pDdfcuCncMXx1TnKxGdMaoIZa7rvPA22x76mYA9My3jLhXr9vLtcn b0Z36SwqTb+KPXtzRj3nFugxnpoiR65OLe8i3ThJNGL1Ti5jhtV7f37jjz5SbFtG4j1H 25Xw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1747038771; x=1747643571; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :x-spam-checked-in-group:list-id:mailing-list:precedence :x-original-authentication-results:x-original-sender:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :x-beenthere:x-gm-message-state:sender:from:to:cc:subject:date :message-id:reply-to; bh=/AUQlwz9dnhxx1sErww4/tLteQK3noEAT+rYtuAorwg=; b=lWbG4fsJI7nOza4NiR/iwdaaDlJiXmgtnrpfvvZEjhlG4RsgINdlbiejVSfBVvwar5 moADKp/U1cZVRusiku5ECMN+tkRKvUs/KU7qKJfbPOH04JQMikTG2rtbcItNNABdtWyx fI82KVSESfixBZgU3KHFxiX++7IvWHkJOmH0fsqY0ApW4IrwsdGa+Py4XkYjeuu/mbGc oFf9qQ2Oqry4AUmZVVBE5vBvPHIVbdxW7J90XFgDSTssp1fuoEi5HxsHd9GXknaXisXS lRD30TuGmhsjzY/MDItodaGzVjgws6iujusrgm7G9yqX5Szy68Uu6UeeUH2Rabw+kw98 b8ow== Sender: isar-users@googlegroups.com X-Forwarded-Encrypted: i=2; AJvYcCXsi7TnhRPtd6CUw2H4bM4GKbO4sqkbVII7WAjSSZQlb4BPlOMeMvsJFIyh++t3lWUAYXlq1mI=@isar-build.org X-Gm-Message-State: AOJu0Yw8D856wR98pYkWcrLiCvqTX3GOZE53SstkJKGO/9nK+laoh1Ny iHjMhzzNfiuYI8quL+npvOLprbiTWuUMNph36pp9nd1iMtPxJK08 X-Google-Smtp-Source: AGHT+IFRdDFQ6+EeHr+Lqpf5rdoTIgp2zqOrkY/VYh7fsDAkZwm65UJ4iaFSMy8shhYfnuguHHL1Mg== X-Received: by 2002:a05:600c:4694:b0:440:94a2:95b8 with SMTP id 5b1f17b1804b1-442d6d6b65fmr108789675e9.16.1747038770317; Mon, 12 May 2025 01:32:50 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com; h=AVT/gBHHNSeREIZNnI3loZKBW/5f4IWbc/5KJbQzfQABgkD2+Q== Received: by 2002:a05:600c:1e09:b0:43b:cfb8:a5d7 with SMTP id 5b1f17b1804b1-442d0200361ls8769445e9.0.-pod-prod-06-eu; Mon, 12 May 2025 01:32:48 -0700 (PDT) X-Received: by 2002:a05:600c:1550:b0:43c:f6c6:578c with SMTP id 5b1f17b1804b1-442d6d6b5f6mr111275605e9.15.1747038768217; Mon, 12 May 2025 01:32:48 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1747038768; cv=none; d=google.com; s=arc-20240605; b=HGpqpmCDTQrkEDPVQsE3evpSloNTwuhe8kAyKlAw7crYWLCb9DaVs0EXpmUZ4TKrmM liTWZsSzT/7lNZu5mICq8SbVuWASy8shQdKJPJyyZ4NS54IkCJO4R18RzFpK+T7aAP3Q vx4PFBvz9ToOKbZQfcy472Dau3m7p5XzQvGHq+xHeNof/54MrjzQ0UKVgPWsu7I8C5TO zJYr2X48L5kxpC79r9Z7T9T1eitUu+AQtWl6/Jp5csCXnJGXfyN7454XJWqGjk/6Qe83 lk0FTwXbDwRN0a+wa2nErJhDR76SU2glQMIJSbtde5Yj/csEwolSdeiKRT1Fxtnn6ZYN MTeg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20240605; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=0UzBLveZlDM4pfeYnpal8s55z7yXTjC9O02HdCeIFDc=; fh=/h9QQkzJ8EboVkWg45aWwpaUro6WMavIVd2OhN45RtE=; b=Pp2UfNlVGOjT9y72crytjhH1Zmme61nm4XBtm6eAVSCCGoBlFVAyxWLtWRDPR4Tb7q tZKcee6bCZOs1I6F6GP8V7sWrj8xskM8W8fDC5ilv+rkwwoLfBZfNw0Q7bdqpyrLJ2PD HoLTg6HyxWe6hu8FSLJSjjtTo98LBALok6EoPX4NmJ80PI0NcTcHcsqxahyRphfgIYPq syOZ7LBejbSM4JyIC3xNiPRXjKbx3/qT2N8rm6j69IGCaxofEl4WVLjOXQ17+5A3/B5r PiKW4eeGuT602hf6rF1yAEyB988N1txU6cFupixftQONCcGvzrTLVDc/MiwtuX0/H4N0 tJnw==; dara=google.com ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=amikan@ilbers.de Received: from shymkent.ilbers.de (shymkent.ilbers.de. [85.214.156.166]) by gmr-mx.google.com with ESMTPS id 5b1f17b1804b1-442cd34bcc8si1866295e9.1.2025.05.12.01.32.48 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Mon, 12 May 2025 01:32:48 -0700 (PDT) Received-SPF: pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) client-ip=85.214.156.166; Received: from user-B660.promwad.corp ([159.148.83.123]) (authenticated bits=0) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPSA id 54C8Wjvq018145 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 12 May 2025 10:32:47 +0200 From: Anton Mikanovich To: isar-users@googlegroups.com Cc: Anton Mikanovich Subject: [PATCH 2/3] CI: Add make world test case Date: Mon, 12 May 2025 11:32:37 +0300 Message-Id: <20250512083238.2089774-3-amikan@ilbers.de> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20250512083238.2089774-1-amikan@ilbers.de> References: <20250512083238.2089774-1-amikan@ilbers.de> MIME-Version: 1.0 X-Spam-Status: No, score=-4.6 required=5.0 tests=DKIMWL_WL_MED,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_EF,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,RCVD_IN_DNSWL_BLOCKED,RCVD_IN_MSPIKE_H2, RCVD_IN_RP_CERTIFIED,RCVD_IN_RP_RNBL,RCVD_IN_RP_SAFE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on shymkent.ilbers.de X-Original-Sender: amikan@ilbers.de X-Original-Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=amikan@ilbers.de Precedence: list Mailing-list: list isar-users@googlegroups.com; contact isar-users+owners@googlegroups.com List-ID: X-Spam-Checked-In-Group: isar-users@googlegroups.com X-Google-Group-Id: 914930254986 List-Post: , List-Help: , List-Archive: , List-Unsubscribe: , X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= Add test case which builds all available targets. Single bitbake usage (directly from mc.conf): $ avocado run testsuite/citest.py -t world Multiple bitbakes usage (from generated yaml file): $ avocado run testsuite/citest.py -t world -m testsuite/data/targets.yml Or the same with ci_build.sh: There is also a script to regenerate yaml from mc.conf: $ ./testsuite/utils/targets_gen.py This test case requires avocado-framework-plugin-varianter-yaml-to-mux. Signed-off-by: Anton Mikanovich --- .gitignore | 2 +- scripts/ci_build.sh | 8 ++- testsuite/citest.py | 33 ++++++++++ testsuite/data/targets.yml | 132 +++++++++++++++++++++++++++++++++++++ testsuite/targets_gen.py | 13 ++++ testsuite/utils.py | 35 +++++++++- 6 files changed, 220 insertions(+), 3 deletions(-) create mode 100644 testsuite/data/targets.yml create mode 100755 testsuite/targets_gen.py diff --git a/.gitignore b/.gitignore index 81c09fc1..fff155f1 100644 --- a/.gitignore +++ b/.gitignore @@ -3,4 +3,4 @@ __pycache__ .config.yaml* .kas_shell_history -build/ +build* diff --git a/scripts/ci_build.sh b/scripts/ci_build.sh index 338557ad..690dadd4 100755 --- a/scripts/ci_build.sh +++ b/scripts/ci_build.sh @@ -23,6 +23,7 @@ if ! command -v avocado > /dev/null; then # shellcheck source=/dev/null source /tmp/avocado_venv/bin/activate pip install avocado-framework==103.0 + pip install avocado-framework-plugin-varianter-yaml-to-mux==103.0 fi # Get Avocado build tests path @@ -49,6 +50,7 @@ show_help() { echo " -b, --base BASE_DIR set path to base directory. If not set," echo " the tests will be started in current path." echo " -d, --debug enable debug bitbake output." + echo " -m, --muxfile input file for yaml-to-mux plugin." echo " -T, --tags specify basic avocado tags." echo " --help display this message and exit." echo @@ -94,6 +96,10 @@ do NORUN="1" echo "warning: deprecated parameter '$key', consider using '-T ,-startvm' instead" ;; + -m|--muxfile) + MUXFILE="-m $2" + shift + ;; -t|--timeout) TIMEOUT="-p time_to_wait=$2" shift @@ -149,7 +155,7 @@ set -x avocado ${VERBOSE} run "${TESTSUITE_DIR}/citest.py" \ -t "${TAGS}" --max-parallel-tasks=1 --disable-sysinfo \ - ${SSTATE} ${TIMEOUT} \ + ${SSTATE} ${MUXFILE} ${TIMEOUT} \ || ret=$? python3 ${TESTSUITE_DIR}/cleanup.py diff --git a/testsuite/citest.py b/testsuite/citest.py index 9f18a51f..501dbf37 100755 --- a/testsuite/citest.py +++ b/testsuite/citest.py @@ -1,4 +1,10 @@ #!/usr/bin/env python3 +# +# This software is a part of ISAR. +# Copyright (C) 2022-2025 ilbers GmbH +# Copyright (C) 2022-2025 Siemens AG +# +# SPDX-License-Identifier: MIT from avocado import skipUnless from avocado.core import exceptions @@ -734,3 +740,30 @@ class VmBootTestFull(CIBaseTest): self.init() self.vm_start('amd64-iso', 'bookworm', image='isar-image-ci', script='test_system_running.sh 30') + + +class World(CIBaseTest): + + """ + All targets build test + + :avocado: tags=world + """ + + def test_world(self): + name = self.params.get('name') + image = self.params.get('image', default='isar-image-ci') + targets = [] + + if name is None: + self.init() + for target in CIUtils.get_targets(): + for image in CIUtils.get_test_images(): + targets.append(f'mc:{target}:{image}') + else: + targets.append(f'mc:{name}:{image}') + self.init(f'build-{name}') + + self.perform_build_test( + targets, container=name and name.startswith('container') + ) diff --git a/testsuite/data/targets.yml b/testsuite/data/targets.yml new file mode 100644 index 00000000..5c9b5196 --- /dev/null +++ b/testsuite/data/targets.yml @@ -0,0 +1,132 @@ +a: !mux + qemuarm-buster: + name: qemuarm-buster + qemuarm-bullseye: + name: qemuarm-bullseye + qemuarm-bookworm: + name: qemuarm-bookworm + qemuarm-trixie: + name: qemuarm-trixie + qemuarm64-buster: + name: qemuarm64-buster + qemuarm64-bullseye: + name: qemuarm64-bullseye + qemuarm64-bookworm: + name: qemuarm64-bookworm + qemuarm64-trixie: + name: qemuarm64-trixie + qemui386-buster: + name: qemui386-buster + qemui386-bullseye: + name: qemui386-bullseye + qemui386-bookworm: + name: qemui386-bookworm + qemuamd64-buster: + name: qemuamd64-buster + qemuamd64-bullseye: + name: qemuamd64-bullseye + qemuamd64-sb-bullseye: + name: qemuamd64-sb-bullseye + qemuamd64-bookworm: + name: qemuamd64-bookworm + qemuamd64-cip-bookworm: + name: qemuamd64-cip-bookworm + qemuamd64-iso-bookworm: + name: qemuamd64-iso-bookworm + qemuamd64-trixie: + name: qemuamd64-trixie + container-amd64-buster: + name: container-amd64-buster + container-amd64-bullseye: + name: container-amd64-bullseye + container-amd64-bookworm: + name: container-amd64-bookworm + qemumipsel-bullseye: + name: qemumipsel-bullseye + qemumipsel-bookworm: + name: qemumipsel-bookworm + qemuriscv64-trixie: + name: qemuriscv64-trixie + bananapi-buster: + name: bananapi-buster + bananapi-bullseye: + name: bananapi-bullseye + bananapi-bookworm: + name: bananapi-bookworm + beagleplay-bookworm: + name: beagleplay-bookworm + de0-nano-soc-buster: + name: de0-nano-soc-buster + de0-nano-soc-bullseye: + name: de0-nano-soc-bullseye + de0-nano-soc-bookworm: + name: de0-nano-soc-bookworm + hikey-bullseye: + name: hikey-bullseye + hikey-bookworm: + name: hikey-bookworm + imx6-sabrelite-buster: + name: imx6-sabrelite-buster + imx6-sabrelite-bullseye: + name: imx6-sabrelite-bullseye + phyboard-mira-bullseye: + name: phyboard-mira-bullseye + phyboard-mira-bookworm: + name: phyboard-mira-bookworm + nanopi-neo-buster: + name: nanopi-neo-buster + nanopi-neo-bullseye: + name: nanopi-neo-bullseye + nanopi-neo-bookworm: + name: nanopi-neo-bookworm + nanopi-neo-efi-bookworm: + name: nanopi-neo-efi-bookworm + stm32mp15x-bullseye: + name: stm32mp15x-bullseye + virtualbox-bullseye: + name: virtualbox-bullseye + virtualbox-bookworm: + name: virtualbox-bookworm + x86-pc-bookworm: + name: x86-pc-bookworm + rpi-arm-bullseye: + name: rpi-arm-bullseye + rpi-arm-bookworm: + name: rpi-arm-bookworm + rpi-arm-v7-bullseye: + name: rpi-arm-v7-bullseye + rpi-arm-v7-bookworm: + name: rpi-arm-v7-bookworm + rpi-arm-v7l-bullseye: + name: rpi-arm-v7l-bullseye + rpi-arm-v7l-bookworm: + name: rpi-arm-v7l-bookworm + rpi-arm64-v8-bullseye: + name: rpi-arm64-v8-bullseye + rpi-arm64-v8-bookworm: + name: rpi-arm64-v8-bookworm + sifive-fu540-trixie: + name: sifive-fu540-trixie + starfive-visionfive2-trixie: + name: starfive-visionfive2-trixie + qemuarm64-focal: + name: qemuarm64-focal + qemuarm64-jammy: + name: qemuarm64-jammy + qemuarm64-noble: + name: qemuarm64-noble + qemuamd64-focal: + name: qemuamd64-focal + qemuamd64-jammy: + name: qemuamd64-jammy + qemuamd64-noble: + name: qemuamd64-noble + qemuamd64-sid: + name: qemuamd64-sid + qemuarm64-sid: + name: qemuarm64-sid +b: !mux + base: + image: isar-image-base + ci: + image: isar-image-ci diff --git a/testsuite/targets_gen.py b/testsuite/targets_gen.py new file mode 100755 index 00000000..41b4d644 --- /dev/null +++ b/testsuite/targets_gen.py @@ -0,0 +1,13 @@ +#!/usr/bin/env python3 + +""" +# This software is a part of Isar. +# Copyright (C) 2025 ilbers GmbH + +# targets_gen.py: Generates yaml for yaml-to-mux Avocado varianter plugin. +""" + +from utils import CIUtils + +if __name__ == "__main__": + CIUtils.gen_targets_yaml() diff --git a/testsuite/utils.py b/testsuite/utils.py index ffcb5cc7..9b18999b 100755 --- a/testsuite/utils.py +++ b/testsuite/utils.py @@ -1,4 +1,9 @@ #!/usr/bin/env python3 +# +# This software is a part of ISAR. +# Copyright (C) 2024-2025 ilbers GmbH +# +# SPDX-License-Identifier: MIT import os import sys @@ -10,8 +15,10 @@ sys.path.append(os.path.join(os.path.dirname(__file__), '../bitbake/lib')) import bb import bb.tinfoil +isar_root = os.path.abspath(os.path.join(os.path.dirname(__file__), '..')) -class CIUtils(): + +class CIUtils: @staticmethod def getVars(*vars, target=None): def fixStream(stream): @@ -53,3 +60,29 @@ class CIUtils(): return tar.getnames() except Exception: return [] + + @staticmethod + def get_test_images(): + return ['isar-image-base', 'isar-image-ci'] + + @staticmethod + def get_targets(): + d = bb.data.init() + d.setVar('BBPATH', os.path.join(isar_root, 'meta-isar')) + d = bb.cookerdata.parse_config_file('conf/mc.conf', d, False) + return d.getVar('BBMULTICONFIG').split() + + @staticmethod + def gen_targets_yaml(fn='targets.yml'): + targetsfile = os.path.join(os.path.dirname(__file__), 'data', fn) + with open(targetsfile, 'w') as f: + f.write('a: !mux\n') + for target in CIUtils.get_targets(): + f.write(f' {target}:\n name: {target}\n') + f.write('b: !mux\n') + prefix = 'isar-image-' + for image in CIUtils.get_test_images(): + nodename = image + if image.startswith(prefix): + nodename = image[len(prefix):] + f.write(f' {nodename}:\n image: {image}\n')