From patchwork Mon Jun 17 15:09:00 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anton Mikanovich X-Patchwork-Id: 3618 Return-Path: Received: from shymkent.ilbers.de ([unix socket]) by shymkent (Cyrus 2.5.10-Debian-2.5.10-3+deb9u2) with LMTPA; Mon, 17 Jun 2024 17:09:23 +0200 X-Sieve: CMU Sieve 2.4 Received: from mail-wr1-f55.google.com (mail-wr1-f55.google.com [209.85.221.55]) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPS id 45HF9MA4019153 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Mon, 17 Jun 2024 17:09:22 +0200 Received: by mail-wr1-f55.google.com with SMTP id ffacd0b85a97d-35f142464b9sf2641954f8f.0 for ; Mon, 17 Jun 2024 08:09:22 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1718636956; cv=pass; d=google.com; s=arc-20160816; b=xos+FAdO0jQZ+aA2HUCIFcm/VWAC97luhUSHwlf3KAoGBsNeIP1DULPL9l2wSnmILD y3manmqGjwNMd2OKyeE8akJRuvSLBrfLYUAFL+lRISLbh6/ZzyH7MKljQ6XGmi2i86ib /wJ5HDDvslgvdtdLCdmAFM/mEB9S1jmLIa84kQ79nDOH1YkbXqpC28hmIYjyd1q68q24 iX7ZemZQYeAIPfvYndWXpxzcq27M+GlrLl0umRVf0RfyuzQ4f5MEB3S2JiS/8jkCPWli oSS2Wwgod9Fb2zc+nWY1wEJbUHLPp4mHMYuzYiWqaZc8ZlmAtrZtun4iMdMBWkG42kqF nENg== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:dkim-signature; bh=XVD39J/VZuljEtgobw9FOihI40himGhca7t79laY2ms=; fh=5HWWLyi/Ef21V3n+0TUiJgJNs09iEE90Q0NWEQhpVmw=; b=1A4dAjpOSRpGZp/suiUZ5WCRJsrxOvj4pRln0PDXasl3b/ZlLSGGJFI/EzyfGiacAM cPNgad9KupFOjaekzFAxYuzWnfPMTffyllqXpMTP6l6s2tvdCo2+AZYt52yH9/oG2nFv sYky2Yk2xt2fQ0vPshn08wRF0imnXh9qE+d1Ra4wPyVwfaL5a+TB/GNU3qhZlx3b1WXP m078FJDpANOkN4TH2lepvUhDfeh1gs6j0CTUTAsMA4gqQ0UsQhzVdWa5n6qMCts1raRy bSssJvblQ4mRmUjrcHCrPbBu34uh4n3zyLlQ7H6s8p8FKEFsFT8VelJpnORauCYIUsYr eX2Q==; darn=isar-build.org ARC-Authentication-Results: i=2; gmr-mx.google.com; spf=pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=amikan@ilbers.de DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlegroups.com; s=20230601; t=1718636956; x=1719241756; darn=isar-build.org; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:x-original-authentication-results :x-original-sender:mime-version:references:in-reply-to:message-id :date:subject:cc:to:from:sender:from:to:cc:subject:date:message-id :reply-to; bh=XVD39J/VZuljEtgobw9FOihI40himGhca7t79laY2ms=; b=xbtrR9ogYnGrjV0XgKK7q7tqEGq6agmQkgLE7t+xOHLcznzHT1cTyi4i+6ON5z4BW9 V70Ph+B1IxzF5894jHN2eGZI0CBdYUsPqOoHP1h2hREKOkZ8/2bQPrz6WrHy/brgGDnF 4ONl8FmKonsvhC9LDVHE48RpvpxF4N5VOyoaEkJjBoZurm0bXymfqgRN+Rtmp7iw96c/ V1timXyv7p5sC5j7FbDqZjirrl2YP+uYkowmnoysqZfpHNrkcC9E7Kzi9f0OMjqGZPGT r8/flyoLGhfoyp0S5vcbTk+h7vr3jJ8zazIC42YL1GRBsBGNinrtj9XfqAZJYbsuvVh+ 93nA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1718636956; x=1719241756; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :x-spam-checked-in-group:list-id:mailing-list:precedence :x-original-authentication-results:x-original-sender:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :x-beenthere:x-gm-message-state:sender:from:to:cc:subject:date :message-id:reply-to; bh=XVD39J/VZuljEtgobw9FOihI40himGhca7t79laY2ms=; b=Bi7WMu3Q1GGOCFTPHwpGQPN9NCvqZ3M9AdZhlU7KRNOi0LdomoAs2mTUVcwAVZllML xN46WVBSzUi1fupPpdDLd+swZ7tNllvNEmhNvGKiL7lukNMgqWG/ImDbxXhJcTG/mcVV f8vVtA39EV87SSlMyVgguaqJXdabK+5mWECIgiCxLuVilFlDdTLtdGQxBGH41oDnL+Be LHkfMvq93BRxO86c+wlMeN/xl7lRRuVVWQQMfRTv4RMo00zCQCjTpTgMUu7yCNIwbqHa ZfK0KAYzQDI/JAHYAKAn86tJ4U4yJegOoeKw++pAO30M0TxRF4yxkSIX6C2ZAS+xhKi4 OJzg== Sender: isar-users@googlegroups.com X-Forwarded-Encrypted: i=2; AJvYcCUPReJT8kIt1CyEtQD/l9GBhqNyuTD5BaCvhITCyhl1OScUUP6AWtNxFQfh4Uf5Osz4xs24+rgLMdGUldzQO/H9HWpS1+4= X-Gm-Message-State: AOJu0Yz41J98Bji3py15nBvz5IrRQP0BkAZd2YlqZo56Gc/eettWN9tw 08iF/1kzMGDUcjlWyKV3xcj6Cuf+nbJf8ggLbTudc2WpmFUB1wV4 X-Google-Smtp-Source: AGHT+IHGOb6mzJGTzrFf1/Jm1Tmyu+KDQ1sDlL8ByKzhPFoNKy4TXFBUai3X4rcbBTjS/B1Pz0merw== X-Received: by 2002:adf:cc91:0:b0:360:8f8d:9e3 with SMTP id ffacd0b85a97d-3608f8d0c81mr4013450f8f.61.1718636955413; Mon, 17 Jun 2024 08:09:15 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a05:6000:1941:b0:35f:b57:e1c9 with SMTP id ffacd0b85a97d-360718cc178ls1649776f8f.0.-pod-prod-02-eu; Mon, 17 Jun 2024 08:09:14 -0700 (PDT) X-Received: by 2002:a05:6000:1fa1:b0:35f:1ae1:d252 with SMTP id ffacd0b85a97d-3607a783919mr8970267f8f.53.1718636953748; Mon, 17 Jun 2024 08:09:13 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1718636953; cv=none; d=google.com; s=arc-20160816; b=irhM/umRaReFKrq+a1bI7mL0lDrnwHA0UpTOpdrNwTehBjKJCsQbP4xUlA1k3iovs9 HhDc3JZfspKn343m/R3KZSh8Khph0YXP5BIwSyMDm4flOpFkDGmeZIMqzQeJApX7n+su qhvD2vMfIldlQZY+LK9y9uFLILSPzEmZL+tzlMQ3xB/uCHB2Z1WALopNjoB9voJdq/tM g8xHxNnfwpQO5iTHIRcwtjqfg+3foU+Y82eVM4AaWPiSBvPdsL8BMhyg7nA0U8f2hUvq avw/r6NpeRMUIPr+xgnt3Ld0ohlL/Ddl1KH3CQ+elxF0IHqntKijJjfVfaViMERrodYx a+ZQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=a07wbHghMLOtZnFmjWXm5kOQzgrErgkvQ+d+lwJ6VVU=; fh=/h9QQkzJ8EboVkWg45aWwpaUro6WMavIVd2OhN45RtE=; b=y6MCeldsEFOdY14vC183hun1fwMuBkPn6cGuJ2oOLLRkuF7Q6V2HQVJ/v4JRY2ZXmX OF/ZjV/VtmmNh0aGNwCzwjkonMYH59RDwaIJhj6MVSmBp/PgCHouhCTvsJI2IJmb1Zsd Nx1+3XAUVbHCZ8sNzbZ8Qqwu2Wgb9RjzwFPC2wVe7o/IPRUs+i5peIHsdCCnvVf5XEJQ 0ezeX9HtUyiqmMiQPgnvMzyvaYWmDGzFHjtJthXiHda2xpikV6HJ5KOlcrgsOn8RmPT7 HyAjI9wSntfWmHhyoUFpklkzDlb/YqJagT/ecPNm7xcT7VnmFNDAalbfh5Lf3znvB+cH +OlA==; dara=google.com ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=amikan@ilbers.de Received: from shymkent.ilbers.de (shymkent.ilbers.de. [85.214.156.166]) by gmr-mx.google.com with ESMTPS id ffacd0b85a97d-360867eb198si127459f8f.8.2024.06.17.08.09.13 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Mon, 17 Jun 2024 08:09:13 -0700 (PDT) Received-SPF: pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) client-ip=85.214.156.166; Received: from user-B660.promwad.corp ([159.148.83.114]) (authenticated bits=0) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPSA id 45HF9AkP019081 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 17 Jun 2024 17:09:12 +0200 From: Anton Mikanovich To: isar-users@googlegroups.com Cc: Anton Mikanovich Subject: [PATCH v2 4/5] CI: Allow external usage for some APIs Date: Mon, 17 Jun 2024 18:09:00 +0300 Message-Id: <20240617150901.55853-5-amikan@ilbers.de> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240617150901.55853-1-amikan@ilbers.de> References: <20240617150901.55853-1-amikan@ilbers.de> MIME-Version: 1.0 X-Spam-Status: No, score=-4.6 required=5.0 tests=DKIMWL_WL_MED,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_EF,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,RCVD_IN_MSPIKE_H2,RCVD_IN_RP_CERTIFIED, RCVD_IN_RP_RNBL,RCVD_IN_RP_SAFE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on shymkent.ilbers.de X-Original-Sender: amikan@ilbers.de X-Original-Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=amikan@ilbers.de Precedence: list Mailing-list: list isar-users@googlegroups.com; contact isar-users+owners@googlegroups.com List-ID: X-Spam-Checked-In-Group: isar-users@googlegroups.com X-Google-Group-Id: 914930254986 List-Post: , List-Help: , List-Archive: , List-Unsubscribe: , X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= Provide utilities class for external usage to be imported from outside of Avocado framework. Signed-off-by: Anton Mikanovich --- testsuite/cibase.py | 6 +++-- testsuite/cibuilder.py | 50 ++++---------------------------------- testsuite/citest.py | 4 +++- testsuite/utils.py | 54 ++++++++++++++++++++++++++++++++++++++++++ 4 files changed, 66 insertions(+), 48 deletions(-) create mode 100755 testsuite/utils.py diff --git a/testsuite/cibase.py b/testsuite/cibase.py index 0f6997af..c257a465 100755 --- a/testsuite/cibase.py +++ b/testsuite/cibase.py @@ -8,6 +8,8 @@ import tempfile import time from cibuilder import CIBuilder, isar_root +from utils import CIUtils + from avocado.utils import process class CIBaseTest(CIBuilder): @@ -238,13 +240,13 @@ class CIBaseTest(CIBuilder): for target in targets: sfiles[target] = dict() package = target.rsplit(':', 1)[-1] - isar_apt = self.getVars('REPO_ISAR_DB_DIR', target=target) + isar_apt = CIUtils.getVars('REPO_ISAR_DB_DIR', target=target) fpath = f'{package}/{package}*.tar.gz' targz = set(glob.glob(f'{isar_apt}/../apt/*/pool/*/*/{fpath}')) if len(targz) < 1: self.fail('No source packages found') for filename in targz: - sfiles[target][filename] = self.get_tar_content(filename) + sfiles[target][filename] = CIUtils.get_tar_content(filename) return sfiles self.configure(**kwargs) diff --git a/testsuite/cibuilder.py b/testsuite/cibuilder.py index a51d6f7e..4aa06ffb 100755 --- a/testsuite/cibuilder.py +++ b/testsuite/cibuilder.py @@ -9,11 +9,11 @@ import shutil import signal import subprocess import sys -import tarfile import time import tempfile import start_vm +from utils import CIUtils from avocado import Test from avocado.utils import path @@ -22,7 +22,6 @@ from avocado.utils import process sys.path.insert(0, os.path.dirname(os.path.realpath(__file__)) + '/../bitbake/lib') import bb -import bb.tinfoil DEF_VM_TO_SEC = 600 @@ -253,38 +252,6 @@ class CIBuilder(Test): except FileNotFoundError: self.log.warn(path + backup_prefix + ' not exist') - def getVars(self, *vars, target=None): - self.check_init() - def fixStream(stream): - # fix stream objects to emulate _io.TextIOWrapper - stream.isatty = lambda: False - stream.fileno = lambda: False - stream.encoding = sys.getdefaultencoding() - - sl = target is not None - fixStream(sys.stdout) - fixStream(sys.stderr) - - # wait until previous bitbake will be finished - lockfile = os.path.join(self.build_dir, 'bitbake.lock') - checks = 0 - while os.path.exists(lockfile) and checks < 5: - time.sleep(1) - checks += 1 - - with bb.tinfoil.Tinfoil(setup_logging=sl) as tinfoil: - values = () - if target: - tinfoil.prepare(quiet=2) - d = tinfoil.parse_recipe(target) - for var in vars: - values = values + (d.getVar(var, True) or 'None',) - else: - tinfoil.prepare(config_only=True, quiet=2) - for var in vars: - values = values + (tinfoil.config_data.getVar(var, True) or 'None',) - return values if len(values) > 1 else values[0] - def create_tmp_layer(self): tmp_layer_dir = os.path.join(isar_root, 'meta-tmp') @@ -314,13 +281,6 @@ BBPATH .= ":${LAYERDIR}"\ bb.utils.edit_bblayers_conf(bblayersconf_file, None, tmp_layer_dir) bb.utils.prunedir(tmp_layer_dir) - def get_tar_content(self, filename): - try: - tar = tarfile.open(filename) - return tar.getnames() - except Exception: - return [] - def get_ssh_cmd_prefix(self, user, host, port, priv_key): cmd_prefix = 'ssh -o ConnectTimeout=5 -o StrictHostKeyChecking=no '\ '-p %s -o IdentityFile=%s %s@%s ' \ @@ -526,10 +486,10 @@ BBPATH .= ":${LAYERDIR}"\ resize_output = None image_fstypes, \ wks_file, \ - bbdistro = self.getVars('IMAGE_FSTYPES', \ - 'WKS_FILE', \ - 'DISTRO', \ - target=multiconfig) + bbdistro = CIUtils.getVars('IMAGE_FSTYPES', \ + 'WKS_FILE', \ + 'DISTRO', \ + target=multiconfig) # only the first type will be tested in start_vm if image_fstypes.split()[0] == 'wic': diff --git a/testsuite/citest.py b/testsuite/citest.py index 42d44f6a..b84ae0e1 100755 --- a/testsuite/citest.py +++ b/testsuite/citest.py @@ -5,6 +5,7 @@ import os from avocado import skipUnless from avocado.utils import path from cibase import CIBaseTest +from utils import CIUtils UMOCI_AVAILABLE = True SKOPEO_AVAILABLE = True @@ -17,6 +18,7 @@ try: except path.CmdNotFoundError: SKOPEO_AVAILABLE = False + class DevTest(CIBaseTest): """ @@ -46,7 +48,7 @@ class DevTest(CIBaseTest): def test_dev_rebuild(self): self.init() - layerdir_core = self.getVars('LAYERDIR_core') + layerdir_core = CIUtils.getVars('LAYERDIR_core') dpkgbase_file = layerdir_core + '/classes/dpkg-base.bbclass' diff --git a/testsuite/utils.py b/testsuite/utils.py new file mode 100755 index 00000000..a3f643fa --- /dev/null +++ b/testsuite/utils.py @@ -0,0 +1,54 @@ +#!/usr/bin/env python3 + +import os +import sys +import tarfile +import time + +sys.path.insert(0, os.path.dirname(os.path.realpath(__file__)) + '/../bitbake/lib') + +import bb +import bb.tinfoil + +class CIUtils(): + @staticmethod + def getVars(*vars, target=None): + def fixStream(stream): + # fix stream objects to emulate _io.TextIOWrapper + stream.isatty = lambda: False + stream.fileno = lambda: False + stream.encoding = sys.getdefaultencoding() + + sl = target is not None + if not hasattr(sys.stdout, 'isatty'): + fixStream(sys.stdout) + if not hasattr(sys.stderr, 'isatty'): + fixStream(sys.stderr) + + # wait until previous bitbake will be finished + lockfile = os.path.join(os.getcwd(), 'bitbake.lock') + checks = 0 + while os.path.exists(lockfile) and checks < 5: + time.sleep(1) + checks += 1 + + with bb.tinfoil.Tinfoil(setup_logging=sl) as tinfoil: + values = () + if target: + tinfoil.prepare(quiet=2) + d = tinfoil.parse_recipe(target) + for var in vars: + values = values + (d.getVar(var, True) or '',) + else: + tinfoil.prepare(config_only=True, quiet=2) + for var in vars: + values = values + (tinfoil.config_data.getVar(var, True) or '',) + return values if len(values) > 1 else values[0] + + @staticmethod + def get_tar_content(filename): + try: + tar = tarfile.open(filename) + return tar.getnames() + except Exception: + return []