From patchwork Fri Jun 14 11:23:19 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anton Mikanovich X-Patchwork-Id: 3604 Return-Path: Received: from shymkent.ilbers.de ([unix socket]) by shymkent (Cyrus 2.5.10-Debian-2.5.10-3+deb9u2) with LMTPA; Fri, 14 Jun 2024 13:23:43 +0200 X-Sieve: CMU Sieve 2.4 Received: from mail-lj1-f183.google.com (mail-lj1-f183.google.com [209.85.208.183]) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPS id 45EBNgof031700 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Fri, 14 Jun 2024 13:23:42 +0200 Received: by mail-lj1-f183.google.com with SMTP id 38308e7fff4ca-2ebf0863242sf14636821fa.3 for ; Fri, 14 Jun 2024 04:23:42 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1718364217; cv=pass; d=google.com; s=arc-20160816; b=fDqd9o81MoleuFW/aR/WMvRvYNyzomXJRsgfhAJevq+w5BW/9FDBaW50otIBsS7Fuh f1FelafJ4UekA3hkkiaL3Yv1ZLMrW/jqZ7qEWgUYv5+BRMNODaWIb3YHUQ1K7Wbm5GAm EYhQu7bxks6RbnCp3o5ICatZQH4ujs6D5LaMzR5pd07YvvqrVI696TWKt4qZD26jTE4Z JssYyIfwgZKxiR0iF+btoGpZkvs661aybeaR1af5bPt9YUbSFHA8szj7T7djZkWP/zZc x9vBATNeoQj/cvEDteWII8xpiKiU7LdjnwcqsWdoeRXSQFrHKNhND2A3BO3GUeiXQCoz oR0g== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:dkim-signature; bh=NNoAeFpgpot9PlqTvrcYpVHVB/EX9TP9G92au2hioUA=; fh=5QtjtvFcGvoK4DTMUwWgys3F5t9R3BDiUCIjmABbwyg=; b=ZceltdALMKj/x4fQLSUcEVkI/agewJZL9wiMWy++nYgwwzy29B4TnDBcqkBZuUigqZ zIzQrpQE4HP4wKu3f8odzvBJR+6slFRzBfDVR65DzblvwCRVm20OHWNQBwBgQ42WAqxK H68JCkwmcqJZUG51xVhwq7JTM+g/DpEBjiSUlv9LHd3qAPrEBfHU74Ej5WkEBu8qLIPW mPYjBt4Y1hwyAVZOT2cVlGZzID88M5MwEKip0EXx9bq+BO7b+Q07Y5mhBwo2hN95b5em bLHxIIIYn9Fg0d6UcLQQXRP12iOCkvRri1HWTlJ3JREquGfbv+pmI2OW25WrY9dyACnu cOnQ==; darn=isar-build.org ARC-Authentication-Results: i=2; gmr-mx.google.com; spf=pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=amikan@ilbers.de DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlegroups.com; s=20230601; t=1718364217; x=1718969017; darn=isar-build.org; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:x-original-authentication-results :x-original-sender:mime-version:references:in-reply-to:message-id :date:subject:cc:to:from:sender:from:to:cc:subject:date:message-id :reply-to; bh=NNoAeFpgpot9PlqTvrcYpVHVB/EX9TP9G92au2hioUA=; b=T0ZX7audoLJvvNSyRn2QhKuyIBOubO0rCHu/IJA/CcxdwHd/AzaKaZow8bjXTsDgQ2 iOn9vRblsKJGlpxJg4DkG23WKQyfnJOw9P5ISOLGN1EcQcOlqsIEKvFYQsvJleoNU2O/ mN01aFOVrde5At87jcWYSkC+H0LH43i8RrAfWdBstt0SwL8wfBIstYDC0+5nEjaGwS2F 3Wqh/bOP2e+gZoOAtS81p6bEk5+4rJTQbR38xD//Rg7BlpZ4Jyp4k/sazOPL5jOHUah9 td+y9WqMR4cSo/CKev4TTLUO9rQI2lEsARoGS3b6CD7RTQKXiXLjoXHmJ/c96qScG1dr K6fA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1718364217; x=1718969017; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :x-spam-checked-in-group:list-id:mailing-list:precedence :x-original-authentication-results:x-original-sender:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :x-beenthere:x-gm-message-state:sender:from:to:cc:subject:date :message-id:reply-to; bh=NNoAeFpgpot9PlqTvrcYpVHVB/EX9TP9G92au2hioUA=; b=ds29yd3ZghizFVYNTdyLyPOdk7+NWZ/XATkS0tTkrS0e51yiJJJQwPUDYxpiOCrpVn 60rGLDK2Ecf6dTO8HG99t5M1hSgwnR/Q7ngb2GpqMkRY0QfdN7uEda+qpqH29zXj/VPX SHpSkrvUkS0df+8f1o+hpsdbnSVwRGR7ur32IREOGuzKRZZFbQ3JBt9GM1NpYELBF8lJ hDfEnQaKKNVilpn0WhMILWfm2TOOpxQgbTHK3+eas9idHX3WgHYpq9LL0PzNhWm+gVK/ Vq/3TNEpA33w0sD6zv5FvF5J3ndsA9dK6u7ZTut37dg4EneElCNH27+MAjNo9Bs9qUFT F7uQ== Sender: isar-users@googlegroups.com X-Forwarded-Encrypted: i=2; AJvYcCV1GzTqOM0/Wjmmz0IR/4fkwFMTKshi3MvL7qGWB/OMghnSasl5mQGa3OofP20nWlw8WIZtVfSU1p3UMzTjYQy1Vrf4xqs= X-Gm-Message-State: AOJu0Yx2XJVNAytfo6wxgQNf1DhXDaleXv3xdcFg6R2EN2gfwq4P5JO3 FZH+jMzAMlD9COl7Xze6vKpjmD8X+67V8kIvRAVmYjzEUxcJYKAu X-Google-Smtp-Source: AGHT+IGdEycxDasOFa3f8jtL7YVT4MDxdi1E2ncbQseBqRH/zvYiYrYdRr84gLtDtMcKc4B9zPkbzg== X-Received: by 2002:a2e:961a:0:b0:2eb:eb11:199f with SMTP id 38308e7fff4ca-2ec0e60dd4amr14309441fa.51.1718364215352; Fri, 14 Jun 2024 04:23:35 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a05:600c:3b0c:b0:421:7e41:1878 with SMTP id 5b1f17b1804b1-422b6cce65fls9571285e9.0.-pod-prod-07-eu; Fri, 14 Jun 2024 04:23:33 -0700 (PDT) X-Received: by 2002:a05:600c:91e:b0:421:f04d:ebd2 with SMTP id 5b1f17b1804b1-4230482fbafmr24494045e9.22.1718364213390; Fri, 14 Jun 2024 04:23:33 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1718364213; cv=none; d=google.com; s=arc-20160816; b=TtPasHq6kEm7AQh5zsykKDO3NmtiFEddbEVTSvGRXatBTcteNtL/peoEZr+dw9kOsW mFeUr6kMAyd94IOWRqjVT1VRT173MFd0jiVnaa2NQkfs8DRgGgCATgds6WvqXUooYeJv JZDDYAuw9BPyn+0266UDcJRLtAdGD3W/k8kozL/kycw/tqw5ECHO1qjlIXFobN3HAKRC PjbCViQnFt3EvfCiVOqDiqLGF0yL0HU3NZZavpkGvYWZSo5CiGNzblG7ALU+mBpVnp9e SFnJWGhF1F1d/mki+cuRSXGF+KZOTmRdgxnvjJrNGeZYKn55sh5jX7WPZTPvUBZxZOEa 5Ydw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=a07wbHghMLOtZnFmjWXm5kOQzgrErgkvQ+d+lwJ6VVU=; fh=/h9QQkzJ8EboVkWg45aWwpaUro6WMavIVd2OhN45RtE=; b=0SU5zPPv3JcFIhQgu0JLLqaIWm8S1xC+iI3FmL3z8oEugAp7CMKkPgg5JsKY7/l0XA o1HnjNvO6KFIxQnWWFmtHMVwRnOTe4kw7zBZqg2YiV+x1EJzAkL/WgsDQTzSqaEZ+ruB MAUsQ70mqM8N682cNOfI34ZRkEtTxNwh/N2d207mjCBmrSqyhzejx2ah4Bn4iN7xCmyi njE+dSKBIb4FyUnQN5ZXJpf2sDCFBD0+sjbi9w3nyACLQCVDOrpJs5cO3El6g+Hgt6nI TlLw6iR6YVmslNUXgCn8RS/WwaNhw32fno2AXe8JgqCCKW4OS0o/VzBoRZQJm0/Ad8PB fd2w==; dara=google.com ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=amikan@ilbers.de Received: from shymkent.ilbers.de (shymkent.ilbers.de. [85.214.156.166]) by gmr-mx.google.com with ESMTPS id 5b1f17b1804b1-42285317d3bsi4210915e9.1.2024.06.14.04.23.33 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 14 Jun 2024 04:23:33 -0700 (PDT) Received-SPF: pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) client-ip=85.214.156.166; Received: from localhost.localdomain (r154-240-38-77-broadband.btv.lv [77.38.240.154] (may be forged)) (authenticated bits=0) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPSA id 45EBNRq5031631 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 14 Jun 2024 13:23:32 +0200 From: Anton Mikanovich To: isar-users@googlegroups.com Cc: Anton Mikanovich Subject: [PATCH 4/5] CI: Allow external usage for some APIs Date: Fri, 14 Jun 2024 14:23:19 +0300 Message-Id: <20240614112320.122428-5-amikan@ilbers.de> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240614112320.122428-1-amikan@ilbers.de> References: <20240614112320.122428-1-amikan@ilbers.de> MIME-Version: 1.0 X-Spam-Status: No, score=-4.6 required=5.0 tests=DKIMWL_WL_MED,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_EF,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,RCVD_IN_MSPIKE_H2,RCVD_IN_RP_CERTIFIED, RCVD_IN_RP_RNBL,RCVD_IN_RP_SAFE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on shymkent.ilbers.de X-Original-Sender: amikan@ilbers.de X-Original-Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=amikan@ilbers.de Precedence: list Mailing-list: list isar-users@googlegroups.com; contact isar-users+owners@googlegroups.com List-ID: X-Spam-Checked-In-Group: isar-users@googlegroups.com X-Google-Group-Id: 914930254986 List-Post: , List-Help: , List-Archive: , List-Unsubscribe: , X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= Provide utilities class for external usage to be imported from outside of Avocado framework. Signed-off-by: Anton Mikanovich --- testsuite/cibase.py | 6 +++-- testsuite/cibuilder.py | 50 ++++---------------------------------- testsuite/citest.py | 4 +++- testsuite/utils.py | 54 ++++++++++++++++++++++++++++++++++++++++++ 4 files changed, 66 insertions(+), 48 deletions(-) create mode 100755 testsuite/utils.py diff --git a/testsuite/cibase.py b/testsuite/cibase.py index 0f6997af..c257a465 100755 --- a/testsuite/cibase.py +++ b/testsuite/cibase.py @@ -8,6 +8,8 @@ import tempfile import time from cibuilder import CIBuilder, isar_root +from utils import CIUtils + from avocado.utils import process class CIBaseTest(CIBuilder): @@ -238,13 +240,13 @@ class CIBaseTest(CIBuilder): for target in targets: sfiles[target] = dict() package = target.rsplit(':', 1)[-1] - isar_apt = self.getVars('REPO_ISAR_DB_DIR', target=target) + isar_apt = CIUtils.getVars('REPO_ISAR_DB_DIR', target=target) fpath = f'{package}/{package}*.tar.gz' targz = set(glob.glob(f'{isar_apt}/../apt/*/pool/*/*/{fpath}')) if len(targz) < 1: self.fail('No source packages found') for filename in targz: - sfiles[target][filename] = self.get_tar_content(filename) + sfiles[target][filename] = CIUtils.get_tar_content(filename) return sfiles self.configure(**kwargs) diff --git a/testsuite/cibuilder.py b/testsuite/cibuilder.py index a51d6f7e..4aa06ffb 100755 --- a/testsuite/cibuilder.py +++ b/testsuite/cibuilder.py @@ -9,11 +9,11 @@ import shutil import signal import subprocess import sys -import tarfile import time import tempfile import start_vm +from utils import CIUtils from avocado import Test from avocado.utils import path @@ -22,7 +22,6 @@ from avocado.utils import process sys.path.insert(0, os.path.dirname(os.path.realpath(__file__)) + '/../bitbake/lib') import bb -import bb.tinfoil DEF_VM_TO_SEC = 600 @@ -253,38 +252,6 @@ class CIBuilder(Test): except FileNotFoundError: self.log.warn(path + backup_prefix + ' not exist') - def getVars(self, *vars, target=None): - self.check_init() - def fixStream(stream): - # fix stream objects to emulate _io.TextIOWrapper - stream.isatty = lambda: False - stream.fileno = lambda: False - stream.encoding = sys.getdefaultencoding() - - sl = target is not None - fixStream(sys.stdout) - fixStream(sys.stderr) - - # wait until previous bitbake will be finished - lockfile = os.path.join(self.build_dir, 'bitbake.lock') - checks = 0 - while os.path.exists(lockfile) and checks < 5: - time.sleep(1) - checks += 1 - - with bb.tinfoil.Tinfoil(setup_logging=sl) as tinfoil: - values = () - if target: - tinfoil.prepare(quiet=2) - d = tinfoil.parse_recipe(target) - for var in vars: - values = values + (d.getVar(var, True) or 'None',) - else: - tinfoil.prepare(config_only=True, quiet=2) - for var in vars: - values = values + (tinfoil.config_data.getVar(var, True) or 'None',) - return values if len(values) > 1 else values[0] - def create_tmp_layer(self): tmp_layer_dir = os.path.join(isar_root, 'meta-tmp') @@ -314,13 +281,6 @@ BBPATH .= ":${LAYERDIR}"\ bb.utils.edit_bblayers_conf(bblayersconf_file, None, tmp_layer_dir) bb.utils.prunedir(tmp_layer_dir) - def get_tar_content(self, filename): - try: - tar = tarfile.open(filename) - return tar.getnames() - except Exception: - return [] - def get_ssh_cmd_prefix(self, user, host, port, priv_key): cmd_prefix = 'ssh -o ConnectTimeout=5 -o StrictHostKeyChecking=no '\ '-p %s -o IdentityFile=%s %s@%s ' \ @@ -526,10 +486,10 @@ BBPATH .= ":${LAYERDIR}"\ resize_output = None image_fstypes, \ wks_file, \ - bbdistro = self.getVars('IMAGE_FSTYPES', \ - 'WKS_FILE', \ - 'DISTRO', \ - target=multiconfig) + bbdistro = CIUtils.getVars('IMAGE_FSTYPES', \ + 'WKS_FILE', \ + 'DISTRO', \ + target=multiconfig) # only the first type will be tested in start_vm if image_fstypes.split()[0] == 'wic': diff --git a/testsuite/citest.py b/testsuite/citest.py index 42d44f6a..b84ae0e1 100755 --- a/testsuite/citest.py +++ b/testsuite/citest.py @@ -5,6 +5,7 @@ import os from avocado import skipUnless from avocado.utils import path from cibase import CIBaseTest +from utils import CIUtils UMOCI_AVAILABLE = True SKOPEO_AVAILABLE = True @@ -17,6 +18,7 @@ try: except path.CmdNotFoundError: SKOPEO_AVAILABLE = False + class DevTest(CIBaseTest): """ @@ -46,7 +48,7 @@ class DevTest(CIBaseTest): def test_dev_rebuild(self): self.init() - layerdir_core = self.getVars('LAYERDIR_core') + layerdir_core = CIUtils.getVars('LAYERDIR_core') dpkgbase_file = layerdir_core + '/classes/dpkg-base.bbclass' diff --git a/testsuite/utils.py b/testsuite/utils.py new file mode 100755 index 00000000..a3f643fa --- /dev/null +++ b/testsuite/utils.py @@ -0,0 +1,54 @@ +#!/usr/bin/env python3 + +import os +import sys +import tarfile +import time + +sys.path.insert(0, os.path.dirname(os.path.realpath(__file__)) + '/../bitbake/lib') + +import bb +import bb.tinfoil + +class CIUtils(): + @staticmethod + def getVars(*vars, target=None): + def fixStream(stream): + # fix stream objects to emulate _io.TextIOWrapper + stream.isatty = lambda: False + stream.fileno = lambda: False + stream.encoding = sys.getdefaultencoding() + + sl = target is not None + if not hasattr(sys.stdout, 'isatty'): + fixStream(sys.stdout) + if not hasattr(sys.stderr, 'isatty'): + fixStream(sys.stderr) + + # wait until previous bitbake will be finished + lockfile = os.path.join(os.getcwd(), 'bitbake.lock') + checks = 0 + while os.path.exists(lockfile) and checks < 5: + time.sleep(1) + checks += 1 + + with bb.tinfoil.Tinfoil(setup_logging=sl) as tinfoil: + values = () + if target: + tinfoil.prepare(quiet=2) + d = tinfoil.parse_recipe(target) + for var in vars: + values = values + (d.getVar(var, True) or '',) + else: + tinfoil.prepare(config_only=True, quiet=2) + for var in vars: + values = values + (tinfoil.config_data.getVar(var, True) or '',) + return values if len(values) > 1 else values[0] + + @staticmethod + def get_tar_content(filename): + try: + tar = tarfile.open(filename) + return tar.getnames() + except Exception: + return []