From patchwork Sun Jun 12 23:07:59 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Uladzimir Bely X-Patchwork-Id: 1875 Return-Path: Received: from shymkent.ilbers.de ([unix socket]) by shymkent (Cyrus 2.5.10-Debian-2.5.10-3+deb9u2) with LMTPA; Mon, 13 Jun 2022 09:08:10 +0200 X-Sieve: CMU Sieve 2.4 Received: from mail-oa1-f58.google.com (mail-oa1-f58.google.com [209.85.160.58]) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPS id 25D788N6027794 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Mon, 13 Jun 2022 09:08:09 +0200 Received: by mail-oa1-f58.google.com with SMTP id 586e51a60fabf-f1d7f9dc02sf4399080fac.17 for ; Mon, 13 Jun 2022 00:08:09 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1655104083; cv=pass; d=google.com; s=arc-20160816; b=j+hrHfQIe0Szp/VV1uvuHo4FoVWH4SYXlndJ3kLe5DXxR+DLwhwec7nlGfU2hM+vtj s9MQJspO/WuJaAjGIzqEHzxIVlO78cGf97taSOQUr9OdUdCMoOZgKU4cdvJM19YL4oNY L0BelY/yl5G8raguaqQivgJr3aFAj0LuHdEqR138om2VxsVQJYg8GC0dONnueXuHn6aS cX9Ew+GZU+d3BbD0P49YhBjJwWi67xar0WAYCzyVuphkiXHCRGpg95159FGUpfLbhj0s jtbXHOzDY1DoEpA7ewh11qMOwq79FW8Jf4QDAchPJ6NlvX8cda6FqzQ2WQAVVftT44OC 8yWA== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:mime-version:references:in-reply-to :message-id:date:subject:to:from:sender:dkim-signature; bh=Zj+Gn7QvEp6kIkSvfMlTI5sZfuIi8USPqz6yX7fvjyc=; b=eUPgS9Y9cB+NcQJJF9w5YXM6hw9CSrxSyQcWJgQTnWcDKRX84iEHNPDhJsiy/DCQZh BD0/ESXiXoo+oFpe/7zkhyAgbcoUur5htP6JjAYtLp5euokTZEkdEhMu98x+NY58RXQ4 yDSSj8sfUBWUt0ivn1NqtjvK83cqPhaj0Cgnr64p9u3wLO9BYdkI+wIMzuk9HQ54ESUP g/8qpsq1LvoJvgFYGfqDzmMznYz+KkQGMQtMvsR5ZHcRZ3CdZaAK+ka6jIe5LpMRgsMh gtaHtxqcPYUaVBGm5hfLXWwHxuyS+F79pz7WKj0K1nO53S1n4u18GD5ovewO0uOR4mn+ HF1g== ARC-Authentication-Results: i=2; gmr-mx.google.com; spf=pass (google.com: domain of ubely@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=ubely@ilbers.de DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlegroups.com; s=20210112; h=sender:from:to:subject:date:message-id:in-reply-to:references :mime-version:x-original-sender:x-original-authentication-results :precedence:mailing-list:list-id:list-post:list-help:list-archive :list-subscribe:list-unsubscribe; bh=Zj+Gn7QvEp6kIkSvfMlTI5sZfuIi8USPqz6yX7fvjyc=; b=NMLIocy3QnKypu/63Iz9VoBEi8rTOwjJQrkkdEAdsYgd5TblIEb2WMwwJdXAvZ+7RG 9aUhTSbIRuDoyuJnRUaA5APYHB3Nu3bxrsrgJAWMs4cazEZ2qsHan6s7v12tnd3yZ6gk ZMoDjEkRPd0s36jndmG3XxVOp/UXfsZPyqcspLz0TlMZCmYhi1g5nBgTabVX/KL9ewJq aMEI6+kGiXXUTgfPAko2uf/aoRGSXExhe/AY2dMH+/JbEHqSGJfJCVGwtOglBxQ0xzOO H6UpLme71ZVUEBDVNvryrKmFJEIQCsX5/W84s1ix3jO3mqctDW68dAvH58bfOeGo4yra yC0w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=sender:x-gm-message-state:from:to:subject:date:message-id :in-reply-to:references:mime-version:x-original-sender :x-original-authentication-results:precedence:mailing-list:list-id :x-spam-checked-in-group:list-post:list-help:list-archive :list-subscribe:list-unsubscribe; bh=Zj+Gn7QvEp6kIkSvfMlTI5sZfuIi8USPqz6yX7fvjyc=; b=eJ5aWXR1EUI+rTuPXAvH/i5sNkR79Ie+RIqnlwq1avxn+UE0HzSNKn+RwuUuHQVzZv t6E/VyL2iYqXUbz+3Yw/1JjP4aYEky4y1yGnYKMVtmayQVClw7KfO3iQ9ceZkNqI30a0 Q0Yrwv22DpRuVUbGZ5pegAp/O9AMQMQ3KkSwhh/I/zRckBA8PGLl0ZBgulQCT0mlNXqa uZfervv2zNnvsSbCIBXkTuqbLH8bwGeu1dn8Xacjk2YpCTvIU8nhZ/uuqRYyao/OF/Q7 f979+Gxh/SiNMX7R9sFgCF6kNxOPz0qljASjhTV5ar214wBdW7nXwcubZFAIiMVJMy/5 Eofw== Sender: isar-users@googlegroups.com X-Gm-Message-State: AOAM530PDyVPfZEryaBvdZM6/YJJLoaEcQ850ujIxiCYi4yGJ0BjCWuz Lej5svnAuwkU993iwXzgJ5g= X-Google-Smtp-Source: ABdhPJwKoCirZvRua5IUDtP7vUmYMKmQmf/KVmQZ8w+jzfQDd0vApRHHP8AuUkXcr68o2Nx6QJZZJg== X-Received: by 2002:a05:6808:1794:b0:32f:c68:c3db with SMTP id bg20-20020a056808179400b0032f0c68c3dbmr5673768oib.209.1655104083145; Mon, 13 Jun 2022 00:08:03 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a05:6870:3c06:b0:fa:8e9d:2fea with SMTP id gk6-20020a0568703c0600b000fa8e9d2feals806693oab.6.gmail; Mon, 13 Jun 2022 00:08:02 -0700 (PDT) X-Received: by 2002:a05:6870:d355:b0:101:2030:268c with SMTP id h21-20020a056870d35500b001012030268cmr4215239oag.264.1655104082639; Mon, 13 Jun 2022 00:08:02 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1655104082; cv=none; d=google.com; s=arc-20160816; b=ZlibrrnCuuighdAvYeieBi43XLin5O599Wv0y6HcJ3XhDUUVzmqAHkK1BWIJJS59+J wadyJlhH8uhz+sO5t04TVqNfWyeGj03qwLpJts7KBEHTkSZf1wx+85bJDr7kQYvj9w3U 8ss6uYoqpXYdpTr5mSPGqruNKMOg55eXaDv4vtj9Yo+p6L9KP1MaliiIhpFtjH4FRUTp XlJZJaa5zTmwMNCQl7kcAiEfSlqU9Wpb13zA4t1pCo0EtGhVOStA1657rVbMeWpd7Zxl CTX43dstj1kU+pYRcqiOXktEwaUfbKPMuREBuBMp1Wb7yCD4n72kS1IOivjA7AXhaawQ R7fw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from; bh=SyO7+QZIvFKLoiOxURkyFXh0RCKqXkologR65BtUtkE=; b=MT/8d+sf4P85tczozXlxP85MQj+a+r2g83zi6KvYOe1ASrqveluwUddnT0L2EQkby+ kd9QVp/p1lGGF20ESnSVMYh+ziR97tTyomfL/IGI1Q6G11ot7m9iQfoyWsx2rjjcPRCD hLATUpvhlQZE2CaQXOnIgvu4sc+QkEutzNEOs5yTSNdVNZ0mi6n/5Fzvu0AvQoc+Z1S4 mLcviOyby++2WpqiQa6wznt6cAxMDqVa8t7h1qu7bHrdYcmlA/xNeiHtJGsjodJ6oJ2i 9dVq8Sz5cAs/ww2Kj2AF05TbzSiyTKORJOt7E37zjLIm3g7yW+7PY50nvsw9tYVVDtuu qpRw== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of ubely@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=ubely@ilbers.de Received: from shymkent.ilbers.de (shymkent.ilbers.de. [85.214.156.166]) by gmr-mx.google.com with ESMTPS id q23-20020a9d4b17000000b0060bade020f3si304481otf.5.2022.06.13.00.08.02 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Mon, 13 Jun 2022 00:08:02 -0700 (PDT) Received-SPF: pass (google.com: domain of ubely@ilbers.de designates 85.214.156.166 as permitted sender) client-ip=85.214.156.166; Received: from baighyz.m.ilbers.de (host-80-81-17-52.static.customer.m-online.net [80.81.17.52]) (authenticated bits=0) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPSA id 25D77xZA027766 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Mon, 13 Jun 2022 09:08:00 +0200 From: Uladzimir Bely To: isar-users@googlegroups.com Subject: [RFC v2 3/3] ci: Added test for isar-apt reuse functionality Date: Mon, 13 Jun 2022 09:07:59 +0200 Message-Id: <20220613070759.16949-4-ubely@ilbers.de> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20220613070759.16949-1-ubely@ilbers.de> References: <20220613070759.16949-1-ubely@ilbers.de> MIME-Version: 1.0 X-Spam-Status: No, score=-0.8 required=5.0 tests=DKIMWL_WL_MED, DKIM_SIGNED, DKIM_VALID,DKIM_VALID_EF,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,RCVD_IN_MSPIKE_H3,RCVD_IN_MSPIKE_WL,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on shymkent.ilbers.de X-Original-Sender: ubely@ilbers.de X-Original-Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of ubely@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=ubely@ilbers.de Precedence: list Mailing-list: list isar-users@googlegroups.com; contact isar-users+owners@googlegroups.com List-ID: X-Spam-Checked-In-Group: isar-users@googlegroups.com X-Google-Group-Id: 914930254986 List-Post: , List-Help: , List-Archive: , List-Unsubscribe: , X-getmail-retrieved-from-mailbox: INBOX Test makes clean build of "libhello" package that is placed to local isar-apt repo outside of 'tmp'. Second build with cleaned 'tmp' should use previously built package that is detected by missing Makefile in source directory. Signed-off-by: Uladzimir Bely --- testsuite/cibase.py | 42 ++++++++++++++++++++++++++++++++++++++++++ testsuite/cibuilder.py | 7 ++++++- testsuite/citest.py | 12 ++++++++++++ 3 files changed, 60 insertions(+), 1 deletion(-) diff --git a/testsuite/cibase.py b/testsuite/cibase.py index 2ffb8191..9ef9d247 100755 --- a/testsuite/cibase.py +++ b/testsuite/cibase.py @@ -73,6 +73,48 @@ class CIBaseTest(CIBuilder): self.delete_from_build_dir('tmp') self.delete_from_build_dir('ccache') + def perform_isarapt_test(self, package_target, **kwargs): + def check_compiled(package_name): + # Makefile should exist if there was compilation + makefile = glob.glob(f'{self.build_dir}/tmp/work/*/{package_name}/*/{package_name}-*/Makefile') + if makefile: + self.log.info("Makefile was generated") + return True + + self.log.info("Makefile was not generated") + return False + + self.configure(isar_apt_reuse=True, **kwargs) + + package_name = package_target.split(':')[-1] + + # Cleanup sstate and tmp before test` + self.delete_from_build_dir('tmp') + self.delete_from_build_dir('sstate-cache') + self.delete_from_build_dir('isar-apt') + + self.log.info('Start build and populate isar-apt...') + self.bitbake(package_target, **kwargs) + compiled1 = check_compiled(package_name) + + if not compiled1: + self.fail('isar-apt repo used at first build') + + # Cleanup build stuff + self.delete_from_build_dir('tmp') + self.delete_from_build_dir('sstate-cache') + + self.log.info('Starting build and reuse isar-apt repo...') + self.bitbake(package_target, **kwargs) + compiled2 = check_compiled(package_name) + + if compiled2: + self.fail('isar-apt repo not used at second build') + + # Cleanup + self.delete_from_build_dir('tmp') + self.delete_from_build_dir('sstate-cache') + def perform_sstate_test(self, image_target, package_target, **kwargs): def check_executed_tasks(target, expected): taskorder_file = glob.glob(f'{self.build_dir}/tmp/work/*/{target}/*/temp/log.task_order') diff --git a/testsuite/cibuilder.py b/testsuite/cibuilder.py index bc48d47f..42e344a3 100755 --- a/testsuite/cibuilder.py +++ b/testsuite/cibuilder.py @@ -54,7 +54,7 @@ class CIBuilder(Test): def configure(self, compat_arch=True, cross=None, debsrc_cache=False, container=False, ccache=False, sstate=False, offline=False, - gpg_pub_key=None, **kwargs): + isar_apt_reuse=False, gpg_pub_key=None, **kwargs): # write configuration file and set bitbake_args # can run multiple times per test case self.check_init() @@ -76,6 +76,7 @@ class CIBuilder(Test): f' container = {container}\n' f' ccache = {ccache}\n' f' sstate = {sstate}\n' + f' isar_apt_reuse = {isar_apt_reuse}\n' f' gpg_pub_key = {gpg_pub_key}\n' f'===================================================') @@ -107,6 +108,10 @@ class CIBuilder(Test): f.write('BASE_REPO_KEY="file://' + gpg_pub_key + '"\n') if distro_apt_premir: f.write('DISTRO_APT_PREMIRRORS = "%s"\n' % distro_apt_premir) + if isar_apt_reuse: + f.write('ISAR_APT_REUSE = "1"\n') + f.write('REPO_ISAR_DIR = "${TOPDIR}/isar-apt/${DISTRO}-${DISTRO_ARCH}/apt"\n') + f.write('REPO_ISAR_DB_DIR = "${TOPDIR}/isar-apt/${DISTRO}-${DISTRO_ARCH}/db"\n') # include ci_build.conf in local.conf with open(self.build_dir + '/conf/local.conf', 'r+') as f: diff --git a/testsuite/citest.py b/testsuite/citest.py index 16e38d07..7545d9e6 100755 --- a/testsuite/citest.py +++ b/testsuite/citest.py @@ -60,6 +60,18 @@ class CcacheTest(CIBaseTest): self.init() self.perform_ccache_test(targets) +class IsarAptTest(CIBaseTest): + + """ + Test rebuild speed improve with isar-apt reuse enabled + + :avocado: tags=isarapt,full + """ + def test_isarapt_reuse(self): + target = 'mc:qemuamd64-bullseye:libhello' + self.init() + self.perform_isarapt_test(target) + class CrossTest(CIBaseTest): """