From patchwork Thu Sep 21 06:32:47 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anton Mikanovich X-Patchwork-Id: 3103 Return-Path: Received: from shymkent.ilbers.de ([unix socket]) by shymkent (Cyrus 2.5.10-Debian-2.5.10-3+deb9u2) with LMTPA; Thu, 21 Sep 2023 08:33:08 +0200 X-Sieve: CMU Sieve 2.4 Received: from mail-wr1-f64.google.com (mail-wr1-f64.google.com [209.85.221.64]) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPS id 38L6X7He025707 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Thu, 21 Sep 2023 08:33:07 +0200 Received: by mail-wr1-f64.google.com with SMTP id ffacd0b85a97d-30932d15a30sf394402f8f.1 for ; Wed, 20 Sep 2023 23:33:07 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1695277982; cv=pass; d=google.com; s=arc-20160816; b=dOPwxVzLn1cbOmUf2WVSa2a57aNw5f4YCE9XQV+Eddg07whO/bUqLXAoZ0FM1tuVt0 oOfMZyvyd6Dk3+FyYvG6IuDeNrn3dLvOXQTaMOAAnDyyJAb5UwIx7LREa4BzEa/TqbkS a425D8FiSIJAVdMYGSsRPaeW/BO79gckhdP1Pje0vkXyIr6I8ONVVBjooaqAH4+NTDSU 3eLadz7uech2I+g3nZ5AwBsRv87mxbvf28rczFuW9f9aH9JtC2UZq6LdcOw1ILB7Fgs9 /gJksYSC8exoeEHUsX906UqJsI4L7YdXpV4XWaQ1ICC7NuF2PeSozr2kr0DQCdDVMG+H ZonQ== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:dkim-signature; bh=ciDAFCmLDobcs4akP9DoqJAI6CJ0buokKIQ2E76a1zo=; fh=/h9QQkzJ8EboVkWg45aWwpaUro6WMavIVd2OhN45RtE=; b=lVpb+vuD01oKQBxVRpb3ne6g++XJUB8Jteb7nxtYW9jeqaxQoTe+NDYI4+FYW9ac5O ME9UM6Dwo4hBdPGp+cx10ug/M+BXvBLHRdfJDO8vdvCUwHLGdiy//mvENR5+fbAA4L6f plz8hRxEjOALnCx+D6+wHGWdVLI7IywYkiHoUvEfBgsQlLdh9ogmxb7WS53fVrz1oADz BNpqvucWTdX/9rGMs7HQi8A8g0xwVBsv9kyoJIFN5VbAqPin6YEhzJmUziYLkGeCyYCa mlUOl7jNQeVixEVSOQ5r8qymsnv1CXFphW7IQjIwVWEop6pB/ELppkb5SvaC8KIToe7r ZVtg== ARC-Authentication-Results: i=2; gmr-mx.google.com; spf=pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=amikan@ilbers.de DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlegroups.com; s=20230601; t=1695277982; x=1695882782; darn=isar-build.org; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:x-original-authentication-results :x-original-sender:mime-version:references:in-reply-to:message-id :date:subject:cc:to:from:sender:from:to:cc:subject:date:message-id :reply-to; bh=ciDAFCmLDobcs4akP9DoqJAI6CJ0buokKIQ2E76a1zo=; b=MPg+Lo1PVBp5sfRcKJZKtx5FnXe5k61X+lHRNzVXL+2sa+bubyC9Z7/u7eZp/OX9Ay gj56OBOVqSEEqxWcAhVW5IHY5HCaupjrdOOsL+eUt3qsA8a2A5OPgAQjrV6BQudOKi8A +3x8Ziw6lI2juYSIWtvFy7BTo1FgC03ykIYl38HKrlneJaX5SVBaLazazxEcHDO02vlQ KGLwyBY5eDqp4yCa4sVyoRi1QYexnWhDlYzy7X/FV+6v+jTagXGqys/UDU02qh51O2Xg GhojYSsZP6/D5XAedjd1XAtG9laZLB5H0ntvyR5ScR8u83WzOqs/72lPPP/xQqQ/YpGB 7KoA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1695277982; x=1695882782; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :x-spam-checked-in-group:list-id:mailing-list:precedence :x-original-authentication-results:x-original-sender:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :x-beenthere:x-gm-message-state:sender:from:to:cc:subject:date :message-id:reply-to; bh=ciDAFCmLDobcs4akP9DoqJAI6CJ0buokKIQ2E76a1zo=; b=nx6ARyBx+Vft7KH2E3HIyWeqCf0jwDrtBl2mX99RhmSa5RA6MeecxR6jYAsJCje+9s gNO5KT9fCseYemiIsNFZNXigsCCipl42B5QtE/e91+kbZbLB41/UmAajX4r3XL2YuQQ0 zuboG38EkqdvBpduy7ShvJ8z5CLeL9uo9mTGT37Cn0YUvbyNAAZ684beTCwjLaTyNmiV OtSZETak/eC2NJTgsGIz/lgpzLRoyI0fSSCdpMLDjD3BLdldjPSRXpLytvgH1tb7LDLB xeODpYXXV7x68ouEuZf3fOJqnpg91ceML7k7/HM1deenYpurNVkonrQp1lHLd6l63YKq 5tJA== Sender: isar-users@googlegroups.com X-Gm-Message-State: AOJu0Ywif3tu+zOpjewRdYVmH3usTbrHQ0wJ9obn9pBmtVUIngch1KuJ dnxiXk8+UJ2GPwcNlfn5Azw= X-Google-Smtp-Source: AGHT+IGB89t+7PK7D/GYhzwkxUe2pbbRplSCeM4AIxwH6QRMQI4JbMqJFnTr7sRXm5snJvwCk4ZRgw== X-Received: by 2002:adf:e803:0:b0:31f:a9db:b838 with SMTP id o3-20020adfe803000000b0031fa9dbb838mr4147092wrm.15.1695277981304; Wed, 20 Sep 2023 23:33:01 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com Received: by 2002:adf:ffc6:0:b0:317:6a90:8149 with SMTP id x6-20020adfffc6000000b003176a908149ls290409wrs.0.-pod-prod-07-eu; Wed, 20 Sep 2023 23:32:59 -0700 (PDT) X-Received: by 2002:adf:e888:0:b0:320:baa:eba1 with SMTP id d8-20020adfe888000000b003200baaeba1mr4454041wrm.5.1695277979758; Wed, 20 Sep 2023 23:32:59 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1695277979; cv=none; d=google.com; s=arc-20160816; b=Ba3bsqS+q99t8Emoc9IQ7rUyj+Pnmh4k3TpdIKW3v89hhAR9Zck4VPBzPcuWP5SNUN u3hdeEnwJR+m/rVIHoyDv/oFil/5uOXfPfKXaHHi7w16QAfOrdAAnEXTMTLwlv1OnsBW a5UwBMsX1tVdk+/X5AdPQi06GPoiXtE3QKvJGVCEy4aCtz/nmWVlGl2yKIDtoZscJLVe iDvCcOSUuFAmNmXlqbVsZ86pn7od9uxynZ/vgraxv0Ls+Wosy2m9ynIm2FJEPpiXSoYM gUrbU1LsvW/qcTeyIphlumr0VkdXp/CEOEmZbyFS82WuSsSpoEErvTZpVedDKBC7v16+ 02QA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=TmqN0Il4QSKboMNB0Qw1h9XFB4AD2nkHJj2iDFxaBbw=; fh=/h9QQkzJ8EboVkWg45aWwpaUro6WMavIVd2OhN45RtE=; b=RNbvbLkweLsrptDGXDU2LUmwlPyGeZ5xMnyFtWDMuNehna0OTI5T0nRSjQphw9dq5k IvFlf+uNb9LPZ94oP10MBbYByhHdKifKGjXeS5jOXzDRN4X6466ymlINBcEKRsXw2Whc 1fuAYhDH5Lir59R3rOt2umsxFFh8G8MyiUBt3pbR0AXrlNp9bWf/r9mb9olr3ZXY5Hns DPq+liNtjIyY2SYtfyzll+kfDxcoC7HVcTh/yxm675RpAss1XOY4Pg4bkkIoGGwA1wct lAAfqfKVfpVpN+9Z5twOwbTxdYDbMHJbBbydUOvSd95I49RcTWFyVWwF2pEe3C2KJltn yavw== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=amikan@ilbers.de Received: from shymkent.ilbers.de (shymkent.ilbers.de. [85.214.156.166]) by gmr-mx.google.com with ESMTPS id y14-20020adfe6ce000000b0031c3528356asi47904wrm.2.2023.09.20.23.32.59 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Wed, 20 Sep 2023 23:32:59 -0700 (PDT) Received-SPF: pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) client-ip=85.214.156.166; Received: from localhost.localdomain (r154-240-38-77-broadband.btv.lv [77.38.240.154] (may be forged)) (authenticated bits=0) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPSA id 38L6Wt0j025667 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 21 Sep 2023 08:32:59 +0200 From: Anton Mikanovich To: isar-users@googlegroups.com Cc: Anton Mikanovich Subject: [PATCH 3/3] isar: Add provides explorer Date: Thu, 21 Sep 2023 09:32:47 +0300 Message-Id: <20230921063247.3114251-4-amikan@ilbers.de> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230921063247.3114251-1-amikan@ilbers.de> References: <20230921063247.3114251-1-amikan@ilbers.de> MIME-Version: 1.0 X-Spam-Status: No, score=-0.9 required=5.0 tests=DKIMWL_WL_MED,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_EF,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,RCVD_IN_MSPIKE_H2,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on shymkent.ilbers.de X-Original-Sender: amikan@ilbers.de X-Original-Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=amikan@ilbers.de Precedence: list Mailing-list: list isar-users@googlegroups.com; contact isar-users+owners@googlegroups.com List-ID: X-Spam-Checked-In-Group: isar-users@googlegroups.com X-Google-Group-Id: 914930254986 List-Post: , List-Help: , List-Archive: , List-Unsubscribe: , X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= Explore Isar recipes for any packages declared in debian/control but not added as PROVIDES of recipe. Usage (inside build dir): $ ../scripts/explore.py provides mc:qemuamd64-bullseye:isar-image-base This script requires python3-apt to be installed. Signed-off-by: Anton Mikanovich --- scripts/explore.py | 95 ++++++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 92 insertions(+), 3 deletions(-) diff --git a/scripts/explore.py b/scripts/explore.py index 56c7378a..ca387e6e 100755 --- a/scripts/explore.py +++ b/scripts/explore.py @@ -71,6 +71,89 @@ class IsarBuilder: else: return True + def provides_lookup(self): + try: + self.isar_apt_update() + self.sr = apt_pkg.SourceRecords() + except: + print('No apt-cache found') + + sources_build = [] + provides_all = [] + for pn, fns in self.recipecache.pkg_pn.items(): + if pn.endswith('-native') or pn.endswith('-compat'): + continue + + provider = self.tinfoil.find_best_provider(pn) + if provider[3] is None or provider[3].endswith('linux-distro.bb'): + continue + + pn_mc = pn if not self.mc else ':'.join(['mc', self.mc, pn]) + + local_d = self.tinfoil.parse_recipe(pn_mc) + source_task = local_d.getVar('do_deploy_source', expand=True) + if not source_task: + continue + + # Filter out conflict packages + list = 0 + provides = local_d.getVar('PROVIDES', expand=True) + for provides_list in provides_all: + for prov in provides.split(): + if prov in provides_list: + break + else: + break + list += 1 + + if list >= len(provides_all): + provides_all.append([]) + if list >= len(sources_build): + sources_build.append([]) + + provides_all[list].extend(provides.split()) + if hasattr(self, 'sr'): + self.sr.restart() + if not self.sr.lookup(pn): + sources_build[list].append(pn) + else: + sources_build[list].append(pn) + + for sources in sources_build: + if not self.bitbake(sources, 'do_deploy_source'): + return + + if not hasattr(self, 'sr'): + try: + self.isar_apt_update() + self.sr = apt_pkg.SourceRecords() + except: + print('No apt-cache found') + + for pn, fns in self.recipecache.pkg_pn.items(): + if pn.endswith('-native') or pn.endswith('-compat'): + continue + + provider = self.tinfoil.find_best_provider(pn) + + self.sr.restart() + while self.sr.lookup(pn): + pn_mc = pn if not self.mc else ':'.join(['mc', self.mc, pn]) + local_d = self.tinfoil.parse_recipe(pn_mc) + provides = local_d.getVar('PROVIDES', expand=True) + + for bin in self.sr.binaries: + if bin == pn: + continue + + if bin in provides: + continue + + print(f'{pn} provides {bin}') + append_filename = provider[3].rsplit(':', 1)[-1] + 'append' + with open(append_filename, 'a') as f: + f.write(f'PROVIDES += "{bin}"\n') + def deps_lookup(self, pkg, parent_provider, add=True): if pkg in self.cache: return @@ -154,8 +237,8 @@ class IsarBuilder: if self.need_source: print('Following packages still left unchecked: ' + str(self.need_source)) -if len(sys.argv) != 3: - print('Usage: ../scripts/explore.py deps mc:qemuamd64-bullseye:isar-image-base') +if len(sys.argv) != 3 or sys.argv[1] not in ['all', 'deps', 'provides']: + print('Usage: ../scripts/explore.py [ all | deps | provides ] mc:qemuamd64-bullseye:isar-image-base') exit(1) mc = '' @@ -170,6 +253,12 @@ else: ib = IsarBuilder() try: ib.init(mc, target) - ib.explore_deps() + if sys.argv[1] == 'all': + ib.provides_lookup() + ib.explore_deps() + elif sys.argv[1] == 'deps': + ib.explore_deps() + elif sys.argv[1] == 'provides': + ib.provides_lookup() finally: ib.shutdown()