From patchwork Fri Apr 5 16:31:26 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Larson, Chris" X-Patchwork-Id: 167 Return-Path: Received: from shymkent.ilbers.de ([unix socket]) by shymkent (Cyrus 2.5.10-Debian-2.5.10-3+deb9u2) with LMTPA; Fri, 05 Apr 2024 18:32:33 +0200 X-Sieve: CMU Sieve 2.4 Received: from mail-pf1-f184.google.com (mail-pf1-f184.google.com [209.85.210.184]) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPS id 435GWVZG003029 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Fri, 5 Apr 2024 18:32:32 +0200 Received: by mail-pf1-f184.google.com with SMTP id d2e1a72fcca58-6e6bab4b84dsf2280452b3a.0 for ; Fri, 05 Apr 2024 09:32:32 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1712334745; cv=pass; d=google.com; s=arc-20160816; b=KzBMBkS6aW5OTnZkDfpPp4YQ6licDi0G3shlqRYcD857Tz/opDvg2AAqycoLwddo7d XjHx0cWG9GfeDnP2K15nOKYo/AIP3EYnZeYZRHsZ2qO6Aw2QV1M4w1YESEdzNnRo41VH 1DPh0RRBVGGUTWJ2n9fjNKDWpDs8LXCFt6l0i6mj2IMihNKheWenM2lPJaovTpQbaLEO h+1glSHn+C+Em6BOfPNzEKlwXyPs9/bbTV99xhWZKfU8MfFHuOXOgjbZ+KPC57WyfUiD QmYIGeMus0Hw024BXSZzchPP56xqbcGQb/Ul/ROZvGpdW0okIxiaS3Of94bz4uprT+07 UUaQ== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:reply-to:feedback-id :content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:dkim-signature; bh=XTSs71MpwrXXIIYorPQBgqXnIvX4AAIxNG17+My2xbA=; fh=vp027gQjJSJllucTMyNU/OCimSRtjNBRcB2omlgfQ6c=; b=A5UW6ajRXT1YNZChDRKruo38B2cVMbhecNmrfs0Xrlwztqzt6cnAeXvKGeTe4oZ5vG FfvO5gd9GHpA7mnEGs9bMPor5AH8sAJVBL3kdAgAVIMOQb90jseOWQ//OFsapUuiV0ba lIioi/Xu76pZTPOxZpGmgdrYHDIu3YKKHW4W8czi3S0+ALH5MsNk7b4A7beezI/Uaz6I NSVeO1UhOn2E7d5YKk5YjJf7QG8tWoHuBvSMzlKju6MBAO+IggCEQ1W3hwrSA7AnbAr4 1YFcOrX1m7gsMPLtb716PzzcZElFAZeeFWT2ayHeJlBq6nYag1Gpg5ZHNAoHrnZCfTP5 FUEQ==; darn=isar-build.org ARC-Authentication-Results: i=2; gmr-mx.google.com; dkim=pass header.i=@siemens.com header.s=fm1 header.b=iTMiieUv; spf=pass (google.com: domain of fm-1325773-20240405163220b4bfc19647cb4060b4-djltbs@rts-flowmailer.siemens.com designates 185.136.64.228 as permitted sender) smtp.mailfrom=fm-1325773-20240405163220b4bfc19647cb4060b4-dJlTBs@rts-flowmailer.siemens.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=siemens.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlegroups.com; s=20230601; t=1712334745; x=1712939545; darn=isar-build.org; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:reply-to :x-original-authentication-results:x-original-sender:feedback-id :content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=XTSs71MpwrXXIIYorPQBgqXnIvX4AAIxNG17+My2xbA=; b=cL0stS9Et92xh0Px8gA0NXS/qHF1aoshrVc2/o3lu0f5WA8fHAT1vANW0I1mdz+mR/ gRIjAiZ2zIot+CtJPKa4AQSmHoAXCWyvj1pmTi/xHO8mA9Dh6o9QWQMNDvgIkL5mSPzd gj9P160jG7A9SCetvFui7wF5v7i1HXroUbbsoU2vOCLNBacCsE6cqv/vvKn8aOp52eea pdonvK4NLNn3CqAzSyLEZzlW3AmBtEq9pf+Kevv5sPgz/4eAPqVdU0uN27QpH0i+a4q1 Cwuc4dWUtlSbtloYMbMdIPn0hCHDfRUG+sdiu2i3eEPM2qPTn0C5vwGnlq8UykOWHuzI AjxA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1712334745; x=1712939545; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :x-spam-checked-in-group:list-id:mailing-list:precedence:reply-to :x-original-authentication-results:x-original-sender:feedback-id :content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-beenthere:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=XTSs71MpwrXXIIYorPQBgqXnIvX4AAIxNG17+My2xbA=; b=YtCU7FLBuMRMb3pJJuWYlB16+r2puek1elJmm50OOaLpbISFp0BkjAUGBPJenBp5F/ 9A23jaocevGRJUR7zwAP+F+o8oYyVnv6Vjd6zKP5V9AIkQD+G2vZj9XKh7uuqwKrbVy9 s3A21+VS77Y7ZNIUqltwf/zZ2ZejdPrhzD5UA/50ToCR1tNNRPLgiFdlZs0VIZFnNujR dQDi4tI3CWp16Tt7iwH45Kd1Ppd5xkTrd0JK4TNDhNXzHcnHg0/F6ZZl1hBU6YSvs0TR rtQZUcKjNstiGMtq6Cd9IOnuj8IhAGp1EkVaauYbMsNpa4hK/aC/HPnL4SIfpkCwpc/k jYGA== X-Forwarded-Encrypted: i=2; AJvYcCWCfN4/FtQXgznSgkK28+OLaWxiibl8m5DZvCIn2+XFWiFXR9Kq83KzLnL0+WQm6Ku2eKBQHqryuhBNl33EMrl20+EPKFQ= X-Gm-Message-State: AOJu0YzoaafEBYBm/hiz86InAbCqwGuDZyDGKfQl5vzg/8YSGgnALDRB +3a90snDdCxqtQ1gkyFwCK+NT3ZbaKKOhtU/rtOJ7gW7yc1Qh36B X-Google-Smtp-Source: AGHT+IFfDmXtSKFnCfF9jMFXUbjSB5eivqAmmfOi1utTUAW7CCdd+MQAPmHFT1jl60r47vES70N/XA== X-Received: by 2002:a05:6a20:9718:b0:1a7:2b68:a404 with SMTP id hr24-20020a056a20971800b001a72b68a404mr1959901pzc.31.1712334745526; Fri, 05 Apr 2024 09:32:25 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a05:6a00:a08:b0:6ec:d942:1606 with SMTP id p8-20020a056a000a0800b006ecd9421606ls1672018pfh.2.-pod-prod-07-us; Fri, 05 Apr 2024 09:32:23 -0700 (PDT) X-Received: by 2002:a17:902:f611:b0:1e2:2587:d43f with SMTP id n17-20020a170902f61100b001e22587d43fmr2733857plg.36.1712334742957; Fri, 05 Apr 2024 09:32:22 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1712334742; cv=none; d=google.com; s=arc-20160816; b=eEKZvsZIymMm83Kk/ktnHvHcmQq35u2EBXcNJuIroMkAZdZzCzaYdJKbqBz9hHgZSe 1n8pAqQSx8PFl/xxy0mr3rlSbOvdgc78OD8ylCUQey7fb3tV29rbGSGs60VBBQbF5cV1 19ErgB7KGUx2WXT6d3+d80dWd9fjeRbtnsHO2FjOZ5g/ilhfXPSi2fy2L2wG3dq19S8E iT9XhG7NW6MILsvXvDLByVHrlIfxVFZSvZEApLMH0HDydiBICKEZYj86wybjKbsUx97P XP9xO/S3yFOuYtHgcB8cCgf4GNpO1iylSSJ45xDK8tXxS2RcIC0hEnNA16ckqDgPIV5U IM7w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=feedback-id:content-transfer-encoding:mime-version:message-id:date :subject:cc:to:from:dkim-signature; bh=+fV1RXziguR83yrysLAByW6r/CHw+cnS8GN4MxXn+Sk=; fh=3bh9z2BWAzGAc1byRhMi0K+arMZmX5BQHE0sOxINdyU=; b=AaHRq6JlWUw6DLOnINvwBp7ZWTgLoAXbg9qJcMq42v/IFzlODP2ZhMZms1mfKHe8Oh /LYxYklmCTF24Xs0S2Yz0YBFRk5MFcFgVsYn+O1obqKfroPjbv1nYyXFS0YXVkOevm4n XxWa5fEsK58g3NVbz0H80163Y8iQ05FTIdQzYjkK1ivorVj06fIeLBteb4TlJc+gytUF hsH08xlQBkJHNj7w98GjoDB9qL/qI/6wp+N4cI8A+d6rk4J0splnJ8U2q7S4XJWwiROL XKiQoVzsWLr4WetsngaKzuCLgYBSjGii4zvTxAkD4KUsqSX3/rJtvUGVWQsZD98jCWwv 5e9g==; dara=google.com ARC-Authentication-Results: i=1; gmr-mx.google.com; dkim=pass header.i=@siemens.com header.s=fm1 header.b=iTMiieUv; spf=pass (google.com: domain of fm-1325773-20240405163220b4bfc19647cb4060b4-djltbs@rts-flowmailer.siemens.com designates 185.136.64.228 as permitted sender) smtp.mailfrom=fm-1325773-20240405163220b4bfc19647cb4060b4-dJlTBs@rts-flowmailer.siemens.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=siemens.com Received: from mta-64-228.siemens.flowmailer.net (mta-64-228.siemens.flowmailer.net. [185.136.64.228]) by gmr-mx.google.com with ESMTPS id mp6-20020a170902fd0600b001e2c4ecc402si110212plb.8.2024.04.05.09.32.22 for (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 05 Apr 2024 09:32:22 -0700 (PDT) Received-SPF: pass (google.com: domain of fm-1325773-20240405163220b4bfc19647cb4060b4-djltbs@rts-flowmailer.siemens.com designates 185.136.64.228 as permitted sender) client-ip=185.136.64.228; Received: by mta-64-228.siemens.flowmailer.net with ESMTPSA id 20240405163220b4bfc19647cb4060b4 for ; Fri, 05 Apr 2024 18:32:20 +0200 X-Patchwork-Original-From: "chris.larson via isar-users" From: "Larson, Chris" To: isar-users@googlegroups.com Cc: Christopher Larson Subject: [PATCHv2 0/9] Add more signature cachability tests to the testsuite Date: Fri, 5 Apr 2024 16:31:26 +0000 Message-Id: <20240405163135.2987489-1-chris.larson@siemens.com> MIME-Version: 1.0 X-Flowmailer-Platform: Siemens Feedback-ID: 519:519-1325773:519-21489:flowmailer X-Original-Sender: chris.larson@siemens.com X-Original-Authentication-Results: gmr-mx.google.com; dkim=pass header.i=@siemens.com header.s=fm1 header.b=iTMiieUv; spf=pass (google.com: domain of fm-1325773-20240405163220b4bfc19647cb4060b4-djltbs@rts-flowmailer.siemens.com designates 185.136.64.228 as permitted sender) smtp.mailfrom=fm-1325773-20240405163220b4bfc19647cb4060b4-dJlTBs@rts-flowmailer.siemens.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=siemens.com X-Original-From: chris.larson@siemens.com Reply-To: chris.larson@siemens.com Precedence: list Mailing-list: list isar-users@googlegroups.com; contact isar-users+owners@googlegroups.com List-ID: X-Spam-Checked-In-Group: isar-users@googlegroups.com X-Google-Group-Id: 914930254986 List-Post: , List-Help: , List-Archive: , List-Unsubscribe: , X-Spam-Status: No, score=-1.2 required=5.0 tests=DKIMWL_WL_MED,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,MAILING_LIST_MULTI, RCVD_IN_MSPIKE_H3,RCVD_IN_MSPIKE_WL,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on shymkent.ilbers.de X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= From: Christopher Larson This series improves isar-sstate's lint command to support checking sigdata in tmp/stamps and to check for absolute paths in SRC_URI, and adds an additional test to the testsuite to exercise this capability. The new test checks for signature cachability issues much more quickly, as it does so without a full build, so it checks more target configurations. This uses bitbake's `-S none` option to avoid building anything, and then runs isar-sstate lint on the resulting sigdata in tmp/stamps. This test will accept a verbose parameter on the command-line to pass --verbose to isar-sstate lint, which will show the "other" absolute paths found, not just source and build directory. This series also fixes two of the failures which were identified by the new test. Here we can see the issues identified by the new test, which are fixed in this series: ``` $ avocado run testsuite/citest.py -t signatures --max-parallel-tasks=1 JOB ID : 3a3308967946663d9b239f638b030502fa80ef0a JOB LOG : /builder/avocado/job-results/job-2024-03-29T19.24-3a33089/job.log (1/1) testsuite/citest.py:SignatureTest.test_signature_lint: STARTED (1/1) testsuite/citest.py:SignatureTest.test_signature_lint: FAIL: Detected cachability issues (42.93 s) RESULTS : PASS 0 | ERROR 0 | FAIL 1 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 0 JOB TIME : 44.63 s Test summary: 1-testsuite/citest.py:SignatureTest.test_signature_lint: FAIL ``` Applicable section of the avocado full.log: ``` 1-testsuite/citest.py:SignatureTest.test_signature_lint: 2024-03-29 19:25:10,818 avocado.test cibase L0142 ERROR| ==== issues found in ubuntu-focal-amd64:isar-bootstrap-target:unpack (95af4581) ==== 1-testsuite/citest.py:SignatureTest.test_signature_lint: 2024-03-29 19:25:10,818 avocado.test cibase L0142 ERROR| -> path in sources-dir: SRC_URI entry "file:///home/kergoth/Code/industrial/signatures/isar/meta-isar/conf/distro/ubuntu.public.key;sha256sum=36a38199a4bf4eae1e7f574891f7dfcb79b91b87a33a499383265e1224b5e989" 1-testsuite/citest.py:SignatureTest.test_signature_lint: 2024-03-29 19:25:10,818 avocado.test cibase L0142 ERROR| ==== issues found in ubuntu-focal-amd64:isar-bootstrap-target:fetch (e8249cb2) ==== 1-testsuite/citest.py:SignatureTest.test_signature_lint: 2024-03-29 19:25:10,818 avocado.test cibase L0142 ERROR| -> path in sources-dir: SRC_URI entry "file:///home/kergoth/Code/industrial/signatures/isar/meta-isar/conf/distro/ubuntu.public.key;sha256sum=36a38199a4bf4eae1e7f574891f7dfcb79b91b87a33a499383265e1224b5e989" 1-testsuite/citest.py:SignatureTest.test_signature_lint: 2024-03-29 19:25:10,818 avocado.test cibase L0142 ERROR| ==== issues found in debian-bullseye-amd64:isar-ci-ssh-setup:install (1c0a8b21) ==== 1-testsuite/citest.py:SignatureTest.test_signature_lint: 2024-03-29 19:25:10,818 avocado.test cibase L0142 ERROR| -> path in sources-dir: TESTSUITEDIR = "/home/kergoth/Code/industrial/signatures/isar/testsuite" 1-testsuite/citest.py:SignatureTest.test_signature_lint: 2024-03-29 19:25:10,818 avocado.test cibase L0142 ERROR| found cachability issues (scanned 602 signatures) 1-testsuite/citest.py:SignatureTest.test_signature_lint: 2024-03-29 19:25:10,818 avocado.test cibase L0142 ERROR| -> absolute paths: sources-dir 3, build-dir 0, other 184 ``` If we strip off the prefix for legibility: ``` ERROR| ==== issues found in ubuntu-focal-amd64:isar-bootstrap-target:unpack (95af4581) ==== ERROR| -> path in sources-dir: SRC_URI entry "file:///home/kergoth/Code/industrial/signatures/isar/meta-isar/conf/distro/ubuntu.public.key;sha256sum=36a38199a4bf4eae1e7f574891f7dfcb79b91b87a33a499383265e1224b5e989" ERROR| ==== issues found in ubuntu-focal-amd64:isar-bootstrap-target:fetch (e8249cb2) ==== ERROR| -> path in sources-dir: SRC_URI entry "file:///home/kergoth/Code/industrial/signatures/isar/meta-isar/conf/distro/ubuntu.public.key;sha256sum=36a38199a4bf4eae1e7f574891f7dfcb79b91b87a33a499383265e1224b5e989" ERROR| ==== issues found in debian-bullseye-amd64:isar-ci-ssh-setup:install (1c0a8b21) ==== ERROR| -> path in sources-dir: TESTSUITEDIR = "/home/kergoth/Code/industrial/signatures/isar/testsuite" ERROR| found cachability issues (scanned 602 signatures) ERROR| -> absolute paths: sources-dir 3, build-dir 0, other 184 ``` The sstate/signatures tests were run for each commit in this series using `git rebase -x` to ensure it does not break a bisect. I welcome any and all feedback on this. v2 updates: - Explain the use of --dump-signatures in the docs on the lint command in isar-sstate - Explain the stamps regex the way the sstate regex is explained, for consistency - Support wildcards for the --excluded-tasks argument and default exclusions - Drop the additional default task exclusions, as this should be revisited - Fix a lint traceback if a variable's value is only whitespacxe Christopher Larson (9): isar-bootstrap: avoid forced early expansion of key vars isar-ci-ssh-setup: avoid abs path in signatures isar-sstate: lint: check for absolute paths in SRC_URI isar-sstate: lint: add support for checking stamps isar-sstate: add --excluded-tasks argument isar-sstate: fix failures if a variable is set to just whitespace cibuilder.py: add -S support to the bitbake method testsuite: add perform_signature_lint method testsuite: add signature cachability checks .../isar-ci-ssh-setup_0.1.bb | 3 + .../isar-bootstrap/isar-bootstrap.inc | 8 +- scripts/isar-sstate | 85 +++++++++++++++---- testsuite/cibase.py | 19 +++++ testsuite/cibuilder.py | 5 +- testsuite/citest.py | 22 ++++- 6 files changed, 121 insertions(+), 21 deletions(-) Signed-off-by: Christopher Larson Acked-by: felix.moessbauer