From patchwork Tue Apr 2 17:28:04 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: kergoth@gmail.com X-Patchwork-Id: 166 Return-Path: Received: from shymkent.ilbers.de ([unix socket]) by shymkent (Cyrus 2.5.10-Debian-2.5.10-3+deb9u2) with LMTPA; Tue, 02 Apr 2024 19:29:05 +0200 X-Sieve: CMU Sieve 2.4 Received: from mail-pj1-f59.google.com (mail-pj1-f59.google.com [209.85.216.59]) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPS id 432HT3H8019561 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Tue, 2 Apr 2024 19:29:04 +0200 Received: by mail-pj1-f59.google.com with SMTP id 98e67ed59e1d1-29f96135606sf5334804a91.1 for ; Tue, 02 Apr 2024 10:29:04 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1712078937; cv=pass; d=google.com; s=arc-20160816; b=ZMVRcAv4UiBX7R9HaDVE3Y6CVn3Z0wDz6cHUUOCgGWto78qc2m7uGb4NSbxPwdZ7JB xuznKv8Rb5ApZgAuFGQHcQHa+Ax5OMKiqgwlS3y7hZ3wXSJAwWL4l6NTCFeQPhN1SNNQ ocMwonl3k3SZfngqIyZSy1+ulqH/BoTjAPze9VZNXb09fHne7Xx9Sihjaey/8Emk8ESH Hc70R/ZJtd3ODSVBLYY9PfNnKYT08FNacwBe45jqsF3hEkmoNfzYnCigVdskcwcirmRR FchNvYhEwujHNmwhgvMk9GVVQBb/DNa8cEi0887sRLPEmCDOkZIIO/wHofQo9oo2RKtc UeYA== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:content-transfer-encoding :mime-version:message-id:date:subject:cc:to:from:sender :dkim-signature:dkim-signature; bh=VpRNWAbEaMvDNHzmEuxpQmIk/Q4BluGzwRDmP2jtgTQ=; fh=chAZqS4aI9ESEbP2egkiLknfwCuNp89pLVrcqJdI7/c=; b=pSXE+D7jzygQ2yaaadHyVptuLDhWOd8M6jrrFlobjtEwZwEv0obhbKQvIMA+3HyRvG rlHr6UD8LIx+XAROU2UjzgMrsxlTQYtLMNelfT0qZuRO2ygeODvif4/1oSvvcjGbslz1 L3JNYET34w17Cyayan9yiHZ6xjK3ZsNJn+IYwenQgdJxt9IBNeNI73cfOCWh0+YRGDvg Mjdy4L0nwVjEHU/EjiFNrhozBOCfFvHrRIZZMRAx4C6t8sNZdhHt9mJIDcUTZIl0z1NL aDjcJ4MYO/2DVXr1HPF6BjzYSx+e/CGJ4ryrv09xc4br6hSEfhegyBBRlNFe5YaLWSwU IGQw==; darn=isar-build.org ARC-Authentication-Results: i=2; gmr-mx.google.com; dkim=pass header.i=@gmail.com header.s=20230601 header.b=iM5auxIV; spf=pass (google.com: domain of kergoth@gmail.com designates 2607:f8b0:4864:20::529 as permitted sender) smtp.mailfrom=kergoth@gmail.com; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlegroups.com; s=20230601; t=1712078937; x=1712683737; darn=isar-build.org; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:x-original-authentication-results :x-original-sender:content-transfer-encoding:mime-version:message-id :date:subject:cc:to:from:sender:from:to:cc:subject:date:message-id :reply-to; bh=VpRNWAbEaMvDNHzmEuxpQmIk/Q4BluGzwRDmP2jtgTQ=; b=PEnKGI6NgMyDZwhxBWbH6iW1Z2S1RSDjh8EADupeVlXsFgzzCce5kMOBVI8QTz8hmr WUGdgmqGzFWFreUr158PFv1dnABdDrkQx0mgmxe0ckegx7ccXpl27SiM1ah2aJADj+4z gyTDrDwZmOaipgR2W2ThzoZXQKKvyiC7JPz2FyGdzaoisXqGtrFXj+96S/Fso0lmD/lf a5Ccj5pcqT4/B4k5Vtd+jGDS+CAZ029lV8d1JpofFHbSqloykGYJ87lGXEqMlPBppSY/ 65YI1laQtOBQ9bJfUAtVYnLPj5qB2nJIDzTuaDNUfiFjfBph9wubRG0wlKmT4Cp3Rvr1 Ixuw== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1712078937; x=1712683737; darn=isar-build.org; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:x-original-authentication-results :x-original-sender:content-transfer-encoding:mime-version:message-id :date:subject:cc:to:from:from:to:cc:subject:date:message-id:reply-to; bh=VpRNWAbEaMvDNHzmEuxpQmIk/Q4BluGzwRDmP2jtgTQ=; b=iOrAZT8k5M4aQzBrkRxqDczDgJ0yvzH8p0HmNkP5xSYzAvAySac6NDfzKt9PLh8/76 EPiOdJ8D87kTUriwo+92s/Sud+ahSgjYXcIawpkJurSb+cdKBbBsnKC8Y99j8JMxbGev WrmW6rM5OlSgj25DHhDGPQa+mWfn6L5K+cy3E7ulCl5F3CCejjZjRgvbT4wqRVxF7EtL yk9/vkhbZf74etMVcjlwKsioFouqI0Y4Nx/qyYn4VdCfJpz03rZnjEowGIekVD4I4Adw yeiFzKP+OPSBA5t1p2bDxIWNBLr+qQqoWSPrqeO0dMUUn+Pe0yqFPHeqIG6jq08DkI84 R/Nw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1712078937; x=1712683737; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :x-spam-checked-in-group:list-id:mailing-list:precedence :x-original-authentication-results:x-original-sender :content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-beenthere:x-gm-message-state:sender:from:to:cc:subject :date:message-id:reply-to; bh=VpRNWAbEaMvDNHzmEuxpQmIk/Q4BluGzwRDmP2jtgTQ=; b=C3K0C9Wq7odH4/V/LtsFFDS1HFLzBCjWnre6U8UQ3+Ab4pvRsShqsX9b0uJYkd+dPC 0Bmt+ThJPtHQIn7ZteB//pX2dxB6nv+Fu+c8kXeHMGQSrkkxdax5AaFgtXn0D9gpp5a9 S99LXM3isJnCnSyMZ6xdpnqSVaoD7c6KyIn1YdyOYA/FEdtG3BNNxKEJOXPHkntRlIoc 3McXl5j8rAvUAn/8niem53jLRnY9OtUyLq2NqIn+BH0P8Y5n1pszLvRjzmfvcaPyhc8p JCHZtJiTNmIqcGOJEt2PnOaNgpJ0XMrZTKVconU1xEWgTxIqkAa46dyR07ToMZSJWZLP ntMQ== Sender: isar-users@googlegroups.com X-Forwarded-Encrypted: i=2; AJvYcCX9lwXIUBso++vcxqLsmIHQWJSL/byoYjaloDRRUgyLo9J3ka5uuuTVExZ8MGiyKNgE22Q/WFWScTySn4OGqRZJjSAX0DI= X-Gm-Message-State: AOJu0YwsTkoOPJ7ECAyXb2AhtNZ9OP6Q+dn41+l1SBXS+tCvNOgeWmB0 bAKSlFXCuNRtw+eDBRCwjPOCewbtg4OsNqdu2OWbZPJv1z5o2Agj X-Google-Smtp-Source: AGHT+IE13QNIS8ipy4YkhtI7npIfBun2+SHhUkvfLXnNffJlyh6biyXGkVxm6Sfz+zdFcBZDWaCACQ== X-Received: by 2002:a17:90b:201:b0:2a2:6e81:cc42 with SMTP id fy1-20020a17090b020100b002a26e81cc42mr310980pjb.24.1712078937360; Tue, 02 Apr 2024 10:28:57 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a17:90b:4f88:b0:2a1:f3c8:8df4 with SMTP id qe8-20020a17090b4f8800b002a1f3c88df4ls3362139pjb.2.-pod-prod-03-us; Tue, 02 Apr 2024 10:28:56 -0700 (PDT) X-Received: by 2002:a17:90a:468f:b0:2a2:11f1:8630 with SMTP id z15-20020a17090a468f00b002a211f18630mr399557pjf.5.1712078935850; Tue, 02 Apr 2024 10:28:55 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1712078935; cv=none; d=google.com; s=arc-20160816; b=qDT+QmPSDrIjWAYxUJGYcyzkgxf6XXxWmoVZdgUxgXuVVxzfmcCByvzuK7lpOuvFpx wjGhOYsj6ArDElQc8SmK0XrgP69QpWZIXEEMbUilnACXk4AOTrBLjeJKvGhJCsMCTDUi nOhSjIBw11W5vHp3ZmgPirgxj0UDcLgYT+y6gcEc2mJ5vuQwha8bgVzHCnpmKf0hC4eC WA7cA4THjbAayiF1mYb1EAzRAbGo9iiKMjJzF7McTB9vwid/rYdj7CmbbE0jCovRnCbf 0XPWx/DDZOVrdSu7Upw3ODmVgGuzjXqj19k/Cz0/kMyuYeAFTj17u9occFQSCyc7s2FM 1LaA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:dkim-signature; bh=Blp8Hggosrp44tSJ9ehomIsGFj3w9hdwq7i90GkGkQ4=; fh=3bh9z2BWAzGAc1byRhMi0K+arMZmX5BQHE0sOxINdyU=; b=h9QKV3/VdA3u7Dt2BKkDNY4n2ZKgA9FphwHfq1XD4kcuyYkMDDsVBJAWQQjE1I32/d Iv1kAZAzzdKMwxwewz8rbkoqpQ1Pl/XhGyrHQuRIb0rI+W/qI/ySbLqiiyDVyANCCUGG GMrahy0rsZbL+B4/GAfvZ4du9iE4qPEs/LAnjXxPISS+jR7oGbJON6R6MzPdVAZI/nQS /d0d/onSneJUpOQu/AoMUJWOG1sreCtSxWp5dCKQDhGB/wfCNtG1O1wayYOPivT8+LiG pO24G//o8CiaudUEVQ09rBYt4WZGP8cZrwyu8u5TdDyqNYMqNDHkyKt4B8hCqM7XroTH MqNw==; dara=google.com ARC-Authentication-Results: i=1; gmr-mx.google.com; dkim=pass header.i=@gmail.com header.s=20230601 header.b=iM5auxIV; spf=pass (google.com: domain of kergoth@gmail.com designates 2607:f8b0:4864:20::529 as permitted sender) smtp.mailfrom=kergoth@gmail.com; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from mail-pg1-x529.google.com (mail-pg1-x529.google.com. [2607:f8b0:4864:20::529]) by gmr-mx.google.com with ESMTPS id np17-20020a17090b4c5100b002a28309e16bsi11798pjb.0.2024.04.02.10.28.55 for (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 02 Apr 2024 10:28:55 -0700 (PDT) Received-SPF: pass (google.com: domain of kergoth@gmail.com designates 2607:f8b0:4864:20::529 as permitted sender) client-ip=2607:f8b0:4864:20::529; Received: by mail-pg1-x529.google.com with SMTP id 41be03b00d2f7-53fbf2c42bfso4237342a12.3 for ; Tue, 02 Apr 2024 10:28:55 -0700 (PDT) X-Received: by 2002:a17:90a:ea0b:b0:2a1:2506:b937 with SMTP id w11-20020a17090aea0b00b002a12506b937mr375371pjy.23.1712078935365; Tue, 02 Apr 2024 10:28:55 -0700 (PDT) Received: from dc4670097af0.alm.mentorg.com (nat-mam.mentorg.com. [139.181.8.34]) by smtp.googlemail.com with ESMTPSA id bt19-20020a17090af01300b002a2313f62dcsm66705pjb.0.2024.04.02.10.28.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 02 Apr 2024 10:28:55 -0700 (PDT) From: kergoth@gmail.com To: isar-users@googlegroups.com Cc: Christopher Larson Subject: [PATCH 0/9] Add more signature cachability tests to the testsuite Date: Tue, 2 Apr 2024 17:28:04 +0000 Message-Id: <20240402172813.418770-1-chris.larson@siemens.com> X-Mailer: git-send-email 2.39.2 MIME-Version: 1.0 X-Original-Sender: kergoth@gmail.com X-Original-Authentication-Results: gmr-mx.google.com; dkim=pass header.i=@gmail.com header.s=20230601 header.b=iM5auxIV; spf=pass (google.com: domain of kergoth@gmail.com designates 2607:f8b0:4864:20::529 as permitted sender) smtp.mailfrom=kergoth@gmail.com; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Precedence: list Mailing-list: list isar-users@googlegroups.com; contact isar-users+owners@googlegroups.com List-ID: X-Spam-Checked-In-Group: isar-users@googlegroups.com X-Google-Group-Id: 914930254986 List-Post: , List-Help: , List-Archive: , List-Unsubscribe: , X-Spam-Status: No, score=-0.9 required=5.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,RCVD_IN_MSPIKE_H3, RCVD_IN_MSPIKE_WL,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on shymkent.ilbers.de X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= From: Christopher Larson This series improves isar-sstate's lint command to support checking sigdata in tmp/stamps and to check for absolute paths in SRC_URI, and adds an additional test to the testsuite to exercise this capability. The new test checks for signature cachability issues much more quickly, as it does so without a full build, so it checks more target configurations. This uses bitbake's `-S none` option to avoid building anything, and then runs isar-sstate lint on the resulting sigdata in tmp/stamps. This test will accept a verbose parameter on the command-line to pass --verbose to isar-sstate lint, which will show the "other" absolute paths found, not just source and build directory. This series also fixes two of the failures which were identified by the new test. Here we can see the issues identified by the new test, which are fixed in this series: ``` $ avocado run testsuite/citest.py -t signatures --max-parallel-tasks=1 JOB ID : 3a3308967946663d9b239f638b030502fa80ef0a JOB LOG : /builder/avocado/job-results/job-2024-03-29T19.24-3a33089/job.log (1/1) testsuite/citest.py:SignatureTest.test_signature_lint: STARTED (1/1) testsuite/citest.py:SignatureTest.test_signature_lint: FAIL: Detected cachability issues (42.93 s) RESULTS : PASS 0 | ERROR 0 | FAIL 1 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 0 JOB TIME : 44.63 s Test summary: 1-testsuite/citest.py:SignatureTest.test_signature_lint: FAIL ``` Applicable section of the avocado full.log: ``` 1-testsuite/citest.py:SignatureTest.test_signature_lint: 2024-03-29 19:25:10,818 avocado.test cibase L0142 ERROR| ==== issues found in ubuntu-focal-amd64:isar-bootstrap-target:unpack (95af4581) ==== 1-testsuite/citest.py:SignatureTest.test_signature_lint: 2024-03-29 19:25:10,818 avocado.test cibase L0142 ERROR| -> path in sources-dir: SRC_URI entry "file:///home/kergoth/Code/industrial/signatures/isar/meta-isar/conf/distro/ubuntu.public.key;sha256sum=36a38199a4bf4eae1e7f574891f7dfcb79b91b87a33a499383265e1224b5e989" 1-testsuite/citest.py:SignatureTest.test_signature_lint: 2024-03-29 19:25:10,818 avocado.test cibase L0142 ERROR| ==== issues found in ubuntu-focal-amd64:isar-bootstrap-target:fetch (e8249cb2) ==== 1-testsuite/citest.py:SignatureTest.test_signature_lint: 2024-03-29 19:25:10,818 avocado.test cibase L0142 ERROR| -> path in sources-dir: SRC_URI entry "file:///home/kergoth/Code/industrial/signatures/isar/meta-isar/conf/distro/ubuntu.public.key;sha256sum=36a38199a4bf4eae1e7f574891f7dfcb79b91b87a33a499383265e1224b5e989" 1-testsuite/citest.py:SignatureTest.test_signature_lint: 2024-03-29 19:25:10,818 avocado.test cibase L0142 ERROR| ==== issues found in debian-bullseye-amd64:isar-ci-ssh-setup:install (1c0a8b21) ==== 1-testsuite/citest.py:SignatureTest.test_signature_lint: 2024-03-29 19:25:10,818 avocado.test cibase L0142 ERROR| -> path in sources-dir: TESTSUITEDIR = "/home/kergoth/Code/industrial/signatures/isar/testsuite" 1-testsuite/citest.py:SignatureTest.test_signature_lint: 2024-03-29 19:25:10,818 avocado.test cibase L0142 ERROR| found cachability issues (scanned 602 signatures) 1-testsuite/citest.py:SignatureTest.test_signature_lint: 2024-03-29 19:25:10,818 avocado.test cibase L0142 ERROR| -> absolute paths: sources-dir 3, build-dir 0, other 184 ``` If we strip off the prefix for legibility: ``` ERROR| ==== issues found in ubuntu-focal-amd64:isar-bootstrap-target:unpack (95af4581) ==== ERROR| -> path in sources-dir: SRC_URI entry "file:///home/kergoth/Code/industrial/signatures/isar/meta-isar/conf/distro/ubuntu.public.key;sha256sum=36a38199a4bf4eae1e7f574891f7dfcb79b91b87a33a499383265e1224b5e989" ERROR| ==== issues found in ubuntu-focal-amd64:isar-bootstrap-target:fetch (e8249cb2) ==== ERROR| -> path in sources-dir: SRC_URI entry "file:///home/kergoth/Code/industrial/signatures/isar/meta-isar/conf/distro/ubuntu.public.key;sha256sum=36a38199a4bf4eae1e7f574891f7dfcb79b91b87a33a499383265e1224b5e989" ERROR| ==== issues found in debian-bullseye-amd64:isar-ci-ssh-setup:install (1c0a8b21) ==== ERROR| -> path in sources-dir: TESTSUITEDIR = "/home/kergoth/Code/industrial/signatures/isar/testsuite" ERROR| found cachability issues (scanned 602 signatures) ERROR| -> absolute paths: sources-dir 3, build-dir 0, other 184 ``` The sstate/signatures tests were run for each commit in this series using `git rebase -x` to ensure it does not break a bisect. I welcome any and all feedback on this. In particular I don't love extending the hardcoded list of image tasks being ignored in isar-sstate lint, but it's a start. I'm open to suggestions. Christopher Larson (9): isar-bootstrap: avoid forced early expansion of key vars isar-ci-ssh-setup: avoid abs path in signatures isar-sstate: lint: check for absolute paths in SRC_URI isar-sstate: lint: add support for checking stamps isar-sstate: lint: ignore more image tasks isar-sstate: add --excluded-tasks argument cibuilder.py: add -S support to the bitbake method testsuite: add perform_signature_lint method testsuite: add signature cachability checks .../isar-ci-ssh-setup_0.1.bb | 3 + .../isar-bootstrap/isar-bootstrap.inc | 8 ++- scripts/isar-sstate | 60 +++++++++++++++---- testsuite/cibase.py | 19 ++++++ testsuite/cibuilder.py | 5 +- testsuite/citest.py | 22 ++++++- 6 files changed, 100 insertions(+), 17 deletions(-)