From patchwork Fri Apr 5 16:31:31 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Larson, Chris" X-Patchwork-Id: 3480 Return-Path: Received: from shymkent.ilbers.de ([unix socket]) by shymkent (Cyrus 2.5.10-Debian-2.5.10-3+deb9u2) with LMTPA; Fri, 05 Apr 2024 18:32:35 +0200 X-Sieve: CMU Sieve 2.4 Received: from mail-il1-f185.google.com (mail-il1-f185.google.com [209.85.166.185]) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPS id 435GWXJt003062 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Fri, 5 Apr 2024 18:32:34 +0200 Received: by mail-il1-f185.google.com with SMTP id e9e14a558f8ab-36a0a33c1bbsf12086225ab.2 for ; Fri, 05 Apr 2024 09:32:34 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1712334747; cv=pass; d=google.com; s=arc-20160816; b=sKefdYxB2bMV9YW/NhrZFM1SO7GDaRAEyAJJ9hEt/W0LkyuCvrrfOL31pPLZPCq6pU i+m/ZTY1JVM3yFEFJNY4hE8iu9Kr7LmWn82/pXXhLXbCbQ/6PI1lqPVTPPnoV5KswMc4 f9mf0c5StfDbiQkZ/tOjAu/EloPYmLJD3oleZTEMvd1eViuXDskm2ceqEmMlHuzpYyCN Aw7yiD5c71B3QErF6JQ7/qQt6NV57r2/IfpvUL025gOqQl8ZkmZKm8NYwwku4Wyd4iZF sc6MnDGsSPTQXxVoGMz7r/3Wte2Dg0qxEbeFG39/2v64gsdyh5mWVTF6txuK/LjVeuI8 nQTw== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:reply-to:feedback-id:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=VcK7V5ToDgf51xCSMJmWWvyT+322XAZnmAQxgUJb0TY=; fh=gQZCXYx1L/I3D4Ax//p6oN8XIG+D1zpGvpy7mGZ+Eho=; b=Vr7pDdVFazRe5qSMyU9mmCWtzTVhzhnxvWqHVO7pbQxNMvTyKS5R1udzSMSKwc+lPj Wqla0VXks6Li6cPL6ZbJRKT2IGucuKISNq13tqmsVwoaBqfxo6gGXBe9RVZ4Ozwj1qQo sV8t79Ghw8tRxb/Cc9pIFkHY0Cu0Kk/ScMW9QqPsPRMmDm8Rd1kLvWlJCsr85kACbVJh YD/Oe4rjiZPyJkHsWIh0CwblAn60bW/tSNJbX4j3ZhPe6xyhe0fEVHyYZt/418x979q3 E3gzXXA9KdZZWLI2zjGGQQgwZcPn6ssLkEIQ/oB+tHpYLUpIbT5jLBSf2Fc6J78nokLj C/5w==; darn=isar-build.org ARC-Authentication-Results: i=2; gmr-mx.google.com; dkim=pass header.i=@siemens.com header.s=fm1 header.b=R44fIvUL; spf=pass (google.com: domain of fm-1325773-20240405163225b1cf2978f16d381092-7thw9a@rts-flowmailer.siemens.com designates 185.136.64.228 as permitted sender) smtp.mailfrom=fm-1325773-20240405163225b1cf2978f16d381092-7tHw9a@rts-flowmailer.siemens.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=siemens.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlegroups.com; s=20230601; t=1712334747; x=1712939547; darn=isar-build.org; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:reply-to :x-original-authentication-results:x-original-sender:feedback-id :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:from:to:cc:subject:date:message-id:reply-to; bh=VcK7V5ToDgf51xCSMJmWWvyT+322XAZnmAQxgUJb0TY=; b=oB2XfwW6IGgXu1XFNjLXTk84Rj3j0M2qyhmrJpyQ/eta0Hw9N2WVkhg8EX//8fATs3 mGxjmX0ei0x6AyG5cn+PN9Pj662MyJqcR6RDh9yyhjKr+2OCHuSBcENQJb1SyM9J3im5 nJy7VLnKIOdRqKl4w9xk/3T08PDtv3df0L2aIFj0mj479oQnfR/o7tdb6Ime25BuBc3n Z9J4S6zVrsc/HOqfFMayWInhld5fAk+10RH+98zc2IaSraXlwSvqm9tQWpkBcDtzoFVM pyLRCCEw5nzrz7d9rhZU0WCNhnaj0eG+KMmQhx7aCDAR/QZcfpHl3GOjYEH2t7GQtVgI Vs4g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1712334747; x=1712939547; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :x-spam-checked-in-group:list-id:mailing-list:precedence:reply-to :x-original-authentication-results:x-original-sender:feedback-id :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:x-beenthere:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=VcK7V5ToDgf51xCSMJmWWvyT+322XAZnmAQxgUJb0TY=; b=Q28Dp959SUnOYdceAL2bmHFYVAT8RhYblN5DvEbhD+6DFGF/puysfO0f7m+QBj61Gd Rwu6bg3J3sZ8M5Zh0sufPv5i1TUI6VHawWPRVgpxK6U6xEOMI/kgtBwenHlkRQvccLq1 wDP5pj1KUW/9b4H2n6aZWHLFmcUsE1hMEm9RiQZaH1+XhJ99N1BamQyxK2o5puJJXrbe A7Wh2k5MRWrOyE59TI0gE6qX4UY0dAjBwC74p8grpqchIzk9pgB4NuESs06+D78+7n5q SfQvhp3OiNnTY4CjX615iGsc1gKHuV2Gzg8qQUK+s/Rutl3vgtJ9cnikjHVWG4bbzGbM BZwg== X-Forwarded-Encrypted: i=2; AJvYcCUEqMLCjqkrxQ/owzHK52PKbVPKiXgtVcEav+vK9HF8+n4TcglXeOPilHsP2ty8pFt5067EDU0bFhNAkVwdOKwttA1y2rQ= X-Gm-Message-State: AOJu0YzpE1ru9/OdRXt97oMSyTgBmqFPA9bkqriPWhiv938Fk0R/6ZYs w3bbxKAMFx3qzWUU6YTz0bs5Crpkzcc4qOTz+ISsmMtt851k2l6O X-Google-Smtp-Source: AGHT+IHxAa30GTUOfrZmM8HO/5rsrwxbEOCaAqT8OFaKV9C2j/1ptnMBCN0Fjl7FxvZ32iWKy1Vd0A== X-Received: by 2002:a92:c54b:0:b0:368:4a3d:e8c5 with SMTP id a11-20020a92c54b000000b003684a3de8c5mr2267196ilj.15.1712334747367; Fri, 05 Apr 2024 09:32:27 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a05:6e02:2206:b0:368:8295:6251 with SMTP id j6-20020a056e02220600b0036882956251ls1569854ilf.0.-pod-prod-04-us; Fri, 05 Apr 2024 09:32:26 -0700 (PDT) X-Received: by 2002:a05:6e02:2197:b0:368:82f1:802f with SMTP id j23-20020a056e02219700b0036882f1802fmr2118596ila.25.1712334746160; Fri, 05 Apr 2024 09:32:26 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1712334746; cv=none; d=google.com; s=arc-20160816; b=YXhv2iEzr/UdNFz7apRoCjY+6HasgMbG/GSZywOpOZZ5dWgM9beZEEAvtAgCbji4B3 7QqAqI5cZm8vN/+M+SOYl7rZIZe1rcq/onzHTtDDvYNnZQ7Bnud1IgNkO6I40DStV5gW 7aM5EwYm5oO/hmmyTpKiSvQTpierYO/ed5LtEIhKTtR5LqLlrsVBbVWLY5TfLNaSo8JA ACbkXXWBl8q4TLGHga1vc+ZjJtUT84DJYKjJrv73igoVC9dsKb9H37ufnecoz4VccZ5p ThkWwwJXS5XomezpxZ7i1vFXmLTEAxZ/MwqF0TA0LjuMThNGq3jE/0KKSKh+cSPU5MzS w/QA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=feedback-id:content-transfer-encoding:mime-version:references :in-reply-to:message-id:date:subject:cc:to:from:dkim-signature; bh=hh9kCRsfi7eiBKmPpoEUDqaDW8052964g5TJn1B/qcw=; fh=3bh9z2BWAzGAc1byRhMi0K+arMZmX5BQHE0sOxINdyU=; b=CJjxUxhQne5pxB9bNsDxkCLg7jgEMNx1+sLtyz3UZARZ6XBFxiBNmhwmq0oCOyEtqa Mhyh265kGAiQE/+YEYa+6ARWcTzvRh5/IA6pjBJGWoiW9L2Ag7hJxWKhxkrY6I7jjal7 eSAqwpk2NykXL2n/LkF1Ezr0y6nspYIB3z++WY6JVMh4KTr/PoXPNjLF67i2KOK3UhXZ KzGZqChg4n3mS4JxsWEJWP6Vqx4YxP/ikQ4ljgq5n3jgBC9zj8bOEb0D0Qqakcse2x13 BDVRTdPugQgXgjxibiUF1APoY2igDm4fQ9fXrY5Ui67SqrRIaZ+86/JT6RkBKxMM7ij0 IFtw==; dara=google.com ARC-Authentication-Results: i=1; gmr-mx.google.com; dkim=pass header.i=@siemens.com header.s=fm1 header.b=R44fIvUL; spf=pass (google.com: domain of fm-1325773-20240405163225b1cf2978f16d381092-7thw9a@rts-flowmailer.siemens.com designates 185.136.64.228 as permitted sender) smtp.mailfrom=fm-1325773-20240405163225b1cf2978f16d381092-7tHw9a@rts-flowmailer.siemens.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=siemens.com Received: from mta-64-228.siemens.flowmailer.net (mta-64-228.siemens.flowmailer.net. [185.136.64.228]) by gmr-mx.google.com with ESMTPS id c4-20020a63d504000000b005f0a5113629si153309pgg.3.2024.04.05.09.32.25 for (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 05 Apr 2024 09:32:26 -0700 (PDT) Received-SPF: pass (google.com: domain of fm-1325773-20240405163225b1cf2978f16d381092-7thw9a@rts-flowmailer.siemens.com designates 185.136.64.228 as permitted sender) client-ip=185.136.64.228; Received: by mta-64-228.siemens.flowmailer.net with ESMTPSA id 20240405163225b1cf2978f16d381092 for ; Fri, 05 Apr 2024 18:32:25 +0200 X-Patchwork-Original-From: "chris.larson via isar-users" From: "Larson, Chris" To: isar-users@googlegroups.com Cc: Christopher Larson Subject: [PATCH 5/9] isar-sstate: add --excluded-tasks argument Date: Fri, 5 Apr 2024 16:31:31 +0000 Message-Id: <20240405163135.2987489-6-chris.larson@siemens.com> In-Reply-To: <20240405163135.2987489-1-chris.larson@siemens.com> References: <20240405163135.2987489-1-chris.larson@siemens.com> MIME-Version: 1.0 X-Flowmailer-Platform: Siemens Feedback-ID: 519:519-1325773:519-21489:flowmailer X-Original-Sender: chris.larson@siemens.com X-Original-Authentication-Results: gmr-mx.google.com; dkim=pass header.i=@siemens.com header.s=fm1 header.b=R44fIvUL; spf=pass (google.com: domain of fm-1325773-20240405163225b1cf2978f16d381092-7thw9a@rts-flowmailer.siemens.com designates 185.136.64.228 as permitted sender) smtp.mailfrom=fm-1325773-20240405163225b1cf2978f16d381092-7tHw9a@rts-flowmailer.siemens.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=siemens.com X-Original-From: chris.larson@siemens.com Reply-To: chris.larson@siemens.com Precedence: list Mailing-list: list isar-users@googlegroups.com; contact isar-users+owners@googlegroups.com List-ID: X-Spam-Checked-In-Group: isar-users@googlegroups.com X-Google-Group-Id: 914930254986 List-Post: , List-Help: , List-Archive: , List-Unsubscribe: , X-Spam-Status: No, score=-1.2 required=5.0 tests=DKIMWL_WL_MED,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,MAILING_LIST_MULTI, RCVD_IN_MSPIKE_H2,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on shymkent.ilbers.de X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= From: Christopher Larson This allows the user to override the default lists of tasks to ignore when linting the sstate cache. Submitted at https://groups.google.com/g/isar-users/c/2NB-PXyswq8/m/Ca1TO65TAQAJ. Signed-off-by: Christopher Larson --- scripts/isar-sstate | 14 +++++++++++--- 1 file changed, 11 insertions(+), 3 deletions(-) diff --git a/scripts/isar-sstate b/scripts/isar-sstate index d68b8938..a4429b7a 100755 --- a/scripts/isar-sstate +++ b/scripts/isar-sstate @@ -128,6 +128,7 @@ apt-get install python3-botocore import argparse from collections import namedtuple import datetime +from fnmatch import fnmatchcase import os import re import shutil @@ -154,6 +155,8 @@ try: except ModuleNotFoundError: s3_supported = False +DEFAULT_IGNORED_TASKS = "rootfs_wicenv,image_wic" + SstateCacheEntry = namedtuple( 'SstateCacheEntry', 'hash path arch pn task suffix islink age size'.split()) @@ -614,6 +617,9 @@ def arguments(): parser.add_argument( '--lint-stamps', default=False, action='store_true', help="lint: assume target is a stamps directory (target must be a local path)") + parser.add_argument( + '--excluded-tasks', type=str, default=DEFAULT_IGNORED_TASKS, + help="lint: comma-separated list of tasks to ignore (default: %(default)s)") args = parser.parse_args() if args.command in 'upload analyze'.split() and args.source is None: @@ -622,6 +628,7 @@ def arguments(): elif args.command in 'info clean'.split() and args.source is not None: print(f"ERROR: '{args.command}' must not have a source (only a target)") sys.exit(1) + args.excluded_tasks = args.excluded_tasks.split(',') return args @@ -820,13 +827,14 @@ def sstate_analyze(source, target, **kwargs): print('\n'.join(out)) -def sstate_lint(target, verbose, sources_dir, build_dir, exit_code, pedantic, lint_stamps, **kwargs): +def sstate_lint(target, verbose, sources_dir, build_dir, exit_code, pedantic, lint_stamps, + excluded_tasks, **kwargs): ADDITIONAL_IGNORED_VARNAMES = 'PP'.split() # only list non-cacheable tasks here # note that these still can break caching of other tasks that depend on these. # Run in pedantic mode to also look for these issues (e.g. in image-in-image builds) # To make a build fully cacheable, avoid absolute paths in BBLAYERS - ADDITIONAL_IGNORED_TASKS = list() if pedantic else 'rootfs_wicenv image_wic'.split() + ADDITIONAL_IGNORED_TASKS = list() if pedantic else excluded_tasks if not target.exists(): print(f"WARNING: target {target} does not exist. Nothing to analyze.") return 0 @@ -840,7 +848,7 @@ def sstate_lint(target, verbose, sources_dir, build_dir, exit_code, pedantic, li hits_builddir = 0 hits_other = 0 for sig in cache_sigs.values(): - if sig.task in ADDITIONAL_IGNORED_TASKS: + if any(fnmatchcase(sig.task, pattern) for pattern in ADDITIONAL_IGNORED_TASKS): continue sig_file = target.download(sig.path)