From patchwork Fri Oct 11 10:00:44 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anton Mikanovich X-Patchwork-Id: 3862 Return-Path: Received: from shymkent.ilbers.de ([unix socket]) by shymkent (Cyrus 2.5.10-Debian-2.5.10-3+deb9u2) with LMTPA; Fri, 11 Oct 2024 12:01:11 +0200 X-Sieve: CMU Sieve 2.4 Received: from mail-qt1-f184.google.com (mail-qt1-f184.google.com [209.85.160.184]) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPS id 49BA196M008701 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Fri, 11 Oct 2024 12:01:10 +0200 Received: by mail-qt1-f184.google.com with SMTP id d75a77b69052e-4604a54afccsf13973991cf.0 for ; Fri, 11 Oct 2024 03:01:10 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1728640863; cv=pass; d=google.com; s=arc-20240605; b=OMoPLnMSEU1CQ85B6kAeTH79mYTKTSTSZsvf3H0qQa8L+9SKVrMIrjE2FYKiQB4rtC VAfF0eoLu7PAV59E/x0TfRip5jxea//1zOAKUcaGqJBT0zr/ds9eyxSFdKS+lPzM8tnE 9LeRezF4u5595uaa0jeBF7uVBBk8FfmJwGuQydZMCfDW5bKBkOUrq0+5IpmBNRARbduf 7lvIL/1U1tqaczjUOSmWA5Todm2dTQrHU3AvvXARUecpPoE/x6ZPji47+4OZpIwXTYFr FbdisL7p3L4TmYs35qqeWeUG3T6uqQMwhNplPGEa9An5mDgPcdrfYN88QLsTZE3vDN1E Dxpw== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20240605; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:dkim-signature; bh=Uw9JeSEoykYDykaYbLpJ4C3+r3jn6umIavz42tu6Ep4=; fh=7IzGFiPMfuwMhe766N289imhS+TH69zg6+ObVaRwKjc=; b=e/zaUc94I5zO3CECEvx0XanN6pcONOPt9Vp0fsUkmqrgVDtacrrvPK5opPzaIiCpEY 22SKiLOiPqmMUhvly/lqdg0IVO1qDURsV2T5Zni7ZCkzqSpxnoZ9JXs37T9+dhkyIKLK pCRIiPBSqvX9yo7hbr5XdGKZyorDrnMrsyCr00POp7FAxoV7dxVsJ5Gdb28XfTvaUbTh 8rb4ojKtbhzimlElaJfMPZMTcZAX65jwYjJIUOLbEa0kW2XiVWlhKDFa8tQPcBVv5WKx hRVMWtImcs51VQ9McQ/2N4b4pFLs6vmLHm0GwoOeXlb1VJq4Cf/rHCUgyjGmmO2/TAgS cQAw==; darn=isar-build.org ARC-Authentication-Results: i=2; gmr-mx.google.com; spf=pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=amikan@ilbers.de DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlegroups.com; s=20230601; t=1728640863; x=1729245663; darn=isar-build.org; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:x-original-authentication-results :x-original-sender:mime-version:references:in-reply-to:message-id :date:subject:cc:to:from:sender:from:to:cc:subject:date:message-id :reply-to; bh=Uw9JeSEoykYDykaYbLpJ4C3+r3jn6umIavz42tu6Ep4=; b=v0f6H67uZim7CbyKEsRJVw7fkD7lB+af/Ix8DM+1NxoQ9bkWez3wf5N+ShHeo5qJfe uZLRkEGQbQyqNXrwyYnNGDgCF8r6fJE/wWdULH61hdAOhK8GcZfWO4EZwiTE5Uf2VD+8 yfEF1F4BtwkyY8XaqolZVjPLiRKU5F5XKk7QgZzQgDNKs03tdFCBZaynGT6bZs0bTZZE pf2OMwtTWpPVf1AUAYzLBD787sVRfh0DLjOmofkjrzK9lHXSiH8A7x4IWCAqftqrdXrM EwnDHxcG3ZnrDQ5pGilVxCk9RAys1M5kOSq7bDz8iczg5NrYUMURuyuEbiVg7SG3PsAv 58Vw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728640863; x=1729245663; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :x-spam-checked-in-group:list-id:mailing-list:precedence :x-original-authentication-results:x-original-sender:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :x-beenthere:x-gm-message-state:sender:from:to:cc:subject:date :message-id:reply-to; bh=Uw9JeSEoykYDykaYbLpJ4C3+r3jn6umIavz42tu6Ep4=; b=TjcvvtNYR7DrXAhTdJA1zWFGdlCZrK/iyigBeOpbht55yjTsg9X3stMR8txd3ibf47 2OKoCC+vu3s4EMsclLgQ2PPLWDasKZctHljXcJHyYKc1UC7KJ4pkjvF6E8E/XC0Pccwu jDQFiWZNUcpvRdgjCIrBUmqmc5Oj0FIDYI5Ft5e1UOrXiqZ6lOvwS649ZFfIAm2c15g3 Uw8vsQD34i6uZhK6u37MotS3rDwJy8groYU2BIAWXdLpPozNmRUCC0VGycLtY7suo2oK NLVZ8mqX4+T07dRBKKR1nrx3HOjHCh0MufeJXKb9tpu+Q7Hmvr9U5aEn4Y5eca6jyndH RXFQ== Sender: isar-users@googlegroups.com X-Forwarded-Encrypted: i=2; AJvYcCXw1E/RXgqHO+kTCbBS91qQH1/DIPUsvJ5TTzUP0yUTMnr2yYcZfaFSTml9TNijyLcGgVAFoE8=@isar-build.org X-Gm-Message-State: AOJu0YwrOXsPWbpCnkIoBVdfr9lLNRAgC97k/du6Ezz+PSGeuTRlCQXF rHzw6XIXe0amwZYv0EJPVG3519ygqQJQUGh5J0O6NTDQXjw/bIUf X-Google-Smtp-Source: AGHT+IEsAoxe5SZyElVFIdfrAjDnnT1Ph//cG2YeSVcnQRJ0h/hhkUV2aNrdU455ty0TEwE/nClKXw== X-Received: by 2002:a05:622a:305:b0:45f:873:ff5e with SMTP id d75a77b69052e-4604bc9d9cfmr26576001cf.57.1728640863117; Fri, 11 Oct 2024 03:01:03 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a05:622a:312:b0:458:2bcd:ed1 with SMTP id d75a77b69052e-4603fb37be4ls33164321cf.0.-pod-prod-05-us; Fri, 11 Oct 2024 03:01:02 -0700 (PDT) X-Received: by 2002:a05:622a:401a:b0:45b:16f5:6c1a with SMTP id d75a77b69052e-4604bbb86abmr28112111cf.16.1728640862106; Fri, 11 Oct 2024 03:01:02 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1728640862; cv=none; d=google.com; s=arc-20240605; b=L+6N1KYJ+fxQZrWy6qWB2s2xnuH4i9wEmeJiyNPXVYM4YannE/qAcotB+sKSR1T1Z6 Yc9x20uZBX93wgM4XBpJ1fADQVBAwctJbp/bVVtLqyBZZrl+kTg/dU01PTk/VUsSvwZ/ BLWdK1y1qHWMdGbsA0Rn+l98R7Kfx7SKWsx24jZg7uloyzJCIecLTGqMvk9nC+H5qbph gBlCGLAjpQudZEebowNPSvQq3tPe7kGIh35DDvCcJ7DdN+4QPTchxjGHG+TJn8kQxdMM HDOC5BebFd/1fVmvxMblZxwc2/mLzYtG+9weiUa7UeCUzw1z37K1RK+cPO1UL4HsAb9R 6UiA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20240605; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=rggpTtItiookqved/UxmACuuVr3M6G11mbLjx7k1C+c=; fh=/h9QQkzJ8EboVkWg45aWwpaUro6WMavIVd2OhN45RtE=; b=BhEPLjwjEL7M2bLfUEQg0mdUmfnXNMHbQ+jhT/Oei5bdBXuBzKQWHdqQeOR6VqGUux A6hIYsqJg9zGpJ3rys8J0QV5m0Gkxrd92rpXfcco8pSGAtbiDz1Qf4eJgI2e4EqO/Yzh 8OyZZy9FEnirrWxbj2lMIvN9xz6cFzTXOHHwJ9SnABvLWe7KHuG9aGfulj1XN453xQFB RCOI/e0nU85rh9WoqNPtExM8DuLI/I9eKiqLYPuRy30X61QyyQhHoh+bbRMHG1FzcQdy UNaszd7uV4fDpcJ1fI+LZYytv81A5u/KJt6+pNsmY9eLYkYX6WaHSvaP8GrY+wpizZYt 0VUw==; dara=google.com ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=amikan@ilbers.de Received: from shymkent.ilbers.de (shymkent.ilbers.de. [85.214.156.166]) by gmr-mx.google.com with ESMTPS id d75a77b69052e-4604be1effdsi560991cf.3.2024.10.11.03.01.01 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 11 Oct 2024 03:01:01 -0700 (PDT) Received-SPF: pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) client-ip=85.214.156.166; Received: from user-B660.promwad.corp ([159.148.83.114]) (authenticated bits=0) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPSA id 49BA0vWJ008632 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 11 Oct 2024 12:00:59 +0200 From: Anton Mikanovich To: isar-users@googlegroups.com Cc: Anton Mikanovich Subject: [PATCH v3 2/8] isar-events: Unhide mounts left behind Date: Fri, 11 Oct 2024 13:00:44 +0300 Message-Id: <20241011100050.322686-3-amikan@ilbers.de> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20241011100050.322686-1-amikan@ilbers.de> References: <20241011100050.322686-1-amikan@ilbers.de> MIME-Version: 1.0 X-Spam-Status: No, score=-4.6 required=5.0 tests=DKIMWL_WL_MED,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_EF,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,RCVD_IN_DNSWL_BLOCKED,RCVD_IN_MSPIKE_H3, RCVD_IN_MSPIKE_WL,RCVD_IN_RP_CERTIFIED,RCVD_IN_RP_RNBL,RCVD_IN_RP_SAFE, SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on shymkent.ilbers.de X-Original-Sender: amikan@ilbers.de X-Original-Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=amikan@ilbers.de Precedence: list Mailing-list: list isar-users@googlegroups.com; contact isar-users+owners@googlegroups.com List-ID: X-Spam-Checked-In-Group: isar-users@googlegroups.com X-Google-Group-Id: 914930254986 List-Post: , List-Help: , List-Archive: , List-Unsubscribe: , X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= Fail the existing build testcases on mounts left behind by calling the cleanup handler once per build (controlled by a new variable). Specifically: 1. Execute cleanup handler only once per build. 2. Output error messages in case of umount failure. 3. Unmount /proc/mounts in reverse order to unmount subdirectories first. 4. Introduce ISAR_FAIL_ON_CLEANUP bitbake variable: - 0 or unset: Output a warning, unmount, build succeeds (default). - 1: Output a warning, keep mounts left behind, build fails. Signed-off-by: Anton Mikanovich --- RECIPE-API-CHANGELOG.md | 9 +++++++ meta/classes/isar-events.bbclass | 40 +++++++++++++++++++++++++------- 2 files changed, 41 insertions(+), 8 deletions(-) diff --git a/RECIPE-API-CHANGELOG.md b/RECIPE-API-CHANGELOG.md index 8eeaf325..608d0cc3 100644 --- a/RECIPE-API-CHANGELOG.md +++ b/RECIPE-API-CHANGELOG.md @@ -665,3 +665,12 @@ package on an `amd64` host: you would expect the -native variant to produce an `amd64` package and -compat an 'armhf` package: it will however remain `arm64` and build of dependent recipes (image or dpkg) may fail because of the architecture mismatch. + +### Changes in cleanup handler + +Bitbake BuildCompleted event handler is now executed only once per build and +always outputs a warning if mounts are left behind after the build. + +Bitbake exit status depends on ISAR_FAIL_ON_CLEANUP bitbake variable: + - 0 or unset: Output a warning, unmount, build succeeds (default). + - 1: Output a warning, keep mounts left behind, build fails. diff --git a/meta/classes/isar-events.bbclass b/meta/classes/isar-events.bbclass index f5061a8b..76b713c7 100644 --- a/meta/classes/isar-events.bbclass +++ b/meta/classes/isar-events.bbclass @@ -4,6 +4,10 @@ # Copyright (C) 2015-2017 ilbers GmbH # Copyright (c) Siemens AG, 2018 +# If set to 1, the build will fail on mounts found during cleanup, +# keeping those mounts left behind +ISAR_FAIL_ON_CLEANUP ?= "0" + addhandler build_started python build_started() { @@ -48,17 +52,37 @@ python build_completed() { if not tmpdir: return + # bitbake calls cleanup for every multiconfig listed in BBMULTICONFIG plus + # one for the entire build. E.g., if BBMULTICONFIG="mc1 mc2 mc3", we call + # "bitbake mc1 mc2", the following cleanups would be called: + # "c1 c2 c3 cdefault". + # Skip running cleanup for additional multiconfigs + mc = d.getVar('BB_CURRENT_MC') + if mc != 'default': + return + + fail_on_cleanup = bb.utils.to_boolean(d.getVar('ISAR_FAIL_ON_CLEANUP')) + basepath = tmpdir + '/work/' - with open('/proc/mounts') as f: - for line in f.readlines(): - if basepath in line: - bb.debug(1, '%s left mounted, unmounting...' % line.split()[1]) - subprocess.call( - ["sudo", "umount", line.split()[1]], - stdout=subprocess.DEVNULL, - stderr=subprocess.DEVNULL, + for line in reversed(list(open('/proc/mounts'))): + if basepath not in line: + continue + msg_line = f"{line.split()[1]} left mounted" + # If bitbake is started manually, bb.warn and bb.error go to stdout; + # with bb.error, bitbake additionally fails the build. Under CI, + # bb.warn and bb.error currently go to a file. + if fail_on_cleanup: + bb.error(msg_line) + else: + msg_line += ', unmounting...' + bb.warn(msg_line) + try: + subprocess.run( + f"sudo umount {line.split()[1]}", shell=True, check=True ) + except subprocess.CalledProcessError as e: + bb.error(str(e)) # Cleanup build UUID, the next bitbake run will generate new one bb.persist_data.persist('BB_ISAR_UUID_DATA', d).clear()