From patchwork Thu Sep 25 06:54:21 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "cedric.hombourger@siemens.com" X-Patchwork-Id: 4376 Return-Path: Received: from shymkent.ilbers.de ([unix socket]) by shymkent (Cyrus 2.5.10-Debian-2.5.10-3+deb9u2) with LMTPA; Thu, 25 Sep 2025 08:54:53 +0200 X-Sieve: CMU Sieve 2.4 Received: from mail-oa1-f56.google.com (mail-oa1-f56.google.com [209.85.160.56]) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPS id 58P6spxC002097 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Thu, 25 Sep 2025 08:54:52 +0200 Received: by mail-oa1-f56.google.com with SMTP id 586e51a60fabf-34f747ca47esf328845fac.2 for ; Wed, 24 Sep 2025 23:54:52 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1758783286; cv=pass; d=google.com; s=arc-20240605; b=RPFyORMjaiTv0pksi3DEYRukteDpbBkFZNoC7URTrY1BcxqBRLKQBWyGCosFmjrTum 13/A5BEV3engc39VyUZK4WnYAE56MIbTJFclE7qZlFKPuY2e4su/pbPmWVXu9cQdFO0A jRWggaF7Wz87B6K/eowmJEZ12jRspDa0S2nxKzoEMv/phB0jhROC+V58LyIZcmdyJ4KM WsxQgI1MdUeH61Up2JeHzBEvsiy3P25ufY4Rvs6yeqpZ+1Z4CE9bAoq3R3rMDIIwPqcH uDuitTMkqPT7d2GruJ2Uy7KtT0EhmKSpfhzTy/NQ3NrGDgJHoNcUWvIw/7tvk66Okael UCOw== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20240605; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:reply-to:feedback-id:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=R41i/p75R333fNOeB3le5U9c9x/ZqaYsLWEtB7ty2lM=; fh=scU7IWRfW4+U8WobqjlR/C7zb/tTZLiMDqCMx6AQiF8=; b=SLGUTCs5hbkyCuTjdB7ZakggQwMzySyJSmyOF45ha0kFbjK+vXcxg5ImKgKSNP0+OT kH946zl1PtbVejb48aW6JF8xpmz9nsGun8o68/3hY7SoUo6fNUL/eiZPNR4M1BobdFd/ +a7s3XSxD6DP0lMmjH+3NR8hg0LhMQbPZ35kn4yvyRKSPwi1IBf3uQFcNT/hlsEuzlll h1bUSL/YoQxrWcMbM35EHpgm9/9IlB00OtGwJgNGqfYk33Wxt5WJug0nsrhbkJJn1R0X pEwtfJ5vrunqdxO3Z6MFqdg5IVxkq38YHd+7UbyglcARtrxuEcVhjlrirdHswxfSDX9y pZZQ==; darn=isar-build.org ARC-Authentication-Results: i=2; gmr-mx.google.com; dkim=pass header.i=@siemens.com header.s=fm1 header.b=tvk9QJry; spf=pass (google.com: domain of fm-1212295-20250925065442d1e28865bc000207cb-wduftc@rts-flowmailer.siemens.com designates 185.136.64.226 as permitted sender) smtp.mailfrom=fm-1212295-20250925065442d1e28865bc000207cb-WDuFTc@rts-flowmailer.siemens.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=siemens.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlegroups.com; s=20230601; t=1758783286; x=1759388086; darn=isar-build.org; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:reply-to :x-original-authentication-results:x-original-sender:feedback-id :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:from:to:cc:subject:date:message-id:reply-to; bh=R41i/p75R333fNOeB3le5U9c9x/ZqaYsLWEtB7ty2lM=; b=LRUnRThwo1hBZqsXE584VkPXWddp3yAoRk7+kTudZlrUtYHWb/SkRBuxY+dpdx5BzW NXkwXlba6JLV5JpVzJAwpUFoKE1ASLX+x6X1gYgBKtcppDZkBsIJRzio4UWk7rlWHGD8 HWxWbjJJ78hXefCdZiqDHkcItGrwN9N7MwFuRRe9m71RaHfArLeL0TCrdojRbDTwX0/6 WYfGIsnjlSVkelD8konhtOqbzfusfF64OI9j8IdUksw2rg3l7b1mFzgAlewOig+ROl88 u2EVD64wYx4DHWx4vFe4YtYXd+0x3S1KMvPjIFw5L9Xqyvncl84rj4ONpvmhagB2IX4X NK/Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1758783286; x=1759388086; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :x-spam-checked-in-group:list-id:mailing-list:precedence:reply-to :x-original-authentication-results:x-original-sender:feedback-id :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:x-beenthere:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=R41i/p75R333fNOeB3le5U9c9x/ZqaYsLWEtB7ty2lM=; b=ObcXOsVj0yOyRsqsfHFjdXKbkt01UtceJP+cj23UQUcjYhPdj5uaEshudOxZ6FNYNh IbqLWxaXGCILmjQGOKjuUP6cjQrR8ZuoEqxP+GXzMzR+UTtvsjzxAylnkRINw3R2TlcR Xe4dPI4nAUQl9QW/sal/uy07eRRPesY/DYSEE4c14Jylz6FqEcrOfLnzRHaZz/zB2ayc vKy2Z2kK7o7yo05KHLn76iXBCqftluC0ctG0xN9xee5gtaRfSjuhqGAeUfkAWk8idXwm pZ8VFdlGxM8/KYbkuK2AUjSPrpVYt+TfnUq6rOB+xAtCEtkMDhuifhUiqj2hQ5hLBtbK iNJg== X-Forwarded-Encrypted: i=2; AJvYcCVzMtGnxUe7df2+3I38b44KGuO0pMZe2cgp/W0e71GxVbBYIyhiIjWdH9rRBtk7yOYNjThVFQ0=@isar-build.org X-Gm-Message-State: AOJu0YwzcBPrb1+VgPXUGeIi+X051TOyLwng2sSrK870/+67mNWkc0q0 PMN4DV/Oh3tQ2kGOa+JMWEO+Uid1IkOtIQVPyutpUIMFf1kSuZpmFfhF X-Google-Smtp-Source: AGHT+IFaba/6KVkMKS9dYKNuufxKl5Ik7AIHiSNH3P8fBofaUIzWQhtUKsTKQwQkZvS83vmdGJO8+g== X-Received: by 2002:a05:6870:2199:b0:30b:cb2f:bae4 with SMTP id 586e51a60fabf-35ebf3f478bmr1016573fac.12.1758783286111; Wed, 24 Sep 2025 23:54:46 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com; h="ARHlJd6aRN7FSodpgpnfgNZub5UtGlF57S31beXYwfZ2w6NWuQ==" Received: by 2002:a05:6871:c687:b0:315:531e:fdba with SMTP id 586e51a60fabf-35eef9c998cls250353fac.1.-pod-prod-02-us; Wed, 24 Sep 2025 23:54:45 -0700 (PDT) X-Received: by 2002:a05:6870:d306:b0:346:865e:d044 with SMTP id 586e51a60fabf-35ebf3f130dmr946432fac.11.1758783284843; Wed, 24 Sep 2025 23:54:44 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1758783284; cv=none; d=google.com; s=arc-20240605; b=VyXnpa1P3VfB+InbXqXABgJFaQCQze7YzPjNv0BiY+5qiPuE3bannEKaxa7dVr9J6J EPWTEd+1VjTJHPYoWYZnpz5As6YxPHgARhH9ZANP7bYu3kFK49a9wKRfe5Aug6oYM1uz Ti2LEYd+ghAKNvZZZkJbGab7S8XCvSK57OAYlw8EGDRnbKlpcXhYgrmae0CsXWeWIwgC ywV4lFjkXgvtxcKUJtTlWd2JLA6XDlgwRCYqfqn8PnsApk8OOsBigog3U6XgLKxSJiEY 1NFRGNTuXoGUEoJRwHedEvSI+Zzk8KLqHVyi31PLOqbJ3yyiqzt8cl5nWVxPO+tl9aGY PPGg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20240605; h=feedback-id:content-transfer-encoding:mime-version:references :in-reply-to:message-id:date:subject:cc:to:from:dkim-signature; bh=Z6WOuK2vtn4PCzrtIVXEPUdXQE8WcnGh+X4uoipbsg0=; fh=GJh20mB+jF6oQ+me1R+hLO+vPKOaUsE8susJALlxkPc=; b=X1F6lrvTE7exh+DwyyPZlo6v4FKR9EES2VDo493jVX6LFHU1BfQdaX+l3VKn8pxE19 Jh+0pBInAt58ag1rbaSfsBm5rBP6dumaCrMO/BNrkzxksa+sn6FUKduVR6CIzW16Re7H xOeYIALYArS9053/ALYHCGgAo8GuPtTKQUFU868yX32/0d5RDkPOJnUdEWu0K8zX6jJZ yGFqC6QGNSPU2yG+OpZeO4YdHQ3jz2edQfcJC122FI4uf//2BzkJbFVgXD2vxxqRZU+X F92pDNatDEF860PDW56jGFvRrWlry5hLImjzuYRxKdFa69quYXySdFnB6n9Sd44bvPsX 9MGg==; dara=google.com ARC-Authentication-Results: i=1; gmr-mx.google.com; dkim=pass header.i=@siemens.com header.s=fm1 header.b=tvk9QJry; spf=pass (google.com: domain of fm-1212295-20250925065442d1e28865bc000207cb-wduftc@rts-flowmailer.siemens.com designates 185.136.64.226 as permitted sender) smtp.mailfrom=fm-1212295-20250925065442d1e28865bc000207cb-WDuFTc@rts-flowmailer.siemens.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=siemens.com Received: from mta-64-226.siemens.flowmailer.net (mta-64-226.siemens.flowmailer.net. [185.136.64.226]) by gmr-mx.google.com with ESMTPS id 586e51a60fabf-363b4995370si42775fac.4.2025.09.24.23.54.44 for (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Wed, 24 Sep 2025 23:54:44 -0700 (PDT) Received-SPF: pass (google.com: domain of fm-1212295-20250925065442d1e28865bc000207cb-wduftc@rts-flowmailer.siemens.com designates 185.136.64.226 as permitted sender) client-ip=185.136.64.226; Received: by mta-64-226.siemens.flowmailer.net with ESMTPSA id 20250925065442d1e28865bc000207cb for ; Thu, 25 Sep 2025 08:54:42 +0200 X-Patchwork-Original-From: "'Cedric Hombourger' via isar-users" From: "cedric.hombourger@siemens.com" To: isar-users@googlegroups.com Cc: "cedric.hombourger@siemens.com" Subject: [PATCH v4 1/4] rootfs: introduce wrapper to run commands against a rootfs Date: Thu, 25 Sep 2025 08:54:21 +0200 Message-ID: <20250925065433.4180883-2-cedric.hombourger@siemens.com> In-Reply-To: <20250925065433.4180883-1-cedric.hombourger@siemens.com> References: <20250925065433.4180883-1-cedric.hombourger@siemens.com> MIME-Version: 1.0 X-Flowmailer-Platform: Siemens Feedback-ID: 519:519-1212295:519-21489:flowmailer X-Original-Sender: cedric.hombourger@siemens.com X-Original-Authentication-Results: gmr-mx.google.com; dkim=pass header.i=@siemens.com header.s=fm1 header.b=tvk9QJry; spf=pass (google.com: domain of fm-1212295-20250925065442d1e28865bc000207cb-wduftc@rts-flowmailer.siemens.com designates 185.136.64.226 as permitted sender) smtp.mailfrom=fm-1212295-20250925065442d1e28865bc000207cb-WDuFTc@rts-flowmailer.siemens.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=siemens.com X-Original-From: Cedric Hombourger Reply-To: Cedric Hombourger Precedence: list Mailing-list: list isar-users@googlegroups.com; contact isar-users+owners@googlegroups.com List-ID: X-Spam-Checked-In-Group: isar-users@googlegroups.com X-Google-Group-Id: 914930254986 List-Post: , List-Help: , List-Archive: , List-Unsubscribe: , X-Spam-Status: No, score=-4.9 required=5.0 tests=DKIMWL_WL_MED,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,MAILING_LIST_MULTI, RCVD_IN_DNSWL_BLOCKED,RCVD_IN_MSPIKE_H3,RCVD_IN_MSPIKE_WL, RCVD_IN_RP_CERTIFIED,RCVD_IN_RP_RNBL,RCVD_IN_RP_SAFE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on shymkent.ilbers.de X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= From: "cedric.hombourger@siemens.com" "sudo chroot" is used in several places to run commands inside rootfs directories constructed by Isar. There are cases where a command could be used without elevated privileges as long as special folders such as /isar-apt are mounted (they are often referenced as /isar-apt in configuration files found in the target rootfs). For such cases, bubblewrap may be used to create a non-privileged namespace (either in a bare/native environment or within a docker/podman container) where the command will be executed as if chroot had been used. The rootfs may also be the host root file-system: this should however be used with care to avoid host contamination problems (note: Isar already relies on a number of host tools). Signed-off-by: Cedric Hombourger --- RECIPE-API-CHANGELOG.md | 8 +++++ doc/user_manual.md | 1 + meta/classes/rootfs.bbclass | 67 +++++++++++++++++++++++++++++++++++++ 3 files changed, 76 insertions(+) diff --git a/RECIPE-API-CHANGELOG.md b/RECIPE-API-CHANGELOG.md index 92e7811c..53e650d4 100644 --- a/RECIPE-API-CHANGELOG.md +++ b/RECIPE-API-CHANGELOG.md @@ -741,3 +741,11 @@ By setting `MS_TPM_20_REF_DIR` in an optee-ftpm recipe, it is now possible to use the new optee_ftpm code base from the OP-TEE project. That variable has to point to a subdir in `WORKDIR` which contains the unpacked ms-tpm-20-ref source code. + +### Require bubblewrap to run non-privileged commands with bind-mounts + +Isar occasionally needs to run commands within root file-systems that it +builds and with several bind-mounts (e.g. /isar-apt). bubblewrap may be +used in Isar classes instead of `sudo chroot` to avoid unecessary privilege +elevations (when we "just" need to chroot but do not require root). It is +pre-installed in kas-container version 4.8 (or later). diff --git a/doc/user_manual.md b/doc/user_manual.md index 67f91973..be89ce1d 100644 --- a/doc/user_manual.md +++ b/doc/user_manual.md @@ -75,6 +75,7 @@ Install the following packages: ``` apt install \ binfmt-support \ + bubblewrap \ bzip2 \ mmdebstrap \ arch-test \ diff --git a/meta/classes/rootfs.bbclass b/meta/classes/rootfs.bbclass index ebe3bf4a..f740c6e1 100644 --- a/meta/classes/rootfs.bbclass +++ b/meta/classes/rootfs.bbclass @@ -34,6 +34,73 @@ export LANG = "C" export LANGUAGE = "C" export LC_ALL = "C" +# Execute a command against a rootfs and with isar-apt bind-mounted. +# Additional mounts may be specified using --bind and a +# custom directory for the command to be executed with --chdir . The +# command is assumed to follow the special "--" argument. This would replace +# "sudo chroot" calls especially when a native command may be used instead of +# chroot'ed command and without elevated privileges (the command will likely +# take the rootfs as argument; e.g. apt-get -o Dir=${ROOTFSDIR}). If the +# optional rootfs argument is omitted, the host rootfs will be used (e.g. to +# run native commands): this should be used with care. +# +# Usage: rootfs_cmd [options] [rootfs] -- command +# +rootfs_cmd() { + set -- "$@" + bwrap_args="--bind ${REPO_ISAR_DIR}/${DISTRO} /isar-apt" + bwrap_binds="" + bwrap_rootfs="" + + while [ "${#}" -gt "0" ] && [ "${1}" != "--" ]; do + case "${1}" in + --bind) + if [ "${#}" -lt "3" ]; then + bbfatal "--bind requires two arguments" + fi + bwrap_binds="${bwrap_binds} --bind ${2} ${3}" + shift 3 + ;; + --chdir) + if [ "${#}" -lt "2" ]; then + bbfatal "${1} requires an argument" + fi + bwrap_args="${bwrap_args} ${1} ${2}" + shift 2 + ;; + -*) + bbfatal "${1} is not a supported option!" + ;; + *) + if [ -z "${bwrap_rootfs}" ]; then + bwrap_rootfs="${1}" + shift + else + bbfatal "unexpected argument '${1}'" + fi + ;; + esac + done + + if [ -n "${bwrap_rootfs}" ]; then + bwrap_args="${bwrap_args} --bind ${bwrap_rootfs} /" + fi + + if [ "${#}" -le "1" ] || [ "${1}" != "--" ]; then + bbfatal "no command specified (missing --)" + fi + shift # remove "--", command and its arguments follows + + for ro_d in bin etc lib lib64 sys usr var; do + [ -d ${bwrap_rootfs}/${ro_d} ] || continue + bwrap_args="${bwrap_args} --ro-bind ${bwrap_rootfs}/${ro_d} /${ro_d}" + done + + bwrap --unshare-user --unshare-pid ${bwrap_args} \ + --dev-bind /dev /dev --proc /proc --tmpfs /tmp \ + ${bwrap_binds} -- "${@}" +} + rootfs_do_mounts[weight] = "3" rootfs_do_mounts() { sudo -s <<'EOSUDO' From patchwork Thu Sep 25 06:54:22 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "cedric.hombourger@siemens.com" X-Patchwork-Id: 4375 Return-Path: Received: from shymkent.ilbers.de ([unix socket]) by shymkent (Cyrus 2.5.10-Debian-2.5.10-3+deb9u2) with LMTPA; Thu, 25 Sep 2025 08:54:53 +0200 X-Sieve: CMU Sieve 2.4 Received: from mail-qv1-f60.google.com (mail-qv1-f60.google.com [209.85.219.60]) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPS id 58P6sp7Q002095 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Thu, 25 Sep 2025 08:54:52 +0200 Received: by mail-qv1-f60.google.com with SMTP id 6a1803df08f44-78ea15d3548sf6000496d6.1 for ; Wed, 24 Sep 2025 23:54:52 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1758783286; cv=pass; d=google.com; s=arc-20240605; b=Sfeq1nRCFRCJsvRt+2+Pyakfivk8teVeKhGz3u3FO04pOFQKwg68RjIr6WMumnSXc/ SFCe7q9cUL3BdcsU33qgSoCXT/hdYtLDLIANfky83UtOMkwLfEMo+yBf6q/By0YKhVGE KbeGLmh2YRssBK3YpfFO5S7I3/RsLiEyN7IkNv9sghCaWxHWA0m4Mdw8Py0XWGj8izl7 JkKjhqEZ7EA6E50Nae+xl33ZBEGs2EOECny42ZLezcmKS+pPWu4Dq/WSMnjvFF8pzNfZ 7bv684sIdLUzYpglSEh0TBzD4MOhmLJLOPmqQlHYEkRA0n3oithSd7/4tX5WdbCjrCmF AD9Q== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20240605; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:reply-to:feedback-id:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=dxQxB3b/kgeO8JFoU+eboZZJNuWc6TcvHJdoBPTmRUw=; fh=HkM6Fbs97+oxq0RJthArMZGsB/RLyAQSCmrq3paIY1s=; b=RwKev2lYU0mQZ7OrluGAUj8lH6dqfsEKhJemWV782V4U2KUnBGQiAX86wpiQZCiO87 aIo2gOLfCDKNjZdPWEVjD2MACld0P4cnqulEh2hOINA+sFBCW1OVWsiikEtxvES9njhn SEI4cuDFSva3dB0gqDMBrxZDty0N3/QqSf+2Bp396YNnfDu38byHgqlM5EH+dUNg6fEy CIdego+6aSbeZQB4kVGhr1ACczDZs9qjmkVsb+1RnP9HnoSrDt96675/0MDWlgG8bA0Y ppEeFoS7SzWDn+gcaZ5hDYN2W+r/F7/PFv5/6a7fqeTFYoPa0tRWvcj8BZM7F5nrMD/4 NQBA==; darn=isar-build.org ARC-Authentication-Results: i=2; gmr-mx.google.com; dkim=pass header.i=@siemens.com header.s=fm1 header.b=XJ0TK+TU; spf=pass (google.com: domain of fm-1212295-20250925065442cda6ee4b9f0002079f-atmcct@rts-flowmailer.siemens.com designates 185.136.64.228 as permitted sender) smtp.mailfrom=fm-1212295-20250925065442cda6ee4b9f0002079f-ATmcCt@rts-flowmailer.siemens.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=siemens.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlegroups.com; s=20230601; t=1758783286; x=1759388086; darn=isar-build.org; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:reply-to :x-original-authentication-results:x-original-sender:feedback-id :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:from:to:cc:subject:date:message-id:reply-to; bh=dxQxB3b/kgeO8JFoU+eboZZJNuWc6TcvHJdoBPTmRUw=; b=uv0GxkDbHHbOmzDieetPL1ZtQb6BJgXlRYvjmRx/Cdt3DC/qFEeBkBn69P5cftAHwG WBbMDGMep26Rw9Pl6HwkDYEQ9sYvfqrRnHKMx0TndEghz8gam7NO3RoV68nvukTby7vr fBQSs7xtFpjIkAOEI4wnb/hQJKiiozEwPsyj1la0IlYgXFaooPu+4vZi04OMJZumSX55 uYQ5GygRCHnYRw5DTjpVQF9ASNxV0RrB6qqgAheSddLtNx1UhLjjmZj4X5+72A+mIKPB sGG5C97Ir1fq68WuEKlLePrWJLhgaRxYP7+b2Ty8eod5fPZliRla1MEjgFU8PUnLd3i2 /fIw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1758783286; x=1759388086; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :x-spam-checked-in-group:list-id:mailing-list:precedence:reply-to :x-original-authentication-results:x-original-sender:feedback-id :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:x-beenthere:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=dxQxB3b/kgeO8JFoU+eboZZJNuWc6TcvHJdoBPTmRUw=; b=Fkp7a13b0JqtR4IpAL9JzpRVT3u8qKzzY5VsU1gxdMI1+hGrupmgn1EwX4XThL2410 xBMc6HyxPFevZ8NKBfltncOKT+ByPlc4QypfEvizHZEbinP6X3frB+PulheQ9+1URf0Q uBYtSTEyqlE2kVM+4SnZYEKke58cAxf/aCzFnkdmM7LmT34HPUTLsA/N5Lq3fRkWdDlz oh8S2ktG/hWsfrgb/c4X+VY06c9r0w2RabChWyFNgzixJqan6oDIRazpwWP+R4Gl1UHv oz+9kxKFItHQxWg4Bsl6PTUK4BqgNT1SRXte9OYSH+O2LWY4EPbQznxUg9UcIigwxLeE UBng== X-Forwarded-Encrypted: i=2; AJvYcCX+M6mkRKMet4rqKtnGTLDJXobziWwebrhisze4HpNVAdYP0JKVkbA803R5al0InkC53fbFq/Y=@isar-build.org X-Gm-Message-State: AOJu0YxSZlwhftNcVs4IEmMrfTOKwgjuDrNmP2EVn0RRySkChIdutRTU 1r+z/75mtTnTMJTDWkyS11vPLPetYIouWC3qkHQgV/OyiM3uxsScBcM7 X-Google-Smtp-Source: AGHT+IFuyck1rbYQ3fvOmE0mCYzjIGRfaGX6M/CElCZv5ko9XdHNizF51y82gD62a4Pamcaq+pxELA== X-Received: by 2002:a05:6214:6117:b0:807:ca4f:5661 with SMTP id 6a1803df08f44-807ca4f588dmr5570306d6.63.1758783285830; Wed, 24 Sep 2025 23:54:45 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com; h="ARHlJd7TWIPDkDI/NLanCaqCZ2ItDLW18GzMuPCpGuQwDRigKw==" Received: by 2002:ad4:46c7:0:b0:70b:acc1:ba4f with SMTP id 6a1803df08f44-7fd7d048470ls8936276d6.1.-pod-prod-08-us; Wed, 24 Sep 2025 23:54:44 -0700 (PDT) X-Received: by 2002:a05:6214:5018:b0:7ea:9df1:78de with SMTP id 6a1803df08f44-7fc3d6859b2mr29862496d6.46.1758783284785; Wed, 24 Sep 2025 23:54:44 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1758783284; cv=none; d=google.com; s=arc-20240605; b=bNHXXrxolF1rLgzOFvxoklGSTAlgft+YDHRPLTDNqhlKz1R0558VdRbag/xTYZz4dF IWjGbe0kjZQMsR7ncX5SrdFcdrgNE/8XHg6KZnsbVfsBuBAdIrvtwKwFjuYBsluWQoxK zhYNVh0qNhy/QktT8aPGBYV3RESU3mLuVd+1KMjSJmQPIOeFunvAYRRkIJWK6AUiRc8m sIOOiV93FDFhzVVgHqAetrla7wzg9WD235SdcqrHtEe/WHM34xjpcP0ELNG1SbVEd6Zy JEy705UPtYeTOq0wkFwKpMYph9J7qabSyy9m9Gx02k616KysLaWRca3SuIqg7l6TZROn zHmQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20240605; h=feedback-id:content-transfer-encoding:mime-version:references :in-reply-to:message-id:date:subject:cc:to:from:dkim-signature; bh=BRiD8X4g3wKQTvAoxp17V2dY9+s3wi0V2twfBcf6teU=; fh=GJh20mB+jF6oQ+me1R+hLO+vPKOaUsE8susJALlxkPc=; b=byuyjIoygw+vN1i3Us85OKMpxP+EtqSDptNHoY1V0heVDipJfKAs+tkWeuoLIvl4Vu EeogJB/uv0tBXKT49BX37ysYx0DqeUh8YX1o/AnUlRmXWgPy64xUvKLl+CzJTXLqaleu rzEFpB5Rj2SuI+YO4v67bRqotBANAI+2aUhUCd/lTIX1qr27CGEEIXq9k3gkf3MfvRrC gVD2ebCFp91KvRwUY1K3DwEszay10p6sngonSV/TNyWIeC9Ga3Vice/5oCHt73HwXzmh Gr4JjFqTQlt8C24CDHAvhTPntvEAgA5NvfnIjOoagE/J0L7DyrE67ExeSwtctWNX/Luu w25A==; dara=google.com ARC-Authentication-Results: i=1; gmr-mx.google.com; dkim=pass header.i=@siemens.com header.s=fm1 header.b=XJ0TK+TU; spf=pass (google.com: domain of fm-1212295-20250925065442cda6ee4b9f0002079f-atmcct@rts-flowmailer.siemens.com designates 185.136.64.228 as permitted sender) smtp.mailfrom=fm-1212295-20250925065442cda6ee4b9f0002079f-ATmcCt@rts-flowmailer.siemens.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=siemens.com Received: from mta-64-228.siemens.flowmailer.net (mta-64-228.siemens.flowmailer.net. [185.136.64.228]) by gmr-mx.google.com with ESMTPS id 6a1803df08f44-8016e3993bdsi425896d6.7.2025.09.24.23.54.44 for (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Wed, 24 Sep 2025 23:54:44 -0700 (PDT) Received-SPF: pass (google.com: domain of fm-1212295-20250925065442cda6ee4b9f0002079f-atmcct@rts-flowmailer.siemens.com designates 185.136.64.228 as permitted sender) client-ip=185.136.64.228; Received: by mta-64-228.siemens.flowmailer.net with ESMTPSA id 20250925065442cda6ee4b9f0002079f for ; Thu, 25 Sep 2025 08:54:42 +0200 X-Patchwork-Original-From: "'Cedric Hombourger' via isar-users" From: "cedric.hombourger@siemens.com" To: isar-users@googlegroups.com Cc: "cedric.hombourger@siemens.com" Subject: [PATCH v4 2/4] deb-dl-dir: optimize caching of source packages using apt natively Date: Thu, 25 Sep 2025 08:54:22 +0200 Message-ID: <20250925065433.4180883-3-cedric.hombourger@siemens.com> In-Reply-To: <20250925065433.4180883-1-cedric.hombourger@siemens.com> References: <20250925065433.4180883-1-cedric.hombourger@siemens.com> MIME-Version: 1.0 X-Flowmailer-Platform: Siemens Feedback-ID: 519:519-1212295:519-21489:flowmailer X-Original-Sender: cedric.hombourger@siemens.com X-Original-Authentication-Results: gmr-mx.google.com; dkim=pass header.i=@siemens.com header.s=fm1 header.b=XJ0TK+TU; spf=pass (google.com: domain of fm-1212295-20250925065442cda6ee4b9f0002079f-atmcct@rts-flowmailer.siemens.com designates 185.136.64.228 as permitted sender) smtp.mailfrom=fm-1212295-20250925065442cda6ee4b9f0002079f-ATmcCt@rts-flowmailer.siemens.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=siemens.com X-Original-From: Cedric Hombourger Reply-To: Cedric Hombourger Precedence: list Mailing-list: list isar-users@googlegroups.com; contact isar-users+owners@googlegroups.com List-ID: X-Spam-Checked-In-Group: isar-users@googlegroups.com X-Google-Group-Id: 914930254986 List-Post: , List-Help: , List-Archive: , List-Unsubscribe: , X-Spam-Status: No, score=-4.9 required=5.0 tests=DKIMWL_WL_MED,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,MAILING_LIST_MULTI, RCVD_IN_DNSWL_BLOCKED,RCVD_IN_MSPIKE_H2,RCVD_IN_RP_CERTIFIED, RCVD_IN_RP_RNBL,RCVD_IN_RP_SAFE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on shymkent.ilbers.de X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= From: "cedric.hombourger@siemens.com" source package are downloaded by entering the target rootfs and run apt there. For foreign architectures, this results in apt being executed under QEMU and leads to poor performance. By using the recently introduced rootfs_native_cmd command wrapper, apt will be executed natively against the target rootfs and without elevated privileges. For our test work-load, caching was reduced from more than 10 hours to an hour. Performance is also more consistent as it will no longer depend as to when bitbake kicks caching of source packages for foreign architecture rootfs vs rootfs for the host (in multiconfig builds). Signed-off-by: Cedric Hombourger --- meta/classes/deb-dl-dir.bbclass | 37 ++++++++------------------------- 1 file changed, 9 insertions(+), 28 deletions(-) diff --git a/meta/classes/deb-dl-dir.bbclass b/meta/classes/deb-dl-dir.bbclass index 4780be20..ea0ed3d2 100644 --- a/meta/classes/deb-dl-dir.bbclass +++ b/meta/classes/deb-dl-dir.bbclass @@ -5,25 +5,6 @@ inherit repository -debsrc_do_mounts() { - sudo -s <"${DEBSRCDIR}/${rootfs_distro}.lock" - - debsrc_undo_mounts "${rootfs}" } dbg_pkgs_download() { From patchwork Thu Sep 25 06:54:23 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "cedric.hombourger@siemens.com" X-Patchwork-Id: 4377 Return-Path: Received: from shymkent.ilbers.de ([unix socket]) by shymkent (Cyrus 2.5.10-Debian-2.5.10-3+deb9u2) with LMTPA; Thu, 25 Sep 2025 08:54:55 +0200 X-Sieve: CMU Sieve 2.4 Received: from mail-yb1-f189.google.com (mail-yb1-f189.google.com [209.85.219.189]) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPS id 58P6sro4002123 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Thu, 25 Sep 2025 08:54:54 +0200 Received: by mail-yb1-f189.google.com with SMTP id 3f1490d57ef6-eb0718279d2sf691567276.3 for ; Wed, 24 Sep 2025 23:54:54 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1758783287; cv=pass; d=google.com; s=arc-20240605; b=FMfxDa5loLl52gfeB2oJLHY3m8A0nEidnxjOiBYCtXoVADSJzDkDD1GyzkU733w0YR XxzP2T1impRmxDu31Dr2qTudsG69ENhTxaIlYVJtNz8N9jgjVyipn7aGq1GnRgoxKW/Y I3+xrhxODbQuPhEJIap+KnDRC5nOslqALdKById7OJ3ACcnNcMudlkKQwMJCiWfAIkQF 6tJrTSQkEyc+fMwAmZiS9Wt7PFEvRKlX8vAw176N5QENsvnHA6rhdt8DNbEeWTUmV+X+ 4yLQ4s6qGTKmaxo3gI25JXBIMwemq8uHLGWLXkcwMBLxNlcNR5M05HDskW1Om0Bkw9Sx wAhQ== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20240605; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:reply-to:feedback-id:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=e+I1/EeLJRMir3sFwe5PQHNtfAVebKPzHRiyQhXsE3U=; fh=zAR3sTIEOLsLnaijWfqRRcQqnHSUTbUuJhmx7l67m9M=; b=L1FlWJ0ORX1yXBx7Cp/EqWvujWqceEMSW8XLGSLWVLXDC5L1ZK8rNB7QpfdahQY6XK 0/swU/UoeBdad2yeBy+7yZYsuo/ZRVDUzkAshCy4tLAigQMTbF8D7ojy1GpcxIZrjCXM edAXHn6Me/v9Jhe+H66Ab4ozARvfZf9ZpiiNiyppJzdkvvrcHIiN3OnDy4vBcB/fTdxB A7I4Ih5iQpjl7KOrXFjlRKXdxDu96VLZpSFuOhYJKLN7Kn4EPpwssTIv3unrN/jajweC 9advfdQKz0XQeYVAy92JSuU5vok3OeJUaFbtx3/JMSnt9Z1WjN5ucGc24opr5VASjECk VUaw==; darn=isar-build.org ARC-Authentication-Results: i=2; gmr-mx.google.com; dkim=pass header.i=@siemens.com header.s=fm1 header.b=JxeIdMN7; spf=pass (google.com: domain of fm-1212295-2025092506544306043743660002077a-t7uhgd@rts-flowmailer.siemens.com designates 185.136.64.228 as permitted sender) smtp.mailfrom=fm-1212295-2025092506544306043743660002077a-T7UHgD@rts-flowmailer.siemens.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=siemens.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlegroups.com; s=20230601; t=1758783287; x=1759388087; darn=isar-build.org; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:reply-to :x-original-authentication-results:x-original-sender:feedback-id :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:from:to:cc:subject:date:message-id:reply-to; bh=e+I1/EeLJRMir3sFwe5PQHNtfAVebKPzHRiyQhXsE3U=; b=mTnyq+LUg4f6cQBetHTvnRx4/apRd1F0D+uIa7zeaIScMSznqXdCzKGxSCufGlydPV +dwo1a21TwZmXljA9oTEfYRluwoNkxssb13mQU6ArxaXaxu0R4PTlDUvGu1P4cZH379B ExKX6oOES9GlwlJn0ioCRaSXY041bE+tSnhhSvQS45GyD5/JI/ECl4RTYtgj2X7os2ZS jaboxRLpf8DEtLppBv7Hto7uRN7zdCzHO9Hv9rv78nbKbvQ+QF8qu6jDgMTjYuK0llCg VD4itPMEDFdQkHRl7RsxIUA/AvVPcigDCjBCwp2V1P5gAH7cafMDAb43mr786IMMUZL0 KdyQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1758783287; x=1759388087; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :x-spam-checked-in-group:list-id:mailing-list:precedence:reply-to :x-original-authentication-results:x-original-sender:feedback-id :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:x-beenthere:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=e+I1/EeLJRMir3sFwe5PQHNtfAVebKPzHRiyQhXsE3U=; b=dS+3DsEV2vsd9V0lsTbHNgnRMWlSX83bXVXD63DNsAe1I74XLwR53EWjuoFvVxPia+ ukptY6sb2mtsy8nMCQnO+d1AZgi8jQn+aeQ40pS12xT5yfIJMJt9IiKkzST6C1XiCgRv l6JDwZe6BF0aFupXcfRkBFmt8OZy0+CAc2oPMbdZfhigs0VnrgnT6RXzZ0oLLLhhxL+2 S7RtjRnzmotZyHvOvjp6QrT9gixKpXG4HZqdPKLfPfKTv6VEHjCgCGjUbCVgz64MsL70 WnQF74cRPVBY6UxoyCnwOf81ViWeR+kwyHFYIp8b5E1+pmeRH92LS8N0YfcinwGsnQ0D tvIg== X-Forwarded-Encrypted: i=2; AJvYcCWhB3lTv5g/LabrB5lQH6mFBd5NYHYJuRHDZtS6PbWWYhmlHJgji4cc8KWtevXtDpjxsGWBcPk=@isar-build.org X-Gm-Message-State: AOJu0Yz5j/SAazejtiDVpzgn5cT1FbFdc9/qkhLrtFfSnbPyjjkjMnjE FVtpIwn1FpJY8x+1gMb4MApZhw/B6H9nLWmmQtuMqIo+km8OxtOfZIK+ X-Google-Smtp-Source: AGHT+IE5XSXFw7mNbWmHWsyYfIPxxyIh+nH37dKZ0SW9sOctu/DWCePhHAE2ddJs8KqSUB0jpHe+JA== X-Received: by 2002:a05:690e:4241:b0:635:4ecf:bdcd with SMTP id 956f58d0204a3-6361a8ad2b9mr1266447d50.47.1758783286785; Wed, 24 Sep 2025 23:54:46 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com; h="ARHlJd5UH32tFFzxaomcoOCRsmGXddJk6mx5Xmovnpm3wWT/aw==" Received: by 2002:a05:690e:42d3:b0:601:f279:9614 with SMTP id 956f58d0204a3-6361b37da85ls203979d50.0.-pod-prod-02-us; Wed, 24 Sep 2025 23:54:45 -0700 (PDT) X-Received: by 2002:a05:690e:2442:b0:600:4a6c:bed6 with SMTP id 956f58d0204a3-6361a890925mr1262087d50.36.1758783284832; Wed, 24 Sep 2025 23:54:44 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1758783284; cv=none; d=google.com; s=arc-20240605; b=RxzgyymlszFEztq2RHe/ecbyErF2KEKrQC8s9uQeFeQccyL4vYh2Fnp0j/Vwoc54Td yQvHK6/HiM1gZLarjYyZ52VVzfHOOpNFi7bLaNfsELNlA2vEnIDhujJtMUN59CcnS5ec cUWd3Y8lgIR5CLokmpLM3Ej8KeJOS+FiM7wUSvvt/HuXB0YmQ+qmldbLB+tS23jqedn7 h7VNExS6Nq2yPZZ/S0YwQEOZ17oXUmVRVnhtf1R8ke0/jk9UZILgChGNqVpFshZ1dRsE /EDjFCGizsiUgVFBZEmOuceHcZ19/9QyWZKzfHA3Eb3XKbTtBuXLL3/Xrq/P0ZV59TlP /kww== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20240605; h=feedback-id:content-transfer-encoding:mime-version:references :in-reply-to:message-id:date:subject:cc:to:from:dkim-signature; bh=jr2C+oYYPggLCfbJoYlAW3QELe9Nj37+1sbAp4rI8xM=; fh=GJh20mB+jF6oQ+me1R+hLO+vPKOaUsE8susJALlxkPc=; b=lRqJJT1pMgTsY2r+sDYKuhOzuEZZ7APbgWriIAH18+jthvQuOhYDGDPeV4w6SY01WX HMM2e4cfYa3p/jElq+JW1vDHrOQ3VSKHOJ6+a1rdjWV4Fph6HmXYEhehmBPOx3KGqgX1 Zj5Rx1QUO4xDimS6d86dc6W35R+yx27AQrr+lKR8sFsFI9di62EjbrR7+ifumo4Ta+Ye QkrqlkIgoroG0T8jR7Zrf74KAbgGdvVtkU3kdE0FoByIY4icZfKTqGEW19k1z2VQiLbJ ANP8ybPcL9yz+vPUUUO+j86bxEHm9aV5foeNvaH6OgZRrTGbkY8ds6nW4mHWxNw/s6n1 dnIA==; dara=google.com ARC-Authentication-Results: i=1; gmr-mx.google.com; dkim=pass header.i=@siemens.com header.s=fm1 header.b=JxeIdMN7; spf=pass (google.com: domain of fm-1212295-2025092506544306043743660002077a-t7uhgd@rts-flowmailer.siemens.com designates 185.136.64.228 as permitted sender) smtp.mailfrom=fm-1212295-2025092506544306043743660002077a-T7UHgD@rts-flowmailer.siemens.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=siemens.com Received: from mta-64-228.siemens.flowmailer.net (mta-64-228.siemens.flowmailer.net. [185.136.64.228]) by gmr-mx.google.com with ESMTPS id 956f58d0204a3-6361e540b5csi68328d50.0.2025.09.24.23.54.44 for (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Wed, 24 Sep 2025 23:54:44 -0700 (PDT) Received-SPF: pass (google.com: domain of fm-1212295-2025092506544306043743660002077a-t7uhgd@rts-flowmailer.siemens.com designates 185.136.64.228 as permitted sender) client-ip=185.136.64.228; Received: by mta-64-228.siemens.flowmailer.net with ESMTPSA id 2025092506544306043743660002077a for ; Thu, 25 Sep 2025 08:54:43 +0200 X-Patchwork-Original-From: "'Cedric Hombourger' via isar-users" From: "cedric.hombourger@siemens.com" To: isar-users@googlegroups.com Cc: "cedric.hombourger@siemens.com" Subject: [PATCH v4 3/4] bootstrap: create lock for downloads/deb without sudo Date: Thu, 25 Sep 2025 08:54:23 +0200 Message-ID: <20250925065433.4180883-4-cedric.hombourger@siemens.com> In-Reply-To: <20250925065433.4180883-1-cedric.hombourger@siemens.com> References: <20250925065433.4180883-1-cedric.hombourger@siemens.com> MIME-Version: 1.0 X-Flowmailer-Platform: Siemens Feedback-ID: 519:519-1212295:519-21489:flowmailer X-Original-Sender: cedric.hombourger@siemens.com X-Original-Authentication-Results: gmr-mx.google.com; dkim=pass header.i=@siemens.com header.s=fm1 header.b=JxeIdMN7; spf=pass (google.com: domain of fm-1212295-2025092506544306043743660002077a-t7uhgd@rts-flowmailer.siemens.com designates 185.136.64.228 as permitted sender) smtp.mailfrom=fm-1212295-2025092506544306043743660002077a-T7UHgD@rts-flowmailer.siemens.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=siemens.com X-Original-From: Cedric Hombourger Reply-To: Cedric Hombourger Precedence: list Mailing-list: list isar-users@googlegroups.com; contact isar-users+owners@googlegroups.com List-ID: X-Spam-Checked-In-Group: isar-users@googlegroups.com X-Google-Group-Id: 914930254986 List-Post: , List-Help: , List-Archive: , List-Unsubscribe: , X-Spam-Status: No, score=-4.9 required=5.0 tests=DKIMWL_WL_MED,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,MAILING_LIST_MULTI, RCVD_IN_DNSWL_BLOCKED,RCVD_IN_MSPIKE_H2,RCVD_IN_RP_CERTIFIED, RCVD_IN_RP_RNBL,RCVD_IN_RP_SAFE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on shymkent.ilbers.de X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= From: "cedric.hombourger@siemens.com" The syncin/syncout commands passed to mmdebstrap will create a lock file in downloads/deb if it does not exist. As mmdebstrap is being executed as root, the lock would also be owned by root and this will cause problems for rootless commands that may be executed later (such as downloading of Debian packages). Create the lock file without sudo prior to running mmdebstrap for it to be owned by the build user rather than root. Signed-off-by: Cedric Hombourger Tested-by: Andreas Naumann --- meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc b/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc index 931f6f13..b2de61ad 100644 --- a/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc +++ b/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc @@ -181,6 +181,10 @@ do_bootstrap() { && sudo umount $base_apt_tmp \ && rm -rf --one-file-system $base_apt_tmp' EXIT + # Create lock file so that it is owned by the user running the build (not root) + mkdir -p ${DEBDIR} + touch ${DEB_DL_LOCK} + sudo TMPDIR="${BOOTSTRAP_TMPDIR}" mmdebstrap $bootstrap_args \ $arch_param \ --mode=unshare \ From patchwork Thu Sep 25 06:54:24 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "cedric.hombourger@siemens.com" X-Patchwork-Id: 4378 Return-Path: Received: from shymkent.ilbers.de ([unix socket]) by shymkent (Cyrus 2.5.10-Debian-2.5.10-3+deb9u2) with LMTPA; Thu, 25 Sep 2025 09:06:47 +0200 X-Sieve: CMU Sieve 2.4 Received: from mail-qk1-f192.google.com (mail-qk1-f192.google.com [209.85.222.192]) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPS id 58P76k4j002505 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Thu, 25 Sep 2025 09:06:46 +0200 Received: by mail-qk1-f192.google.com with SMTP id af79cd13be357-854bec86266sf212775085a.1 for ; Thu, 25 Sep 2025 00:06:46 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1758784000; cv=pass; d=google.com; s=arc-20240605; b=H2XZUnBUxE03eQaonnVKsMLQVSODZPWMrgfBThLkKbCvfbz4Ru+L2FVSZy7jSl1L9j JirT1CnLSWLNMYA3hkN5lOORWl2cPbmu+5hfyfdx8zo/+dyLM/bzzM2ccWmXTfPPklOB jSVl4fZcaN9WMm4Jgd5rx9jHzR0MxFqvLK6jvn3sllnCivdSyJsKV2tAJb9qWsh/ZL2O WURFkMSXxlx7NiZUbKkp3lrYun45XAzt1W9un56maWxtFBKyqIGJNlJzcHnRD0Ycj7Op 2MhO2vH0YX2JKUTWjrMa2AvZ4ruYhRiJ8Q95/ziLGes4oe8SZXQq2UslNZob3tpJ9K4L 7SRQ== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20240605; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:reply-to:feedback-id:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=PdYikJ2oKUaRRcraHjnVTE+lzeR3iyIQY8Cddkm2i8c=; fh=sSsAkDG1GCGTkDU+vUd6dlRqxF5vkeUm9ZHx0kWULhA=; b=a4WYpI0FXCvHLU082chf2B5Lue/W+gK1ft6s2uGABix4KhrPya7DXdxcq3UnyvVVON OFZzbhbJ9R4l5WKiG6cqbtkV6sr1YJ5yBs1cUtv1Ko10Pokmsaxntw/sFRdEHF/zf8DC /40xMk8TIzI1eXN+VJhu01HgZc7U0tCbL3I8dvK3PanIodT+4mplb/Lab42HO0IaD1mu QYxZgGLSXzCGGtuoofGpaZ8S85zPTk8KcEOaj5MHMHgPKtIc4tfLRKfGuqtHdlNLyDBJ gHOWniRw/pWF+CmDfP0XIoBSNtFxe6WQ0XiHstIgYqNdC07CNxyx4QC8wb2QpUGh1qMj PnuA==; darn=isar-build.org ARC-Authentication-Results: i=2; gmr-mx.google.com; dkim=pass header.i=@siemens.com header.s=fm1 header.b=JgnVmhrS; spf=pass (google.com: domain of fm-1212295-202509250654437bebb669bf000207ae-v_jj66@rts-flowmailer.siemens.com designates 185.136.65.227 as permitted sender) smtp.mailfrom=fm-1212295-202509250654437bebb669bf000207ae-v_Jj66@rts-flowmailer.siemens.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=siemens.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlegroups.com; s=20230601; t=1758784000; x=1759388800; darn=isar-build.org; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:reply-to :x-original-authentication-results:x-original-sender:feedback-id :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:from:to:cc:subject:date:message-id:reply-to; bh=PdYikJ2oKUaRRcraHjnVTE+lzeR3iyIQY8Cddkm2i8c=; b=POB9ptRs30NJFILt/ehr+wecA0xfJsvP32lrvSCAOnEZA6SuThikoTZG8Kz0PlRiVt Vn1x8rCyiE5o/SATqN0VW0vP18jYav2FcRhmLOwjXfFaUIxDTlLqi8V0V5/3w/NYmdCT B1eX1dkYKTTPR0g+TU/3khI3jfbAJnBdYDaIX65STL2LRlQV3sXG0CgiNvA/l6wtjfUQ VK7gLUhuFcud3OF5XP6r/z9wRsDwWTxGSW5Gq9LHgIS2Nq2RH6YlagJ13ULw4ii5LnRt PJJERTdtBWEjLBWm0YjMnBWVhX1Y8EYfzMl/40HzgafJkAFVbXV7G7yX6PhPhHyiHizV f1Og== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1758784000; x=1759388800; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :x-spam-checked-in-group:list-id:mailing-list:precedence:reply-to :x-original-authentication-results:x-original-sender:feedback-id :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:x-beenthere:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=PdYikJ2oKUaRRcraHjnVTE+lzeR3iyIQY8Cddkm2i8c=; b=G31NC3VVq4uHlTKY7IAy16noanXwwEpkUcpBDRedGWbhZFxxKwWCxwoSc3zOGoH5qM kOj9vLkGW5EeBuP9Z+t8XZExpOW1EYHhaCpJBXZ5NOG6ve/RekOVsPy9qjshnT5miF0i Il9AqRvT3cDaDnfcFFWfjPWkyjapp3FFy4wU7i6aavGeVu48RTd2DwUIP+BrLBeoJMYo +EDx7cCm+1+iyDnPKFMnsh1dewUcXNh4G/EFSCvxjhsj5c7BniMBEE7SXAB4antBfELz NnBrzRm2xVEomGNJz1l9cljIyLcUruLTY/mW8tZPYjJkodk3ZQ4z+C/Hi6lLOrkMJ1Jz 8YcQ== X-Forwarded-Encrypted: i=2; AJvYcCV+5u/YwEuE5abxJohH82GfMcHGtsyhd4jg6LFrx7N7+HkGqqXnVWMRqw2BWgr4ZHHVXlit+AI=@isar-build.org X-Gm-Message-State: AOJu0YxnU8Ne0QxNCot7icwhAxdAdaSQ4Dcr03oFfXgKTf3/HCABItvA +MYNjIxr0T4bbkQxAncAxXRhEWWJrYc8OTqX/Ql5sh1GeUF03HZJeK+h X-Google-Smtp-Source: AGHT+IHkqBT/1nYJU5hG8M4aI0Y++bKiCPxdFR71CRamySXYZIGCnBnaiCDOCNVqQR4Eg9hBLNLrGw== X-Received: by 2002:a05:690c:930d:20b0:733:3aa3:674e with SMTP id 00721157ae682-763f87775efmr22951847b3.10.1758783286232; Wed, 24 Sep 2025 23:54:46 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com; h="ARHlJd7a2elMcIAngLuCKiaj0gF7N8fOEuGRBklIF55NDo+9zA==" Received: by 2002:a05:690e:2547:b0:5f3:b853:aca3 with SMTP id 956f58d0204a3-6361b58b7fcls225089d50.1.-pod-prod-06-us; Wed, 24 Sep 2025 23:54:45 -0700 (PDT) X-Received: by 2002:a05:690c:7485:b0:74b:e290:1e2e with SMTP id 00721157ae682-7640624f3a5mr21826197b3.52.1758783285071; Wed, 24 Sep 2025 23:54:45 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1758783285; cv=none; d=google.com; s=arc-20240605; b=GNgFDYD1rRf8kjnWdZoaFN+rVxlr3zTaQ5bvrak+LO5JBYXKa36jTUAuPJPXn7x+q2 nX4M9/yKaq9eoUwAW61mhGZ9OqfaHg9bmTAwr/e6F2N+qB8kQIlE4YK2vfRhpfeYhpLh NNtL1YKcXSDxNKyrioaN6/miqCsn19XbI0D9tg4fNom6Zon9fAigJD3pgURfESNS7wK+ RWI2K2M+mzCgd7szZ6BWo86AqGaKImNJbc1mirsAzGnTwZfllmIglTqqDTkQUxbr1MI4 +8OXNPNdae1TgE1oA/gu5t9sQTuvVmzVISmu/OSkiLfraWHg0mOnULzgb5l3i32VQx+3 hRSA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20240605; h=feedback-id:content-transfer-encoding:mime-version:references :in-reply-to:message-id:date:subject:cc:to:from:dkim-signature; bh=SJ4rAl5rQYFYautfdxrU6ErymrRU4/HpwhLTlztHuXk=; fh=GJh20mB+jF6oQ+me1R+hLO+vPKOaUsE8susJALlxkPc=; b=Y56S2WiUjEUyFuQS9wT9oylhFlXfCYPhwa2kPqi5JXuw217n8aXMb43ApTsL+Zy97s nM90L2tZryrO6asTfzVLiVd7PXxK+X5FofLdsfkiFBgtwcWgQU34LO/FmzM0PFxiCtxD jzI9QErKJThk+SDrmfTcoxPp+BSLquLiH4cg3ZvddrkFcx//6JBdXoQ1ulFqT0a2VXKy qtrpBEZ2ipM5HWcy26Hf+nRMExA406p7zTQquzDc2tq9+/5FcYd+yqZEbR3kXKgxEqyn JJHImnzUokOuzK9q3BMwJgI3cQGutl79OJ1xl+VHcc0n8fAUkP+NhYWllkNeVvT1AFKM umLw==; dara=google.com ARC-Authentication-Results: i=1; gmr-mx.google.com; dkim=pass header.i=@siemens.com header.s=fm1 header.b=JgnVmhrS; spf=pass (google.com: domain of fm-1212295-202509250654437bebb669bf000207ae-v_jj66@rts-flowmailer.siemens.com designates 185.136.65.227 as permitted sender) smtp.mailfrom=fm-1212295-202509250654437bebb669bf000207ae-v_Jj66@rts-flowmailer.siemens.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=siemens.com Received: from mta-65-227.siemens.flowmailer.net (mta-65-227.siemens.flowmailer.net. [185.136.65.227]) by gmr-mx.google.com with ESMTPS id 00721157ae682-76724498746si161807b3.0.2025.09.24.23.54.44 for (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Wed, 24 Sep 2025 23:54:44 -0700 (PDT) Received-SPF: pass (google.com: domain of fm-1212295-202509250654437bebb669bf000207ae-v_jj66@rts-flowmailer.siemens.com designates 185.136.65.227 as permitted sender) client-ip=185.136.65.227; Received: by mta-65-227.siemens.flowmailer.net with ESMTPSA id 202509250654437bebb669bf000207ae for ; Thu, 25 Sep 2025 08:54:43 +0200 X-Patchwork-Original-From: "'Cedric Hombourger' via isar-users" From: "cedric.hombourger@siemens.com" To: isar-users@googlegroups.com Cc: "cedric.hombourger@siemens.com" Subject: [PATCH v4 4/4] rootfs: do not get elevated privileges when downloading packages Date: Thu, 25 Sep 2025 08:54:24 +0200 Message-ID: <20250925065433.4180883-5-cedric.hombourger@siemens.com> In-Reply-To: <20250925065433.4180883-1-cedric.hombourger@siemens.com> References: <20250925065433.4180883-1-cedric.hombourger@siemens.com> MIME-Version: 1.0 X-Flowmailer-Platform: Siemens Feedback-ID: 519:519-1212295:519-21489:flowmailer X-Original-Sender: cedric.hombourger@siemens.com X-Original-Authentication-Results: gmr-mx.google.com; dkim=pass header.i=@siemens.com header.s=fm1 header.b=JgnVmhrS; spf=pass (google.com: domain of fm-1212295-202509250654437bebb669bf000207ae-v_jj66@rts-flowmailer.siemens.com designates 185.136.65.227 as permitted sender) smtp.mailfrom=fm-1212295-202509250654437bebb669bf000207ae-v_Jj66@rts-flowmailer.siemens.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=siemens.com X-Original-From: Cedric Hombourger Reply-To: Cedric Hombourger Precedence: list Mailing-list: list isar-users@googlegroups.com; contact isar-users+owners@googlegroups.com List-ID: X-Spam-Checked-In-Group: isar-users@googlegroups.com X-Google-Group-Id: 914930254986 List-Post: , List-Help: , List-Archive: , List-Unsubscribe: , X-Spam-Status: No, score=-4.9 required=5.0 tests=DKIMWL_WL_MED,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,MAILING_LIST_MULTI, RCVD_IN_DNSWL_BLOCKED,RCVD_IN_MSPIKE_H2,RCVD_IN_RP_CERTIFIED, RCVD_IN_RP_RNBL,RCVD_IN_RP_SAFE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on shymkent.ilbers.de X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= From: "cedric.hombourger@siemens.com" Use rootfs_cmd() to run "apt-get install --download-only" without sudo. This requires /var/cache/apt/archives/ to be writable by the build user: change ownership while populating that folder with previously downloaded packages (those in downloads/deb/). Signed-off-by: Cedric Hombourger --- meta/classes/deb-dl-dir.bbclass | 21 ++++++++++++++++++--- meta/classes/rootfs.bbclass | 16 +++++++++++++--- 2 files changed, 31 insertions(+), 6 deletions(-) diff --git a/meta/classes/deb-dl-dir.bbclass b/meta/classes/deb-dl-dir.bbclass index ea0ed3d2..16ccd426 100644 --- a/meta/classes/deb-dl-dir.bbclass +++ b/meta/classes/deb-dl-dir.bbclass @@ -107,9 +107,24 @@ dbg_pkgs_download() { deb_dl_dir_import() { export pc="${DEBDIR}/${2}" export rootfs="${1}" - sudo mkdir -p "${rootfs}"/var/cache/apt/archives/ + export uid=$(id -u) + export gid=$(id -g) + + # let our unprivileged user place downloaded packages in /var/cache/apt/archives/ + sudo -Es << ' EOSUDO' + mkdir -p "${rootfs}"/var/cache/apt/archives/partial/ + touch "${rootfs}"/var/cache/apt/archives/lock + chown -R ${uid}:${gid} "${rootfs}"/var/cache/apt/archives/ + EOSUDO + + # nothing to copy if download directory does not exist just yet [ ! -d "${pc}" ] && return 0 - flock -s "${pc}".lock sudo -Es << 'EOSUDO' + + # attempt to create hard-links for .deb files from downloads/ into + # /var/cache/apt/archives/ so apt will only download packages we + # have not yet downloaded. perform a regular copy whenever hard-links + # cannot be created + ( flock 9 set -e printenv | grep -q BB_VERBOSE_LOGS && set -x @@ -118,7 +133,7 @@ deb_dl_dir_import() { ln -Pf -t "${rootfs}"/var/cache/apt/archives/ "$p" 2>/dev/null || cp -n --no-preserve=owner -t "${rootfs}"/var/cache/apt/archives/ "$p" done -EOSUDO + ) 9>"${pc}".lock } deb_dl_dir_export() { diff --git a/meta/classes/rootfs.bbclass b/meta/classes/rootfs.bbclass index f740c6e1..684d04c4 100644 --- a/meta/classes/rootfs.bbclass +++ b/meta/classes/rootfs.bbclass @@ -286,10 +286,20 @@ ROOTFS_INSTALL_COMMAND += "rootfs_install_pkgs_download" rootfs_install_pkgs_download[weight] = "600" rootfs_install_pkgs_download[progress] = "custom:rootfs_progress.PkgsDownloadProgressHandler" rootfs_install_pkgs_download[isar-apt-lock] = "release-after" -rootfs_install_pkgs_download[network] = "${TASK_USE_NETWORK_AND_SUDO}" +rootfs_install_pkgs_download[network] = "${TASK_USE_NETWORK}" rootfs_install_pkgs_download() { - sudo -E chroot '${ROOTFSDIR}' \ - /usr/bin/apt-get ${ROOTFS_APT_ARGS} --download-only ${ROOTFS_PACKAGES} + mkdir -p "${WORKDIR}/dpkg" + + # Use our own dpkg lock files rather than those in the rootfs since we are not root + # (this is safe as there are no concurrent apt/dpkg operations for that rootfs) + touch "${WORKDIR}/dpkg/lock" "${WORKDIR}/dpkg/lock-frontend" + + # download packages using apt in a non-privileged namespace + rootfs_cmd --bind "${ROOTFSDIR}/var/cache/apt/archives" /var/cache/apt/archives \ + --bind "${WORKDIR}/dpkg/lock" /var/lib/dpkg/lock \ + --bind "${WORKDIR}/dpkg/lock-frontend" /var/lib/dpkg/lock-frontend \ + ${ROOTFSDIR} \ + -- /usr/bin/apt-get ${ROOTFS_APT_ARGS} --download-only ${ROOTFS_PACKAGES} } ROOTFS_INSTALL_COMMAND_BEFORE_EXPORT ??= ""