From patchwork Thu Sep 25 06:54:22 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "cedric.hombourger@siemens.com" X-Patchwork-Id: 4375 Return-Path: Received: from shymkent.ilbers.de ([unix socket]) by shymkent (Cyrus 2.5.10-Debian-2.5.10-3+deb9u2) with LMTPA; Thu, 25 Sep 2025 08:54:53 +0200 X-Sieve: CMU Sieve 2.4 Received: from mail-qv1-f60.google.com (mail-qv1-f60.google.com [209.85.219.60]) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPS id 58P6sp7Q002095 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Thu, 25 Sep 2025 08:54:52 +0200 Received: by mail-qv1-f60.google.com with SMTP id 6a1803df08f44-78ea15d3548sf6000496d6.1 for ; Wed, 24 Sep 2025 23:54:52 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1758783286; cv=pass; d=google.com; s=arc-20240605; b=Sfeq1nRCFRCJsvRt+2+Pyakfivk8teVeKhGz3u3FO04pOFQKwg68RjIr6WMumnSXc/ SFCe7q9cUL3BdcsU33qgSoCXT/hdYtLDLIANfky83UtOMkwLfEMo+yBf6q/By0YKhVGE KbeGLmh2YRssBK3YpfFO5S7I3/RsLiEyN7IkNv9sghCaWxHWA0m4Mdw8Py0XWGj8izl7 JkKjhqEZ7EA6E50Nae+xl33ZBEGs2EOECny42ZLezcmKS+pPWu4Dq/WSMnjvFF8pzNfZ 7bv684sIdLUzYpglSEh0TBzD4MOhmLJLOPmqQlHYEkRA0n3oithSd7/4tX5WdbCjrCmF AD9Q== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20240605; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:reply-to:feedback-id:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=dxQxB3b/kgeO8JFoU+eboZZJNuWc6TcvHJdoBPTmRUw=; fh=HkM6Fbs97+oxq0RJthArMZGsB/RLyAQSCmrq3paIY1s=; b=RwKev2lYU0mQZ7OrluGAUj8lH6dqfsEKhJemWV782V4U2KUnBGQiAX86wpiQZCiO87 aIo2gOLfCDKNjZdPWEVjD2MACld0P4cnqulEh2hOINA+sFBCW1OVWsiikEtxvES9njhn SEI4cuDFSva3dB0gqDMBrxZDty0N3/QqSf+2Bp396YNnfDu38byHgqlM5EH+dUNg6fEy CIdego+6aSbeZQB4kVGhr1ACczDZs9qjmkVsb+1RnP9HnoSrDt96675/0MDWlgG8bA0Y ppEeFoS7SzWDn+gcaZ5hDYN2W+r/F7/PFv5/6a7fqeTFYoPa0tRWvcj8BZM7F5nrMD/4 NQBA==; darn=isar-build.org ARC-Authentication-Results: i=2; gmr-mx.google.com; dkim=pass header.i=@siemens.com header.s=fm1 header.b=XJ0TK+TU; spf=pass (google.com: domain of fm-1212295-20250925065442cda6ee4b9f0002079f-atmcct@rts-flowmailer.siemens.com designates 185.136.64.228 as permitted sender) smtp.mailfrom=fm-1212295-20250925065442cda6ee4b9f0002079f-ATmcCt@rts-flowmailer.siemens.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=siemens.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlegroups.com; s=20230601; t=1758783286; x=1759388086; darn=isar-build.org; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:reply-to :x-original-authentication-results:x-original-sender:feedback-id :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:from:to:cc:subject:date:message-id:reply-to; bh=dxQxB3b/kgeO8JFoU+eboZZJNuWc6TcvHJdoBPTmRUw=; b=uv0GxkDbHHbOmzDieetPL1ZtQb6BJgXlRYvjmRx/Cdt3DC/qFEeBkBn69P5cftAHwG WBbMDGMep26Rw9Pl6HwkDYEQ9sYvfqrRnHKMx0TndEghz8gam7NO3RoV68nvukTby7vr fBQSs7xtFpjIkAOEI4wnb/hQJKiiozEwPsyj1la0IlYgXFaooPu+4vZi04OMJZumSX55 uYQ5GygRCHnYRw5DTjpVQF9ASNxV0RrB6qqgAheSddLtNx1UhLjjmZj4X5+72A+mIKPB sGG5C97Ir1fq68WuEKlLePrWJLhgaRxYP7+b2Ty8eod5fPZliRla1MEjgFU8PUnLd3i2 /fIw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1758783286; x=1759388086; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :x-spam-checked-in-group:list-id:mailing-list:precedence:reply-to :x-original-authentication-results:x-original-sender:feedback-id :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:x-beenthere:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=dxQxB3b/kgeO8JFoU+eboZZJNuWc6TcvHJdoBPTmRUw=; b=Fkp7a13b0JqtR4IpAL9JzpRVT3u8qKzzY5VsU1gxdMI1+hGrupmgn1EwX4XThL2410 xBMc6HyxPFevZ8NKBfltncOKT+ByPlc4QypfEvizHZEbinP6X3frB+PulheQ9+1URf0Q uBYtSTEyqlE2kVM+4SnZYEKke58cAxf/aCzFnkdmM7LmT34HPUTLsA/N5Lq3fRkWdDlz oh8S2ktG/hWsfrgb/c4X+VY06c9r0w2RabChWyFNgzixJqan6oDIRazpwWP+R4Gl1UHv oz+9kxKFItHQxWg4Bsl6PTUK4BqgNT1SRXte9OYSH+O2LWY4EPbQznxUg9UcIigwxLeE UBng== X-Forwarded-Encrypted: i=2; AJvYcCX+M6mkRKMet4rqKtnGTLDJXobziWwebrhisze4HpNVAdYP0JKVkbA803R5al0InkC53fbFq/Y=@isar-build.org X-Gm-Message-State: AOJu0YxSZlwhftNcVs4IEmMrfTOKwgjuDrNmP2EVn0RRySkChIdutRTU 1r+z/75mtTnTMJTDWkyS11vPLPetYIouWC3qkHQgV/OyiM3uxsScBcM7 X-Google-Smtp-Source: AGHT+IFuyck1rbYQ3fvOmE0mCYzjIGRfaGX6M/CElCZv5ko9XdHNizF51y82gD62a4Pamcaq+pxELA== X-Received: by 2002:a05:6214:6117:b0:807:ca4f:5661 with SMTP id 6a1803df08f44-807ca4f588dmr5570306d6.63.1758783285830; Wed, 24 Sep 2025 23:54:45 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com; h="ARHlJd7TWIPDkDI/NLanCaqCZ2ItDLW18GzMuPCpGuQwDRigKw==" Received: by 2002:ad4:46c7:0:b0:70b:acc1:ba4f with SMTP id 6a1803df08f44-7fd7d048470ls8936276d6.1.-pod-prod-08-us; Wed, 24 Sep 2025 23:54:44 -0700 (PDT) X-Received: by 2002:a05:6214:5018:b0:7ea:9df1:78de with SMTP id 6a1803df08f44-7fc3d6859b2mr29862496d6.46.1758783284785; Wed, 24 Sep 2025 23:54:44 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1758783284; cv=none; d=google.com; s=arc-20240605; b=bNHXXrxolF1rLgzOFvxoklGSTAlgft+YDHRPLTDNqhlKz1R0558VdRbag/xTYZz4dF IWjGbe0kjZQMsR7ncX5SrdFcdrgNE/8XHg6KZnsbVfsBuBAdIrvtwKwFjuYBsluWQoxK zhYNVh0qNhy/QktT8aPGBYV3RESU3mLuVd+1KMjSJmQPIOeFunvAYRRkIJWK6AUiRc8m sIOOiV93FDFhzVVgHqAetrla7wzg9WD235SdcqrHtEe/WHM34xjpcP0ELNG1SbVEd6Zy JEy705UPtYeTOq0wkFwKpMYph9J7qabSyy9m9Gx02k616KysLaWRca3SuIqg7l6TZROn zHmQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20240605; h=feedback-id:content-transfer-encoding:mime-version:references :in-reply-to:message-id:date:subject:cc:to:from:dkim-signature; bh=BRiD8X4g3wKQTvAoxp17V2dY9+s3wi0V2twfBcf6teU=; fh=GJh20mB+jF6oQ+me1R+hLO+vPKOaUsE8susJALlxkPc=; b=byuyjIoygw+vN1i3Us85OKMpxP+EtqSDptNHoY1V0heVDipJfKAs+tkWeuoLIvl4Vu EeogJB/uv0tBXKT49BX37ysYx0DqeUh8YX1o/AnUlRmXWgPy64xUvKLl+CzJTXLqaleu rzEFpB5Rj2SuI+YO4v67bRqotBANAI+2aUhUCd/lTIX1qr27CGEEIXq9k3gkf3MfvRrC gVD2ebCFp91KvRwUY1K3DwEszay10p6sngonSV/TNyWIeC9Ga3Vice/5oCHt73HwXzmh Gr4JjFqTQlt8C24CDHAvhTPntvEAgA5NvfnIjOoagE/J0L7DyrE67ExeSwtctWNX/Luu w25A==; dara=google.com ARC-Authentication-Results: i=1; gmr-mx.google.com; dkim=pass header.i=@siemens.com header.s=fm1 header.b=XJ0TK+TU; spf=pass (google.com: domain of fm-1212295-20250925065442cda6ee4b9f0002079f-atmcct@rts-flowmailer.siemens.com designates 185.136.64.228 as permitted sender) smtp.mailfrom=fm-1212295-20250925065442cda6ee4b9f0002079f-ATmcCt@rts-flowmailer.siemens.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=siemens.com Received: from mta-64-228.siemens.flowmailer.net (mta-64-228.siemens.flowmailer.net. [185.136.64.228]) by gmr-mx.google.com with ESMTPS id 6a1803df08f44-8016e3993bdsi425896d6.7.2025.09.24.23.54.44 for (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Wed, 24 Sep 2025 23:54:44 -0700 (PDT) Received-SPF: pass (google.com: domain of fm-1212295-20250925065442cda6ee4b9f0002079f-atmcct@rts-flowmailer.siemens.com designates 185.136.64.228 as permitted sender) client-ip=185.136.64.228; Received: by mta-64-228.siemens.flowmailer.net with ESMTPSA id 20250925065442cda6ee4b9f0002079f for ; Thu, 25 Sep 2025 08:54:42 +0200 X-Patchwork-Original-From: "'Cedric Hombourger' via isar-users" From: "cedric.hombourger@siemens.com" To: isar-users@googlegroups.com Cc: "cedric.hombourger@siemens.com" Subject: [PATCH v4 2/4] deb-dl-dir: optimize caching of source packages using apt natively Date: Thu, 25 Sep 2025 08:54:22 +0200 Message-ID: <20250925065433.4180883-3-cedric.hombourger@siemens.com> In-Reply-To: <20250925065433.4180883-1-cedric.hombourger@siemens.com> References: <20250925065433.4180883-1-cedric.hombourger@siemens.com> MIME-Version: 1.0 X-Flowmailer-Platform: Siemens Feedback-ID: 519:519-1212295:519-21489:flowmailer X-Original-Sender: cedric.hombourger@siemens.com X-Original-Authentication-Results: gmr-mx.google.com; dkim=pass header.i=@siemens.com header.s=fm1 header.b=XJ0TK+TU; spf=pass (google.com: domain of fm-1212295-20250925065442cda6ee4b9f0002079f-atmcct@rts-flowmailer.siemens.com designates 185.136.64.228 as permitted sender) smtp.mailfrom=fm-1212295-20250925065442cda6ee4b9f0002079f-ATmcCt@rts-flowmailer.siemens.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=siemens.com X-Original-From: Cedric Hombourger Reply-To: Cedric Hombourger Precedence: list Mailing-list: list isar-users@googlegroups.com; contact isar-users+owners@googlegroups.com List-ID: X-Spam-Checked-In-Group: isar-users@googlegroups.com X-Google-Group-Id: 914930254986 List-Post: , List-Help: , List-Archive: , List-Unsubscribe: , X-Spam-Status: No, score=-4.9 required=5.0 tests=DKIMWL_WL_MED,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,MAILING_LIST_MULTI, RCVD_IN_DNSWL_BLOCKED,RCVD_IN_MSPIKE_H2,RCVD_IN_RP_CERTIFIED, RCVD_IN_RP_RNBL,RCVD_IN_RP_SAFE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on shymkent.ilbers.de X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= From: "cedric.hombourger@siemens.com" source package are downloaded by entering the target rootfs and run apt there. For foreign architectures, this results in apt being executed under QEMU and leads to poor performance. By using the recently introduced rootfs_native_cmd command wrapper, apt will be executed natively against the target rootfs and without elevated privileges. For our test work-load, caching was reduced from more than 10 hours to an hour. Performance is also more consistent as it will no longer depend as to when bitbake kicks caching of source packages for foreign architecture rootfs vs rootfs for the host (in multiconfig builds). Signed-off-by: Cedric Hombourger --- meta/classes/deb-dl-dir.bbclass | 37 ++++++++------------------------- 1 file changed, 9 insertions(+), 28 deletions(-) diff --git a/meta/classes/deb-dl-dir.bbclass b/meta/classes/deb-dl-dir.bbclass index 4780be20..ea0ed3d2 100644 --- a/meta/classes/deb-dl-dir.bbclass +++ b/meta/classes/deb-dl-dir.bbclass @@ -5,25 +5,6 @@ inherit repository -debsrc_do_mounts() { - sudo -s <"${DEBSRCDIR}/${rootfs_distro}.lock" - - debsrc_undo_mounts "${rootfs}" } dbg_pkgs_download() {