From patchwork Wed Jun 18 13:50:38 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Cedric Hombourger X-Patchwork-Id: 4234 Return-Path: Received: from shymkent.ilbers.de ([unix socket]) by shymkent (Cyrus 2.5.10-Debian-2.5.10-3+deb9u2) with LMTPA; Wed, 18 Jun 2025 16:04:02 +0200 X-Sieve: CMU Sieve 2.4 Received: from mail-qt1-f192.google.com (mail-qt1-f192.google.com [209.85.160.192]) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPS id 55IE40lK003439 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Wed, 18 Jun 2025 16:04:01 +0200 Received: by mail-qt1-f192.google.com with SMTP id d75a77b69052e-4a587c85a60sf145149111cf.2 for ; Wed, 18 Jun 2025 07:04:01 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1750255435; cv=pass; d=google.com; s=arc-20240605; b=Clw008oixXApm2OZiJmHK+5c8sw32KMW13K6PP86XWSGaMqKqVtJfgMbSUne6IZpec m1hmTnjPN4JfIU8smGtWhFZ6MgC8XticMDsz2CFxjgOrXvGRidTIlXr13YSU71ocQC99 8aU42Ddc6FmWrkg71q8LJM3rOorqTZorqVTyWoBL8WwVDu0kqGo4rSMEdUPPsRmMmOE9 SHsrwoqqveV1M01fQt9o+aW1vr/FEFwyBJnvPcJP+00zuK9JLL0j5+0MnMSEv3ZrqtXa 6OVBAmpTXn4l6Lc52u/ISO4Yrt6WUIKoKTMugrkStdcQeWTJVHoqiuNUfe5DG2rn7UpO 1L+w== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20240605; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:reply-to:feedback-id:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=ZoEr9NAm7l2LXz+/DLEyA4hDoO4RkGdTpMYFtAo8D8w=; fh=ZkShzVbw6GXqqrCX57R3KbrfGbvCnUutZ/DrDFyGxbE=; b=WmP/VZhCRt/LOPYdzP+9xXnNMRpAMtiC8ioZhOKLHpcBZTeZHNYdj6gT+5lbbQCG5m bddpkeVeHu7F1L3RPrkyBmRpivszdUJDqsZRSw4yHsBylJhBQpv7ELEZ8lvJAXUzKVu4 f5J1uW7kpND2uBkzORyyftPNTZzf7hKUIXYpCVdG9w52YivDvzoqCLqGcYz7WjPzvhra IiPsANMwCahDZYoFyQu824aX+6jcOCA8SZ8WzSh9pe3nUs1NrCY1fNk9SfCAwRJ4YnMH DTyF7/nHRwAUYZMIPWlvCHkT0vn8NZo7e4PsxcL+ahJsKsuvpfsKenEP910oh2xAszBy Ggtw==; darn=isar-build.org ARC-Authentication-Results: i=2; gmr-mx.google.com; dkim=pass header.i=@siemens.com header.s=fm2 header.b=bDfz7K6A; spf=pass (google.com: domain of fm-1212295-20250618135101fe6c60d88081e4770d-ybgaqx@rts-flowmailer.siemens.com designates 185.136.65.225 as permitted sender) smtp.mailfrom=fm-1212295-20250618135101fe6c60d88081e4770d-ybgaQx@rts-flowmailer.siemens.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=siemens.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlegroups.com; s=20230601; t=1750255435; x=1750860235; darn=isar-build.org; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:reply-to :x-original-authentication-results:x-original-sender:feedback-id :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:from:to:cc:subject:date:message-id:reply-to; bh=ZoEr9NAm7l2LXz+/DLEyA4hDoO4RkGdTpMYFtAo8D8w=; b=eRRpHQuKKugTVVIM1o+Mj0SMtFZGiuMtzyvaMdxpzz+aV2B7t9u+HWgAymgChyecJ2 EQRNOc2OSPOLtlQWBFoe0lir69uyyLKqoxJW6lkEpXCiVcb4ROesFb8aISu2WnhwRtEc zvvHvO2dj8ft2dYtQr0rg1fARQthwJe1ZKCEpqQ2IeAaN/+U4SsUySszXm75LGJiEQZ4 mNHHTEYqVrnsxFyXy8EAjA2wY38y6q8WfjepAcWY7vSEQRx7+0gV8AEN06JzXHX9WFUy ZYVhRXgI/EE/2pjwUFeHHbS9E1cNlTy2sMfEzO8cfsFMy8GXsibUuxhodkIOlZu/v7H9 JU6w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1750255435; x=1750860235; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :x-spam-checked-in-group:list-id:mailing-list:precedence:reply-to :x-original-authentication-results:x-original-sender:feedback-id :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:x-beenthere:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=ZoEr9NAm7l2LXz+/DLEyA4hDoO4RkGdTpMYFtAo8D8w=; b=Wmz2UlV1J5tZ4fswMo4mIBfGYzoI/lZtCPtTqHpwIvKeykfjUsW7jZcugSszBKkQcX jdauJmk/yWAi4+KL7Ir0+pktnSz4zdqJWaBLw+AgAw8VLNqqEwv3FDk78GKumftbwwbP AMRt/7Ux9fWlOeCV1N7BzVVkEGskRXzmd9qeOCDHi9T5MOLnlLcPA7Tw8HMzJaO2MXj3 PSUEtN5a/6RyKOlIfcKCb38T4e2rDhABm3xJgVuqSbHJR7wYYpWuW05ilMkljriB3Dg5 eRUKhwsmRBOuMnMAfAa+G/Q3dgwlG1uUn+I3dAXuivRslMMgYvZUHHtUl1SDNFkYTO2P k5tg== X-Forwarded-Encrypted: i=2; AJvYcCX7Q+GJkyPnFZLh2kABSiJcN4ztIcR/UT+U1wEYYUkVgQgGA9eJFNjMfDEIk+agqT2ft7+9DBo=@isar-build.org X-Gm-Message-State: AOJu0YyAAO/a0dTq6II1Hf7A8jnAX49af6Vo2crVIU3Z7eZ9xApVQGjB 37UWi4xtc0HcE0KWLnpmaSvFlv9ipDhuzPQUvGeiSM5mhj665LE1lFQM X-Google-Smtp-Source: AGHT+IEKWL/kAE/Sqy7Uxfycg3hv3wbtPCiM+5N5XoMMdbPOLQqxleHpve+UtOXmxmL4j8zlpJNrHA== X-Received: by 2002:a17:903:1b66:b0:236:8db5:7e50 with SMTP id d9443c01a7336-2368db57ed5mr126675525ad.9.1750254664619; Wed, 18 Jun 2025 06:51:04 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com; h=AZMbMZc8n3Hd1ohw7fk8YORlVtW4GdUbBp3oaYvQgaXToHdeqQ== Received: by 2002:a17:903:1112:b0:236:6f3e:191 with SMTP id d9443c01a7336-2366f3e0300ls52317875ad.2.-pod-prod-03-us; Wed, 18 Jun 2025 06:51:03 -0700 (PDT) X-Received: by 2002:a17:902:ea0d:b0:234:8e54:2d53 with SMTP id d9443c01a7336-2366b3f8713mr267144165ad.45.1750254663298; Wed, 18 Jun 2025 06:51:03 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1750254663; cv=none; d=google.com; s=arc-20240605; b=ayJtOh3ldVumQbD0wbbQ1MfhYkfS0ry5LT5+2Dzb9Mvyc91QZg3PM0AH5ynwsJD7Yn SAMv34Sw2PQ7giWX1etZMIS1iDSly0+jLxKMeLW/JinLC7bCJCNLvdKYIqtegbruSd6n 3ca6YQMR0G3w8YeO/2gRtwDVBYeULSLQXIEgHc8ZYTLJQmI/igUycVCoz21MadZjd5zG u2Uyoa26bvWT5mDIyMGkxNPXnYWj7tXU975yAGzEZS47iNuJSe5FBsHG0nOpS6kuZAWU SMRlFo4cXeM2E1yDtySvgbdWqZiOn69tok+qk/cVX0yGyPSX+VUePifwtTWG3YJ/JkW8 Tb+Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20240605; h=feedback-id:content-transfer-encoding:mime-version:references :in-reply-to:message-id:date:subject:cc:to:from:dkim-signature; bh=yTHyZtp/Gm/LWcW3yXuXiRnaKYLCtILET9xhsLbpom4=; fh=D/q4xMKxZDyLo2GtmwQ/2prSr9aCFD3HVqTCj43epLY=; b=hGVg6tgQTyUmx8CCiJPQ6Kt3CqRUVu2c2Og8K/QSy6mJJ8DOcIHHEokE/cqgBISt7P 5Q4dWamup7mIfKEvrDcw1ey2wVrc8KsvLYlGkNEnOl6KaNB+QtnA21mArnpFaGu8vKlF nSkvVnodbM7nfI8BEVu7675SdL9pxVfzD80ULdQ9xqlMCXugMeC/gPH2t6wN+62ZhiSQ iTNSGbP7mq0zzp7RaZ1KPTEeUzuO/mNyTIZK+h6YKzjS2Jr1CeN7MTkNeM/GZzEViAf8 YrfbHhxqxUQin+PMcJM4SJ8vkk2AaWJfjYk2OquMgmtOupr9em6Cpwte/sRjLeEE8Bdm ohhA==; dara=google.com ARC-Authentication-Results: i=1; gmr-mx.google.com; dkim=pass header.i=@siemens.com header.s=fm2 header.b=bDfz7K6A; spf=pass (google.com: domain of fm-1212295-20250618135101fe6c60d88081e4770d-ybgaqx@rts-flowmailer.siemens.com designates 185.136.65.225 as permitted sender) smtp.mailfrom=fm-1212295-20250618135101fe6c60d88081e4770d-ybgaQx@rts-flowmailer.siemens.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=siemens.com Received: from mta-65-225.siemens.flowmailer.net (mta-65-225.siemens.flowmailer.net. [185.136.65.225]) by gmr-mx.google.com with ESMTPS id d9443c01a7336-2365dc38e44si5168215ad.7.2025.06.18.06.51.03 for (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Wed, 18 Jun 2025 06:51:03 -0700 (PDT) Received-SPF: pass (google.com: domain of fm-1212295-20250618135101fe6c60d88081e4770d-ybgaqx@rts-flowmailer.siemens.com designates 185.136.65.225 as permitted sender) client-ip=185.136.65.225; Received: by mta-65-225.siemens.flowmailer.net with ESMTPSA id 20250618135101fe6c60d88081e4770d for ; Wed, 18 Jun 2025 15:51:01 +0200 X-Patchwork-Original-From: "'Cedric Hombourger' via isar-users" From: Cedric Hombourger To: isar-users@googlegroups.com Cc: felix.moessbauer@siemens.com, Cedric Hombourger Subject: [PATCH v2 2/4] deb-dl-dir: optimize caching of source packages using apt natively Date: Wed, 18 Jun 2025 15:50:38 +0200 Message-Id: <20250618135040.8252-3-cedric.hombourger@siemens.com> In-Reply-To: <20250618135040.8252-1-cedric.hombourger@siemens.com> References: <20250519115750.3195300-1-cedric.hombourger@siemens.com> <20250618135040.8252-1-cedric.hombourger@siemens.com> MIME-Version: 1.0 X-Flowmailer-Platform: Siemens Feedback-ID: 519:519-1212295:519-21489:flowmailer X-Original-Sender: cedric.hombourger@siemens.com X-Original-Authentication-Results: gmr-mx.google.com; dkim=pass header.i=@siemens.com header.s=fm2 header.b=bDfz7K6A; spf=pass (google.com: domain of fm-1212295-20250618135101fe6c60d88081e4770d-ybgaqx@rts-flowmailer.siemens.com designates 185.136.65.225 as permitted sender) smtp.mailfrom=fm-1212295-20250618135101fe6c60d88081e4770d-ybgaQx@rts-flowmailer.siemens.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=siemens.com X-Original-From: Cedric Hombourger Reply-To: Cedric Hombourger Precedence: list Mailing-list: list isar-users@googlegroups.com; contact isar-users+owners@googlegroups.com List-ID: X-Spam-Checked-In-Group: isar-users@googlegroups.com X-Google-Group-Id: 914930254986 List-Post: , List-Help: , List-Archive: , List-Unsubscribe: , X-Spam-Status: No, score=-4.9 required=5.0 tests=DKIMWL_WL_MED,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,MAILING_LIST_MULTI, RCVD_IN_DNSWL_BLOCKED,RCVD_IN_MSPIKE_H2,RCVD_IN_RP_CERTIFIED, RCVD_IN_RP_RNBL,RCVD_IN_RP_SAFE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on shymkent.ilbers.de X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= source package are downloaded by entering the target rootfs and run apt there. For foreign architectures, this results in apt being executed under QEMU and leads to poor performance. By using the recently introduced rootfs_native_cmd command wrapper, apt will be executed natively against the target rootfs and without elevated privileges. For our test work-load, caching was reduced from more than 10 hours to an hour. Performance is also more consistent as it will no longer depend as to when bitbake kicks caching of source packages for foreign architecture rootfs vs rootfs for the host (in multiconfig builds). Signed-off-by: Cedric Hombourger --- meta/classes/deb-dl-dir.bbclass | 37 ++++++++------------------------- 1 file changed, 9 insertions(+), 28 deletions(-) diff --git a/meta/classes/deb-dl-dir.bbclass b/meta/classes/deb-dl-dir.bbclass index 3f560da4..7026f4f4 100644 --- a/meta/classes/deb-dl-dir.bbclass +++ b/meta/classes/deb-dl-dir.bbclass @@ -5,25 +5,6 @@ inherit repository -debsrc_do_mounts() { - sudo -s <"${DEBSRCDIR}/${rootfs_distro}.lock" - - debsrc_undo_mounts "${rootfs}" } dbg_pkgs_download() {