From patchwork Mon May 19 11:57:48 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "cedric.hombourger@siemens.com" X-Patchwork-Id: 4212 Return-Path: Received: from shymkent.ilbers.de ([unix socket]) by shymkent (Cyrus 2.5.10-Debian-2.5.10-3+deb9u2) with LMTPA; Mon, 19 May 2025 13:58:24 +0200 X-Sieve: CMU Sieve 2.4 Received: from mail-ed1-f63.google.com (mail-ed1-f63.google.com [209.85.208.63]) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPS id 54JBwNZV018147 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Mon, 19 May 2025 13:58:23 +0200 Received: by mail-ed1-f63.google.com with SMTP id 4fb4d7f45d1cf-601fb2b7859sf613725a12.3 for ; Mon, 19 May 2025 04:58:23 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1747655897; cv=pass; d=google.com; s=arc-20240605; b=CotrhaG1iX8ZsR16j14L7foFYC3zEH2NvT40nhvY5MeeR3OP01z5MnZvLbzuEJVDNA f1ac0QLfYA0PnqabT9xqTL8ToMUMKURrqk6AieR0u4NxWLWQOCMUA0WSouIsRtggjJHc O6iAHfIp2qupfFABKd/vCzpaR7ruQ7THxQKPu4o+Eb2tgk6huwMPMskD1Av/Q6/Ve8jb USa4oQqhBRcsEVh4w+ZvQEhAejjZvRmALp3mRlDJJRVXA+tOS/CMASFeSU2oUTwGEsxb qP/YF51sdlItrCofh+yCf0meA+c1j/8lkNMXOl/M7jvjN7AIqRITnieV8rGfYL1KinUY wumg== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20240605; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:reply-to:feedback-id:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=Yf8PsUeaBFA/I140D4g4QDIFxECafAciEj0ApQQFsEU=; fh=KL8XXpOp4J0Fyhc3F4xao1i7lA8raw+xdTB3D3mXhfo=; b=iNWYsdDBwMG0Er0lWIlsXw3Sj/YVQaA8KTiz5fDo5mH+JzpleX3QIQPV3JEGfncjcg UZi5HGU7w6X3uZrfju/5mXYv3ZhnrU3JH8rOc8pn9XSwN//METhvMvx40x6AEYolgoXH X209pKwIb5Qq/WDFZutJ4E+L48XAbECgpVLCjSbP9Z/LHwYvRbHJ8BDjOt9/dxivC+1z JJ+32/4ljmAtQWfUVK3i457PT+vQSTdY6IBrOTbuHWQvNT71VgrvHM/aKqSi+63m3YAZ IXIlEBsmwiayVS66fBLxtm7ZcLUY3QgK73Y7TIsDUcfAafFPRr0qvOt5TcWffL7eTXJz Gr6w==; darn=isar-build.org ARC-Authentication-Results: i=2; gmr-mx.google.com; dkim=pass header.i=@siemens.com header.s=fm1 header.b=PhCQ4kVY; spf=pass (google.com: domain of fm-1212295-2025051911581484ad99ed22a832611e-cfne2l@rts-flowmailer.siemens.com designates 185.136.64.228 as permitted sender) smtp.mailfrom=fm-1212295-2025051911581484ad99ed22a832611e-CFNe2l@rts-flowmailer.siemens.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=siemens.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlegroups.com; s=20230601; t=1747655897; x=1748260697; darn=isar-build.org; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:reply-to :x-original-authentication-results:x-original-sender:feedback-id :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:from:to:cc:subject:date:message-id:reply-to; bh=Yf8PsUeaBFA/I140D4g4QDIFxECafAciEj0ApQQFsEU=; b=Mm7xbJKNp3FwZyINMk9Xh01wy4o/6S2HGU0/Rlz48/nYy+w0ZG6nt5nHRUS6dVWgh0 JDoNrdrAyVvHMLKzH1vK/QIxGZm5W/yxLRGNE/q5b7HhM6KDrKwHGwBxySy3NDQlfJAM fDmhTc8H2fF0ASIuQgRZpXOJj1YTj+UuFhyJhKY0a82K2a+Iaf4/yiFXj7N2UpXC0JUB xgYKl6YxtNT8aPTbsuvssjMFbyZVvW4WbSq8yqu6R6X6Y0L7btZNJ6lmmOj9DgTxJAKj NRwh9Z6djVc8/eU1COfgCFI/UDVDHt6zEjb7aWTDt9G/cAWkSxG7uSZXSFsimlilMmIr Srkw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1747655897; x=1748260697; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :x-spam-checked-in-group:list-id:mailing-list:precedence:reply-to :x-original-authentication-results:x-original-sender:feedback-id :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:x-beenthere:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=Yf8PsUeaBFA/I140D4g4QDIFxECafAciEj0ApQQFsEU=; b=w0HmjSGqG10cEuMeh3dYWoKrpXINMQ1ANyYf4dcHBGXyqv5ucby+nOdaiLhI1wqqsf bwh/ZorDZisSyyj43RVvsuLbyuGxseZwGIw4gGp5MN5JWjVzA2rsULuFRKEDtGtfQRiE GSfeBp3CNzkR1Vfj49pVk3p7kcWKZGAAv/crJ1Rl7xC369Uk45jM1/rak/LwWJnmwxAd 8NWW79qnhv5IKF/paJSztZehAVC2u6t0PWc64mFhYLPeNH2uKY2bx8emvz4Zzta/YXa0 sc52R2Ru9kW2ho509JHVCJNdTV07NOOUw/K3rBTsIlNoQJukW8XPm0vURDA/XEYJ7XHi Neow== X-Forwarded-Encrypted: i=2; AJvYcCUowdkF/QYF/LDLkN10sHMm7sQh79Z+b5oci1U4kfsbIoXmEMKE/Qon1QJux49mpyl2jEaIRoU=@isar-build.org X-Gm-Message-State: AOJu0Yy8hwVvAYqCKwrSdBfRu1bmjKxBqu2vw2Q7nWj1Meg1KFV1EUh9 SV2Ss+6dNuH7sR0jKLx8/j0Ea30YLnBYtUlyfxl0VqYQm2U/LZ9DhlGk X-Google-Smtp-Source: AGHT+IEfMOp091TQ6yCdGbgR1mSnngfichFVHQdbIDNfwLxJBc6J2/kfD/5VDFxNshqmWzpUNYwMwA== X-Received: by 2002:a05:6402:40cc:b0:5f4:d57e:4ab6 with SMTP id 4fb4d7f45d1cf-6011411a954mr8882528a12.24.1747655897309; Mon, 19 May 2025 04:58:17 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com; h=AVT/gBG7kf+mbAz6uJkaImUitv+F7BmfclKZUEF2osPWrLOo2g== Received: by 2002:a05:6402:1ecd:b0:601:a35e:6dd5 with SMTP id 4fb4d7f45d1cf-601a35e7064ls120714a12.0.-pod-prod-05-eu; Mon, 19 May 2025 04:58:14 -0700 (PDT) X-Received: by 2002:a17:906:c14d:b0:ad2:3efc:dd7a with SMTP id a640c23a62f3a-ad536b57a13mr1147253766b.4.1747655894439; Mon, 19 May 2025 04:58:14 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1747655894; cv=none; d=google.com; s=arc-20240605; b=N92k5rzRLHgTt0k1IciBuKgQsWmo+ROD2s94Zc/Sk21NF+iqxVVtmhb7V1hzPECQmk T9/EkD9r5sq6DXhgYLKxqNo4kecpUUqx8mh6rS+lHdrTHueaZciyLZfPK+EL/BclPJA+ nvtOg9mn6BYeW1y2LLoDBXpame/zT8hXuS1wBAWcAefABjFBHQcFKtUivPij7EbBL/LM AIA7iCfQsbftAwv7cUoTJAFl1WdCnq/iTHf+AvDQNTHbNYkKbpPQlgoesPAmQXLcD5uH fRFZhZg4sVNRSKo8X9hnyVA+c44BTcQ4I2z5LEY+R2fTlE4YpJZ8ZQce3Xad9MkRkS3a toeA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20240605; h=feedback-id:content-transfer-encoding:mime-version:references :in-reply-to:message-id:date:subject:cc:to:from:dkim-signature; bh=yTHyZtp/Gm/LWcW3yXuXiRnaKYLCtILET9xhsLbpom4=; fh=D/q4xMKxZDyLo2GtmwQ/2prSr9aCFD3HVqTCj43epLY=; b=QdtCP1JwFUvitOjgzyLncqWMmjq33OqgANWWacAe9uodZy2k0ikxbdfp5crPmF2JLm gk2lnCqN4aX/y9QJlDZdIv+3CWSAD6jw8texSJo+e5wsCtyDZNBfL0TGyCMY4OscbYbU DzFqgxoFb8zYI93+udLmvv/eHk7TEigkbXzIzcNLAx9F0PN+sHPd6Yf/XFdYY9a0w9pO +3Q42CBu06RPyicsxhkCO1Z/BLuntnam75kMA2iSUEpb9OT2tHBbc5KGam66hf8Yn1jU BniwVYyCyyPCWbg0WhZSowHlMJUUd1nvoVUhpcn1tNkX7WP0LKr/rVqwQSgtGWCDW3jP JFQQ==; dara=google.com ARC-Authentication-Results: i=1; gmr-mx.google.com; dkim=pass header.i=@siemens.com header.s=fm1 header.b=PhCQ4kVY; spf=pass (google.com: domain of fm-1212295-2025051911581484ad99ed22a832611e-cfne2l@rts-flowmailer.siemens.com designates 185.136.64.228 as permitted sender) smtp.mailfrom=fm-1212295-2025051911581484ad99ed22a832611e-CFNe2l@rts-flowmailer.siemens.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=siemens.com Received: from mta-64-228.siemens.flowmailer.net (mta-64-228.siemens.flowmailer.net. [185.136.64.228]) by gmr-mx.google.com with ESMTPS id 4fb4d7f45d1cf-6005ae4a03esi108842a12.3.2025.05.19.04.58.14 for (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Mon, 19 May 2025 04:58:14 -0700 (PDT) Received-SPF: pass (google.com: domain of fm-1212295-2025051911581484ad99ed22a832611e-cfne2l@rts-flowmailer.siemens.com designates 185.136.64.228 as permitted sender) client-ip=185.136.64.228; Received: by mta-64-228.siemens.flowmailer.net with ESMTPSA id 2025051911581484ad99ed22a832611e for ; Mon, 19 May 2025 13:58:14 +0200 X-Patchwork-Original-From: "'Cedric Hombourger' via isar-users" From: "cedric.hombourger@siemens.com" To: isar-users@googlegroups.com Cc: felix.moessbauer@siemens.com, Cedric Hombourger Subject: [PATCH 2/4] deb-dl-dir: optimize caching of source packages using apt natively Date: Mon, 19 May 2025 13:57:48 +0200 Message-Id: <20250519115750.3195300-3-cedric.hombourger@siemens.com> In-Reply-To: <20250519115750.3195300-1-cedric.hombourger@siemens.com> References: <20250515150727.1764989-2-cedric.hombourger@siemens.com> <20250519115750.3195300-1-cedric.hombourger@siemens.com> MIME-Version: 1.0 X-Flowmailer-Platform: Siemens Feedback-ID: 519:519-1212295:519-21489:flowmailer X-Original-Sender: cedric.hombourger@siemens.com X-Original-Authentication-Results: gmr-mx.google.com; dkim=pass header.i=@siemens.com header.s=fm1 header.b=PhCQ4kVY; spf=pass (google.com: domain of fm-1212295-2025051911581484ad99ed22a832611e-cfne2l@rts-flowmailer.siemens.com designates 185.136.64.228 as permitted sender) smtp.mailfrom=fm-1212295-2025051911581484ad99ed22a832611e-CFNe2l@rts-flowmailer.siemens.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=siemens.com X-Original-From: Cedric Hombourger Reply-To: Cedric Hombourger Precedence: list Mailing-list: list isar-users@googlegroups.com; contact isar-users+owners@googlegroups.com List-ID: X-Spam-Checked-In-Group: isar-users@googlegroups.com X-Google-Group-Id: 914930254986 List-Post: , List-Help: , List-Archive: , List-Unsubscribe: , X-Spam-Status: No, score=-4.9 required=5.0 tests=DKIMWL_WL_MED,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,MAILING_LIST_MULTI, RCVD_IN_DNSWL_BLOCKED,RCVD_IN_MSPIKE_H2,RCVD_IN_RP_CERTIFIED, RCVD_IN_RP_RNBL,RCVD_IN_RP_SAFE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on shymkent.ilbers.de X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= source package are downloaded by entering the target rootfs and run apt there. For foreign architectures, this results in apt being executed under QEMU and leads to poor performance. By using the recently introduced rootfs_native_cmd command wrapper, apt will be executed natively against the target rootfs and without elevated privileges. For our test work-load, caching was reduced from more than 10 hours to an hour. Performance is also more consistent as it will no longer depend as to when bitbake kicks caching of source packages for foreign architecture rootfs vs rootfs for the host (in multiconfig builds). Signed-off-by: Cedric Hombourger --- meta/classes/deb-dl-dir.bbclass | 37 ++++++++------------------------- 1 file changed, 9 insertions(+), 28 deletions(-) diff --git a/meta/classes/deb-dl-dir.bbclass b/meta/classes/deb-dl-dir.bbclass index 3f560da4..7026f4f4 100644 --- a/meta/classes/deb-dl-dir.bbclass +++ b/meta/classes/deb-dl-dir.bbclass @@ -5,25 +5,6 @@ inherit repository -debsrc_do_mounts() { - sudo -s <"${DEBSRCDIR}/${rootfs_distro}.lock" - - debsrc_undo_mounts "${rootfs}" } dbg_pkgs_download() {