From patchwork Wed Jun 25 19:37:44 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Cedric Hombourger X-Patchwork-Id: 4239 Return-Path: Received: from shymkent.ilbers.de ([unix socket]) by shymkent (Cyrus 2.5.10-Debian-2.5.10-3+deb9u2) with LMTPA; Wed, 25 Jun 2025 21:39:39 +0200 X-Sieve: CMU Sieve 2.4 Received: from mail-oo1-f60.google.com (mail-oo1-f60.google.com [209.85.161.60]) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPS id 55PJdBD6007182 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Wed, 25 Jun 2025 21:39:17 +0200 Received: by mail-oo1-f60.google.com with SMTP id 006d021491bc7-60f132ad457sf2826eaf.3 for ; Wed, 25 Jun 2025 12:39:12 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1750880346; cv=pass; d=google.com; s=arc-20240605; b=YsceAqBtE813AmIKiwrH8hgMrQUr+pv7zE6cv9xfuu5LIJih6oAvOlzpyb7WUirDEV ZTiw599DfkwEOE6F3LZoy5OXdzuEO1yeoj2x+A7wlgEL1WggtKgYUp+iP4PhHWmzTYDd EDd0ifl9iltaUoQhTWNVc1QxMtNH54MGkQM5cw2BeMlSRF9TmU7LJ4h4mcNPqCEByKOf wMZS0Cwmdi40UDHWVgPG/FlAHwlnSE5d344H4C71+0L190DhvQOPImSm/ZD9yyh5emBJ ShNFp809WiZ8465FMatf+crCTLGDFHAvPZ6I+3YoqzCAuViq9FXrGvfDzHeL3EYiptFc 4zjQ== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20240605; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:reply-to:feedback-id:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=fWEhrPcG9jjLDFgXGjAcw4XIUAV2GeMTzYWUcNStluE=; fh=JvTZpVpvb4RdHT+fCeABGyX/7zqRADsHWSNamGY799E=; b=jiSzbcEYxvZyofdn+hBlDLnX0ERO2CpJEOJU+3sh63cIWz6eSq6HAtpVutUUP9/drl zEragXYs8OVDEH52Luui/kBBeYjVdn0EL4IdrrrP2BeQ86gZytt0gVuIe9ceJDqIvvAN wamqOZpHeexjnYJGgt7j2KI3dTgkwn7I/iO/34/H8lcBaHUxLak9YmoywQDwjS46Lr/R QUwta7a9rzhbz7j6JS/xMwUv4HguT4spAoe23qx00mptTKfgp/32SPWVPNvkW1K2lFGR Fuf8bQ4E9n3qfNSSK659efuSeJ4wqIfoCZuJGANQmV+DZ2uP1QDqSMRnw44oZi86mOqr LhwA==; darn=isar-build.org ARC-Authentication-Results: i=2; gmr-mx.google.com; dkim=pass header.i=@siemens.com header.s=fm2 header.b=VYmysbnv; spf=pass (google.com: domain of fm-1212295-20250625193904d657f811301bf74e88-rn0ms2@rts-flowmailer.siemens.com designates 185.136.65.228 as permitted sender) smtp.mailfrom=fm-1212295-20250625193904d657f811301bf74e88-rN0Ms2@rts-flowmailer.siemens.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=siemens.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlegroups.com; s=20230601; t=1750880346; x=1751485146; darn=isar-build.org; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:reply-to :x-original-authentication-results:x-original-sender:feedback-id :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:from:to:cc:subject:date:message-id:reply-to; bh=fWEhrPcG9jjLDFgXGjAcw4XIUAV2GeMTzYWUcNStluE=; b=dv747IF8gETHZCyNRU21alcdFUOfdTjrXk76PEq5Mrdr4iTfYoPCTRuiBrd6e1jPm5 skHmXI4eNo8ExzklUVUGO7HCip9lAhXLvLfmjEDiNLLJVhkPnjki/oB6VcmbHo+ALvNa pspv0exwjf9xN83bRSUzYMn+V4lX5cHY1qKG4bmbsTIT+VPOpKUmx2gDvOH6IhNPOMe7 b2y01lOevUvVME2OIIsj2cyPNULwO8d1EJg8/x6yyH78/ngEK+DyVmp2ZES5w1ghM8CS Uhe0yOYB4ZfYgx1xcNhdpP4iIHMWe9sfofglofeHPBrj8vH7ToSdno4xNeULsoS2bN1h J/Bg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1750880346; x=1751485146; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :x-spam-checked-in-group:list-id:mailing-list:precedence:reply-to :x-original-authentication-results:x-original-sender:feedback-id :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:x-beenthere:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=fWEhrPcG9jjLDFgXGjAcw4XIUAV2GeMTzYWUcNStluE=; b=PEdOv6wPoZIKIesn2sEvTq7c9k50NJLB+EWmWBQ90jsx8bfogIFxzTjCWi5zRNUTLm WSOy/dv+B2NwCblpDa03ZJlSMoMmj1fZPUK474aIiTEzTNsY6bPPH0GFriw1Tqcc9zWr 0UPm+v4gLzjSGR8Jm0nNa9tVCzYlNCQA733ZHMIznxtK8fU83iRgfMzfrJdBXHXiak9d ua6PxB/QFV+mVS2USNnzqCqlwwiCLNrlA0uUcuMG7TjKEv7XmAYwuadZ2ngbmIjb9ZGG 0a6FToHilgSX2jrVPFE/4rrAYJCJT7/5TXC81wXY8JEnhU2NJE1bHSlrj6Kq6pQgJwnJ 2NWw== X-Forwarded-Encrypted: i=2; AJvYcCUiUItVjJVwJRJYa6WTPwO2mzj1G9wVBOsVCKmowBXN5ov73K+knPtWLrN0FTl5UK1HAljKLKM=@isar-build.org X-Gm-Message-State: AOJu0YwvsbOCwAWL8V51TxvKeCjyiWO6rOGZmtCIL6n8mil9ea+kPkf7 c9X3Q6lim4qq6NGbHPtlkabazjZXxc6vHse5gmK9S9kC31P1hM1gce4t X-Google-Smtp-Source: AGHT+IFin+Qcbl84ySfeOzxQn0/tF1A6CRlikA0tExepOxa+grJ3rT6wmDts83MOuBthbNPoGevi3g== X-Received: by 2002:a05:6820:508e:b0:611:a243:3889 with SMTP id 006d021491bc7-611a2433b41mr525899eaf.1.1750880345947; Wed, 25 Jun 2025 12:39:05 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com; h=AZMbMZd7kzGt0uKb3GTP0r3H8AqGH+NA84tXk9mmX/7AZqUqgw== Received: by 2002:a05:6820:4886:b0:611:7896:5575 with SMTP id 006d021491bc7-611ab014a7cls93641eaf.0.-pod-prod-09-us; Wed, 25 Jun 2025 12:39:05 -0700 (PDT) X-Received: by 2002:a05:6808:4f20:b0:403:3195:58cb with SMTP id 5614622812f47-40b05c15b91mr4031610b6e.28.1750880344907; Wed, 25 Jun 2025 12:39:04 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1750880344; cv=none; d=google.com; s=arc-20240605; b=gr8L0VcjqEXcYulgSmrhI+EdE4nXZcmxZvgVsDBBPr+9nN/QKdAAW6YbamUnXSvRVH ikc9iM3rvihD8J3VR+6WAq3BNZwl2iMogMD4Am1q0jTUFIsGZBeG6RNiBS/7H9lhwZhF JnkvfXihpPymGMQZlJhmc5dhUMl4N3rDElhq8K+K9pRC2wiwZt0s6SoBinFOwuela8VP I3tJrXdDZQ7m37Tti/aHb8NZdEjCwsBKzRjzdl6j3EumqMdP9RvTVWtNQ+tYUthGKhOB n+stLbWNWOL4OiGbuCb4q603kc6f4cD17couODnKbrrr7xBJ3ouzFQbPCBxV+BexPxX4 aHug== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20240605; h=feedback-id:content-transfer-encoding:mime-version:references :in-reply-to:message-id:date:subject:cc:to:from:dkim-signature; bh=yTHyZtp/Gm/LWcW3yXuXiRnaKYLCtILET9xhsLbpom4=; fh=D/q4xMKxZDyLo2GtmwQ/2prSr9aCFD3HVqTCj43epLY=; b=b1hCdUe4wdGhy5h2zwTCuvNwFA63j93KRIF2ZJciSsUw9OWAOtd8xxXdLc6HC9E5Bl j/MEFt1nQslYGNVQrBYYVMKuWfWuBVEc4QqyQgZv8uOcbTNR1i84ZkMs5SDyKMDeEUAG 2uPdQ2mW48O0O7BGXHgEwrbggpF37z4Obgkrm4VYFPCFzuP7IWTa8eIpMEJzxLH96zxc 1LdHBUJxeOxwRMGaVJJP6USi11W09hjVRUVjthuJ6prH6OYTSkZ64CLewPm6Mlt2MUJj fSc3cUUVdKL7qDTvobkSb5zKFQm/39/dq/mQqlFBFRNi87vPXFi+s+W7ES1jXFbvVqhk bnSg==; dara=google.com ARC-Authentication-Results: i=1; gmr-mx.google.com; dkim=pass header.i=@siemens.com header.s=fm2 header.b=VYmysbnv; spf=pass (google.com: domain of fm-1212295-20250625193904d657f811301bf74e88-rn0ms2@rts-flowmailer.siemens.com designates 185.136.65.228 as permitted sender) smtp.mailfrom=fm-1212295-20250625193904d657f811301bf74e88-rN0Ms2@rts-flowmailer.siemens.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=siemens.com Received: from mta-65-228.siemens.flowmailer.net (mta-65-228.siemens.flowmailer.net. [185.136.65.228]) by gmr-mx.google.com with ESMTPS id 5614622812f47-40ac6d458c8si636949b6e.5.2025.06.25.12.39.04 for (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Wed, 25 Jun 2025 12:39:04 -0700 (PDT) Received-SPF: pass (google.com: domain of fm-1212295-20250625193904d657f811301bf74e88-rn0ms2@rts-flowmailer.siemens.com designates 185.136.65.228 as permitted sender) client-ip=185.136.65.228; Received: by mta-65-228.siemens.flowmailer.net with ESMTPSA id 20250625193904d657f811301bf74e88 for ; Wed, 25 Jun 2025 21:39:04 +0200 X-Patchwork-Original-From: "'Cedric Hombourger' via isar-users" From: Cedric Hombourger To: isar-users@googlegroups.com Cc: felix.moessbauer@siemens.com, Cedric Hombourger Subject: [PATCH v3 2/6] deb-dl-dir: optimize caching of source packages using apt natively Date: Thu, 26 Jun 2025 03:37:44 +0800 Message-Id: <20250625193748.2681-3-cedric.hombourger@siemens.com> In-Reply-To: <20250625193748.2681-1-cedric.hombourger@siemens.com> References: <20250625193748.2681-1-cedric.hombourger@siemens.com> MIME-Version: 1.0 X-Flowmailer-Platform: Siemens Feedback-ID: 519:519-1212295:519-21489:flowmailer X-Original-Sender: cedric.hombourger@siemens.com X-Original-Authentication-Results: gmr-mx.google.com; dkim=pass header.i=@siemens.com header.s=fm2 header.b=VYmysbnv; spf=pass (google.com: domain of fm-1212295-20250625193904d657f811301bf74e88-rn0ms2@rts-flowmailer.siemens.com designates 185.136.65.228 as permitted sender) smtp.mailfrom=fm-1212295-20250625193904d657f811301bf74e88-rN0Ms2@rts-flowmailer.siemens.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=siemens.com X-Original-From: Cedric Hombourger Reply-To: Cedric Hombourger Precedence: list Mailing-list: list isar-users@googlegroups.com; contact isar-users+owners@googlegroups.com List-ID: X-Spam-Checked-In-Group: isar-users@googlegroups.com X-Google-Group-Id: 914930254986 List-Post: , List-Help: , List-Archive: , List-Unsubscribe: , X-Spam-Status: No, score=-4.9 required=5.0 tests=DKIMWL_WL_MED,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,MAILING_LIST_MULTI, RCVD_IN_DNSWL_BLOCKED,RCVD_IN_MSPIKE_H3,RCVD_IN_MSPIKE_WL, RCVD_IN_RP_CERTIFIED,RCVD_IN_RP_RNBL,RCVD_IN_RP_SAFE,T_SPF_TEMPERROR autolearn=unavailable autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on shymkent.ilbers.de X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= source package are downloaded by entering the target rootfs and run apt there. For foreign architectures, this results in apt being executed under QEMU and leads to poor performance. By using the recently introduced rootfs_native_cmd command wrapper, apt will be executed natively against the target rootfs and without elevated privileges. For our test work-load, caching was reduced from more than 10 hours to an hour. Performance is also more consistent as it will no longer depend as to when bitbake kicks caching of source packages for foreign architecture rootfs vs rootfs for the host (in multiconfig builds). Signed-off-by: Cedric Hombourger --- meta/classes/deb-dl-dir.bbclass | 37 ++++++++------------------------- 1 file changed, 9 insertions(+), 28 deletions(-) diff --git a/meta/classes/deb-dl-dir.bbclass b/meta/classes/deb-dl-dir.bbclass index 3f560da4..7026f4f4 100644 --- a/meta/classes/deb-dl-dir.bbclass +++ b/meta/classes/deb-dl-dir.bbclass @@ -5,25 +5,6 @@ inherit repository -debsrc_do_mounts() { - sudo -s <"${DEBSRCDIR}/${rootfs_distro}.lock" - - debsrc_undo_mounts "${rootfs}" } dbg_pkgs_download() {