From patchwork Thu Apr 10 05:27:49 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Felix Moessbauer X-Patchwork-Id: 257 Return-Path: Received: from shymkent.ilbers.de ([unix socket]) by shymkent (Cyrus 2.5.10-Debian-2.5.10-3+deb9u2) with LMTPA; Thu, 10 Apr 2025 07:28:05 +0200 X-Sieve: CMU Sieve 2.4 Received: from mail-ed1-f61.google.com (mail-ed1-f61.google.com [209.85.208.61]) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPS id 53A5S4GR009423 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Thu, 10 Apr 2025 07:28:04 +0200 Received: by mail-ed1-f61.google.com with SMTP id 4fb4d7f45d1cf-5e5be5ec846sf349340a12.2 for ; Wed, 09 Apr 2025 22:28:04 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1744262879; cv=pass; d=google.com; s=arc-20240605; b=OmcCO6BEXwYIoGlgjWTf8UAjysEng+a1Orysiy8hfJuOieYFGaaUoWe0F9Bu1WthmY hJ5WbPjTTGOACQGburbSOIgqdCuZN2Gy/SMxjnR5K4ejw3JQoDfjf/76mJaZ7OnHPG6j 788NDb/1MZibQjMO0tcXYkBIozHaxTr9IK1gVQdwhellltFxdDDELIv5yswmwGRt+E8I NdqWG5Rxg6brgpNlT66u8XvQ9kf9yWQXTSmEZaMyN85ZKcKQexn8AKuwCF5GEaIKjLrc rzsY6lnrritKOjd0b155LMHSkyuQEG+7yLG9X/PrmfwHhDY3Vw3iVn+5CttihOGMtTov 3F0Q== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20240605; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:reply-to:feedback-id:mime-version :message-id:date:subject:cc:to:from:dkim-signature; bh=vCI2/K7hC2YU4neaOFbOz4Crsu8wc6sx4c5XnbyOqE4=; fh=B8CuTd95CsVq/gH6zU9FdKjg8lFbtgsoCusPHCi2WX0=; b=P1ubyrxIuKXEgQBacminGZkfcGBCQAj/qaGGdvyOChS4ChPP9q3lSigFeiNh75gKlk 6yrtuKmX+tmRephV56dyYSTE/LcnLudFxPPzTyrnuFMNu/f0krHolMJl7Mk47Imu70lY MOlriFnNfSJwzeep7iZa3Cq02jD/2E8v8fR77e83ufyDnEzNVxfGWgnMLXXiFjh717+y Dn4mF/cZ5kLDGM62EscXG9bFuOuvhDKmkC5LOu9tyeIKAw1DjSelWgMIsyWCr0DhK9OY rzKwioJ3740q063ROJIGMHffH/0JqELpadUQC6iXQ4UaY8mIBDxCr1zmb0xdMmaKfOBQ XMZg==; darn=isar-build.org ARC-Authentication-Results: i=2; gmr-mx.google.com; dkim=pass header.i=@siemens.com header.s=fm2 header.b=KUOsWkOr; spf=pass (google.com: domain of fm-1321639-202504100527542323b3dd088583a57e-ptwbwf@rts-flowmailer.siemens.com designates 185.136.65.228 as permitted sender) smtp.mailfrom=fm-1321639-202504100527542323b3dd088583a57e-ptwBwf@rts-flowmailer.siemens.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=siemens.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlegroups.com; s=20230601; t=1744262879; x=1744867679; darn=isar-build.org; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:reply-to :x-original-authentication-results:x-original-sender:feedback-id :mime-version:message-id:date:subject:cc:to:from:from:to:cc:subject :date:message-id:reply-to; bh=vCI2/K7hC2YU4neaOFbOz4Crsu8wc6sx4c5XnbyOqE4=; b=BmndPMhYeCjVS7HtVp+AnU2MOw20XA71zE1o4ka8nWotCF5zd9Po4uJlKoMkxqV+f1 n9VfOaDEPWCnn1viN1YrGiUjJ0Q7rRLAk8ocqtbnvUD/JLVxP7lI6J3X6JY9UiiaZeI/ IHB72GxFvePzJGUltWciXtDXZ0o/jvzrxt0hhpx7qSqwwR2CMZPgi+gmr5GRAP81oiH4 oDDXwVBqqkICcDMtfdmI4aDS+lAIB2/UdXi88+B9Ke1kLKEm7PdTLiWQcK77Jhh+JBGC K98L829cfA/4GROB9klWovea5jcUCkZABw1lFNERS6C+7K8usA+8MoenxSLsawtpzKOa g0Yw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1744262879; x=1744867679; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :x-spam-checked-in-group:list-id:mailing-list:precedence:reply-to :x-original-authentication-results:x-original-sender:feedback-id :mime-version:message-id:date:subject:cc:to:from:x-beenthere :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=vCI2/K7hC2YU4neaOFbOz4Crsu8wc6sx4c5XnbyOqE4=; b=vF8L5YLE6IEb/8m9gDtcKUAcXr0HMks1rK80zNOVURh2CKjiK/p6sC7gsE7pUa1mFI PsQSu6HeXA9LSlHPpvq6QTJEmi0yFAzfxGS0U7TMcThkOGuoVtV+mE0MG5PBEd/gjybh p9tkwREMTq3+9uuzN3nyioZehzewGb3CuZUXZlQQFN/azgfENvPCoJnlAo3AMqU109mS Ole6ucu7NXiOE0I6EVrZqGmMUE3jiKIdanKGkGu4cXmBgr7qeA7hlHHWV4M021+HzW7v pNYPyzVuK66kbyOKhf+UScGFB9CniK//QGiS/4qx6lnUuA1GpZyoM3KBRABzBjcIOpdA 3Llg== X-Forwarded-Encrypted: i=2; AJvYcCXMgF8fHsO7PKX7LI+/zwXcVY980UiMM5TI6o0gs7vY23Dsnx3tqYp65AgELAXC8oTh5Ds5WhU=@isar-build.org X-Gm-Message-State: AOJu0YwGbpowbz5TvSs91k2kmgPPXzyVzfiOncc1y/bxSAnMl7sKeVE+ wiDWeemOO/3vprIpwDFo+9T7KhG5WCi/2C8McY9skprdHojOLqHe X-Google-Smtp-Source: AGHT+IFNxGeyS/e9G7vu54oLE+LRxx/BWTRGl7NAxHDYgttUtlGMCfcpfrwSJObJF5tQWV1ridf/zw== X-Received: by 2002:a05:6402:538e:b0:5e8:bf8b:4396 with SMTP id 4fb4d7f45d1cf-5f32c455309mr827869a12.13.1744262877441; Wed, 09 Apr 2025 22:27:57 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com; h=ARLLPAJIKYYk5+12MbzAtUty0SoYK2GDeDX2IpmIBaTQP26q+g== Received: by 2002:a50:ab14:0:b0:5dc:d1a6:1880 with SMTP id 4fb4d7f45d1cf-5f32765091als404159a12.2.-pod-prod-07-eu; Wed, 09 Apr 2025 22:27:55 -0700 (PDT) X-Received: by 2002:a05:6402:274c:b0:5e0:9252:3550 with SMTP id 4fb4d7f45d1cf-5f32c42e29fmr679851a12.2.1744262874755; Wed, 09 Apr 2025 22:27:54 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1744262874; cv=none; d=google.com; s=arc-20240605; b=QCrs56ZJ4cwcmtELc8+yK04zvaLLdbtKQiu+NGeRPhynyUA1b7qqqFzl20qUdeTMga xmF13KkG/fVIJjlgljbA450aoKoeLLv/dlVlvjHUVQkzQkqJsXOBOeJjc1dO6664hj/D BOoKVDi4VujIm/nJYqXK5Gevq27CzJF4tKf7yYnXvR2euQVzpPfI6tblBYxVNG9NoX2W 3dCP7+4MqC9ZLsjyUxybZOkO469YRctTvKvegnrSFplsiFG/GWQPWKGxES+GbgkgcA60 4fOdDnxcK1fCmmDrIfDgVsF3Q18JOIG1jM1LqszB+CzPO+gigVj1+knQT3Vj/1Nrckjb 2BRA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20240605; h=feedback-id:content-transfer-encoding:mime-version:message-id:date :subject:cc:to:from:dkim-signature; bh=q3TRnSCSkHR99cAJJqo7aslJVrSyDRWLO6fHP/sZWbw=; fh=ihiGjvo7NUvpz5Iu2nNH4iKRnWbkT1eQ96FbYrmsmqY=; b=gsAyCG0kQJn9rDS/wur7+3gLxSF5bL3xIWGlrGciRmdsGrwBVcp5wuAlV43wkeAW9I a8db+lOOX00aeo6JhYzM1G1sWU5nN0iTAeisJQ6cilIq8JdkBCzjE6Zz6dom3AZUVOPF lwGat81MyIdjSY/D6MgqYbPihVNCM3iPTLgl5PoHoj1l8wPbwy2+AMHzmWBS5WrZ8lLQ TG6n1jnERC0nxPx4GOSHhnfrt1UJwmXNhDZs39eG93x4SukQUZJl9x6kn0DY32oJUGET QrGPKVZn+MosUfsyLU57H2vpdevByuY0/pbdTK3upkIv1uAPdHRwHderzECjcoTznlzM ouzg==; dara=google.com ARC-Authentication-Results: i=1; gmr-mx.google.com; dkim=pass header.i=@siemens.com header.s=fm2 header.b=KUOsWkOr; spf=pass (google.com: domain of fm-1321639-202504100527542323b3dd088583a57e-ptwbwf@rts-flowmailer.siemens.com designates 185.136.65.228 as permitted sender) smtp.mailfrom=fm-1321639-202504100527542323b3dd088583a57e-ptwBwf@rts-flowmailer.siemens.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=siemens.com Received: from mta-65-228.siemens.flowmailer.net (mta-65-228.siemens.flowmailer.net. [185.136.65.228]) by gmr-mx.google.com with ESMTPS id 4fb4d7f45d1cf-5f2fbbac511si82322a12.1.2025.04.09.22.27.54 for (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Wed, 09 Apr 2025 22:27:54 -0700 (PDT) Received-SPF: pass (google.com: domain of fm-1321639-202504100527542323b3dd088583a57e-ptwbwf@rts-flowmailer.siemens.com designates 185.136.65.228 as permitted sender) client-ip=185.136.65.228; Received: by mta-65-228.siemens.flowmailer.net with ESMTPSA id 202504100527542323b3dd088583a57e for ; Thu, 10 Apr 2025 07:27:54 +0200 X-Patchwork-Original-From: "'Felix Moessbauer' via isar-users" From: Felix Moessbauer To: isar-users@googlegroups.com Cc: adriaan.schmidt@siemens.com, cedric.hombourger@siemens.com, Felix Moessbauer Subject: [PATCH 0/2] Significantly speedup image generation Date: Thu, 10 Apr 2025 07:27:49 +0200 Message-Id: <20250410052751.3835154-1-felix.moessbauer@siemens.com> MIME-Version: 1.0 X-Flowmailer-Platform: Siemens Feedback-ID: 519:519-1321639:519-21489:flowmailer X-Original-Sender: felix.moessbauer@siemens.com X-Original-Authentication-Results: gmr-mx.google.com; dkim=pass header.i=@siemens.com header.s=fm2 header.b=KUOsWkOr; spf=pass (google.com: domain of fm-1321639-202504100527542323b3dd088583a57e-ptwbwf@rts-flowmailer.siemens.com designates 185.136.65.228 as permitted sender) smtp.mailfrom=fm-1321639-202504100527542323b3dd088583a57e-ptwBwf@rts-flowmailer.siemens.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=siemens.com X-Original-From: Felix Moessbauer Reply-To: Felix Moessbauer Precedence: list Mailing-list: list isar-users@googlegroups.com; contact isar-users+owners@googlegroups.com List-ID: X-Spam-Checked-In-Group: isar-users@googlegroups.com X-Google-Group-Id: 914930254986 List-Post: , List-Help: , List-Archive: , List-Unsubscribe: , X-Spam-Status: No, score=-4.9 required=5.0 tests=DKIMWL_WL_MED,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,MAILING_LIST_MULTI, RCVD_IN_DNSWL_BLOCKED,RCVD_IN_MSPIKE_H2,RCVD_IN_RP_CERTIFIED, RCVD_IN_RP_RNBL,RCVD_IN_RP_SAFE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on shymkent.ilbers.de X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= Especially on non native machines, the image generation can take a long time. This is due to the fact that the initrd is generated during the rootfs installation. By that, also the initrd is generated multiple times, despite only one final version is needed. In case of a out-of-rootfs initrd, it is built 5 times (2x in rootfs, 2x in initramfs +1x manually in initramfs). We optimize this by manually controlling when the initrd is built instead of relying on the triggers from the initramfs-tools. As initramfs-tools does not provide a conf.d style mechanism to completely disable the generation, we now install this package during bootstrap and then stub the update-initramfs calls during rootfs installation. In rootfs_postprocess, we restore the original update-initramfs calls and build the initrd manually. Changes since RFC v3: - Use path-injection during do_packages_install to replace the update-initramfs script - No need to add packages to bootstrap - The initrd stub is deployed and removed in do_rootfs_install. By that, the clean rootfs is deployed into the sstate cache Changes since RFC v2: - Complete re-design of approach to better integrate with sstate cache - Unify logic in rootfs.bblcass and initramfs.bbclass Best regards, Felix Felix Moessbauer (2): delay creation of initrd until end of rootfs install report approximate progress during initrd generation meta/classes/image.bbclass | 17 ++--- meta/classes/initramfs.bbclass | 39 +--------- meta/classes/rootfs.bbclass | 72 ++++++++++++++++++- meta/lib/rootfs_progress.py | 33 +++++++-- .../sbuild-chroot/sbuild-chroot.inc | 1 + 5 files changed, 106 insertions(+), 56 deletions(-)