From patchwork Wed Apr 23 12:43:57 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Felix Moessbauer X-Patchwork-Id: 266 Return-Path: Received: from shymkent.ilbers.de ([unix socket]) by shymkent (Cyrus 2.5.10-Debian-2.5.10-3+deb9u2) with LMTPA; Wed, 23 Apr 2025 14:44:17 +0200 X-Sieve: CMU Sieve 2.4 Received: from mail-wm1-f58.google.com (mail-wm1-f58.google.com [209.85.128.58]) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPS id 53NCiGst011682 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Wed, 23 Apr 2025 14:44:16 +0200 Received: by mail-wm1-f58.google.com with SMTP id 5b1f17b1804b1-43ceb011ea5sf35947045e9.2 for ; Wed, 23 Apr 2025 05:44:16 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1745412250; cv=pass; d=google.com; s=arc-20240605; b=RNphMi6PjCkVTkh9exQgXoxY3+jpbGYaDJX8KHmyAZ7QJflq5D3CN8fhZUEKA73lAP IqwoAJujZjv1nwhBNw8VgFynV2ClQe9m0rPao6hunCsUa3XgSjmKnE7kdV889BAFBE8/ g1Zwvhag14jzjtl/o1+p7JRP/oEW4BCJkcTT/6qVFWlI0qEihAjjdoXPp7tb/uQGPcdI Hj+5hyCrGZm8stVcdFQjfHGJFNjs65FMFD0WxglEIU6CtklRWYskeDgmSt83503WPjBR 2cKX9eEWMXAWCOJpRVdOYNMjU/cLOxbHKKJic6EneB1571XDT9bDgn/GKztrOozb9f/N L3uw== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20240605; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:reply-to:feedback-id:mime-version :message-id:date:subject:cc:to:from:dkim-signature; bh=tvDhs1ZWdj9Bw69pqJ2Si9QaEqbgWtWb7hV0yLFiIaQ=; fh=lGCnJrxq+847wkYYDwhPXokwD8qkVWLxD7o+XK1Tv7g=; b=hAugEADc+iaLkAKXNY27V6yyC3kcVBoZeHjDPLvHcE0Dd23Ty1KXUy3Mjgyhy8JUs3 R1pa5P2EO8TJUNYKz18CJk2zrAGaak/HL2uERuRZBinYN5hX4K0YOC0gGgt2NZd97tgY ZukYKM0UY9pHrq8dIiFN6we1yG4TMOd6BEH2tgxEh2rXb6utX0l2mSu5WX9jEJRcyFJ4 hX4cFVJeHvr06ADxaaDVEQTzfsJ26l+MEWCB/W0Qyxnbh5YnyP2MkWCWiWlI+nNqZVaG Br+KDkEEVmPVG+PKIA5C6CyiWCXO2lQyVc1GteIyFoq47c1Fth/yjzOgXH4hkml6ZCdz 93yQ==; darn=isar-build.org ARC-Authentication-Results: i=2; gmr-mx.google.com; dkim=pass header.i=@siemens.com header.s=fm2 header.b=dbPdfrmp; spf=pass (google.com: domain of fm-1321639-20250423124405dfa34a6108760d0642-3mczry@rts-flowmailer.siemens.com designates 185.136.65.226 as permitted sender) smtp.mailfrom=fm-1321639-20250423124405dfa34a6108760d0642-3mCzrY@rts-flowmailer.siemens.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=siemens.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlegroups.com; s=20230601; t=1745412250; x=1746017050; darn=isar-build.org; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:reply-to :x-original-authentication-results:x-original-sender:feedback-id :mime-version:message-id:date:subject:cc:to:from:from:to:cc:subject :date:message-id:reply-to; bh=tvDhs1ZWdj9Bw69pqJ2Si9QaEqbgWtWb7hV0yLFiIaQ=; b=RzOHNp8TfpwLLAH/8JP97I8La0Yrb0TVfE0lXaXj8bvSRqxLfbNsg9IDEV3oBrx7o0 zoeFeXLvFwPtJQ1RFz3i2mgvP/qwiR5RnI2g2tYhTNyp+DxnXZAYrSbr1dTLYhNxDFqy IdxnJIGtfk0y4vH4teDcGikKN924hi9j7kNBN1kBhcTu+iXUnpY69ssg6XxBwAHETn2I HLcMi0S/4NAqQ/PWCn5Ld19ALFn0czvZZo3mMCq/Ca5LbfPSXnc2E2azQWMd7nGBgCTu jg1v5sGJZpY93MSgSxu/KvI04gn8lCOU2wdk9fjsCKJYERl6bMswKE0TVJfMQoBpG2mf /8ng== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1745412250; x=1746017050; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :x-spam-checked-in-group:list-id:mailing-list:precedence:reply-to :x-original-authentication-results:x-original-sender:feedback-id :mime-version:message-id:date:subject:cc:to:from:x-beenthere :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=tvDhs1ZWdj9Bw69pqJ2Si9QaEqbgWtWb7hV0yLFiIaQ=; b=liDk2APyMWwQQyrTYhc+hsfl7hdhn23rGdRrCuYrzByGQaxT2IN+fwVtnolGYEXH9B V4wTpYmz20o4NvCD4WaJ5SmQYSgKAv32L8W6dF4xaD2LJ6BhzZgmZcudnmwaGm8bjsTM ju59mwBt2/Wzql9zQO2+zxaI/srHlIMmcTEaQfCsY1WUliwp+TItQMQX0gEyOYtfUXBQ 0M/1ckUOxsB1EFqQhnu9LlIIfp1FilqCyPlGjnLkTGnSpFiNligWj/tIu9akpyBoD5/Q yk1MEd0+o/cW8AFLdSfssPb+sB03yPa7X/1Jrrzn3qUhgwnM1JEiMkh14tXlscNfo9zc G/Mw== X-Forwarded-Encrypted: i=2; AJvYcCXEnZC1kyzaxHykRCIbEElFygGTPm1KtZaZG7ElHLukqRFXYeBp6iwnRwOccXrIknorXno1wEE=@isar-build.org X-Gm-Message-State: AOJu0YxI0tKlfDQKF5XMx9LL1c4q/mUwn/YuabyVj3GM8dftLRetwQvW 6A5bXvY3zxVkpOO8CQ2QFs9K4qZJfHuxXr7fzHSMaUWkp0nNt7JL X-Google-Smtp-Source: AGHT+IGXt2AR9q5mgCttkDTggikYCEniknV4ivxB7/1GCTs3hcPMY4aVtfKMPrapheudJPNobPODLA== X-Received: by 2002:a05:600c:a363:b0:43d:2230:303b with SMTP id 5b1f17b1804b1-4406ac0a8b4mr156074515e9.20.1745412249106; Wed, 23 Apr 2025 05:44:09 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com; h=ARLLPAIlNRtP5qWgb5rMX8bvNlBy6XfUIKjarkDY0t6i6XUhzg== Received: by 2002:a05:600c:6d54:b0:43c:ec29:489d with SMTP id 5b1f17b1804b1-440623bdefals16457495e9.2.-pod-prod-04-eu; Wed, 23 Apr 2025 05:44:06 -0700 (PDT) X-Received: by 2002:a05:600c:1911:b0:43d:9d5:474d with SMTP id 5b1f17b1804b1-4406aa8b882mr184380445e9.0.1745412246169; Wed, 23 Apr 2025 05:44:06 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1745412246; cv=none; d=google.com; s=arc-20240605; b=bRIWzjLvw/IG/RafxYjbPwTPTCfGGTbajLIJ6N10162bZOEw6t/kLvOr5p7tZJbdmo Rf9ZMfXJl5ytp7N0roJXOyRqlV/pZOIUmpwnfJ59qHP3uf97AjYFp3ndK3U98xopDQY9 H5rGtT4ZLuQmI+1NvhMLMKp8CFWeDrpRGFPN9gOBUhhVCEkkGwQknMDFkEFWBlSbczOZ yPeIdTIB5pakAwqN/mbtqHvnVdN5x5FnJrzlu9s0Gyibs24VObIiwzf+englrW/s07zK KSfGvOwXbm5eQruWI3LRvQINCmdCGg8nmpnNgWHopFW+FJKWzM/FVeuCflt8Cpb4ePTp 6i0A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20240605; h=feedback-id:content-transfer-encoding:mime-version:message-id:date :subject:cc:to:from:dkim-signature; bh=d6Oty8zNrtnMmXra+AZg6Ma4zOwEdRdHXwPKHcRUNYQ=; fh=ihiGjvo7NUvpz5Iu2nNH4iKRnWbkT1eQ96FbYrmsmqY=; b=l3RI+SLiix1lnnhWgbGAc5tJMbo4kpTUYMBO89/YHA2ru2SwNGtn9h/ChJCeuTp6Bg 0tXn0eRPGxcTv1cs7eJXw7o8BaEQXyFIzQezs24nSe6Cestep46KgPlsXm0EWplIxxVP JR8IQeUDgcRwOYO0CXJ9KE9gQ6qr+h7v3wjS1lzqt/aeByhRjlMkScUbTaeiVQfcxQsE NzMuJ07jma2dzgH/xDK9eoC9vRqxjEgt57sPlAOJLvzLg7J0udpyxVXsZAny4e3N7Znm neuVNPGZAMuwk5FVBek7xVcj/R4DcIxjOlYm22UxL3yclDPht5I4R/qh7Qp5mpqyP3F+ G+cg==; dara=google.com ARC-Authentication-Results: i=1; gmr-mx.google.com; dkim=pass header.i=@siemens.com header.s=fm2 header.b=dbPdfrmp; spf=pass (google.com: domain of fm-1321639-20250423124405dfa34a6108760d0642-3mczry@rts-flowmailer.siemens.com designates 185.136.65.226 as permitted sender) smtp.mailfrom=fm-1321639-20250423124405dfa34a6108760d0642-3mCzrY@rts-flowmailer.siemens.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=siemens.com Received: from mta-65-226.siemens.flowmailer.net (mta-65-226.siemens.flowmailer.net. [185.136.65.226]) by gmr-mx.google.com with ESMTPS id 5b1f17b1804b1-4408c7d0d16si1016195e9.0.2025.04.23.05.44.06 for (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Wed, 23 Apr 2025 05:44:06 -0700 (PDT) Received-SPF: pass (google.com: domain of fm-1321639-20250423124405dfa34a6108760d0642-3mczry@rts-flowmailer.siemens.com designates 185.136.65.226 as permitted sender) client-ip=185.136.65.226; Received: by mta-65-226.siemens.flowmailer.net with ESMTPSA id 20250423124405dfa34a6108760d0642 for ; Wed, 23 Apr 2025 14:44:05 +0200 X-Patchwork-Original-From: "'Felix Moessbauer' via isar-users" From: Felix Moessbauer To: isar-users@googlegroups.com Cc: adriaan.schmidt@siemens.com, cedric.hombourger@siemens.com, Felix Moessbauer Subject: [PATCH v2 0/2] Significantly speedup image generation Date: Wed, 23 Apr 2025 14:43:57 +0200 Message-Id: <20250423124359.1081781-1-felix.moessbauer@siemens.com> MIME-Version: 1.0 X-Flowmailer-Platform: Siemens Feedback-ID: 519:519-1321639:519-21489:flowmailer X-Original-Sender: felix.moessbauer@siemens.com X-Original-Authentication-Results: gmr-mx.google.com; dkim=pass header.i=@siemens.com header.s=fm2 header.b=dbPdfrmp; spf=pass (google.com: domain of fm-1321639-20250423124405dfa34a6108760d0642-3mczry@rts-flowmailer.siemens.com designates 185.136.65.226 as permitted sender) smtp.mailfrom=fm-1321639-20250423124405dfa34a6108760d0642-3mCzrY@rts-flowmailer.siemens.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=siemens.com X-Original-From: Felix Moessbauer Reply-To: Felix Moessbauer Precedence: list Mailing-list: list isar-users@googlegroups.com; contact isar-users+owners@googlegroups.com List-ID: X-Spam-Checked-In-Group: isar-users@googlegroups.com X-Google-Group-Id: 914930254986 List-Post: , List-Help: , List-Archive: , List-Unsubscribe: , X-Spam-Status: No, score=-4.9 required=5.0 tests=DKIMWL_WL_MED,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,MAILING_LIST_MULTI, RCVD_IN_DNSWL_BLOCKED,RCVD_IN_MSPIKE_H2,RCVD_IN_RP_CERTIFIED, RCVD_IN_RP_RNBL,RCVD_IN_RP_SAFE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on shymkent.ilbers.de X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= Especially on non native machines, the image generation can take a long time. This is due to the fact that the initrd is generated during the rootfs installation. By that, also the initrd is generated multiple times, despite only one final version is needed. In case of a out-of-rootfs initrd, it is built 5 times (2x in rootfs, 2x in initramfs +1x manually in initramfs). We optimize this by manually controlling when the initrd is built instead of relying on the triggers from the initramfs-tools. As initramfs-tools does not provide a conf.d style mechanism to completely disable the generation, we now install this package during bootstrap and then stub the update-initramfs calls during rootfs installation. In rootfs_postprocess, we restore the original update-initramfs calls and build the initrd manually. Changes since Patch v1: - fix deployment of initrd for ubuntu (dracut) targets - fix incorrect path-globbing in rootfs_quality_check Changes since RFC v3: - Use path-injection during do_packages_install to replace the update-initramfs script - No need to add packages to bootstrap - The initrd stub is deployed and removed in do_rootfs_install. By that, the clean rootfs is deployed into the sstate cache Changes since RFC v2: - Complete re-design of approach to better integrate with sstate cache - Unify logic in rootfs.bblcass and initramfs.bbclass Best regards, Felix Felix Moessbauer (2): delay creation of initrd until end of rootfs install report approximate progress during initrd generation meta/classes/image.bbclass | 15 +--- meta/classes/initramfs.bbclass | 39 +--------- meta/classes/rootfs.bbclass | 78 ++++++++++++++++++- meta/lib/rootfs_progress.py | 33 ++++++-- .../sbuild-chroot/sbuild-chroot.inc | 1 + 5 files changed, 110 insertions(+), 56 deletions(-)