From patchwork Wed Apr 23 12:43:59 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Felix Moessbauer X-Patchwork-Id: 4195 Return-Path: Received: from shymkent.ilbers.de ([unix socket]) by shymkent (Cyrus 2.5.10-Debian-2.5.10-3+deb9u2) with LMTPA; Wed, 23 Apr 2025 14:44:17 +0200 X-Sieve: CMU Sieve 2.4 Received: from mail-ed1-f64.google.com (mail-ed1-f64.google.com [209.85.208.64]) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPS id 53NCiFwT011672 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Wed, 23 Apr 2025 14:44:15 +0200 Received: by mail-ed1-f64.google.com with SMTP id 4fb4d7f45d1cf-5f05d36993asf6561345a12.0 for ; Wed, 23 Apr 2025 05:44:15 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1745412250; cv=pass; d=google.com; s=arc-20240605; b=lUOhdvopigosDG4vjbQkFRSo2p22DWpKooHzLEHYtuUqf7AiMlMHHU2jS5tsH11m0J uDfmR/PMcNV7WdjEXlWHn/9vehumRHJwGvE9G/fLPmYY3SNiLZtnT0ONH2uJMj8yeCbg VwwOacZCcPhp/qOn+X03ro8xCdN+pEAAVQTgaCbbJYaqrW1vY9aMFJ7uh4WbvBe6U9nt s7bYxXP7loV3ZtL5PZZ+pJsJvDfyguO022ykLFsVP7RCEIU79bzZS1htp+t2lHJpkGU3 /IbzDx6rupDLOnowJOQl1NG9mEBAz+ugbUcKxJiXbx21nxbfd7dBNuFo0s2Xu+2bL98c qR2A== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20240605; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:reply-to:feedback-id:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=t5edAy4yFNmFWmYFwaBG027bNLZyJJwgqu307084pZA=; fh=QJCGRiqUBMM3y8sb729fQRJwf02xsB9p2lNZWN4eJzI=; b=lCokagxUdM8KVW92k0MfBadj9g251+brSZlpm7Vdky/1TTmO8XIOPPerXlfTGhDBEc BexDzFOdPboDifVK2Lky6UW4M8IZF4Pue42JhCmxU4zImJxnoCOrrU61rq1pMfwgoAIV WB6bVaML1JpVB6H47C4WZKkqSWMhslXvBKODtoLiy+0Z9IkpU1WcSdhXupTUICO8ggSl mQyzLB27OSVOfQ6+dSkWdT0dd/d65Vta9luBOmTvIsIbsN8quS06JBpDGp1rOkHbASzt SvSc2R9MwvTxBJtCf+fTddBsX6Yo5rBm6x+WzuXEVZXGdc3eLm3SmWws7Ch0ANRcre+2 Mjkg==; darn=isar-build.org ARC-Authentication-Results: i=2; gmr-mx.google.com; dkim=pass header.i=@siemens.com header.s=fm2 header.b=eD0nlK9V; spf=pass (google.com: domain of fm-1321639-202504231244058ce314500f709e708a-gfsaf1@rts-flowmailer.siemens.com designates 185.136.64.227 as permitted sender) smtp.mailfrom=fm-1321639-202504231244058ce314500f709e708a-gFSaF1@rts-flowmailer.siemens.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=siemens.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlegroups.com; s=20230601; t=1745412250; x=1746017050; darn=isar-build.org; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:reply-to :x-original-authentication-results:x-original-sender:feedback-id :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:from:to:cc:subject:date:message-id:reply-to; bh=t5edAy4yFNmFWmYFwaBG027bNLZyJJwgqu307084pZA=; b=sK5tn+oEXT5KzNG/S/JDf2QDZ0kH30w85Z3hq/TQAcBhB8YPwVhP0emIwFEssZO4pH sfcnXhNReTOsFxBQS9tPllAymA+gp6pNrG6KFpsEYWjW7+VMEmjfUKDGP3c1Bgppe0Kx 1wMb7gqhWok5F/R7U1Z8UzJiTBH4I9uLHLt+7UtGY/NhrAl7nDrwq6YJkOzqeTQZNxBe RRmqns/oNVPyHsmmbeJ16meoVp/mSABvc9RFxA199ONpSY3LNI5YUvr6U5ui5TwbO5Ue z1z5n9WPxl0PtukQhkc5BhuAkOlVL72KTNl6nCYl+8iY5uFwN6Pir4hnH+boE6ExGR4t MQxw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1745412250; x=1746017050; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :x-spam-checked-in-group:list-id:mailing-list:precedence:reply-to :x-original-authentication-results:x-original-sender:feedback-id :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:x-beenthere:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=t5edAy4yFNmFWmYFwaBG027bNLZyJJwgqu307084pZA=; b=QEHzjMVmpQbSZXdOXwBNf/vkFduYxiBGFy+XVWqedDniVIXRXR1UDLavPNbP7Ar1um fPDDfpdA7pLgcMNH7AQyDsR2CpB1n09LuigM8bdflYqUr6FAnR88GMOQNz4NGGg/Ql+s dsMfecJzV4m1eQ5LqNCpnhsSlVIvxi7Zz1B/+Cc0oiu5B6LayKe+frlW4OF9eUPYo0gg TiDrBdanEXE2pWvi8ad2AyqMbsGW0rips/7YTlfGZPW7SfJB6XwOjKQ8emS3vDT67Ad1 DJTHfOAz10PaJhCIZu67GHdryq0j78OLkVMmwXzXj2jEgckLfH2XO22Qkxfn9jX4IkaS OTZQ== X-Forwarded-Encrypted: i=2; AJvYcCVlIb6SjtCRn4vI8IaAKIXB9cgUNFiIPFV4UDLW7t46EjF4HuHYRkgQNwAXSVPmQCsQHh8pRfY=@isar-build.org X-Gm-Message-State: AOJu0YzjNu1nZFBjCsEuaalISym06LI9Ki4raxMQCYoI+Q4Q6FlTR89I CTzNSKMYOtcjrB0wnN1p9ll0YV1m6ZGjLbTpgIPqps1nFoqQPOjF X-Google-Smtp-Source: AGHT+IHmeTwPZGFJ/XNzktWfeIRkacI2ximTZ5vrPj6c0RhnXxBmk9jG0eTVDFmjUrI1QTV/RGPz3g== X-Received: by 2002:a05:6402:40cf:b0:5f4:cad5:360f with SMTP id 4fb4d7f45d1cf-5f6285d430dmr17237115a12.28.1745412249182; Wed, 23 Apr 2025 05:44:09 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com; h=ARLLPAJpA9k1rbqEsu4U4D0G70TdW9s6LQUg3giemLOna2LIrg== Received: by 2002:a05:6402:42c8:b0:5dc:cf28:2128 with SMTP id 4fb4d7f45d1cf-5f6b4f88bfals169167a12.0.-pod-prod-03-eu; Wed, 23 Apr 2025 05:44:06 -0700 (PDT) X-Received: by 2002:a05:6402:13ce:b0:5e6:de5:312f with SMTP id 4fb4d7f45d1cf-5f6285a89e4mr16260393a12.23.1745412246288; Wed, 23 Apr 2025 05:44:06 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1745412246; cv=none; d=google.com; s=arc-20240605; b=RzdkKRzckGuiKJOEQ2Zf490SpwD7ui3VkcB69bQp20tC4oJsvUg3aKwwRKowQCQViS xZP717rF1tg6+tRrYLKiN3Jo9JGeLO48s9orcqIOU+yr3rf7/nuVWK9BMmH3KSHqs08t YxbKwf3VAmCDxu2no1FFkKpfcCikuMW/r3WXd76VyW+NRH7JzfGLCCzuSwQzZFMQe8zH GSsZ3NDI7CDeKhql0rOoDlQPbZ5+tiEbC5/lSNUXl/3fdrCbT/zSf4EUdz3YHD7vfydx 0vQi6SUUioxvmadwPudUeLX5fEV+o+B9mufH1ai+z/jmXhtkzIIsNKOgIAJAov2BxYTH dOXQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20240605; h=feedback-id:content-transfer-encoding:mime-version:references :in-reply-to:message-id:date:subject:cc:to:from:dkim-signature; bh=vVVAObvm+npG1BgVLJffsHmP8s57zQVpO2J5vj9c9pw=; fh=ihiGjvo7NUvpz5Iu2nNH4iKRnWbkT1eQ96FbYrmsmqY=; b=KeH5ITjqIFgbDeo2i9qRNZ9VNURK4a3PScpnWL4eAdshAuGKsc61SDOrE9glGfJWSG PMen7bwFf7euAWJRoccZPzyfq+P6L+eQdLlOmFVHjFo8JwaQ7B8MYuVNgg7IDbeuGa+o Z/te29R90CowqBdGuchXgJbVuHgHN8+thHtFxKmPPh8UB1d2c/9cTc2R8S8CDuDHzKx1 GopVKR2MfoG2PZD6/sDBXbTTESPTCQuW6Z7qtYM+A2d3XdJuJ8HoMzIju+vWfkQKMMR5 qcoOKjU5TgcQzNrSnpYoCEtBiyY3x2QQCL9HLTQvCbv5dxaGUdKGnS6jvTev8A0zopFq 7/qQ==; dara=google.com ARC-Authentication-Results: i=1; gmr-mx.google.com; dkim=pass header.i=@siemens.com header.s=fm2 header.b=eD0nlK9V; spf=pass (google.com: domain of fm-1321639-202504231244058ce314500f709e708a-gfsaf1@rts-flowmailer.siemens.com designates 185.136.64.227 as permitted sender) smtp.mailfrom=fm-1321639-202504231244058ce314500f709e708a-gFSaF1@rts-flowmailer.siemens.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=siemens.com Received: from mta-64-227.siemens.flowmailer.net (mta-64-227.siemens.flowmailer.net. [185.136.64.227]) by gmr-mx.google.com with ESMTPS id 4fb4d7f45d1cf-5f6258343edsi427225a12.4.2025.04.23.05.44.06 for (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Wed, 23 Apr 2025 05:44:06 -0700 (PDT) Received-SPF: pass (google.com: domain of fm-1321639-202504231244058ce314500f709e708a-gfsaf1@rts-flowmailer.siemens.com designates 185.136.64.227 as permitted sender) client-ip=185.136.64.227; Received: by mta-64-227.siemens.flowmailer.net with ESMTPSA id 202504231244058ce314500f709e708a for ; Wed, 23 Apr 2025 14:44:06 +0200 X-Patchwork-Original-From: "'Felix Moessbauer' via isar-users" From: Felix Moessbauer To: isar-users@googlegroups.com Cc: adriaan.schmidt@siemens.com, cedric.hombourger@siemens.com, Felix Moessbauer Subject: [PATCH v2 2/2] report approximate progress during initrd generation Date: Wed, 23 Apr 2025 14:43:59 +0200 Message-Id: <20250423124359.1081781-3-felix.moessbauer@siemens.com> In-Reply-To: <20250423124359.1081781-1-felix.moessbauer@siemens.com> References: <20250423124359.1081781-1-felix.moessbauer@siemens.com> MIME-Version: 1.0 X-Flowmailer-Platform: Siemens Feedback-ID: 519:519-1321639:519-21489:flowmailer X-Original-Sender: felix.moessbauer@siemens.com X-Original-Authentication-Results: gmr-mx.google.com; dkim=pass header.i=@siemens.com header.s=fm2 header.b=eD0nlK9V; spf=pass (google.com: domain of fm-1321639-202504231244058ce314500f709e708a-gfsaf1@rts-flowmailer.siemens.com designates 185.136.64.227 as permitted sender) smtp.mailfrom=fm-1321639-202504231244058ce314500f709e708a-gFSaF1@rts-flowmailer.siemens.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=siemens.com X-Original-From: Felix Moessbauer Reply-To: Felix Moessbauer Precedence: list Mailing-list: list isar-users@googlegroups.com; contact isar-users+owners@googlegroups.com List-ID: X-Spam-Checked-In-Group: isar-users@googlegroups.com X-Google-Group-Id: 914930254986 List-Post: , List-Help: , List-Archive: , List-Unsubscribe: , X-Spam-Status: No, score=-4.9 required=5.0 tests=DKIMWL_WL_MED,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,MAILING_LIST_MULTI, RCVD_IN_DNSWL_BLOCKED,RCVD_IN_MSPIKE_H2,RCVD_IN_RP_CERTIFIED, RCVD_IN_RP_RNBL,RCVD_IN_RP_SAFE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on shymkent.ilbers.de X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= On non native architectures, the initrd generation has to be emulated. In combination with kernels that provide many modules (like the distro kernels), this leads to long initrd build times. To give a rough estimation of the duration, we add progress reporting to that step. As we always build the initrd with MODULES=most, we know from experiments, that ~50% of the kernel modules of the rootfs are included in the initrd. We use that number as a guesstimate, as we don't have precise numbers when starting the task. Signed-off-by: Felix Moessbauer --- meta/classes/rootfs.bbclass | 6 ++++++ meta/lib/rootfs_progress.py | 33 ++++++++++++++++++++++++++++----- 2 files changed, 34 insertions(+), 5 deletions(-) diff --git a/meta/classes/rootfs.bbclass b/meta/classes/rootfs.bbclass index fd6bb24a..ab30ab91 100644 --- a/meta/classes/rootfs.bbclass +++ b/meta/classes/rootfs.bbclass @@ -454,6 +454,10 @@ do_generate_initramfs[sstate-outputdirs] = "${DEPLOY_DIR_IMAGE}" python do_generate_initramfs() { bb.build.exec_func('rootfs_do_mounts', d) bb.build.exec_func('rootfs_do_qemu', d) + + progress_reporter = bb.progress.ProgressHandler(d) + d.rootfs_progress = progress_reporter + try: bb.build.exec_func('rootfs_generate_initramfs', d) finally: @@ -468,7 +472,9 @@ rootfs_generate_initramfs[progress] = "custom:rootfs_progress.InitrdProgressHand rootfs_generate_initramfs() { if [ -n "$(sudo find '${ROOTFSDIR}/boot' -type f -name 'vmlinu[xz]*')" ]; then sudo -E chroot "${ROOTFSDIR}" sh -c '\ + mods_total="$(find /usr/lib/modules -type f -name '*.ko*' | wc -l)"; \ export kernel_version=$(basename /boot/vmlinu[xz]-* | cut -d'-' -f2-); \ + echo "Total number of modules: $mods_total"; \ echo "Generating initrd for kernel version: $kernel_version"; \ update-initramfs -u -v -k "$kernel_version";' if [ -n "${INITRD_DEPLOY_FILE}" ]; then diff --git a/meta/lib/rootfs_progress.py b/meta/lib/rootfs_progress.py index f808852f..1cc70c87 100644 --- a/meta/lib/rootfs_progress.py +++ b/meta/lib/rootfs_progress.py @@ -28,14 +28,16 @@ class PkgsProgressHandler(bb.progress.ProgressHandler): self._linebuffer = self._linebuffer[breakpos:] if self._stage == 'prepare': - match = re.search( - r'^([0-9]+) upgraded, ([0-9]+) newly installed', line) - if match: - self._num_pkgs = int(match.group(1)) + int(match.group(2)) - self._stage = 'post-prepare' + self.process_total(line) else: self.process_line(line) + def process_total(self, line): + m = re.search(r'^([0-9]+) upgraded, ([0-9]+) newly installed', line) + if m: + self._num_pkgs = int(m.group(1)) + int(m.group(2)) + self._stage = 'post-prepare' + def process_line(self, line): return @@ -65,3 +67,24 @@ class PkgsInstallProgressHandler(PkgsProgressHandler): progress = 99 * (self._pkg + self._pkg_set_up) / (self._num_pkgs * 2) self._progress.update(progress) + + +class InitrdProgressHandler(PkgsProgressHandler): + def __init__(self, d, outfile, otherargs=None): + super().__init__(d, outfile) + + def process_total(self, line): + m = re.search(r'^Total number of modules: ([0-9]+)', line) + if m: + # in MODULES=most mode, we install ~half of all modules + self._num_pkgs = int(m.group(1)) // 2 + self._stage = 'post-prepare' + + def process_line(self, line): + if line.startswith('Adding module'): + self._pkg += 1 + elif line.startswith('(excluding'): + self._pkg += len(line.split(' ')) - 1 + else: + return + self._progress.update(99 * self._pkg / self._num_pkgs)