From patchwork Sat Mar 22 18:30:30 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Badrikesh Prusty X-Patchwork-Id: 4125 Return-Path: Received: from shymkent.ilbers.de ([unix socket]) by shymkent (Cyrus 2.5.10-Debian-2.5.10-3+deb9u2) with LMTPA; Sat, 22 Mar 2025 19:31:17 +0100 X-Sieve: CMU Sieve 2.4 Received: from mail-wm1-f56.google.com (mail-wm1-f56.google.com [209.85.128.56]) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPS id 52MIVGjs014441 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Sat, 22 Mar 2025 19:31:16 +0100 Received: by mail-wm1-f56.google.com with SMTP id 5b1f17b1804b1-43ced8c2eb7sf22754615e9.1 for ; Sat, 22 Mar 2025 11:31:16 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1742668271; cv=pass; d=google.com; s=arc-20240605; b=RNfoSg8q1xliAF9qnI70PO2TPcd7+1VJqJw6cWX31FWxOwMBLWxbJkol+ZBp8s98px NQuUOnZKQoChIEii4HfloiewUhjzKqdpfWCPvWL/qD56pgpowVoIAR/V1RA+ogK6dm8b Gg8VOyjkeKHyvwe+45r3RhjlEcF1ZfPmLBLERZmdpp3mO74fACKlRRC4we1Ap3nOUBxc ZD/B5kunV+eBIjeE4VAxDBE3T8O+WowSit1Eet/kZlVRMW/JFSoDySeNl3W1C6/0CAOs 77NOQ3qBO4GvwwqHNPNlst7QHjtRvRnzs3V+PhL7KCiyWbu7u/g46VSPeWwyqf6ciw/m QufA== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20240605; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:reply-to:feedback-id:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=vgw8jxeMPKu425Aogpo2+rCPLqUV8YKPGZXVqeb2plk=; fh=3RhuqSooMgQmHJ0ro+7qFoKf4uaPe392Axc1bk8ZF2Y=; b=LCFDG6YiHPvFV1B4ZYZH/eFNcoUYiA/z2F4DBE9up9AaLE1ytopIc6V4CGJss4qngj LZ0kTCqjiqnkIZ14W9KRqNLwUrnsPhIpsKj4Wn6WjKDc7HjfC1ZFgBfEe0bkrEfx6Bps PGlhEBdNu7lcu6eB4NT4VpLg7euQGBfCww/+UeKHziZ4vLiuoNXLfbR9bDlsdD84LHpJ QiOnALRlua0yw/xL3E8l9h7AvH9k7DreHKvwQa7SBDprCWkagaWFHfig1fwgrKPX38Pr 9wQyPywCgfKqiJrtAXBsn58aM9z0MoLvqBTNaT1nJouiwa3MSkJlyzoX1kW9usJlXLQM KQpQ==; darn=isar-build.org ARC-Authentication-Results: i=2; gmr-mx.google.com; dkim=pass header.i=@siemens.com header.s=fm1 header.b=P2pfRGVX; spf=pass (google.com: domain of fm-1328765-2025032218310450c0b41fe44d114125-ojinqh@rts-flowmailer.siemens.com designates 185.136.64.225 as permitted sender) smtp.mailfrom=fm-1328765-2025032218310450c0b41fe44d114125-oJInqh@rts-flowmailer.siemens.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=siemens.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlegroups.com; s=20230601; t=1742668271; x=1743273071; darn=isar-build.org; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:reply-to :x-original-authentication-results:x-original-sender:feedback-id :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:from:to:cc:subject:date:message-id:reply-to; bh=vgw8jxeMPKu425Aogpo2+rCPLqUV8YKPGZXVqeb2plk=; b=Id+qYPKMoK+gDz2N2sHWrnrsWIoEBsmnrp4c89Ls2zLIZbcSYI4MMwo0lGYhAPjb74 yFTy9J7tFmP+Cm+h64WTEislywu+s1lW+rrid2CBGF49hCK59krl2Z6E69bB541gnL3L Q5ad8z6xOOfzGaUd1DYhZeIzDpRA6S4vkHnNZTj+7UYb2t41bxp3MRFnS0UtA/hRcf55 Ru4HWupL40L/CMaQFPxiVQrQmV2QoaWrM6LjAsFlHJiNZENnbVNVF48zF4DS4DNs0c/E vgty+dw1MIIhrSrXO+dUR2xOi1hmvN2Rp+00uScQ4b8er5FAIHjCqbI9IqF+Er4j4npZ u81Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1742668271; x=1743273071; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :x-spam-checked-in-group:list-id:mailing-list:precedence:reply-to :x-original-authentication-results:x-original-sender:feedback-id :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:x-beenthere:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=vgw8jxeMPKu425Aogpo2+rCPLqUV8YKPGZXVqeb2plk=; b=KoH4lxthLTdqIpzyZPOF7PeiEFtxciY3pgAhFjyWKc7bbi8yT5aawsHBqngqQldylc z6MBYiKw5XcCpbwz5MsPUPjMv019avMC60WG6+SoRGopq9zy3922Y2VHviAv44WcUh+5 uo4HwK282ybZsSccWJKOeSu/TOjj9kTPZO+bGNW6GU1o8Fr8lcCfvVGgrPJOrYYttwb2 JbI6ZQTiZH5ikemqXtDBB+2K8pf+betme+Diy4gjyHNOa7xDKpi+O6m3fCvVmJgzq6eT niCOkOIyNyh5/GXXJRkUTdw/HAibqhRqCIKKPuyH4OPBzjD2lbGiH8easpFJLHZOy/XY fK8w== X-Forwarded-Encrypted: i=2; AJvYcCXGh/oj1C1D3b+Glgv7OKAuyZxr3PyaHjPNxzIfWODOHOHG7mon97rbNZ+LEkdHB1YQktq8ct0=@isar-build.org X-Gm-Message-State: AOJu0Yy1WFHztKZ8hwpG/2VJmiS+vwLNIcdOPtt8e6MVM6afFxpMt+vh j//PQxYfojg7HIS1Pek/YP74lG++ZmlGO7H3Z/OwvzLoODQLBoF7 X-Google-Smtp-Source: AGHT+IGMD5qp7TgOPfJaUrxQzhaZN2nMGpP0HAmJJBgl1aMNvTqDbQKruGq9e8ySmq9582O5GsRleA== X-Received: by 2002:a05:600c:1d9f:b0:43c:eeee:b713 with SMTP id 5b1f17b1804b1-43d50a201a9mr73392505e9.20.1742668269730; Sat, 22 Mar 2025 11:31:09 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com; h=ARLLPALEtKNi496gbBXZHz8db1ZBt2hoSYR8IOzPq+WkKfTpQQ== Received: by 2002:a05:600c:3d8a:b0:43c:edda:8108 with SMTP id 5b1f17b1804b1-43d4ec8d47fls14748675e9.1.-pod-prod-07-eu; Sat, 22 Mar 2025 11:31:05 -0700 (PDT) X-Received: by 2002:a05:600c:470f:b0:43c:f597:d582 with SMTP id 5b1f17b1804b1-43d509e0f82mr63108685e9.1.1742668265317; Sat, 22 Mar 2025 11:31:05 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1742668265; cv=none; d=google.com; s=arc-20240605; b=B1cS8Epg5dEFY63SeIMMWL3AeXMARRhDRFHCzAvcuoo06/cZiw7Ycz2lTDdkNj3he3 neOxZ0mWEgH9HJbYmTMGUcjPN3YYB305PWvW+AhgdFp6/hwXEXvACAPh/mBMfhwANZR1 H5C2IsWrOWcqvmMGyCA8oHN+xU3VybTkO+OtfJBbj2lwkVhLBo12a1ugZKBtpJbu00ID bcQZ89oI53hZOEBX64FqiKGUO59YcHYjStJiCR1UFOdvM52qQ2uTksx1S5ox0imHO6bm PF4IgyOx6B9vKYCmgoincqs8z5oqku+x1katS1Q9peNpQrcxL7C7NkX/9JT+aIQP/m39 1odw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20240605; h=feedback-id:content-transfer-encoding:mime-version:references :in-reply-to:message-id:date:subject:cc:to:from:dkim-signature; bh=RlKJTDG2NENPGBT/oHHLx8inBENJ+xK1fUH/e8shJSg=; fh=IVe7tedBjrVO7NMfxxhSNfR9HdaoPf8yp1e13zSf5B8=; b=NZoVoW3U/MbrMkrJG3asgZiFZwPxT0qnzvCpuelKgmBubiaIKfhzHOPjgORh02/Dok e90jfuFcKIovJRVyehDmOVZniFKbxHm2aKKR8tVUhZoz7xg9vMkAwJnl1ubKg1EkzbwP tS683orVo2JaQ0t2gTaMF7SZ20kDOsT9ZWaAHPXsDs9G+31lllDg2aVC4BMvOEut4CkP nQJI0mCx1vYqx/FOLDPTpinEGxpJmWUV5fbPwcr41l5foHq7VtkE6BJQEK/HJKhIFoAR b6teWV7h4abfarW0MyvjT1QH/Kmu09OAR072SJltNOpCVUYNwJQJK/WAU3iKUZbOLAtE QUFQ==; dara=google.com ARC-Authentication-Results: i=1; gmr-mx.google.com; dkim=pass header.i=@siemens.com header.s=fm1 header.b=P2pfRGVX; spf=pass (google.com: domain of fm-1328765-2025032218310450c0b41fe44d114125-ojinqh@rts-flowmailer.siemens.com designates 185.136.64.225 as permitted sender) smtp.mailfrom=fm-1328765-2025032218310450c0b41fe44d114125-oJInqh@rts-flowmailer.siemens.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=siemens.com Received: from mta-64-225.siemens.flowmailer.net (mta-64-225.siemens.flowmailer.net. [185.136.64.225]) by gmr-mx.google.com with ESMTPS id 5b1f17b1804b1-43d3ad387dbsi5565265e9.0.2025.03.22.11.31.05 for (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Sat, 22 Mar 2025 11:31:05 -0700 (PDT) Received-SPF: pass (google.com: domain of fm-1328765-2025032218310450c0b41fe44d114125-ojinqh@rts-flowmailer.siemens.com designates 185.136.64.225 as permitted sender) client-ip=185.136.64.225; Received: by mta-64-225.siemens.flowmailer.net with ESMTPSA id 2025032218310450c0b41fe44d114125 for ; Sat, 22 Mar 2025 19:31:04 +0100 X-Patchwork-Original-From: "'Badrikesh Prusty' via isar-users" From: Badrikesh Prusty To: isar-users@googlegroups.com Cc: badrikesh prusty Subject: [PATCH v2] meta: cache dbg/dbgsym packages during rootfs_postprocess Date: Sun, 23 Mar 2025 00:00:30 +0530 Message-Id: <20250322183030.1022828-1-badrikesh.prusty@siemens.com> In-Reply-To: References: MIME-Version: 1.0 X-Flowmailer-Platform: Siemens Feedback-ID: 519:519-1328765:519-21489:flowmailer X-Original-Sender: badrikesh.prusty@siemens.com X-Original-Authentication-Results: gmr-mx.google.com; dkim=pass header.i=@siemens.com header.s=fm1 header.b=P2pfRGVX; spf=pass (google.com: domain of fm-1328765-2025032218310450c0b41fe44d114125-ojinqh@rts-flowmailer.siemens.com designates 185.136.64.225 as permitted sender) smtp.mailfrom=fm-1328765-2025032218310450c0b41fe44d114125-oJInqh@rts-flowmailer.siemens.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=siemens.com X-Original-From: Badrikesh Prusty Reply-To: Badrikesh Prusty Precedence: list Mailing-list: list isar-users@googlegroups.com; contact isar-users+owners@googlegroups.com List-ID: X-Spam-Checked-In-Group: isar-users@googlegroups.com X-Google-Group-Id: 914930254986 List-Post: , List-Help: , List-Archive: , List-Unsubscribe: , X-Spam-Status: No, score=-4.9 required=5.0 tests=DKIMWL_WL_MED,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,MAILING_LIST_MULTI, RCVD_IN_DNSWL_BLOCKED,RCVD_IN_MSPIKE_H2,RCVD_IN_RP_CERTIFIED, RCVD_IN_RP_RNBL,RCVD_IN_RP_SAFE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on shymkent.ilbers.de X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= From: badrikesh prusty When software is compiled, it's often "stripped" of debugging symbols to reduce its size. Debug packages contain these debugging symbols. This allows developers to get detailed stack traces when a program crashes. This helps developers identify the exact location and cause of the crash. Debug symbol packages (-dbg/-dbgsym) for .deb binaries in DEBDIR are cached during rootfs_postprocess. Caching is disabled by default. To enable it, set BASE_REPO_FEATURES:append = " cache-dbg-pkgs". During image build, the rootfs_postprocess task downloads these packages to the rootfs, and all the dbg packages syncing with the DEBDIR with the help of rootfs_export_package_cache function. A local.conf.sample option is added to enable this feature, allowing users to uncomment and activate caching as needed. Signed-off-by: badrikesh prusty --- meta-isar/conf/local.conf.sample | 3 +++ meta/classes/deb-dl-dir.bbclass | 18 ++++++++++++++++++ meta/classes/image.bbclass | 1 + meta/classes/rootfs.bbclass | 22 ++++++++++++++++++++++ 4 files changed, 44 insertions(+) diff --git a/meta-isar/conf/local.conf.sample b/meta-isar/conf/local.conf.sample index b59f310b..52ea2fb5 100644 --- a/meta-isar/conf/local.conf.sample +++ b/meta-isar/conf/local.conf.sample @@ -172,6 +172,9 @@ IMAGE_INSTALL:append:qemuamd64-sb = " example-module-signed-${KERNEL_NAME}" # Without this feature, only sources of packages downloaded with apt:// are downloaded. #BASE_REPO_FEATURES ?= "cache-deb-src" +# Uncomment this to enable caching of all debug symbol packages. +#BASE_REPO_FEATURES += "cache-dbg-pkgs" + # # Uncomment this to enable use of cached base repository #ISAR_USE_CACHED_BASE_REPO ?= "1" diff --git a/meta/classes/deb-dl-dir.bbclass b/meta/classes/deb-dl-dir.bbclass index 7ebd057e..1e3f9a16 100644 --- a/meta/classes/deb-dl-dir.bbclass +++ b/meta/classes/deb-dl-dir.bbclass @@ -70,6 +70,24 @@ debsrc_download() { debsrc_undo_mounts "${rootfs}" } +dbg_pkgs_download() { + export rootfs="$1" + + apt-ftparchive --md5=no --sha1=no --sha256=no --sha512=no \ + -a "${DISTRO_ARCH}" packages \ + "${rootfs}/var/cache/apt/archives" \ + | awk '/^Package:/ {print $2}' \ + | sort -u \ + | while read pkg; do + apt-cache -o Dir=${rootfs} showsrc ${pkg} \ + | awk '/^Package-List:/,/^$/' \ + | grep -E "${pkg}-(dbg|dbgsym)" \ + | grep "${DISTRO_ARCH}" \ + | awk '!/Binary:/ {print $1}' \ + | sort -u + done | xargs -r sudo -E chroot ${rootfs} sh -c '/usr/bin/apt-get -y --download-only install "$@"' -- +} + deb_dl_dir_import() { export pc="${DEBDIR}/${2}" export rootfs="${1}" diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass index 56eca202..24c1cb38 100644 --- a/meta/classes/image.bbclass +++ b/meta/classes/image.bbclass @@ -73,6 +73,7 @@ ROOTFS_DPKGSTATUS_DEPLOY_DIR ?= "${DEPLOY_DIR_IMAGE}" ROOTFS_PACKAGE_SUFFIX ?= "${PN}-${DISTRO}-${MACHINE}" ROOTFS_POSTPROCESS_COMMAND:prepend = "${@bb.utils.contains('BASE_REPO_FEATURES', 'cache-deb-src', 'cache_deb_src', '', d)} " +ROOTFS_POSTPROCESS_COMMAND:prepend = "${@bb.utils.contains('BASE_REPO_FEATURES', 'cache-dbg-pkgs', 'cache_dbg_pkgs', '', d)} " inherit rootfs inherit sdk diff --git a/meta/classes/rootfs.bbclass b/meta/classes/rootfs.bbclass index 2348e269..406e9192 100644 --- a/meta/classes/rootfs.bbclass +++ b/meta/classes/rootfs.bbclass @@ -293,6 +293,28 @@ cache_deb_src() { fi } +ROOTFS_POSTPROCESS_COMMAND += "${@bb.utils.contains('ROOTFS_FEATURES', 'cache-dbg-pkgs', 'rootfs_export_package_cache', '', d)}" +cache_dbg_pkgs() { + if [ -e "${ROOTFSDIR}"/etc/resolv.conf ] || + [ -h "${ROOTFSDIR}"/etc/resolv.conf ]; then + sudo mv "${ROOTFSDIR}"/etc/resolv.conf "${ROOTFSDIR}"/etc/resolv.conf.isar + fi + rootfs_install_resolvconf + # Note: ISAR updates the apt state information(apt-get update) only once during bootstrap and + # relies on that through out the build. Copy that state information instead of apt-get update + # which generates a new state from upstream. + sudo tar -xf "${BOOTSTRAP_SRC}" ./var/lib/apt/lists --one-top-level="${ROOTFSDIR}" + + deb_dl_dir_import ${ROOTFSDIR} ${ROOTFS_BASE_DISTRO}-${BASE_DISTRO_CODENAME} + dbg_pkgs_download ${ROOTFSDIR} + + sudo rm -f "${ROOTFSDIR}"/etc/resolv.conf + if [ -e "${ROOTFSDIR}"/etc/resolv.conf.isar ] || + [ -h "${ROOTFSDIR}"/etc/resolv.conf.isar ]; then + sudo mv "${ROOTFSDIR}"/etc/resolv.conf.isar "${ROOTFSDIR}"/etc/resolv.conf + fi +} + ROOTFS_POSTPROCESS_COMMAND += "${@bb.utils.contains('ROOTFS_FEATURES', 'clean-package-cache', 'rootfs_postprocess_clean_package_cache', '', d)}" rootfs_postprocess_clean_package_cache() { sudo -E chroot '${ROOTFSDIR}' \