From patchwork Mon Feb 6 13:09:19 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: roberto.foglietta@linuxteam.org X-Patchwork-Id: 2612 Return-Path: Received: from shymkent.ilbers.de ([unix socket]) by shymkent (Cyrus 2.5.10-Debian-2.5.10-3+deb9u2) with LMTPA; Mon, 06 Feb 2023 14:09:33 +0100 X-Sieve: CMU Sieve 2.4 Received: from mail-lj1-f188.google.com (mail-lj1-f188.google.com [209.85.208.188]) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPS id 316D9TeK005805 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 6 Feb 2023 14:09:30 +0100 Received: by mail-lj1-f188.google.com with SMTP id r12-20020a2e80cc000000b002904f4b5fd2sf2682178ljg.11; Mon, 06 Feb 2023 05:09:30 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1675688964; cv=pass; d=google.com; s=arc-20160816; b=BVMSl2iOCvpE7xpC7jrR7YzGNBKJ+/RPtPGgDMrF75nEven5t4EyZm2l/3c7JCbxzL ngmKeEL8WG1Rk9oCj5RE6XOQAaxk2CUqvSdMrFofIEPelzIQD5WUFywzWSU7XJH8qrln IV4F0A3vgdEt2q69N1TyHHx0hW/hqeO6PBLG1qsMLYODQ6q3upGynva10J2lr6G+wIxv zBcXt3KX2U0fgwPa+UoUrKcQ6z0MCpAJkww4qfJF0tTRMJL80H2HRhvg5lhhj2+aQ0Mo dHcPZsvI0cebgfJvDpKnyDUtAkK5Qt51C/nDbIpKsiUdiX2W1xKmuskERxCMIzgTCCC5 u6AA== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:mime-version:message-id:date :subject:cc:to:from:sender:dkim-signature; bh=3t0OXJ/OoX2c1Iz4Nm6G0vlDc0b5e2dK3G5URvc5XIM=; b=BEeMw+MNAHyUliRsz6x7mmok3oWdp9MPYkhWn2bPOJ0LNjayO/j0EIQte00G2Xywsf vphxuwUeYqM0UrSosks9AKkyy404lMGSzdTOwwR7dfRoGcEoosCsv42f/+r9BWr6DkKz BGGr2XxK2fOBUkvHutyFqQjUpolDFsijbawVaJKC2Yqe/VrmDyYCcu8AidQ++bfROdEU Tn4/OFnzgCLrOsdDdPjJgglmL5LExOqWQ7GTY3UgqrpEt4/cK3qmjZe+iQVYY1d6gwAx RJGHuTc9jpCDzjOlGnz/ZCs3je7qlX2li1fLvvlSUqaq4HLoSkvyTzwCTbSEOrsd5Xa8 nNVg== ARC-Authentication-Results: i=2; gmr-mx.google.com; spf=pass (google.com: domain of roberto.foglietta@linuxteam.org designates 5.144.164.169 as permitted sender) smtp.mailfrom=roberto.foglietta@linuxteam.org DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlegroups.com; s=20210112; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:x-original-authentication-results :x-original-sender:mime-version:message-id:date:subject:cc:to:from :sender:from:to:cc:subject:date:message-id:reply-to; bh=3t0OXJ/OoX2c1Iz4Nm6G0vlDc0b5e2dK3G5URvc5XIM=; b=f6CZeaOnnp9qvDCogBuJhCVIeAyEGeeojH1Mdc5dpAlWt6cVpJ69exWxfoa8XkvK9f N1WQt2V0BNT9PyjoGNvAzd7CrecvlWTltXdqxxXVJR4jD8FAnJHmXvuYNy/8H1K7ENZn zo8i5cw3o0wIsm1AlmEWnEJXjmqipeoiMsPF9ccuN5QZAMTO3ylV0tpxRCZJMZLw+IbM ptVgh7OCu56zCjL2kD6Qz3dWu2jZufe1Ccdtj6YbL7n0i2XTLiDLYW9AcXdKMqHAdJYT Ay0cQJwQrQsDnMDcae4Ph9wBm0n16Hj062U/jmzb5WkzePpevMiYZ+LKajnu6mYP6iMc kzDQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :x-spam-checked-in-group:list-id:mailing-list:precedence :x-original-authentication-results:x-original-sender:mime-version :message-id:date:subject:cc:to:from:x-gm-message-state:sender:from :to:cc:subject:date:message-id:reply-to; bh=3t0OXJ/OoX2c1Iz4Nm6G0vlDc0b5e2dK3G5URvc5XIM=; b=Ye63E1SyIQ2x22qmUO1wwvF7WGB+zWgVWT1gWLupDJXFoUXp1QJWHJbluaj051AdK4 yfGuy2E1TXr+lG8t3Ngh4uVvxXgnr9S4J0nTutPj8R0gs0IeZLm02MrhtLJvqBoHv8iD YXNT4Dlc/2Jfy7Zr9ghjJt2gtbzKQ3tppUfCRKC6Qo52Hd5OpwYM/+lEQBdoabYyNKbt BnhhNH8fXt/qJDloHH3xgxJC/PyfcsnhZs4LnjRv0HSWB093Vzug3kDK7EiRzNyoGrW2 /Fw/cwKpCbUJfNFNuQt2p35xWM46zsh8R9N8BCVh9DQIKv1kT9PBDFfaDHByiSbPQXit APHg== Sender: isar-users@googlegroups.com X-Gm-Message-State: AO0yUKWanjccGtGdJuIQJmphG8psdzpb8wKbbhjagV0oyxRBN1nuuziZ SvKi8+D3YJWFNhAo5DsLSwc= X-Google-Smtp-Source: AK7set87tDRDWuet/orWU4XA75s/+kaijsRp5dhUO2JF8+dcRD2C7CFJcAU4y/y+W9CR/qP9D+ZdwQ== X-Received: by 2002:a05:6512:962:b0:4b5:7290:3dd8 with SMTP id v2-20020a056512096200b004b572903dd8mr3948995lft.254.1675688964262; Mon, 06 Feb 2023 05:09:24 -0800 (PST) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a2e:9048:0:b0:28b:86c7:a456 with SMTP id n8-20020a2e9048000000b0028b86c7a456ls1534660ljg.11.-pod-prod-gmail; Mon, 06 Feb 2023 05:09:22 -0800 (PST) X-Received: by 2002:a2e:8297:0:b0:289:ea7b:8978 with SMTP id y23-20020a2e8297000000b00289ea7b8978mr6760346ljg.8.1675688962804; Mon, 06 Feb 2023 05:09:22 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1675688962; cv=none; d=google.com; s=arc-20160816; b=dGn4hd/Ah+b5TacDVsZSoySR8TmLcxFfACTDohryjT1SmiO8mHCFc8vWFk/flPdmUR 8TC+8qndctnYQmyc/rZTHrQ+VLD8ScBqMWRwDtQsFsd3mX6svkYklJhGZ8/La8BXGUWT THBNC/6Le0tN/zzLlLglWN3CA3qEsAV+NQB69PA1jkK2l2G1ZiS0AWfLUEflePUoLjYz vCyap6TxKUttaDqF0XaCRd8QMtVJdtBUjRQ9IJ92VoO+A9X5b+Fn4iEbZvAxj8mSRKaa Yh7ruqlpkqiO8rkWFhxYB3tx1YvXo6KRj73OlgDR+frl2G2BeMcQAJ5RO5srZreeyH60 HMIw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from; bh=WVeESxTUTC2XOhssbDUkZisL//GqS6vL5q9r5TrmnSg=; b=bO09+ww76R/1hbArHYk+ljaNkdogmYJKaa6lLdpf2O1D7LPgM4XZicgf7fImsPmUA4 4+879VENzfIf1ZtJdyGaZW6RmgKdrFGLCkNxwGFSjmYe5f0eXzKLYDx2gFiWYr46uhpT 4GfhoqTdXmnMGtE+nGiAU9YpAi6IuQX0I+PJYVaGRI5ynDqdq9JG2C56lwkZmCq/G6y0 NwEG90Wl4I4GkeIvbTOlN4NsHkP5fC1mmGJnWPpQcQNuTwk80rZdd8UjVZfbS7DsG2Mr IXs3Ws9kIyqiRs0nhdo4Hey8bdPWrQF8aQp4tm8qDo0/5ZdeTP6hv+hb4P5HzS9/jR7O bBwQ== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of roberto.foglietta@linuxteam.org designates 5.144.164.169 as permitted sender) smtp.mailfrom=roberto.foglietta@linuxteam.org Received: from relay08.th.seeweb.it (relay08.th.seeweb.it. [5.144.164.169]) by gmr-mx.google.com with ESMTPS id d21-20020a05651c089500b0029059a3a89asi421054ljq.4.2023.02.06.05.09.22 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 06 Feb 2023 05:09:22 -0800 (PST) Received-SPF: pass (google.com: domain of roberto.foglietta@linuxteam.org designates 5.144.164.169 as permitted sender) client-ip=5.144.164.169; Received: from localhost.localdomain (unknown [IPv6:2a02:8071:3187:7b80:4081:d461:5a4:5549]) by m-r2.th.seeweb.it (Postfix) with ESMTPA id 9FC103FD02; Mon, 6 Feb 2023 14:09:21 +0100 (CET) From: roberto.foglietta@linuxteam.org To: isar-users@googlegroups.com Cc: roberto.foglietta@gmail.com Subject: [PATCH] deb-dl-dir class rework to use faster ln -P or fallback to cp Date: Mon, 6 Feb 2023 14:09:19 +0100 Message-Id: <20230206130919.327653-1-roberto.foglietta@linuxteam.org> X-Mailer: git-send-email 2.34.1 MIME-Version: 1.0 X-Original-Sender: roberto.foglietta@linuxteam.org X-Original-Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of roberto.foglietta@linuxteam.org designates 5.144.164.169 as permitted sender) smtp.mailfrom=roberto.foglietta@linuxteam.org Precedence: list Mailing-list: list isar-users@googlegroups.com; contact isar-users+owners@googlegroups.com List-ID: X-Spam-Checked-In-Group: isar-users@googlegroups.com X-Google-Group-Id: 914930254986 List-Post: , List-Help: , List-Archive: , List-Unsubscribe: , X-Spam-Status: No, score=-0.9 required=5.0 tests=DKIMWL_WL_MED,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_EF,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,RCVD_IN_MSPIKE_H2,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on shymkent.ilbers.de X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= From: "Roberto A. Foglietta" deb-dl-dir, feature: faster when using ln -P otherwise fallback to cp The original class functions deb_dl_dir_import/export were using cp to copy debian package to the target rootfs but this approach is quite slow while using hard link does not work if the destination and source dirs are not lying on the same filesystem. Thus, ln -P should fallback to cp when it does not work (which is different from complaining on stderr). Moreover, these two functions have been reworked to reach a straight forward and more compact form. In particular, export function was using bashism to do some kind of comparison which after all is useless because copying back without overwriting just fulfills that part. Signed-off-by: Roberto A. Foglietta --- meta/classes/deb-dl-dir.bbclass | 41 ++++++++++++++++++--------------- 1 file changed, 23 insertions(+), 18 deletions(-) diff --git a/meta/classes/deb-dl-dir.bbclass b/meta/classes/deb-dl-dir.bbclass index 3b1517dc..22de7bf6 100644 --- a/meta/classes/deb-dl-dir.bbclass +++ b/meta/classes/deb-dl-dir.bbclass @@ -78,39 +78,44 @@ debsrc_download() { deb_dl_dir_import() { export pc="${DEBDIR}/${2}" - export rootfs="${1}" - sudo mkdir -p "${rootfs}"/var/cache/apt/archives/ + export sc="${1}/var/cache/apt/archives/" + sudo mkdir -p "${sc}" [ ! -d "${pc}" ] && return 0 + export tf=$(ls -1 "${pc}"/*.deb | head -n1) + [ ! -e "${tf}" ] && return 0 flock -s "${pc}".lock -c ' set -e printenv | grep -q BB_VERBOSE_LOGS && set -x - sudo find "${pc}" -type f -iname "*\.deb" -exec \ - cp -n --no-preserve=owner -t "${rootfs}"/var/cache/apt/archives/ {} + + sudo ln -Pf -t "${sc}" "${tf}" 2>/dev/null ||: + if [ -r "${tf}" ]; then + sudo find "${pc}" -type f -iname "*\.deb" -exec \ + ln -Pf -t "${sc}" {} + + else + sudo find "${pc}" -type f -iname "*\.deb" -exec \ + cp -np owner --reflink=auto -t "${sc}" {} + + fi ' } deb_dl_dir_export() { export pc="${DEBDIR}/${2}" - export rootfs="${1}" + export sc="${1}/var/cache/apt/archives/" mkdir -p "${pc}" + export tf=$(ls -1 "${sc}"/*.deb | head -n1) + [ ! -e "${tf}" ] && return 0 flock "${pc}".lock -c ' set -e printenv | grep -q BB_VERBOSE_LOGS && set -x - find "${rootfs}"/var/cache/apt/archives/ \ - -maxdepth 1 -type f -iname '*\.deb' |\ - while read p; do - # skip files from a previous export - [ -f "${pc}/${p##*/}" ] && continue - # can not reuse bitbake function here, this is basically - # "repo_contains_package" - package=$(find "${REPO_ISAR_DIR}"/"${DISTRO}" -name ${p##*/}) - if [ -n "$package" ]; then - cmp --silent "$package" "$p" && continue - fi - sudo cp -n "${p}" "${pc}" - done + sudo ln -Pf -t "${pc}" "${tf}" 2>/dev/null ||: + if [ -r "${tf}" ]; then + sudo find "${sc}" -maxdepth 1 -type f -iname '*\.deb' \ + -exec ln -P -t "${pc}" {} + 2>/dev/null ||: + else + sudo find "${sc}" -maxdepth 1 -type f -iname '*\.deb' \ + -exec cp -n --reflink=auto -t "${pc}" {} + + fi sudo chown -R $(id -u):$(id -g) "${pc}" ' }