From patchwork Fri Feb 10 11:09:08 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: roberto.foglietta@linuxteam.org X-Patchwork-Id: 2628 Return-Path: Received: from shymkent.ilbers.de ([unix socket]) by shymkent (Cyrus 2.5.10-Debian-2.5.10-3+deb9u2) with LMTPA; Fri, 10 Feb 2023 12:09:21 +0100 X-Sieve: CMU Sieve 2.4 Received: from mail-lj1-f185.google.com (mail-lj1-f185.google.com [209.85.208.185]) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPS id 31AB9J1d023745 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 10 Feb 2023 12:09:20 +0100 Received: by mail-lj1-f185.google.com with SMTP id b27-20020a2ebc1b000000b002932b3ecf09sf1345055ljf.10; Fri, 10 Feb 2023 03:09:20 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1676027354; cv=pass; d=google.com; s=arc-20160816; b=GyFEWYZ8KPZzY8Ae3sE7Jv1Om6Ht2Ef8XQ2jedWr38DU9kcLPGCc03YxFY5KVO5rHV 1P+rcdrBwGAoODaI/id//rWncjQj6yaoiZGRMvLuYfGgBJCPekL6c0zcUNYhIpp+/ttW NJIYAgZ5IT2hhpVdewLDGY3RNVel0RRNgfvW8F3mKZZonN6omu0TuglwMwB5q8wv09xb DgGGfHcYjcOVbfAEAlhaenbOh1a+kX3HyDSZTNpDFVjCFmplhkOkFd7sv9EUkqdWWji6 N1KCzuVBdA7KKIIm4rWGX6pvwPl8WXS0M0ZFVavu142Hm7O7AEb/8orOdMc2FEKwBZt+ Ij8w== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:mime-version:message-id:date :subject:cc:to:from:sender:dkim-signature; bh=Gq/95pPErUkOI9OqEYU6MHklG9git3cPr/GT7qnVm88=; b=a1QSTP9+v3tqfbi5rvLR/LKM9R14VxsCIMK+setR73LuFgy1veqfX2exqb4BMVM/GA /AJHMN0sEmPTmx2LwFGw8TBnaRLdbjbsm4fF5sKSE7OmRyzLm4UoFYy43mNxppCj6th7 uE6RZ0A3oFl1pvolGIOHGDpvnqZhUI7l+2SZZx3DXRXTUeA9WR1rFdYsq+82a/go3SKu gQC1Fr635jGL7Isr2XshBQG4W6QW942bHFqJ4dGcFHf+lgPsQAIkMnQ8GEoOOafn9dA8 7U3jL+dhMpAMc9/NevQD+dQWWiiAfAF9P351f0jhtqoKY+9AKFNJ+4Ys8AsErOkFe9dm WjGQ== ARC-Authentication-Results: i=2; gmr-mx.google.com; spf=pass (google.com: domain of roberto.foglietta@linuxteam.org designates 5.144.164.165 as permitted sender) smtp.mailfrom=roberto.foglietta@linuxteam.org DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlegroups.com; s=20210112; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:x-original-authentication-results :x-original-sender:mime-version:message-id:date:subject:cc:to:from :sender:from:to:cc:subject:date:message-id:reply-to; bh=Gq/95pPErUkOI9OqEYU6MHklG9git3cPr/GT7qnVm88=; b=VpKIz+SBV9SqpKYsTTCK+IoGgO5FIXXSw4vF+njoiVe+vYbdbsee9M+E33xrMIsHQI WQD68tjduTFTw1DZWLArSCLco1wlRX8WnAhgBuNfhfPVku0wHIum16+MsZ0cz2Mzc5oa A5o42D2jywFxatT8rc9jQ21fDosR5tUI6HgvOh0KQD9OTasa0jDhuG6iQQTIjhuCoZjR 44g4aV+oD/r3n5SnGm73kAwIi/VZsXh2e+Bvsqp7IxARsPMbJMN8DXii93GXtU1WxePy amevViozpQBuR4nZC/WW/jaYOOEboBR+zAa6J9eWgelNe6DNVZwx+9XcQf1FU1eO9zjW FHTA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :x-spam-checked-in-group:list-id:mailing-list:precedence :x-original-authentication-results:x-original-sender:mime-version :message-id:date:subject:cc:to:from:x-gm-message-state:sender:from :to:cc:subject:date:message-id:reply-to; bh=Gq/95pPErUkOI9OqEYU6MHklG9git3cPr/GT7qnVm88=; b=7KOI0I8s0acjFFC/2D9iIAs3K3MYwh6vN3wsUMS4yueRYzxOsQepWhS3h6jwFjqJwP Qh6EOAzaVPbQxnkOTi4x4bCa4t+tC7wuSXMda7nl7SDhZ7HBHgG0JE/gdqcTY622MEkJ 1vsmQ/nz8I3/3XXda40t1LW2wVlbrAtf2bv+tIJ5yl7ljt++lvo5E2j0mFZ/TZ8y3O4H +4rW65KuVOHDdrfTlj7CaLHZ/0sVQBTgdiDQaYp2+DVICpz6ybfp/4fLrXi3iri9tjZn 4ILv+EvKrslu6ElcW6M56/u6LqoeHxdXZfWAFjsj14ZTDkgC5psLR8SjTkcAfp/G9QVn TQWw== Sender: isar-users@googlegroups.com X-Gm-Message-State: AO0yUKW+esjahpWdXgNh1s5cw9A7tdS5db8+YmT1HaMLj2quaF424UJ1 acgglsJE46tFn2jiPyXNt+E= X-Google-Smtp-Source: AK7set8uB34+6vGef1K+cRvc7KorQItCwYCHcUTbWQfoORMKaUsGIDAPCUO2ybGdbyEs8mHL5zEg9w== X-Received: by 2002:a2e:9bcc:0:b0:293:2d3b:29d4 with SMTP id w12-20020a2e9bcc000000b002932d3b29d4mr1422098ljj.91.1676027354302; Fri, 10 Feb 2023 03:09:14 -0800 (PST) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a2e:a178:0:b0:293:32ba:16c6 with SMTP id u24-20020a2ea178000000b0029332ba16c6ls699220ljl.3.-pod-prod-gmail; Fri, 10 Feb 2023 03:09:12 -0800 (PST) X-Received: by 2002:a2e:575d:0:b0:292:7e94:82e5 with SMTP id r29-20020a2e575d000000b002927e9482e5mr4170493ljd.22.1676027352803; Fri, 10 Feb 2023 03:09:12 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1676027352; cv=none; d=google.com; s=arc-20160816; b=xJuH5Sa/cC6BBQp9vriUDJaE0cf281vy4qtJ74GyeiOeccsMJfGEbG35xvJQWQwXgk RuKVXs4Ub0ksp91RKmr2jJS4GjI6VujkbleC/pZ4ULIUr7Fs6KdtO/5HqDkCLwZeycYZ waX3NKHrkMc8hiWzwU7YOyBQ+tgAc7gkZee5Ok5FPDspCODFUA5Cw62OplVq07k03BRr VUjB49Gd+P58z9IX2SkBJxPmpQeRz0oZHcox/tGJ8boximxMrJoiY9eX983qJ02BPAyv Xp3YV6KhYtMH8igHqn9ACbXWITAxrVuqs+xWLdu9v41issRt51eGyu/VzN8uyDqHDDyV VZig== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from; bh=app0qQSaT6llQdUKo9/v2+mE0KtIF5Q31OcTwqAapcc=; b=i4ksVcVqgA8+78pDUD9pB+vUOOuCVEWCC4Ql0BfVkP2oiuoENtXJAmOXfGrBGSydY+ hJeF4kKg59K6P9m3Zs12CWoBxYkTBacg8I9l+AJCInLK5yGa7Wjt34zfKOofEFuPQRj8 Arnbhvgfr+YXMuWEX4JRfu4sge8v9w/jig2Dv9oEz8xDlDNjDAOxr419r/OGiWmqDIWM u0uOeAaiDfjzztDq3qbYvn3U+0J22tVeflMf/H+QzDtreRpqeoSIa7fkdLPmKWXwNgIX /c+Pm19tm0FHRR/aBVmt1tGF+9yBv3EICyde8PIHAFrFn5QfvoxC8lh1pmk+mH90ZhtV V2qg== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of roberto.foglietta@linuxteam.org designates 5.144.164.165 as permitted sender) smtp.mailfrom=roberto.foglietta@linuxteam.org Received: from relay04.th.seeweb.it (relay04.th.seeweb.it. [5.144.164.165]) by gmr-mx.google.com with ESMTPS id z7-20020a2ebcc7000000b00293215eee9esi220068ljp.2.2023.02.10.03.09.12 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 10 Feb 2023 03:09:12 -0800 (PST) Received-SPF: pass (google.com: domain of roberto.foglietta@linuxteam.org designates 5.144.164.165 as permitted sender) client-ip=5.144.164.165; Received: from localhost.localdomain (unknown [IPv6:2a02:8071:3187:7b80:e47:5fa:2d03:1a15]) by m-r1.th.seeweb.it (Postfix) with ESMTPA id 935A12049A; Fri, 10 Feb 2023 12:09:11 +0100 (CET) From: roberto.foglietta@linuxteam.org To: isar-users@googlegroups.com Cc: roberto.foglietta@gmail.com Subject: [PATCH v4] deb-dl-dir class rework to use faster ln -P or fallback to cp Date: Fri, 10 Feb 2023 12:09:08 +0100 Message-Id: <20230210110908.1507520-1-roberto.foglietta@linuxteam.org> X-Mailer: git-send-email 2.34.1 MIME-Version: 1.0 X-Original-Sender: roberto.foglietta@linuxteam.org X-Original-Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of roberto.foglietta@linuxteam.org designates 5.144.164.165 as permitted sender) smtp.mailfrom=roberto.foglietta@linuxteam.org Precedence: list Mailing-list: list isar-users@googlegroups.com; contact isar-users+owners@googlegroups.com List-ID: X-Spam-Checked-In-Group: isar-users@googlegroups.com X-Google-Group-Id: 914930254986 List-Post: , List-Help: , List-Archive: , List-Unsubscribe: , X-Spam-Status: No, score=-0.9 required=5.0 tests=DKIMWL_WL_MED,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_EF,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,RCVD_IN_MSPIKE_H2,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on shymkent.ilbers.de X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= From: "Roberto A. Foglietta" deb-dl-dir, feature: faster when using ln -P otherwise fallback to cp The original class functions deb_dl_dir_import/export were using cp to copy debian package to the target rootfs but this approach is quite slow while using hard link does not work if the destination and source dirs are not lying on the same filesystem. Thus, ln -P should fallback to cp when it does not work (which is different from complaining on stderr). Moreover, these two functions have been reworked to reach a straight forward and more compact form. In particular, export function was using bashism to do some kind of comparison which after all is useless because copying back without overwriting just fulfills that part. More rework using sudo in a different way plus a corner case addressingi, in case the spia file exists for some other reasons. Rebased on the current next and bugfix about checking the destination not the source file. Signed-off-by: Roberto A. Foglietta --- meta/classes/deb-dl-dir.bbclass | 53 +++++++++++++++++++-------------- 1 file changed, 30 insertions(+), 23 deletions(-) diff --git a/meta/classes/deb-dl-dir.bbclass b/meta/classes/deb-dl-dir.bbclass index 7db25251..2a3c7508 100644 --- a/meta/classes/deb-dl-dir.bbclass +++ b/meta/classes/deb-dl-dir.bbclass @@ -78,39 +78,46 @@ debsrc_download() { deb_dl_dir_import() { export pc="${DEBDIR}/${2}" - export rootfs="${1}" - sudo mkdir -p "${rootfs}"/var/cache/apt/archives/ + export sc="${1}/var/cache/apt/archives/" + sudo mkdir -p "${sc}" [ ! -d "${pc}" ] && return 0 - flock -s "${pc}".lock -c ' + export tf=$(cd "${pc}"; ls -1 *.deb | head -n1) + [ ! -e "${pc}/${tf}" ] && return 0 + flock -Fs "${pc}".lock sudo -Es << 'EOFSUDO' set -e printenv | grep -q BB_VERBOSE_LOGS && set -x - sudo find "${pc}" -type f -iname "*\.deb" -exec \ - ln -Pf -t "${rootfs}"/var/cache/apt/archives/ {} + - ' + rm -f "${sc}/${tf}" + ln -Pf -t "${sc}" "${pc}/${tf}" 2>/dev/null ||: + if [ -r "${sc}/${tf}" ]; then + find "${pc}" -type f -iname "*\.deb" -exec \ + ln -Pf -t "${sc}" {} + + else + find "${pc}" -type f -iname "*\.deb" -exec \ + cp -np owner --reflink=auto -t "${sc}" {} + + fi +EOFSUDO } deb_dl_dir_export() { export pc="${DEBDIR}/${2}" - export rootfs="${1}" + export sc="${1}/var/cache/apt/archives/" mkdir -p "${pc}" - flock "${pc}".lock -c ' + export tf=$(cd "${sc}"; ls -1 *.deb | head -n1) + [ ! -e "${sc}/${tf}" ] && return 0 + flock -F "${pc}".lock sudo -Es << 'EOFSUDO' set -e printenv | grep -q BB_VERBOSE_LOGS && set -x - find "${rootfs}"/var/cache/apt/archives/ \ - -maxdepth 1 -type f -iname '*\.deb' |\ - while read p; do - # skip files from a previous export - [ -f "${pc}/${p##*/}" ] && continue - # can not reuse bitbake function here, this is basically - # "repo_contains_package" - package=$(find "${REPO_ISAR_DIR}"/"${DISTRO}" -name ${p##*/}) - if [ -n "$package" ]; then - cmp --silent "$package" "$p" && continue - fi - sudo ln -Pf "${p}" "${pc}" - done - sudo chown -R $(id -u):$(id -g) "${pc}" - ' + rm -f "${pc}/${tf}" + ln -Pf -t "${pc}" "${sc}/${tf}" 2>/dev/null ||: + if [ -r "${pc}/${tf}" ]; then + find "${sc}" -maxdepth 1 -type f -iname '*\.deb' \ + -exec ln -P -t "${pc}" {} + 2>/dev/null ||: + else + find "${sc}" -maxdepth 1 -type f -iname '*\.deb' \ + -exec cp -n --reflink=auto -t "${pc}" {} + + fi + chown -R $(id -u):$(id -g) "${pc}" +EOFSUDO }