From patchwork Mon Aug 24 02:59:00 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vijai Kumar K X-Patchwork-Id: 12 Return-Path: Delivered-To: ilbers.mnt@gmail.com Received: by 2002:a05:7000:4605:0:0:0:0 with SMTP id b5csp2195665mae; Mon, 24 Aug 2020 03:59:31 -0700 (PDT) X-Received: by 2002:a1c:1bc2:: with SMTP id b185mr5562357wmb.168.1598266771534; Mon, 24 Aug 2020 03:59:31 -0700 (PDT) ARC-Seal: i=3; a=rsa-sha256; t=1598266771; cv=pass; d=google.com; s=arc-20160816; b=Y9kLQHBZ40JawFml7m9tgFR966x+qvoc/z1mI9tTsIuq2XOq4GyHjPnRUDpx85gJYa aRvViDjGCc7hScvTMvVOJVhAGJ5DK4oupUNmprfKZ2ssGBExo/MvouQlZxJof+LsU0ou blRPSIb2LXswYdINfbMY307WrrvSxeozmtr2B+GFteblDhAMwfB7BQrOAkXZlWw476VG xTg1KzyFaIWWwwg+uVFUWYw71gIt9fYBo84jYots3OzabAg48RSFi0eB0TebB3hCe1tH SgEcewI7+xI8okOynYdnsveira5DEJfOrC0MwTJlzhS2lh6hAHGXJFaBmH9uxUGZoTjb h3Vw== ARC-Message-Signature: i=3; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:mime-version:references:in-reply-to :message-id:date:subject:to:from:ironport-sdr:ironport-sdr:sender :dkim-signature; bh=evrsmO3eYXtyAUpBfp9bKLb6E16kXCFw4yqZi15uzWQ=; b=uZqK90XF302QHYY0sgEzi+f0Zh05buQniHR87egDmGih+6NR+mnhOc5+GfVGOOrpzy 9W7zlWhZEvIWCkUGmJXjP6GZX8nCK3gs8+suS3bAr08Z4U4AIBqJznTqaynSvYgYxA5+ Jh5oAQqV1c1OEgAOJml2BZ/WriSqBAGCDOvceRzJdPRnIfGn8gr6Ay9q9kbpms9/Qaon QejyFB4LPSoNgedXIz+GOhab/Z7skq3c4ERK/KfvTgfpHh8kj0XbdwYVIUhZCns55/ve 5jucaIhWxB3GA7Lq3zKIYCy4i1VE64gebrqRsaKJIDFCOXEceD2oxuMn4AdaAaZA1u5n +Qfg== ARC-Authentication-Results: i=3; mx.google.com; dkim=pass header.i=@googlegroups.com header.s=20161025 header.b=Ahjhj0o2; arc=pass (i=2 spf=pass spfdomain=mentor.com); spf=pass (google.com: domain of isar-users+bncbcw7ho6fuybrbev3r35akgqeifszrry@googlegroups.com designates 209.85.220.55 as permitted sender) smtp.mailfrom=isar-users+bncBCW7HO6FUYBRBEV3R35AKGQEIFSZRRY@googlegroups.com Received: from mail-sor-f55.google.com (mail-sor-f55.google.com. [209.85.220.55]) by mx.google.com with SMTPS id d4sor3941850wre.30.2020.08.24.03.59.31 (Google Transport Security); Mon, 24 Aug 2020 03:59:31 -0700 (PDT) Received-SPF: pass (google.com: domain of isar-users+bncbcw7ho6fuybrbev3r35akgqeifszrry@googlegroups.com designates 209.85.220.55 as permitted sender) client-ip=209.85.220.55; Authentication-Results: mx.google.com; dkim=pass header.i=@googlegroups.com header.s=20161025 header.b=Ahjhj0o2; arc=pass (i=2 spf=pass spfdomain=mentor.com); spf=pass (google.com: domain of isar-users+bncbcw7ho6fuybrbev3r35akgqeifszrry@googlegroups.com designates 209.85.220.55 as permitted sender) smtp.mailfrom=isar-users+bncBCW7HO6FUYBRBEV3R35AKGQEIFSZRRY@googlegroups.com ARC-Seal: i=2; a=rsa-sha256; t=1598266771; cv=pass; d=google.com; s=arc-20160816; b=AbFk0h5aXt96p+Y9TG5EvxP216sv6P7Y1sk5PmvJ3NvVzMJlbEU6o7Ynf6fD8K1exp ni1zF6KKBrRJuj2Y8YWQXFaGK6Hpq/tJo5kZNsYXkDvY4Vk9d+ddm0ZHA4YyNSKOyF48 HcrV1MnBQNHcxk2vYw4zxgXBsF3Q8WzJphEBva3tNJ4vffhOVVvkYsir0GBlFpcyagzX VV/k98XNL9hjRepdCShwtvo5kdofFj77H65tZOVS0RMXWeneKUSO/ffq1JeWThFuercY ydf3e2Oncpa/emkr5pQy71S/hJN7FnCd4+gXGocrHhpZSYKCFQ1Am96m8kQTHXK//iqW Eq7w== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:mime-version:references:in-reply-to :message-id:date:subject:to:from:ironport-sdr:ironport-sdr:sender :dkim-signature; bh=evrsmO3eYXtyAUpBfp9bKLb6E16kXCFw4yqZi15uzWQ=; b=JtdShOhXDJmMea/Wd7lKeXv2+NhUWPILHanOGOkW7ytbFYgEQWqlaV9j/JpOztVJEo bfVlDggPYFKPk8RYAOPTtmOCd3vx7P2eQiCz0k6QjU9A7VWMnalO2MwnyvBV+3ABa8SS IGM61S2BsYEXoEnTAlFi0e0EUDBV7P6B1bQMIocUKvUjAqsLHeFYFDSBq8HJMixkaHqk dMRZntHVpmDtM/wp7isDK53KOvt8yu2HoNXl55RNFKZ10XSEOg4Txd06VGr+vCvvjj7x zENFpjJQDgvsNoTjOIh/mo1qnO2GHAKY8xcC4ermOw8WFgHUpi3w63NIgKrAMsBh+VwL XjFQ== ARC-Authentication-Results: i=2; gmr-mx.google.com; spf=pass (google.com: domain of vijaikumar_kanagarajan@mentor.com designates 68.232.137.252 as permitted sender) smtp.mailfrom=Vijaikumar_Kanagarajan@mentor.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlegroups.com; s=20161025; h=sender:ironport-sdr:ironport-sdr:from:to:subject:date:message-id :in-reply-to:references:mime-version:x-original-sender :x-original-authentication-results:precedence:mailing-list:list-id :list-post:list-help:list-archive:list-subscribe:list-unsubscribe; bh=evrsmO3eYXtyAUpBfp9bKLb6E16kXCFw4yqZi15uzWQ=; b=Ahjhj0o2a2LN25dD6Dry0DEa5b9KnK9mOnI6txaH6Zu/XDFDtklj2B7YsCTEAFqO8m t1306Py8n2WFHNAauIJzVgAVp8QRkmOJLPbjxKlI/wA/fgUgQs/jQyAiZ4PXjSkt1X/e +ptSEZaQjl1XdAqliTi0bEv4VRYhZiyMwvQXSvIX+KwxOw7ZrvMlBeZi3EQszD4xgD6D cjhCqGXH0Yu6/w6UO9SOhHZAY/MA5EVJ1xz7ycV3MdAnHFdvTqJmEJd+I0esheVZiDAI 5a6eo2i47Ck744r+IHZ1XV1xr1AKW7nfoQT4YQbB5z63AfmYudAi7ye7tzQfWhA0QUJw a97w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=sender:x-gm-message-state:ironport-sdr:ironport-sdr:from:to:subject :date:message-id:in-reply-to:references:mime-version :x-original-sender:x-original-authentication-results:precedence :mailing-list:list-id:x-spam-checked-in-group:list-post:list-help :list-archive:list-subscribe:list-unsubscribe; bh=evrsmO3eYXtyAUpBfp9bKLb6E16kXCFw4yqZi15uzWQ=; b=G8nGMIcrbgQIxpqDKuFgdQLVBlswIZV2KJ4wzZShvd1xALHXZffMtNOwb43KB2NmZy LakXiSdtUjMpwWvS30Ie22lRT8SOAKoy8GUmUUl6tbeKXJhUn6hhj5YIPeED1BRSoe8J PhPJylVs0S/IBR/SnVMAPBLABT6Epe4Q3BDgQBOdM3gLDThNLoQs8Y9RXQyN7EQd01MS EULva2IXM86pDhRMfJ6yPVT1BMKmubBNxiaDE2KUe/P7/V5N56Hquiv968lKJvzFtxOJ B2q7m5H43ne4PRpp47txkTV8oUBSbtV+C6JS13/pAMzHTFfrOxahafGjzr+B5VUXQ3u5 8mZA== Sender: isar-users@googlegroups.com X-Gm-Message-State: AOAM532kmGX57ftNL0eYDS7m77Cd1ehUR0yfrL5cm00Vkd9RnqCP+ZfF YUQsuTD4ymiVory/0ro9VNs= X-Google-Smtp-Source: ABdhPJxdXmYVUiJaDz8gFX7PCNJiefwV9jqW/RNLzmbjFVlkOWhaOMOWZcni65s/8deoKZmxf8LmgA== X-Received: by 2002:a05:6000:152:: with SMTP id r18mr5841795wrx.204.1598266770868; Mon, 24 Aug 2020 03:59:30 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a05:600c:2ca3:: with SMTP id h3ls4201102wmc.1.canary-gmail; Mon, 24 Aug 2020 03:59:30 -0700 (PDT) X-Received: by 2002:a1c:dc45:: with SMTP id t66mr5377622wmg.17.1598266770282; Mon, 24 Aug 2020 03:59:30 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1598266770; cv=none; d=google.com; s=arc-20160816; b=IZ1qivvRthWJvR99Xgefj6pK5O/iJG0mXGt4pORmvMrsfxUCdk+BssrJK8x7GYVYVl 1Wpx+1uPsJAGPkpN/obsJOhIbMVvjMmK384+yinxuI1NxAUTTGepm+Ny+hZ0cX81CWuR u+d4lU8Oi9zCo427Fh2FpbgFNFFK8EOu4uGu+BzzBgfm9CIYJ7tO91MkvRSS9nK4gnNP KlucqUoShzFskzkQzeHzIY5Z+j1gvyRRGfMDB35AgH3rcNjoDdvFiWDnUfJNJZcgu9bj 8wXgLuoaLm6mSL+9HrJjfpiALL9056A9+NAYazT58g0OFjE+CsV2z7J+u5HL3uGK4NxF pYMQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=mime-version:references:in-reply-to:message-id:date:subject:to:from :ironport-sdr:ironport-sdr; bh=3hESc+kJKtwqKrBuuST3QyiQQb+6pEbodWyFniokofA=; b=G9a5sSfrRY/Bl3T33hpdX8khXO9heXML9vgF70A7C98tbAkGBexYFv0F7GNqNl4EPO q1hIUvYJYK/6K1P1cpC/DYc2cO51EMsrgLoeayZL98HqkyDL8pTQcVQ8C/cFJTgoySi8 5OskPZgTYD+iCKz9OW3/9j5AXx0ytDuXoDJ+EFa1S7cuUDJlGvzLHE+EBnMlSlO+MJgA M1r0pWiG0Vre8yWDWNy4BUemj2oZNknKIrQ/S3YC0GyQgGepmrheb+oA0Pb52SFFwjma 8n7VkvwqlzK4lUrOLQYUoQNSBXsaVnWOxBaTybPs3bM36Yo4I9psaHMLsfh4aOXWa1MO M76w== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of vijaikumar_kanagarajan@mentor.com designates 68.232.137.252 as permitted sender) smtp.mailfrom=Vijaikumar_Kanagarajan@mentor.com Received: from esa4.mentor.iphmx.com (esa4.mentor.iphmx.com. [68.232.137.252]) by gmr-mx.google.com with ESMTPS id i11si331160wra.3.2020.08.24.03.59.29 for (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Mon, 24 Aug 2020 03:59:30 -0700 (PDT) Received-SPF: pass (google.com: domain of vijaikumar_kanagarajan@mentor.com designates 68.232.137.252 as permitted sender) client-ip=68.232.137.252; IronPort-SDR: tTdPFp6Lgy7WMELVL0LwhMjauhf85qs+viCChWkNQz3iV8kQYnGVfttgkVKJHtz79DQdz0tkdS MdD6nuJnUmfbdj0bosrFTLYnSAWXTMyASVskAeOr/iDDLl6Lk2kkTuAPYL5JCs6BkpkjVLS8XB nKxjFifNRIi868iD9llLuMZme2TjoMiMSsrNOupzKYOwdB1fSudGwRe3d2+4D5V4XpL5jpfAje shj4VE85HoAR6jCPyfFhrxAXwfAtzQIXT/fGEQdpALx8e5jiK15fxIttm0/ckdaGjRMOi3stcC 25c= X-IronPort-AV: E=Sophos;i="5.76,348,1592899200"; d="scan'208";a="52309707" Received: from orw-gwy-01-in.mentorg.com ([192.94.38.165]) by esa4.mentor.iphmx.com with ESMTP; 24 Aug 2020 02:59:28 -0800 IronPort-SDR: yQ2RtRZmQg4UncDX3/gnC17Y1tFD67v67zEaui3Rn6L7RVEglD6O9DWnfYZDC24CFQ90Z3qaQd Gn7B8dfywwCVFBtKHAN2om/c/e6IHiJ6K01xckfLOEfTfPMDJABMB+gFm0MqzzBgfU25J2MJ8f RT1kjzfzTvDGCAwWkLMwL5uec8NLj78FmbvGAX1FirvwkXz9f5PZiW0Tq/IABjZq406aPGXTul Y3xG/wWMBefoSJc9Fxurje0BYjjf6KEs5AjL6xjG5YTTC0Flx+r9IY7Z9aB/wu9tLyNlVTvSsF Fr8= From: Vijai Kumar K To: , Subject: [RFC PATCH 05/10] lib/oe/path: try hardlinking instead of guessing when it might fail Date: Mon, 24 Aug 2020 16:29:00 +0530 Message-ID: <20200824105905.21110-1-Vijaikumar_Kanagarajan@mentor.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200824092750.19642-5-Vijaikumar_Kanagarajan@mentor.com> References: <20200824092750.19642-5-Vijaikumar_Kanagarajan@mentor.com> MIME-Version: 1.0 X-ClientProxiedBy: svr-orw-mbx-08.mgc.mentorg.com (147.34.90.208) To svr-orw-mbx-01.mgc.mentorg.com (147.34.90.201) X-Original-Sender: vijaikumar_kanagarajan@mentor.com X-Original-Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of vijaikumar_kanagarajan@mentor.com designates 68.232.137.252 as permitted sender) smtp.mailfrom=Vijaikumar_Kanagarajan@mentor.com Precedence: list Mailing-list: list isar-users@googlegroups.com; contact isar-users+owners@googlegroups.com List-ID: X-Spam-Checked-In-Group: isar-users@googlegroups.com X-Google-Group-Id: 914930254986 List-Post: , List-Help: , List-Archive: , List-Unsubscribe: , X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: =?utf-8?q?1675904178420789218?= X-GMAIL-MSGID: =?utf-8?q?1675904178420789218?= From: Henning Schild The comparison of the stat st_dev is not enough to judge whether hardlinking will work. One example would be where you try and hardlink across two bind-mounts of a directory. The st_dev will be the same and the operation will still fail. Instead of implementing a check to try and figure out hardlink support just try hardlinking and fall back to a copy when running into an exception. Signed-off-by: Henning Schild Signed-off-by: Richard Purdie [Vijai: Imported from OE core f5571bda8327f927feb23b167ab4594b7d0c95bc] Signed-off-by: Vijai Kumar K --- meta/lib/oe/path.py | 24 ++++++++++++++++++------ 1 file changed, 18 insertions(+), 6 deletions(-) diff --git a/meta/lib/oe/path.py b/meta/lib/oe/path.py index c6bb604..3506e2c 100644 --- a/meta/lib/oe/path.py +++ b/meta/lib/oe/path.py @@ -99,7 +99,22 @@ def copyhardlinktree(src, dst): if os.path.isdir(src) and not len(os.listdir(src)): return - if (os.stat(src).st_dev == os.stat(dst).st_dev): + canhard = False + testfile = None + for root, dirs, files in os.walk(src): + if len(files): + testfile = os.path.join(root, files[0]) + break + + if testfile is not None: + try: + os.link(testfile, os.path.join(dst, 'testfile')) + os.unlink(os.path.join(dst, 'testfile')) + canhard = True + except Exception as e: + bb.debug(2, "Hardlink test failed with " + str(e)) + + if (canhard): # Need to copy directories only with tar first since cp will error if two # writers try and create a directory at the same time cmd = "cd %s; find . -type d -print | tar --xattrs --xattrs-include='*' -cf - -S -C %s -p --no-recursion --files-from - | tar --xattrs --xattrs-include='*' -xhf - -C %s" % (src, src, dst) @@ -121,12 +136,9 @@ def copyhardlinktree(src, dst): def copyhardlink(src, dst): """Make a hard link when possible, otherwise copy.""" - # We need to stat the destination directory as the destination file probably - # doesn't exist yet. - dstdir = os.path.dirname(dst) - if os.stat(src).st_dev == os.stat(dstdir).st_dev: + try: os.link(src, dst) - else: + except OSError: shutil.copy(src, dst) def remove(path, recurse=True):