From patchwork Wed Sep 2 10:56:19 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vijai Kumar K X-Patchwork-Id: 24 Return-Path: Delivered-To: ilbers.mnt@gmail.com Received: by 2002:a4a:eb04:0:0:0:0:0 with SMTP id f4csp654361ooj; Wed, 2 Sep 2020 11:57:14 -0700 (PDT) X-Received: by 2002:a17:902:8208:: with SMTP id x8mr3298751pln.65.1599073034121; Wed, 02 Sep 2020 11:57:14 -0700 (PDT) ARC-Seal: i=3; a=rsa-sha256; t=1599073034; cv=pass; d=google.com; s=arc-20160816; b=MA+d92ONq5Ls6Imwo+XjMY326fwse7DOpJ1OoC48HCC2aGK8AmUig/0CFsWQjxv3xG zpXqRSea6B1/Opi8mJrZSDV1GSHash2wTN189DSg5PU6mIEM4xRMNB1CSpRtCGQKlWHv OmrgK/g5kJPzp9f9mx6VtWSoR/vCzRha9rw7e4Mdjho4ufCQfbWgPXyBLrVVO/xhLATU SQHBuTtvnsLCKBr69R7x0IT8bQZVXgaPMyT9enPmuyp250TXNbIk7G6p1eNgp5SnVqrO 2EY3DsE+0vVT1n6Jyo2SAX83x1rgFT9rZ6n7AjdjQ6wymdBIFvMQSivF5Cj0LBpO/W5l N44g== ARC-Message-Signature: i=3; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:mime-version:references:in-reply-to :message-id:date:subject:to:from:ironport-sdr:ironport-sdr:sender :dkim-signature; bh=NT/kY5ztWyru9umpiBp+RiaMhbJAL1UwQQgqik0G7Lw=; b=oz6GCzrdAFPU5m2mSmlvYXIYtpfFvV/+boiehoDP/P3OscKheDarzDdLMeJHiFtLFw OyaBK0Xj6JmJekalsoEOsCugvdIEw9kWyz4keVy6e27k1jRV9hUzTtpuv5zoMuX/HGUz DhsOwIAkLpI/fEUrn5ECp/IXVrUtFJEaGUPNTHB4sY2vKSeZRcTQOqZfScEo4EKYEiSu Hs/Gc3Z2SXe6iYnIOhe7cz3EBzSlkugccQAEsCYkDB1k8npttwJHPdpeylcC3Eowicty CTItTH7W7w9LzBnqrvgfPlOUqwirg/CY/66uwX8WZRWZJTwNuaR3v5obByHRRdrLJkSu H2Gg== ARC-Authentication-Results: i=3; mx.google.com; dkim=pass header.i=@googlegroups.com header.s=20161025 header.b=P5HNinNr; arc=pass (i=2 spf=pass spfdomain=mentor.com); spf=pass (google.com: domain of isar-users+bncbcwp7lf37ukrbcowx75akgqef7ttcxi@googlegroups.com designates 209.85.220.55 as permitted sender) smtp.mailfrom=isar-users+bncBCWP7LF37UKRBCOWX75AKGQEF7TTCXI@googlegroups.com Received: from mail-sor-f55.google.com (mail-sor-f55.google.com. [209.85.220.55]) by mx.google.com with SMTPS id lw4sor269221pjb.0.2020.09.02.11.57.13 (Google Transport Security); Wed, 02 Sep 2020 11:57:14 -0700 (PDT) Received-SPF: pass (google.com: domain of isar-users+bncbcwp7lf37ukrbcowx75akgqef7ttcxi@googlegroups.com designates 209.85.220.55 as permitted sender) client-ip=209.85.220.55; Authentication-Results: mx.google.com; dkim=pass header.i=@googlegroups.com header.s=20161025 header.b=P5HNinNr; arc=pass (i=2 spf=pass spfdomain=mentor.com); spf=pass (google.com: domain of isar-users+bncbcwp7lf37ukrbcowx75akgqef7ttcxi@googlegroups.com designates 209.85.220.55 as permitted sender) smtp.mailfrom=isar-users+bncBCWP7LF37UKRBCOWX75AKGQEF7TTCXI@googlegroups.com ARC-Seal: i=2; a=rsa-sha256; t=1599073033; cv=pass; d=google.com; s=arc-20160816; b=Or7e4AStBM/cpc5Su4a/34svmMTiYvbOgiRArA3e/bBkgN/+gPWZJS7pEgx/8tOT/F 3vfV2lr5qqvPoVmusi84CNNcTBmU1JOSZdC1pKTMUxWJ0RsaX95z/2rTgs2mAGEJeVHU Ho8zkz0sg4GYSgJvktfLJjwd3H5Rzl0czpO38FXtdwkNAeHrJHBztgO7uXyCRFBC9Jcz VYxo5sYnEKnu7LAsqGMwnKATCvlV+WZQX9wiUe5AN6naivN4HnLXujDPO0VZZlEOfwhy Vn7nERB+jgC6AORx9ezosRxmB1z9tFZhnGwIQQTy0woGXI30YeFiUK5epAIAR66ah8cv Ha0Q== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:mime-version:references:in-reply-to :message-id:date:subject:to:from:ironport-sdr:ironport-sdr:sender :dkim-signature; bh=NT/kY5ztWyru9umpiBp+RiaMhbJAL1UwQQgqik0G7Lw=; b=hc+KhKRJVwAsCbqQz1BMlWyym501ny98vzU51Jz9JBbBJzWM0+yEjEeLGO+sqYaH8O iGhD3cUPO6Ua/BfiVDFiW1WwI7UXVrKL3nqOvyiUe4doZF4WIoVAcF84YdIEVcMF5kcJ Cj7Q6232nmKIsokrnIzYg+9ZDpXmvtV7/2jXH0WQEQS+F88TNc3zuMCA53IMXmH4KlYB UHK0C4LIu2qEOxrtkG6onQrKXVB4jGsdheCHgaWTxMT5RvYyeK94A8+Bokpz7cmBn6X4 NAh/Kt3Vn5IVo7C1MDG6hpIq/mK42/7t1TeHaeH/5ergVtrCa09FqWQBAiCQTCUHQKfu ZCSQ== ARC-Authentication-Results: i=2; gmr-mx.google.com; spf=pass (google.com: domain of vijaikumar_kanagarajan@mentor.com designates 68.232.129.153 as permitted sender) smtp.mailfrom=Vijaikumar_Kanagarajan@mentor.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlegroups.com; s=20161025; h=sender:ironport-sdr:ironport-sdr:from:to:subject:date:message-id :in-reply-to:references:mime-version:x-original-sender :x-original-authentication-results:precedence:mailing-list:list-id :list-post:list-help:list-archive:list-subscribe:list-unsubscribe; bh=NT/kY5ztWyru9umpiBp+RiaMhbJAL1UwQQgqik0G7Lw=; b=P5HNinNr7ee4OsrUGkOEpOpYe5LlyOdz4GWhmxmrvHYMWLy+JAtY0/qSXOlfp3gKQb mW/OXQyZBCBIVdTCtD7x9B2rHjknOWQ8bVS2ATyiqrNYalNQk4jLCSXMOzIQsLtbSGjC +L5DZ83LPwbnZaD/5dyhHtUUulfzo0hEyM9oROAkSGZd1UYynPdp4i3hbQhqUe+kpej+ DoOUYti5L6vkSRu7GXuMDuduxWk5kPJ2ZoGiL+Es2JbU2LoDb1vwiswMrVibiIyxAUMP feSbLAScvDUZ/QACrzx2hpRgujJzzuIs9YelYqvakex3QKlyv3JIELbSV+jBKRgM43l0 efig== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=sender:x-gm-message-state:ironport-sdr:ironport-sdr:from:to:subject :date:message-id:in-reply-to:references:mime-version :x-original-sender:x-original-authentication-results:precedence :mailing-list:list-id:x-spam-checked-in-group:list-post:list-help :list-archive:list-subscribe:list-unsubscribe; bh=NT/kY5ztWyru9umpiBp+RiaMhbJAL1UwQQgqik0G7Lw=; b=K4zPBHnYkSwnRFXEaVJUaUGJ192yFw6Ti/ueBZKV9BZshMg5cMfwUenVpChdY/ULMy a88M6UeRmGKH9XONIJ1+VdZqrGZoqAZ4O2fEKrYK+Mr1zcHF53sF4nLWyjQXpl4/rLyl WCLfU8krjPgiaYtkTjWU94VlcIr+FWZmQF0bF9mt7FFDT1B8Guvj5oDfpQxANwNc60cW d0admyraKTfaMaLERdmN91URUVpdVxaH8qx02fhamMeD3o9UbyHaXiX/FBbxKWQ1GAoB 4P7MyLCPP399aRR17koaCTvPMKLk63NlyZIZ3SnSkE4rhdxIMIwndO++3pjt646dIYEJ fiCw== Sender: isar-users@googlegroups.com X-Gm-Message-State: AOAM5303q0uI2m1Zi2Cfs2sm54hwLF+KokLDkwz06BQa3eP+gkepfncQ naeEQ+0EetVYX7m7IjnAhTg= X-Google-Smtp-Source: ABdhPJwgoI5JFhqeYdq1z2Exgf7xUgSEzz6J/ENXRKNxNjvDsDAItZxG5x92e42x+INccDwNIVgwOQ== X-Received: by 2002:a17:90a:e016:: with SMTP id u22mr3605923pjy.178.1599073033636; Wed, 02 Sep 2020 11:57:13 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a17:90a:73c9:: with SMTP id n9ls146697pjk.1.experimental-gmail; Wed, 02 Sep 2020 11:57:13 -0700 (PDT) X-Received: by 2002:a17:902:7689:b029:cf:85aa:69f4 with SMTP id m9-20020a1709027689b02900cf85aa69f4mr4771581pll.1.1599073032987; Wed, 02 Sep 2020 11:57:12 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1599073032; cv=none; d=google.com; s=arc-20160816; b=JAcdFrcHDjo+1mrx2rdwHmVPtsq3v9PKDuPNqmIwEWcGf0EAnMq8wxr3Qo59BO8MVq IRCEZTGD+VyCHN9ISjAjt8pBZASTDhCEyeWVERbDCkrqHEpCBg/wFS6+ZiqL8vUrUn4Q kRwmNYfqAKSju8V4SJDqCH9MR8OMxXFAOmO8Z8FnawbKoJUnGQ9I6LsgnuU4ylVVm7pn yZF6BXtM+dBDgEjIiNQO/ymplmWiJfeB5Bmn/s3Yk0UEyIvEzrkHcRjwnPCMy/FtzeRT bgBS2doS/S5bIgT1K8h9dpSwm831zu3zO/qUw8540EgQrBfzeI1ozdZyNB/Fayv+bT1C Vlqw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=mime-version:references:in-reply-to:message-id:date:subject:to:from :ironport-sdr:ironport-sdr; bh=3hESc+kJKtwqKrBuuST3QyiQQb+6pEbodWyFniokofA=; b=zWJUxF6CZxU+0M0u0o81afBunohqXSowFXsr/m8uO9l3YsHtScdnW9t5j1Z0mcjvBv AAz2hLv7R4hyTHsBBsgboLaZaxu/KgHsBmt0lmGG42CUj7e6Ucpm2C1ieWovHgvTKYA+ zp5uAXCjOW7hIYRjkca2GGfm13VK7eBYj7jauorgUnQvtymE+3cujGlT9TCTrPXhcc7B omQ8CviuGDazU5iGBxAJwJCktEfDxrJ+L5S58VT0xFbkQ8YkanPebUJHjX8tTsMtWcq1 38UzAdTDFc2Y7ojqghmezJb9zt4gexNgWl8F4ay6ZxFsZFt/JCz66/Eem62wilkcKyhn xsZA== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of vijaikumar_kanagarajan@mentor.com designates 68.232.129.153 as permitted sender) smtp.mailfrom=Vijaikumar_Kanagarajan@mentor.com Received: from esa1.mentor.iphmx.com (esa1.mentor.iphmx.com. [68.232.129.153]) by gmr-mx.google.com with ESMTPS id a199si43434pfd.1.2020.09.02.11.57.12 for (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Wed, 02 Sep 2020 11:57:12 -0700 (PDT) Received-SPF: pass (google.com: domain of vijaikumar_kanagarajan@mentor.com designates 68.232.129.153 as permitted sender) client-ip=68.232.129.153; IronPort-SDR: /Z7ndzDMf3CO/hoEpgfXhU5NA/uQbZcbMhugy4hndRdxFCQ6FvsHO96uvvDj9KioFvI03XMD1Z /6f6uXwDCkcuKhEt796r0xZYml+Ik4KmMotSRv0RAWBQKX6/SrecA0+XGRdpOvWlWjYCh1Nhxs oOuMOx5MshzZSf7F7x9SNWuzlhr3OEE8eIa1rqyJCrHOs88XRXIhoOBDscnWhTPvnkH8y/BI8y U+7jMgt2jZJvBtMuJFfi2J1QpTB4IxVT+9kZAajPEOUofegqAg6L57bVMFX+2UIc4ZLvLRZtAk +Kc= X-IronPort-AV: E=Sophos;i="5.76,383,1592899200"; d="scan'208";a="54698486" Received: from orw-gwy-02-in.mentorg.com ([192.94.38.167]) by esa1.mentor.iphmx.com with ESMTP; 02 Sep 2020 10:57:12 -0800 IronPort-SDR: i0PZNpSXHXXW3qvHi40ZLsbLbBQYBieV2151Nrl/SYFnNuVO2HjIccpQxMs0XkOBvVWg+UcGXL fQ7bc4BpSt3EI8JAMrRspPL30bR3GzM3QovIs9MDsWsKA42YFvrYjqVNxnFYcTGMdmIpL4CTgg Ec7sf70ebFl/P0gkyroKO/fyR/0RaevX5+958NbAA3JwD056q6Sa5UHDQvSFX88h01Qnzky/a1 yW9pMvtFxPwlECHZ0I7KiTD4GFU3zX+fF47fsiHZXtE6YelnYdjxHoAW/y9J1v/PO3I990EHJW bkw= From: Vijai Kumar K To: , Subject: [PATCH v2 05/10] lib/oe/path: try hardlinking instead of guessing when it might fail Date: Thu, 3 Sep 2020 00:26:19 +0530 Message-ID: <20200902185624.15044-6-Vijaikumar_Kanagarajan@mentor.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200902185624.15044-1-Vijaikumar_Kanagarajan@mentor.com> References: <20200902185624.15044-1-Vijaikumar_Kanagarajan@mentor.com> MIME-Version: 1.0 X-ClientProxiedBy: SVR-ORW-MBX-07.mgc.mentorg.com (147.34.90.207) To svr-orw-mbx-01.mgc.mentorg.com (147.34.90.201) X-Original-Sender: vijaikumar_kanagarajan@mentor.com X-Original-Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of vijaikumar_kanagarajan@mentor.com designates 68.232.129.153 as permitted sender) smtp.mailfrom=Vijaikumar_Kanagarajan@mentor.com Precedence: list Mailing-list: list isar-users@googlegroups.com; contact isar-users+owners@googlegroups.com List-ID: X-Spam-Checked-In-Group: isar-users@googlegroups.com X-Google-Group-Id: 914930254986 List-Post: , List-Help: , List-Archive: , List-Unsubscribe: , X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: =?utf-8?q?1676749605862142312?= X-GMAIL-MSGID: =?utf-8?q?1676749605862142312?= From: Henning Schild The comparison of the stat st_dev is not enough to judge whether hardlinking will work. One example would be where you try and hardlink across two bind-mounts of a directory. The st_dev will be the same and the operation will still fail. Instead of implementing a check to try and figure out hardlink support just try hardlinking and fall back to a copy when running into an exception. Signed-off-by: Henning Schild Signed-off-by: Richard Purdie [Vijai: Imported from OE core f5571bda8327f927feb23b167ab4594b7d0c95bc] Signed-off-by: Vijai Kumar K --- meta/lib/oe/path.py | 24 ++++++++++++++++++------ 1 file changed, 18 insertions(+), 6 deletions(-) diff --git a/meta/lib/oe/path.py b/meta/lib/oe/path.py index c6bb604..3506e2c 100644 --- a/meta/lib/oe/path.py +++ b/meta/lib/oe/path.py @@ -99,7 +99,22 @@ def copyhardlinktree(src, dst): if os.path.isdir(src) and not len(os.listdir(src)): return - if (os.stat(src).st_dev == os.stat(dst).st_dev): + canhard = False + testfile = None + for root, dirs, files in os.walk(src): + if len(files): + testfile = os.path.join(root, files[0]) + break + + if testfile is not None: + try: + os.link(testfile, os.path.join(dst, 'testfile')) + os.unlink(os.path.join(dst, 'testfile')) + canhard = True + except Exception as e: + bb.debug(2, "Hardlink test failed with " + str(e)) + + if (canhard): # Need to copy directories only with tar first since cp will error if two # writers try and create a directory at the same time cmd = "cd %s; find . -type d -print | tar --xattrs --xattrs-include='*' -cf - -S -C %s -p --no-recursion --files-from - | tar --xattrs --xattrs-include='*' -xhf - -C %s" % (src, src, dst) @@ -121,12 +136,9 @@ def copyhardlinktree(src, dst): def copyhardlink(src, dst): """Make a hard link when possible, otherwise copy.""" - # We need to stat the destination directory as the destination file probably - # doesn't exist yet. - dstdir = os.path.dirname(dst) - if os.stat(src).st_dev == os.stat(dstdir).st_dev: + try: os.link(src, dst) - else: + except OSError: shutil.copy(src, dst) def remove(path, recurse=True):