From patchwork Mon Sep 7 08:20:14 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vijai Kumar K X-Patchwork-Id: 39 Return-Path: Delivered-To: ilbers.mnt@gmail.com Received: by 2002:a4a:eb04:0:0:0:0:0 with SMTP id f4csp3467831ooj; Mon, 7 Sep 2020 09:20:42 -0700 (PDT) X-Received: by 2002:a17:906:cec9:: with SMTP id si9mr21298007ejb.351.1599495642363; Mon, 07 Sep 2020 09:20:42 -0700 (PDT) ARC-Seal: i=3; a=rsa-sha256; t=1599495642; cv=pass; d=google.com; s=arc-20160816; b=xsNYsFjRCZXA4cOhBu0dzAs5hrCJnT9Fu7bmUk2gyaNCN8oDsHfiqtLhIaaA53GJU9 X0EOgBoQwTxnTj3JYOMtxUVzaoPmE2kE8cGFmJFvxOHo4rM+B+W/RA6SBdYjWzONqxpi Kn1aAHSgKK1K4pQ/jNjk3sDzq9xE/eznAQWkh4hmZiW1QWA9i3xYzScFNNVQDbDTZvA+ kFLSBHSpkyrwMk8i1ECdMqeOw9LnDauBR5ykSSSo0ghLbrf8at6Bm/IvHJXXnWLxF0L0 FDenPLDw13VbmDzJ82Cgz1xIl2+kccHxLXAzx7ouKfTeijzM6cpzSjTlaqi9nARsYusI JcSw== ARC-Message-Signature: i=3; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:mime-version:references:in-reply-to :message-id:date:subject:to:from:ironport-sdr:ironport-sdr:sender :dkim-signature; bh=fSp4QXVeISrSejrnjQJxKzyoVbBtcuDtlYgvgFnaUcs=; b=HuB9N+v+tW5Qp/bjdWexMc6S0HZvp4QUpnzx/UXsVvjRAlT5BsFDoh/EBh+q1EAYkC 3EN5le7YkHNGHpAUZfGEBHIMu8x1WM+MzbwGT9t0GDtMM4///m+lZyWXjLg24pghMqqf wD3HiSDb3UeHa7rEH9CqZo3eqLffGLRHlTAkt8YqJWaf4L9DxJurMW+AiwHaMzn371Eb fZQA2MIm1KOtDLwMTP7YUC69is59CuokCb5VQPW2SAr9lYEKcWDXwhcmzEtUX5l4rfqu zPMjPBwEtEzRcAszFXaujifUNYJSufrxm1P15RtYxScU6t1dhtUXUDeuKHnbmTvH3kjO I/iw== ARC-Authentication-Results: i=3; mx.google.com; dkim=pass header.i=@googlegroups.com header.s=20161025 header.b=Sxedsqpw; arc=pass (i=2 spf=pass spfdomain=mentor.com); spf=pass (google.com: domain of isar-users+bncbcwp7lf37ukrbwn33h5akgqeoxaljjy@googlegroups.com designates 209.85.220.55 as permitted sender) smtp.mailfrom=isar-users+bncBCWP7LF37UKRBWN33H5AKGQEOXALJJY@googlegroups.com Received: from mail-sor-f55.google.com (mail-sor-f55.google.com. [209.85.220.55]) by mx.google.com with SMTPS id i20sor1084751edu.10.2020.09.07.09.20.42 (Google Transport Security); Mon, 07 Sep 2020 09:20:42 -0700 (PDT) Received-SPF: pass (google.com: domain of isar-users+bncbcwp7lf37ukrbwn33h5akgqeoxaljjy@googlegroups.com designates 209.85.220.55 as permitted sender) client-ip=209.85.220.55; Authentication-Results: mx.google.com; dkim=pass header.i=@googlegroups.com header.s=20161025 header.b=Sxedsqpw; arc=pass (i=2 spf=pass spfdomain=mentor.com); spf=pass (google.com: domain of isar-users+bncbcwp7lf37ukrbwn33h5akgqeoxaljjy@googlegroups.com designates 209.85.220.55 as permitted sender) smtp.mailfrom=isar-users+bncBCWP7LF37UKRBWN33H5AKGQEOXALJJY@googlegroups.com ARC-Seal: i=2; a=rsa-sha256; t=1599495642; cv=pass; d=google.com; s=arc-20160816; b=cij9/Gz5XsdqwdJb6Vs8d/BlsKrK10XZBnar65ORjVTwcbMBP22duz/plmksm5VX+0 enOXm3YKmawnO4fmEeEfgYuLY2K9NQze9c+L1YPhGs+pdJhOxzX7VTFgNz3xOcLKrqIb +ZIIX2ozeM55ALnbIqJ6iqzdaIuvC1yZ4Ck8vMVy6W+MLmHEkea0ShsbpN555tkPGb+n Eis9KTrBwHd/5W4ZhrUnBJGPg9RQjELEU1GAX+twrv88B90W9T742DbX1A5AEeGkN8PX o9AGZTZ2+khUJ+YYv/G0IjMSjcr63f25p5FYcm2hCZBTH7jq7iOApTKOpgZ2uIoHs1Je E9sQ== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:mime-version:references:in-reply-to :message-id:date:subject:to:from:ironport-sdr:ironport-sdr:sender :dkim-signature; bh=fSp4QXVeISrSejrnjQJxKzyoVbBtcuDtlYgvgFnaUcs=; b=C34jPGvLH6IewwgnLu0UJ01P0ursuJgyIqLaeKR7L5dLHrQah0+BTzbPntr8RwmQtk 9KyeoE21iVBseoryp6gorjbFLKXgR13cgtUUmJAnrsfL1Q/oDSxWCgz2YfGt2/JojCPC is5YgwxFGQ4kUFIjmOCaxCMOk2yIT7yy2J1fPt5sncVIhXi8l4Vt4HR15AWjfP1w/uBZ GsjJudeagk7jd5myZOwO8f77L32vkdz2siiAORtj/Wxn1yaL2m5OeTqhVziS/FF6Dh0y bLIX/C1WcRR6YbuAB0FIhEerhJF2dOVW+ySmeXlFd2Guac08C9y6QHKVSfIVTyo3iuy8 gIYw== ARC-Authentication-Results: i=2; gmr-mx.google.com; spf=pass (google.com: domain of vijaikumar_kanagarajan@mentor.com designates 68.232.137.252 as permitted sender) smtp.mailfrom=Vijaikumar_Kanagarajan@mentor.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlegroups.com; s=20161025; h=sender:ironport-sdr:ironport-sdr:from:to:subject:date:message-id :in-reply-to:references:mime-version:x-original-sender :x-original-authentication-results:precedence:mailing-list:list-id :list-post:list-help:list-archive:list-subscribe:list-unsubscribe; bh=fSp4QXVeISrSejrnjQJxKzyoVbBtcuDtlYgvgFnaUcs=; b=SxedsqpwCu11mu7x39LkOrVripnVswkL7LV1qQpSwllNGKMMMwGE9Eoa157h1paq4u s7/6sCkvu1wj+R/PCDCgAiKj0hwdYe4MFGVCoqp/R70cxdibpTzvXH1iz8pn14C8HsZx V59cCnwixj3UOGAOoC4DTmrDrX6OnHeYoirYWVZi/72O9RWUmJ5ArsB6OM/h+nrX2XRm Tj7MJo6nBaZcvMJl+QS0vKSkQyK3nQrLmtNBg+n+w5A6c90JXjYYNj4NpO92dVoMnweJ KSyoiuOUFBvQMZFD0sKwCJSOTotEneyO7D2ay4sCad54IvrCY1Kfcr0w1grZSawVI+TB ghOw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=sender:x-gm-message-state:ironport-sdr:ironport-sdr:from:to:subject :date:message-id:in-reply-to:references:mime-version :x-original-sender:x-original-authentication-results:precedence :mailing-list:list-id:x-spam-checked-in-group:list-post:list-help :list-archive:list-subscribe:list-unsubscribe; bh=fSp4QXVeISrSejrnjQJxKzyoVbBtcuDtlYgvgFnaUcs=; b=knWjgQmaGQmtoAaWyntT0EMElPWh0zGBD19l1WAYgtABCcdDh3auAYoFWAVPKRtwf9 6GjGaOtExlsmscmuOIrHtnKVhMfEd7ec3qWfHiuANW1ssAvBRcfl2pVczNnaspwLkeAQ sNG7sHvfcsSFYq3MsNR1+JajrYm9dznLJfOszDGRUEVrU33oh2P3ASIZKwQ4vjJkhGA6 Zxmt/5i/67n551Cs0jpcMBeohrp0hmWjgzwSwAMFX/lUQMfN/wrkkg2NgXTgJUUieaiz LIMwJc197uEf2S9bTURB5BxNF0G5u+4S5PZFgLqSH/508u0MsEie+jweDiVRH+dbHC7E B8rA== Sender: isar-users@googlegroups.com X-Gm-Message-State: AOAM531irBPoAzjxA4AIgylLSBeWT3UQZGBHjFAnQfBXMFmqiz+J+Lsz SXP7rrnM4PcoEcW5xYGVlQo= X-Google-Smtp-Source: ABdhPJxEBpNMOK5eBLfjSHEw+hjMv5z/ZQXsXVPFvhAQzrHauvXnL33Y7qwtFW5wEXyQbS4ivdYroQ== X-Received: by 2002:a05:6402:1fb:: with SMTP id i27mr13202623edy.379.1599495642083; Mon, 07 Sep 2020 09:20:42 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com Received: by 2002:aa7:c545:: with SMTP id s5ls233984edr.3.gmail; Mon, 07 Sep 2020 09:20:41 -0700 (PDT) X-Received: by 2002:aa7:d40f:: with SMTP id z15mr23003966edq.247.1599495641131; Mon, 07 Sep 2020 09:20:41 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1599495641; cv=none; d=google.com; s=arc-20160816; b=akrhoFFAxQhga/bACoHewseKVOe+N//qyDdMXS7ADunrXcodEWsxIjMbMpR2jedZng EVKkgP2BtUpNJ3mYvNlVlj5KA3McO7yHTrQe8MMlG2KND9KctEnJzmqJCJiWJ27GwdbO zFGiu4iILTD611orMWxvp8jD12zjs4VZyOyI4qDWo6SM6SOw7VDxtKX9FXkGvfJYwLng piVITik8NTYkffwTCFZa1fqAzW2IbsWN60Av5ePRM0QuTmT4M9fN/VQizC++EnioYDeo Wj7E7GLRG/BZHJCxsZM/tJFuvYtKEWZrLdbxBmnPhutEhdp9LxbgYb40CJ8FdTEh51V7 5ZhA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=mime-version:references:in-reply-to:message-id:date:subject:to:from :ironport-sdr:ironport-sdr; bh=3hESc+kJKtwqKrBuuST3QyiQQb+6pEbodWyFniokofA=; b=Ixo5lmBDQYaRVvdXGyFVNkCJRIgGg+a5FpBzonK/HfKsU9s+ndMeLKg9PwSUkME9FK Io43A1jn1idJj+bnI/CNfrdTNZLp2U//45y7TLYajuHCEVi5EswL148H9pXWFI8mLZLq FIr1ubsQ2WmWvaJY5+jZbh3HYXNBxD61bYGe67KHSOfDw2Oq4K/40MbiFN3J/mIxCBFZ QwXxIf0puEqog0W8nJ4jJuSiLkC0YiKAatpiU3V5NjeEgQIWkbSK4aT7WOJDc1n0PuKa 2T+8q4zn8mV5HhOgibq3eSRNHu/e2LSanritwe8yT2sljDwj4w75SvCdiI2giwHiED/0 Wlfg== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of vijaikumar_kanagarajan@mentor.com designates 68.232.137.252 as permitted sender) smtp.mailfrom=Vijaikumar_Kanagarajan@mentor.com Received: from esa4.mentor.iphmx.com (esa4.mentor.iphmx.com. [68.232.137.252]) by gmr-mx.google.com with ESMTPS id w16si466474edq.4.2020.09.07.09.20.40 for (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Mon, 07 Sep 2020 09:20:41 -0700 (PDT) Received-SPF: pass (google.com: domain of vijaikumar_kanagarajan@mentor.com designates 68.232.137.252 as permitted sender) client-ip=68.232.137.252; IronPort-SDR: 9UZDv6AjVkeXLYKpYKp7A2i2So49N8XMf9JDEPILRr9Na+Q/MpPDcDhyhWfWbATFGWHQ0dZAM/ m6+29DJIIPWmu1GBb4HWVJmrJaoaTomRPzySW+ky51NVdK6rwqFA/yz0JKUPXPF74f1pHe2vsO 92yT2yXFEbJuqGWkLuMUG4WWPvxUrenXg9vFHErxA1mDX6xJdxATIHClnO5UuSXFcPq3wU41Xy Gyb646sf1Tzxps2qGg0fH9pjOQGzjtdjqwOVoZ7aEMxlgd+aeXNX//Uf05AHCYpwXL47pnTHTy /GY= X-IronPort-AV: E=Sophos;i="5.76,402,1592899200"; d="scan'208";a="52770194" Received: from orw-gwy-02-in.mentorg.com ([192.94.38.167]) by esa4.mentor.iphmx.com with ESMTP; 07 Sep 2020 08:20:40 -0800 IronPort-SDR: sjsLotMhpUxhKVIs/Na7Fu5g8Xwn8fe0mhhPMONKix8pz+3tXU51+fPalqn8YMIFcc1s007k2Y RXFtFGnzQ1Ksd7esqUlkSSt8yHGww2Xi6yIvVaxw99gciOaf6B4ab76GDl9/VmLoD33Mo73kSi Zr9gUcT5cic7DGWHOR8tWsMe0TXphSjNQtd2V1oKiYhLj1gZ+WvpsU32AfITkKqfIqs+EjYT8S gKxy2X9PrizCtindW3CZoFb5Cl7ImP98SO+QcPsGa58CSX4y0lyxsvbrwfALHe+t1JE6dRS/7m 1OY= From: Vijai Kumar K To: , Subject: [PATCH v3 5/9] lib/oe/path: try hardlinking instead of guessing when it might fail Date: Mon, 7 Sep 2020 21:50:14 +0530 Message-ID: <20200907162018.16636-2-Vijaikumar_Kanagarajan@mentor.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200907162018.16636-1-Vijaikumar_Kanagarajan@mentor.com> References: <20200907161756.16372-4-Vijaikumar_Kanagarajan@mentor.com> <20200907162018.16636-1-Vijaikumar_Kanagarajan@mentor.com> MIME-Version: 1.0 X-ClientProxiedBy: SVR-ORW-MBX-09.mgc.mentorg.com (147.34.90.209) To svr-orw-mbx-01.mgc.mentorg.com (147.34.90.201) X-Original-Sender: vijaikumar_kanagarajan@mentor.com X-Original-Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of vijaikumar_kanagarajan@mentor.com designates 68.232.137.252 as permitted sender) smtp.mailfrom=Vijaikumar_Kanagarajan@mentor.com Precedence: list Mailing-list: list isar-users@googlegroups.com; contact isar-users+owners@googlegroups.com List-ID: X-Spam-Checked-In-Group: isar-users@googlegroups.com X-Google-Group-Id: 914930254986 List-Post: , List-Help: , List-Archive: , List-Unsubscribe: , X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: =?utf-8?q?1677192742527937168?= X-GMAIL-MSGID: =?utf-8?q?1677192742527937168?= From: Henning Schild The comparison of the stat st_dev is not enough to judge whether hardlinking will work. One example would be where you try and hardlink across two bind-mounts of a directory. The st_dev will be the same and the operation will still fail. Instead of implementing a check to try and figure out hardlink support just try hardlinking and fall back to a copy when running into an exception. Signed-off-by: Henning Schild Signed-off-by: Richard Purdie [Vijai: Imported from OE core f5571bda8327f927feb23b167ab4594b7d0c95bc] Signed-off-by: Vijai Kumar K --- meta/lib/oe/path.py | 24 ++++++++++++++++++------ 1 file changed, 18 insertions(+), 6 deletions(-) diff --git a/meta/lib/oe/path.py b/meta/lib/oe/path.py index c6bb604..3506e2c 100644 --- a/meta/lib/oe/path.py +++ b/meta/lib/oe/path.py @@ -99,7 +99,22 @@ def copyhardlinktree(src, dst): if os.path.isdir(src) and not len(os.listdir(src)): return - if (os.stat(src).st_dev == os.stat(dst).st_dev): + canhard = False + testfile = None + for root, dirs, files in os.walk(src): + if len(files): + testfile = os.path.join(root, files[0]) + break + + if testfile is not None: + try: + os.link(testfile, os.path.join(dst, 'testfile')) + os.unlink(os.path.join(dst, 'testfile')) + canhard = True + except Exception as e: + bb.debug(2, "Hardlink test failed with " + str(e)) + + if (canhard): # Need to copy directories only with tar first since cp will error if two # writers try and create a directory at the same time cmd = "cd %s; find . -type d -print | tar --xattrs --xattrs-include='*' -cf - -S -C %s -p --no-recursion --files-from - | tar --xattrs --xattrs-include='*' -xhf - -C %s" % (src, src, dst) @@ -121,12 +136,9 @@ def copyhardlinktree(src, dst): def copyhardlink(src, dst): """Make a hard link when possible, otherwise copy.""" - # We need to stat the destination directory as the destination file probably - # doesn't exist yet. - dstdir = os.path.dirname(dst) - if os.stat(src).st_dev == os.stat(dstdir).st_dev: + try: os.link(src, dst) - else: + except OSError: shutil.copy(src, dst) def remove(path, recurse=True):