From patchwork Fri Jul 12 06:56:44 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anton Mikanovich X-Patchwork-Id: 195 Return-Path: Received: from shymkent.ilbers.de ([unix socket]) by shymkent (Cyrus 2.5.10-Debian-2.5.10-3+deb9u2) with LMTPA; Fri, 12 Jul 2024 08:57:04 +0200 X-Sieve: CMU Sieve 2.4 Received: from mail-lf1-f63.google.com (mail-lf1-f63.google.com [209.85.167.63]) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPS id 46C6v3fP021197 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Fri, 12 Jul 2024 08:57:03 +0200 Received: by mail-lf1-f63.google.com with SMTP id 2adb3069b0e04-52e969d34a9sf1833982e87.2 for ; Thu, 11 Jul 2024 23:57:03 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1720767418; cv=pass; d=google.com; s=arc-20160816; b=WY6xep+Glj9+baOGZP5Tvg6FaXauFPFy3prEcL0xW+PXlwOiOlnr/qxZguf6vFaavH d9loQBC+06YTWpzQKDnNZvoIPncJvaKeSDS7lyN92R1GlalEmSFXki7cNEswSatznuYX cZWvbAh+dJ018jhB/7Odh3/T/HVF20puVyOJ6ZfN4UXnLeQvZqxQfCbb+lDSvgFdLWus RHY7g2iYtFT5ARyA63xwt6m6tXPgoKKvx1RNbkLOaqhYAo4XWevq/gCMKduNhon6NE1G 21/yinFu5LvHoDaQliBvrQKZlyWVYNNubqqSWIsT4/O+e2bxxdWDU7B/iI6O3pYzrIUw IgEQ== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:mime-version:message-id:date :subject:cc:to:from:sender:dkim-signature; bh=ZKn4wS1oimPwigFOAN1KJQqz5UqAmXhnfbATD1ZwjDc=; fh=le1RV5EWiBsXfjI/X1jKmz/3DJkNwpfSs3dOlGR3/II=; b=Qa/YOyyxaA7IkdXOBBzEkCnOqGfXzkIng9g6KIQvjufIVQybhD2wfDSTH5mzjjly0N thb8HNiz/WPwD8PqkGmdqiJgHDEWCXIAna//h+TYrGNlrQRuhyYct4eJdO4y3x/mlmyK r6JhugNOVwMON5XFfiVP08oXo4u4pwR2zXEfFcNFhC1sehQgfszpVvTE1P9bCm2ixmKU VdE3cli6gLcI6naujehCMeV+sjYnckYFiFgqqJokLXgrwsLcU9bMJCFX/GI7dgbyiwRK /bpAZG/xFE4RgLFKvdOdHROa0GOLyBjb6aXWN3aNprha7Mseuadpxk9DQliCtWDaOj4v kmFQ==; darn=isar-build.org ARC-Authentication-Results: i=2; gmr-mx.google.com; spf=pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=amikan@ilbers.de DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlegroups.com; s=20230601; t=1720767418; x=1721372218; darn=isar-build.org; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:x-original-authentication-results :x-original-sender:mime-version:message-id:date:subject:cc:to:from :sender:from:to:cc:subject:date:message-id:reply-to; bh=ZKn4wS1oimPwigFOAN1KJQqz5UqAmXhnfbATD1ZwjDc=; b=Ctlr0yC6z39M362rSLi8hIE6w77lxMIDtHgoqMB1gMcPAYcxVXSS61JPEM6C+a3vlW U9niQvL/qpOYSIiGAbM4UMUNgHLQS6GDxePlTkI3Gp6BqU5RRkC7g2sfJD1oZAM+4h9X DUvKUKdykQ6NUsp/pVGAtQrcNhJzW8GRuokRiFY5s1VZLOlT/5OBLU5JMT3fooo/b5hu U4C0yCKqAckX2bGLF62AolA/3Y5/5iVqY3zvWFXiSt7EOFHHfbAvwBwLiEHhjvwvVSvV c6RuVVPqdg953R224y0NW4NTJpowHIURWqaw4GbNLHOxck6EDYXDGji6U+makUBAIq20 OUgw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1720767418; x=1721372218; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :x-spam-checked-in-group:list-id:mailing-list:precedence :x-original-authentication-results:x-original-sender:mime-version :message-id:date:subject:cc:to:from:x-beenthere:x-gm-message-state :sender:from:to:cc:subject:date:message-id:reply-to; bh=ZKn4wS1oimPwigFOAN1KJQqz5UqAmXhnfbATD1ZwjDc=; b=QE/LsYo0TYCT3TrljUd7CSuhdD68pvQrtQ4/TS7MdHlQglJQ7mQzbkJxZtkIOYDyL1 LMANipp3c3UaSxa/lAN/Y/y4ld5i3DnHnHsLZK8Aqwt72iN8Qkt31W6/QNoCBbOGzRbz IoUQNObPvXg9NjWeBEqo6+Rp+9yM6sRky0j/aB0sJO51h3i9vqWWLEqdNYD3t2iuwnty iUhg2vaPr+RhYhoOK7Rr8+im2yXxmx6AstwLJARoHb7t7wT3uJn1DA3Z13CVQPwmnFNC xdgX8Rwr+KYHqWz0ggoqEkQ8u0DEMjXzb26XZvFiNOTnW8lVpkzkpr+k/MWIrWk8kVKT c2Pw== Sender: isar-users@googlegroups.com X-Forwarded-Encrypted: i=2; AJvYcCU3AB0sT/gPy9QEaAW0WP1PsR2QvxW9B1l5nAHZFQGhmHF84Jy/YMP7Kq+1RjWEMTTtwH5IuUeMIoDofhFm0MXEVmAdT1I= X-Gm-Message-State: AOJu0Yw5Howb08zIveAYHde/aDyotJWSEnBFQA/N1m1WSSqPcyZp2Sr9 zYrVbDM1ESw+wlOAWpLkLLvSBfLv8O9b43nkqK2+BcEO3cfFK9zb X-Google-Smtp-Source: AGHT+IHQ8TWTDPsJyCfyFaph5KTwYy86H2xqxKJlViBV3lCZtr7jmrj0s2nMEIhAeS9SvddDQUWWiA== X-Received: by 2002:a05:6512:239c:b0:52c:8b69:e039 with SMTP id 2adb3069b0e04-52eb9991e44mr10393305e87.4.1720767416812; Thu, 11 Jul 2024 23:56:56 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a19:ac02:0:b0:52e:97da:24d7 with SMTP id 2adb3069b0e04-52ec4504d84ls907840e87.1.-pod-prod-05-eu; Thu, 11 Jul 2024 23:56:55 -0700 (PDT) X-Received: by 2002:a05:6512:239c:b0:52c:8b69:e039 with SMTP id 2adb3069b0e04-52eb9991e44mr10393248e87.4.1720767414732; Thu, 11 Jul 2024 23:56:54 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1720767414; cv=none; d=google.com; s=arc-20160816; b=yn8AiUZXT/IK/SuKJhjwOzuc4QD4s1GR4VEpvc9zHxGlGer2/tD0xK+x7x4efPs5xC X+1lB0vcng5bkLNu//xgQVR/zQ25Qnty6o7oN12UIPjwLBmUdoRfjFahBqbwQZKlD0q4 Nl2Cjs1VEBjC/Chkdklkytjpi1aWNo4Hqgor95r+iNbsX7w1FGzVJgxFRe6t4qO1bfLI KAT6Rzk1yGXy9+OJNBiaFx5bMK2GNYN5jlcHS2Hg9Ln3DAvEsTa+Do8/L2qLbxd8X7yz oWcE4t2hUIXqbuHs/xj4IKnNDqd7aIdSyygM1zCdJnjD0WnxGt9pAtR54xyyT4h7SFtT OgGA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from; bh=OI4Oivmifq5HVS6uUSjcyZK2LepSgMvG1yIAYsvbZMk=; fh=/h9QQkzJ8EboVkWg45aWwpaUro6WMavIVd2OhN45RtE=; b=UgmPv3rwjdiI6ovI+sL7o+UerIZ8+7vyOjjlFRCDG7TFYBAtAZtjsHvCZPy4CbhFf4 MlPWnVj3RQL8ea5AtNBBdFRMTa4GXWYMlHaGDf/rSvHULMmIMDEzs8EQOvn6pkFHPVfT y46BauindElX94Dv9drG/3z443D5yKshJHNHLMKqRR5mGfyQZaa+nL28GNQ+gq9moHy+ nB8Bq0pjmgs8iRTTPc5vWBsMCCsxC1s6/Hpy2qxL1/fWP3TAT+MyJmh1n9HhdtQ+ngQ/ mD1WGx2UWVM4M0Ks5GvsCZvtHpKZioGFxO4UwhOwsVA5x3zGMFMLsR7kUdROWcPiGFkv T5sA==; dara=google.com ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=amikan@ilbers.de Received: from shymkent.ilbers.de (shymkent.ilbers.de. [85.214.156.166]) by gmr-mx.google.com with ESMTPS id a640c23a62f3a-a780a7f7fbesi15820166b.2.2024.07.11.23.56.54 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Thu, 11 Jul 2024 23:56:54 -0700 (PDT) Received-SPF: pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) client-ip=85.214.156.166; Received: from user-B660.promwad.corp ([159.148.83.114]) (authenticated bits=0) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPSA id 46C6urcK021176 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 12 Jul 2024 08:56:53 +0200 From: Anton Mikanovich To: isar-users@googlegroups.com Cc: Anton Mikanovich Subject: [PATCH 0/2] Protect isar-apt usages in shell Date: Fri, 12 Jul 2024 09:56:44 +0300 Message-Id: <20240712065646.2156320-1-amikan@ilbers.de> X-Mailer: git-send-email 2.34.1 MIME-Version: 1.0 X-Spam-Status: No, score=-4.6 required=5.0 tests=DKIMWL_WL_MED,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_EF,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,RCVD_IN_MSPIKE_H2,RCVD_IN_RP_CERTIFIED, RCVD_IN_RP_RNBL,RCVD_IN_RP_SAFE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on shymkent.ilbers.de X-Original-Sender: amikan@ilbers.de X-Original-Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=amikan@ilbers.de Precedence: list Mailing-list: list isar-users@googlegroups.com; contact isar-users+owners@googlegroups.com List-ID: X-Spam-Checked-In-Group: isar-users@googlegroups.com X-Google-Group-Id: 914930254986 List-Post: , List-Help: , List-Archive: , List-Unsubscribe: , X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= Read operations on isar-apt should be covered by shared lock. This was not implemented for deb_dl_dir_export and imager_run. At least following CI error was caused by isar-apt race: DEBUG: Executing shell function rootfs_export_package_cache find: 'isar-apt/debian-bullseye-arm64/apt/debian-bullseye/pool/main/e/enable-fsck': No such file or directory Find call inside deb_dl_dir_export was trying to look into enable-fsck directory while is was deleted by deb_clean function of do_deploy_deb task of enable-fsck package. The same situation can potentially happen on imager_run call, causing races during image preparations. To protect isar-apt usages inside big shell functions introduce lockrun.py helper for using with isar-apt lock. More clear implementation will require tasks rewrite in python, which is not an option for the big downstream projects. Anton Mikanovich (2): deb-dl-dir: Avoid multiple find run on export meta: Cover bash isar-apt usages by isar.lock meta/classes/deb-dl-dir.bbclass | 9 +++-- meta/classes/image-tools-extension.bbclass | 2 + scripts/lockrun.py | 44 ++++++++++++++++++++++ 3 files changed, 52 insertions(+), 3 deletions(-) create mode 100755 scripts/lockrun.py