From patchwork Mon Jul 8 06:17:30 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anton Mikanovich X-Patchwork-Id: 192 Return-Path: Received: from shymkent.ilbers.de ([unix socket]) by shymkent (Cyrus 2.5.10-Debian-2.5.10-3+deb9u2) with LMTPA; Mon, 08 Jul 2024 08:17:58 +0200 X-Sieve: CMU Sieve 2.4 Received: from mail-wm1-f60.google.com (mail-wm1-f60.google.com [209.85.128.60]) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPS id 4686HvNL030717 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Mon, 8 Jul 2024 08:17:58 +0200 Received: by mail-wm1-f60.google.com with SMTP id 5b1f17b1804b1-42668699453sf7066135e9.3 for ; Sun, 07 Jul 2024 23:17:58 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1720419472; cv=pass; d=google.com; s=arc-20160816; b=sDcxe9iWz+NJ47VCgBZAZhOVBWV3jXp+3X4CGsL8kiJeHYKDCls2IDQ6dd85FgcSwb AcpWT5SadQoiTm9iHLkHCIgUfT5m18sNRBuH78LY/SiGBd5N9Kdn67LaoIM3Byxf1WyI Ofh6/h0B7GfZcVOymeKr+qszMq2++QYCSZNr5UMZiB/5RP+j/gGRrit6gZ9sqCiOcv4H bnJJzk/jWStaXBubKH5P6R3PNMnYP1V6BnOi/QinYsoaLa5Wof+TzbCorglt/rcGZ7+O +ipq7BfkuL0lLWfJZaugoYebrTsLRZ2uepgZ8ppG+jlTAqXxlTr0QDDoy5jTTyX7kMEv dmGg== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:mime-version:message-id:date :subject:cc:to:from:sender:dkim-signature; bh=Szm+TZ2L5pRtvvE7jp6qoiQTy3jHtQXqbHIWjIAKTmk=; fh=8vtGeVMb4ZuNZDCmFhKrdvZy0dVpHH/kZz7Adsn0iWg=; b=zwtdM17yhGSC9Q+UDXIfeR8Y5eAs8ZdLGjANtwoOJkVLq69WVxryXpzAi8pASVaw2R nvjCbZzpxTihZLLyNUtz9jTlE06shCFwmvWtC2a0Qyxx644Ol2JOWYuiilf5DhSen6rx R3YiWcxrujK6xEZ6kJ/mYyIe3ga7kCKBKkZFBKwO1fvOBk9UFzokVDmEQdq4OEoA+VR7 u6FE7QToWY2PY59cpAWR0IWeIJ3BiLT2GlpAJu5gK7sho8ooCk/TQ8vYty6vm7KIUyAA oLTtiH2HOGU80AxUzosEmAKszNlx+3NTvcAkmRmjtyTYKciA7kvRmV89PQLPMOTUMvms 2M3g==; darn=isar-build.org ARC-Authentication-Results: i=2; gmr-mx.google.com; spf=pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=amikan@ilbers.de DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlegroups.com; s=20230601; t=1720419472; x=1721024272; darn=isar-build.org; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:x-original-authentication-results :x-original-sender:mime-version:message-id:date:subject:cc:to:from :sender:from:to:cc:subject:date:message-id:reply-to; bh=Szm+TZ2L5pRtvvE7jp6qoiQTy3jHtQXqbHIWjIAKTmk=; b=fqutI0LcKDPkONEk7veyIWgPnZ6dOgzF0ubguLMBZWGDghjE+2oU7pNdgbyFaZ5utV BX8rODDhBpPp2DQH0Q9Xlt756L67I9IeHl4M+TgOn9k5k5Z4n7sHIuGs65Unjtfp8jPr F+G9rqAGW9qS8KpO2qwSfH2VamZz2PqqoA6P+3+ueJCr/TvTuZ5tK5h9xtDSrGIMIMLp TlcR5iVsbKkgI0aCz5cah7sd/t9EVKRLCFrGU9WJSzZpiCaTC+lqvrMfScVgGyjtxf+g 8wMBVGFfQTEswJOuH/SYUBgb+KUqSksCFHEbmJ3jumfkLzGp8ah+pSHgD8vC0ggNQKmE gGdw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1720419472; x=1721024272; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :x-spam-checked-in-group:list-id:mailing-list:precedence :x-original-authentication-results:x-original-sender:mime-version :message-id:date:subject:cc:to:from:x-beenthere:x-gm-message-state :sender:from:to:cc:subject:date:message-id:reply-to; bh=Szm+TZ2L5pRtvvE7jp6qoiQTy3jHtQXqbHIWjIAKTmk=; b=aJWLGaCq7bXpK/gjy4WZj3cX38KEKdQFXf18qJgygSGPFT9UohxYcjOLkvbHwcuA60 dNTV6sdgdrdhgLmalQuxgSKhIZ+225fujNjB7J+CTqojPd8aehEp5nzyrl4j4oHROlG3 BRXlGOiRWj7IHGCtQf69YOZkTx9cNMhTho2J0guvcttZbRiD/ja1apW0NklpqIscVy8y NtIASkJYjbl17WQls5ZqExu4nyh/sNBL2tcZBFQrvY01Lt5IfnmXUNdOHcNStF74Dh0Z 9Ig8SBsPVQBMyiRujsU0gMMQNRXOUPfMxYsV+5MZ+WDbNfwnBtJyxGDGuDlDIyi13CQK 0MKQ== Sender: isar-users@googlegroups.com X-Forwarded-Encrypted: i=2; AJvYcCVrCQUq2qqf13xU5Ns17A8uYuoUjVZCZ0lNppwAcbFCmr8hYAIt94kzKDWK1CvIC7M23r9W0TeMkABw6pUf07WjrTjZPr4= X-Gm-Message-State: AOJu0Yx9K3tTqWz5OBXpks9Q3nv0wGGza3wP4Lxpgn1XuBwV4iAE5aAb wjzefnseUe6yhYebwg2fLKvwpKgjuI1iR7nNC045JfhosorlLZFH X-Google-Smtp-Source: AGHT+IG3rW/T8Pdkq3LxfPawUAKH8QSzrpKU7iyPAgP41xi72NLuuA7R62AW9vJXMlUPdS8L9lWaTQ== X-Received: by 2002:a05:600c:1c8e:b0:426:6b8a:84f7 with SMTP id 5b1f17b1804b1-4266b8a8807mr6935385e9.20.1720419471839; Sun, 07 Jul 2024 23:17:51 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a05:600c:4508:b0:426:6bb9:87ac with SMTP id 5b1f17b1804b1-4266bb98ae5ls1702485e9.2.-pod-prod-05-eu; Sun, 07 Jul 2024 23:17:50 -0700 (PDT) X-Received: by 2002:a05:600c:920:b0:426:5cd6:8307 with SMTP id 5b1f17b1804b1-4265cd683bbmr55533045e9.31.1720419469596; Sun, 07 Jul 2024 23:17:49 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1720419469; cv=none; d=google.com; s=arc-20160816; b=JOO+rVTvHLrdQyF3TSDHHPF9myGalRjNI6lC1z/SGkGCpje8UHLVnJQa99sQMxAvnW raUFQazTDO4kSFWyLthUmalF03zbP+fwHCKqXWFwTAWAzXcVhJxGyddK9ZRQqsRmdkIL bDAGUKFc6kmrw7GoBCAgI4kzHPnpK0IFD+IhqAIKAvNB5R0iuCA24CB/vtiz0yLJ1/49 3YMaEChOtFNuy3xZV/8Bd7RKrSJtR/rA46mLBL1xoXgN/789M8dwjqvtEKK5SD/bFfs5 Gc06Kz+frOjaseMITCKd14HEV+XIEubUeyGTTG1XKUEThluofpHkUfD547aFcGjf5wkL jOcQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from; bh=188WgJGDu4vYnOo/SvI6rnVK+ZbEN+77gecrLW2UjJU=; fh=/h9QQkzJ8EboVkWg45aWwpaUro6WMavIVd2OhN45RtE=; b=j+FECTRcF0wkzoPyBRrb/Z+J8VZpRDIc1Fu6NZohoyleyFK/kRaxoTBqoot8VmuKgR boIrDd6uKygagR0PwBelBXF5nKtDNto6sFEsvo03C9jCxT+tjX5u205Aqn9HVkOtK0TH 63VLG7u3eiZOlYJHi1avvTj7dJ+FHCDyhenetpWH1b0OjKyspkkiBCi84bC+mAEY7q0C Z81kInWX5WBrdJY2AmSXEvFVlVdWxGQHmz2KD91L/RYKlzUSpwmtwEi68eoYk1xrGw+j 3oXmbxDgq18BmTEdsBKW+GWDczgRjVGvF3fON/6j2cesIRZd/DyYdEtaX6c+9KWichFO rTlA==; dara=google.com ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=amikan@ilbers.de Received: from shymkent.ilbers.de (shymkent.ilbers.de. [85.214.156.166]) by gmr-mx.google.com with ESMTPS id 5b1f17b1804b1-42662c6a252si1532685e9.0.2024.07.07.23.17.49 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Sun, 07 Jul 2024 23:17:49 -0700 (PDT) Received-SPF: pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) client-ip=85.214.156.166; Received: from user-B660.promwad.corp ([159.148.83.114]) (authenticated bits=0) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPSA id 4686HmUq030688 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 8 Jul 2024 08:17:48 +0200 From: Anton Mikanovich To: isar-users@googlegroups.com Cc: Anton Mikanovich Subject: [PATCH v10 0/8] Migrate to mmdebstrap Date: Mon, 8 Jul 2024 09:17:30 +0300 Message-Id: <20240708061738.311938-1-amikan@ilbers.de> X-Mailer: git-send-email 2.34.1 MIME-Version: 1.0 X-Spam-Status: No, score=-4.6 required=5.0 tests=DKIMWL_WL_MED,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_EF,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,RCVD_IN_MSPIKE_H2,RCVD_IN_RP_CERTIFIED, RCVD_IN_RP_RNBL,RCVD_IN_RP_SAFE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on shymkent.ilbers.de X-Original-Sender: amikan@ilbers.de X-Original-Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=amikan@ilbers.de Precedence: list Mailing-list: list isar-users@googlegroups.com; contact isar-users+owners@googlegroups.com List-ID: X-Spam-Checked-In-Group: isar-users@googlegroups.com X-Google-Group-Id: 914930254986 List-Post: , List-Help: , List-Archive: , List-Unsubscribe: , X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= Switch Isar from debootstrap to mmdebstrap. It makes isar-bootstrap target ~10..30% faster on both cross and native rootfs types. Some measurements on qemuarm64-bookworm (required debs are predownloaded and used during deb-dl-import): - 20s vs 28s for "isar-bootstrap-host" - 1m32s vs 1m48s for "isar-bootstrap-target Also mmdebstrap support using SOURCE_DATE_EPOCH for images reproducibly out of the box. But the main advantage is an ability to remove sudo for debootstrapping later on (this needs "uidmap" package installed). This change breaks Debian Buster support as host distro, but previous isar-bootstrap (with Buster support) still can be selected by setting in local.conf: PREFERRED_PROVIDER_bootstrap-host ?= "isar-bootstrap-host" PREFERRED_PROVIDER_bootstrap-target ?= "isar-bootstrap-target" Changes since v9: - Rebase on next. - Separate common code into bbclass. - Skip deb cache on rootfs packing. Changes since v8: - Rebase on next. - Keep isar-bootstrap recipes. - Implement bootstrap provider selection. Changes since v7: - Rebase on next. Changes since v6: - Reorder patches. Changes since v5: - Improve keyring handling. - Fix mmdebstrap hooks. Changes since v4: - Rebase on next. - Migrate from apt-key to gpg. - Fix Ubuntu and RPi targets. - Cleanup debootstrap mentions. - Fix commit messages. Changes since v3: - Rebase on latest next. - Remove temporary patch 7 since kas:4.0 already includes mmdebstrap. Changes since v2: - Rebase on next. - Fix commit messages. - Update host requirements. Changes since v1: - Move mmdebstrap tmpdir to workdir. - Fix kas building. - Improve mounts cleanup. Anton Mikanovich (8): isar-bootstrap: Move common parts to bbclass meta: Add mmdebstrap recipe meta: Allow selecting bootstrap providers testsuite: Allow variable bootstrap providers mmdebstrap: Fix missing dpkg available mmdebstrap: Move preparations to hooks isar-bootstrap: Use tar output instead of directory user_manual.md: Update boostrap related documentation RECIPE-API-CHANGELOG.md | 13 + doc/user_manual.md | 15 +- meta-isar/conf/local.conf.sample | 4 + meta/classes/bootstrap.bbclass | 226 ++++++++++++++++ meta/classes/rootfs.bbclass | 13 +- meta/conf/bitbake.conf | 3 + .../isar-bootstrap/isar-bootstrap-host.bb | 2 + .../isar-bootstrap/isar-bootstrap-target.bb | 2 + .../isar-bootstrap/isar-bootstrap.inc | 242 +----------------- .../isar-mmdebstrap/isar-mmdebstrap-host.bb | 19 ++ .../isar-mmdebstrap/isar-mmdebstrap-target.bb | 14 + .../isar-mmdebstrap/isar-mmdebstrap.inc | 215 ++++++++++++++++ testsuite/cibase.py | 9 +- 13 files changed, 538 insertions(+), 239 deletions(-) create mode 100644 meta/classes/bootstrap.bbclass create mode 100644 meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap-host.bb create mode 100644 meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap-target.bb create mode 100644 meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc