From patchwork Wed Nov 6 08:21:09 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anton Mikanovich X-Patchwork-Id: 222 Return-Path: Received: from shymkent.ilbers.de ([unix socket]) by shymkent (Cyrus 2.5.10-Debian-2.5.10-3+deb9u2) with LMTPA; Wed, 06 Nov 2024 09:21:35 +0100 X-Sieve: CMU Sieve 2.4 Received: from mail-wm1-f64.google.com (mail-wm1-f64.google.com [209.85.128.64]) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPS id 4A68LYc9016992 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Wed, 6 Nov 2024 09:21:34 +0100 Received: by mail-wm1-f64.google.com with SMTP id 5b1f17b1804b1-43159603c92sf43220475e9.2 for ; Wed, 06 Nov 2024 00:21:34 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1730881289; cv=pass; d=google.com; s=arc-20240605; b=Utohxjxp6BSX8IILaAGm6B7QWovXI1yW/o1kDTopcAKTTXcklmeuvhY/P8RgDyD17d V/p5HF0CEwoK96GKD3sMQog/v4UsLqcggtxOXEeAoTszUJ2NQY6z+v3eJ+f/khu/4e6l 0GugvuwSQMXhc3S8DALwhiTNxvB3Pw2ZPg1l5lDIE399ZTLd23GWDmM9XEU97FAvyb12 yZ8VjR6lGZ0FY5rTkRrFBea0OWrhRApqLzDeTfgJ0GqkYE2dkE2UT3bxgr8fOWYem2Qy ZCdM1Et5h/g6y3lTAZ/I9O6RS9bW45lB+F3TkrZFeud/241BVe+lqywFBmOaAbqH8Q9/ qVXg== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20240605; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:mime-version:message-id:date :subject:cc:to:from:sender:dkim-signature; bh=jc9+MNvjb4StupLtlTzN3aEgmEMsaEzwsr/zzs+udBk=; fh=ybIdCP+nmRNbgLY05KveGgFN5a4g3r+AS56A6q8DVC0=; b=ItrrQJsViLaWEEbktYUWqu8MwNWfrZp7VbWTBCBaCBQyxA63D67XBg3OmkH0JF0MQ5 uCKTGcAUjPMAMfbdVmXvLV/30+y3eURCR7aJ4S8AJB1Nk/30viwm7LjtqRaI7C1bWxPA vAoNwl+5YhpLGbJ0phBytA03lXELezFernZUvO7Gt0jrXRpOkao9eHro/2eLGCk6Yntp I0ulvalsWyRHqu5UAqEDPRgUtiXoKvcmMw4yw7lfT6ojvuwNSpULFNnsYxd2nLShoYhy iPgd4acP9LagJDPDCFCJvgazrBQBkSmt9kleL+Xl1Yk/3VZuYJjlWewghfzFc8VGkkr+ VXQQ==; darn=isar-build.org ARC-Authentication-Results: i=2; gmr-mx.google.com; spf=pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=amikan@ilbers.de DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlegroups.com; s=20230601; t=1730881289; x=1731486089; darn=isar-build.org; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:x-original-authentication-results :x-original-sender:mime-version:message-id:date:subject:cc:to:from :sender:from:to:cc:subject:date:message-id:reply-to; bh=jc9+MNvjb4StupLtlTzN3aEgmEMsaEzwsr/zzs+udBk=; b=j9SCUqANb1dZEzKhL0Hs5cIbP1wFXCwXTHM6jMbVR8SwpQzSiAzAM42Dr3pdjpQvmk G0SGkgvcAjDKvcgINwOUS1M78vrYbfnAYrH0jdyYYOuonZowBBRm5KJSw/GEWNmOHQHF 8BT6TJ/c4jHCQS22mAq+oHcIUV2N513q7gUmlNiR0Z+5waA1Eddkr9pXNq/ncgVLMV3K maEx61y4rFYwaBZ5z/xAvBaNBF26Zpg5Ll8SpUSdERiXbCYdhcNJQllrD8CVlVqNfE4w Sv39EDmVraJvPjn2LiWrZd2uW35Hy7779VkAblnuGISzwHJ9cVYXuCGgq2vCQYgRXyPx 6wJA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1730881289; x=1731486089; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :x-spam-checked-in-group:list-id:mailing-list:precedence :x-original-authentication-results:x-original-sender:mime-version :message-id:date:subject:cc:to:from:x-beenthere:x-gm-message-state :sender:from:to:cc:subject:date:message-id:reply-to; bh=jc9+MNvjb4StupLtlTzN3aEgmEMsaEzwsr/zzs+udBk=; b=R9p3W3I4PEpEzhWzmBfmOPDJjM7UMGaLPlyk5C0EaISU4tl1DnoSoOW1gG4w3Zkyp2 BlW/EmVLTIPnphhLX4rccg92C68Lu6/xFHSV6TV4jZugSWaMEDAvuO5UNwag2Yn3vfKu EequnwD62e3AsFtaGpobnV28hV6IM8jGJxCpR1PqZWw1qfltAwZoxK2pGTlk9JqSCFhK OtiCXIzjZHsNMFjHEzvNNubMBvwgxnxFmM1FCFWLZKud9AKH4k4m6jRBFM7MuYkiYbjV ePUNbhV9L2opuaa0nhVvRg6S96S4xEvaQsxr4Oca43afhLekQiK8MaD0/oqV8hKBhZM7 ARMg== Sender: isar-users@googlegroups.com X-Forwarded-Encrypted: i=2; AJvYcCU7sXXBz/0grgackTG02IvsalFjACaNcxO/RKROu08O6p+fZ+EgV9uE5ZTnTPxihkHfl3fiO1w=@isar-build.org X-Gm-Message-State: AOJu0YzzPerie8gH94XAnY25SAO4/4xKqZ1uLFhmJ9HqPQlzY11UsL47 EkqBbmPETVVYso6sQjqBTxBsMZ4C1MFAzv7LmcHuDuBwg5TxzfQn X-Google-Smtp-Source: AGHT+IGkPqseUN7vQLLRwjoBHIJSn1x9VOhHufpoLAITvDYCUyp+TvbRxSZseg+VWvS+Gcyj5w+lTg== X-Received: by 2002:a05:600c:5493:b0:431:5533:8f0c with SMTP id 5b1f17b1804b1-43285649fe6mr147377375e9.29.1730881288367; Wed, 06 Nov 2024 00:21:28 -0800 (PST) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a05:600c:1548:b0:42c:b037:5fba with SMTP id 5b1f17b1804b1-4327b8195a0ls13494795e9.2.-pod-prod-06-eu; Wed, 06 Nov 2024 00:21:26 -0800 (PST) X-Received: by 2002:a05:600c:1909:b0:431:542d:2592 with SMTP id 5b1f17b1804b1-4328327eba3mr145698435e9.27.1730881285753; Wed, 06 Nov 2024 00:21:25 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1730881285; cv=none; d=google.com; s=arc-20240605; b=bJl4BZoVDv+os4HxkeRqi807/U4soRbryGdgNnDCuW70Uraj5G83ehFK1KYtQgOycm yI5E3EvDjtmtgpe6qui2o6VsdlXqWBmw8eoKiBkgjjkTkw0Aj4CiXsKwHXU83i2hNc3w JACAC5Uvt7llO51d54O8d+3Fwcefhipt1d1qFW5twnDOFH2r0iYkAIPtbeG3sPRh2k3I mwAsTLYjv0XL3xDdloIlW1KsZzCVWysmCQtroKbzfTjHNDClVcDi1xXsx6RDzYiAFndm ZAHd470Jp1Qva1aMLO9exKg7fsabY3HiUv9K1YsbZO53+rz4Er8eOmr0GaubRwlwU8bQ da8w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20240605; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from; bh=n+zHe0Y39xNIlrfVoQRZjCj83fmqqywhWpHZ0IfcUpQ=; fh=/h9QQkzJ8EboVkWg45aWwpaUro6WMavIVd2OhN45RtE=; b=IK7xpAo8JYk8XXJ0uXW1tebmhs4S9psyqKZeXj3kiOfijNNwEKKx3J9HBKf0BVpjJ5 H16St+CXjQZ+CBO9mxPVzCRePi+plDDti/rmwvZx9gWxSgQ3DqnWB/7w4CF/rB6lvYJQ zLdgS0HveK3ixsdkV6RhX8MpFKH/TPVopXOml4oU6Xp9BQYnp32/8/wiv6lzSTGVsiIr RqWg2TN7UXTwR8rETrhs/TDu3YvY5Gbc9FNyO0JRBhnSEyRc4pBXXhkVTmhRDX0/I1De PMhb2gp3kdrYXZki61S/eQtrqZEqeNj4JVPQD8cKCJFIMHmAKgYZ6GIXqO3RZZg/ldAU Gf7A==; dara=google.com ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=amikan@ilbers.de Received: from shymkent.ilbers.de (shymkent.ilbers.de. [85.214.156.166]) by gmr-mx.google.com with ESMTPS id ffacd0b85a97d-381c1161a94si270194f8f.4.2024.11.06.00.21.25 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Wed, 06 Nov 2024 00:21:25 -0800 (PST) Received-SPF: pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) client-ip=85.214.156.166; Received: from user-B660.promwad.corp ([159.148.83.114]) (authenticated bits=0) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPSA id 4A68LOjp016956 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 6 Nov 2024 09:21:24 +0100 From: Anton Mikanovich To: isar-users@googlegroups.com Cc: Anton Mikanovich Subject: [PATCH v11 0/8] Migrate to mmdebstrap Date: Wed, 6 Nov 2024 10:21:09 +0200 Message-Id: <20241106082117.1089554-1-amikan@ilbers.de> X-Mailer: git-send-email 2.34.1 MIME-Version: 1.0 X-Spam-Status: No, score=-4.6 required=5.0 tests=DKIMWL_WL_MED,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_EF,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,RCVD_IN_DNSWL_BLOCKED,RCVD_IN_MSPIKE_H2, RCVD_IN_RP_CERTIFIED,RCVD_IN_RP_RNBL,RCVD_IN_RP_SAFE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on shymkent.ilbers.de X-Original-Sender: amikan@ilbers.de X-Original-Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=amikan@ilbers.de Precedence: list Mailing-list: list isar-users@googlegroups.com; contact isar-users+owners@googlegroups.com List-ID: X-Spam-Checked-In-Group: isar-users@googlegroups.com X-Google-Group-Id: 914930254986 List-Post: , List-Help: , List-Archive: , List-Unsubscribe: , X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= Switch Isar from debootstrap to mmdebstrap. It makes isar-bootstrap target ~10..30% faster on both cross and native rootfs types. Some measurements on qemuarm64-bookworm (required debs are predownloaded and used during deb-dl-import): - 20s vs 28s for "isar-bootstrap-host" - 1m32s vs 1m48s for "isar-bootstrap-target Also mmdebstrap support using SOURCE_DATE_EPOCH for images reproducibly out of the box. But the main advantage is an ability to remove sudo for debootstrapping later on (this needs "uidmap" package installed). This patchset was successfully tested with following downstreams: - cip-core - meta-iot2050 - xenomai-images This change breaks Debian Buster support as host distro, but previous isar-bootstrap (with Buster support) still can be selected by setting in local.conf: PREFERRED_PROVIDER_bootstrap-host ?= "isar-bootstrap-host" PREFERRED_PROVIDER_bootstrap-target ?= "isar-bootstrap-target" Changes since v10: - Rebase on next. Changes since v9: - Rebase on next. - Separate common code into bbclass. - Skip deb cache on rootfs packing. Changes since v8: - Rebase on next. - Keep isar-bootstrap recipes. - Implement bootstrap provider selection. Changes since v7: - Rebase on next. Changes since v6: - Reorder patches. Changes since v5: - Improve keyring handling. - Fix mmdebstrap hooks. Changes since v4: - Rebase on next. - Migrate from apt-key to gpg. - Fix Ubuntu and RPi targets. - Cleanup debootstrap mentions. - Fix commit messages. Changes since v3: - Rebase on latest next. - Remove temporary patch 7 since kas:4.0 already includes mmdebstrap. Changes since v2: - Rebase on next. - Fix commit messages. - Update host requirements. Changes since v1: - Move mmdebstrap tmpdir to workdir. - Fix kas building. - Improve mounts cleanup. Anton Mikanovich (8): isar-bootstrap: Move common parts to bbclass meta: Add mmdebstrap recipe meta: Allow selecting bootstrap providers testsuite: Allow variable bootstrap providers mmdebstrap: Fix missing dpkg available mmdebstrap: Move preparations to hooks isar-bootstrap: Use tar output instead of directory user_manual.md: Update boostrap related documentation RECIPE-API-CHANGELOG.md | 13 + doc/user_manual.md | 15 +- meta-isar/conf/local.conf.sample | 4 + meta/classes/bootstrap.bbclass | 226 ++++++++++++++++ meta/classes/rootfs.bbclass | 13 +- meta/conf/bitbake.conf | 3 + .../isar-bootstrap/isar-bootstrap-host.bb | 2 + .../isar-bootstrap/isar-bootstrap-target.bb | 2 + .../isar-bootstrap/isar-bootstrap.inc | 244 ++---------------- .../isar-mmdebstrap/isar-mmdebstrap-host.bb | 19 ++ .../isar-mmdebstrap/isar-mmdebstrap-target.bb | 14 + .../isar-mmdebstrap/isar-mmdebstrap.inc | 216 ++++++++++++++++ testsuite/cibase.py | 9 +- 13 files changed, 540 insertions(+), 240 deletions(-) create mode 100644 meta/classes/bootstrap.bbclass create mode 100644 meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap-host.bb create mode 100644 meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap-target.bb create mode 100644 meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc