From patchwork Wed Oct 19 10:47:45 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Schmidt, Adriaan" X-Patchwork-Id: 2173 Return-Path: Received: from shymkent.ilbers.de ([unix socket]) by shymkent (Cyrus 2.5.10-Debian-2.5.10-3+deb9u2) with LMTPA; Wed, 19 Oct 2022 12:48:30 +0200 X-Sieve: CMU Sieve 2.4 Received: from mail-lf1-f60.google.com (mail-lf1-f60.google.com [209.85.167.60]) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPS id 29JAmTQb024460 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 19 Oct 2022 12:48:29 +0200 Received: by mail-lf1-f60.google.com with SMTP id bp18-20020a056512159200b004a2c88a4e1esf5549372lfb.3; Wed, 19 Oct 2022 03:48:29 -0700 (PDT) ARC-Seal: i=3; a=rsa-sha256; t=1666176503; cv=pass; d=google.com; s=arc-20160816; b=rIXEjZPngV3HF2WQfzvuVrSYe6UjaggC16kvZ7RrnwQRWDLzdR0XZZMpDGsGW5Fmc1 NQ7HhLCahqKUHpKeNvZk9kfZxFqcJmhkMYkiYAhtAcb9921MR97Brwt8JNkwYxWNqWEe yim9O613q2eNfXvyuvO9/W6uNzT9uXu7xdj7hKAxj111Qm6HuZ1obpON2bzRvtzUfVgP XDu6vaFAh3tKNGBTwIK2e71EXcWJYTT3NggWuv8Kwx/lUKAORqBGin8DR9oVDEwBQPgU HjkZadP+YBVppOD2EaE2kPght7bhFwf3i7NnNvO7AHakD/c+YVtzUqPhoGlFOPZ/UBGg ezpw== ARC-Message-Signature: i=3; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:mime-version:message-id:date :subject:cc:to:from:sender:dkim-signature; bh=30ChZtjzqVz3GSWAm5a355HwP/mqsUTfYufGwlTzn70=; b=nSV33rrBMdABLKfp7FSvZTVJ8Zpko6yAbJ+w46P89Fk4o2G5tNDnmZxPKv2CsmIS2O FADbZCR4Q0hW+Q45QDTnee+1nPe+2T2Oi12aCYzEfzedoSY53ra8FPHjNTwElHHgkkM7 D+oTA5MXFe4zVPbXXtbNXbC4w+Q6ibDcZPiZ18qatzTJSZo7zQpxQ5RF3hHVp5iBuGsS xHjMAccl7HY8EfcdkERQhCBVDS9qZfvwPSZqqMHFe/KihbgZ/HFOEqX/5dztaCgL75VK 5UJQoWBHk6fpqKVvhoaY0tBO8OCw2ng/ZfznaTWBak7wkiOJI/KL3JNM7FgjTVNYarDY 2DFQ== ARC-Authentication-Results: i=3; gmr-mx.google.com; dkim=pass header.i=@siemens.com header.s=selector2 header.b=H9JbffUC; arc=pass (i=1 spf=pass spfdomain=siemens.com dmarc=pass fromdomain=siemens.com); spf=pass (google.com: domain of adriaan.schmidt@siemens.com designates 40.107.2.81 as permitted sender) smtp.mailfrom=adriaan.schmidt@siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlegroups.com; s=20210112; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:x-original-authentication-results :x-original-sender:mime-version:message-id:date:subject:cc:to:from :sender:from:to:cc:subject:date:message-id:reply-to; bh=30ChZtjzqVz3GSWAm5a355HwP/mqsUTfYufGwlTzn70=; b=A9r8vxitbgIZbWLMVBR6pUrpcf7fCxRMXal+yHfkurW0EUGIYTnYPKfJjwVlk25x02 rxuWzrqgMy7+JUdegSxkC1taBiOp1NoD+fZcDoyUUZRKO5ecpcOIaWyGwXkf8jP+jiqT lNhc08rzwqqkDvhoBrADu4jl0adeYeeDdzJrTGFjP7bmKopznX+Z2/9OzOKWDGH9yI50 8jhIWTOsy6TUC4HLiar0KRzIeCq8BPtNHvUgj0mHlKWX/1VXgYFM07319l4A22qQjiE7 0OL2AlC4TpXTEUf/mmKcSorqtxQrD6MIQTqJG8JvdfJLnhPexZ2xJxkC17WdtgCh9Wjv Em9Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :x-spam-checked-in-group:list-id:mailing-list:precedence :x-original-authentication-results:x-original-sender:mime-version :message-id:date:subject:cc:to:from:x-gm-message-state:sender:from :to:cc:subject:date:message-id:reply-to; bh=30ChZtjzqVz3GSWAm5a355HwP/mqsUTfYufGwlTzn70=; b=DDLIvMRKLg46cxAOVQEtgZcTViQbSJYkqrh4YvU3ZCw8nx2q3SwhhKwha2588S3elR INbTJ7RnqpfLMenOXYhrFRNKA9iTZJ25ymHfDJDBYftsnfi7dEQ1HiPwO79DLRmUAKKL YDe7yes3LTlcGUevi84hYLVFjsScJ0/2poz2K9GuSjfBDLzatBVUQcYcinsYnLzHmHlK 5J8gqPBitIkme8LDbZnDHPFdaYQT7DoYuzfHFvXDp2GSVERVhRDf0NBRjb5EW1tjFkvl gjU+aORDhNcI8lDTs4SZJiwUcbStlsObMl2VZEIJSfWxjLehfntgWmcQrUUnZGXL84hp KwTw== Sender: isar-users@googlegroups.com X-Gm-Message-State: ACrzQf2VeJ6SumJ7oGG25kaYEAxmpd81dt5Zt6s3KbMJzVczEaVH3g2Y S8GM+RFWh+1r7uNo2p0Kjwc= X-Google-Smtp-Source: AMsMyM7wRJFzLY9Miq1bcoX0v1iHXcnEk/nlHAaReQCGAopYnMGCkLOmhPWza/rZPoxwtnisCvb/Yw== X-Received: by 2002:a2e:2d0a:0:b0:26c:a1c:cdf with SMTP id t10-20020a2e2d0a000000b0026c0a1c0cdfmr2893765ljt.352.1666176503558; Wed, 19 Oct 2022 03:48:23 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a2e:2c18:0:b0:26d:cf5c:d952 with SMTP id s24-20020a2e2c18000000b0026dcf5cd952ls3292609ljs.5.-pod-prod-gmail; Wed, 19 Oct 2022 03:48:22 -0700 (PDT) X-Received: by 2002:a2e:6812:0:b0:26e:624:52d6 with SMTP id c18-20020a2e6812000000b0026e062452d6mr2568343lja.427.1666176502146; Wed, 19 Oct 2022 03:48:22 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1666176502; cv=pass; d=google.com; s=arc-20160816; b=x+bWKZX4QPKdEG6knMTy1rOACNcGqvquwzK6HZtQ6op4z/kUkoAq9x19z4lDxPWVgV a6E5CtaZiB1zee/BQU/xE+BQy+/dZNoTSLTMokYv7CKBYbo4wkCE0eFfZYWWIiDp7tLL +jP+G5x18S4e2mDQ9Xhe+8aMPEdRfQU6KzZ6PhN/AhHUa3BBehh+L94U5Fyp4WAXb1zG jIPN7AGAXJMcS5hjngakluOdKPkBg251B9d1nq/DpNf4JcJysYYV5XAZPIuuSroIvkE4 yiVUs6u9asXQF7m3+9rHJ+OaLKfGV3GHshGj3P7LcgwVi7u02i1Mzvs1WqCVRVVHhE8i mmdw== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:dkim-signature; bh=NvX7cDyKbKQ1uWcQjgTGsEsB+L1LzlWTnyes4oRLY+8=; b=C80sjJfbQN+lv95qI4ZVy2N6X1JLjYdDrNh8sLnMkw6mZyewMzhVkbiBbBL9RbseHw VXdPpPKuddZbdrb9VxZLF+qmmIOCXbdc55RbAPJiDDhSp03qhVES3EwYxEVXRfCr25A/ yJqUZ6J+36ImjQcrZMYDUvo5qWBZADs6mQ8f5Zj+63vJ1t2+h0BDHoPcaDkxZR3XeesA +7sqw/ls1evVyku/j1lZIFJ0LBBDD7TsQkCosdBlJyVxk0O1cj0oVrtNem+d5GLt5RQc 1xNbo6nCLRmHC5Uj+w/bj6TiXuiYBvT6CUz/BB+wyEIVbmSU44Vak3/91QQX3X32RKB8 AN+w== ARC-Authentication-Results: i=2; gmr-mx.google.com; dkim=pass header.i=@siemens.com header.s=selector2 header.b=H9JbffUC; arc=pass (i=1 spf=pass spfdomain=siemens.com dmarc=pass fromdomain=siemens.com); spf=pass (google.com: domain of adriaan.schmidt@siemens.com designates 40.107.2.81 as permitted sender) smtp.mailfrom=adriaan.schmidt@siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com Received: from EUR02-VE1-obe.outbound.protection.outlook.com (mail-eopbgr20081.outbound.protection.outlook.com. [40.107.2.81]) by gmr-mx.google.com with ESMTPS id x12-20020a056512078c00b0048b224551b6si617834lfr.12.2022.10.19.03.48.21 for (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Wed, 19 Oct 2022 03:48:22 -0700 (PDT) Received-SPF: pass (google.com: domain of adriaan.schmidt@siemens.com designates 40.107.2.81 as permitted sender) client-ip=40.107.2.81; ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=De1tgIqEk7iHHvSatEOk65NPB8cGRsqnKnOaUK/V4beopjQrPMjPmBrH3D8uKJL/iCD4paf1vWAj64ieIwokFFRcNOCSpE9j7Ei3JtY7N08AaCV2Gq+KwUvI1DBfqOvMIf3a6IEEyfl1m+PJOcwqmMjA2sklRxYh4n1qeWv03ssdhUingm2wd3B1yanhJpwFuxx56JWdkp67COpMQVjiWcGdD2+IZof9/slP36nl5jJH7SnC73qExcXPaXnV1O6PJyFoMs2agSf0G0L1Wlxx6P4S5h9glLY1vIB8M4TUhDA18MrSp+aBYdfhENsi9LbGSDDc6VtJlQMR78CZVF/5yw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=NvX7cDyKbKQ1uWcQjgTGsEsB+L1LzlWTnyes4oRLY+8=; b=ZvxHuQNzH9vCbQr3Es/41Ki5e9VuLDYT5OW+Y2z34g3EbzKVJOYtHmR7daMg3Rop34KwJqGGFSNKS7OC6Qo4MqxvsQkcB80+2yxy4t6jTPq+J2DAu4bYuAtM6LL9IPnXnVrA8uIHJtDVNrjKs0WPkV4f0dpboTWCmaAWDADGReGwqbcH6pRvm1i+fr9JL83+kow5E4pOwr4Nlrm2hfQNHqHTBgbIJZQ9aH0KVgPFKJ93e7N3mr+GMMCxJwC/ripiIpTOL1q3uuy8jpw6klNR9B9iKvr8snvzpznLfeutTgeUan5t/XeRpisYf0+WKvc/FmK7+2mhD93rbbXGZn+1QQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 194.138.21.75) smtp.rcpttodomain=googlegroups.com smtp.mailfrom=siemens.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=siemens.com; dkim=none (message not signed); arc=none Received: from AS9PR06CA0591.eurprd06.prod.outlook.com (2603:10a6:20b:486::23) by DB9PR10MB6380.EURPRD10.PROD.OUTLOOK.COM (2603:10a6:10:3dc::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5723.30; Wed, 19 Oct 2022 10:48:19 +0000 Received: from VE1EUR01FT067.eop-EUR01.prod.protection.outlook.com (2603:10a6:20b:486:cafe::50) by AS9PR06CA0591.outlook.office365.com (2603:10a6:20b:486::23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5723.34 via Frontend Transport; Wed, 19 Oct 2022 10:48:19 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 194.138.21.75) smtp.mailfrom=siemens.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=siemens.com; Received-SPF: Pass (protection.outlook.com: domain of siemens.com designates 194.138.21.75 as permitted sender) receiver=protection.outlook.com; client-ip=194.138.21.75; helo=hybrid.siemens.com; pr=C Received: from hybrid.siemens.com (194.138.21.75) by VE1EUR01FT067.mail.protection.outlook.com (10.152.3.83) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5746.16 via Frontend Transport; Wed, 19 Oct 2022 10:48:19 +0000 Received: from DEMCHDC8WAA.ad011.siemens.net (139.25.226.104) by DEMCHDC8VRA.ad011.siemens.net (194.138.21.75) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1118.15; Wed, 19 Oct 2022 12:48:19 +0200 Received: from random.ppmd.siemens.net (139.25.68.25) by DEMCHDC8WAA.ad011.siemens.net (139.25.226.104) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1118.12; Wed, 19 Oct 2022 12:48:18 +0200 From: Adriaan Schmidt To: CC: Adriaan Schmidt Subject: [RFC PATCH] Remove isar-apt and the corresponding lock Date: Wed, 19 Oct 2022 12:47:45 +0200 Message-ID: <20221019104745.479166-1-adriaan.schmidt@siemens.com> X-Mailer: git-send-email 2.30.2 MIME-Version: 1.0 X-Originating-IP: [139.25.68.25] X-ClientProxiedBy: DEMCHDC8WBA.ad011.siemens.net (139.25.226.105) To DEMCHDC8WAA.ad011.siemens.net (139.25.226.104) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: VE1EUR01FT067:EE_|DB9PR10MB6380:EE_ X-MS-Office365-Filtering-Correlation-Id: cc435852-ba4c-4c3b-14bd-08dab1bf6f9a X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: I2+Ff74RJ6EYAqzQFmh41iiLeCzTq/3hE9HeM8+DW5nPGF/xp18ZH+5jIrkGpn8jn0pm/n8exjVyqRUQdTagwaSGQPDw5wKbp+ruT54NQ0of/2xrr1/sCdTNdDc7w770lxW4U3agmtlBdLoOmZ/fAkUG16JUGhAgVJdB4RW6VmeKiYOoRhVmuNS+rr9jtiFj8ss5ilKgkM++Vrm/goMVLhT79doq1IWB2t+VjAvm8mQJDb+36m69mF5cBbFbiXAb6DC/Mi8WTAZvypCdXO10BMm+EEXIPMqHS4zjextv0kqRXHzLvgN14nhZ6PolORVIYO6EqMkr4ittlqEIAE05k0zL7HvyMOoU5AIHZm0R11zNGzFQkYW7YaYLvKYUafVZhinYR0n3fqnxMxisUi2GiEbDatJ6WvqRann6wo8T0nWJwi20TpbbusFePbwJ0Oxw7Jv8tC7C/vRXVm89XKy02+xInH1dDDHTNYNsDxLjQdNIdYP6kPlL7wsI19eYCISkH9JB0SuIyrg0x206wefllC3lVnY3LO5s3fWyq/d++yFkb0fYOC/xStsEnjbLRp4kBoKwFB1u5YdVNLKfsrNR2Yb+fTTFN9xiA8EcLqzve2SlXB6O61mw53Z4jIIzbtl0Yyjr5YRvlbYRbd6uZrnNKbEy5a58FQjpoLRYLf/bAqwosMXtD06/JPgN0XSX6x0GsacPAzpYQghxF9YONAUK2/f9TdWt5+xSYDae7UdAT4WSec1E5fyodK00jX2FGO1ccTg1/hA/D2G4VbDHP4TUFbiRg1j4A6TF3uidwZ0GjIDh6BxAJNuhHc/cqMsBBFWgzZT+3MMeqooJciZdo9P7aw== X-Forefront-Antispam-Report: CIP:194.138.21.75;CTRY:DE;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:hybrid.siemens.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230022)(4636009)(376002)(346002)(396003)(39860400002)(136003)(451199015)(46966006)(36840700001)(40470700004)(82310400005)(6916009)(34020700004)(16526019)(1076003)(186003)(956004)(47076005)(83380400001)(356005)(81166007)(2616005)(2906002)(5660300002)(36860700001)(44832011)(30864003)(82740400003)(478600001)(41300700001)(4326008)(86362001)(40480700001)(107886003)(40460700003)(8936002)(26005)(8676002)(316002)(6666004)(70586007)(70206006)(36756003)(336012)(82960400001)(36900700001);DIR:OUT;SFP:1101; X-OriginatorOrg: siemens.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 19 Oct 2022 10:48:19.5116 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: cc435852-ba4c-4c3b-14bd-08dab1bf6f9a X-MS-Exchange-CrossTenant-Id: 38ae3bcd-9579-4fd4-adda-b42e1495d55a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=38ae3bcd-9579-4fd4-adda-b42e1495d55a;Ip=[194.138.21.75];Helo=[hybrid.siemens.com] X-MS-Exchange-CrossTenant-AuthSource: VE1EUR01FT067.eop-EUR01.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB9PR10MB6380 X-Original-Sender: adriaan.schmidt@siemens.com X-Original-Authentication-Results: gmr-mx.google.com; dkim=pass header.i=@siemens.com header.s=selector2 header.b=H9JbffUC; arc=pass (i=1 spf=pass spfdomain=siemens.com dmarc=pass fromdomain=siemens.com); spf=pass (google.com: domain of adriaan.schmidt@siemens.com designates 40.107.2.81 as permitted sender) smtp.mailfrom=adriaan.schmidt@siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com Precedence: list Mailing-list: list isar-users@googlegroups.com; contact isar-users+owners@googlegroups.com List-ID: X-Spam-Checked-In-Group: isar-users@googlegroups.com X-Google-Group-Id: 914930254986 List-Post: , List-Help: , List-Archive: , List-Unsubscribe: , X-Spam-Status: No, score=-0.9 required=5.0 tests=DKIMWL_WL_MED,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_EF,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,RCVD_IN_MSPIKE_H2,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on shymkent.ilbers.de X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= Hi all, the isar-apt file lock has been bothering me for a while now, mainly because it leads to situations where we see many bitbake tasks running in parallel, but unable to make any progress (deploy_deb, install_builddeps, ...). I ran a quick experiment on what would be required to change that, and came up with this approach: - isar-apt/ is no longer a repository maintained with reprepro - each recipe has "its own" directory below isar-apt/ - there are directories isar-apt/${PN}/in and isar-apt/${PN}/out - out/ contains the deb packages produced, put there by deploy_deb - in/ is populated by prepare_build and contains the contents of in/ and out/ of all DEPENDS (need all the in/ contents due to transitive dependencies) - then, again in prepare_build, we copy our in/ directory to WORKDIR, and create an apt repository (actually just a "Packages" file) using dpkg-scanpackages. In addition to removing the lock, this also gives us better control over what goes into a package build, and eliminates effects due to "accidental" packages in isar-apt. I've implemented a very rough PoC, just to give you an idea. Please don't look too closely at the code, there are many things missing (e.g. SDKs, architectures != amd64, ...). But it can build isar-image-base for amd64. If we decide that this would be a worthwile change, I will spend some more time on it. Best, Adriaan Signed-off-by: Adriaan Schmidt --- meta/classes/dpkg-base.bbclass | 46 ++++++++++--------- meta/classes/image-tools-extension.bbclass | 2 +- meta/classes/rootfs.bbclass | 27 ++++++----- meta/conf/bitbake.conf | 3 +- .../isar-bootstrap/isar-bootstrap.inc | 2 +- .../isar-apt/files/distributions.in | 3 -- meta/recipes-devtools/isar-apt/isar-apt.bb | 23 ---------- 7 files changed, 42 insertions(+), 64 deletions(-) delete mode 100644 meta/recipes-devtools/isar-apt/files/distributions.in delete mode 100644 meta/recipes-devtools/isar-apt/isar-apt.bb diff --git a/meta/classes/dpkg-base.bbclass b/meta/classes/dpkg-base.bbclass index 12d18592..a5ed95b1 100644 --- a/meta/classes/dpkg-base.bbclass +++ b/meta/classes/dpkg-base.bbclass @@ -17,7 +17,7 @@ DEPENDS_append_riscv64 = "${@' crossbuild-essential-riscv64' if d.getVar('ISAR_C DEB_BUILD_PROFILES ?= "" DEB_BUILD_OPTIONS ?= "" -ISAR_APT_REPO ?= "deb [trusted=yes] file:///home/builder/${PN}/isar-apt/${DISTRO}-${DISTRO_ARCH}/apt/${DISTRO} ${DEBDISTRONAME} main" +ISAR_APT_REPO ?= "deb [trusted=yes] file:///home/builder/${PN}/isar-apt ${DEBDISTRONAME} main" python do_adjust_git() { import subprocess @@ -179,7 +179,6 @@ PPS ?= "${@get_package_srcdir(d)}" # Empty do_prepare_build() implementation, to be overwritten if needed do_prepare_build() { - true } addtask prepare_build after do_patch do_transform_template before do_dpkg_build @@ -212,15 +211,26 @@ dpkg_undo_mounts() { sudo rmdir ${BUILDROOT} } +do_prepare_build[cleandirs] += "${REPO_ISAR_DIR}/${DISTRO}/${PN}/in ${WORKDIR}/isar-apt" + do_prepare_build_append() { - # Make a local copy of isar-apt repo that is not affected by other parallel builds - mkdir -p ${WORKDIR}/isar-apt/${DISTRO}-${DISTRO_ARCH} - rm -rf ${WORKDIR}/isar-apt/${DISTRO}-${DISTRO_ARCH}/* - cp -Rl ${REPO_ISAR_DIR} ${WORKDIR}/isar-apt/${DISTRO}-${DISTRO_ARCH} + # in deploy/isar-apt: collect all (transitive) deps in .../in/ + for dep in ${DEPENDS}; do + for f in $(find ${REPO_ISAR_DIR}/${DISTRO}/${dep}/in/ ${REPO_ISAR_DIR}/${DISTRO}/${dep}/out/ -maxdepth 1 -name '*.deb' -print); do + cp -l "$f" ${REPO_ISAR_DIR}/${DISTRO}/${PN}/in/ + done + done + # for the build itself: copy everything into workdir + cd ${WORKDIR}/isar-apt + mkdir pool dists + for f in $(find ${REPO_ISAR_DIR}/${DISTRO}/${PN}/in/ -maxdepth 1 -name '*.deb' -print); do + cp -l "$f" pool/ + done + # make it a deb repo + mkdir -p dists/${DEBDISTRONAME}/main/binary-${DISTRO_ARCH}/ + dpkg-scanpackages pool/ > dists/${DEBDISTRONAME}/main/binary-${DISTRO_ARCH}/Packages } -do_prepare_build[lockfiles] += "${REPO_ISAR_DIR}/isar.lock" - # Placeholder for actual dpkg_runbuild() implementation dpkg_runbuild() { die "This should never be called, overwrite it in your derived class" @@ -282,28 +292,20 @@ do_dpkg_build[depends] = "${SCHROOT_DEP}" CLEANFUNCS += "deb_clean" deb_clean() { - DEBS=$( find ${WORKDIR} -maxdepth 1 -name "*.deb" || [ ! -d ${S} ] ) - if [ -n "${DEBS}" ]; then - for d in ${DEBS}; do - repo_del_package "${REPO_ISAR_DIR}"/"${DISTRO}" \ - "${REPO_ISAR_DB_DIR}"/"${DISTRO}" "${DEBDISTRONAME}" "${d}" - done - fi + rm -rf ${REPO_ISAR_DIR}/${DISTRO}/${PN} } -# the clean function modifies isar-apt -do_clean[lockfiles] = "${REPO_ISAR_DIR}/isar.lock" do_deploy_deb() { - deb_clean - repo_add_packages "${REPO_ISAR_DIR}"/"${DISTRO}" \ - "${REPO_ISAR_DB_DIR}"/"${DISTRO}" "${DEBDISTRONAME}" ${WORKDIR}/*.deb + # place our results in .../out/ + for f in $(find ${WORKDIR} -maxdepth 1 -name '*.deb' -print); do + cp -l "$f" ${REPO_ISAR_DIR}/${DISTRO}/${PN}/out/ + done } addtask deploy_deb after do_dpkg_build before do_build do_deploy_deb[deptask] = "do_deploy_deb" -do_deploy_deb[depends] += "isar-apt:do_cache_config" -do_deploy_deb[lockfiles] = "${REPO_ISAR_DIR}/isar.lock" do_deploy_deb[dirs] = "${S}" +do_deploy_deb[cleandirs] = "${REPO_ISAR_DIR}/${DISTRO}/${PN}/out" python do_devshell() { bb.build.exec_func('schroot_create_configs', d) diff --git a/meta/classes/image-tools-extension.bbclass b/meta/classes/image-tools-extension.bbclass index e13d4a3f..1a516dcf 100644 --- a/meta/classes/image-tools-extension.bbclass +++ b/meta/classes/image-tools-extension.bbclass @@ -11,7 +11,7 @@ IMAGER_INSTALL ??= "" IMAGER_BUILD_DEPS ??= "" DEPENDS += "${IMAGER_BUILD_DEPS}" -do_install_imager_deps[depends] = "${BUILDCHROOT_DEP} isar-apt:do_cache_config" +do_install_imager_deps[depends] = "${BUILDCHROOT_DEP}" do_install_imager_deps[deptask] = "do_deploy_deb" do_install_imager_deps[lockfiles] += "${REPO_ISAR_DIR}/isar.lock" do_install_imager_deps() { diff --git a/meta/classes/rootfs.bbclass b/meta/classes/rootfs.bbclass index bbb5ac0a..036925a4 100644 --- a/meta/classes/rootfs.bbclass +++ b/meta/classes/rootfs.bbclass @@ -78,6 +78,19 @@ BOOTSTRAP_SRC_${ROOTFS_ARCH} = "${DEPLOY_DIR_BOOTSTRAP}/${ROOTFS_DISTRO}-${ROOTF rootfs_prepare[weight] = "25" rootfs_prepare(){ sudo cp -Trpfx '${BOOTSTRAP_SRC}/' '${ROOTFSDIR}' + mkdir -p ${REPO_ISAR_DIR}/${DISTRO}/${PN}/pool + echo "deps: ${DEPENDS}" + for dep in ${DEPENDS}; do + # hackity-hack: for some reason the kernel pkg is in DEPDENDS, even if there's no recipe + # could this be an isar-bug!? + [ "${dep}" = "${KERNEL_IMAGE_PKG}" ] && continue + for f in $(find ${REPO_ISAR_DIR}/${DISTRO}/${dep}/in/ ${REPO_ISAR_DIR}/${DISTRO}/${dep}/out/ -maxdepth 1 -name '*.deb' -print); do + cp -l "$f" ${REPO_ISAR_DIR}/${DISTRO}/${PN}/pool + done + done + # make it a deb repo + mkdir -p ${REPO_ISAR_DIR}/${DISTRO}/${PN}/dists/${DEBDISTRONAME}/main/binary-${DISTRO_ARCH}/ + cd ${REPO_ISAR_DIR}/${DISTRO}/${PN} && dpkg-scanpackages pool/ > dists/${DEBDISTRONAME}/main/binary-${DISTRO_ARCH}/Packages } ROOTFS_CONFIGURE_COMMAND += "rootfs_configure_isar_apt" @@ -87,7 +100,7 @@ rootfs_configure_isar_apt() { set -e mkdir -p '${ROOTFSDIR}/etc/apt/sources.list.d' - echo 'deb [trusted=yes] file:///isar-apt ${DEBDISTRONAME} main' > \ + echo 'deb [trusted=yes] file:///isar-apt/${PN} ${DEBDISTRONAME} main' > \ '${ROOTFSDIR}/etc/apt/sources.list.d/isar-apt.list' mkdir -p '${ROOTFSDIR}/etc/apt/preferences.d' @@ -117,7 +130,6 @@ EOSUDO ROOTFS_INSTALL_COMMAND += "rootfs_install_pkgs_update" rootfs_install_pkgs_update[weight] = "5" -rootfs_install_pkgs_update[isar-apt-lock] = "acquire-before" rootfs_install_pkgs_update() { sudo -E chroot '${ROOTFSDIR}' /usr/bin/apt-get update \ -o Dir::Etc::SourceList="sources.list.d/isar-apt.list" \ @@ -142,7 +154,6 @@ rootfs_import_package_cache() { ROOTFS_INSTALL_COMMAND += "rootfs_install_pkgs_download" rootfs_install_pkgs_download[weight] = "600" -rootfs_install_pkgs_download[isar-apt-lock] = "release-after" rootfs_install_pkgs_download() { sudo -E chroot '${ROOTFSDIR}' \ /usr/bin/apt-get ${ROOTFS_APT_ARGS} --download-only ${ROOTFS_PACKAGES} @@ -195,15 +206,8 @@ python do_rootfs_install() { for cmd in cmds: progress_reporter.next_stage() - - if (d.getVarFlag(cmd, 'isar-apt-lock') or "") == "acquire-before": - lock = bb.utils.lockfile(d.getVar("REPO_ISAR_DIR") + "/isar.lock", - shared=True) - bb.build.exec_func(cmd, d) - if (d.getVarFlag(cmd, 'isar-apt-lock') or "") == "release-after": - bb.utils.unlockfile(lock) progress_reporter.finish() } addtask rootfs_install before do_rootfs_postprocess after do_unpack @@ -294,7 +298,7 @@ python do_rootfs() { } addtask rootfs before do_build -do_rootfs_postprocess[depends] = "base-apt:do_cache isar-apt:do_cache_config" +do_rootfs_postprocess[depends] = "base-apt:do_cache" SSTATETASKS += "do_rootfs_install" SSTATECREATEFUNCS += "rootfs_install_sstate_prepare" @@ -311,7 +315,6 @@ rootfs_install_sstate_prepare() { sudo umount ${WORKDIR}/mnt/rootfs sudo chown $(id -u):$(id -g) rootfs.tar } -do_rootfs_install_sstate_prepare[lockfiles] = "${REPO_ISAR_DIR}/isar.lock" rootfs_install_sstate_finalize() { # this runs in SSTATE_INSTDIR diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf index 98412e02..e7874a2b 100644 --- a/meta/conf/bitbake.conf +++ b/meta/conf/bitbake.conf @@ -88,8 +88,7 @@ TARGET_OS = "isar" PACKAGE_ARCH ?= "${DISTRO_ARCH}" # Isar apt repository paths -REPO_ISAR_DIR = "${DEPLOY_DIR}/isar-apt/${DISTRO}-${DISTRO_ARCH}/apt" -REPO_ISAR_DB_DIR = "${DEPLOY_DIR}/isar-apt/${DISTRO}-${DISTRO_ARCH}/db" +REPO_ISAR_DIR = "${DEPLOY_DIR}/isar-apt" THIRD_PARTY_APT_KEYRING = "/etc/apt/trusted.gpg.d/third_party.gpg" # Base apt repository paths diff --git a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc index fdaad61b..4df21d99 100644 --- a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc +++ b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc @@ -276,7 +276,7 @@ do_bootstrap[vardeps] += " \ ${DISTRO_VARS_PREFIX}DISTRO_APT_SOURCES \ " do_bootstrap[dirs] = "${DEPLOY_DIR_BOOTSTRAP}" -do_bootstrap[depends] = "base-apt:do_cache isar-apt:do_cache_config" +do_bootstrap[depends] = "base-apt:do_cache" do_bootstrap() { if [ "${ISAR_ENABLE_COMPAT_ARCH}" = "1" ]; then diff --git a/meta/recipes-devtools/isar-apt/files/distributions.in b/meta/recipes-devtools/isar-apt/files/distributions.in deleted file mode 100644 index 3cf7ea55..00000000 --- a/meta/recipes-devtools/isar-apt/files/distributions.in +++ /dev/null @@ -1,3 +0,0 @@ -Codename: {CODENAME} -Architectures: i386 armhf arm64 amd64 mipsel riscv64 source -Components: main diff --git a/meta/recipes-devtools/isar-apt/isar-apt.bb b/meta/recipes-devtools/isar-apt/isar-apt.bb deleted file mode 100644 index 0458bac7..00000000 --- a/meta/recipes-devtools/isar-apt/isar-apt.bb +++ /dev/null @@ -1,23 +0,0 @@ -# This software is a part of ISAR. -# Copyright (C) 2015-2017 ilbers GmbH -# Copyright (C) 2020 Siemens AG -# -# SPDX-License-Identifier: MIT - -inherit repository - -SRC_URI = "file://distributions.in" - -do_cache_config[stamp-extra-info] = "${DISTRO}" -do_cache_config[lockfiles] = "${REPO_ISAR_DIR}/isar.lock" - -# Generate reprepro config for current distro if it doesn't exist. Once it's -# generated, this task should do nothing. -do_cache_config() { - repo_create "${REPO_ISAR_DIR}"/"${DISTRO}" \ - "${REPO_ISAR_DB_DIR}"/"${DISTRO}" \ - "${DEBDISTRONAME}" \ - "${WORKDIR}/distributions.in" -} - -addtask cache_config after do_unpack before do_build