From patchwork Fri Feb 4 06:41:22 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Henning Schild X-Patchwork-Id: 1421 Return-Path: Received: from shymkent.ilbers.de ([unix socket]) by shymkent (Cyrus 2.5.10-Debian-2.5.10-3) with LMTPA; Fri, 04 Feb 2022 17:41:37 +0100 X-Sieve: CMU Sieve 2.4 Received: from mail-lj1-f188.google.com (mail-lj1-f188.google.com [209.85.208.188]) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8) with ESMTPS id 214Gfa8n001589 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Fri, 4 Feb 2022 17:41:36 +0100 Received: by mail-lj1-f188.google.com with SMTP id c31-20020a2ebf1f000000b0022d87a28911sf1286725ljr.1 for ; Fri, 04 Feb 2022 08:41:36 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1643992891; cv=pass; d=google.com; s=arc-20160816; b=SGGSACZ6/HoqH08oS0pYTWUAbqky1KSJ6Et1lJVxCdbAY8b3Sm2OvrnEYGR/LXxCC/ XPxaiz/IUlVreTshp68R0hahDwSK5JcbpOxx9sHFRty6ir+FWEycD/nA6aXkxycoHI+2 aE0ORPnqnNVHLF3NtKuYzyGQPZFprvtyG4NfX9Ag6kmxZMntv5u87mq26TeIIqfrZybT kKnNtEMxAMihL9rfFfkZJErdT7EzMka30nWhpPUyv2QQ5lQPptjERoo2u/xqhais9byp 5bRiHrwH8a0Tv/p3y8Cq/Q2+7d4dQzL6bUGUHb8hUdCi1VZXpCPEsitu7fjBDPdHioi2 XtRw== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:feedback-id:mime-version:references :in-reply-to:message-id:date:subject:cc:to:from:sender :dkim-signature; bh=Abys+VaBi0Qh8p/WMkeJEKWLluPJp5lvj8VKO4ux0j0=; b=xEkxx8i88AYKg1/DAhSR1fniFcshDbD+VTyZ+MUTQMKBjrZWLSBRfdAX7RA6Thj9vw FlYDtQ2b9oCRHO1AbpgtIEm2/aGmdrfufd6qNqFENJNkzmrN+bbxvqmFK/SDuFoCOG7n 8ZEYhBI3uTBYHMusDbM6IC6TRIXEaFgEH+D6qfjx2bMU1jffdUGDs9w8AY4G4dYFvbi9 rIE42/+SV4wCQIZ6q930TzEIbJmDWByhWrWvkvEUjsAVwNbIgesdO6cvXGJQCaxkvFtB R+GzoOmK2JiIY4ZRJR4u+rW38DvrZjbvNm8jmE1FFz0ESRfzyu5OS8dQoTHn1AMG6zQZ 1TaQ== ARC-Authentication-Results: i=2; gmr-mx.google.com; dkim=pass header.i=@siemens.com header.s=fm1 header.b=GOLlDiVT; spf=pass (google.com: domain of fm-52304-20220204164128bf7d40ada8c29bf488-_8ww14@rts-flowmailer.siemens.com designates 185.136.65.227 as permitted sender) smtp.mailfrom=fm-52304-20220204164128bf7d40ada8c29bf488-_8WW14@rts-flowmailer.siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlegroups.com; s=20210112; h=sender:from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:feedback-id:x-original-sender :x-original-authentication-results:precedence:mailing-list:list-id :list-post:list-help:list-archive:list-subscribe:list-unsubscribe; bh=Abys+VaBi0Qh8p/WMkeJEKWLluPJp5lvj8VKO4ux0j0=; b=bevKRrxna25bm0IiwyPv3vMP3I8V3cMwwD2MZlEttk+EsC55gk0fVDcZoxzVO5utG2 DLVtgn5rEWDJ4CDtOYJNhpxFxlhtgi/4eVV66buezRQLv0eRGtZQRN5DM1PYKuYGVN2R Ahonp+fr0IKx+m16nE3GSDBmmyC5VHm+2XYTAQWbZYgdHmzxUL6gyh5MriLykTXNna3f VQ67EiqWLF4A5l8As9h8dpKm/yWJB8bNPBLhym6XB2BxbZl1UZXjgDIW4dBzgunDXgx7 HTkLaWaGKeFcbbTJDok5vm2bRQ0KCkSJE8cOtajvv5p+BpMImfpvrzS03gsPi7+HWxX9 F8aA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=sender:x-gm-message-state:from:to:cc:subject:date:message-id :in-reply-to:references:mime-version:feedback-id:x-original-sender :x-original-authentication-results:precedence:mailing-list:list-id :x-spam-checked-in-group:list-post:list-help:list-archive :list-subscribe:list-unsubscribe; bh=Abys+VaBi0Qh8p/WMkeJEKWLluPJp5lvj8VKO4ux0j0=; b=Sga0WvVK11qTG3KbJdnWiQyFBnZ+o2lpedh118fEfiP4afNYAsuzV2vdY3YStX67Cl FrAX6LWdlr+nXQvrR2Ea3Dfvu5rlhbTuMOc4g7XADAPJY2dU1FQtzxk7d8hD5dp11bNw 1ZsVvEqfsVGjarzHwXxq5GdA4IGzwJCXHWgowWaTSOeVMpMTS+LT9kbnFNSfNNxj1o1u MgyGp179O6QcqCFhZaGsSoqibDOF9xbXsrnNXAl5HYWYP2UDEnFqygjhgVk5RSHR/7L3 n3SRkGA7vz2vod0VeR2qWi25V6uNM0klbpUGpwxDDq+dxqMxnCr7Bjb64r9e8XdihCjf pBRg== Sender: isar-users@googlegroups.com X-Gm-Message-State: AOAM532HJblC2neM54S4Teb3fKm25TAG42PK/MGLnwmxSTpD0x/DXOAa 9WqDRYF3LqQACUiZAC5bDZo= X-Google-Smtp-Source: ABdhPJwvHsJ8oL6+bmaNPKu/NVmxVeSjeyMWcT/oeWmg0uPJUMLXVlTKhQ3kGTJm7pznUHdOACPVWw== X-Received: by 2002:a2e:b05a:: with SMTP id d26mr2251272ljl.425.1643992890846; Fri, 04 Feb 2022 08:41:30 -0800 (PST) X-BeenThere: isar-users@googlegroups.com Received: by 2002:ac2:4e11:: with SMTP id e17ls1823596lfr.1.gmail; Fri, 04 Feb 2022 08:41:30 -0800 (PST) X-Received: by 2002:a05:6512:228a:: with SMTP id f10mr2823536lfu.22.1643992889938; Fri, 04 Feb 2022 08:41:29 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1643992889; cv=none; d=google.com; s=arc-20160816; b=F2p6eWJmRevusimgJ1OKEvrmPGz5MPXXj3SCAgpOB0TxMN/pRAy8ysJG3DQC/ZoUXs QhntUnevgRmDP2SA3vDS/Zp45d6BGgp/PnC2uv0ygItZHMX7ybfEoe37ZOFrhJ3YZZam u8Z/TfbR75Nn8N8EBp3hRLH5Zy5rFD2y2LecMDHWYtCPg+tt8Tu7dBpOUIoRknSFcnm9 F4PqtgitUQ0h8vWUkE6hT+8LOcINzNh1rSLhB5z5WXN+/yfFUpEi6JKBm8TclbNTFx1X 8KxedRUG0mdJ9zBZHVRKiLZSVjuk/6PzcHHaPdfKWZEzeZGWtsWVNrAURZ4gGo/e5e5e NPlg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=feedback-id:content-transfer-encoding:mime-version:references :in-reply-to:message-id:date:subject:cc:to:from:dkim-signature; bh=j+LKr3XzyiDw+F0RhPLy/T/R0XmFJZ1ZokbSq5m0ZSo=; b=kqq8KfL0+HmcD+AUwu1vEI95/6c6uPcE0L67Phb5pL1mLphDOBv3IIWU9Byce0jWL1 THNDR1luNwzbryurl/dWmR9XZA/K6qIUrYhFXQmEmRf1h42WBxTlwaV3LyNol2b/kbjz QY2YlNnBj8hgZCsijakApoBHlRfhQfH1jlqSiLVYQLBb35o9EJ/v5WkJ+R4XeTPTRSD6 OTDE7Ca72ZsWJMsLMxSJcSNEsZdgH4/O54lwdq2QF7NcDgEJE0iQ8Bf6RweGH37swt41 kSBvd5btjuBQKkLq9LhxIUDBlyg/4embA9braFfeI7vjXqn+ub6BVET/IpH/Qr/a3EnZ MOfw== ARC-Authentication-Results: i=1; gmr-mx.google.com; dkim=pass header.i=@siemens.com header.s=fm1 header.b=GOLlDiVT; spf=pass (google.com: domain of fm-52304-20220204164128bf7d40ada8c29bf488-_8ww14@rts-flowmailer.siemens.com designates 185.136.65.227 as permitted sender) smtp.mailfrom=fm-52304-20220204164128bf7d40ada8c29bf488-_8WW14@rts-flowmailer.siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com Received: from mta-65-227.siemens.flowmailer.net (mta-65-227.siemens.flowmailer.net. [185.136.65.227]) by gmr-mx.google.com with ESMTPS id k18si78884lfe.8.2022.02.04.08.41.29 for (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 04 Feb 2022 08:41:29 -0800 (PST) Received-SPF: pass (google.com: domain of fm-52304-20220204164128bf7d40ada8c29bf488-_8ww14@rts-flowmailer.siemens.com designates 185.136.65.227 as permitted sender) client-ip=185.136.65.227; Received: by mta-65-227.siemens.flowmailer.net with ESMTPSA id 20220204164128bf7d40ada8c29bf488 for ; Fri, 04 Feb 2022 17:41:28 +0100 From: henning.schild@siemens.com To: isar-users@googlegroups.com Cc: Henning Schild Subject: [PATCHv3 1/3] sstate: change rootfs task to cache Date: Fri, 4 Feb 2022 17:41:22 +0100 Message-Id: <20220204164124.10396-2-henning.schild@siemens.com> In-Reply-To: <20220204164124.10396-1-henning.schild@siemens.com> References: <20220204164124.10396-1-henning.schild@siemens.com> MIME-Version: 1.0 X-Flowmailer-Platform: Siemens Feedback-ID: 519:519-52304:519-21489:flowmailer X-Original-Sender: henning.schild@siemens.com X-Original-Authentication-Results: gmr-mx.google.com; dkim=pass header.i=@siemens.com header.s=fm1 header.b=GOLlDiVT; spf=pass (google.com: domain of fm-52304-20220204164128bf7d40ada8c29bf488-_8ww14@rts-flowmailer.siemens.com designates 185.136.65.227 as permitted sender) smtp.mailfrom=fm-52304-20220204164128bf7d40ada8c29bf488-_8WW14@rts-flowmailer.siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com Precedence: list Mailing-list: list isar-users@googlegroups.com; contact isar-users+owners@googlegroups.com List-ID: X-Spam-Checked-In-Group: isar-users@googlegroups.com X-Google-Group-Id: 914930254986 List-Post: , List-Help: , List-Archive: , List-Unsubscribe: , X-Spam-Status: No, score=-0.9 required=5.0 tests=DKIMWL_WL_MED, DKIM_SIGNED, DKIM_VALID,DKIM_VALID_EF,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,RCVD_IN_MSPIKE_H2,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on shymkent.ilbers.de X-getmail-retrieved-from-mailbox: INBOX This patch moves sstate caching from the task "do_rootfs" to "do_rootfs_install". Which is earlier in the task chain. There are two "bugs" that we fix by doing so, while hopefully not giving up on too much caching speed. - rootfs_postprocess: - image_postprocess_mark: potentially outdated BUILD_ID and friends - rootfs_generate_manifest: manifest file gone missing in DEPLOY_DIR Since rootfs_postprocess is kind of an open field it might also fix issues in ROOTFS_POSTPROCESS_COMMAND functions in layers. Any function deriving stuff (like a manifest) or changing stuff (like the mark) could potentially benefit from "not caching". Signed-off-by: Henning Schild --- meta/classes/rootfs.bbclass | 25 ++++++++++++------------- 1 file changed, 12 insertions(+), 13 deletions(-) diff --git a/meta/classes/rootfs.bbclass b/meta/classes/rootfs.bbclass index 1fc487896f4d..78fb2cc71350 100644 --- a/meta/classes/rootfs.bbclass +++ b/meta/classes/rootfs.bbclass @@ -270,29 +270,28 @@ python do_rootfs() { } addtask rootfs before do_build -do_rootfs[depends] = "base-apt:do_cache isar-apt:do_cache_config" - -SSTATETASKS += "do_rootfs" +SSTATETASKS += "do_rootfs_install" ROOTFS_SSTATE = "${WORKDIR}/rootfs-sstate" -do_rootfs[dirs] += "${ROOTFS_SSTATE} ${WORKDIR}/mnt/rootfs" -do_rootfs[cleandirs] += "${ROOTFS_SSTATE}" -do_rootfs[sstate-plaindirs] = "${ROOTFS_SSTATE}" -do_rootfs[sstate-interceptfuncs] = "rootfs_sstate_prepare" +do_rootfs_install[depends] += " base-apt:do_cache isar-apt:do_cache_config" +do_rootfs_install[dirs] += "${ROOTFS_SSTATE} ${WORKDIR}/mnt/rootfs" +do_rootfs_install[cleandirs] += "${ROOTFS_SSTATE}" +do_rootfs_install[sstate-plaindirs] = "${ROOTFS_SSTATE}" +do_rootfs_install[sstate-interceptfuncs] = "rootfs_install_sstate_prepare" # the buildchroot is owned by root, so we need some sudoing to pack and unpack -rootfs_sstate_prepare() { +rootfs_install_sstate_prepare() { sudo mount --bind ${WORKDIR}/rootfs ${WORKDIR}/mnt/rootfs -o ro sudo tar -C ${WORKDIR}/mnt -cpf ${ROOTFS_SSTATE}/rootfs.tar --one-file-system rootfs sudo umount ${WORKDIR}/mnt/rootfs } -do_rootfs_sstate_prepare[lockfiles] = "${REPO_ISAR_DIR}/isar.lock" +do_rootfs_install_sstate_prepare[lockfiles] = "${REPO_ISAR_DIR}/isar.lock" -rootfs_sstate_finalize() { +rootfs_install_sstate_finalize() { sudo tar -C ${WORKDIR} -xpf ${ROOTFS_SSTATE}/rootfs.tar } -python do_rootfs_setscene() { +python do_rootfs_install_setscene() { sstate_setscene(d) - bb.build.exec_func('rootfs_sstate_finalize', d) + bb.build.exec_func('rootfs_install_sstate_finalize', d) } -addtask do_rootfs_setscene +addtask do_rootfs_install_setscene