From patchwork Mon Feb 27 07:50:32 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: venkata.pyla@toshiba-tsip.com X-Patchwork-Id: 2701 Return-Path: Received: from shymkent.ilbers.de ([unix socket]) by shymkent (Cyrus 2.5.10-Debian-2.5.10-3+deb9u2) with LMTPA; Mon, 27 Feb 2023 08:50:09 +0100 X-Sieve: CMU Sieve 2.4 Received: from mail-oa1-f55.google.com (mail-oa1-f55.google.com [209.85.160.55]) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPS id 31R7o7Vc005916 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 27 Feb 2023 08:50:08 +0100 Received: by mail-oa1-f55.google.com with SMTP id 586e51a60fabf-1728701fdb1sf3773617fac.11; Sun, 26 Feb 2023 23:50:08 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1677484202; cv=pass; d=google.com; s=arc-20160816; b=jqyasCw47uMBAUZgpGq7GY2IwjArSzFviCSFVePkP8pnnzBLtBN0FB3DxmS44/om4X JL3WTjttmu6ocDlttDs21dFZvxLoe3laGyMlseBYsPU1FxuEGxiuxMyHeI7otdl0pdIS bojbWjAvc2iBfOwuM/enOzOYgR+67r1goBH8d8JYzya+LuCPYKbLYC0ZH7LvjgFzIa9z 8DGt3gQcj4NvSfOhdd+hblGnKfnqMp6E+Xz/nLUR2f22GPjvV4Vi3Wv48+pZ62/kovRx m//NWPor3UboblTcJjobhJi88tqmZLNrtmkqPYiqWQsgEqqObbjtghKvg6GVHHT78gGl XuZg== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:mime-version:message-id:date :subject:cc:to:from:sender:dkim-signature; bh=bE/Zhm5smQaIBBBgU3z4KTBVRwvFk8QHHRWyaucbEC8=; b=Uyk6y2pS7aX/WI3ofZwNebX2mHoSqmSucNEE/cAoS7NdYl3lMvgAJKF57gep5URIr1 2ldKGMh2jxYadm3sT+tUuvg277tQEDNzdS/MCdgJuKcd0BiEWpuz6+clffM0HOMN/w20 6VuwQVy6mDBQZqS4axdyLkXNH3YRnaplFCK/IVVgp2gTpkAkFp6nuuzn/dnz5ko+8wzk kMr315eWkczwPjtcet/IdwDcvon0RHxDd5AotuHWLRljhRfRg7wUofKmXZ/n4qTIhx7p iwVogoplMD+LnZPgEyU+PCCU7rFKElWSzgMJVgkoAARGX9F9pmUsNonqKBX7qrOVY6la 0C+g== ARC-Authentication-Results: i=2; gmr-mx.google.com; spf=pass (google.com: domain of venkata.pyla@toshiba-tsip.com designates 210.130.202.153 as permitted sender) smtp.mailfrom=venkata.pyla@toshiba-tsip.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlegroups.com; s=20210112; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:x-original-authentication-results :x-original-sender:mime-version:message-id:date:subject:cc:to:from :sender:from:to:cc:subject:date:message-id:reply-to; bh=bE/Zhm5smQaIBBBgU3z4KTBVRwvFk8QHHRWyaucbEC8=; b=KCew9iwmN1jJ9wC3dFLdyaSKSbtF+3M1UlHPYptE6Cau2qT0zW0fenGhhvhzdj/P77 YJaWTciOdGK4+esIK9Mcn0+JRBL18vIpXLunzTdVJDyLudJmZ6ibRo5u5hpJRd6EMLBI 1KJb+omfVoDoqwnHDPZEHMsDBpBRRNA6wnMaV3N8OnYS+3BYws1HIwdEJklN26t7zmgP 8byfwrKtd3/RBp3fzXEzcXTwxM4o1mScRaet2waO4Gt+uGSFb0kUHU/3Wiehm15NMlqT dd6rAOsbcgy0HV0h3T0ELR1mO2uD3F1tIEjLJTrb+NXSolGeRk39WdxybM7N/7P/k/OR IiuA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :x-spam-checked-in-group:list-id:mailing-list:precedence :x-original-authentication-results:x-original-sender:mime-version :message-id:date:subject:cc:to:from:x-gm-message-state:sender:from :to:cc:subject:date:message-id:reply-to; bh=bE/Zhm5smQaIBBBgU3z4KTBVRwvFk8QHHRWyaucbEC8=; b=1SlafMbcr7VfvIges54DNE2RarjnzVT3SBiBkF5mj9MoVag7WGK4+BBqTg/8U7arZF l0fa7YOm+F9VBJcZ5WxesLYQwFDcZlvgtV/MWMmzeEdOEuG76QEExrfugaVjkcFcqPCu uSl7XZTYNkSx7+Me53yGYmh5u2RKmZse3WG15L/HW2j0G0BHZlNUqjRsXUUdxD/Rbxra anl2aj+VMlON7mag2JjpSCRfuDN2RZEGxmV2RTzx9qf2YILjGVRYc9wyn2atjOm+czlJ sX81Qs7h46QhXRch8flqNqAIny52yE7TaVcU87FNvw88chmDhvlGmY8Wmq1utL0zIW5G WnTQ== Sender: isar-users@googlegroups.com X-Gm-Message-State: AO0yUKUqaUg/QOdgQyCdNPOXNe9Mj1sej+87Y3rwYPrfISI9wJk+Sgup YRM21IJa2PIPIg+pkIEOPQE= X-Google-Smtp-Source: AK7set8l1zpsf57hKyfxXQhzRW87u8b0yW9QlxjelEZFj6bV5j4prZjGARSZnawUVMPpb+A+DsvKMw== X-Received: by 2002:a05:6870:98aa:b0:172:4745:f577 with SMTP id eg42-20020a05687098aa00b001724745f577mr3338835oab.1.1677484202274; Sun, 26 Feb 2023 23:50:02 -0800 (PST) X-BeenThere: isar-users@googlegroups.com Received: by 2002:aca:2209:0:b0:383:b2c0:4e6a with SMTP id b9-20020aca2209000000b00383b2c04e6als2837106oic.2.-pod-prod-gmail; Sun, 26 Feb 2023 23:50:01 -0800 (PST) X-Received: by 2002:a54:4692:0:b0:37a:f32f:730c with SMTP id k18-20020a544692000000b0037af32f730cmr9551790oic.35.1677484201759; Sun, 26 Feb 2023 23:50:01 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1677484201; cv=none; d=google.com; s=arc-20160816; b=Uhv9X4sYGgRakjBEvOc5nMI1qN7tl7ybcUPPexgcGNP28wHZ6oEjJo1zQ70JYXJB3m vxtkCxWpXEgIDoxpBwxJriqCXd2fzc6twDvbo1+nQXLaHglTqogCwrmLQFbn4Fb3exq4 V0SLTHvPAPV7Qcfu82PoD0uzNZDqqDxySP0ZScAHmAM5tDfgC7PcQ332PtDUAyn12dXq ebxRU9EN7EmxDgfM4b7/GXpLDMH3BWjCKv5GvVx+syXJ0xz3dKnnqt8TG1bubxxdzhCE yMe/2AhZVYtwsJszkAlssbF57vtvxLnE+H6469mtPUyscva5wlyDumRWbwryE3P29nvA bBUw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from; bh=lCyY+xKdxx/tf0yG0iZgRg9aOe5uAhKo5F/N8f37fzQ=; b=NomiIVSquM3hlc1fx0wMi7KFTyGSlVV8LhBZ5b2n5vRPzSt8E2VlgxLwk4xg1Fx/c4 oXG0VIjIqolrxSg5em0vqvE8GRVqAafpDgIeu/wjeENioR/ofYUqZDCXPqVVE1WmhhSj B7cvNfkwbHnvsgb6DPwiPLA4/8WpjoKB1JYhAJFxYsD7tcwgxjo/I1QEmAzIeA3DRlzp NC4fmMVD4b3P+txeI9e9dq+YAygHkte1kTKuGYMxFYJidxIM3QyNNL9b9j2L+Rtu+yJT RQskZp5z3yRnyrQGemyBj/HW7VUqtviX7ItzYcpOeA8IpNQsZ4qGxk7AlImasPS945I/ zgmg== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of venkata.pyla@toshiba-tsip.com designates 210.130.202.153 as permitted sender) smtp.mailfrom=venkata.pyla@toshiba-tsip.com Received: from mo-csw.securemx.jp (mo-csw1514.securemx.jp. [210.130.202.153]) by gmr-mx.google.com with ESMTPS id bh4-20020a056808180400b0037fa46467c0si329926oib.0.2023.02.26.23.50.01 for (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Sun, 26 Feb 2023 23:50:01 -0800 (PST) Received-SPF: pass (google.com: domain of venkata.pyla@toshiba-tsip.com designates 210.130.202.153 as permitted sender) client-ip=210.130.202.153; Received: by mo-csw.securemx.jp (mx-mo-csw1514) id 31R7nxGs007160; Mon, 27 Feb 2023 16:49:59 +0900 X-Iguazu-Qid: 34tro5DMQMnkZK47sU X-Iguazu-QSIG: v=2; s=0; t=1677484199; q=34tro5DMQMnkZK47sU; m=mXOrX9X4QS4jMEBvqA1xXiVrIMhSHwJ6Z7yd8FwSmy0= Received: from imx2-a.toshiba.co.jp (imx2-a.toshiba.co.jp [106.186.93.35]) by relay.securemx.jp (mx-mr1510) id 31R7nwiA017099 (version=TLSv1.2 cipher=AES128-GCM-SHA256 bits=128 verify=NOT); Mon, 27 Feb 2023 16:49:58 +0900 From: venkata.pyla@toshiba-tsip.com To: isar-users@googlegroups.com Cc: venkata pyla , felix.moessbauer@siemens.com, roberto.foglietta@gmail.com, jan.kiszka@siemens.com, henning.schild@siemens.com, kazuhiro3.hayashi@toshiba.co.jp, dinesh.kumar@toshiba-tsip.com Subject: [RFC PATCH] image.bbclass: create separate task for creating reproducible image Date: Mon, 27 Feb 2023 13:20:32 +0530 X-TSB-HOP2: ON Message-Id: <20230227075032.28099-1-venkata.pyla@toshiba-tsip.com> X-Mailer: git-send-email 2.20.1 MIME-Version: 1.0 X-OriginalArrivalTime: 27 Feb 2023 07:49:56.0591 (UTC) FILETIME=[15C917F0:01D94A80] X-Original-Sender: venkata.pyla@toshiba-tsip.com X-Original-Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of venkata.pyla@toshiba-tsip.com designates 210.130.202.153 as permitted sender) smtp.mailfrom=venkata.pyla@toshiba-tsip.com Precedence: list Mailing-list: list isar-users@googlegroups.com; contact isar-users+owners@googlegroups.com List-ID: X-Spam-Checked-In-Group: isar-users@googlegroups.com X-Google-Group-Id: 914930254986 List-Post: , List-Help: , List-Archive: , List-Unsubscribe: , X-Spam-Status: No, score=-0.8 required=5.0 tests=DKIMWL_WL_MED,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_EF,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,RCVD_IN_MSPIKE_H3,RCVD_IN_MSPIKE_WL,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on shymkent.ilbers.de X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= From: venkata pyla Currently the reproducible fix for setting same file timestamps across builds is present in the function `do_rootfs_finalize` and this fix may be skipped for the tasks that are added after this task or `do_rootfs` for e.g. in one of the child project(CIP) for swupdate target it adds additional task `do_generate_image_uuid` that is added after `do_rootfs` this task is updating some contents in the rootfs which is skipping the reproducible fix was applied in `do_rootfs_finalize`. For this reason a separate task `do_image_make_reproducible` is created which should call before the final target image is created [tar, wic, squashfs] so that it applies reproducible fixes just before image creation. RFC: In this patch I tried below dependency[1] for the function to solve the above problem, it works now but I am not sure whether it guarantee always this task `do_image_make_reproducible` will run just before `do_image_[tar,wic,squashfs]` tasks or does it need more changes to make it guarantee, any suggestions? [1] addtask image_make_reprodcible after do_rootfs before do_image Signed-off-by: venkata pyla --- meta/classes/image.bbclass | 33 ++++++++++++++++++++------------- 1 file changed, 20 insertions(+), 13 deletions(-) diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass index ef7d5a2..c3a3d45 100644 --- a/meta/classes/image.bbclass +++ b/meta/classes/image.bbclass @@ -446,19 +446,6 @@ do_rootfs_finalize() { fi EOSUDO - # Set same time-stamps to the newly generated file/folders in the - # rootfs image for the purpose of reproducible builds. - if [ -n "${SOURCE_DATE_EPOCH}" ]; then - fn="${DEPLOY_DIR_IMAGE}/files.modified_timestamps" - if sudo find ${ROOTFSDIR} -newermt "$(date -d@${SOURCE_DATE_EPOCH} '+%Y-%m-%d %H:%M:%S')" \ - -printf "%y %p\n" -exec touch '{}' -h -d@${SOURCE_DATE_EPOCH} ';' | egrep ^f >"$fn"; then - if [ -e "$fn" ]; then - bbwarn "modified timestamp (${SOURCE_DATE_EPOCH}) of $(cat "$fn" | wc -l) files for image reproducibly." \ - "List of files modified can be found in: .${DEPLOY_DIR_IMAGE}/files.modified_timestamps" - fi - fi - fi - } do_rootfs_finalize[network] = "${TASK_USE_SUDO}" addtask rootfs_finalize before do_rootfs after do_rootfs_postprocess @@ -502,3 +489,23 @@ do_rootfs_quality_check() { do_rootfs_quality_check[network] = "${TASK_USE_SUDO}" addtask rootfs_quality_check after do_rootfs_finalize before do_rootfs + +# Run this task just before final image creation of different image types +# i.e do_image_tar, do_image_squashfs, do_image_wic etc. +# to avoid modification of image contents that leads to non-reproducible +# image +do_image_make_reproducible() { + # Set same time-stamps to the newly generated file/folders in the + # rootfs image for the purpose of reproducible builds. + if [ -n "${SOURCE_DATE_EPOCH}" ]; then + fn="${DEPLOY_DIR_IMAGE}/files.modified_timestamps" + if sudo find ${ROOTFSDIR} -newermt "$(date -d@${SOURCE_DATE_EPOCH} '+%Y-%m-%d %H:%M:%S')" \ + -printf "%y %p\n" -exec touch '{}' -h -d@${SOURCE_DATE_EPOCH} ';' | egrep ^f >"$fn"; then + if [ -e "$fn" ]; then + bbwarn "modified timestamp (${SOURCE_DATE_EPOCH}) of $(cat "$fn" | wc -l) files for image reproducibly." \ + "List of files modified can be found in: .${DEPLOY_DIR_IMAGE}/files.modified_timestamps" + fi + fi + fi +} +addtask image_make_reprodcible after do_rootfs before do_image