From patchwork Tue Nov 29 17:58:12 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Uladzimir Bely X-Patchwork-Id: 29 Return-Path: Received: from shymkent.ilbers.de ([unix socket]) by shymkent (Cyrus 2.5.10-Debian-2.5.10-3+deb9u2) with LMTPA; Tue, 29 Nov 2022 18:58:26 +0100 X-Sieve: CMU Sieve 2.4 Received: from mail-pg1-f189.google.com (mail-pg1-f189.google.com [209.85.215.189]) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPS id 2ATHwOk0003028 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 29 Nov 2022 18:58:25 +0100 Received: by mail-pg1-f189.google.com with SMTP id x16-20020a63b210000000b0045f5c1e18d0sf9693152pge.0; Tue, 29 Nov 2022 09:58:25 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1669744699; cv=pass; d=google.com; s=arc-20160816; b=WifwDp1x9uTJPeKiWRE6fOmW50glyp+dTyZxsdf7Vt8z6vpx4sUbsNB9JcbgG+bsRh VdyVoIMVEHMkokXU4WdDiNhrkRA0FVshlUDnAjZIxM6kQ9VWflficxygEpeqH+gplyVI 7HpknpzRZSbnE4V1QId8Z61bCCR1/KltpLWbvW1k1ykh03fMIR1++FPrl4DgvfEf2yn7 ew8phZ0NnriELQK+rGBdAZFe0DUGK+GXgUw9EDxZMw7eM15VAhKoYdAh8eBEvm21wtiL Ge6HQyVrg4fc6Qhk/vqEYd6RT0zspFdM0NeIkIt400RUuHcq3bvJURNbDpMX+Ysq/Yxv FLog== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:mime-version:message-id:date :subject:to:from:sender:dkim-signature; bh=9nq4b4aHQB1E1MZ6yBsVtFVkP15+3rHGuQ/l1hG/I4Q=; b=d8/I4W1CvDKUadtpH+WHt9hB5V90Ev5tSL+PhjBMY910ZcRbp1WY5k5In577o8L7Qh iNCLBUud1VmEJsJ9S+MTmdcDpjCoDoFuLkp62OLdpSsmNyWDhJOeK3mQyfqtqVOqc0A5 4odaI69cYncYugR7jdg+4Qk2qy1S7/8DNqyQ5eQ95OTPmufRWnXcHM6yc+W67PhoIZLK +hkopidXDifFTQSNYoha/Oag3FlFdmDnwPCQl64t+c/Vdq0um7iTgmIUnwj6l8L8d27Q mrcc1o9lQ9v9Mog9IXW4Ol0L7m6zvSwLAQWtPiKMkxW1phVwQiygfa15u5ilA2JnBJ0i jG7A== ARC-Authentication-Results: i=2; gmr-mx.google.com; spf=pass (google.com: domain of ubely@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=ubely@ilbers.de DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlegroups.com; s=20210112; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:x-original-authentication-results :x-original-sender:mime-version:message-id:date:subject:to:from :sender:from:to:cc:subject:date:message-id:reply-to; bh=9nq4b4aHQB1E1MZ6yBsVtFVkP15+3rHGuQ/l1hG/I4Q=; b=p48I10w/JI0oLModqNSoZxP9VTKQK5YedhI0G7dP75epYaQ8q7nDfpRYR/Krig2FoC 8LzfE5m0oe4Gulwa2gqF7tqJMn+3YdRzeXIdNHpwDI9aib2DPuN3KxJJRgEUGZSWLPrw 9L3YoGHf78ruGnwSpbZm9ctlzKYrYxBI9eDuBkawW7da8eKdgEQvWcGwwfVR2//5Onqv 9Vnv/IuMwYWumSvRaA+vTzIXHJXL4fPUtISVFQFboDH12uM53IcPxLvB5Zdsz7gHyf9k HUy1XlE7xGYfPyIxq9iFrA/pCXCGLr7WYgmxNgHHKSGrobESaGCW2oEMdXcS1xNsv2oi dmbw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :x-spam-checked-in-group:list-id:mailing-list:precedence :x-original-authentication-results:x-original-sender:mime-version :message-id:date:subject:to:from:x-gm-message-state:sender:from:to :cc:subject:date:message-id:reply-to; bh=9nq4b4aHQB1E1MZ6yBsVtFVkP15+3rHGuQ/l1hG/I4Q=; b=NuI59iWoW3SmSKTu7njOm+guRg7HN0EIdx0qtAj8Rmuj9MW6jupzmY58fjEqcPjpqs i8ulUjPo1T4UjB5RK057tHq5IR5FVYJxCg4dqsAyR8vx8vcKQ0w73fzlQU8LIoxT26gb P0SZ7+VaFdaDayT7aMI43Vq1MgLqmf7SLFOKOoOjPlQao3mkB+GDlhI35Ds3Qinh4muE YTLiaAX26mmzx/0/ioASNbqfncnI3/XNMEceeGgpUdTdAlAoNtMDFZ0xpGN4WMRW1Ly/ g/gX6fsQJyUyaFpJLW3FcEhWpNV4qZPE9k2LtnJo9JytdburQDMn4H27v34so0c73/+o aOpQ== Sender: isar-users@googlegroups.com X-Gm-Message-State: ANoB5pmWHIoVV/bbxetBSa54iJNM4LYGRSgvyoyj1DdTXxJJpzwvS7bh h5zyUH3vLAPvlqZ9+X0LJgw= X-Google-Smtp-Source: AA0mqf7jTCZB6TEq4ngq9iqP7E0aUMZ62XIUNmblIDjABJKcEWyI5XwebrUSCUdJtXlIG/DttNAiJQ== X-Received: by 2002:a05:6a00:1941:b0:572:6e9b:9fa2 with SMTP id s1-20020a056a00194100b005726e9b9fa2mr38624646pfk.8.1669744698828; Tue, 29 Nov 2022 09:58:18 -0800 (PST) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a17:903:40c3:b0:189:33cd:635 with SMTP id t3-20020a17090340c300b0018933cd0635ls11256887pld.5.-pod-prod-gmail; Tue, 29 Nov 2022 09:58:17 -0800 (PST) X-Received: by 2002:a17:903:555:b0:189:959a:84cc with SMTP id jo21-20020a170903055500b00189959a84ccmr7023079plb.41.1669744697809; Tue, 29 Nov 2022 09:58:17 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1669744697; cv=none; d=google.com; s=arc-20160816; b=b9TaUSibRBcP9liOQm9fQL1IiQdR1cNd7ccDiIh7Yr2QXmZikdtSms1Oakmi75PmCc u5ASKuOK+o4PvkrOTZd2muNaVNmifGE/cUQnnd+ZHc1uhBdmL4Js6JzDMAe25O80TpSm /peKvsEs2Vbf/pqnC5fbyCwiASRkl/2jccVoqr2Rl56cZsz03mtguoL1Lhf3ge8bZu/n E5FU1TQpU2eAf2VnYeFUUiHxG6AQeH0Q1HUZFiyiJ+dkxDOoHrDf7sZURTgAhI7jUHS5 TSz+BpSEyfNHaAEKkRJqXvkzVfBZxJemgvYuPF1b6iBU+cUQP+VK6gHIkXIQo8CrSfmn JW6A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:message-id:date:subject:to :from; bh=Ja29AW6VLupyLjccoR4F4JC781mPJz17iQPpeauah0s=; b=aWPcuq2nsOCw/rlY5TuVpFo+dfDEoKBttYgx+ohZj3C5JamOZ1k3lhRYu7u9PqJvyA yIA2WdnWh7E9I7yrODo2ietRkEwNNlRPseh6fbD+p4sBvzrTChDXMs7hUqzpcyfotvNr 2b8SRtF+fe/HT85suLvgBX3r2i7zUyFAQu60pvjEWvSLVEA+EZm+GGbXoWei7ogEA7Hf rFarD460ngYoPo/s/zH6zZwj1yCiX3wYJPw2LStu/RbgTsHygCCH9fw5b5M/PyN2plMa W14QrELylyZdO6oPCI0UrzydOMkMw/qaoJmNKGlMlK4bi7jgD5A/h2D9nDMUNNxa4xnR Ck5Q== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of ubely@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=ubely@ilbers.de Received: from shymkent.ilbers.de (shymkent.ilbers.de. [85.214.156.166]) by gmr-mx.google.com with ESMTPS id i1-20020a625401000000b0056d189fd322si670359pfb.2.2022.11.29.09.58.16 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 29 Nov 2022 09:58:17 -0800 (PST) Received-SPF: pass (google.com: domain of ubely@ilbers.de designates 85.214.156.166 as permitted sender) client-ip=85.214.156.166; Received: from baighyz.m.ilbers.de (host-80-81-17-52.static.customer.m-online.net [80.81.17.52]) (authenticated bits=0) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPSA id 2ATHwEbP003006 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Tue, 29 Nov 2022 18:58:14 +0100 From: Uladzimir Bely To: isar-users@googlegroups.com Subject: [RFC PATCH 0/2] check Isar image for reproducability Date: Tue, 29 Nov 2022 18:58:12 +0100 Message-Id: <20221129175814.16355-1-ubely@ilbers.de> X-Mailer: git-send-email 2.20.1 MIME-Version: 1.0 X-Spam-Status: No, score=-0.8 required=5.0 tests=DKIMWL_WL_MED,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_EF,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,RCVD_IN_MSPIKE_H3,RCVD_IN_MSPIKE_WL,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on shymkent.ilbers.de X-Original-Sender: ubely@ilbers.de X-Original-Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of ubely@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=ubely@ilbers.de Precedence: list Mailing-list: list isar-users@googlegroups.com; contact isar-users+owners@googlegroups.com List-ID: X-Spam-Checked-In-Group: isar-users@googlegroups.com X-Google-Group-Id: 914930254986 List-Post: , List-Help: , List-Archive: , List-Unsubscribe: , X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= This is an reimplementation of the idea proposed in https://groups.google.com/g/isar-users/c/4ZIuKCOQzVc/m/sPUafWDgAgAJ that uses avocado test instead of custom script. The test is placed under KFAIL while images are even not close to be considered reproducable for now. Some things to improve the patchset: - Check not only `tar.gz` images, but target rootfs directories, so it should save time and allow supporting various multiconfigs. - Not only generate diff statistics, but also analyze "total" number of differencies we see. - ... Uladzimir Bely (2): Install diffoscope binary if not installed testsuite: check Isar image for reproducability. scripts/ci_build.sh | 5 +++++ testsuite/cibase.py | 44 ++++++++++++++++++++++++++++++++++++++++++++ testsuite/citest.py | 18 ++++++++++++++++++ 3 files changed, 67 insertions(+)