From patchwork Tue Jul 9 17:31:37 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jan Kiszka X-Patchwork-Id: 3673 Return-Path: Received: from shymkent.ilbers.de ([unix socket]) by shymkent (Cyrus 2.5.10-Debian-2.5.10-3+deb9u2) with LMTPA; Tue, 09 Jul 2024 19:31:50 +0200 X-Sieve: CMU Sieve 2.4 Received: from mail-lf1-f62.google.com (mail-lf1-f62.google.com [209.85.167.62]) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPS id 469HVn1x006532 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Tue, 9 Jul 2024 19:31:50 +0200 Received: by mail-lf1-f62.google.com with SMTP id 2adb3069b0e04-52ea96517a6sf4654957e87.0 for ; Tue, 09 Jul 2024 10:31:50 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1720546304; cv=pass; d=google.com; s=arc-20160816; b=KtzLsS2zdi6yG3iikI28zpsGKURTpM4AeC6lQuZdFkoc+HusnoBX8G1c0LMdPu4ubW 1+4CLM8R5W+qDtI0DQzdkSy4lvQVMIexMlaazOEEGIzZrY+zNbUfBzKhta3vqfdEB2Xj BimxH0QEwZrDCxIplbZJqqEBQP0BV5Kch6Xs1WH4dbh4q5v1b9GKweQi62JgLUg3WpFd sXDC5/F8TCw5cRFzs7cnQOcrt0kYGngmbG98wYf2j+QCe1Etr1u40M/H+pZvfp9q2Xb3 FSoqY1QZekQd/owMLOfcF37dH6x09SbwSkqgffgUZMvV+dwImGVHmZhCnaKUpIdkD/D1 cfnQ== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:reply-to:feedback-id:mime-version :references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=wUR46tngSfB5cZc8nNKXaUBTvDRWVT0hOZzOppbQG3o=; fh=zAgQillEVyp+qXIT1i2gYHOKKVEx56lyLs3BNR2VE2I=; b=DLkFQnEhiSbS/0xtiYKoU+tQFquGjwBQTTxNkPtEPu7pJOmq/PQRNkrZ6pF7EQzgZ6 /wYEgjXg9iJQlU55rbYDKI0EGaPaDA6fjob5QwWCUcDKsoHCxH/66e4PcKH/mjW+drnM nVEqweHy+RPSN1LO6glUsMgs7E4SiXZbMmQAosL1DlbiCnSbM3E4Bdlv8lFCyA37vvEg culokzfF1HamS5Zsb8AP047ME7gOS2aK81UxSUJMwdYlAK7fz630W9jUMxXm6gGrJXEQ KTTt2U4c4NVaeDgmR6ywPq3F74FKuxEB6uj0WK2orVfj8EynYLK6rxnKJJKlfhF5fF0g DTYg==; darn=isar-build.org ARC-Authentication-Results: i=2; gmr-mx.google.com; dkim=pass header.i=@siemens.com header.s=fm2 header.b=L3VrYHtm; spf=pass (google.com: domain of fm-294854-2024070917314078e9b583f9bda058c6-fiuspa@rts-flowmailer.siemens.com designates 185.136.64.225 as permitted sender) smtp.mailfrom=fm-294854-2024070917314078e9b583f9bda058c6-FIUSpa@rts-flowmailer.siemens.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=siemens.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlegroups.com; s=20230601; t=1720546304; x=1721151104; darn=isar-build.org; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:reply-to :x-original-authentication-results:x-original-sender:feedback-id :mime-version:references:in-reply-to:message-id:date:subject:to:from :from:to:cc:subject:date:message-id:reply-to; bh=wUR46tngSfB5cZc8nNKXaUBTvDRWVT0hOZzOppbQG3o=; b=lzobixiV5rrc/A2TVPp+KeGEwpGtSgoK061g3O4dja6iaA0xNG2Dt4HkBOM6Ndiv9G MHJKUyoe/Y9m+ITlHttYR6qRINT8LtfBJIfyWj2EsPbuGzcNWXa4wmKZUvnMRrvD46/+ QgcUGS4bKo2AXSM/3cCImAPgYFqv2qUSSe0ggNGsMl3O1Ca/Lx+VRgSFsk9A6VYbhuvO vb81gq/DMgz8s+m0366bZvgN/ffiISM8CldMO9eucIf1rDTngyATS+vXj3Uf4yjnw+B9 ai3K9MFHbkdBsp4CzWJChBNcYQ6/eEeOR1l5cw2TSFnp3SO1KLkzGCpXlN+jyVfjNiRd 3HBw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1720546304; x=1721151104; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :x-spam-checked-in-group:list-id:mailing-list:precedence:reply-to :x-original-authentication-results:x-original-sender:feedback-id :mime-version:references:in-reply-to:message-id:date:subject:to:from :x-beenthere:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=wUR46tngSfB5cZc8nNKXaUBTvDRWVT0hOZzOppbQG3o=; b=khdPX+ATShHQETROFjpmTgaCIYbqTuv34xwK4b8IeXGGlYo+Y58db60jUD5PGwcaEN vpIQSYqQTJzk0GeutDC64PBExsVcR0rV4lfmClxldZrYXG6bj6jvruMxtPPW1uVX0GpH PRRQoRTRTuCS5B4uxPE1YIh4VFxX4A+9bP3hFu6hS/THHw8/ZAu8XmXY6rHM4UmcHNL0 d/ud03hQvU5S0JQ9zxGrJl/4y5pMURhUws3Hyc7MNyoO/hOb+9BeLRkafq+5lzl5e7Vt 7m1Q+ZiABR+AQ3wPr5oiigFM1NXczZ/ESVTynNvFdZ8WJ/9Kz06Sar6I5rU0Jeqd4DQY IuVg== X-Forwarded-Encrypted: i=2; AJvYcCWF451OccETgoEsl9ly/Z8qkA44MAOgf9em4bzS7KjwRuAhoVXzxT8dgMzzLrXX0EaQB09aqjNa7l4o9sK+x+MKhwloH6Q= X-Gm-Message-State: AOJu0YzcIdq2gPupBPjQ/hZguZb5MTzZxCly+79Ccx12YfGssJKkSqUO wPzkHpUc3TyYGjH6CoHdwc+JUROd21X7luavkq6h+o+Xn0z+UvNV X-Google-Smtp-Source: AGHT+IFbzF9vTf5QEz/NXr6d4bHeiD8Y5qbN/UinPZKyVhF9jEHU8xhSb+v26eFdi1D/nf2tP/6MNA== X-Received: by 2002:a05:6512:3b87:b0:52e:9d2c:1c81 with SMTP id 2adb3069b0e04-52eb9990ccamr2336007e87.13.1720546303719; Tue, 09 Jul 2024 10:31:43 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a05:6512:31d3:b0:52c:a105:90ee with SMTP id 2adb3069b0e04-52e9f301e67ls1488533e87.1.-pod-prod-01-eu; Tue, 09 Jul 2024 10:31:41 -0700 (PDT) X-Received: by 2002:a05:6512:3d8f:b0:52e:7688:2817 with SMTP id 2adb3069b0e04-52eb9994ab3mr1863067e87.17.1720546301347; Tue, 09 Jul 2024 10:31:41 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1720546301; cv=none; d=google.com; s=arc-20160816; b=LNuVddjc93aBv7Ulep3tn4Z7qY/62ThyFWDJ05Z1Kh/rqH1CVe/LRU9fLrrG63LTVo pw0F40l8J3Uxy8lk0x9MTQfJNM6mzW28zVG7Stn2Vp0ep+qUE0XazlL1hjCDoWSU/VRr FuYGUzwJZPcFbVRicFGq6M/xnpMUK6mdPV2DO5nP7Tz9mPuV3W+MTekN12BucEgyu9gM acJe/2iDo+4pslaNuFRB0sv+XCd7kCKfQ1TEZc/7REw7xhc9s5z+lwngrwqDkZN5d4z5 LC7W+CryOWkzyWTGSAqoG0+C0gxlYeK4KxT87wWwA2F4vAsNYQqP64ZYDsdFyBN2hCIb 6hiw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=feedback-id:content-transfer-encoding:mime-version:references :in-reply-to:message-id:date:subject:to:from:dkim-signature; bh=fanyJ2CdPImoQzmlj8y6E58TDb+QAfcBjuwhNoStB48=; fh=hOBXq6nisExkIRIwN8IDOmOCLKwX8PM0ANlxQbNMEqA=; b=hGom/G19dkdpbmfRCVWh3ft5rZjChIdc6XNke4ElhMcrr23MkYGtipEb196XjXle17 gfqMqv9gjyF93zFq32Nuvw2oRmAvUWj8t5qpgP1uORzdJKoOHM1TIq/baOIA28MPrkxu Rfy0qZkSuwNh7NiWKzpjKHQkaQluX5i3vl8tfXuv7ZaDiGOevg4oYfXYsNtCjRlSpS+g 4wtIiM88+TOi1g+wL4lXQPr2gCaNIAsOws2uhpP9SulP6XBP0PIx/nJfiN5U6kMqN/Rj KYnpENROYRqJO0xo6aUD9O/1HIDMSmzkp5YT0JPN8XWRhmO5d2M8Jz2CGK7NQOlQ72cc A6hg==; dara=google.com ARC-Authentication-Results: i=1; gmr-mx.google.com; dkim=pass header.i=@siemens.com header.s=fm2 header.b=L3VrYHtm; spf=pass (google.com: domain of fm-294854-2024070917314078e9b583f9bda058c6-fiuspa@rts-flowmailer.siemens.com designates 185.136.64.225 as permitted sender) smtp.mailfrom=fm-294854-2024070917314078e9b583f9bda058c6-FIUSpa@rts-flowmailer.siemens.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=siemens.com Received: from mta-64-225.siemens.flowmailer.net (mta-64-225.siemens.flowmailer.net. [185.136.64.225]) by gmr-mx.google.com with ESMTPS id 2adb3069b0e04-52eb8e5237bsi89199e87.7.2024.07.09.10.31.41 for (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 09 Jul 2024 10:31:41 -0700 (PDT) Received-SPF: pass (google.com: domain of fm-294854-2024070917314078e9b583f9bda058c6-fiuspa@rts-flowmailer.siemens.com designates 185.136.64.225 as permitted sender) client-ip=185.136.64.225; Received: by mta-64-225.siemens.flowmailer.net with ESMTPSA id 2024070917314078e9b583f9bda058c6 for ; Tue, 09 Jul 2024 19:31:40 +0200 X-Patchwork-Original-From: "'Jan Kiszka' via isar-users" From: Jan Kiszka To: isar-users Subject: [RFC][PATCH 1/3] Introduce fetcher from container registries Date: Tue, 9 Jul 2024 19:31:37 +0200 Message-ID: In-Reply-To: References: MIME-Version: 1.0 X-Flowmailer-Platform: Siemens Feedback-ID: 519:519-294854:519-21489:flowmailer X-Original-Sender: jan.kiszka@siemens.com X-Original-Authentication-Results: gmr-mx.google.com; dkim=pass header.i=@siemens.com header.s=fm2 header.b=L3VrYHtm; spf=pass (google.com: domain of fm-294854-2024070917314078e9b583f9bda058c6-fiuspa@rts-flowmailer.siemens.com designates 185.136.64.225 as permitted sender) smtp.mailfrom=fm-294854-2024070917314078e9b583f9bda058c6-FIUSpa@rts-flowmailer.siemens.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=siemens.com X-Original-From: Jan Kiszka Reply-To: Jan Kiszka Precedence: list Mailing-list: list isar-users@googlegroups.com; contact isar-users+owners@googlegroups.com List-ID: X-Spam-Checked-In-Group: isar-users@googlegroups.com X-Google-Group-Id: 914930254986 List-Post: , List-Help: , List-Archive: , List-Unsubscribe: , X-Spam-Status: No, score=-4.9 required=5.0 tests=DKIMWL_WL_MED,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,MAILING_LIST_MULTI, RCVD_IN_MSPIKE_H2,RCVD_IN_RP_CERTIFIED,RCVD_IN_RP_RNBL,RCVD_IN_RP_SAFE, SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on shymkent.ilbers.de X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= From: Jan Kiszka This bitbake fetcher allows to pull container images from registries, store them in the download cache and transfer them into the workdir of recipes requesting the image. The format of the URL is docker://[/];digest=sha256:...[;tag=] Fetching without digest is supported but will cause a warning, just like downloading via wget without a checksum. If tag is left out, "latest" is used. The fetcher will try to pull all available variants of a multi-arch image. If this is not needed, you can also directly specify the image digest of a specific architecture. Future versions may also introduce full unpacking of the fetched container layers in workdir if use case come up. Signed-off-by: Jan Kiszka --- meta/classes/dpkg-base.bbclass | 6 +++ meta/lib/container_fetcher.py | 90 ++++++++++++++++++++++++++++++++++ 2 files changed, 96 insertions(+) create mode 100644 meta/lib/container_fetcher.py diff --git a/meta/classes/dpkg-base.bbclass b/meta/classes/dpkg-base.bbclass index 789d6c74..d90b32a9 100644 --- a/meta/classes/dpkg-base.bbclass +++ b/meta/classes/dpkg-base.bbclass @@ -98,6 +98,12 @@ python() { if len(d.getVar('SRC_APT').strip()) > 0: bb.build.addtask('apt_unpack', 'do_patch', '', d) bb.build.addtask('cleanall_apt', 'do_cleanall', '', d) + + # container docker fetcher + import container_fetcher + from bb.fetch2 import methods + + methods.append(container_fetcher.Container()) } do_apt_fetch() { diff --git a/meta/lib/container_fetcher.py b/meta/lib/container_fetcher.py new file mode 100644 index 00000000..8c713795 --- /dev/null +++ b/meta/lib/container_fetcher.py @@ -0,0 +1,90 @@ +# This software is a part of ISAR. +# Copyright (c) Siemens AG, 2024 +# +# SPDX-License-Identifier: MIT + +import os +import re +from bb.fetch2 import FetchMethod +from bb.fetch2 import logger +from bb.fetch2 import MissingChecksumEvent +from bb.fetch2 import NoChecksumError +from bb.fetch2 import runfetchcmd +from bb.progress import LineFilterProgressHandler + +class SkopeoProgressHandler(LineFilterProgressHandler): + def __init__(self, d): + super(SkopeoProgressHandler, self).__init__(d) + self._fire_progress(0) + + def writeline(self, line): + match = re.findall(r'^Copying image .*\(([0-9]+/[0-9]+)\)$', line) + if match: + state = match[0].split('/') + progress = (int(state[0]) * 100) / int(state[1]) + self.update(progress) + return True + + +class Container(FetchMethod): + def supports(self, ud, d): + return ud.type in ['docker'] + + def urldata_init(self, ud, d): + ud.tag = "latest" + if "tag" in ud.parm: + ud.tag = ud.parm["tag"] + + ud.digest = None + if "digest" in ud.parm: + ud.digest = ud.parm["digest"] + + container_name = ud.host + (ud.path if ud.path != "/" else "") + ud.container_src = container_name + \ + ("@" + ud.digest if ud.digest else ":" + ud.tag) + ud.localname = container_name.replace('/', '.') + ud.localfile = "container-images/" + ud.localname + \ + "_" + (ud.digest.replace(":", "-") if ud.digest else ud.tag) + + def download(self, ud, d): + progresshandler = SkopeoProgressHandler(d) + runfetchcmd(f"skopeo copy --preserve-digests --all docker://{ud.container_src} dir:{ud.localfile}", + d, log=progresshandler) + + if ud.digest: + return + + checksum = bb.utils.sha256_file(ud.localpath + "/manifest.json") + checksum_line = f"SRC_URI = \"{ud.url};digest=sha256:{checksum}\"" + + strict = d.getVar("BB_STRICT_CHECKSUM") or "0" + + # If strict checking enabled and neither sum defined, raise error + if strict == "1": + raise NoChecksumError(checksum_line) + + checksum_event = {"sha256sum": checksum} + bb.event.fire(MissingChecksumEvent(ud.url, **checksum_event), d) + + if strict == "ignore": + return + + # Log missing digest so user can more easily add it + logger.warning( + f"Missing checksum for '{ud.localpath}', consider using this " \ + f"SRC_URI in the recipe:\n{checksum_line}") + + def unpack(self, ud, rootdir, d): + arch = d.getVar('DISTRO_ARCH') + variant_opt = "" + if arch == "armhf": + arch = "arm" + variant_opt = "--override-variant v7" + elif arch == "armel": + arch = "arm" + variant_opt = "--override-variant v6" + runfetchcmd(f"skopeo --override-arch {arch} {variant_opt} " \ + f"copy dir:{ud.localpath} dir:{rootdir + '/' + ud.localname}", d) + + def clean(self, ud, d): + bb.utils.remove(ud.localpath, recurse=True) From patchwork Tue Jul 9 17:31:38 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jan Kiszka X-Patchwork-Id: 3672 Return-Path: Received: from shymkent.ilbers.de ([unix socket]) by shymkent (Cyrus 2.5.10-Debian-2.5.10-3+deb9u2) with LMTPA; Tue, 09 Jul 2024 19:31:50 +0200 X-Sieve: CMU Sieve 2.4 Received: from mail-wm1-f57.google.com (mail-wm1-f57.google.com [209.85.128.57]) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPS id 469HVnLc006522 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Tue, 9 Jul 2024 19:31:49 +0200 Received: by mail-wm1-f57.google.com with SMTP id 5b1f17b1804b1-42725ef39e2sf4178925e9.0 for ; Tue, 09 Jul 2024 10:31:49 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1720546304; cv=pass; d=google.com; s=arc-20160816; b=wpiwVzkNpQYezsCZLkqHDOuoHRUMn6RtremqLM6y2+dNhXX1lkEMPyZBsN6NimTMOF FvfNmBQ128oniI5Cod7Qe8JpeYcGwdKSgA4XGGkoCtuKNrb7Ydz9736JJahOzPPO2o4Z KfU4h+2TG6RQl/2sCU21mmAPCmooykzvXGe+PVDrURvS+SmOFPXEmKXnXhxpsrWNscgP 1lrufylL4WFLcIu+r9k2pai8RetOKHExTO8vvi3bPLgTRQRaSObfxFgMiWb1tywcZCjM dyz2Mlc4q0lIrJhxvYqzGSQE3LvkIGxv2j8wYVpbKTR+0uliMP1MlcTwXhnDaUuQB093 OJ5w== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:reply-to:feedback-id:mime-version :references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=9KFqqe0+zUNuf2b3S+LuEbkWCkj4tHfVFrRH80sjhzs=; fh=Hcc+btNqJwyKxqPaL7+rDw/HEUNK1itHI9e7YqVOdlY=; b=vemFuBxqsRWDNjLAab5XLotRKQs9Or5/K1AOq7DUdpxAobG1q0aAroTz0VPawM4T/G TAef+TWq5eQHt8qyWE0us3jMVch6yOndgwCH3UEvT+sDwMzIhzW3VxQjA5zysJoCgLlv tJD81jukbgkLyNkGm7ln+FS7ee+aGdG+LdjfAXUEtAdklHsDnPrso3cEURdvtsQBbG4q 9yPEPPztcNdv+yX5W2qPa9WyOT0c7p9WHYjC2AYpjILIeBjXMzylUaPkw68ZHlJOkIZL OSsIk0fH/PG183Q1nvp+YreGJ9RpJx6jAHjwPtb8zjRzhOgXAv03BwIWiZPtz2dE/cmE XU5Q==; darn=isar-build.org ARC-Authentication-Results: i=2; gmr-mx.google.com; dkim=pass header.i=@siemens.com header.s=fm2 header.b=BxUVJ49q; spf=pass (google.com: domain of fm-294854-20240709173140ddaf302001721ef599-8lvdhi@rts-flowmailer.siemens.com designates 185.136.64.228 as permitted sender) smtp.mailfrom=fm-294854-20240709173140ddaf302001721ef599-8LvdHi@rts-flowmailer.siemens.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=siemens.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlegroups.com; s=20230601; t=1720546304; x=1721151104; darn=isar-build.org; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:reply-to :x-original-authentication-results:x-original-sender:feedback-id :mime-version:references:in-reply-to:message-id:date:subject:to:from :from:to:cc:subject:date:message-id:reply-to; bh=9KFqqe0+zUNuf2b3S+LuEbkWCkj4tHfVFrRH80sjhzs=; b=tVRl5DKuA1wS+VYDmaC8SI0e2OUgDt8rLnlCGfz/GKlggq6K778nyPbRA565n9mXDf UsNfhdkDZj6l8vy0ujCwDI80urvULZ0kYtdGx/SORzOSsupqD++glc8C+eiKgxoSjkG+ ITtD7NxTIgoIJr/twmbPrjUNMDZmOQ9hxL2f/MFPvabLCNxCV0+ywXh7a4gNw15VzjQt VkqRnV+mObDNSNSDPgcY65JYuwYJKVqrL6cb55WAVEjYLUD/FNbIcockSIjUR4kIFtER Vz8HcVTaln1Y8HTQTXUS6dvZsPhfBAGGxY2o0BRF4F4PwymUXPuAglMOCUH/IrmEx9/9 QVLg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1720546304; x=1721151104; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :x-spam-checked-in-group:list-id:mailing-list:precedence:reply-to :x-original-authentication-results:x-original-sender:feedback-id :mime-version:references:in-reply-to:message-id:date:subject:to:from :x-beenthere:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=9KFqqe0+zUNuf2b3S+LuEbkWCkj4tHfVFrRH80sjhzs=; b=lTy3zLJr2wZKanJAawCajI1XYbn0tK/1JlACaI6qn5t7xmGzhSI+5gxtZ5I/aW+rG+ BmFaM5oj4V2SWcbf5I2Jnsc5roq+2v9GNfnzCa+t7H3uFMOhWR/LF5ygkGd0VBGCSPo4 j4XL4LjxIeB/vxhnjiaWZDtd44ISWIsecCraVJcHTTiD+fBenJhVk9tl2aEG1/Iqs5SR fEICkEymiOd9lytQJebjNDA1WuUmaXhaO0s4MyPIphApYJ/ipDfZR1N/kwY8K/ZJDY6e QK2mqADH9boyGvqEpGGxMmu8SOXHHz1aq1DdD4PL33g4JxSAOM/qb2aXe6k4iZZN0An2 DocQ== X-Forwarded-Encrypted: i=2; AJvYcCWO63QwHxHhDTS0QzqDQzFMg/vy/ygMArwA9HaippKKhxcIEIEzqaF3SYp+rl+h4XqaeyqVkJ9oUEcZ6LRyEUbF0Dzgee8= X-Gm-Message-State: AOJu0YyezILJ/Bwq1WO9MwhP08k+QqsfSCSK+feWQwfdVIfRXXEmp5xH 2WS06XoTKmYLkGXJLn567yLCZjv/sG6t5+z5rEtuZ99sqIeUJzck X-Google-Smtp-Source: AGHT+IEqrfNvHf47zObvKMmE00TDrqxXlXjjSEudiBz2D49RPGGE50DE4+Kp8e1PgImprDSE9/ah3g== X-Received: by 2002:a05:600c:4ba2:b0:426:6ead:5709 with SMTP id 5b1f17b1804b1-426707cf736mr19336035e9.9.1720546303397; Tue, 09 Jul 2024 10:31:43 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a05:600c:1d0f:b0:427:9377:3cc4 with SMTP id 5b1f17b1804b1-42793774063ls94135e9.1.-pod-prod-06-eu; Tue, 09 Jul 2024 10:31:41 -0700 (PDT) X-Received: by 2002:a05:600c:4341:b0:424:ad14:6b79 with SMTP id 5b1f17b1804b1-426706c9159mr23700795e9.8.1720546301252; Tue, 09 Jul 2024 10:31:41 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1720546301; cv=none; d=google.com; s=arc-20160816; b=X3kRsZe08+yj2m3sBtOSLuq6/h4kMGaXEW4p/DnJ2P+ydDuPLyh+RFieEeh9WoJLx9 o/IfGEIuCx1YI76XC/l48p0pOd2DYe3RD0hVCZBzsZ1jvXK/7ruMS/hYS7NA/6XMfhxT q9zqsZG1Ps4HZT3uzuWw3A0gLNnwoWoHhHyRsoODOiqZnVlBz0WJOwhFnKwX683sF+/D 52cQzr/Sp4hSEuYfU/MrCwiomHIO+KesC7Xh73cBsglbnTjVXOy7jBcI4DkjyN02B5C8 6zdMvH67urCUCCvtQ1JzTbg2Ee6V7Po4BjgUHJ5f6OeKhoc3aM6WV/k2OX30rlNWcfW7 ccWQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=feedback-id:content-transfer-encoding:mime-version:references :in-reply-to:message-id:date:subject:to:from:dkim-signature; bh=L8JXHy4gkDOvNTnuYGp+qJaSa8/+nONEOLnUEdxdg5k=; fh=hOBXq6nisExkIRIwN8IDOmOCLKwX8PM0ANlxQbNMEqA=; b=E6KNMnoCk5v+UzF2/oao2KvYbQkKCxy4wuuBGMT/55O2fnfOvMLsxdv09ZtfktZMsz sj2o1/KYykRlkz5TFhF/I4gWfhKIfBMxLHmBrFUEmLptKXgjdbthcEJugNqjZPQ6TIRN d7SyrNoCZwApcQ5ntQ8bVLBw682O/h87cK6cfCMHUKLj3XS7RYj6oF5RSyeAfJZ92vPo ZfpO8FZrheewZ+9j/wc0pa3wHZ/Zbt4W+GPXv3Tk4qA6UrANKNo2BEc1rc72btK6sYaj /GcXkhYpRMjXisw86pOhzETs+WRJ2Mbr13sxZklo8gf4zxtUtiu6i/1a+4sjOAgzNNk9 53iw==; dara=google.com ARC-Authentication-Results: i=1; gmr-mx.google.com; dkim=pass header.i=@siemens.com header.s=fm2 header.b=BxUVJ49q; spf=pass (google.com: domain of fm-294854-20240709173140ddaf302001721ef599-8lvdhi@rts-flowmailer.siemens.com designates 185.136.64.228 as permitted sender) smtp.mailfrom=fm-294854-20240709173140ddaf302001721ef599-8LvdHi@rts-flowmailer.siemens.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=siemens.com Received: from mta-64-228.siemens.flowmailer.net (mta-64-228.siemens.flowmailer.net. [185.136.64.228]) by gmr-mx.google.com with ESMTPS id 5b1f17b1804b1-426725584d2si626425e9.0.2024.07.09.10.31.41 for (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 09 Jul 2024 10:31:41 -0700 (PDT) Received-SPF: pass (google.com: domain of fm-294854-20240709173140ddaf302001721ef599-8lvdhi@rts-flowmailer.siemens.com designates 185.136.64.228 as permitted sender) client-ip=185.136.64.228; Received: by mta-64-228.siemens.flowmailer.net with ESMTPSA id 20240709173140ddaf302001721ef599 for ; Tue, 09 Jul 2024 19:31:40 +0200 X-Patchwork-Original-From: "'Jan Kiszka' via isar-users" From: Jan Kiszka To: isar-users Subject: [RFC][PATCH 2/3] container-loader: Introduce helper to load container images into local registry Date: Tue, 9 Jul 2024 19:31:38 +0200 Message-ID: In-Reply-To: References: MIME-Version: 1.0 X-Flowmailer-Platform: Siemens Feedback-ID: 519:519-294854:519-21489:flowmailer X-Original-Sender: jan.kiszka@siemens.com X-Original-Authentication-Results: gmr-mx.google.com; dkim=pass header.i=@siemens.com header.s=fm2 header.b=BxUVJ49q; spf=pass (google.com: domain of fm-294854-20240709173140ddaf302001721ef599-8lvdhi@rts-flowmailer.siemens.com designates 185.136.64.228 as permitted sender) smtp.mailfrom=fm-294854-20240709173140ddaf302001721ef599-8LvdHi@rts-flowmailer.siemens.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=siemens.com X-Original-From: Jan Kiszka Reply-To: Jan Kiszka Precedence: list Mailing-list: list isar-users@googlegroups.com; contact isar-users+owners@googlegroups.com List-ID: X-Spam-Checked-In-Group: isar-users@googlegroups.com X-Google-Group-Id: 914930254986 List-Post: , List-Help: , List-Archive: , List-Unsubscribe: , X-Spam-Status: No, score=-4.9 required=5.0 tests=DKIMWL_WL_MED,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,MAILING_LIST_MULTI, RCVD_IN_MSPIKE_H2,RCVD_IN_RP_CERTIFIED,RCVD_IN_RP_RNBL,RCVD_IN_RP_SAFE, SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on shymkent.ilbers.de X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= From: Jan Kiszka This allows to write dpkg-raw recipes which packages archived container images and load them into a local docker or podman registry on boot. The scenario behind this is to pre-fill local registries in a way that still permits live updates during runtime. The loader script only process images which are not yet available under the same name and tag in the local registry. Also after loading, the archived images stay on the local file system. This allows to perform reloading in case the local registry should be emptied (e.g. reset to factory state). To reduce the space those original images need, they are compressed, by default with xz. Separate include files are available to cater the main container engines, one for docker and one for podman. Signed-off-by: Jan Kiszka --- .../container-loader/container-loader.inc | 76 +++++++++++++++++++ .../container-loader/docker-loader.inc | 10 +++ .../files/container-loader.service.tmpl | 11 +++ .../files/container-loader.sh.tmpl | 13 ++++ .../container-loader/podman-loader.inc | 10 +++ 5 files changed, 120 insertions(+) create mode 100644 meta/recipes-support/container-loader/container-loader.inc create mode 100644 meta/recipes-support/container-loader/docker-loader.inc create mode 100644 meta/recipes-support/container-loader/files/container-loader.service.tmpl create mode 100755 meta/recipes-support/container-loader/files/container-loader.sh.tmpl create mode 100644 meta/recipes-support/container-loader/podman-loader.inc diff --git a/meta/recipes-support/container-loader/container-loader.inc b/meta/recipes-support/container-loader/container-loader.inc new file mode 100644 index 00000000..8e352214 --- /dev/null +++ b/meta/recipes-support/container-loader/container-loader.inc @@ -0,0 +1,76 @@ +# This software is a part of ISAR. +# Copyright (c) Siemens AG, 2024 +# +# SPDX-License-Identifier: MIT + +FILESPATH:append := ":${FILE_DIRNAME}/files" + +inherit dpkg-raw + +SRC_URI += " \ + file://container-loader.service.tmpl \ + file://container-loader.sh.tmpl" + +TEMPLATE_FILES += " \ + container-loader.service.tmpl \ + container-loader.sh.tmpl" +TEMPLATE_VARS += "CONTAINER_ENGINE" + +CONTAINER_COMPRESSION ?= "xz" + +DEBIAN_DEPENDS += " \ + ${CONTAINER_ENGINE_PACKAGES} \ + ${@', xz-utils' if d.getVar('CONTAINER_COMPRESSION') == 'xz' else \ + ', gzip' if d.getVar('CONTAINER_COMPRESSION') == 'gz' else \ + ''}" + +CONTAINER_COMPRESSOR = "${@ \ + 'xz' if d.getVar('CONTAINER_COMPRESSION') == 'xz' else \ + 'gzip' if d.getVar('CONTAINER_COMPRESSION') == 'gz' else \ + ''}" + +python do_install() { + import os + + workdir = d.getVar('WORKDIR') + D = d.getVar('D') + PN= d.getVar('PN') + + image_list = open(D + "/usr/share/" + PN +"/image.list", "w") + + src_uri = d.getVar('SRC_URI').split() + for uri in src_uri: + scheme, host, path, _, _, parm = bb.fetch.decodeurl(uri) + if scheme != "docker": + continue + + image_name = host + (path if path != "/" else "") + unpacked_image = workdir + "/" + image_name.replace('/', '.') + dest_dir = D + "/usr/share/" + PN + "/images" + tar_image = dest_dir + "/" + image_name.replace('/', '.') + ".tar" + docker_ref = ":" + parm["tag"] if "tag" in parm else "latest" + + cmd = f"skopeo copy dir:{unpacked_image} " \ + f"docker-archive:{tar_image}:{image_name}{docker_ref}" + bb.note(f"running: {cmd}") + bb.process.run(cmd) + + cmd = f"{d.getVar('CONTAINER_COMPRESSOR')} {tar_image}" + bb.note(f"running: {cmd}") + bb.process.run(cmd) + + line = f"{os.path.basename(tar_image)}.{d.getVar('CONTAINER_COMPRESSION')} " + \ + image_name + docker_ref + bb.note(f"adding '{line}' to image.list") + image_list.write(line + "\n") + + image_list.close() + + bb.utils.copyfile(workdir + "/container-loader.sh", + D + "/usr/share/" + PN + "/container-loader.sh") +} +do_install[cleandirs] += "${D}/usr/share/${PN}/images" + +do_prepare_build:append() { + install -v -m 644 ${WORKDIR}/container-loader.service ${S}/debian/${PN}.service +} diff --git a/meta/recipes-support/container-loader/docker-loader.inc b/meta/recipes-support/container-loader/docker-loader.inc new file mode 100644 index 00000000..b864c854 --- /dev/null +++ b/meta/recipes-support/container-loader/docker-loader.inc @@ -0,0 +1,10 @@ +# This software is a part of ISAR. +# Copyright (c) Siemens AG, 2024 +# +# SPDX-License-Identifier: MIT + +require container-loader.inc + +CONTAINER_ENGINE = "docker" + +CONTAINER_ENGINE_PACKAGES ?= "docker.io, apparmor" diff --git a/meta/recipes-support/container-loader/files/container-loader.service.tmpl b/meta/recipes-support/container-loader/files/container-loader.service.tmpl new file mode 100644 index 00000000..afde55d3 --- /dev/null +++ b/meta/recipes-support/container-loader/files/container-loader.service.tmpl @@ -0,0 +1,11 @@ +[Unit] +Description=Load archived container images on boot +After=${CONTAINER_ENGINE}.service + +[Service] +Type=oneshot +ExecStart=/usr/share/${PN}/container-loader.sh +RemainAfterExit=true + +[Install] +WantedBy=multi-user.target diff --git a/meta/recipes-support/container-loader/files/container-loader.sh.tmpl b/meta/recipes-support/container-loader/files/container-loader.sh.tmpl new file mode 100755 index 00000000..31d27865 --- /dev/null +++ b/meta/recipes-support/container-loader/files/container-loader.sh.tmpl @@ -0,0 +1,13 @@ +#!/bin/sh +# +# Copyright (c) Siemens AG, 2024 +# +# SPDX-License-Identifier: MIT + +set -eu + +while read -r image ref; do + if [ -z "$(${CONTAINER_ENGINE} images -q "$ref")" ]; then + ${CONTAINER_ENGINE} load -i /usr/share/${PN}/images/"$image" + fi +done < /usr/share/${PN}/image.list diff --git a/meta/recipes-support/container-loader/podman-loader.inc b/meta/recipes-support/container-loader/podman-loader.inc new file mode 100644 index 00000000..d2c9a12d --- /dev/null +++ b/meta/recipes-support/container-loader/podman-loader.inc @@ -0,0 +1,10 @@ +# This software is a part of ISAR. +# Copyright (c) Siemens AG, 2024 +# +# SPDX-License-Identifier: MIT + +require container-loader.inc + +CONTAINER_ENGINE = "podman" + +CONTAINER_ENGINE_PACKAGES ?= "podman" From patchwork Tue Jul 9 17:31:39 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jan Kiszka X-Patchwork-Id: 3674 Return-Path: Received: from shymkent.ilbers.de ([unix socket]) by shymkent (Cyrus 2.5.10-Debian-2.5.10-3+deb9u2) with LMTPA; Tue, 09 Jul 2024 19:31:52 +0200 X-Sieve: CMU Sieve 2.4 Received: from mail-lj1-f190.google.com (mail-lj1-f190.google.com [209.85.208.190]) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPS id 469HVp6s006541 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Tue, 9 Jul 2024 19:31:51 +0200 Received: by mail-lj1-f190.google.com with SMTP id 38308e7fff4ca-2ee94b0e2e1sf49617101fa.2 for ; Tue, 09 Jul 2024 10:31:51 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1720546304; cv=pass; d=google.com; s=arc-20160816; b=nXJp+XF/mcVSydV13qEx23NgiNkfkz+mLPbCvw923m2rswmZhP/lHy1/RTNpwSGgMJ nS450TwCQR7Shnju6HhN8mkNq4TGZloxgh+VYqvv3OV8sx/CeCIDelIDmh5FaRAZQcQi 8H2aCUsu6P69HkfMsUaCI4ITHzElF1b2OxcC2D3lTD8sPEX91nT2Nj/M/Enp8tWVe7eR C8jZn8CEDszGIa3rwsK7xnTlsEzVdJmYfSNm4vgJVHOA/8FKtlMYoq2lkPm3a/InAcGc WAmCfWnS2GHTHGegs7LQZ4BRkVsfafIWP0cPnNeOQAdm7sJfOzcxu047pSe7OEJNcuEY KQmw== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:reply-to:feedback-id:mime-version :references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=Id5SXKvhjtOcsoeF9UxkAvXiD+xR/HiJiARDkzXdbfk=; fh=1xYbwTbtEtQLqoyd2YjetyaPkUXBf054iV62kdctTnc=; b=bxBPiVdOj6kah935UzkDM+jnv311y1oDJoiM2LIZt0fCjwGaAHpRsdjZV7GvwKRGgX qnCt8l2/hThjty5zyc3U7Kie3ACnHM2pVqI+nkPpc2Hi9RGErs+WUQm6c6zI3EN30hoZ jsSOmOq920V1VAuRzpejPleJun2flAs+3qrtREIy184hN8Q/IAQqNo3O5m8EPENn80bK G333ltHnC9yY24QP7OOwnzV8vIPqz7Ea6dTr0E9gAqcoE3YwXYYDzN89QrjLug24s6e9 uN9PSnbNav0nloycXTERB+zzqHrhPHJHfRNsgL+KZt+dPX1zVnuqKnBi5Uwemcpbd5jR 7xHg==; darn=isar-build.org ARC-Authentication-Results: i=2; gmr-mx.google.com; dkim=pass header.i=@siemens.com header.s=fm2 header.b=PSCyfgNq; spf=pass (google.com: domain of fm-294854-20240709173140a1e2c3fb75b7f274d9-hc2wsk@rts-flowmailer.siemens.com designates 185.136.64.225 as permitted sender) smtp.mailfrom=fm-294854-20240709173140a1e2c3fb75b7f274d9-Hc2wSK@rts-flowmailer.siemens.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=siemens.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlegroups.com; s=20230601; t=1720546304; x=1721151104; darn=isar-build.org; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:reply-to :x-original-authentication-results:x-original-sender:feedback-id :mime-version:references:in-reply-to:message-id:date:subject:to:from :from:to:cc:subject:date:message-id:reply-to; bh=Id5SXKvhjtOcsoeF9UxkAvXiD+xR/HiJiARDkzXdbfk=; b=X5+cRrCdssU8VLOPlRN0pr5THG3Jv0DGqzF+1L1vR9QwD7BfP/L6k3EQFNE91OB2ad I4ONwvP0W0pgNvAS3dof+dsgvYwqs7TmlyWY+rdsEHB4GpAywkachjonqLHkhdnmFiRn F4XTuTmZt1NVWUKQRYsgyy+Zq3MCbqh28LnS1epkN9mzGioxuX+D5hFw/bUN3cZOiH37 Yc2EtkIMh2qWfW4xIyQeSIiwQR6DFUrRwPbPPR9C2SzFyM1piID6r5b+jO1G7XWrD+Rl ukrQyqOxxikQlpmL+2/BzEbLSNPzhQVXzGf1MLvt+QFMIzEkMS2Fd3okDtpSuyAEb3t+ 2a5Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1720546304; x=1721151104; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :x-spam-checked-in-group:list-id:mailing-list:precedence:reply-to :x-original-authentication-results:x-original-sender:feedback-id :mime-version:references:in-reply-to:message-id:date:subject:to:from :x-beenthere:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=Id5SXKvhjtOcsoeF9UxkAvXiD+xR/HiJiARDkzXdbfk=; b=Okq8JmWMYICkUb0eA4R6TK+UjZK6ZYLcY0NXq6xuDa9X0YeUiOzcUAVUHvc7WMnV19 k87oSdAzlFZajGbgVOnoGGOavu+9iJEW5+Vg01yGuPlq8atWnB9ZpqVdVarGwrXVrhvn mM446LEj7cm+RvgBT/IFoX2FGFyb8r9znXJwPOONStNgtGsno6gqxbhX0RpdMzTNVzW5 u9r8OMz2wuW4h4opdIx+EvgnNIQPdas4y80jlhYCUdqssQJiHl58hLyQlcqfWWTmInSv IrFMppLyzKURDkHAB7HsYm2jJd3Mqs3xBlYbbpiND+I8dE8BdBhyOdlZO+nxerJorc9B 6zjg== X-Forwarded-Encrypted: i=2; AJvYcCX006fd1cS4oY0duDIUpWraQ/Qqy2rHdMdPpS5BqukdPjvLj1BPpAu/nI946I60EKGQrugAA9UNm8YywxeyfsM97dzyrko= X-Gm-Message-State: AOJu0Yxomqw4mzOXv8mIMlGERrahd8GwgDNTDa9fhemeSvxiB4ysET5B wBhe4yY7fYxn/9eNC/xJdXJd8ESPNfMQqewqxoiXzZw4QWAEog5i X-Google-Smtp-Source: AGHT+IE4uG7PajlI/AghRkLCJd3kWK5OY1kOORIZuCpaZgiiL8WJ2Q7ydcxIg1X4SF/2TVnX8naf7Q== X-Received: by 2002:a2e:a415:0:b0:2ee:8ce9:3075 with SMTP id 38308e7fff4ca-2eeb3197a5fmr19240371fa.49.1720546304079; Tue, 09 Jul 2024 10:31:44 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a05:651c:305:b0:2ee:8918:7088 with SMTP id 38308e7fff4ca-2ee8cdf2a69ls23906151fa.1.-pod-prod-01-eu; Tue, 09 Jul 2024 10:31:42 -0700 (PDT) X-Received: by 2002:a2e:b059:0:b0:2ec:53fb:39cb with SMTP id 38308e7fff4ca-2eeb30ba0edmr21915551fa.6.1720546301659; Tue, 09 Jul 2024 10:31:41 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1720546301; cv=none; d=google.com; s=arc-20160816; b=vXxKLPgsZUU/871Ug75q6hdtwo1aXCGd3M4rkR+XI10kKuu/nYJKoO3TEVQroAnXSV 371wfQKdFLrfQcZBh7AVIogKqjk3Lb3u/W3kmK1TPiq/Vg8bbxhVGZtT0pEX0FGg0VUg PZbAWxZ3JcX1UFy1k29MpGi0+2sVdjrxCPyhA4HWwPR5QXlsBVn32M4kZ1CLNocLLOUv mgyMBQg9n33W/NqS9/3jAWWAdNLIIj5LQiZl03Dbt9S42R0ih2uL/3Y2KRkfw7EhopQT QPPg8zI/a+0qCqvv7in+IlCXGoxWLLGZEX4d5jlBWuImIDYHm6QbQ5Fyn7IAXxRMJt6I obWA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=feedback-id:content-transfer-encoding:mime-version:references :in-reply-to:message-id:date:subject:to:from:dkim-signature; bh=BA20LfX5WBzZRAGQkD2H3FyvqlNHsjhg+FxhPrEIsuw=; fh=hOBXq6nisExkIRIwN8IDOmOCLKwX8PM0ANlxQbNMEqA=; b=rukpYMhaDkGtgox9c2/9TmdYTnIzeY4GRzfWpEJp1MVAzOcYYzO++VqP7u1fVTsemR FqT3vFR1FfWfnuZJUvSnzdz2FUb6a4HEcMnLR83YBDFSs+2BcR+KEXQotNGUDOD1ELN6 S1rYtpWZN54D4gm5jTW20HxfHSkhIHN6bQh4X5+aMSRHL/9nHQRyylrCcsw5+Fe9/DXS i1Z8A8LLznUSrz+xU8rEheFWdG2MjFoeJgLvOOMWrPYv25C2fTSf+bA5co39ljQ5k/E9 FhC/AhX8UT50qoN0ZRCHpn54uMi0DDk7zV1rHvPVKSlKCtkOi5cs8CDi99nIJ1eTHg66 gmMg==; dara=google.com ARC-Authentication-Results: i=1; gmr-mx.google.com; dkim=pass header.i=@siemens.com header.s=fm2 header.b=PSCyfgNq; spf=pass (google.com: domain of fm-294854-20240709173140a1e2c3fb75b7f274d9-hc2wsk@rts-flowmailer.siemens.com designates 185.136.64.225 as permitted sender) smtp.mailfrom=fm-294854-20240709173140a1e2c3fb75b7f274d9-Hc2wSK@rts-flowmailer.siemens.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=siemens.com Received: from mta-64-225.siemens.flowmailer.net (mta-64-225.siemens.flowmailer.net. [185.136.64.225]) by gmr-mx.google.com with ESMTPS id 4fb4d7f45d1cf-594bfe70f1fsi60773a12.5.2024.07.09.10.31.41 for (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 09 Jul 2024 10:31:41 -0700 (PDT) Received-SPF: pass (google.com: domain of fm-294854-20240709173140a1e2c3fb75b7f274d9-hc2wsk@rts-flowmailer.siemens.com designates 185.136.64.225 as permitted sender) client-ip=185.136.64.225; Received: by mta-64-225.siemens.flowmailer.net with ESMTPSA id 20240709173140a1e2c3fb75b7f274d9 for ; Tue, 09 Jul 2024 19:31:41 +0200 X-Patchwork-Original-From: "'Jan Kiszka' via isar-users" From: Jan Kiszka To: isar-users Subject: [RFC][PATCH 3/3] meta-isar: Add demo packages for installing prebuilt containers Date: Tue, 9 Jul 2024 19:31:39 +0200 Message-ID: In-Reply-To: References: MIME-Version: 1.0 X-Flowmailer-Platform: Siemens Feedback-ID: 519:519-294854:519-21489:flowmailer X-Original-Sender: jan.kiszka@siemens.com X-Original-Authentication-Results: gmr-mx.google.com; dkim=pass header.i=@siemens.com header.s=fm2 header.b=PSCyfgNq; spf=pass (google.com: domain of fm-294854-20240709173140a1e2c3fb75b7f274d9-hc2wsk@rts-flowmailer.siemens.com designates 185.136.64.225 as permitted sender) smtp.mailfrom=fm-294854-20240709173140a1e2c3fb75b7f274d9-Hc2wSK@rts-flowmailer.siemens.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=siemens.com X-Original-From: Jan Kiszka Reply-To: Jan Kiszka Precedence: list Mailing-list: list isar-users@googlegroups.com; contact isar-users+owners@googlegroups.com List-ID: X-Spam-Checked-In-Group: isar-users@googlegroups.com X-Google-Group-Id: 914930254986 List-Post: , List-Help: , List-Archive: , List-Unsubscribe: , X-Spam-Status: No, score=-4.9 required=5.0 tests=DKIMWL_WL_MED,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,MAILING_LIST_MULTI, RCVD_IN_MSPIKE_H2,RCVD_IN_RP_CERTIFIED,RCVD_IN_RP_RNBL,RCVD_IN_RP_SAFE, SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on shymkent.ilbers.de X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= From: Jan Kiszka One recipe for docker, one for podman. Both pull from a registry that, in contrast to infamous dockerhub, should not throttle CI jobs running these frequently for testing purposes. The podman variant of the recipe is intentionally leaving out the digest to trigger the related warning of the container fetcher. These demos also come with kas integration. Signed-off-by: Jan Kiszka --- kas/package/Kconfig | 19 +++++++++++++++++++ kas/package/pkg_prebuilt-docker-img.yaml | 9 +++++++++ kas/package/pkg_prebuilt-podman-img.yaml | 9 +++++++++ .../prebuilt-docker-img_0.1.bb | 10 ++++++++++ .../prebuilt-podman-img_0.1.bb | 10 ++++++++++ 5 files changed, 57 insertions(+) create mode 100644 kas/package/pkg_prebuilt-docker-img.yaml create mode 100644 kas/package/pkg_prebuilt-podman-img.yaml create mode 100644 meta-isar/recipes-app/prebuilt-container/prebuilt-docker-img_0.1.bb create mode 100644 meta-isar/recipes-app/prebuilt-container/prebuilt-podman-img_0.1.bb diff --git a/kas/package/Kconfig b/kas/package/Kconfig index 35ba7cf1..395c3a08 100644 --- a/kas/package/Kconfig +++ b/kas/package/Kconfig @@ -146,6 +146,25 @@ config KAS_INCLUDE_PACKAGE_ISAR_CI_SSH_SETUP default "kas/package/pkg_sshd-regen-keys.yaml" depends on PACKAGE_ISAR_CI_SSH_SETUP + +config PACKAGE_PREBUILT_DOCKER_IMG + bool "prebuilt-docker-img" + default y + +config KAS_INCLUDE_PACKAGE_PREBUILT_DOCKER_IMG + string + default "kas/package/pkg_prebuilt-docker-img.yaml" + depends on PACKAGE_PREBUILT_DOCKER_IMG + +config PACKAGE_PREBUILT_PODMAN_IMG + bool "prebuilt-podman-img" + default y + +config KAS_INCLUDE_PACKAGE_PREBUILT_PODMAN_IMG + string + default "kas/package/pkg_prebuilt-podman-img.yaml" + depends on PACKAGE_PREBUILT_PODMAN_IMG + endmenu config KAS_IMAGE_PREINSTALL diff --git a/kas/package/pkg_prebuilt-docker-img.yaml b/kas/package/pkg_prebuilt-docker-img.yaml new file mode 100644 index 00000000..df96a484 --- /dev/null +++ b/kas/package/pkg_prebuilt-docker-img.yaml @@ -0,0 +1,9 @@ +# This software is a part of ISAR. +# Copyright (c) Siemens AG, 2024 + +header: + version: 14 + +local_conf_header: + package-prebuilt-docker-img: | + IMAGE_INSTALL:append = " prebuilt-docker-img" diff --git a/kas/package/pkg_prebuilt-podman-img.yaml b/kas/package/pkg_prebuilt-podman-img.yaml new file mode 100644 index 00000000..d0b8da1c --- /dev/null +++ b/kas/package/pkg_prebuilt-podman-img.yaml @@ -0,0 +1,9 @@ +# This software is a part of ISAR. +# Copyright (c) Siemens AG, 2024 + +header: + version: 14 + +local_conf_header: + package-prebuilt-podman-img: | + IMAGE_INSTALL:append = " prebuilt-podman-img" diff --git a/meta-isar/recipes-app/prebuilt-container/prebuilt-docker-img_0.1.bb b/meta-isar/recipes-app/prebuilt-container/prebuilt-docker-img_0.1.bb new file mode 100644 index 00000000..3f337d92 --- /dev/null +++ b/meta-isar/recipes-app/prebuilt-container/prebuilt-docker-img_0.1.bb @@ -0,0 +1,10 @@ +# This software is a part of ISAR. +# Copyright (c) Siemens AG, 2024 +# +# SPDX-License-Identifier: MIT + +require recipes-support/container-loader/docker-loader.inc + +SRC_URI += "\ + docker://quay.io/libpod/alpine;digest=sha256:fa93b01658e3a5a1686dc3ae55f170d8de487006fb53a28efcd12ab0710a2e5f;tag=3.10.2 \ + " diff --git a/meta-isar/recipes-app/prebuilt-container/prebuilt-podman-img_0.1.bb b/meta-isar/recipes-app/prebuilt-container/prebuilt-podman-img_0.1.bb new file mode 100644 index 00000000..e671a494 --- /dev/null +++ b/meta-isar/recipes-app/prebuilt-container/prebuilt-podman-img_0.1.bb @@ -0,0 +1,10 @@ +# This software is a part of ISAR. +# Copyright (c) Siemens AG, 2024 +# +# SPDX-License-Identifier: MIT + +require recipes-support/container-loader/podman-loader.inc + +SRC_URI += "\ + docker://quay.io/libpod/alpine;tag=latest \ + "