| Message ID | 20241007122709.215901-1-alexander.heinisch@siemens.com |
|---|---|
| State | Accepted, archived |
| Headers | show |
| Series | [v3] meta: Allow to specify the snapshot date in the upstream format. | expand |
On 07.10.24 14:27, alexander.heinisch@siemens.com wrote: > From: Alexander Heinisch <alexander.heinisch@siemens.com> > > Currently, it is only possible to specify a unix timestamp to resolve > the snapshot version used. Since this is not very user friendly, most > downstream projects put comments like `# 20240702T082400Z` next to the > snapshot assignment `ISAR_APT_SNAPSHOT_TIMESTAMP = ...`. > > This patch makes it easier for users to specify the snapshot version > in a more natural way by `ISAR_APT_SNAPSHOT_DATE = 20240702T082400Z` > > Signed-off-by: Alexander Heinisch <alexander.heinisch@siemens.com> > --- > v2: Added documentation > v3: Renamed APT_SNAPSHOT_DATE to ISAR_APT_SNAPSHOT_DATE > > doc/user_manual.md | 3 ++- > kas/opt/Kconfig | 2 +- > meta-isar/conf/distro/ubuntu-common.inc | 2 +- > meta/conf/distro/debian-common.conf | 2 +- > meta/recipes-core/isar-bootstrap/isar-bootstrap.inc | 4 ++-- > 5 files changed, 7 insertions(+), 6 deletions(-) > > diff --git a/doc/user_manual.md b/doc/user_manual.md > index 706c74bf..cf3c3b12 100644 > --- a/doc/user_manual.md > +++ b/doc/user_manual.md > @@ -435,7 +435,8 @@ Some other variables include: > - `ISAR_USE_APT_SNAPSHOT` - Use a frozen apt snapshot instead of the live mirror. Optional. > `ISAR_APT_DL_LIMIT` - Rate limit the apt fetching to n kB / s. Optional. > - `DISTRO_APT_SNAPSHOT_PREMIRROR` - Similar to `DISTRO_APT_PREMIRRORS` but for a snapshot, pre-defined for supported distros. > - - `ISAR_APT_SNAPSHOT_TIMESTAMP` - Timestamp of the apt snapshot. Automatically derived from `SOURCE_DATE_EPOCH` if not overwritten. > + - `ISAR_APT_SNAPSHOT_TIMESTAMP` - Unix timestamp of the apt snapshot. Automatically derived from `SOURCE_DATE_EPOCH` if not overwritten. (Consider `ISAR_APT_SNAPSHOT_DATE` for a more user friendly format) > + - `ISAR_APT_SNAPSHOT_DATE` - Timestamp in upstream format (e.g. `20240702T082400Z`) of the apt snapshot. Overrides `ISAR_APT_SNAPSHOT_TIMESTAMP` if set. Otherwise, will be automatically derived from `ISAR_APT_SNAPSHOT_TIMESTAMP` > - `THIRD_PARTY_APT_KEYS` - List of gpg key URIs used to verify apt repos for apt installation after bootstrapping. > - `FILESEXTRAPATHS` - The default directories BitBake uses when it processes recipes are initially defined by the FILESPATH variable. You can extend FILESPATH variable by using FILESEXTRAPATHS. > - `FILESOVERRIDES` - A subset of OVERRIDES used by the build system for creating FILESPATH. The FILESOVERRIDES variable uses overrides to automatically extend the FILESPATH variable. > diff --git a/kas/opt/Kconfig b/kas/opt/Kconfig > index 65482e3a..1e56c29f 100644 > --- a/kas/opt/Kconfig > +++ b/kas/opt/Kconfig > @@ -44,7 +44,7 @@ config USE_APT_SNAPSHOT > bool "Use the distros snapshot mirror" > depends on !MIRROR_DEBIAN && (DEBIAN_BUSTER || DEBIAN_BULLSEYE || DEBIAN_BOOKWORM || DEBIAN_TRIXIE || DEBIAN_SID || UBUNTU_FOCAL || UBUNTU_JAMMY) > help > - Use a snapshot mirror for the selected distribution. The date is defined by ISAR_APT_SNAPSHOT_TIMESTAMP. > + Use a snapshot mirror for the selected distribution. The date is defined by ISAR_APT_SNAPSHOT_DATE. > > config KAS_INCLUDE_APT_SNAPSHOT > string > diff --git a/meta-isar/conf/distro/ubuntu-common.inc b/meta-isar/conf/distro/ubuntu-common.inc > index 232bee35..be7fd047 100644 > --- a/meta-isar/conf/distro/ubuntu-common.inc > +++ b/meta-isar/conf/distro/ubuntu-common.inc > @@ -37,4 +37,4 @@ SYSTEMD_BOOTLOADER_INSTALL:focal = "systemd:${DISTRO_ARCH}" > SYSTEMD_BOOTLOADER_INSTALL:jammy = "systemd:${DISTRO_ARCH}" > > # snapshot mirror for reproducible builds > -DISTRO_APT_SNAPSHOT_PREMIRROR ??= "(http|https)://archive.ubuntu.com/(.*) https://snapshot.ubuntu.com/\2/${APT_SNAPSHOT_DATE}/\n" > +DISTRO_APT_SNAPSHOT_PREMIRROR ??= "(http|https)://archive.ubuntu.com/(.*) https://snapshot.ubuntu.com/\2/${ISAR_APT_SNAPSHOT_DATE}/\n" > diff --git a/meta/conf/distro/debian-common.conf b/meta/conf/distro/debian-common.conf > index 8c6a3f63..65b5b083 100644 > --- a/meta/conf/distro/debian-common.conf > +++ b/meta/conf/distro/debian-common.conf > @@ -40,4 +40,4 @@ COMPAT_DISTRO_ARCH:amd64 = "i386" > COMPAT_DISTRO_ARCH:arm64 = "armhf" > > # snapshot mirror for reproducible builds > -DISTRO_APT_SNAPSHOT_PREMIRROR ??= "deb.debian.org/(.*) snapshot.debian.org/archive/\1/${APT_SNAPSHOT_DATE}/\n" > +DISTRO_APT_SNAPSHOT_PREMIRROR ??= "deb.debian.org/(.*) snapshot.debian.org/archive/\1/${ISAR_APT_SNAPSHOT_DATE}/\n" > diff --git a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc > index 12f32ff0..92974eb5 100644 > --- a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc > +++ b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc > @@ -34,7 +34,7 @@ DISTRO_VARS_PREFIX ?= "${@'HOST_' if bb.utils.to_boolean(d.getVar('BOOTSTRAP_FOR > BOOTSTRAP_DISTRO = "${@d.getVar('HOST_DISTRO' if bb.utils.to_boolean(d.getVar('BOOTSTRAP_FOR_HOST')) else 'DISTRO')}" > BOOTSTRAP_BASE_DISTRO = "${@d.getVar('HOST_BASE_DISTRO' if bb.utils.to_boolean(d.getVar('BOOTSTRAP_FOR_HOST')) else 'BASE_DISTRO')}" > FILESEXTRAPATHS:append = ":${BBPATH}" > -APT_SNAPSHOT_DATE = "${@ get_apt_snapshot_date(d)}" > +ISAR_APT_SNAPSHOT_DATE ?= "${@ get_isar_apt_snapshot_date(d)}" > > inherit deb-dl-dir > > @@ -110,7 +110,7 @@ def parse_aptsources_list_line(source_list_line): > > return [type, options, source, suite, components] > > -def get_apt_snapshot_date(d): > +def get_isar_apt_snapshot_date(d): > import time > source_date_epoch = d.getVar('ISAR_APT_SNAPSHOT_TIMESTAMP') > return time.strftime('%Y%m%dT%H%M%SZ', time.gmtime(int(source_date_epoch))) Reviewed-by: Jan Kiszka <jan.kiszka@siemens.com> Thanks, Jan
On Mon, 2024-10-07 at 14:27 +0200, alexander.heinisch via isar-users wrote: > From: Alexander Heinisch <alexander.heinisch@siemens.com> > > Currently, it is only possible to specify a unix timestamp to resolve > the snapshot version used. Since this is not very user friendly, most > downstream projects put comments like `# 20240702T082400Z` next to > the > snapshot assignment `ISAR_APT_SNAPSHOT_TIMESTAMP = ...`. > > This patch makes it easier for users to specify the snapshot version > in a more natural way by `ISAR_APT_SNAPSHOT_DATE = 20240702T082400Z` > > Signed-off-by: Alexander Heinisch <alexander.heinisch@siemens.com> > --- > v2: Added documentation > v3: Renamed APT_SNAPSHOT_DATE to ISAR_APT_SNAPSHOT_DATE > > doc/user_manual.md | 3 ++- > kas/opt/Kconfig | 2 +- > meta-isar/conf/distro/ubuntu-common.inc | 2 +- > meta/conf/distro/debian-common.conf | 2 +- > meta/recipes-core/isar-bootstrap/isar-bootstrap.inc | 4 ++-- > 5 files changed, 7 insertions(+), 6 deletions(-) > > diff --git a/doc/user_manual.md b/doc/user_manual.md > index 706c74bf..cf3c3b12 100644 > --- a/doc/user_manual.md > +++ b/doc/user_manual.md > @@ -435,7 +435,8 @@ Some other variables include: > - `ISAR_USE_APT_SNAPSHOT` - Use a frozen apt snapshot instead of > the live mirror. Optional. > `ISAR_APT_DL_LIMIT` - Rate limit the apt fetching to n kB / s. > Optional. > - `DISTRO_APT_SNAPSHOT_PREMIRROR` - Similar to > `DISTRO_APT_PREMIRRORS` but for a snapshot, pre-defined for supported > distros. > - - `ISAR_APT_SNAPSHOT_TIMESTAMP` - Timestamp of the apt snapshot. > Automatically derived from `SOURCE_DATE_EPOCH` if not overwritten. > + - `ISAR_APT_SNAPSHOT_TIMESTAMP` - Unix timestamp of the apt > snapshot. Automatically derived from `SOURCE_DATE_EPOCH` if not > overwritten. (Consider `ISAR_APT_SNAPSHOT_DATE` for a more user > friendly format) > + - `ISAR_APT_SNAPSHOT_DATE` - Timestamp in upstream format (e.g. > `20240702T082400Z`) of the apt snapshot. Overrides > `ISAR_APT_SNAPSHOT_TIMESTAMP` if set. Otherwise, will be > automatically derived from `ISAR_APT_SNAPSHOT_TIMESTAMP` > - `THIRD_PARTY_APT_KEYS` - List of gpg key URIs used to verify apt > repos for apt installation after bootstrapping. > - `FILESEXTRAPATHS` - The default directories BitBake uses when it > processes recipes are initially defined by the FILESPATH variable. > You can extend FILESPATH variable by using FILESEXTRAPATHS. > - `FILESOVERRIDES` - A subset of OVERRIDES used by the build system > for creating FILESPATH. The FILESOVERRIDES variable uses overrides to > automatically extend the FILESPATH variable. > diff --git a/kas/opt/Kconfig b/kas/opt/Kconfig > index 65482e3a..1e56c29f 100644 > --- a/kas/opt/Kconfig > +++ b/kas/opt/Kconfig > @@ -44,7 +44,7 @@ config USE_APT_SNAPSHOT > bool "Use the distros snapshot mirror" > depends on !MIRROR_DEBIAN && (DEBIAN_BUSTER || > DEBIAN_BULLSEYE || DEBIAN_BOOKWORM || DEBIAN_TRIXIE || DEBIAN_SID || > UBUNTU_FOCAL || UBUNTU_JAMMY) > help > - Use a snapshot mirror for the selected distribution. > The date is defined by ISAR_APT_SNAPSHOT_TIMESTAMP. > + Use a snapshot mirror for the selected distribution. > The date is defined by ISAR_APT_SNAPSHOT_DATE. > > config KAS_INCLUDE_APT_SNAPSHOT > string > diff --git a/meta-isar/conf/distro/ubuntu-common.inc b/meta- > isar/conf/distro/ubuntu-common.inc > index 232bee35..be7fd047 100644 > --- a/meta-isar/conf/distro/ubuntu-common.inc > +++ b/meta-isar/conf/distro/ubuntu-common.inc > @@ -37,4 +37,4 @@ SYSTEMD_BOOTLOADER_INSTALL:focal = > "systemd:${DISTRO_ARCH}" > SYSTEMD_BOOTLOADER_INSTALL:jammy = "systemd:${DISTRO_ARCH}" > > # snapshot mirror for reproducible builds > -DISTRO_APT_SNAPSHOT_PREMIRROR ??= > "(http|https)://archive.ubuntu.com/(.*) > https://snapshot.ubuntu.com/\2/${APT_SNAPSHOT_DATE}/\n" > +DISTRO_APT_SNAPSHOT_PREMIRROR ??= > "(http|https)://archive.ubuntu.com/(.*) > https://snapshot.ubuntu.com/\2/${ISAR_APT_SNAPSHOT_DATE}/\n" > diff --git a/meta/conf/distro/debian-common.conf > b/meta/conf/distro/debian-common.conf > index 8c6a3f63..65b5b083 100644 > --- a/meta/conf/distro/debian-common.conf > +++ b/meta/conf/distro/debian-common.conf > @@ -40,4 +40,4 @@ COMPAT_DISTRO_ARCH:amd64 = "i386" > COMPAT_DISTRO_ARCH:arm64 = "armhf" > > # snapshot mirror for reproducible builds > -DISTRO_APT_SNAPSHOT_PREMIRROR ??= "deb.debian.org/(.*) > snapshot.debian.org/archive/\1/${APT_SNAPSHOT_DATE}/\n" > +DISTRO_APT_SNAPSHOT_PREMIRROR ??= "deb.debian.org/(.*) > snapshot.debian.org/archive/\1/${ISAR_APT_SNAPSHOT_DATE}/\n" > diff --git a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc > b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc > index 12f32ff0..92974eb5 100644 > --- a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc > +++ b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc > @@ -34,7 +34,7 @@ DISTRO_VARS_PREFIX ?= "${@'HOST_' if > bb.utils.to_boolean(d.getVar('BOOTSTRAP_FOR > BOOTSTRAP_DISTRO = "${@d.getVar('HOST_DISTRO' if > bb.utils.to_boolean(d.getVar('BOOTSTRAP_FOR_HOST')) else 'DISTRO')}" > BOOTSTRAP_BASE_DISTRO = "${@d.getVar('HOST_BASE_DISTRO' if > bb.utils.to_boolean(d.getVar('BOOTSTRAP_FOR_HOST')) else > 'BASE_DISTRO')}" > FILESEXTRAPATHS:append = ":${BBPATH}" > -APT_SNAPSHOT_DATE = "${@ get_apt_snapshot_date(d)}" > +ISAR_APT_SNAPSHOT_DATE ?= "${@ get_isar_apt_snapshot_date(d)}" > > inherit deb-dl-dir > > @@ -110,7 +110,7 @@ def parse_aptsources_list_line(source_list_line): > > return [type, options, source, suite, components] > > -def get_apt_snapshot_date(d): > +def get_isar_apt_snapshot_date(d): > import time > source_date_epoch = d.getVar('ISAR_APT_SNAPSHOT_TIMESTAMP') > return time.strftime('%Y%m%dT%H%M%SZ', > time.gmtime(int(source_date_epoch))) > -- > 2.43.0 > Applied to next, thanks.
diff --git a/doc/user_manual.md b/doc/user_manual.md index 706c74bf..cf3c3b12 100644 --- a/doc/user_manual.md +++ b/doc/user_manual.md @@ -435,7 +435,8 @@ Some other variables include: - `ISAR_USE_APT_SNAPSHOT` - Use a frozen apt snapshot instead of the live mirror. Optional. `ISAR_APT_DL_LIMIT` - Rate limit the apt fetching to n kB / s. Optional. - `DISTRO_APT_SNAPSHOT_PREMIRROR` - Similar to `DISTRO_APT_PREMIRRORS` but for a snapshot, pre-defined for supported distros. - - `ISAR_APT_SNAPSHOT_TIMESTAMP` - Timestamp of the apt snapshot. Automatically derived from `SOURCE_DATE_EPOCH` if not overwritten. + - `ISAR_APT_SNAPSHOT_TIMESTAMP` - Unix timestamp of the apt snapshot. Automatically derived from `SOURCE_DATE_EPOCH` if not overwritten. (Consider `ISAR_APT_SNAPSHOT_DATE` for a more user friendly format) + - `ISAR_APT_SNAPSHOT_DATE` - Timestamp in upstream format (e.g. `20240702T082400Z`) of the apt snapshot. Overrides `ISAR_APT_SNAPSHOT_TIMESTAMP` if set. Otherwise, will be automatically derived from `ISAR_APT_SNAPSHOT_TIMESTAMP` - `THIRD_PARTY_APT_KEYS` - List of gpg key URIs used to verify apt repos for apt installation after bootstrapping. - `FILESEXTRAPATHS` - The default directories BitBake uses when it processes recipes are initially defined by the FILESPATH variable. You can extend FILESPATH variable by using FILESEXTRAPATHS. - `FILESOVERRIDES` - A subset of OVERRIDES used by the build system for creating FILESPATH. The FILESOVERRIDES variable uses overrides to automatically extend the FILESPATH variable. diff --git a/kas/opt/Kconfig b/kas/opt/Kconfig index 65482e3a..1e56c29f 100644 --- a/kas/opt/Kconfig +++ b/kas/opt/Kconfig @@ -44,7 +44,7 @@ config USE_APT_SNAPSHOT bool "Use the distros snapshot mirror" depends on !MIRROR_DEBIAN && (DEBIAN_BUSTER || DEBIAN_BULLSEYE || DEBIAN_BOOKWORM || DEBIAN_TRIXIE || DEBIAN_SID || UBUNTU_FOCAL || UBUNTU_JAMMY) help - Use a snapshot mirror for the selected distribution. The date is defined by ISAR_APT_SNAPSHOT_TIMESTAMP. + Use a snapshot mirror for the selected distribution. The date is defined by ISAR_APT_SNAPSHOT_DATE. config KAS_INCLUDE_APT_SNAPSHOT string diff --git a/meta-isar/conf/distro/ubuntu-common.inc b/meta-isar/conf/distro/ubuntu-common.inc index 232bee35..be7fd047 100644 --- a/meta-isar/conf/distro/ubuntu-common.inc +++ b/meta-isar/conf/distro/ubuntu-common.inc @@ -37,4 +37,4 @@ SYSTEMD_BOOTLOADER_INSTALL:focal = "systemd:${DISTRO_ARCH}" SYSTEMD_BOOTLOADER_INSTALL:jammy = "systemd:${DISTRO_ARCH}" # snapshot mirror for reproducible builds -DISTRO_APT_SNAPSHOT_PREMIRROR ??= "(http|https)://archive.ubuntu.com/(.*) https://snapshot.ubuntu.com/\2/${APT_SNAPSHOT_DATE}/\n" +DISTRO_APT_SNAPSHOT_PREMIRROR ??= "(http|https)://archive.ubuntu.com/(.*) https://snapshot.ubuntu.com/\2/${ISAR_APT_SNAPSHOT_DATE}/\n" diff --git a/meta/conf/distro/debian-common.conf b/meta/conf/distro/debian-common.conf index 8c6a3f63..65b5b083 100644 --- a/meta/conf/distro/debian-common.conf +++ b/meta/conf/distro/debian-common.conf @@ -40,4 +40,4 @@ COMPAT_DISTRO_ARCH:amd64 = "i386" COMPAT_DISTRO_ARCH:arm64 = "armhf" # snapshot mirror for reproducible builds -DISTRO_APT_SNAPSHOT_PREMIRROR ??= "deb.debian.org/(.*) snapshot.debian.org/archive/\1/${APT_SNAPSHOT_DATE}/\n" +DISTRO_APT_SNAPSHOT_PREMIRROR ??= "deb.debian.org/(.*) snapshot.debian.org/archive/\1/${ISAR_APT_SNAPSHOT_DATE}/\n" diff --git a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc index 12f32ff0..92974eb5 100644 --- a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc +++ b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc @@ -34,7 +34,7 @@ DISTRO_VARS_PREFIX ?= "${@'HOST_' if bb.utils.to_boolean(d.getVar('BOOTSTRAP_FOR BOOTSTRAP_DISTRO = "${@d.getVar('HOST_DISTRO' if bb.utils.to_boolean(d.getVar('BOOTSTRAP_FOR_HOST')) else 'DISTRO')}" BOOTSTRAP_BASE_DISTRO = "${@d.getVar('HOST_BASE_DISTRO' if bb.utils.to_boolean(d.getVar('BOOTSTRAP_FOR_HOST')) else 'BASE_DISTRO')}" FILESEXTRAPATHS:append = ":${BBPATH}" -APT_SNAPSHOT_DATE = "${@ get_apt_snapshot_date(d)}" +ISAR_APT_SNAPSHOT_DATE ?= "${@ get_isar_apt_snapshot_date(d)}" inherit deb-dl-dir @@ -110,7 +110,7 @@ def parse_aptsources_list_line(source_list_line): return [type, options, source, suite, components] -def get_apt_snapshot_date(d): +def get_isar_apt_snapshot_date(d): import time source_date_epoch = d.getVar('ISAR_APT_SNAPSHOT_TIMESTAMP') return time.strftime('%Y%m%dT%H%M%SZ', time.gmtime(int(source_date_epoch)))