[RFC] native.bbclass

Hi all,

as mentioned by Jan, I have another approach of building native/host/compat
arch packages, inspired by OE's native class.

The way it works is:
- A recipe can inherit native.bbclass (or, if we like we could even add it
  to all recipes via the dpkg class).
- For recipes that inherit native.bbclass, bitbake generates a new bitbake
  target with suffix `-native`. Recipes can DEPEND on those `-native` packages.
  My use case that made me develop this is goreleaser, a build tool
  for which I have a bitbake recipe. If I want to crosscompile, I need it
  for the host architecture. By DEPENDing on `goreleaser-native`, I can
  then add `goreleaser:native` to the build dependencies in my control file
  (the `:native` makes dpkg-buildpackage choose the right architecture).
- When building the recipe, the only change is that we set PACKAGE_ARCH=HOST_ARCH.
  In addition we have the override `class-native`, which is set when building
  the native variant of the recipe and lets us make conditional changes to
  what is built.

Some caveats/notes:
- Both "normal" target arch and `-native` builds generate ARCH=all packages
  and source packages. So there is potential duplication. In theory those
  packages should be identical, but we currently don't have any way of knowing.
  This is in part due to using shared isar-apt to transfer packages between
  jobs, where the last job to push a package always wins.
  My earlier RFC on removing (that use of) isar-apt and explicitly
  transferring dependent packages might help here. In that case we have full
  control of what packages are provided, and can detect such duplicates.
- I'm changing the default overrides to include PACKAGE_ARCH instead of
  DISTRO_ARCH. Maybe this is how it always should have been?
- This is currently only for "native", but I think we can simply duplicate
  the pattern to also support compat arch packages.
- Using PN in recipes can become tricky. For example, I sometimes see the
  pattern of setting SRC_URI="git://github.com/${PN}/${PN}.git".
  This would break when building the native variant, as this will then
  have a different PN. Personally, I don't think this is a big problem.
  Using ${PN} in SRC_URI may not be a good idea anyways, and if we really
  need a variable, there is BPN, which does not have the -native suffix.
- We currently don't have a use case for this in meta-isar. But maybe we
  can come up with a good example.

I'd like to try if this approach works for the other use cases we have.

Adriaan

PS: If you'd like some more examples of the BBCLASSEXTEND feature, have
    a look at the SDK generation, which has been refactored in that way.

---
 meta/classes/native.bbclass | 13 +++++++++++++
 meta/conf/bitbake.conf      |  5 +++--
 2 files changed, 16 insertions(+), 2 deletions(-)
 create mode 100644 meta/classes/native.bbclass

On 17.01.23 19:03, Adriaan Schmidt wrote:
> Hi all,
> 
> as mentioned by Jan, I have another approach of building native/host/compat
> arch packages, inspired by OE's native class.
> 
> The way it works is:
> - A recipe can inherit native.bbclass (or, if we like we could even add it
>   to all recipes via the dpkg class).

I think we should not start spreading these inherits across random
recipes when there are no costs and risks (as I believe) in adding that
to dpkg directly, likely even dpkg-base.

> - For recipes that inherit native.bbclass, bitbake generates a new bitbake
>   target with suffix `-native`. Recipes can DEPEND on those `-native` packages.
>   My use case that made me develop this is goreleaser, a build tool
>   for which I have a bitbake recipe. If I want to crosscompile, I need it
>   for the host architecture. By DEPENDing on `goreleaser-native`, I can
>   then add `goreleaser:native` to the build dependencies in my control file
>   (the `:native` makes dpkg-buildpackage choose the right architecture).
> - When building the recipe, the only change is that we set PACKAGE_ARCH=HOST_ARCH.
>   In addition we have the override `class-native`, which is set when building
>   the native variant of the recipe and lets us make conditional changes to
>   what is built.

So, if I decide to pre-populate an image with libfoo not only for the
target arch but maybe also others (thinking of compat here), I would
have to do

IMAGE_INSTALL += "meta-package"

and meta-package would have

DEBIAN_DEPENDS += "libfoo, libfoo:${COMPAT_DISTRO_ARCH}"
DEPENDS += "libfoo libfoo-compat"

Unless we find a way to teach IMAGE_INSTALL resolving "...:<some-arch>"
into the right -native/-compat DEPENDS.

Obviously simpler is to have some 32-bit-only legacy app that states

PACKAGE_ARCH = "${COMPAT_DISTRO_ARCH}"
DEBIAN_DEPENDS = "libfoo:${COMPAT_DISTRO_ARCH}"
DEPENDS = "libfoo-compat"

libfoo itself would no longer have to worry about becoming compat as well.

> 
> Some caveats/notes:
> - Both "normal" target arch and `-native` builds generate ARCH=all packages
>   and source packages. So there is potential duplication. In theory those
>   packages should be identical, but we currently don't have any way of knowing.
>   This is in part due to using shared isar-apt to transfer packages between
>   jobs, where the last job to push a package always wins.
>   My earlier RFC on removing (that use of) isar-apt and explicitly
>   transferring dependent packages might help here. In that case we have full
>   control of what packages are provided, and can detect such duplicates.

We have a second duplication issue, and that is around the debian source
package. Again, all targets should normally generate a bit-identical
version of that so that only identical versions will be found when
collecting them for potential redistribution. I was assuming Isar would
already build a deb-src repo for isar-apt, but it this feature seems to
be missing. Once we add it, the topic above will become relevant for it
as well.

> - I'm changing the default overrides to include PACKAGE_ARCH instead of
>   DISTRO_ARCH. Maybe this is how it always should have been?
> - This is currently only for "native", but I think we can simply duplicate
>   the pattern to also support compat arch packages.

This should be done in the same run, we already need it as urgently as
-native.

> - Using PN in recipes can become tricky. For example, I sometimes see the
>   pattern of setting SRC_URI="git://github.com/${PN}/${PN}.git".
>   This would break when building the native variant, as this will then
>   have a different PN. Personally, I don't think this is a big problem.
>   Using ${PN} in SRC_URI may not be a good idea anyways, and if we really
>   need a variable, there is BPN, which does not have the -native suffix.

I agree.

> - We currently don't have a use case for this in meta-isar. But maybe we
>   can come up with a good example.

Typical use cases are self-built build tools for other self-built
packages. Or cleaning up the kbuild issue that also Stefan Koch was
working on. In fact, kbuild is an existing in-tree -native use case that
we hacked so far to generate only -native packages. When converting
that, we could add a test case that uses the to-be-generated
kbuild:target package on the generated image, e.g. to install a dkms
kernel module live.

Jan

> 
> I'd like to try if this approach works for the other use cases we have.
> 
> Adriaan
> 
> PS: If you'd like some more examples of the BBCLASSEXTEND feature, have
>     a look at the SDK generation, which has been refactored in that way.
> 
> ---
>  meta/classes/native.bbclass | 13 +++++++++++++
>  meta/conf/bitbake.conf      |  5 +++--
>  2 files changed, 16 insertions(+), 2 deletions(-)
>  create mode 100644 meta/classes/native.bbclass
> 
> diff --git a/meta/classes/native.bbclass b/meta/classes/native.bbclass
> new file mode 100644
> index 00000000..896c8a06
> --- /dev/null
> +++ b/meta/classes/native.bbclass
> @@ -0,0 +1,13 @@
> +BBCLASSEXTEND = "native"
> +BPN = "${PN}"
> +
> +python native_virtclass_handler() {
> +    pn = e.data.getVar('PN')
> +    if pn.endswith('-native'):
> +        e.data.setVar('BPN', pn[:-len('-native')])
> +        e.data.appendVar('OVERRIDES', ':class-native')
> +}
> +addhandler native_virtclass_handler
> +native_virtclass_handler[eventmask] = "bb.event.RecipePreFinalise"
> +
> +PACKAGE_ARCH_class-native = "${HOST_ARCH}"
> diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf
> index 98412e02..ef962fc7 100644
> --- a/meta/conf/bitbake.conf
> +++ b/meta/conf/bitbake.conf
> @@ -67,8 +67,8 @@ KERNEL_FILE_mipsel ?= "vmlinux"
>  KERNEL_FILE_riscv64 ?= "vmlinux"
>  KERNEL_FILE_arm64 ?= "vmlinux"
>  
> -OVERRIDES = "${DISTRO_ARCH}:${COMPAT_OVERRIDE}:${MACHINE}:${DISTRO}:${BASE_DISTRO_CODENAME}:forcevariable"
> -FILESOVERRIDES = "${DISTRO_ARCH}:${MACHINE}"
> +OVERRIDES = "${PACKAGE_ARCH}:${COMPAT_OVERRIDE}:${MACHINE}:${DISTRO}:${BASE_DISTRO_CODENAME}:forcevariable"
> +FILESOVERRIDES = "${PACKAGE_ARCH}:${MACHINE}"
>  COMPAT_OVERRIDE = "${@'compat-arch' if d.getVar('ISAR_ENABLE_COMPAT_ARCH') == '1' else ''}"
>  
>  # Setting default QEMU_ARCH variables for different DISTRO_ARCH:
> @@ -81,6 +81,7 @@ QEMU_ARCH_riscv64 = "riscv64"
>  
>  # Codename of the repository created by the caching class
>  DEBDISTRONAME ?= "isar"
> +NATIVELSBSTRING ?= "isarnative"
>  
>  # Strings used in sstate signature files
>  TARGET_VENDOR = ""

On Tue, 2023-01-17 at 19:03 +0100, Adriaan Schmidt wrote:
> Hi all,
> 
> as mentioned by Jan, I have another approach of building
> native/host/compat
> arch packages, inspired by OE's native class.
> 
> The way it works is:
> - A recipe can inherit native.bbclass (or, if we like we could even
> add it
>   to all recipes via the dpkg class).
> - For recipes that inherit native.bbclass, bitbake generates a new
> bitbake
>   target with suffix `-native`. Recipes can DEPEND on those `-native`
> packages.
>   My use case that made me develop this is goreleaser, a build tool
>   for which I have a bitbake recipe. If I want to crosscompile, I
> need it
>   for the host architecture. By DEPENDing on `goreleaser-native`, I
> can
>   then add `goreleaser:native` to the build dependencies in my
> control file
>   (the `:native` makes dpkg-buildpackage choose the right
> architecture).
> - When building the recipe, the only change is that we set
> PACKAGE_ARCH=HOST_ARCH.
>   In addition we have the override `class-native`, which is set when
> building
>   the native variant of the recipe and lets us make conditional
> changes to
>   what is built.
> 
> Some caveats/notes:
> - Both "normal" target arch and `-native` builds generate ARCH=all
> packages
>   and source packages. So there is potential duplication. In theory
> those
>   packages should be identical, but we currently don't have any way
> of knowing.

Yes, that is indeed problematic. But we already have this problem today
when doing manual duplication to get multi-arch support.

>   This is in part due to using shared isar-apt to transfer packages
> between
>   jobs, where the last job to push a package always wins.
>   My earlier RFC on removing (that use of) isar-apt and explicitly
>   transferring dependent packages might help here. In that case we
> have full
>   control of what packages are provided, and can detect such
> duplicates.
> - I'm changing the default overrides to include PACKAGE_ARCH instead
> of
>   DISTRO_ARCH. Maybe this is how it always should have been?

Hmm... I'm not 100% sure if that is correct. The PACKAGE_ARCH is what
is passed to --host= of sbuild. This describes the toolchain that is
used to generate the package and somehow corresponds to DISTRO_ARCH
(for target packages and HOST_ARCH for host tools). The architecture of
the package that is built is specified by DPKG_ARCH. IMHO the naming
scheme here is quite confusing.

> - This is currently only for "native", but I think we can simply
> duplicate
>   the pattern to also support compat arch packages.

We have an internal layer that heavily uses multi-arch and builds
library packages for at least two architectures at the same time.
Currently this is done via a manual duplication of the recipes (using a
shared .inc), but this is error prone and tedious.
Having this in ISAR would significantly reduce the maintenance effort
there.

> - Using PN in recipes can become tricky. For example, I sometimes see
> the
>   pattern of setting SRC_URI="git://github.com/${PN}/${PN}.git".
>   This would break when building the native variant, as this will
> then
>   have a different PN. Personally, I don't think this is a big
> problem.
>   Using ${PN} in SRC_URI may not be a good idea anyways, and if we
> really
>   need a variable, there is BPN, which does not have the -native
> suffix.

We even have that pattern in the hello.bb example. It's pretty
prominent.

Felix

> - We currently don't have a use case for this in meta-isar. But maybe
> we
>   can come up with a good example.
> 
> I'd like to try if this approach works for the other use cases we
> have.
> 
> Adriaan
> 
> PS: If you'd like some more examples of the BBCLASSEXTEND feature,
> have
>     a look at the SDK generation, which has been refactored in that
> way.
> 
> ---
>  meta/classes/native.bbclass | 13 +++++++++++++
>  meta/conf/bitbake.conf      |  5 +++--
>  2 files changed, 16 insertions(+), 2 deletions(-)
>  create mode 100644 meta/classes/native.bbclass
> 
> diff --git a/meta/classes/native.bbclass
> b/meta/classes/native.bbclass
> new file mode 100644
> index 00000000..896c8a06
> --- /dev/null
> +++ b/meta/classes/native.bbclass
> @@ -0,0 +1,13 @@
> +BBCLASSEXTEND = "native"
> +BPN = "${PN}"
> +
> +python native_virtclass_handler() {
> +    pn = e.data.getVar('PN')
> +    if pn.endswith('-native'):
> +        e.data.setVar('BPN', pn[:-len('-native')])
> +        e.data.appendVar('OVERRIDES', ':class-native')
> +}
> +addhandler native_virtclass_handler
> +native_virtclass_handler[eventmask] = "bb.event.RecipePreFinalise"
> +
> +PACKAGE_ARCH_class-native = "${HOST_ARCH}"
> diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf
> index 98412e02..ef962fc7 100644
> --- a/meta/conf/bitbake.conf
> +++ b/meta/conf/bitbake.conf
> @@ -67,8 +67,8 @@ KERNEL_FILE_mipsel ?= "vmlinux"
>  KERNEL_FILE_riscv64 ?= "vmlinux"
>  KERNEL_FILE_arm64 ?= "vmlinux"
>  
> -OVERRIDES =
> "${DISTRO_ARCH}:${COMPAT_OVERRIDE}:${MACHINE}:${DISTRO}:${BASE_DISTRO
> _CODENAME}:forcevariable"
> -FILESOVERRIDES = "${DISTRO_ARCH}:${MACHINE}"
> +OVERRIDES =
> "${PACKAGE_ARCH}:${COMPAT_OVERRIDE}:${MACHINE}:${DISTRO}:${BASE_DISTR
> O_CODENAME}:forcevariable"
> +FILESOVERRIDES = "${PACKAGE_ARCH}:${MACHINE}"
>  COMPAT_OVERRIDE = "${@'compat-arch' if
> d.getVar('ISAR_ENABLE_COMPAT_ARCH') == '1' else ''}"
>  
>  # Setting default QEMU_ARCH variables for different DISTRO_ARCH:
> @@ -81,6 +81,7 @@ QEMU_ARCH_riscv64 = "riscv64"
>  
>  # Codename of the repository created by the caching class
>  DEBDISTRONAME ?= "isar"
> +NATIVELSBSTRING ?= "isarnative"
>  
>  # Strings used in sstate signature files
>  TARGET_VENDOR = ""
> -- 
> 2.30.2
>

On Wed, 2023-01-18 at 07:09 +0100, Jan Kiszka wrote:
> On 17.01.23 19:03, Adriaan Schmidt wrote:
> > Hi all,
> > 
> > as mentioned by Jan, I have another approach of building
> > native/host/compat
> > arch packages, inspired by OE's native class.
> > 
> > The way it works is:
> > - A recipe can inherit native.bbclass (or, if we like we could even
> > add it
> >   to all recipes via the dpkg class).
> 
> I think we should not start spreading these inherits across random
> recipes when there are no costs and risks (as I believe) in adding
> that
> to dpkg directly, likely even dpkg-base.
> 
> > - For recipes that inherit native.bbclass, bitbake generates a new
> > bitbake
> >   target with suffix `-native`. Recipes can DEPEND on those `-
> > native` packages.
> >   My use case that made me develop this is goreleaser, a build tool
> >   for which I have a bitbake recipe. If I want to crosscompile, I
> > need it
> >   for the host architecture. By DEPENDing on `goreleaser-native`, I
> > can
> >   then add `goreleaser:native` to the build dependencies in my
> > control file
> >   (the `:native` makes dpkg-buildpackage choose the right
> > architecture).
> > - When building the recipe, the only change is that we set
> > PACKAGE_ARCH=HOST_ARCH.
> >   In addition we have the override `class-native`, which is set
> > when building
> >   the native variant of the recipe and lets us make conditional
> > changes to
> >   what is built.
> 
> So, if I decide to pre-populate an image with libfoo not only for the
> target arch but maybe also others (thinking of compat here), I would
> have to do
> 
> IMAGE_INSTALL += "meta-package"
> 
> and meta-package would have
> 
> DEBIAN_DEPENDS += "libfoo, libfoo:${COMPAT_DISTRO_ARCH}"
> DEPENDS += "libfoo libfoo-compat"
> 
> Unless we find a way to teach IMAGE_INSTALL resolving "...:<some-
> arch>"
> into the right -native/-compat DEPENDS.

This is trivial to resolve with python functions.
Currently the IMAGE_INSTALL is added both to DEPENDS, as well as to
apt. For apt, we keep it. For DEPENDS, we replace "<foo>:arch" with
"<foo>-arch", where arch might have to be mapped to "-native / -
compat", depending on how the naming scheme will be implemented.

Felix

> 
> Obviously simpler is to have some 32-bit-only legacy app that states
> 
> PACKAGE_ARCH = "${COMPAT_DISTRO_ARCH}"
> DEBIAN_DEPENDS = "libfoo:${COMPAT_DISTRO_ARCH}"
> DEPENDS = "libfoo-compat"
> 
> libfoo itself would no longer have to worry about becoming compat as
> well.
> 
> > 
> > Some caveats/notes:
> > - Both "normal" target arch and `-native` builds generate ARCH=all
> > packages
> >   and source packages. So there is potential duplication. In theory
> > those
> >   packages should be identical, but we currently don't have any way
> > of knowing.
> >   This is in part due to using shared isar-apt to transfer packages
> > between
> >   jobs, where the last job to push a package always wins.
> >   My earlier RFC on removing (that use of) isar-apt and explicitly
> >   transferring dependent packages might help here. In that case we
> > have full
> >   control of what packages are provided, and can detect such
> > duplicates.
> 
> We have a second duplication issue, and that is around the debian
> source
> package. Again, all targets should normally generate a bit-identical
> version of that so that only identical versions will be found when
> collecting them for potential redistribution. I was assuming Isar
> would
> already build a deb-src repo for isar-apt, but it this feature seems
> to
> be missing. Once we add it, the topic above will become relevant for
> it
> as well.
> 
> > - I'm changing the default overrides to include PACKAGE_ARCH
> > instead of
> >   DISTRO_ARCH. Maybe this is how it always should have been?
> > - This is currently only for "native", but I think we can simply
> > duplicate
> >   the pattern to also support compat arch packages.
> 
> This should be done in the same run, we already need it as urgently
> as
> -native.
> 
> > - Using PN in recipes can become tricky. For example, I sometimes
> > see the
> >   pattern of setting SRC_URI="git://github.com/${PN}/${PN}.git".
> >   This would break when building the native variant, as this will
> > then
> >   have a different PN. Personally, I don't think this is a big
> > problem.
> >   Using ${PN} in SRC_URI may not be a good idea anyways, and if we
> > really
> >   need a variable, there is BPN, which does not have the -native
> > suffix.
> 
> I agree.
> 
> > - We currently don't have a use case for this in meta-isar. But
> > maybe we
> >   can come up with a good example.
> 
> Typical use cases are self-built build tools for other self-built
> packages. Or cleaning up the kbuild issue that also Stefan Koch was
> working on. In fact, kbuild is an existing in-tree -native use case
> that
> we hacked so far to generate only -native packages. When converting
> that, we could add a test case that uses the to-be-generated
> kbuild:target package on the generated image, e.g. to install a dkms
> kernel module live.
> 
> Jan
> 
> > 
> > I'd like to try if this approach works for the other use cases we
> > have.
> > 
> > Adriaan
> > 
> > PS: If you'd like some more examples of the BBCLASSEXTEND feature,
> > have
> >     a look at the SDK generation, which has been refactored in that
> > way.
> > 
> > ---
> >  meta/classes/native.bbclass | 13 +++++++++++++
> >  meta/conf/bitbake.conf      |  5 +++--
> >  2 files changed, 16 insertions(+), 2 deletions(-)
> >  create mode 100644 meta/classes/native.bbclass
> > 
> > diff --git a/meta/classes/native.bbclass
> > b/meta/classes/native.bbclass
> > new file mode 100644
> > index 00000000..896c8a06
> > --- /dev/null
> > +++ b/meta/classes/native.bbclass
> > @@ -0,0 +1,13 @@
> > +BBCLASSEXTEND = "native"
> > +BPN = "${PN}"
> > +
> > +python native_virtclass_handler() {
> > +    pn = e.data.getVar('PN')
> > +    if pn.endswith('-native'):
> > +        e.data.setVar('BPN', pn[:-len('-native')])
> > +        e.data.appendVar('OVERRIDES', ':class-native')
> > +}
> > +addhandler native_virtclass_handler
> > +native_virtclass_handler[eventmask] = "bb.event.RecipePreFinalise"
> > +
> > +PACKAGE_ARCH_class-native = "${HOST_ARCH}"
> > diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf
> > index 98412e02..ef962fc7 100644
> > --- a/meta/conf/bitbake.conf
> > +++ b/meta/conf/bitbake.conf
> > @@ -67,8 +67,8 @@ KERNEL_FILE_mipsel ?= "vmlinux"
> >  KERNEL_FILE_riscv64 ?= "vmlinux"
> >  KERNEL_FILE_arm64 ?= "vmlinux"
> >  
> > -OVERRIDES =
> > "${DISTRO_ARCH}:${COMPAT_OVERRIDE}:${MACHINE}:${DISTRO}:${BASE_DIST
> > RO_CODENAME}:forcevariable"
> > -FILESOVERRIDES = "${DISTRO_ARCH}:${MACHINE}"
> > +OVERRIDES =
> > "${PACKAGE_ARCH}:${COMPAT_OVERRIDE}:${MACHINE}:${DISTRO}:${BASE_DIS
> > TRO_CODENAME}:forcevariable"
> > +FILESOVERRIDES = "${PACKAGE_ARCH}:${MACHINE}"
> >  COMPAT_OVERRIDE = "${@'compat-arch' if
> > d.getVar('ISAR_ENABLE_COMPAT_ARCH') == '1' else ''}"
> >  
> >  # Setting default QEMU_ARCH variables for different DISTRO_ARCH:
> > @@ -81,6 +81,7 @@ QEMU_ARCH_riscv64 = "riscv64"
> >  
> >  # Codename of the repository created by the caching class
> >  DEBDISTRONAME ?= "isar"
> > +NATIVELSBSTRING ?= "isarnative"
> >  
> >  # Strings used in sstate signature files
> >  TARGET_VENDOR = ""
> 
> -- 
> Siemens AG, Technology
> Competence Center Embedded Linux
>