mbox

[FYI,0/2] Migration to bitbake 2.0

Message ID 20220607152531.3498370-1-adriaan.schmidt@siemens.com
State Superseded, archived
Headers show

Message

Schmidt, Adriaan June 7, 2022, 7:25 a.m. UTC 
Hi all,

I recently spent a Friday afternoon playing with bitbake 2.0, and here are the
findings, in patch form, without the actual updated code (which would be bitbake/*,
meta/lib/oe/* and meta/classes/sstate.bbclass.

I made it work for mc:qemuamd64-bullseye:isar-image-base, and for one of
our downstream layers, just to get an idea of the work required.
This post is meant as documentation, for whenever the time comes to apply the
bitbake update...

Changes required:
- new override syntax, with ':' instead of '_'
- some rewording (e.g. WHITELIST -> IGNORE_VARS, ABORT -> HALT)
- new task flag "[network]", which is needed on tasks requiring network
  access. If it is not set, networking is disabled by cloning into a new
  namespace. The current bitbake implementation creates a new namespace for
  network and user, with the latter breaking use of sudo. p1 would be
  the corresponding upstream change in bitbake to make sudo work, as an alternative
  to adding "[network] = 1" to all tasks (which I tested, and which also works).
- SRCREV always needs to be a hash, not a branch or tag. This does not
  actually affect Isar itself, but possibly downstream layers (we often
  use SRCREV="v${PV}").
- sstate internally switches from tgz to zstd compression for artifacts,
  and from pickle to compressed json for siginfo files. This affects the
  isar-sstate script.

Adriaan

Adriaan Schmidt (2):
  bitbake: no user namespace in disable_network
  migrate Isar to bitbake 2.0

 bitbake/lib/bb/utils.py                       |  3 +-
 meta-isar/classes/rpi-sdimg.bbclass           |  4 +-
 meta-isar/classes/ubi-ubifs-img.bbclass       |  4 +-
 meta-isar/conf/distro/raspios-bullseye.conf   |  8 +--
 meta-isar/conf/distro/ubuntu-focal.conf       |  4 +-
 meta-isar/conf/local.conf.sample              |  8 +--
 meta-isar/conf/machine/hikey.conf             |  2 +-
 meta-isar/conf/machine/phyboard-mira.conf     |  2 +-
 meta-isar/conf/machine/qemuamd64.conf         |  6 +-
 meta-isar/conf/machine/qemuriscv64.conf       |  2 +-
 meta-isar/conf/machine/stm32mp15x.conf        |  6 +-
 .../conf/multiconfig/qemuamd64-bullseye.conf  |  2 +-
 meta-isar/recipes-app/cowsay/cowsay_git.bb    |  2 +-
 .../recipes-app/hello-isar/hello-isar.bb      |  4 +-
 meta-isar/recipes-app/libhello/libhello.bb    |  4 +-
 .../bootconfig-rpi/bootconfig-rpi.bb          |  6 +-
 .../optee-os/optee-os-stm32mp15x_3.11.0.bb    |  2 +-
 .../linux/linux-mainline_5.4.70.bb            |  4 +-
 .../linux/linux-phy_5.10.76-phy4.bb           |  2 +-
 meta/classes/base.bbclass                     |  2 +
 meta/classes/dpkg-base.bbclass                |  4 +-
 meta/classes/dpkg-gbp.bbclass                 |  4 +-
 meta/classes/dpkg-prebuilt.bbclass            |  2 +-
 meta/classes/dpkg.bbclass                     |  1 +
 meta/classes/image-tools-extension.bbclass    |  2 +
 meta/classes/image.bbclass                    | 38 ++++++------
 meta/classes/imagetypes.bbclass               | 54 ++++++++---------
 meta/classes/imagetypes_container.bbclass     |  4 +-
 meta/classes/imagetypes_vm.bbclass            |  8 +--
 meta/classes/imagetypes_wic.bbclass           |  5 +-
 meta/classes/rootfs.bbclass                   |  4 +-
 meta/classes/sdk.bbclass                      | 34 +++++------
 meta/conf/bitbake.conf                        | 60 ++++++++++++-------
 meta/conf/distro/debian-common.conf           | 12 ++--
 meta/conf/distro/debian-stretch.conf          |  4 +-
 meta/recipes-bsp/barebox/barebox.inc          |  2 +-
 meta/recipes-bsp/optee-os/optee-os-custom.inc |  2 +-
 .../trusted-firmware-a-custom.inc             |  2 +-
 meta/recipes-bsp/u-boot/u-boot-custom.inc     |  2 +-
 .../isar-bootstrap/isar-bootstrap.inc         | 16 +++--
 .../buildchroot/buildchroot.inc               |  6 +-
 meta/recipes-kernel/kselftest/kselftest.inc   |  8 ++-
 meta/recipes-kernel/linux-module/module.inc   |  2 +-
 meta/recipes-kernel/linux/linux-custom.inc    |  6 +-
 scripts/isar-buildenv-internal                |  4 +-
 scripts/isar-sstate                           | 43 ++++++-------
 testsuite/cibuilder.py                        |  8 +--
 testsuite/citest.py                           |  2 +-
 48 files changed, 227 insertions(+), 189 deletions(-)

Comments

Anton Mikanovich June 7, 2022, 3:46 p.m. UTC | #1 
07.06.2022 18:25, Adriaan Schmidt wrote:
> Hi all,
>
> I recently spent a Friday afternoon playing with bitbake 2.0, and here are the
> findings, in patch form, without the actual updated code (which would be bitbake/*,
> meta/lib/oe/* and meta/classes/sstate.bbclass.
>
> I made it work for mc:qemuamd64-bullseye:isar-image-base, and for one of
> our downstream layers, just to get an idea of the work required.
> This post is meant as documentation, for whenever the time comes to apply the
> bitbake update...
>
> Changes required:
> - new override syntax, with ':' instead of '_'
> - some rewording (e.g. WHITELIST -> IGNORE_VARS, ABORT -> HALT)
> - new task flag "[network]", which is needed on tasks requiring network
>    access. If it is not set, networking is disabled by cloning into a new
>    namespace. The current bitbake implementation creates a new namespace for
>    network and user, with the latter breaking use of sudo. p1 would be
>    the corresponding upstream change in bitbake to make sudo work, as an alternative
>    to adding "[network] = 1" to all tasks (which I tested, and which also works).
> - SRCREV always needs to be a hash, not a branch or tag. This does not
>    actually affect Isar itself, but possibly downstream layers (we often
>    use SRCREV="v${PV}").
> - sstate internally switches from tgz to zstd compression for artifacts,
>    and from pickle to compressed json for siginfo files. This affects the
>    isar-sstate script.
>
> Adriaan

Hello Adriaan,
Thanks for the patch. I'm already working on "Bitbake update prepare" 
patchset
which will include everything needed for migration but without override 
syntax
update itself.
It will also include (already configured for Isar) convert-overrides.py 
script
for automatic downstreams update. After this patch we will have Isar and
downstreams ready for fast syntax change (I hope so).
Unfortunately this task has less priority then Isar release preparation, 
but I
will try to share something soon.