| Message ID | 20240214101025.2123540-1-stefan-koch@siemens.com |
|---|---|
| Headers | show |
| Series | linux-custom: Split up binaries from kernel headers to kbuild package | expand |
On Wed, 2024-02-14 at 11:10 +0100, Stefan Koch wrote: > Hi > > This updated v6 patchset implement suggestions from reviewed v4 > patchset. > It's now a set of five patches: > - linux-custom: Set PROVIDES variable using bitbake overrides > - linux-custom: Split up binaries from kernel headers to kbuild > package > - linux-custom: Provide target and host specific kernel kbuild > packages > - linux-module: Support emulated module build with cross-compiled > kernel > - docs: Update custom_kernel docs for split up of kernel scripts and > tools Hi Stefan, I tested this again and now also the cross-kernel / non-cross module and vice versa properly works. Thanks! Tested-by: Felix Moessbauer <felix.moessbauer@siemens.com> Best regards, Felix > > The first commit sets the PROVIDES variable using > bitbake overrides instead of python code. > > Swap out the binaries from the kernel headers > into kernel kbuild package is the main use-case > introduced by the second commit > "Split up binaries from kernel headers to kbuild package" > > The third commit "Provide target and host specific kernel kbuild > packages" > introduces that the binaries could be swapped out into host and > target > specific kernel kbuild packages. > > Support of emulated module build with a cross-compiled kernel build > is introduced by the fourth commit. > > These are the main development goals: > > 1. Solve already known isar custom kernel > limitations from doc/custom_kernel.inc > - kernel headers package does not support both native > and cross compilation of kernel modules when cross built > > 2. Honor recommendations for future from doc/custom_kernel.inc > - Generate kernel headers packages for both host and target > when using cross build > > 3. Add extensions known from debian kernel packages structure > - Generate a kernel headers package without binaries > - Create specific kernel kbuild packages that > will ship the "scripts" and "tools" binaries > - Use symlinks to point to the "scripts" and "tools" binaries > > 4. Be user friendly > - Avoid redundant configuration of kernel source definitions with > user > actions to enable kbuild package generation > - Use already known way to include linux-custom.inc in just one > own bitbake recipe that provides the kernel source definitions > - Keep known user behavior for existing build configurations: > just update isar, and use the new linux-kbuild bitbake target > to create the kbuild target package when cross building > > Best regards > > Stefan > > Stefan Koch (5): > linux-custom: Set PROVIDES variable using bitbake overrides > linux-custom: Split up binaries from kernel headers to kbuild > package > linux-custom: Provide target and host specific kernel kbuild > packages > linux-module: Support emulated module build with cross-compiled > kernel > docs: Update custom_kernel docs for split up of kernel scripts and > tools > > RECIPE-API-CHANGELOG.md | 31 ++++++++ > doc/custom_kernel.md | 18 +++-- > meta/recipes-kernel/linux-module/module.inc | 3 +- > .../linux/classes/kbuildtarget.bbclass | 8 ++ > .../linux/files/debian/control.tmpl | 15 +++- > .../linux/files/debian/isar/build.tmpl | 13 +++- > .../linux/files/debian/isar/common.tmpl | 11 +++ > .../linux/files/debian/isar/install.tmpl | 75 ++++++++++++++--- > -- > meta/recipes-kernel/linux/linux-custom.inc | 65 +++++++++++++--- > meta/recipes-kernel/linux/linux-distro.bb | 1 + > 10 files changed, 199 insertions(+), 41 deletions(-) > create mode 100644 meta/recipes- > kernel/linux/classes/kbuildtarget.bbclass >
On Wed, 2024-02-14 at 11:10 +0100, 'Stefan Koch' via isar-users wrote: > Hi > > This updated v6 patchset implement suggestions from reviewed v4 > patchset. > It's now a set of five patches: > - linux-custom: Set PROVIDES variable using bitbake overrides > - linux-custom: Split up binaries from kernel headers to kbuild > package > - linux-custom: Provide target and host specific kernel kbuild > packages > - linux-module: Support emulated module build with cross-compiled > kernel > - docs: Update custom_kernel docs for split up of kernel scripts and > tools > > The first commit sets the PROVIDES variable using > bitbake overrides instead of python code. > > Swap out the binaries from the kernel headers > into kernel kbuild package is the main use-case > introduced by the second commit > "Split up binaries from kernel headers to kbuild package" > > The third commit "Provide target and host specific kernel kbuild > packages" > introduces that the binaries could be swapped out into host and > target > specific kernel kbuild packages. > > Support of emulated module build with a cross-compiled kernel build > is introduced by the fourth commit. > > These are the main development goals: > > 1. Solve already known isar custom kernel > limitations from doc/custom_kernel.inc > - kernel headers package does not support both native > and cross compilation of kernel modules when cross built > > 2. Honor recommendations for future from doc/custom_kernel.inc > - Generate kernel headers packages for both host and target > when using cross build > > 3. Add extensions known from debian kernel packages structure > - Generate a kernel headers package without binaries > - Create specific kernel kbuild packages that > will ship the "scripts" and "tools" binaries > - Use symlinks to point to the "scripts" and "tools" binaries > > 4. Be user friendly > - Avoid redundant configuration of kernel source definitions with > user > actions to enable kbuild package generation > - Use already known way to include linux-custom.inc in just one > own bitbake recipe that provides the kernel source definitions > - Keep known user behavior for existing build configurations: > just update isar, and use the new linux-kbuild bitbake target > to create the kbuild target package when cross building > > Best regards > > Stefan > > Stefan Koch (5): > linux-custom: Set PROVIDES variable using bitbake overrides > linux-custom: Split up binaries from kernel headers to kbuild > package > linux-custom: Provide target and host specific kernel kbuild > packages > linux-module: Support emulated module build with cross-compiled > kernel > docs: Update custom_kernel docs for split up of kernel scripts and > tools > > RECIPE-API-CHANGELOG.md | 31 ++++++++ > doc/custom_kernel.md | 18 +++-- > meta/recipes-kernel/linux-module/module.inc | 3 +- > .../linux/classes/kbuildtarget.bbclass | 8 ++ > .../linux/files/debian/control.tmpl | 15 +++- > .../linux/files/debian/isar/build.tmpl | 13 +++- > .../linux/files/debian/isar/common.tmpl | 11 +++ > .../linux/files/debian/isar/install.tmpl | 75 ++++++++++++++--- > -- > meta/recipes-kernel/linux/linux-custom.inc | 65 +++++++++++++--- > meta/recipes-kernel/linux/linux-distro.bb | 1 + > 10 files changed, 199 insertions(+), 41 deletions(-) > create mode 100644 meta/recipes- > kernel/linux/classes/kbuildtarget.bbclass > > -- > 2.39.2 > Hello Stefan. Recently I run the patchset through CI and it failed (test 2 in "dev" set) on amd64 targets with error like: [stdlog] 2024-04-12 16:23:10,235 avocado.app cibuilder L0222 ERROR| ERROR: Nothing PROVIDES 'linux-kbuild-amd64-native' (but mc:qemuamd64-bullseye:/workspace/build/isar_ub_devel_fast/567/meta- isar/recipes-kernel/example-module/example-module.bb DEPENDS on or otherwise requires it). Close matches: [stdlog] 2024-04-12 16:23:10,235 avocado.app cibuilder L0222 ERROR| linux-kbuild-amd64 [stdlog] 2024-04-12 16:23:10,235 avocado.test cibuilder L0220 INFO | [stdlog] 2024-04-12 16:23:10,235 avocado.app cibuilder L0222 ERROR| linux-kbuild-cip-native [stdlog] 2024-04-12 16:23:10,235 avocado.app cibuilder L0222 ERROR| linux-kbuild-mainline-native [stdlog] 2024-04-12 16:23:10,235 avocado.app cibuilder L0222 ERROR| ERROR: Required build target 'isar-image-ci' has no buildable providers. I tried to reproduce it locally, and the easiest way to reproduce was the following: ``` ./kas/kas-container menu <Leave everything as is, just select "Save & Build"> ... Parsing recipes: 100% |###################################################################### #########################################################| Time: 0:00:00 Parsing of 55 .bb files complete (0 cached, 55 parsed). 57 targets, 15 skipped, 0 masked, 0 errors. NOTE: Resolving any missing task queue dependencies ERROR: Nothing PROVIDES 'linux-kbuild-amd64-native' (but /build/../repo/meta-isar/recipes-kernel/example-module/example- module.bb DEPENDS on or otherwise requires it). Close matches: linux-kbuild-amd64 linux-kbuild-cip-native linux-kbuild-mainline-native ERROR: Required build target 'isar-image-base' has no buildable providers. Missing or unbuildable dependency chain was: ['isar-image-base', 'example-module-amd64', 'linux-kbuild-amd64-native'] ``` I also tried the patchset with older Isar (e.g. just some commits before v0.10, when the patchset was released) and the situation looks the same. When we remove "example-module" from list of packages to install, build is OK.
On Mon, 2024-04-15 at 15:47 +0300, Uladzimir Bely wrote: > On Wed, 2024-02-14 at 11:10 +0100, 'Stefan Koch' via isar-users > wrote: > > Hi > > > > This updated v6 patchset implement suggestions from reviewed v4 > > patchset. > > It's now a set of five patches: > > - linux-custom: Set PROVIDES variable using bitbake overrides > > - linux-custom: Split up binaries from kernel headers to kbuild > > package > > - linux-custom: Provide target and host specific kernel kbuild > > packages > > - linux-module: Support emulated module build with cross-compiled > > kernel > > - docs: Update custom_kernel docs for split up of kernel scripts > > and > > tools > > > > The first commit sets the PROVIDES variable using > > bitbake overrides instead of python code. > > > > Swap out the binaries from the kernel headers > > into kernel kbuild package is the main use-case > > introduced by the second commit > > "Split up binaries from kernel headers to kbuild package" > > > > The third commit "Provide target and host specific kernel kbuild > > packages" > > introduces that the binaries could be swapped out into host and > > target > > specific kernel kbuild packages. > > > > Support of emulated module build with a cross-compiled kernel build > > is introduced by the fourth commit. > > > > These are the main development goals: > > > > 1. Solve already known isar custom kernel > > limitations from doc/custom_kernel.inc > > - kernel headers package does not support both native > > and cross compilation of kernel modules when cross built > > > > 2. Honor recommendations for future from doc/custom_kernel.inc > > - Generate kernel headers packages for both host and target > > when using cross build > > > > 3. Add extensions known from debian kernel packages structure > > - Generate a kernel headers package without binaries > > - Create specific kernel kbuild packages that > > will ship the "scripts" and "tools" binaries > > - Use symlinks to point to the "scripts" and "tools" binaries > > > > 4. Be user friendly > > - Avoid redundant configuration of kernel source definitions with > > user > > actions to enable kbuild package generation > > - Use already known way to include linux-custom.inc in just one > > own bitbake recipe that provides the kernel source definitions > > - Keep known user behavior for existing build configurations: > > just update isar, and use the new linux-kbuild bitbake target > > to create the kbuild target package when cross building > > > > Best regards > > > > Stefan > > > > Stefan Koch (5): > > linux-custom: Set PROVIDES variable using bitbake overrides > > linux-custom: Split up binaries from kernel headers to kbuild > > package > > linux-custom: Provide target and host specific kernel kbuild > > packages > > linux-module: Support emulated module build with cross-compiled > > kernel > > docs: Update custom_kernel docs for split up of kernel scripts > > and > > tools > > > > RECIPE-API-CHANGELOG.md | 31 ++++++++ > > doc/custom_kernel.md | 18 +++-- > > meta/recipes-kernel/linux-module/module.inc | 3 +- > > .../linux/classes/kbuildtarget.bbclass | 8 ++ > > .../linux/files/debian/control.tmpl | 15 +++- > > .../linux/files/debian/isar/build.tmpl | 13 +++- > > .../linux/files/debian/isar/common.tmpl | 11 +++ > > .../linux/files/debian/isar/install.tmpl | 75 ++++++++++++++- > > -- > > -- > > meta/recipes-kernel/linux/linux-custom.inc | 65 +++++++++++++-- > > - > > meta/recipes-kernel/linux/linux-distro.bb | 1 + > > 10 files changed, 199 insertions(+), 41 deletions(-) > > create mode 100644 meta/recipes- > > kernel/linux/classes/kbuildtarget.bbclass > > > > -- > > 2.39.2 > > > > Hello Stefan. Hi > > Recently I run the patchset through CI and it failed (test 2 in "dev" > set) on amd64 targets with error like: > > [stdlog] 2024-04-12 16:23:10,235 avocado.app cibuilder L0222 > ERROR| ERROR: Nothing PROVIDES 'linux-kbuild-amd64-native' (but > mc:qemuamd64-bullseye:/workspace/build/isar_ub_devel_fast/567/meta- > isar/recipes-kernel/example-module/example-module.bb DEPENDS on or > otherwise requires it). Close matches: > [stdlog] 2024-04-12 16:23:10,235 avocado.app cibuilder L0222 > ERROR| linux-kbuild-amd64 > [stdlog] 2024-04-12 16:23:10,235 avocado.test cibuilder L0220 > INFO | > [stdlog] 2024-04-12 16:23:10,235 avocado.app cibuilder L0222 > ERROR| linux-kbuild-cip-native > [stdlog] 2024-04-12 16:23:10,235 avocado.app cibuilder L0222 > ERROR| linux-kbuild-mainline-native > [stdlog] 2024-04-12 16:23:10,235 avocado.app cibuilder L0222 > ERROR| ERROR: Required build target 'isar-image-ci' has no buildable > providers. > > I tried to reproduce it locally, and the easiest way to reproduce was > the following: > > ``` > ./kas/kas-container menu > <Leave everything as is, just select "Save & Build"> > ... > Parsing recipes: 100% > > ################################################################### > > ### > #########################################################| Time: > 0:00:00 > Parsing of 55 .bb files complete (0 cached, 55 parsed). 57 targets, > 15 > skipped, 0 masked, 0 errors. > NOTE: Resolving any missing task queue dependencies > ERROR: Nothing PROVIDES 'linux-kbuild-amd64-native' (but > /build/../repo/meta-isar/recipes-kernel/example-module/example- > module.bb DEPENDS on or otherwise requires it). Close matches: > linux-kbuild-amd64 > linux-kbuild-cip-native > linux-kbuild-mainline-native > ERROR: Required build target 'isar-image-base' has no buildable > providers. > Missing or unbuildable dependency chain was: ['isar-image-base', > 'example-module-amd64', 'linux-kbuild-amd64-native'] > ``` > This is caused by commit: "linux-module: Support emulated module build with cross-compiled kernel" It should be possible to skip this commit for the first time, fix it meanwhile and apply this (then fixed) commit later. > I also tried the patchset with older Isar (e.g. just some commits > before v0.10, when the patchset was released) and the situation looks > the same. > > When we remove "example-module" from list of packages to install, > build > is OK. > Thanks and regards
On Wed, 2024-02-14 at 11:10 +0100, 'Stefan Koch' via isar-users wrote: > Hi > > This updated v6 patchset implement suggestions from reviewed v4 > patchset. > It's now a set of five patches: > - linux-custom: Set PROVIDES variable using bitbake overrides > - linux-custom: Split up binaries from kernel headers to kbuild > package > - linux-custom: Provide target and host specific kernel kbuild > packages > - linux-module: Support emulated module build with cross-compiled > kernel > - docs: Update custom_kernel docs for split up of kernel scripts and > tools > > The first commit sets the PROVIDES variable using > bitbake overrides instead of python code. > > Swap out the binaries from the kernel headers > into kernel kbuild package is the main use-case > introduced by the second commit > "Split up binaries from kernel headers to kbuild package" > > The third commit "Provide target and host specific kernel kbuild > packages" > introduces that the binaries could be swapped out into host and > target > specific kernel kbuild packages. > > Support of emulated module build with a cross-compiled kernel build > is introduced by the fourth commit. > > These are the main development goals: > > 1. Solve already known isar custom kernel > limitations from doc/custom_kernel.inc > - kernel headers package does not support both native > and cross compilation of kernel modules when cross built > > 2. Honor recommendations for future from doc/custom_kernel.inc > - Generate kernel headers packages for both host and target > when using cross build > > 3. Add extensions known from debian kernel packages structure > - Generate a kernel headers package without binaries > - Create specific kernel kbuild packages that > will ship the "scripts" and "tools" binaries > - Use symlinks to point to the "scripts" and "tools" binaries > > 4. Be user friendly > - Avoid redundant configuration of kernel source definitions with > user > actions to enable kbuild package generation > - Use already known way to include linux-custom.inc in just one > own bitbake recipe that provides the kernel source definitions > - Keep known user behavior for existing build configurations: > just update isar, and use the new linux-kbuild bitbake target > to create the kbuild target package when cross building > > Best regards > > Stefan > > Stefan Koch (5): > linux-custom: Set PROVIDES variable using bitbake overrides > linux-custom: Split up binaries from kernel headers to kbuild > package > linux-custom: Provide target and host specific kernel kbuild > packages > linux-module: Support emulated module build with cross-compiled > kernel > docs: Update custom_kernel docs for split up of kernel scripts and > tools > > RECIPE-API-CHANGELOG.md | 31 ++++++++ > doc/custom_kernel.md | 18 +++-- > meta/recipes-kernel/linux-module/module.inc | 3 +- > .../linux/classes/kbuildtarget.bbclass | 8 ++ > .../linux/files/debian/control.tmpl | 15 +++- > .../linux/files/debian/isar/build.tmpl | 13 +++- > .../linux/files/debian/isar/common.tmpl | 11 +++ > .../linux/files/debian/isar/install.tmpl | 75 ++++++++++++++--- > -- > meta/recipes-kernel/linux/linux-custom.inc | 65 +++++++++++++--- > meta/recipes-kernel/linux/linux-distro.bb | 1 + > 10 files changed, 199 insertions(+), 41 deletions(-) > create mode 100644 meta/recipes- > kernel/linux/classes/kbuildtarget.bbclass > > -- > 2.39.2 > Applied to next, excluding patch 4 (causing build failures on amd64 arch), as discussed on maillist. Thanks.
Hi This updated v6 patchset implement suggestions from reviewed v4 patchset. It's now a set of five patches: - linux-custom: Set PROVIDES variable using bitbake overrides - linux-custom: Split up binaries from kernel headers to kbuild package - linux-custom: Provide target and host specific kernel kbuild packages - linux-module: Support emulated module build with cross-compiled kernel - docs: Update custom_kernel docs for split up of kernel scripts and tools The first commit sets the PROVIDES variable using bitbake overrides instead of python code. Swap out the binaries from the kernel headers into kernel kbuild package is the main use-case introduced by the second commit "Split up binaries from kernel headers to kbuild package" The third commit "Provide target and host specific kernel kbuild packages" introduces that the binaries could be swapped out into host and target specific kernel kbuild packages. Support of emulated module build with a cross-compiled kernel build is introduced by the fourth commit. These are the main development goals: 1. Solve already known isar custom kernel limitations from doc/custom_kernel.inc - kernel headers package does not support both native and cross compilation of kernel modules when cross built 2. Honor recommendations for future from doc/custom_kernel.inc - Generate kernel headers packages for both host and target when using cross build 3. Add extensions known from debian kernel packages structure - Generate a kernel headers package without binaries - Create specific kernel kbuild packages that will ship the "scripts" and "tools" binaries - Use symlinks to point to the "scripts" and "tools" binaries 4. Be user friendly - Avoid redundant configuration of kernel source definitions with user actions to enable kbuild package generation - Use already known way to include linux-custom.inc in just one own bitbake recipe that provides the kernel source definitions - Keep known user behavior for existing build configurations: just update isar, and use the new linux-kbuild bitbake target to create the kbuild target package when cross building Best regards Stefan Stefan Koch (5): linux-custom: Set PROVIDES variable using bitbake overrides linux-custom: Split up binaries from kernel headers to kbuild package linux-custom: Provide target and host specific kernel kbuild packages linux-module: Support emulated module build with cross-compiled kernel docs: Update custom_kernel docs for split up of kernel scripts and tools RECIPE-API-CHANGELOG.md | 31 ++++++++ doc/custom_kernel.md | 18 +++-- meta/recipes-kernel/linux-module/module.inc | 3 +- .../linux/classes/kbuildtarget.bbclass | 8 ++ .../linux/files/debian/control.tmpl | 15 +++- .../linux/files/debian/isar/build.tmpl | 13 +++- .../linux/files/debian/isar/common.tmpl | 11 +++ .../linux/files/debian/isar/install.tmpl | 75 ++++++++++++++----- meta/recipes-kernel/linux/linux-custom.inc | 65 +++++++++++++--- meta/recipes-kernel/linux/linux-distro.bb | 1 + 10 files changed, 199 insertions(+), 41 deletions(-) create mode 100644 meta/recipes-kernel/linux/classes/kbuildtarget.bbclass