mbox series

[v4,0/3] linux-custom: Split up binaries from kernel headers to kbuild packages

Message ID 20230821134501.2681654-1-stefan-koch@siemens.com
Headers show
Series linux-custom: Split up binaries from kernel headers to kbuild packages | expand

Message

Koch, Stefan Aug. 21, 2023, 1:44 p.m. UTC
Hi

This updated v4 patchset implement suggestions from reviewed v3 patchset.
It's now a set of five patches:
- linux-custom: Split up binaries from kernel headers to kbuild package
- linux-custom: Provide host and target specific kernel kbuild packages
- docs: Update custom_kernel docs for split up of kernel scripts and tools

The main use-case was to swap out the binaries
from the kernel headers into kernel kbuild package.
This is introduced by the first commit
"Split up binaries from kernel headers to kbuild package"

The second commit "Provide host and target specific kernel kbuild packages"
introduces that the binaries could be swapped out into host and target
specific kernel kbuild packages.

The main development goals were these:

1. Solve already known isar custom kernel
limitations from doc/custom_kernel.inc
- kernel headers package does not support both native
  and cross compilation of kernel modules when cross built

2. Honor recommendations for future from doc/custom_kernel.inc
- Generate kernel headers packages for both host and target
  when using cross build

3. Add extensions known from debian kernel packages structure
- Generate a kernel headers package without binaries
- Create specific kernel kbuild packages that
  will ship the "scripts" and "tools" binaries
- Use symlinks to point to the "scripts" and "tools" binaries

4. Be user friendly
- Avoid redundant configuration of kernel source definitions with user
  actions to enable kbuild package generation
- Use already known way to include linux-custom.inc in just one
  own bitbake recipe that provides the kernel source definitions
- Keep known user behavior for existing build configurations:
  just update isar, request "-compat" (and "-native") bitbake targets
  and kbuild packages for target and host will be created automatically

Best regards

Stefan

Stefan Koch (3):
  linux-custom: Split up binaries from kernel headers to kbuild package
  linux-custom: Provide host and target specific kernel kbuild packages
  docs: Update custom_kernel docs for split up of kernel scripts and
    tools

 doc/custom_kernel.md                          | 16 ++--
 .../linux/files/debian/control.tmpl           | 25 ++++++-
 .../linux/files/debian/isar/build.tmpl        | 12 ++-
 .../linux/files/debian/isar/common.tmpl       | 10 +++
 .../linux/files/debian/isar/install.tmpl      | 73 ++++++++++++++-----
 .../linux/files/debian/rules.tmpl             |  2 +-
 meta/recipes-kernel/linux/linux-custom.inc    | 53 ++++++++++++--
 7 files changed, 155 insertions(+), 36 deletions(-)

Comments

Uladzimir Bely Sept. 1, 2023, 10:22 a.m. UTC | #1
On Mon, 2023-08-21 at 15:44 +0200, Stefan Koch wrote:
> Hi
> 
> This updated v4 patchset implement suggestions from reviewed v3
> patchset.
> It's now a set of five patches:
> - linux-custom: Split up binaries from kernel headers to kbuild
> package
> - linux-custom: Provide host and target specific kernel kbuild
> packages
> - docs: Update custom_kernel docs for split up of kernel scripts and
> tools
> 
> The main use-case was to swap out the binaries
> from the kernel headers into kernel kbuild package.
> This is introduced by the first commit
> "Split up binaries from kernel headers to kbuild package"
> 
> The second commit "Provide host and target specific kernel kbuild
> packages"
> introduces that the binaries could be swapped out into host and
> target
> specific kernel kbuild packages.
> 
> The main development goals were these:
> 
> 1. Solve already known isar custom kernel
> limitations from doc/custom_kernel.inc
> - kernel headers package does not support both native
>   and cross compilation of kernel modules when cross built
> 
> 2. Honor recommendations for future from doc/custom_kernel.inc
> - Generate kernel headers packages for both host and target
>   when using cross build
> 
> 3. Add extensions known from debian kernel packages structure
> - Generate a kernel headers package without binaries
> - Create specific kernel kbuild packages that
>   will ship the "scripts" and "tools" binaries
> - Use symlinks to point to the "scripts" and "tools" binaries
> 
> 4. Be user friendly
> - Avoid redundant configuration of kernel source definitions with
> user
>   actions to enable kbuild package generation
> - Use already known way to include linux-custom.inc in just one
>   own bitbake recipe that provides the kernel source definitions
> - Keep known user behavior for existing build configurations:
>   just update isar, request "-compat" (and "-native") bitbake targets
>   and kbuild packages for target and host will be created
> automatically
> 
> Best regards
> 
> Stefan
> 
> Stefan Koch (3):
>   linux-custom: Split up binaries from kernel headers to kbuild
> package
>   linux-custom: Provide host and target specific kernel kbuild
> packages
>   docs: Update custom_kernel docs for split up of kernel scripts and
>     tools
> 
>  doc/custom_kernel.md                          | 16 ++--
>  .../linux/files/debian/control.tmpl           | 25 ++++++-
>  .../linux/files/debian/isar/build.tmpl        | 12 ++-
>  .../linux/files/debian/isar/common.tmpl       | 10 +++
>  .../linux/files/debian/isar/install.tmpl      | 73 ++++++++++++++---
> --
>  .../linux/files/debian/rules.tmpl             |  2 +-
>  meta/recipes-kernel/linux/linux-custom.inc    | 53 ++++++++++++--
>  7 files changed, 155 insertions(+), 36 deletions(-)
> 

This passes fast/full CI, so we tend to apply it next week, if there
are no objections.
Jan Kiszka Sept. 5, 2023, 8:17 p.m. UTC | #2
On 21.08.23 15:44, Stefan Koch wrote:
> Hi
> 
> This updated v4 patchset implement suggestions from reviewed v3 patchset.
> It's now a set of five patches:
> - linux-custom: Split up binaries from kernel headers to kbuild package
> - linux-custom: Provide host and target specific kernel kbuild packages
> - docs: Update custom_kernel docs for split up of kernel scripts and tools
> 
> The main use-case was to swap out the binaries
> from the kernel headers into kernel kbuild package.
> This is introduced by the first commit
> "Split up binaries from kernel headers to kbuild package"
> 
> The second commit "Provide host and target specific kernel kbuild packages"
> introduces that the binaries could be swapped out into host and target
> specific kernel kbuild packages.
> 
> The main development goals were these:
> 
> 1. Solve already known isar custom kernel
> limitations from doc/custom_kernel.inc
> - kernel headers package does not support both native
>   and cross compilation of kernel modules when cross built
> 
> 2. Honor recommendations for future from doc/custom_kernel.inc
> - Generate kernel headers packages for both host and target
>   when using cross build
> 
> 3. Add extensions known from debian kernel packages structure
> - Generate a kernel headers package without binaries
> - Create specific kernel kbuild packages that
>   will ship the "scripts" and "tools" binaries
> - Use symlinks to point to the "scripts" and "tools" binaries
> 
> 4. Be user friendly
> - Avoid redundant configuration of kernel source definitions with user
>   actions to enable kbuild package generation
> - Use already known way to include linux-custom.inc in just one
>   own bitbake recipe that provides the kernel source definitions
> - Keep known user behavior for existing build configurations:
>   just update isar, request "-compat" (and "-native") bitbake targets
>   and kbuild packages for target and host will be created automatically

Most things look fairly good, however - maybe it's just too late today -
I'm not yet getting the scenario where -compat plays a role. My
impression is right now, you are misusing this for some scenario it is
not designed for (compat = 32-bit package for 64-bit target). Or am I
misinterpreting this?

Jan
Jan Kiszka Sept. 5, 2023, 8:18 p.m. UTC | #3
On 01.09.23 12:22, Uladzimir Bely wrote:
> On Mon, 2023-08-21 at 15:44 +0200, Stefan Koch wrote:
>> Hi
>>
>> This updated v4 patchset implement suggestions from reviewed v3
>> patchset.
>> It's now a set of five patches:
>> - linux-custom: Split up binaries from kernel headers to kbuild
>> package
>> - linux-custom: Provide host and target specific kernel kbuild
>> packages
>> - docs: Update custom_kernel docs for split up of kernel scripts and
>> tools
>>
>> The main use-case was to swap out the binaries
>> from the kernel headers into kernel kbuild package.
>> This is introduced by the first commit
>> "Split up binaries from kernel headers to kbuild package"
>>
>> The second commit "Provide host and target specific kernel kbuild
>> packages"
>> introduces that the binaries could be swapped out into host and
>> target
>> specific kernel kbuild packages.
>>
>> The main development goals were these:
>>
>> 1. Solve already known isar custom kernel
>> limitations from doc/custom_kernel.inc
>> - kernel headers package does not support both native
>>   and cross compilation of kernel modules when cross built
>>
>> 2. Honor recommendations for future from doc/custom_kernel.inc
>> - Generate kernel headers packages for both host and target
>>   when using cross build
>>
>> 3. Add extensions known from debian kernel packages structure
>> - Generate a kernel headers package without binaries
>> - Create specific kernel kbuild packages that
>>   will ship the "scripts" and "tools" binaries
>> - Use symlinks to point to the "scripts" and "tools" binaries
>>
>> 4. Be user friendly
>> - Avoid redundant configuration of kernel source definitions with
>> user
>>   actions to enable kbuild package generation
>> - Use already known way to include linux-custom.inc in just one
>>   own bitbake recipe that provides the kernel source definitions
>> - Keep known user behavior for existing build configurations:
>>   just update isar, request "-compat" (and "-native") bitbake targets
>>   and kbuild packages for target and host will be created
>> automatically
>>
>> Best regards
>>
>> Stefan
>>
>> Stefan Koch (3):
>>   linux-custom: Split up binaries from kernel headers to kbuild
>> package
>>   linux-custom: Provide host and target specific kernel kbuild
>> packages
>>   docs: Update custom_kernel docs for split up of kernel scripts and
>>     tools
>>
>>  doc/custom_kernel.md                          | 16 ++--
>>  .../linux/files/debian/control.tmpl           | 25 ++++++-
>>  .../linux/files/debian/isar/build.tmpl        | 12 ++-
>>  .../linux/files/debian/isar/common.tmpl       | 10 +++
>>  .../linux/files/debian/isar/install.tmpl      | 73 ++++++++++++++---
>> --
>>  .../linux/files/debian/rules.tmpl             |  2 +-
>>  meta/recipes-kernel/linux/linux-custom.inc    | 53 ++++++++++++--
>>  7 files changed, 155 insertions(+), 36 deletions(-)
>>
> 
> This passes fast/full CI, so we tend to apply it next week, if there
> are no objections.

I have some remaining question on the -compat scenario, but I'm getting
an OOO notice (internally) from Stefan. Unless someone else can answer
that question, we may need to wait until he returned.

Jan
Koch, Stefan Oct. 4, 2023, 3:36 p.m. UTC | #4
On Tue, 2023-09-05 at 22:17 +0200, Jan Kiszka wrote:
> On 21.08.23 15:44, Stefan Koch wrote:
> > Hi
> > 
> > This updated v4 patchset implement suggestions from reviewed v3
> > patchset.
> > It's now a set of five patches:
> > - linux-custom: Split up binaries from kernel headers to kbuild
> > package
> > - linux-custom: Provide host and target specific kernel kbuild
> > packages
> > - docs: Update custom_kernel docs for split up of kernel scripts
> > and tools
> > 
> > The main use-case was to swap out the binaries
> > from the kernel headers into kernel kbuild package.
> > This is introduced by the first commit
> > "Split up binaries from kernel headers to kbuild package"
> > 
> > The second commit "Provide host and target specific kernel kbuild
> > packages"
> > introduces that the binaries could be swapped out into host and
> > target
> > specific kernel kbuild packages.
> > 
> > The main development goals were these:
> > 
> > 1. Solve already known isar custom kernel
> > limitations from doc/custom_kernel.inc
> > - kernel headers package does not support both native
> >   and cross compilation of kernel modules when cross built
> > 
> > 2. Honor recommendations for future from doc/custom_kernel.inc
> > - Generate kernel headers packages for both host and target
> >   when using cross build
> > 
> > 3. Add extensions known from debian kernel packages structure
> > - Generate a kernel headers package without binaries
> > - Create specific kernel kbuild packages that
> >   will ship the "scripts" and "tools" binaries
> > - Use symlinks to point to the "scripts" and "tools" binaries
> > 
> > 4. Be user friendly
> > - Avoid redundant configuration of kernel source definitions with
> > user
> >   actions to enable kbuild package generation
> > - Use already known way to include linux-custom.inc in just one
> >   own bitbake recipe that provides the kernel source definitions
> > - Keep known user behavior for existing build configurations:
> >   just update isar, request "-compat" (and "-native") bitbake
> > targets
> >   and kbuild packages for target and host will be created
> > automatically
> 
> Most things look fairly good, however - maybe it's just too late
> today -
> I'm not yet getting the scenario where -compat plays a role. My
> impression is right now, you are misusing this for some scenario it
> is
> not designed for (compat = 32-bit package for 64-bit target). Or am I
> misinterpreting this?

Yes, compat is used for the target build. See last answer for [PATCH v4
2/3].

> 
> Jan
>
Uladzimir Bely Nov. 21, 2023, 7:26 a.m. UTC | #5
On Mon, 2023-08-21 at 15:44 +0200, Stefan Koch wrote:
> Hi
> 
> This updated v4 patchset implement suggestions from reviewed v3
> patchset.
> It's now a set of five patches:
> - linux-custom: Split up binaries from kernel headers to kbuild
> package
> - linux-custom: Provide host and target specific kernel kbuild
> packages
> - docs: Update custom_kernel docs for split up of kernel scripts and
> tools
> 
> The main use-case was to swap out the binaries
> from the kernel headers into kernel kbuild package.
> This is introduced by the first commit
> "Split up binaries from kernel headers to kbuild package"
> 
> The second commit "Provide host and target specific kernel kbuild
> packages"
> introduces that the binaries could be swapped out into host and
> target
> specific kernel kbuild packages.
> 
> The main development goals were these:
> 
> 1. Solve already known isar custom kernel
> limitations from doc/custom_kernel.inc
> - kernel headers package does not support both native
>   and cross compilation of kernel modules when cross built
> 
> 2. Honor recommendations for future from doc/custom_kernel.inc
> - Generate kernel headers packages for both host and target
>   when using cross build
> 
> 3. Add extensions known from debian kernel packages structure
> - Generate a kernel headers package without binaries
> - Create specific kernel kbuild packages that
>   will ship the "scripts" and "tools" binaries
> - Use symlinks to point to the "scripts" and "tools" binaries
> 
> 4. Be user friendly
> - Avoid redundant configuration of kernel source definitions with
> user
>   actions to enable kbuild package generation
> - Use already known way to include linux-custom.inc in just one
>   own bitbake recipe that provides the kernel source definitions
> - Keep known user behavior for existing build configurations:
>   just update isar, request "-compat" (and "-native") bitbake targets
>   and kbuild packages for target and host will be created
> automatically
> 
> Best regards
> 
> Stefan
> 
> Stefan Koch (3):
>   linux-custom: Split up binaries from kernel headers to kbuild
> package
>   linux-custom: Provide host and target specific kernel kbuild
> packages
>   docs: Update custom_kernel docs for split up of kernel scripts and
>     tools
> 
>  doc/custom_kernel.md                          | 16 ++--
>  .../linux/files/debian/control.tmpl           | 25 ++++++-
>  .../linux/files/debian/isar/build.tmpl        | 12 ++-
>  .../linux/files/debian/isar/common.tmpl       | 10 +++
>  .../linux/files/debian/isar/install.tmpl      | 73 ++++++++++++++---
> --
>  .../linux/files/debian/rules.tmpl             |  2 +-
>  meta/recipes-kernel/linux/linux-custom.inc    | 53 ++++++++++++--
>  7 files changed, 155 insertions(+), 36 deletions(-)
> 

Hello.

Patch 2 brough up a discussion, but patch 1 itself looks OK.

According to internal discussion, we could apply patch 1 before "v0.10"
released (with documentation changes from patch 3), if it's not too
risky for being in new release without response from multiple
downstreams...

Anyway, since it's not applicable on top of current 'next', it needs
rebasing.