meta/classes/crossvars: move sdk toolchain selection from python block

Message ID 20230604093216.1094289-1-srinuvasan_a@mentor.com
State Changes Requested
Headers show
Series meta/classes/crossvars: move sdk toolchain selection from python block | expand

Commit Message

Srinuvasan Arjunan June 4, 2023, 9:32 a.m. UTC
From: Srinuvasan A <srinuvasan.a@siemens.com>

In the present implementation we are not able to override the SDK
toolchain from downstream layer, this is due to the SDK toolchain
selection part in anonymous python function.

Anonymous python functions always run at the end of parsing, regardless of where they are defined
even when we do override in our recipe, always wins the Anonymous Python
functions variable settings, hence we are not able to override our
downstream toolchain.

Move the SDK toolchian selection from python block, now we can able to
override from downstream layer.

Signed-off-by: Srinuvasan A <srinuvasan.a@siemens.com>
---
 meta/classes/crossvars.bbclass | 9 ++++-----
 1 file changed, 4 insertions(+), 5 deletions(-)

Comments

Jan Kiszka June 5, 2023, 7 a.m. UTC | #1
On 04.06.23 11:32, Srinuvasan Arjunan wrote:
> From: Srinuvasan A <srinuvasan.a@siemens.com>
> 
> In the present implementation we are not able to override the SDK
> toolchain from downstream layer, this is due to the SDK toolchain
> selection part in anonymous python function.
> 
> Anonymous python functions always run at the end of parsing, regardless of where they are defined
> even when we do override in our recipe, always wins the Anonymous Python
> functions variable settings, hence we are not able to override our
> downstream toolchain.
> 
> Move the SDK toolchian selection from python block, now we can able to
> override from downstream layer.
> 
> Signed-off-by: Srinuvasan A <srinuvasan.a@siemens.com>
> ---
>  meta/classes/crossvars.bbclass | 9 ++++-----
>  1 file changed, 4 insertions(+), 5 deletions(-)
> 
> diff --git a/meta/classes/crossvars.bbclass b/meta/classes/crossvars.bbclass
> index 201d460..120e6d1 100644
> --- a/meta/classes/crossvars.bbclass
> +++ b/meta/classes/crossvars.bbclass
> @@ -5,6 +5,10 @@ ISAR_CROSS_COMPILE ??= "0"
>  
>  inherit compat
>  
> +SDK_TOOLCHAIN = "${@'build-essential' if d.getVar('ISAR_CROSS_COMPILE') == '0' or d.getVar('HOST_ARCH') == d.getVar('DISTRO_ARCH') or d.getVar('DISTRO_ARCH') == None else 'crossbuild-essential-${DISTRO_ARCH}'}"
> +
> +SDK_TOOLCHAIN:append = "${@' crossbuild-essential-${COMPAT_DISTRO_ARCH}' if isar_can_build_compat(d) else ''}"
> +

This is still no weak assignment which you will need for overrides. Did
you actually test this? And, again, can't we implement the weak
assignment in python below?

Jan

>  python __anonymous() {
>      import pwd
>      d.setVar('SCHROOT_USER', pwd.getpwuid(os.geteuid()).pw_name)
> @@ -17,19 +21,14 @@ python __anonymous() {
>          sbuild_dep = "sbuild-chroot-target:do_build"
>          buildchroot_dir = d.getVar('BUILDCHROOT_TARGET_DIR', False)
>          buildchroot_dep = "buildchroot-target:do_build"
> -        sdk_toolchain = "build-essential"
>      else:
>          d.setVar('BUILD_HOST_ARCH', d.getVar('HOST_ARCH'))
>          schroot_dir = d.getVar('SCHROOT_HOST_DIR', False)
>          sbuild_dep = "sbuild-chroot-host:do_build"
>          buildchroot_dir = d.getVar('BUILDCHROOT_HOST_DIR', False)
>          buildchroot_dep = "buildchroot-host:do_build"
> -        sdk_toolchain = "crossbuild-essential-" + distro_arch
>      d.setVar('SCHROOT_DIR', schroot_dir)
>      d.setVar('SCHROOT_DEP', sbuild_dep)
>      d.setVar('BUILDCHROOT_DIR', buildchroot_dir)
>      d.setVar('BUILDCHROOT_DEP', buildchroot_dep)
> -    if isar_can_build_compat(d):
> -        sdk_toolchain += " crossbuild-essential-" + d.getVar('COMPAT_DISTRO_ARCH')
> -    d.setVar('SDK_TOOLCHAIN', sdk_toolchain)
>  }
Srinuvasan Arjunan June 5, 2023, 7:07 a.m. UTC | #2
Yes tested working fine, here i do the append overrides in downstream 
layer, like how the previous implementation before things move into the 
python block.

https://github.com/ilbers/isar/commit/43f402f4862c3779a492c50a26475ea1221a2b01

Now in present implementation comapt-arch override was removed and this 
selection of compat will be handled by  ISAR_ENABLE_COMPAT_ARCH 

Thanks,
Srinu

On Monday, June 5, 2023 at 12:31:01 PM UTC+5:30 Jan Kiszka wrote:

> On 04.06.23 11:32, Srinuvasan Arjunan wrote:
> > From: Srinuvasan A <srinuv...@siemens.com>
> > 
> > In the present implementation we are not able to override the SDK
> > toolchain from downstream layer, this is due to the SDK toolchain
> > selection part in anonymous python function.
> > 
> > Anonymous python functions always run at the end of parsing, regardless 
> of where they are defined
> > even when we do override in our recipe, always wins the Anonymous Python
> > functions variable settings, hence we are not able to override our
> > downstream toolchain.
> > 
> > Move the SDK toolchian selection from python block, now we can able to
> > override from downstream layer.
> > 
> > Signed-off-by: Srinuvasan A <srinuv...@siemens.com>
> > ---
> > meta/classes/crossvars.bbclass | 9 ++++-----
> > 1 file changed, 4 insertions(+), 5 deletions(-)
> > 
> > diff --git a/meta/classes/crossvars.bbclass 
> b/meta/classes/crossvars.bbclass
> > index 201d460..120e6d1 100644
> > --- a/meta/classes/crossvars.bbclass
> > +++ b/meta/classes/crossvars.bbclass
> > @@ -5,6 +5,10 @@ ISAR_CROSS_COMPILE ??= "0"
> > 
> > inherit compat
> > 
> > +SDK_TOOLCHAIN = "${@'build-essential' if d.getVar('ISAR_CROSS_COMPILE') 
> == '0' or d.getVar('HOST_ARCH') == d.getVar('DISTRO_ARCH') or 
> d.getVar('DISTRO_ARCH') == None else 'crossbuild-essential-${DISTRO_ARCH}'}"
> > +
> > +SDK_TOOLCHAIN:append = "${@' 
> crossbuild-essential-${COMPAT_DISTRO_ARCH}' if isar_can_build_compat(d) 
> else ''}"
> > +
>
> This is still no weak assignment which you will need for overrides. Did
> you actually test this? And, again, can't we implement the weak
> assignment in python below?
>
> Jan
>
> > python __anonymous() {
> > import pwd
> > d.setVar('SCHROOT_USER', pwd.getpwuid(os.geteuid()).pw_name)
> > @@ -17,19 +21,14 @@ python __anonymous() {
> > sbuild_dep = "sbuild-chroot-target:do_build"
> > buildchroot_dir = d.getVar('BUILDCHROOT_TARGET_DIR', False)
> > buildchroot_dep = "buildchroot-target:do_build"
> > - sdk_toolchain = "build-essential"
> > else:
> > d.setVar('BUILD_HOST_ARCH', d.getVar('HOST_ARCH'))
> > schroot_dir = d.getVar('SCHROOT_HOST_DIR', False)
> > sbuild_dep = "sbuild-chroot-host:do_build"
> > buildchroot_dir = d.getVar('BUILDCHROOT_HOST_DIR', False)
> > buildchroot_dep = "buildchroot-host:do_build"
> > - sdk_toolchain = "crossbuild-essential-" + distro_arch
> > d.setVar('SCHROOT_DIR', schroot_dir)
> > d.setVar('SCHROOT_DEP', sbuild_dep)
> > d.setVar('BUILDCHROOT_DIR', buildchroot_dir)
> > d.setVar('BUILDCHROOT_DEP', buildchroot_dep)
> > - if isar_can_build_compat(d):
> > - sdk_toolchain += " crossbuild-essential-" + 
> d.getVar('COMPAT_DISTRO_ARCH')
> > - d.setVar('SDK_TOOLCHAIN', sdk_toolchain)
> > }
>
> -- 
> Siemens AG, Technology
> Competence Center Embedded Linux
>
>
Jan Kiszka June 5, 2023, 7:18 a.m. UTC | #3
On 05.06.23 09:07, Srinuvasan Arjunan wrote:
> Yes tested working fine, here i do the append overrides in downstream
> layer, like how the previous implementation before things move into the
> python block.
> 
> https://github.com/ilbers/isar/commit/43f402f4862c3779a492c50a26475ea1221a2b01
> 
> Now in present implementation comapt-arch override was removed and this
> selection of compat will be handled by  ISAR_ENABLE_COMPAT_ARCH

So, d.setVar blocks :append and :remove, right? You don't want to set it
downstream completely?

But I dislike this long inline python, specifically as we already have
the logic more readable. How about using an intermediate variable then
which is defined in the anonymous python block while SDK_TOOLCHAIN gets
that value by default outside the block?

Or - to come back to your original issue - we make that use case of
multilib officially an SDK toolchain option so that you just need to
flip a switch, not fiddle with the internals of SDK_TOOLCHAIN.

Jan

> 
> Thanks,
> Srinu
> 
> On Monday, June 5, 2023 at 12:31:01 PM UTC+5:30 Jan Kiszka wrote:
> 
>     On 04.06.23 11:32, Srinuvasan Arjunan wrote:
>     > From: Srinuvasan A <srinuv...@siemens.com>
>     >
>     > In the present implementation we are not able to override the SDK
>     > toolchain from downstream layer, this is due to the SDK toolchain
>     > selection part in anonymous python function.
>     >
>     > Anonymous python functions always run at the end of parsing,
>     regardless of where they are defined
>     > even when we do override in our recipe, always wins the Anonymous
>     Python
>     > functions variable settings, hence we are not able to override our
>     > downstream toolchain.
>     >
>     > Move the SDK toolchian selection from python block, now we can
>     able to
>     > override from downstream layer.
>     >
>     > Signed-off-by: Srinuvasan A <srinuv...@siemens.com>
>     > ---
>     > meta/classes/crossvars.bbclass | 9 ++++-----
>     > 1 file changed, 4 insertions(+), 5 deletions(-)
>     >
>     > diff --git a/meta/classes/crossvars.bbclass
>     b/meta/classes/crossvars.bbclass
>     > index 201d460..120e6d1 100644
>     > --- a/meta/classes/crossvars.bbclass
>     > +++ b/meta/classes/crossvars.bbclass
>     > @@ -5,6 +5,10 @@ ISAR_CROSS_COMPILE ??= "0"
>     >
>     > inherit compat
>     >
>     > +SDK_TOOLCHAIN = "${@'build-essential' if
>     d.getVar('ISAR_CROSS_COMPILE') == '0' or d.getVar('HOST_ARCH') ==
>     d.getVar('DISTRO_ARCH') or d.getVar('DISTRO_ARCH') == None else
>     'crossbuild-essential-${DISTRO_ARCH}'}"
>     > +
>     > +SDK_TOOLCHAIN:append = "${@'
>     crossbuild-essential-${COMPAT_DISTRO_ARCH}' if
>     isar_can_build_compat(d) else ''}"
>     > +
> 
>     This is still no weak assignment which you will need for overrides. Did
>     you actually test this? And, again, can't we implement the weak
>     assignment in python below?
> 
>     Jan
> 
>     > python __anonymous() {
>     > import pwd
>     > d.setVar('SCHROOT_USER', pwd.getpwuid(os.geteuid()).pw_name)
>     > @@ -17,19 +21,14 @@ python __anonymous() {
>     > sbuild_dep = "sbuild-chroot-target:do_build"
>     > buildchroot_dir = d.getVar('BUILDCHROOT_TARGET_DIR', False)
>     > buildchroot_dep = "buildchroot-target:do_build"
>     > - sdk_toolchain = "build-essential"
>     > else:
>     > d.setVar('BUILD_HOST_ARCH', d.getVar('HOST_ARCH'))
>     > schroot_dir = d.getVar('SCHROOT_HOST_DIR', False)
>     > sbuild_dep = "sbuild-chroot-host:do_build"
>     > buildchroot_dir = d.getVar('BUILDCHROOT_HOST_DIR', False)
>     > buildchroot_dep = "buildchroot-host:do_build"
>     > - sdk_toolchain = "crossbuild-essential-" + distro_arch
>     > d.setVar('SCHROOT_DIR', schroot_dir)
>     > d.setVar('SCHROOT_DEP', sbuild_dep)
>     > d.setVar('BUILDCHROOT_DIR', buildchroot_dir)
>     > d.setVar('BUILDCHROOT_DEP', buildchroot_dep)
>     > - if isar_can_build_compat(d):
>     > - sdk_toolchain += " crossbuild-essential-" +
>     d.getVar('COMPAT_DISTRO_ARCH')
>     > - d.setVar('SDK_TOOLCHAIN', sdk_toolchain)
>     > }
> 
>     -- 
>     Siemens AG, Technology
>     Competence Center Embedded Linux
> 
> -- 
> You received this message because you are subscribed to the Google
> Groups "isar-users" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to isar-users+unsubscribe@googlegroups.com
> <mailto:isar-users+unsubscribe@googlegroups.com>.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/isar-users/3f742bff-1ccd-4a88-8c96-8cab068b8eb4n%40googlegroups.com <https://groups.google.com/d/msgid/isar-users/3f742bff-1ccd-4a88-8c96-8cab068b8eb4n%40googlegroups.com?utm_medium=email&utm_source=footer>.
Srinuvasan Arjunan June 5, 2023, 7:43 a.m. UTC | #4
On Monday, June 5, 2023 at 12:48:34 PM UTC+5:30 Jan Kiszka wrote:

On 05.06.23 09:07, Srinuvasan Arjunan wrote: 
> Yes tested working fine, here i do the append overrides in downstream 
> layer, like how the previous implementation before things move into the 
> python block. 
> 
> 
https://github.com/ilbers/isar/commit/43f402f4862c3779a492c50a26475ea1221a2b01 
> 
> Now in present implementation comapt-arch override was removed and this 
> selection of compat will be handled by  ISAR_ENABLE_COMPAT_ARCH 

So, d.setVar blocks :append and :remove, right?


         yes, correct. 

You don't want to set it 
downstream completely? 

But I dislike this long inline python, specifically as we already have 
the logic more readable. How about using an intermediate variable then 
which is defined in the anonymous python block while SDK_TOOLCHAIN gets 
that value by default outside the block?


  We could do this way. 



Or - to come back to your original issue - we make that use case of 
multilib officially an SDK toolchain option so that you just need to 
flip a switch, not fiddle with the internals of SDK_TOOLCHAIN.


  Yes , this also works , i will come back with more details. 



Jan 

> 
> Thanks, 
> Srinu 
> 
> On Monday, June 5, 2023 at 12:31:01 PM UTC+5:30 Jan Kiszka wrote: 
> 
> On 04.06.23 11:32, Srinuvasan Arjunan wrote: 
> > From: Srinuvasan A <srinuv...@siemens.com> 
> > 
> > In the present implementation we are not able to override the SDK 
> > toolchain from downstream layer, this is due to the SDK toolchain 
> > selection part in anonymous python function. 
> > 
> > Anonymous python functions always run at the end of parsing, 
> regardless of where they are defined 
> > even when we do override in our recipe, always wins the Anonymous 
> Python 
> > functions variable settings, hence we are not able to override our 
> > downstream toolchain. 
> > 
> > Move the SDK toolchian selection from python block, now we can 
> able to 
> > override from downstream layer. 
> > 
> > Signed-off-by: Srinuvasan A <srinuv...@siemens.com> 
> > --- 
> > meta/classes/crossvars.bbclass | 9 ++++----- 
> > 1 file changed, 4 insertions(+), 5 deletions(-) 
> > 
> > diff --git a/meta/classes/crossvars.bbclass 
> b/meta/classes/crossvars.bbclass 
> > index 201d460..120e6d1 100644 
> > --- a/meta/classes/crossvars.bbclass 
> > +++ b/meta/classes/crossvars.bbclass 
> > @@ -5,6 +5,10 @@ ISAR_CROSS_COMPILE ??= "0" 
> > 
> > inherit compat 
> > 
> > +SDK_TOOLCHAIN = "${@'build-essential' if 
> d.getVar('ISAR_CROSS_COMPILE') == '0' or d.getVar('HOST_ARCH') == 
> d.getVar('DISTRO_ARCH') or d.getVar('DISTRO_ARCH') == None else 
> 'crossbuild-essential-${DISTRO_ARCH}'}" 
> > + 
> > +SDK_TOOLCHAIN:append = "${@' 
> crossbuild-essential-${COMPAT_DISTRO_ARCH}' if 
> isar_can_build_compat(d) else ''}" 
> > + 
> 
> This is still no weak assignment which you will need for overrides. Did 
> you actually test this? And, again, can't we implement the weak 
> assignment in python below? 
> 
> Jan 
> 
> > python __anonymous() { 
> > import pwd 
> > d.setVar('SCHROOT_USER', pwd.getpwuid(os.geteuid()).pw_name) 
> > @@ -17,19 +21,14 @@ python __anonymous() { 
> > sbuild_dep = "sbuild-chroot-target:do_build" 
> > buildchroot_dir = d.getVar('BUILDCHROOT_TARGET_DIR', False) 
> > buildchroot_dep = "buildchroot-target:do_build" 
> > - sdk_toolchain = "build-essential" 
> > else: 
> > d.setVar('BUILD_HOST_ARCH', d.getVar('HOST_ARCH')) 
> > schroot_dir = d.getVar('SCHROOT_HOST_DIR', False) 
> > sbuild_dep = "sbuild-chroot-host:do_build" 
> > buildchroot_dir = d.getVar('BUILDCHROOT_HOST_DIR', False) 
> > buildchroot_dep = "buildchroot-host:do_build" 
> > - sdk_toolchain = "crossbuild-essential-" + distro_arch 
> > d.setVar('SCHROOT_DIR', schroot_dir) 
> > d.setVar('SCHROOT_DEP', sbuild_dep) 
> > d.setVar('BUILDCHROOT_DIR', buildchroot_dir) 
> > d.setVar('BUILDCHROOT_DEP', buildchroot_dep) 
> > - if isar_can_build_compat(d): 
> > - sdk_toolchain += " crossbuild-essential-" + 
> d.getVar('COMPAT_DISTRO_ARCH') 
> > - d.setVar('SDK_TOOLCHAIN', sdk_toolchain) 
> > } 
> 
> -- 
> Siemens AG, Technology 
> Competence Center Embedded Linux 
> 
> -- 
> You received this message because you are subscribed to the Google 
> Groups "isar-users" group. 
> To unsubscribe from this group and stop receiving emails from it, send 
> an email to isar-users+...@googlegroups.com 
> <mailto:isar-users+...@googlegroups.com>. 
> To view this discussion on the web visit 
> 
https://groups.google.com/d/msgid/isar-users/3f742bff-1ccd-4a88-8c96-8cab068b8eb4n%40googlegroups.com 
<
https://groups.google.com/d/msgid/isar-users/3f742bff-1ccd-4a88-8c96-8cab068b8eb4n%40googlegroups.com?utm_medium=email&utm_source=footer>.

Patch

diff --git a/meta/classes/crossvars.bbclass b/meta/classes/crossvars.bbclass
index 201d460..120e6d1 100644
--- a/meta/classes/crossvars.bbclass
+++ b/meta/classes/crossvars.bbclass
@@ -5,6 +5,10 @@  ISAR_CROSS_COMPILE ??= "0"
 
 inherit compat
 
+SDK_TOOLCHAIN = "${@'build-essential' if d.getVar('ISAR_CROSS_COMPILE') == '0' or d.getVar('HOST_ARCH') == d.getVar('DISTRO_ARCH') or d.getVar('DISTRO_ARCH') == None else 'crossbuild-essential-${DISTRO_ARCH}'}"
+
+SDK_TOOLCHAIN:append = "${@' crossbuild-essential-${COMPAT_DISTRO_ARCH}' if isar_can_build_compat(d) else ''}"
+
 python __anonymous() {
     import pwd
     d.setVar('SCHROOT_USER', pwd.getpwuid(os.geteuid()).pw_name)
@@ -17,19 +21,14 @@  python __anonymous() {
         sbuild_dep = "sbuild-chroot-target:do_build"
         buildchroot_dir = d.getVar('BUILDCHROOT_TARGET_DIR', False)
         buildchroot_dep = "buildchroot-target:do_build"
-        sdk_toolchain = "build-essential"
     else:
         d.setVar('BUILD_HOST_ARCH', d.getVar('HOST_ARCH'))
         schroot_dir = d.getVar('SCHROOT_HOST_DIR', False)
         sbuild_dep = "sbuild-chroot-host:do_build"
         buildchroot_dir = d.getVar('BUILDCHROOT_HOST_DIR', False)
         buildchroot_dep = "buildchroot-host:do_build"
-        sdk_toolchain = "crossbuild-essential-" + distro_arch
     d.setVar('SCHROOT_DIR', schroot_dir)
     d.setVar('SCHROOT_DEP', sbuild_dep)
     d.setVar('BUILDCHROOT_DIR', buildchroot_dir)
     d.setVar('BUILDCHROOT_DEP', buildchroot_dep)
-    if isar_can_build_compat(d):
-        sdk_toolchain += " crossbuild-essential-" + d.getVar('COMPAT_DISTRO_ARCH')
-    d.setVar('SDK_TOOLCHAIN', sdk_toolchain)
 }