[v5] Replace ISARROOT with more relevant variables

When switching between two ISAR workspaces in the same shell
session, the ISARROOT setting of the previous workspace would be
picked up for the new workspace resulting in an incorrect
configuration. The user had to manually unset ISARROOT to avoid
any issues.

As like OEROOT in OE, let us unset ISARROOT at the end of the
initialization script. This helps us to avoid the above issue
and also preserves support for passing ISARROOT from a caller
script as in OE.

ISARROOT is used mostly to derive the location of bitbake, scripts,
testsuite, meta and meta-isar directories.

Layers like meta, meta-isar already have their paths in LAYERDIR_core
and LAYERDIR_isar which can be used. For other directories introduce
dedicated variables BITBAKEDIR, SCRIPTSDIR and TESTSUITEDIR.

Signed-off-by: Vijai Kumar K <Vijaikumar_Kanagarajan@mentor.com>
---
Changes in v5:
Fixed a bug introduced by me in the CI build script in last series.
CI build is successful now.
http://ci.isar-build.org:8080/job/isar_vkk_devel/82/
Changes in v4:
- Rebase against latest next
- Added a entry in RECIPE-API-CHANGELOG.
Changes in v3:
- Send patch using proper send.
Changes in v2:
- Approach using dedicated variables for each commonly used
directories.

 RECIPE-API-CHANGELOG.md                  |  6 +++++
 isar-init-build-env                      |  1 +
 meta/classes/image-sdk-extension.bbclass |  2 +-
 meta/classes/wic-img.bbclass             | 31 ++++++++++++------------
 scripts/ci_build.sh                      | 18 ++++++++------
 scripts/isar-buildenv-internal           |  7 ++++--
 6 files changed, 39 insertions(+), 26 deletions(-)

Can this be merged?

Thanks,
Vijai Kumar K

On Tuesday, September 22, 2020 at 5:39:30 PM UTC+5:30 
vijaikumar_...@mentor.com wrote:

> When switching between two ISAR workspaces in the same shell
> session, the ISARROOT setting of the previous workspace would be
> picked up for the new workspace resulting in an incorrect
> configuration. The user had to manually unset ISARROOT to avoid
> any issues.
>
> As like OEROOT in OE, let us unset ISARROOT at the end of the
> initialization script. This helps us to avoid the above issue
> and also preserves support for passing ISARROOT from a caller
> script as in OE.
>
> ISARROOT is used mostly to derive the location of bitbake, scripts,
> testsuite, meta and meta-isar directories.
>
> Layers like meta, meta-isar already have their paths in LAYERDIR_core
> and LAYERDIR_isar which can be used. For other directories introduce
> dedicated variables BITBAKEDIR, SCRIPTSDIR and TESTSUITEDIR.
>
> Signed-off-by: Vijai Kumar K <Vijaikumar_...@mentor.com>
> ---
> Changes in v5:
> Fixed a bug introduced by me in the CI build script in last series.
> CI build is successful now.
> http://ci.isar-build.org:8080/job/isar_vkk_devel/82/
> Changes in v4:
> - Rebase against latest next
> - Added a entry in RECIPE-API-CHANGELOG.
> Changes in v3:
> - Send patch using proper send.
> Changes in v2:
> - Approach using dedicated variables for each commonly used
> directories.
>
> RECIPE-API-CHANGELOG.md | 6 +++++
> isar-init-build-env | 1 +
> meta/classes/image-sdk-extension.bbclass | 2 +-
> meta/classes/wic-img.bbclass | 31 ++++++++++++------------
> scripts/ci_build.sh | 18 ++++++++------
> scripts/isar-buildenv-internal | 7 ++++--
> 6 files changed, 39 insertions(+), 26 deletions(-)
>
> diff --git a/RECIPE-API-CHANGELOG.md b/RECIPE-API-CHANGELOG.md
> index 1839770..ac9d8d6 100644
> --- a/RECIPE-API-CHANGELOG.md
> +++ b/RECIPE-API-CHANGELOG.md
> @@ -246,3 +246,9 @@ by setting DEBIAN_BUILD_DEPENDS.
>
> ${S} can now be used for checking out sources without being linked 
> implicitly
> with ${D} which needs to be filled explicitly in do_install as before.
> +
> +### Remove ISARROOT from bitbake environment
> +
> +ISARROOT variable is now removed from the bitbake environment. It is unset
> +after the initial setup. It is replaced with dedicated variables like
> +BITBAKEDIR, SCRIPTSDIR and TESTSUITEDIR.
> diff --git a/isar-init-build-env b/isar-init-build-env
> index b08bb59..85fb82f 100755
> --- a/isar-init-build-env
> +++ b/isar-init-build-env
> @@ -55,5 +55,6 @@ export ISARROOT
> unset ISARROOT
> return 1
> }
> +unset ISARROOT
>
> [ -z "$BUILDDIR" ] || cd "$BUILDDIR"
> diff --git a/meta/classes/image-sdk-extension.bbclass 
> b/meta/classes/image-sdk-extension.bbclass
> index b9f2cf4..b91fdd6 100644
> --- a/meta/classes/image-sdk-extension.bbclass
> +++ b/meta/classes/image-sdk-extension.bbclass
> @@ -19,7 +19,7 @@ do_populate_sdk() {
> sudo rm -f ${SDKCHROOT_DIR}/chroot-setup.sh 
> ${SDKCHROOT_DIR}/configscript.sh
>
> # Copy mount_chroot.sh for convenience
> - sudo cp ${ISARROOT}/scripts/mount_chroot.sh ${SDKCHROOT_DIR}
> + sudo cp ${SCRIPTSDIR}/mount_chroot.sh ${SDKCHROOT_DIR}
>
> # Create SDK archive
> cd -P ${SDKCHROOT_DIR}/..
> diff --git a/meta/classes/wic-img.bbclass b/meta/classes/wic-img.bbclass
> index afab394..1bb1021 100644
> --- a/meta/classes/wic-img.bbclass
> +++ b/meta/classes/wic-img.bbclass
> @@ -86,7 +86,7 @@ STAGING_DATADIR ?= "/usr/lib/"
> STAGING_LIBDIR ?= "/usr/lib/"
> STAGING_DIR ?= "${TMPDIR}"
> IMAGE_BASENAME ?= "${PN}-${DISTRO}"
> -FAKEROOTCMD ?= "${ISARROOT}/scripts/wic_fakeroot"
> +FAKEROOTCMD ?= "${SCRIPTSDIR}/wic_fakeroot"
> RECIPE_SYSROOT_NATIVE ?= "/"
> BUILDCHROOT_DIR = "${BUILDCHROOT_TARGET_DIR}"
>
> @@ -132,7 +132,7 @@ do_wic_image() {
> buildchroot_do_mounts
> sudo -s <<'EOSUDO'
> ( flock 9
> - for dir in ${BBLAYERS} ${STAGING_DIR} ${ISARROOT}/scripts 
> ${ISARROOT}/bitbake; do
> + for dir in ${BBLAYERS} ${STAGING_DIR} ${SCRIPTSDIR} ${BITBAKEDIR}; do
> mkdir -p ${BUILDCHROOT_DIR}/$dir
> if ! mountpoint ${BUILDCHROOT_DIR}/$dir >/dev/null 2>&1; then
> mount --bind --make-private $dir ${BUILDCHROOT_DIR}/$dir
> @@ -149,25 +149,26 @@ EOSUDO
>
> sudo -E chroot ${BUILDCHROOT_DIR} \
> sh -c ' \
> - ISARROOT="$1"
> - WKS_FULL_PATH="$2"
> - STAGING_DIR="$3"
> - MACHINE="$4"
> - WICTMP="$5"
> - IMAGE_FULLNAME="$6"
> - IMAGE_BASENAME="$7"
> - shift 7
> -
> - export PATH="$ISARROOT/bitbake/bin:$PATH"
> - "$ISARROOT"/scripts/wic create "$WKS_FULL_PATH" \
> + BITBAKEDIR="$1"
> + SCRIPTSDIR="$2"
> + WKS_FULL_PATH="$3"
> + STAGING_DIR="$4"
> + MACHINE="$5"
> + WICTMP="$6"
> + IMAGE_FULLNAME="$7"
> + IMAGE_BASENAME="$8"
> + shift 8
> +
> + export PATH="$BITBAKEDIR/bin:$PATH"
> + "$SCRIPTSDIR"/wic create "$WKS_FULL_PATH" \
> --vars "$STAGING_DIR/$MACHINE/imgdata/" \
> -o "/$WICTMP/${IMAGE_FULLNAME}.wic/" \
> --bmap \
> -e "$IMAGE_BASENAME" $@' \
> - my_script "${ISARROOT}" "${WKS_FULL_PATH}" "${STAGING_DIR}" \
> + my_script "${BITBAKEDIR}" "${SCRIPTSDIR}" "${WKS_FULL_PATH}" 
> "${STAGING_DIR}" \
> "${MACHINE}" "${WICTMP}" "${IMAGE_FULLNAME}" "${IMAGE_BASENAME}" \
> ${WIC_CREATE_EXTRA_ARGS}
> - sudo chown -R $(stat -c "%U" ${ISARROOT}) ${ISARROOT}/meta 
> ${ISARROOT}/meta-isar ${ISARROOT}/scripts || true
> + sudo chown -R $(stat -c "%U" ${LAYERDIR_core}) ${LAYERDIR_core} 
> ${LAYERDIR_isar} ${SCRIPTSDIR} || true
> WIC_DIRECT=$(ls -t -1 
> ${BUILDCHROOT_DIR}/$WICTMP/${IMAGE_FULLNAME}.wic/*.direct | head -1)
> sudo chown -R $(id -u):$(id -g) ${BUILDCHROOT_DIR}/${WICTMP}
> mv -f ${WIC_DIRECT} ${WIC_IMAGE_FILE}
> diff --git a/scripts/ci_build.sh b/scripts/ci_build.sh
> index af996d1..bf224a6 100755
> --- a/scripts/ci_build.sh
> +++ b/scripts/ci_build.sh
> @@ -144,8 +144,8 @@ if [ -n "$CROSS_BUILD" ]; then
> fi
>
> if [ -n "$REPRO_BUILD" ]; then
> - 
> ISAR_TESTSUITE_GPG_PUB_KEY_FILE="$ISARROOT/testsuite/base-apt/test_pub.key"
> - 
> ISAR_TESTSUITE_GPG_PRIV_KEY_FILE="$ISARROOT/testsuite/base-apt/test_priv.key"
> + ISAR_TESTSUITE_GPG_PUB_KEY_FILE="$TESTSUITEDIR/base-apt/test_pub.key"
> + ISAR_TESTSUITE_GPG_PRIV_KEY_FILE="$TESTSUITEDIR/base-apt/test_priv.key"
> export GNUPGHOME=$(mktemp -d)
> gpg --import $ISAR_TESTSUITE_GPG_PUB_KEY_FILE 
> $ISAR_TESTSUITE_GPG_PRIV_KEY_FILE
>
> @@ -206,21 +206,23 @@ if [ -z "$FAST_BUILD" ]; then
> while [ -e bitbake.sock ]; do sleep 1; done
> fi
>
> -cp -a "${ISARROOT}/meta/classes/dpkg-base.bbclass" 
> "${ISARROOT}/meta/classes/dpkg-base.bbclass.ci-backup"
> -echo -e "do_fetch_append() {\n\n}" >> 
> "${ISARROOT}/meta/classes/dpkg-base.bbclass"
> +eval $(bitbake -e | grep "^LAYERDIR_core=")
> +eval $(bitbake -e | grep "^LAYERDIR_isar=")
> +cp -a "${LAYERDIR_core}/classes/dpkg-base.bbclass" 
> "${LAYERDIR_core}/classes/dpkg-base.bbclass.ci-backup"
> +echo -e "do_fetch_append() {\n\n}" >> 
> "${LAYERDIR_core}/classes/dpkg-base.bbclass"
>
> bitbake $BB_ARGS mc:qemuamd64-stretch:isar-image-base
>
> -mv "${ISARROOT}/meta/classes/dpkg-base.bbclass.ci-backup" 
> "${ISARROOT}/meta/classes/dpkg-base.bbclass"
> +mv "${LAYERDIR_core}/classes/dpkg-base.bbclass.ci-backup" 
> "${LAYERDIR_core}/classes/dpkg-base.bbclass"
>
> # Test wic --exclude-path
> -cp -a "${ISARROOT}/meta-isar/scripts/lib/wic/canned-wks/sdimage-efi.wks" 
> "${ISARROOT}/meta-isar/scripts/lib/wic/canned-wks/sdimage-efi.wks.ci-backup"
> +cp -a "${LAYERDIR_isar}/scripts/lib/wic/canned-wks/sdimage-efi.wks" 
> "${LAYERDIR_isar}/scripts/lib/wic/canned-wks/sdimage-efi.wks.ci-backup"
> mv 
> ${BUILDDIR}/tmp/deploy/images/qemuamd64/isar-image-base-debian-stretch-qemuamd64.wic.img 
> \
>
> ${BUILDDIR}/tmp/deploy/images/qemuamd64/isar-image-base-debian-stretch-qemuamd64.wic.img.ci-backup
> -sed -i -e 's/part \/ /part \/ --exclude-path usr /g' 
> "${ISARROOT}/meta-isar/scripts/lib/wic/canned-wks/sdimage-efi.wks"
> +sed -i -e 's/part \/ /part \/ --exclude-path usr /g' 
> "${LAYERDIR_isar}/scripts/lib/wic/canned-wks/sdimage-efi.wks"
>
> bitbake $BB_ARGS mc:qemuamd64-stretch:isar-image-base
>
> -mv 
> "${ISARROOT}/meta-isar/scripts/lib/wic/canned-wks/sdimage-efi.wks.ci-backup" 
> "${ISARROOT}/meta-isar/scripts/lib/wic/canned-wks/sdimage-efi.wks"
> +mv 
> "${LAYERDIR_isar}/scripts/lib/wic/canned-wks/sdimage-efi.wks.ci-backup" 
> "${LAYERDIR_isar}/scripts/lib/wic/canned-wks/sdimage-efi.wks"
> mv 
> ${BUILDDIR}/tmp/deploy/images/qemuamd64/isar-image-base-debian-stretch-qemuamd64.wic.img.ci-backup 
> \
>
> ${BUILDDIR}/tmp/deploy/images/qemuamd64/isar-image-base-debian-stretch-qemuamd64.wic.img
> diff --git a/scripts/isar-buildenv-internal 
> b/scripts/isar-buildenv-internal
> index 72a91ab..fe1e65f 100755
> --- a/scripts/isar-buildenv-internal
> +++ b/scripts/isar-buildenv-internal
> @@ -59,8 +59,11 @@ fi
> unset BDIR
>
> export BUILDDIR
> +export BITBAKEDIR="${ISARROOT}/bitbake"
> +export SCRIPTSDIR="${ISARROOT}/scripts"
> +export TESTSUITEDIR="${ISARROOT}/testsuite"
>
> -for newpath in "$ISARROOT/bitbake/bin" "$ISARROOT/scripts"; do
> +for newpath in "$BITBAKEDIR/bin" "$SCRIPTSDIR"; do
> # Remove any existences of $newpath from $PATH
> PATH=$(echo $PATH | sed -re "s#(^|:)$newpath(:|$)#\2#g;s#^:##")
>
> @@ -74,5 +77,5 @@ export PATH
> BBPATH="${BUILDDIR}"
> export BBPATH
>
> -BB_ENV_EXTRAWHITE="BUILDDIR ISARROOT http_proxy https_proxy ftp_proxy 
> no_proxy GNUPGHOME"
> +BB_ENV_EXTRAWHITE="BUILDDIR BITBAKEDIR SCRIPTSDIR TESTSUITEDIR http_proxy 
> https_proxy ftp_proxy no_proxy GNUPGHOME"
> export BB_ENV_EXTRAWHITE
> -- 
> 2.17.1
>
>

Hello Vijai Kumar,

On Tue, Sep 22, 2020 at 05:39:08PM +0530, Vijai Kumar K wrote:
> Changes in v5:
> Fixed a bug introduced by me in the CI build script in last series.
> CI build is successful now.
> http://ci.isar-build.org:8080/job/isar_vkk_devel/82/

Thanks for looking at this. I had tried to fix that, but encountered issues and
gave up after two steps. Applied to next.

With kind regards,
Baurzhan.

Thanks Baurzhan. Now this is in, I will rebase the wic series and send it 
out. 

Best,
Vijai Kumar K

On Monday, October 5, 2020 at 8:33:38 PM UTC+5:30 i...@radix50.net wrote:

> Hello Vijai Kumar,
>
> On Tue, Sep 22, 2020 at 05:39:08PM +0530, Vijai Kumar K wrote:
> > Changes in v5:
> > Fixed a bug introduced by me in the CI build script in last series.
> > CI build is successful now.
> > http://ci.isar-build.org:8080/job/isar_vkk_devel/82/
>
> Thanks for looking at this. I had tried to fix that, but encountered 
> issues and
> gave up after two steps. Applied to next.
>
> With kind regards,
> Baurzhan.
>