| Message ID | 20240814173524.383600-3-nicusor.huhulea@siemens.com |
|---|---|
| State | Under Review |
| Headers | show |
| Series | [1/3] linux-custom: refactor kernel's make command | expand |
On Wed, 2024-08-14 at 20:35 +0300, 'Nicusor Huhulea' via isar-users wrote: > The current implementation makes a full kernel build and that means > it will > compile all dtbs specified by the bsp makefile e.g > arch/arm64/boot/dts/ti/Makefile > Currently the required dtbs are given by the DTB_FILES. > > These changes are checking if the DTB_FILES is specified, compile > only those > and avoids compiling all the dtbs from the bsp directory. Otherwise > build all dtbs > specified by the kernel source. > Preventing the automatic dtb builds is made by using target specific > build > commands e.g KERNEL_IMAGETYPE > The compilation of the dtbs accepts the following formats e.g: > Kernel Post 6.x: > a) full path from the architecture directory: > arch/${ARCH}/boot/dts/bsp_dir/test.dts > arch/${ARCH}/boot/dts/bsp_dir/test.dtb > b) relative path from the dts directory: bsp_dir/test.dts > c) device tree blob file: bsp_dir/test.dtb > Kernel Pre 6.x: > The same as on Kernel Post 6.x but without the bsp_dir > > The differences when building a full build and when using a specific > target > build are:(tests were made on the machine phyboard-mira on 6.x and > 4.x versions) > specific target: Build needed 00:02:08, 2219688k disk space linux-phy > 6.x > full build: Build needed 00:02:14, 2271528k disk space linux-phy > 6.x > The time difference: the full build took 6s longer than the specific > target build > The disk space difference: the full build used 50.6MB more disk space > than > the specific target build. > In conclusion the specific target build is slightly faster and uses > less > disk space. > > For now the DTB_FILES variable should accept only Device Tree > Blobs(.dtb) > because some areas still depends on it e.g: do_copy_boot_files, > plugins > > Signed-off-by: Nicusor Huhulea <nicusor.huhulea@siemens.com> > --- > .../linux/files/debian/isar/build.tmpl | 56 > ++++++++++++++++++- > .../linux/files/debian/isar/install.tmpl | 12 +++- > meta/recipes-kernel/linux/linux-custom.inc | 6 ++ > 3 files changed, 71 insertions(+), 3 deletions(-) > > diff --git a/meta/recipes-kernel/linux/files/debian/isar/build.tmpl > b/meta/recipes-kernel/linux/files/debian/isar/build.tmpl > index 1fd6f948..6f186e02 100644 > --- a/meta/recipes-kernel/linux/files/debian/isar/build.tmpl > +++ b/meta/recipes-kernel/linux/files/debian/isar/build.tmpl > @@ -22,8 +22,13 @@ do_build() { > sed -i "s/@KR@/${KR}/g" ${S}/debian/control ${S}/debian/linux- > image-${KERNEL_NAME_PROVIDED}.* > > MAKE_COMMON_ARGS="O=${KERNEL_BUILD_DIR} ${PARALLEL_MAKE} > KCFLAGS=${KCFLAGS} KAFLAGS=${KAFLAGS}" > - if echo "${DEB_BUILD_PROFILES}" | grep -q "kernel"; then # Full > build > - ${MAKE} ${MAKE_COMMON_ARGS} ${KERNEL_EXTRA_BUILDARGS} > + if echo "${DEB_BUILD_PROFILES}" | grep -q "kernel"; then > + if [ -n "${KERNEL_IMAGETYPE}" ]; then # Override the default > compilation > + ${MAKE} ${MAKE_COMMON_ARGS} ${KERNEL_EXTRA_BUILDARGS} > ${KERNEL_IMAGETYPE} modules > + compile_dtbs > + else $ Full build A typo causing build error. '/<<PKGBUILDDIR>>/debian/isar/build: line 29: $: command not found'. > + ${MAKE} ${MAKE_COMMON_ARGS} ${KERNEL_EXTRA_BUILDARGS} > + fi > elif echo "${DEB_BUILD_PROFILES}" | grep -q "kbuild"; then # > Build kernel scripts and tools > ${MAKE} ${MAKE_COMMON_ARGS} ${KERNEL_EXTRA_BUILDARGS} > scripts > if grep -q -E > "CONFIG_STACK_VALIDATION=y|CONFIG_HAVE_OBJTOOL=y" > ${KERNEL_BUILD_DIR}/.config && [ -d "tools/objtool" ]; then > @@ -38,6 +43,53 @@ do_build() { > set +x > } > > +compile_dtbs() { > + local prefix="arch/${ARCH}/boot/dts/" > + > + for dtb in ${DTB_FILES}; do > + # Handle the case where a standalone(with no path .dtb/.dts) > file may reside within > + # a specific BSP directory or not(kernel pre-6.x) > + if [[ ("${dtb}" == *.dtb || "${dtb}" == *.dts) && "${dtb}" > != */* ]]; then > + bsp_dir="" > + dts_path="" > + if [[ "${dtb}" == *.dtb ]]; then > + dts="${dtb%.dtb}.dts" > + else > + dts="${dtb}" > + fi > + > + # recursively search for the dts file in all possible > BSP directories > + while IFS= read -r dts_path; do > + if [ -n "${dts_path}" ]; then > + # Eliminate the prefix and dts to get only the > bsp_dir > + bsp_dir="${dts_path#${prefix}}" > + bsp_dir="${bsp_dir%${dts}}" > + > + relative_dtb_path="${bsp_dir}${dtb}" > + else > + relative_dtb_path="${prefix}${dtb}" > + fi > + done < <(find "${prefix}" -type f -name "${dts}") > + > + # Check if the path is relative (starts with arch/${ARCH}/) > + # and if it does then extract the relative path from the > architecture specific path > + elif [[ "${dtb}" == "${prefix}"* ]]; then > + relative_dtb_path="${dtb#${prefix}}" > + else > + # directly use the given dtb > + relative_dtb_path="${dtb}" > + fi > + > + # Check if it's a .dts file > + if [[ "${relative_dtb_path}" == *.dts ]]; then > + # Change .dts to .dtb > + relative_dtb_path="${relative_dtb_path%.dts}.dtb" > + fi > + > + ${MAKE} ${MAKE_COMMON_ARGS} ${relative_dtb_path} > + done > +} > + > print_settings() { > cat <<EOF > # Build settings: > diff --git a/meta/recipes-kernel/linux/files/debian/isar/install.tmpl > b/meta/recipes-kernel/linux/files/debian/isar/install.tmpl > index 00011517..c0a60f4e 100644 > --- a/meta/recipes-kernel/linux/files/debian/isar/install.tmpl > +++ b/meta/recipes-kernel/linux/files/debian/isar/install.tmpl > @@ -124,7 +124,17 @@ install_hooks() { > > install_dtbs() { > [ -n "${CONFIG_OF}" ] || return 0 > - ${MAKE} O=${O} INSTALL_DTBS_PATH=${deb_img_dir}/usr/lib/linux- > image-${krel} dtbs_install > + prefix="arch/${ARCH}/boot/dts/" > + src_dir="${O}/${prefix}" > + dst_dir="" > + > + dst_dir="${deb_img_dir}/usr/lib/linux-image-${krel}/" > + find "${src_dir}" -type f -name '*.dtb' | while IFS= read -r > dtb_file; do > + local relative_path="${dtb_file#"${src_dir}"}" > + mkdir -p "${dst_dir}$(dirname "${relative_path}")" > + cp "${dtb_file}" "${dst_dir}/${relative_path}" > + echo "Copying ${dtb_file} to ${dest_dir}/${relative_path}" > + done > } > > install_kmods() { > diff --git a/meta/recipes-kernel/linux/linux-custom.inc > b/meta/recipes-kernel/linux/linux-custom.inc > index 6aa70fd3..31cc480b 100644 > --- a/meta/recipes-kernel/linux/linux-custom.inc > +++ b/meta/recipes-kernel/linux/linux-custom.inc > @@ -76,6 +76,7 @@ TEMPLATE_VARS += " \ > KERNEL_DEBIAN_DEPENDS \ > KERNEL_BUILD_DIR \ > KERNEL_FILE \ > + KERNEL_IMAGETYPE \ > KERNEL_HEADERS_DEBIAN_DEPENDS \ > LINUX_VERSION_EXTENSION \ > KERNEL_NAME_PROVIDED \ > @@ -84,12 +85,17 @@ TEMPLATE_VARS += " \ > KAFLAGS \ > DISTRIBUTOR \ > KERNEL_EXTRA_BUILDARGS \ > + DTB_FILES \ > " > > inherit dpkg > inherit template > inherit kbuildtarget > > +DTB_FILES ?= "" > + > +KERNEL_IMAGETYPE ?= "" > + > # Add custom cflags to the kernel build > KCFLAGS ?= "-fdebug-prefix-map=${CURDIR}=." > KAFLAGS ?= "-fdebug-prefix-map=${CURDIR}=." > -- > 2.39.2 >
On 23.08.24 10:44, Uladzimir Bely wrote: > On Wed, 2024-08-14 at 20:35 +0300, 'Nicusor Huhulea' via isar-users > wrote: >> The current implementation makes a full kernel build and that means >> it will >> compile all dtbs specified by the bsp makefile e.g >> arch/arm64/boot/dts/ti/Makefile >> Currently the required dtbs are given by the DTB_FILES. >> >> These changes are checking if the DTB_FILES is specified, compile >> only those >> and avoids compiling all the dtbs from the bsp directory. Otherwise >> build all dtbs >> specified by the kernel source. >> Preventing the automatic dtb builds is made by using target specific >> build >> commands e.g KERNEL_IMAGETYPE >> The compilation of the dtbs accepts the following formats e.g: >> Kernel Post 6.x: >> a) full path from the architecture directory: >> arch/${ARCH}/boot/dts/bsp_dir/test.dts >> arch/${ARCH}/boot/dts/bsp_dir/test.dtb >> b) relative path from the dts directory: bsp_dir/test.dts >> c) device tree blob file: bsp_dir/test.dtb >> Kernel Pre 6.x: >> The same as on Kernel Post 6.x but without the bsp_dir >> >> The differences when building a full build and when using a specific >> target >> build are:(tests were made on the machine phyboard-mira on 6.x and >> 4.x versions) >> specific target: Build needed 00:02:08, 2219688k disk space linux-phy >> 6.x >> full build: Build needed 00:02:14, 2271528k disk space linux-phy >> 6.x >> The time difference: the full build took 6s longer than the specific >> target build >> The disk space difference: the full build used 50.6MB more disk space >> than >> the specific target build. >> In conclusion the specific target build is slightly faster and uses >> less >> disk space. >> >> For now the DTB_FILES variable should accept only Device Tree >> Blobs(.dtb) >> because some areas still depends on it e.g: do_copy_boot_files, >> plugins >> >> Signed-off-by: Nicusor Huhulea <nicusor.huhulea@siemens.com> >> --- >> .../linux/files/debian/isar/build.tmpl | 56 >> ++++++++++++++++++- >> .../linux/files/debian/isar/install.tmpl | 12 +++- >> meta/recipes-kernel/linux/linux-custom.inc | 6 ++ >> 3 files changed, 71 insertions(+), 3 deletions(-) >> >> diff --git a/meta/recipes-kernel/linux/files/debian/isar/build.tmpl >> b/meta/recipes-kernel/linux/files/debian/isar/build.tmpl >> index 1fd6f948..6f186e02 100644 >> --- a/meta/recipes-kernel/linux/files/debian/isar/build.tmpl >> +++ b/meta/recipes-kernel/linux/files/debian/isar/build.tmpl >> @@ -22,8 +22,13 @@ do_build() { >> sed -i "s/@KR@/${KR}/g" ${S}/debian/control ${S}/debian/linux- >> image-${KERNEL_NAME_PROVIDED}.* >> >> MAKE_COMMON_ARGS="O=${KERNEL_BUILD_DIR} ${PARALLEL_MAKE} >> KCFLAGS=${KCFLAGS} KAFLAGS=${KAFLAGS}" >> - if echo "${DEB_BUILD_PROFILES}" | grep -q "kernel"; then # Full >> build >> - ${MAKE} ${MAKE_COMMON_ARGS} ${KERNEL_EXTRA_BUILDARGS} >> + if echo "${DEB_BUILD_PROFILES}" | grep -q "kernel"; then >> + if [ -n "${KERNEL_IMAGETYPE}" ]; then # Override the default >> compilation >> + ${MAKE} ${MAKE_COMMON_ARGS} ${KERNEL_EXTRA_BUILDARGS} >> ${KERNEL_IMAGETYPE} modules >> + compile_dtbs >> + else $ Full build > > A typo causing build error. > > '/<<PKGBUILDDIR>>/debian/isar/build: line 29: $: command not found'. > Just to be clear: even once issues in such details are resolved, I'm still not convinced about the value of the approach, compared to the maintenance risks it would introduce. Jan
diff --git a/meta/recipes-kernel/linux/files/debian/isar/build.tmpl b/meta/recipes-kernel/linux/files/debian/isar/build.tmpl index 1fd6f948..6f186e02 100644 --- a/meta/recipes-kernel/linux/files/debian/isar/build.tmpl +++ b/meta/recipes-kernel/linux/files/debian/isar/build.tmpl @@ -22,8 +22,13 @@ do_build() { sed -i "s/@KR@/${KR}/g" ${S}/debian/control ${S}/debian/linux-image-${KERNEL_NAME_PROVIDED}.* MAKE_COMMON_ARGS="O=${KERNEL_BUILD_DIR} ${PARALLEL_MAKE} KCFLAGS=${KCFLAGS} KAFLAGS=${KAFLAGS}" - if echo "${DEB_BUILD_PROFILES}" | grep -q "kernel"; then # Full build - ${MAKE} ${MAKE_COMMON_ARGS} ${KERNEL_EXTRA_BUILDARGS} + if echo "${DEB_BUILD_PROFILES}" | grep -q "kernel"; then + if [ -n "${KERNEL_IMAGETYPE}" ]; then # Override the default compilation + ${MAKE} ${MAKE_COMMON_ARGS} ${KERNEL_EXTRA_BUILDARGS} ${KERNEL_IMAGETYPE} modules + compile_dtbs + else $ Full build + ${MAKE} ${MAKE_COMMON_ARGS} ${KERNEL_EXTRA_BUILDARGS} + fi elif echo "${DEB_BUILD_PROFILES}" | grep -q "kbuild"; then # Build kernel scripts and tools ${MAKE} ${MAKE_COMMON_ARGS} ${KERNEL_EXTRA_BUILDARGS} scripts if grep -q -E "CONFIG_STACK_VALIDATION=y|CONFIG_HAVE_OBJTOOL=y" ${KERNEL_BUILD_DIR}/.config && [ -d "tools/objtool" ]; then @@ -38,6 +43,53 @@ do_build() { set +x } +compile_dtbs() { + local prefix="arch/${ARCH}/boot/dts/" + + for dtb in ${DTB_FILES}; do + # Handle the case where a standalone(with no path .dtb/.dts) file may reside within + # a specific BSP directory or not(kernel pre-6.x) + if [[ ("${dtb}" == *.dtb || "${dtb}" == *.dts) && "${dtb}" != */* ]]; then + bsp_dir="" + dts_path="" + if [[ "${dtb}" == *.dtb ]]; then + dts="${dtb%.dtb}.dts" + else + dts="${dtb}" + fi + + # recursively search for the dts file in all possible BSP directories + while IFS= read -r dts_path; do + if [ -n "${dts_path}" ]; then + # Eliminate the prefix and dts to get only the bsp_dir + bsp_dir="${dts_path#${prefix}}" + bsp_dir="${bsp_dir%${dts}}" + + relative_dtb_path="${bsp_dir}${dtb}" + else + relative_dtb_path="${prefix}${dtb}" + fi + done < <(find "${prefix}" -type f -name "${dts}") + + # Check if the path is relative (starts with arch/${ARCH}/) + # and if it does then extract the relative path from the architecture specific path + elif [[ "${dtb}" == "${prefix}"* ]]; then + relative_dtb_path="${dtb#${prefix}}" + else + # directly use the given dtb + relative_dtb_path="${dtb}" + fi + + # Check if it's a .dts file + if [[ "${relative_dtb_path}" == *.dts ]]; then + # Change .dts to .dtb + relative_dtb_path="${relative_dtb_path%.dts}.dtb" + fi + + ${MAKE} ${MAKE_COMMON_ARGS} ${relative_dtb_path} + done +} + print_settings() { cat <<EOF # Build settings: diff --git a/meta/recipes-kernel/linux/files/debian/isar/install.tmpl b/meta/recipes-kernel/linux/files/debian/isar/install.tmpl index 00011517..c0a60f4e 100644 --- a/meta/recipes-kernel/linux/files/debian/isar/install.tmpl +++ b/meta/recipes-kernel/linux/files/debian/isar/install.tmpl @@ -124,7 +124,17 @@ install_hooks() { install_dtbs() { [ -n "${CONFIG_OF}" ] || return 0 - ${MAKE} O=${O} INSTALL_DTBS_PATH=${deb_img_dir}/usr/lib/linux-image-${krel} dtbs_install + prefix="arch/${ARCH}/boot/dts/" + src_dir="${O}/${prefix}" + dst_dir="" + + dst_dir="${deb_img_dir}/usr/lib/linux-image-${krel}/" + find "${src_dir}" -type f -name '*.dtb' | while IFS= read -r dtb_file; do + local relative_path="${dtb_file#"${src_dir}"}" + mkdir -p "${dst_dir}$(dirname "${relative_path}")" + cp "${dtb_file}" "${dst_dir}/${relative_path}" + echo "Copying ${dtb_file} to ${dest_dir}/${relative_path}" + done } install_kmods() { diff --git a/meta/recipes-kernel/linux/linux-custom.inc b/meta/recipes-kernel/linux/linux-custom.inc index 6aa70fd3..31cc480b 100644 --- a/meta/recipes-kernel/linux/linux-custom.inc +++ b/meta/recipes-kernel/linux/linux-custom.inc @@ -76,6 +76,7 @@ TEMPLATE_VARS += " \ KERNEL_DEBIAN_DEPENDS \ KERNEL_BUILD_DIR \ KERNEL_FILE \ + KERNEL_IMAGETYPE \ KERNEL_HEADERS_DEBIAN_DEPENDS \ LINUX_VERSION_EXTENSION \ KERNEL_NAME_PROVIDED \ @@ -84,12 +85,17 @@ TEMPLATE_VARS += " \ KAFLAGS \ DISTRIBUTOR \ KERNEL_EXTRA_BUILDARGS \ + DTB_FILES \ " inherit dpkg inherit template inherit kbuildtarget +DTB_FILES ?= "" + +KERNEL_IMAGETYPE ?= "" + # Add custom cflags to the kernel build KCFLAGS ?= "-fdebug-prefix-map=${CURDIR}=." KAFLAGS ?= "-fdebug-prefix-map=${CURDIR}=."
The current implementation makes a full kernel build and that means it will compile all dtbs specified by the bsp makefile e.g arch/arm64/boot/dts/ti/Makefile Currently the required dtbs are given by the DTB_FILES. These changes are checking if the DTB_FILES is specified, compile only those and avoids compiling all the dtbs from the bsp directory. Otherwise build all dtbs specified by the kernel source. Preventing the automatic dtb builds is made by using target specific build commands e.g KERNEL_IMAGETYPE The compilation of the dtbs accepts the following formats e.g: Kernel Post 6.x: a) full path from the architecture directory: arch/${ARCH}/boot/dts/bsp_dir/test.dts arch/${ARCH}/boot/dts/bsp_dir/test.dtb b) relative path from the dts directory: bsp_dir/test.dts c) device tree blob file: bsp_dir/test.dtb Kernel Pre 6.x: The same as on Kernel Post 6.x but without the bsp_dir The differences when building a full build and when using a specific target build are:(tests were made on the machine phyboard-mira on 6.x and 4.x versions) specific target: Build needed 00:02:08, 2219688k disk space linux-phy 6.x full build: Build needed 00:02:14, 2271528k disk space linux-phy 6.x The time difference: the full build took 6s longer than the specific target build The disk space difference: the full build used 50.6MB more disk space than the specific target build. In conclusion the specific target build is slightly faster and uses less disk space. For now the DTB_FILES variable should accept only Device Tree Blobs(.dtb) because some areas still depends on it e.g: do_copy_boot_files, plugins Signed-off-by: Nicusor Huhulea <nicusor.huhulea@siemens.com> --- .../linux/files/debian/isar/build.tmpl | 56 ++++++++++++++++++- .../linux/files/debian/isar/install.tmpl | 12 +++- meta/recipes-kernel/linux/linux-custom.inc | 6 ++ 3 files changed, 71 insertions(+), 3 deletions(-)