mbox series

[v1,0/1] Use oe.utils.directory_size() instead of du for calculating rootfs size

Message ID 20241129062827.527177-1-Adithya.Balakumar@toshiba-tsip.com
Headers show
Series Use oe.utils.directory_size() instead of du for calculating rootfs size | expand

Message

Adithya Balakumar Nov. 29, 2024, 6:28 a.m. UTC 
Currently the "du" command is used to calculate the rootfs size, and sometimes the
reported size of the rootfs by du varies across multiple builds (usually by a few kb).
But the rootfs contents are reproducible.

Although this does not affect any functionality, but it does affect reproducibility
of artifacts that rely on the calculated rootfs size.

There are 2 scenarios (that I noticed) where this is affects reproducibility:
1. In the case where no size parameter is set for a rootfs partition in the wks file,
   wic uses the calculated rootfs size.
2. ext4 filesystem images with the IMAGE_CMD of isar, relies on rootfs size to create image.

Some of the files/directories in the rootfs has different number of blocks allocated.
The files/directories affected are not always the same across builds.

Using the "du" command to measure the size of the rootfs directory in such a case, yields
different values for the ROOTFS_SIZE.

This patch introduces the use of an alternate function from oe.utils to calculate the
directory size. OE-Core uses the same function to calculate the directory size (See [1])

Based on multiple build runs, usage of this function seems to give consistent size results.

Some details on the issue is mentioned in the below thread in the ISAR mailing list (See [2])

[1] https://github.com/openembedded/openembedded-core/commit/d8f1f3a6b024a2ae6631d1ce25421e8d94b69a12
[2] https://groups.google.com/g/isar-users/c/Ll7t4G41Lfo/m/gvxR4hYBAwAJ

Adithya Balakumar (1):
  image.bbclass: use oe.utils.directory_size() instead of du

 meta/classes/image.bbclass | 7 ++-----
 1 file changed, 2 insertions(+), 5 deletions(-)

Comments

Uladzimir Bely Dec. 6, 2024, 7:29 a.m. UTC | #1 
On Fri, 2024-11-29 at 11:58 +0530, Adithya Balakumar wrote:
> Currently the "du" command is used to calculate the rootfs size, and
> sometimes the
> reported size of the rootfs by du varies across multiple builds
> (usually by a few kb).
> But the rootfs contents are reproducible.
> 
> Although this does not affect any functionality, but it does affect
> reproducibility
> of artifacts that rely on the calculated rootfs size.
> 
> There are 2 scenarios (that I noticed) where this is affects
> reproducibility:
> 1. In the case where no size parameter is set for a rootfs partition
> in the wks file,
>    wic uses the calculated rootfs size.
> 2. ext4 filesystem images with the IMAGE_CMD of isar, relies on
> rootfs size to create image.
> 
> Some of the files/directories in the rootfs has different number of
> blocks allocated.
> The files/directories affected are not always the same across builds.
> 
> Using the "du" command to measure the size of the rootfs directory in
> such a case, yields
> different values for the ROOTFS_SIZE.
> 
> This patch introduces the use of an alternate function from oe.utils
> to calculate the
> directory size. OE-Core uses the same function to calculate the
> directory size (See [1])
> 
> Based on multiple build runs, usage of this function seems to give
> consistent size results.
> 
> Some details on the issue is mentioned in the below thread in the
> ISAR mailing list (See [2])
> 
> [1]
> https://github.com/openembedded/openembedded-core/commit/d8f1f3a6b024a2ae6631d1ce25421e8d94b69a12
> [2]
> https://groups.google.com/g/isar-users/c/Ll7t4G41Lfo/m/gvxR4hYBAwAJ
> 
> Adithya Balakumar (1):
>   image.bbclass: use oe.utils.directory_size() instead of du
> 
>  meta/classes/image.bbclass | 7 ++-----
>  1 file changed, 2 insertions(+), 5 deletions(-)
> 
> -- 
> 2.39.5
> 
> 

Applied to next, thanks.