| Message ID | 20211012130413.1719424-1-adriaan.schmidt@siemens.com |
|---|---|
| State | RFC |
| Headers | show |
12.10.2021 16:04, Adriaan Schmidt wrote: > After some reports on problems with mounting and unmounting, here is a proposal > to refactor the whole mount logic. > > The main idea is to have all actual mounting happen in mounts.bbclass, which > also takes care of things like reference counting. > Any task that needs mounts uses a `do_my_task[mounts] = "..."` varflag annotation. > > One thing that may need some thoughts: the original rootfs_do_mounts also > writes /etc/resolv.conf. This proposal introduces another varflag > (`do_my_task[buildchroot] = "1"`) for tasks that need that. Alternatively, > we could simply provide a function that needs to be called explicitly. > > Current status: the code works, but there may be some issues with cleanup/unmounting. > The existing ("legacy") cleanup code in isar-events still finds active mounts, > which should actually be covered by cleanup code in mounts.bbclass. > > > Adriaan Schmidt (5): > oe imports in central location > meta: refactor containerization > meta: add oe.utils > meta: add mounts class > meta: refactor to use the new mounting mechanism > > meta/classes/base.bbclass | 28 + > meta/classes/buildchroot.bbclass | 52 +- > meta/classes/cpiogz-img.bbclass | 3 +- > meta/classes/dpkg-base.bbclass | 49 +- > meta/classes/dpkg-gbp.bbclass | 2 - > meta/classes/dpkg.bbclass | 4 +- > meta/classes/ext4-img.bbclass | 3 +- > meta/classes/fit-img.bbclass | 4 +- > .../classes/image-container-extension.bbclass | 3 +- > meta/classes/image-sdk-extension.bbclass | 6 +- > meta/classes/image-tools-extension.bbclass | 4 +- > meta/classes/image.bbclass | 30 +- > meta/classes/initramfs.bbclass | 2 +- > meta/classes/isar-events.bbclass | 4 +- > meta/classes/mounts.bbclass | 271 +++++++++ > meta/classes/patch.bbclass | 5 - > meta/classes/rootfs.bbclass | 50 +- > meta/classes/ubi-img.bbclass | 3 +- > meta/classes/ubifs-img.bbclass | 3 +- > meta/classes/vm-img.bbclass | 7 +- > meta/classes/wic-img.bbclass | 31 +- > meta/conf/bitbake.conf | 2 +- > meta/lib/oe/utils.py | 569 ++++++++++++++++++ > .../isar-bootstrap/isar-bootstrap.inc | 43 +- > .../buildchroot/buildchroot.inc | 8 +- > 25 files changed, 976 insertions(+), 210 deletions(-) > create mode 100644 meta/classes/mounts.bbclass > create mode 100644 meta/lib/oe/utils.py > There are at least 6 cases needs to be checked after any mount rework: 1) There is no while loop issue during workdir umount 2) Fail on parallel build do not affect other recipes 3) Workdir should be unmounted in case build fail 4) Fail during mounting should not left paths unmounted 5) There is no `Target is busy` issues during high load 6) Host /dev is not get corrupted after `device is busy` issues All of that issues were raised before and we need to be sure any rebuild is safe. That's why we will check those 6 cases on the next version before going forward. Will provide test case details if needed.
After some reports on problems with mounting and unmounting, here is a proposal to refactor the whole mount logic. The main idea is to have all actual mounting happen in mounts.bbclass, which also takes care of things like reference counting. Any task that needs mounts uses a `do_my_task[mounts] = "..."` varflag annotation. One thing that may need some thoughts: the original rootfs_do_mounts also writes /etc/resolv.conf. This proposal introduces another varflag (`do_my_task[buildchroot] = "1"`) for tasks that need that. Alternatively, we could simply provide a function that needs to be called explicitly. Current status: the code works, but there may be some issues with cleanup/unmounting. The existing ("legacy") cleanup code in isar-events still finds active mounts, which should actually be covered by cleanup code in mounts.bbclass. Adriaan Schmidt (5): oe imports in central location meta: refactor containerization meta: add oe.utils meta: add mounts class meta: refactor to use the new mounting mechanism meta/classes/base.bbclass | 28 + meta/classes/buildchroot.bbclass | 52 +- meta/classes/cpiogz-img.bbclass | 3 +- meta/classes/dpkg-base.bbclass | 49 +- meta/classes/dpkg-gbp.bbclass | 2 - meta/classes/dpkg.bbclass | 4 +- meta/classes/ext4-img.bbclass | 3 +- meta/classes/fit-img.bbclass | 4 +- .../classes/image-container-extension.bbclass | 3 +- meta/classes/image-sdk-extension.bbclass | 6 +- meta/classes/image-tools-extension.bbclass | 4 +- meta/classes/image.bbclass | 30 +- meta/classes/initramfs.bbclass | 2 +- meta/classes/isar-events.bbclass | 4 +- meta/classes/mounts.bbclass | 271 +++++++++ meta/classes/patch.bbclass | 5 - meta/classes/rootfs.bbclass | 50 +- meta/classes/ubi-img.bbclass | 3 +- meta/classes/ubifs-img.bbclass | 3 +- meta/classes/vm-img.bbclass | 7 +- meta/classes/wic-img.bbclass | 31 +- meta/conf/bitbake.conf | 2 +- meta/lib/oe/utils.py | 569 ++++++++++++++++++ .../isar-bootstrap/isar-bootstrap.inc | 43 +- .../buildchroot/buildchroot.inc | 8 +- 25 files changed, 976 insertions(+), 210 deletions(-) create mode 100644 meta/classes/mounts.bbclass create mode 100644 meta/lib/oe/utils.py