| Message ID | CAJGKYO58T1LZLMJ84+oLF3b6CtJcfEwz97=WGDcrWCanX+5HVA@mail.gmail.com |
|---|---|
| State | Not Applicable, archived |
| Headers | show |
| Series | [PATCHSET] expand last partition script and its usage | expand |
On Thu, 15 Dec 2022 at 19:06, Roberto A. Foglietta <roberto.foglietta@gmail.com> wrote: > > Hi all, > > all these patches are just sent by elastic mail SMTP and they are > waiting in moderation. > > * 9ab1ca3 - In expand last partition few improvements and a bugfix Version 2 of the last patch * 082d8af - In expand last partition few improvements and a bugfix I forgot to mention Henning on which code I took inspiration from. Best regards, R-
Hi all,
this is the single patch for rework and generalisation of last
partition expansion.
As usual has been sent by elastic mail SMTP and it went in moderation.
UPDATE: the unpartition volume resizing has been tested in this way
git clone https://github.com/robang74/isar-nvidia-debian.git
cd isar-nvidia-debian
source .profile
build basic-os
wicshell
cd /boot
cp -arf vmlinuz-5.10.0-19-amd64 initrd.img-5.10.0-19-amd64 /work
exit
wicinst file:image.wic
sudo losetup -P $(losetup -f) image.wic
sudo dd if=/dev/loop3p2 bs=1M > disk.wic
dd if=/dev/zero bs=1 seek=$[4*1024*1024*1024] count=1 oflag=append
conv=notrunc,sparse status=none of=disk.wic
sudo qemu-system-x86_64 --cpu host --enable-kvm -smp 4 -m 1024 -drive
format=raw,file=disk.wic -device sdhci-pci -kernel
build/tmp/work/vmlinuz-5.10.0-19-amd64 -initrd
build/tmp/work/initrd.img-5.10.0-19-amd64 -append "root=/dev/sda
console=ttyS0" --nographic; echo ">>> press ENTER"; read; reset
dmesg | grep expand
df -h
poweroff
Best regards, R-
===========================================
The script has been reworked to simplify its code:
- the last partition free space is not computed anymore because useless
- on exit a function take care of some final tasks and logs in /dev/kmsg
and to extend its functionalities supporting new filesystems:
- filesystem type is managed by a switch-case block of code
- temporary directory to mount is created relying on TMPDIR
- unsupervised systems have the option to reboot because error
- if temporary directory creation fails 2nd try with /dev/shm
- additional ext2, ext3 and btrfs filesystems supported
- udev support: udevadm settle is the first task to run
- unpartitioned volumes are resized as well (tested)
This patch has been created by a collection of 7 patches to be applied on
ilbers ISAR, branch next, commit d26660b724b034b602f3889f55a23cd9be2e87bd
Author: Roberto A. Foglietta <roberto.foglietta@gmail.com>
This code has been developed taking inspiration by suggestions and code from:
Felix Moessbauer <felix.moessbauer@siemens.com>
Tobias Schmidl <tobiasschmidl@siemens.com>
Henning Schild <henning.schild@siemens.com>
Signed-off-by: Roberto A. Foglietta <roberto.foglietta@gmail.com>
On Fri, 16 Dec 2022 at 00:00, Roberto A. Foglietta <roberto.foglietta@gmail.com> wrote: > > Hi all, > - udev support: udevadm settle is the first task to run [...] > - unpartitioned volumes are resized as well (tested) about these points the latency introduced by udevadm settle is irrelevant in most cases about 0.1 seconds and because I commented the code that checks for the free space in the last partition (all resizing tools do that more efficiently) - in general the execution time will be smaller - faster. In those platforms in which the boot time grows up - most probably are the same that would fail because they boot from a small internal storage device and might use unpartitioned devices for rootfs or RW. Likely the qemu test that I presented here in which vmlinuz and initrd image are separated from the unpartitioned volume. In such a case the mount cannot trigger the kernel to map the device because rootfs and bootfs are on different devices. This is also the reason because I denominated the patch "generalisation" - not only for having added btrfs. That support was been in place since September 30th. Best regards, R-
On Fri, 16 Dec 2022 at 09:14, Roberto A. Foglietta <roberto.foglietta@gmail.com> wrote: > > On Fri, 16 Dec 2022 at 00:00, Roberto A. Foglietta > <roberto.foglietta@gmail.com> wrote: > > > > Hi all, > > > - udev support: udevadm settle is the first task to run > A dependency needed added, this grants that udevd and udevadm will be installed Many systems might not need udevadm settle for expanding last partition but few others which rely on separated couple of bootfs/rootfs where last partition is on a volume connected through a kind of bus which not be fully ready at boot time until udevd will end to populate the /dev tree. Moreover, expanding the last partition is a single time task which runs only on the first machine boot and then will be disabled forever. Thus, even if it would take some time on some systems, it is acceptable in order to grant to the widest variety of systems this service. Alternative is to accept that some system will be left unprovided of the storage space they have been designed for. This patch is for applying on top of the other previous 7 patches - In expand last partition few improvements and a bugfix - In expand last partition trap exitnlog EXIT logs with simpler code - The expand last partition might support full volumes also, to test - In expand last partition checking the size is useless - If expand last partition fails, unsupervised systems reboot - In expand last partition script btrfs support added - In expand last partition wait for udev have finished or their combination: - The expand last partition rework and generalisation Sent by elastic mail SMTP and went in moderation, here in attachment. Best regards, R-
On Fri, 16 Dec 2022 at 12:38, Roberto A. Foglietta <roberto.foglietta@gmail.com> wrote: > > On Fri, 16 Dec 2022 at 09:14, Roberto A. Foglietta > <roberto.foglietta@gmail.com> wrote: > > > > On Fri, 16 Dec 2022 at 00:00, Roberto A. Foglietta > > <roberto.foglietta@gmail.com> wrote: > > > > > > Hi all, > > > > > - udev support: udevadm settle is the first task to run > > > > A dependency needed added, this grants that udevd and udevadm will be installed > > Many systems might not need udevadm settle for expanding last partition but few > others which rely on separated couple of bootfs/rootfs where last > partition is on > a volume connected through a kind of bus which not be fully ready at boot time > until udevd will end to populate the /dev tree. > > Moreover, expanding the last partition is a single time task which runs only on > the first machine boot and then will be disabled forever. Thus, even if it > would take some time on some systems, it is acceptable in order to grant to the > widest variety of systems this service. Alternative is to accept that some > system will be left unprovided of the storage space they have been designed for. > > This patch is for applying on top of the other previous 7 patches > > - In expand last partition few improvements and a bugfix > - In expand last partition trap exitnlog EXIT logs with simpler code > - The expand last partition might support full volumes also, to test > - In expand last partition checking the size is useless > - If expand last partition fails, unsupervised systems reboot > - In expand last partition script btrfs support added > - In expand last partition wait for udev have finished > > or their combination: > > - The expand last partition rework and generalisation > > Sent by elastic mail SMTP and went in moderation, here in attachment. > I decided to leverage the MIT license to change the copyright notice and license after the work and generalisation. As usual, I sent this patch by elastic mail SMTP so it went on moderation. Here in attachment. This patch applies on the top of the patches listed above. From aeb885595b53d08ed101ccc08032a6dfcfe3decb Mon Sep 17 00:00:00 2001 From: "Roberto A. Foglietta" <roberto.foglietta@gmail.com> Date: Sun, 18 Dec 2022 14:10:40 +0100 Subject: [PATCH] Copyright changes of expand last partition script after rework Script to resize last partition to full medium size, changed the copyright and the license by the author of the rework and generalisation as following: . This software was originally a part of ISAR . Copyright (c) Siemens AG, 2018-2022 . SPDX-License-Identifier: MIT . . After its rework and generalisation this file was relicensed . (C) 2022, Roberto A. Foglietta <roberto.foglietta@gmail.com> . SPDX-License-Identifier: all rights reserved, but fair use allowed . Fair use includes test, learning and marketing but not sales, redistribution . leasing, renting or every other commercial/business activities without the . consent of the author. Every company or individual allowed to use this . code behind these limitations will be listed here below, if any. Signed-off-by: Roberto A. Foglietta <roberto.foglietta@gmail.com> Best regards, R-
From f5a72cdfd3f429895893412b0c835edba223c608 Mon Sep 17 00:00:00 2001 From: "Roberto A. Foglietta" <roberto.foglietta@gmail.com> Date: Sat, 10 Dec 2022 04:13:42 +0100 Subject: [PATCH 2/7] In expand last partition script btrfs support added Improvement, expand-on-first-boot: support for btrfs added v2: the mount point for btrfs filesystem to resize is under /dev/shm by default but a variable defined in the running environment can change the temporary directory path. The /dev/shm has been chosen because in some systems /tmp can be on a RO root filesystem and /tmp not yet mounted with tmpfs. This requires the system to have /dev/shm configured into the kernel. v3: the previous version was using a bashism but the shebang is /bin/sh this means that the script will fail in a customized system in which /bin/sh do not link to bash or bash is not available. v4: Felix suggested using mktemp for the temporary directory in which to mount the btrfs filesystem to resize. The default path /tmp could be changed using the variable TMPDIR to set in the running environment. v5: typo fixes into the patch description v6: mktemp does a reasonable second try with -p /dev/shm before failing Signed-off-by: Roberto A. Foglietta <roberto.foglietta@gmail.com> --- .../expand-on-first-boot_1.2.bb | 2 +- .../files/expand-last-partition.sh | 37 ++++++------------- 2 files changed, 13 insertions(+), 26 deletions(-) diff --git a/meta/recipes-support/expand-on-first-boot/expand-on-first-boot_1.2.bb b/meta/recipes-support/expand-on-first-boot/expand-on-first-boot_1.2.bb index 0996000..fe7b72b 100644 --- a/meta/recipes-support/expand-on-first-boot/expand-on-first-boot_1.2.bb +++ b/meta/recipes-support/expand-on-first-boot/expand-on-first-boot_1.2.bb @@ -10,7 +10,7 @@ inherit dpkg-raw DESCRIPTION = "This service grows the last partition to the full medium during first boot" MAINTAINER = "isar-users <isar-users@googlegroups.com>" -DEBIAN_DEPENDS = "systemd, sed, grep, coreutils, mount, e2fsprogs, fdisk (>=2.29.2-3) | util-linux (<2.29.2-3), util-linux" +DEBIAN_DEPENDS = "systemd, sed, grep, coreutils, mount, e2fsprogs, fdisk (>=2.29.2-3) | util-linux (<2.29.2-3), util-linux, btrfs-progs" SRC_URI = " \ file://expand-on-first-boot.service \ diff --git a/meta/recipes-support/expand-on-first-boot/files/expand-last-partition.sh b/meta/recipes-support/expand-on-first-boot/files/expand-last-partition.sh index 4d55645..f5bcad1 100755 --- a/meta/recipes-support/expand-on-first-boot/files/expand-last-partition.sh +++ b/meta/recipes-support/expand-on-first-boot/files/expand-last-partition.sh @@ -59,29 +59,16 @@ sfdisk -d "${BOOT_DEV}" 2>/dev/null | \ # Inform the kernel about the partitioning change partx -u "${LAST_PART}" -# this is for debian stretch or systemd < 236 -if [ ! -x /lib/systemd/systemd-growfs ]; then - # Do not fail resize2fs if no mtab entry is found, e.g., - # when using systemd mount units. - export EXT2FS_NO_MTAB_OK=1 +# Do not fail resize2fs if no mtab entry is found, e.g., +# when using systemd mount units. +export EXT2FS_NO_MTAB_OK=1 - resize2fs "${LAST_PART}" - exit 0 -fi - -if grep -q x-systemd.growfs /etc/fstab; then - echo "Found x-systemd.growfs option in /etc/fstab, won't call it explicitly." >&2 - exit 0 -fi - -# mount $LAST_PART out of tree, so we won't conflict with other mounts -MOUNT_POINT=$(mktemp -d -p /mnt "$(basename "$0").XXXXXXXXXX") -if [ ! -d "${MOUNT_POINT}" ]; then - echo "Cannot create temporary mount point ${MOUNT_POINT}." >&2 - exit 1 -fi - -mount "${LAST_PART}" "${MOUNT_POINT}" -/lib/systemd/systemd-growfs "${MOUNT_POINT}" -umount "${MOUNT_POINT}" -rmdir "${MOUNT_POINT}" +case $(lsblk -fno FSTYPE "${LAST_PART}") in + ext4) resize2fs "${LAST_PART}" + ;; + btrfs) tmpdir=$(mktemp -d -p "$TMPDIR" btrfs.XXXX || mktemp -d -p "/dev/shm" btrfs.XXXX) + mount "${LAST_PART}" $tmpdir + btrfs filesystem resize max $tmpdir + umount $tmpdir && rmdir $tmpdir + ;; +esac -- 2.34.1