image: check if the file is core dump

Message ID 20240416051222.3344127-1-zhibin.dong@siemens.com
State Superseded, archived
Headers show
Series image: check if the file is core dump | expand

Commit Message

Zhibin Dong April 16, 2024, 5:12 a.m. UTC
The previous code does a wrong judgement in two cases:
1. a file is suffixed by .core but is not a core dump file
2. a file is a core dump file but is not suffixed by .core

The new code uses `file` to determine the type of files, which is more
accurate.

Signed-off-by: Zhibin Dong <zhibin.dong@siemens.com>
---
 meta/classes/image.bbclass | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

Comments

MOESSBAUER, Felix April 16, 2024, 9:02 p.m. UTC | #1
On Tue, 2024-04-16 at 13:12 +0800, Zhibin Dong wrote:
> The previous code does a wrong judgement in two cases:
> 1. a file is suffixed by .core but is not a core dump file
> 2. a file is a core dump file but is not suffixed by .core

Hi, just for reference: This is an addition to the pattern implemented
in https://groups.google.com/g/isar-users/c/JtCExK7m4OY/m/F9QaxKBSAQAJ

> 
> The new code uses `file` to determine the type of files, which is
> more
> accurate.
> 
> Signed-off-by: Zhibin Dong <zhibin.dong@siemens.com>
> ---
>  meta/classes/image.bbclass | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass
> index 98741da0..10923947 100644
> --- a/meta/classes/image.bbclass
> +++ b/meta/classes/image.bbclass
> @@ -444,7 +444,7 @@ EOSUDO
>  
>      # Sometimes qemu-user-static generates coredumps in chroot, move
> them
>      # to work temporary directory and inform user about it.
> -    for f in $(sudo find ${ROOTFSDIR} -type f -name *.core); do
> +    for f in $(sudo find ${ROOTFSDIR} -type f -exec file {} \; |

Are we sure this works in cross-arch scenarios? I.e. can an amd64
"file" detect arm64 coredumps? Did you test this Zhibin Dong?
If so, this patch is fine.

Best regards,
Felix

> grep 'core file' | cut -d: -f1); do
>          sudo mv "${f}" "${WORKDIR}/temp/"
>          bbwarn "found core dump in rootfs, check it in
> ${WORKDIR}/temp/${f##*/}"
>      done
> -- 
> 2.39.2
>
Zhibin Dong April 17, 2024, 2:11 a.m. UTC | #2
> Are we sure this works in cross-arch scenarios? I.e. can an amd64 "file" 
detect arm64 coredumps? Did you test this Zhibin Dong?
> If so, this patch is fine. 

On an amd64 platform, I execute these commands.
$ file ./core.amd64
./core.amd64: ELF 64-bit LSB core file, x86-64, version 1 (SYSV), 
SVR4-style, from './crashing_program', real uid: 1000, effective uid: 1000, 
real gid: 1000, effective gid: 1000, execfn: './crashing_program', 
platform: 'x86_64'
$ file ./core.aarch64
./core.aarch64: ELF 64-bit LSB core file, ARM aarch64, version 1 (SYSV), 
SVR4-style, from './crashing_program_aarch64', real uid: 1000, effective 
uid: 1000, real gid: 1001, effective gid: 1001, execfn: 
'./crashing_program_aarch64', platform: 'aarch64'

`file` can determine the architecture of a file and also determine if it is 
a core file.
在2024年4月17日星期三 UTC+8 05:02:35<MOESSBAUER, Felix> 写道:

> On Tue, 2024-04-16 at 13:12 +0800, Zhibin Dong wrote:
> > The previous code does a wrong judgement in two cases:
> > 1. a file is suffixed by .core but is not a core dump file
> > 2. a file is a core dump file but is not suffixed by .core
>
> Hi, just for reference: This is an addition to the pattern implemented
> in https://groups.google.com/g/isar-users/c/JtCExK7m4OY/m/F9QaxKBSAQAJ
>
> > 
> > The new code uses `file` to determine the type of files, which is
> > more
> > accurate.
> > 
> > Signed-off-by: Zhibin Dong <zhibi...@siemens.com>
> > ---
> >  meta/classes/image.bbclass | 2 +-
> >  1 file changed, 1 insertion(+), 1 deletion(-)
> > 
> > diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass
> > index 98741da0..10923947 100644
> > --- a/meta/classes/image.bbclass
> > +++ b/meta/classes/image.bbclass
> > @@ -444,7 +444,7 @@ EOSUDO
> >  
> >      # Sometimes qemu-user-static generates coredumps in chroot, move
> > them
> >      # to work temporary directory and inform user about it.
> > -    for f in $(sudo find ${ROOTFSDIR} -type f -name *.core); do
> > +    for f in $(sudo find ${ROOTFSDIR} -type f -exec file {} \; |
>
> Are we sure this works in cross-arch scenarios? I.e. can an amd64
> "file" detect arm64 coredumps? Did you test this Zhibin Dong?
> If so, this patch is fine.
>
> Best regards,
> Felix
>
> > grep 'core file' | cut -d: -f1); do
> >          sudo mv "${f}" "${WORKDIR}/temp/"
> >          bbwarn "found core dump in rootfs, check it in
> > ${WORKDIR}/temp/${f##*/}"
> >      done
> > -- 
> > 2.39.2
> > 
>
> -- 
> Siemens AG, Technology
> Linux Expert Center
>
>
>
Schmidt, Adriaan April 17, 2024, 4:57 a.m. UTC | #3
Zhibin Dong, Dienstag, 16. April 2024 07:12
> 
> The previous code does a wrong judgement in two cases:
> 1. a file is suffixed by .core but is not a core dump file
> 2. a file is a core dump file but is not suffixed by .core
> 
> The new code uses `file` to determine the type of files, which is more
> accurate.
> 
> Signed-off-by: Zhibin Dong <zhibin.dong@siemens.com>
> ---
>  meta/classes/image.bbclass | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass
> index 98741da0..10923947 100644
> --- a/meta/classes/image.bbclass
> +++ b/meta/classes/image.bbclass
> @@ -444,7 +444,7 @@ EOSUDO
> 
>      # Sometimes qemu-user-static generates coredumps in chroot, move them
>      # to work temporary directory and inform user about it.
> -    for f in $(sudo find ${ROOTFSDIR} -type f -name *.core); do
> +    for f in $(sudo find ${ROOTFSDIR} -type f -exec file {} \; | grep 'core

This runs `file` on every file in the rootfs, which I expect can take a long time.

Also, if you check `file -l | grep core` you will see that not all file types
that have "core" in their description are actually core dumps.

I don't know the details of why we have this code [1], but maybe "scan the whole
rootfs" is not the best solution to the problem...

When specifically can those core dumps happen?
Is it only during update-initramfs, which is mentioned in the bug linked to the original commit [2]?
Maybe also during package installation, which may run commands with qemu-user?
Can we reproduce this?
Where in the rootfs are they stored?
Can our search for them be more targeted?
Would such core dumps be caught by other checks we have in place (e.g., modification time of files)?

Adriaan

[1] introduced in fa10b1d9b3a5e876bbcf556b03d585bf712fa7a5
[2] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1040981

> file' | cut -d: -f1); do
>          sudo mv "${f}" "${WORKDIR}/temp/"
>          bbwarn "found core dump in rootfs, check it in
> ${WORKDIR}/temp/${f##*/}"
>      done
> --
> 2.39.2
> 
> --
> You received this message because you are subscribed to the Google Groups
> "isar-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to isar-users+unsubscribe@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/isar-users/20240416051222.3344127-1-
> zhibin.dong%40siemens.com.

Patch

diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass
index 98741da0..10923947 100644
--- a/meta/classes/image.bbclass
+++ b/meta/classes/image.bbclass
@@ -444,7 +444,7 @@  EOSUDO
 
     # Sometimes qemu-user-static generates coredumps in chroot, move them
     # to work temporary directory and inform user about it.
-    for f in $(sudo find ${ROOTFSDIR} -type f -name *.core); do
+    for f in $(sudo find ${ROOTFSDIR} -type f -exec file {} \; | grep 'core file' | cut -d: -f1); do
         sudo mv "${f}" "${WORKDIR}/temp/"
         bbwarn "found core dump in rootfs, check it in ${WORKDIR}/temp/${f##*/}"
     done