[1/3] fix(isar-sstate): catch errors in signature comparison

Message ID 20220719061628.192078-2-adriaan.schmidt@siemens.com
State Superseded, archived
Headers show
Series isar-sstate improvements | expand

Commit Message

Schmidt, Adriaan July 18, 2022, 10:16 p.m. UTC
Those errors happen when the sstate cache contains both pre and post
bitbake-2.0 signatures, which have a different format (pickle vs. compressed json).

Signed-off-by: Adriaan Schmidt <adriaan.schmidt@siemens.com>
---
 scripts/isar-sstate | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

Comments

Henning Schild July 18, 2022, 11:05 p.m. UTC | #1
Am Tue, 19 Jul 2022 08:16:26 +0200
schrieb Adriaan Schmidt <adriaan.schmidt@siemens.com>:

> Those errors happen when the sstate cache contains both pre and post
> bitbake-2.0 signatures, which have a different format (pickle vs.
> compressed json).
> 
> Signed-off-by: Adriaan Schmidt <adriaan.schmidt@siemens.com>
> ---
>  scripts/isar-sstate | 5 ++++-
>  1 file changed, 4 insertions(+), 1 deletion(-)
> 
> diff --git a/scripts/isar-sstate b/scripts/isar-sstate
> index 8ea85edc..9f5c17b0 100755
> --- a/scripts/isar-sstate
> +++ b/scripts/isar-sstate
> @@ -781,7 +781,10 @@ def sstate_analyze(source, target, **kwargs):
>  
>              local_file = source.download(s.path)
>              remote_file = target.download(t.path)
> -            out = compare_sigfiles(remote_file, local_file,
> recursecb, color=True)
> +            try:
> +                out = compare_sigfiles(remote_file, local_file,
> recursecb, color=True)
> +            except:
> +                out = ["Failed to compare signatures."]

Maybe there is a python exception for just that, or an error code from
which we can take strerr so we do not invent our own.

Henning

>              source.release(local_file)
>              target.release(remote_file)
>              # shorten hashes from 64 to 8 characters for better
> readability
Schmidt, Adriaan July 19, 2022, 12:11 a.m. UTC | #2
Schild, Henning, Dienstag, 19. Juli 2022 09:05:
> Am Tue, 19 Jul 2022 08:16:26 +0200
> schrieb Adriaan Schmidt <adriaan.schmidt@siemens.com>:
> 
> > Those errors happen when the sstate cache contains both pre and post
> > bitbake-2.0 signatures, which have a different format (pickle vs.
> > compressed json).
> >
> > Signed-off-by: Adriaan Schmidt <adriaan.schmidt@siemens.com>
> > ---
> >  scripts/isar-sstate | 5 ++++-
> >  1 file changed, 4 insertions(+), 1 deletion(-)
> >
> > diff --git a/scripts/isar-sstate b/scripts/isar-sstate
> > index 8ea85edc..9f5c17b0 100755
> > --- a/scripts/isar-sstate
> > +++ b/scripts/isar-sstate
> > @@ -781,7 +781,10 @@ def sstate_analyze(source, target, **kwargs):
> >
> >              local_file = source.download(s.path)
> >              remote_file = target.download(t.path)
> > -            out = compare_sigfiles(remote_file, local_file,
> > recursecb, color=True)
> > +            try:
> > +                out = compare_sigfiles(remote_file, local_file,
> > recursecb, color=True)
> > +            except:
> > +                out = ["Failed to compare signatures."]
> 
> Maybe there is a python exception for just that, or an error code from
> which we can take strerr so we do not invent our own.

The exceptions we see here come from different locations during file reading.
I'm seeing things like EOFError or _pickle.UnpicklingError, and when we move
to bitbake 2.0 it may be something compression or json related.
The strings returned are not really useful (e.g. from the unpickler it
might be "Memo value not found at index 97"), that's why I went for
"catch anything and move on".

But I just noticed that sstate_lint() also needs a try/catch when reading
signature files. Will send update.

Adriaan


> Henning
> 
> >              source.release(local_file)
> >              target.release(remote_file)
> >              # shorten hashes from 64 to 8 characters for better
> > readability

Patch

diff --git a/scripts/isar-sstate b/scripts/isar-sstate
index 8ea85edc..9f5c17b0 100755
--- a/scripts/isar-sstate
+++ b/scripts/isar-sstate
@@ -781,7 +781,10 @@  def sstate_analyze(source, target, **kwargs):
 
             local_file = source.download(s.path)
             remote_file = target.download(t.path)
-            out = compare_sigfiles(remote_file, local_file, recursecb, color=True)
+            try:
+                out = compare_sigfiles(remote_file, local_file, recursecb, color=True)
+            except:
+                out = ["Failed to compare signatures."]
             source.release(local_file)
             target.release(remote_file)
             # shorten hashes from 64 to 8 characters for better readability