| Message ID | 20240712092245.47054-1-felix.moessbauer@siemens.com |
|---|---|
| State | Accepted, archived |
| Headers | show |
| Series | [1/1] fix expansion of variables in SRC_URI of dpkg-prebuilt | expand |
On Fri, 2024-07-12 at 11:22 +0200, 'Felix Moessbauer' via isar-users wrote: > The processing of the items in SRC_URI of dpkg-prebuilt previously > was > executed on the non expanded variables. This was introduced to fix > credential leaks and to avoid absolute paths in the signatures > (caching > issues). However, this does not work when putting whole SRC_URI > entries > into variables (which potentially can be empty), as then the > unpack=false is added to the non-expanded variable which either might > already contain this, or is empty. This led to broken urls. > > To fix this, the patch changes the processing logic to work on the > expanded string. As this would re-introduce the credential and > caching > issues, we further add a vardepvalue with the non-expanded string. By > that, the signatures just contain the original string in its non > expanded version. > Yes, this change currently seems to break CI (test_sstate fails). I haven't investigated it deeply yet, so for now just attaching the log. > Signed-off-by: Felix Moessbauer <felix.moessbauer@siemens.com> > --- > meta/classes/dpkg-prebuilt.bbclass | 9 ++++++--- > 1 file changed, 6 insertions(+), 3 deletions(-) > > diff --git a/meta/classes/dpkg-prebuilt.bbclass b/meta/classes/dpkg- > prebuilt.bbclass > index a6187a07..ecf0d383 100644 > --- a/meta/classes/dpkg-prebuilt.bbclass > +++ b/meta/classes/dpkg-prebuilt.bbclass > @@ -7,13 +7,16 @@ inherit dpkg-base > > python do_unpack:prepend() { > # enforce unpack=false > - src_uri = (d.getVar('SRC_URI', False) or '').split() > - if len(src_uri) == 0: > + src_uri_raw = d.getVar('SRC_URI', False) > + src_uri_exp = (d.getVar('SRC_URI', True) or '').split() > + if len(src_uri_exp) == 0: > return > def ensure_unpack_false(uri): > return ';'.join([x for x in uri.split(';') if not > x.startswith('unpack=')] + ['unpack=false']) > - src_uri = [ensure_unpack_false(uri) for uri in src_uri] > + src_uri = [ensure_unpack_false(uri) for uri in src_uri_exp] > d.setVar('SRC_URI', ' '.join(src_uri)) > + if src_uri_raw: > + d.appendVarFlag('SRC_URI', 'vardepvalue', src_uri_raw) > } > > # also breaks inherited (from dpkg-base) dependency on sbuild_chroot > -- > 2.39.2 >
On Mon, 2024-07-22 at 11:57 +0300, Uladzimir Bely wrote: > On Fri, 2024-07-12 at 11:22 +0200, 'Felix Moessbauer' via isar-users > wrote: > > The processing of the items in SRC_URI of dpkg-prebuilt previously > > was > > executed on the non expanded variables. This was introduced to fix > > credential leaks and to avoid absolute paths in the signatures > > (caching > > issues). However, this does not work when putting whole SRC_URI > > entries > > into variables (which potentially can be empty), as then the > > unpack=false is added to the non-expanded variable which either > > might > > already contain this, or is empty. This led to broken urls. > > > > To fix this, the patch changes the processing logic to work on the > > expanded string. As this would re-introduce the credential and > > caching > > issues, we further add a vardepvalue with the non-expanded string. > > By > > that, the signatures just contain the original string in its non > > expanded version. > > > > Yes, this change currently seems to break CI (test_sstate fails). I > haven't investigated it deeply yet, so for now just attaching the > log. Hi, this looks unrelated, but uncovered a bug in the isar-sstate script. I just send a patch for that, which should be applied prior to this one: "fix(isar-sstate): continue on missing varvals value". Felix > > > Signed-off-by: Felix Moessbauer <felix.moessbauer@siemens.com> > > --- > > meta/classes/dpkg-prebuilt.bbclass | 9 ++++++--- > > 1 file changed, 6 insertions(+), 3 deletions(-) > > > > diff --git a/meta/classes/dpkg-prebuilt.bbclass > > b/meta/classes/dpkg- > > prebuilt.bbclass > > index a6187a07..ecf0d383 100644 > > --- a/meta/classes/dpkg-prebuilt.bbclass > > +++ b/meta/classes/dpkg-prebuilt.bbclass > > @@ -7,13 +7,16 @@ inherit dpkg-base > > > > python do_unpack:prepend() { > > # enforce unpack=false > > - src_uri = (d.getVar('SRC_URI', False) or '').split() > > - if len(src_uri) == 0: > > + src_uri_raw = d.getVar('SRC_URI', False) > > + src_uri_exp = (d.getVar('SRC_URI', True) or '').split() > > + if len(src_uri_exp) == 0: > > return > > def ensure_unpack_false(uri): > > return ';'.join([x for x in uri.split(';') if not > > x.startswith('unpack=')] + ['unpack=false']) > > - src_uri = [ensure_unpack_false(uri) for uri in src_uri] > > + src_uri = [ensure_unpack_false(uri) for uri in src_uri_exp] > > d.setVar('SRC_URI', ' '.join(src_uri)) > > + if src_uri_raw: > > + d.appendVarFlag('SRC_URI', 'vardepvalue', src_uri_raw) > > } > > > > # also breaks inherited (from dpkg-base) dependency on > > sbuild_chroot > > -- > > 2.39.2 > > >
On Fri, 2024-07-12 at 11:22 +0200, 'Felix Moessbauer' via isar-users wrote: > The processing of the items in SRC_URI of dpkg-prebuilt previously > was > executed on the non expanded variables. This was introduced to fix > credential leaks and to avoid absolute paths in the signatures > (caching > issues). However, this does not work when putting whole SRC_URI > entries > into variables (which potentially can be empty), as then the > unpack=false is added to the non-expanded variable which either might > already contain this, or is empty. This led to broken urls. > > To fix this, the patch changes the processing logic to work on the > expanded string. As this would re-introduce the credential and > caching > issues, we further add a vardepvalue with the non-expanded string. By > that, the signatures just contain the original string in its non > expanded version. > > Signed-off-by: Felix Moessbauer <felix.moessbauer@siemens.com> > --- > meta/classes/dpkg-prebuilt.bbclass | 9 ++++++--- > 1 file changed, 6 insertions(+), 3 deletions(-) > > diff --git a/meta/classes/dpkg-prebuilt.bbclass b/meta/classes/dpkg- > prebuilt.bbclass > index a6187a07..ecf0d383 100644 > --- a/meta/classes/dpkg-prebuilt.bbclass > +++ b/meta/classes/dpkg-prebuilt.bbclass > @@ -7,13 +7,16 @@ inherit dpkg-base > > python do_unpack:prepend() { > # enforce unpack=false > - src_uri = (d.getVar('SRC_URI', False) or '').split() > - if len(src_uri) == 0: > + src_uri_raw = d.getVar('SRC_URI', False) > + src_uri_exp = (d.getVar('SRC_URI', True) or '').split() > + if len(src_uri_exp) == 0: > return > def ensure_unpack_false(uri): > return ';'.join([x for x in uri.split(';') if not > x.startswith('unpack=')] + ['unpack=false']) > - src_uri = [ensure_unpack_false(uri) for uri in src_uri] > + src_uri = [ensure_unpack_false(uri) for uri in src_uri_exp] > d.setVar('SRC_URI', ' '.join(src_uri)) > + if src_uri_raw: > + d.appendVarFlag('SRC_URI', 'vardepvalue', src_uri_raw) > } > > # also breaks inherited (from dpkg-base) dependency on sbuild_chroot > -- > 2.39.2 > Applied to next, thanks.
diff --git a/meta/classes/dpkg-prebuilt.bbclass b/meta/classes/dpkg-prebuilt.bbclass index a6187a07..ecf0d383 100644 --- a/meta/classes/dpkg-prebuilt.bbclass +++ b/meta/classes/dpkg-prebuilt.bbclass @@ -7,13 +7,16 @@ inherit dpkg-base python do_unpack:prepend() { # enforce unpack=false - src_uri = (d.getVar('SRC_URI', False) or '').split() - if len(src_uri) == 0: + src_uri_raw = d.getVar('SRC_URI', False) + src_uri_exp = (d.getVar('SRC_URI', True) or '').split() + if len(src_uri_exp) == 0: return def ensure_unpack_false(uri): return ';'.join([x for x in uri.split(';') if not x.startswith('unpack=')] + ['unpack=false']) - src_uri = [ensure_unpack_false(uri) for uri in src_uri] + src_uri = [ensure_unpack_false(uri) for uri in src_uri_exp] d.setVar('SRC_URI', ' '.join(src_uri)) + if src_uri_raw: + d.appendVarFlag('SRC_URI', 'vardepvalue', src_uri_raw) } # also breaks inherited (from dpkg-base) dependency on sbuild_chroot
The processing of the items in SRC_URI of dpkg-prebuilt previously was executed on the non expanded variables. This was introduced to fix credential leaks and to avoid absolute paths in the signatures (caching issues). However, this does not work when putting whole SRC_URI entries into variables (which potentially can be empty), as then the unpack=false is added to the non-expanded variable which either might already contain this, or is empty. This led to broken urls. To fix this, the patch changes the processing logic to work on the expanded string. As this would re-introduce the credential and caching issues, we further add a vardepvalue with the non-expanded string. By that, the signatures just contain the original string in its non expanded version. Signed-off-by: Felix Moessbauer <felix.moessbauer@siemens.com> --- meta/classes/dpkg-prebuilt.bbclass | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-)