mbox series

[0/1] fix copy out of apt cache after sbuild

Message ID 20240122132647.779902-1-felix.moessbauer@siemens.com
Headers show
Series fix copy out of apt cache after sbuild | expand

Message

MOESSBAUER, Felix Jan. 22, 2024, 1:26 p.m. UTC
This series fixes the build issues reported in "potential race
condition in sbuild post commands" by not copying symlinks. This has
been tested on various large layers and seems to do the job. However,
it still remains unclear why packages are missing in the apt cache
under some rebuild scenarios.

While investigating this, I also noticed that we AGAIN have a quadratic
space blow-up in the isar-apt directory that is copied into the build
folder of each package. This makes me wonder if the whole APT handling
is properly understood. IMHO this needs another haul-over, which is also
indicated by the number of apt related issues which were reported in the
last months.

Best regards,
Felix

Felix Moessbauer (1):
  fix copy out of apt cache after sbuild

 meta/classes/dpkg.bbclass | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

Comments

Schmidt, Adriaan Jan. 22, 2024, 1:47 p.m. UTC | #1
'Felix Moessbauer' via isar-users <isar-users@googlegroups.com>, Monday, January 22, 2024 2:27 PM
> This series fixes the build issues reported in "potential race
> condition in sbuild post commands" by not copying symlinks. This has
> been tested on various large layers and seems to do the job. However,
> it still remains unclear why packages are missing in the apt cache
> under some rebuild scenarios.
> 
> While investigating this, I also noticed that we AGAIN have a quadratic
> space blow-up in the isar-apt directory that is copied into the build
> folder of each package. This makes me wonder if the whole APT handling
> is properly understood. IMHO this needs another haul-over, which is also
> indicated by the number of apt related issues which were reported in the
> last months.

https://github.com/ilbers/isar/blob/master/meta/classes/dpkg-base.bbclass#L198

Looking at the code, the intention is that hard links are used to pass isar-apt
into each package build. Could your measurement be counting those wrong?

And if we do think that another approach at handling the whole apt topic is worth
considering: I posted about that a while back ("[RFC PATCH] Remove isar-apt and
the corresponding lock" from 2022-10-19)...

Adriaan

> 
> Best regards,
> Felix
> 
> Felix Moessbauer (1):
>   fix copy out of apt cache after sbuild
> 
>  meta/classes/dpkg.bbclass | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> --
> 2.39.2
> 
> --
> You received this message because you are subscribed to the Google Groups
> "isar-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to isar-users+unsubscribe@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/isar-users/20240122132647.779902-1-
> felix.moessbauer%40siemens.com.
MOESSBAUER, Felix Jan. 22, 2024, 3:04 p.m. UTC | #2
On Mon, 2024-01-22 at 13:47 +0000, Schmidt, Adriaan (T CED EDC-DE)
wrote:
> 'Felix Moessbauer' via isar-users <isar-users@googlegroups.com>,
> Monday, January 22, 2024 2:27 PM
> > This series fixes the build issues reported in "potential race
> > condition in sbuild post commands" by not copying symlinks. This
> > has
> > been tested on various large layers and seems to do the job.
> > However,
> > it still remains unclear why packages are missing in the apt cache
> > under some rebuild scenarios.
> > 
> > While investigating this, I also noticed that we AGAIN have a
> > quadratic
> > space blow-up in the isar-apt directory that is copied into the
> > build
> > folder of each package. This makes me wonder if the whole APT
> > handling
> > is properly understood. IMHO this needs another haul-over, which is
> > also
> > indicated by the number of apt related issues which were reported
> > in the
> > last months.
> 
> https://github.com/ilbers/isar/blob/master/meta/classes/dpkg-base.bbclass#L198
> 
> Looking at the code, the intention is that hard links are used to
> pass isar-apt
> into each package build. Could your measurement be counting those
> wrong?

Right. Thanks for the pointer. That's why I prefer to use the long
options of the commands.

> 
> And if we do think that another approach at handling the whole apt
> topic is worth
> considering: I posted about that a while back ("[RFC PATCH] Remove
> isar-apt and
> the corresponding lock" from 2022-10-19)...
> 

I still believe this is a good idea.

Felix

> Adriaan
> 
> > 
> > Best regards,
> > Felix
> > 
> > Felix Moessbauer (1):
> >   fix copy out of apt cache after sbuild
> > 
> >  meta/classes/dpkg.bbclass | 2 +-
> >  1 file changed, 1 insertion(+), 1 deletion(-)
> > 
> > --
> > 2.39.2
> > 
> > --
> > You received this message because you are subscribed to the Google
> > Groups
> > "isar-users" group.
> > To unsubscribe from this group and stop receiving emails from it,
> > send an
> > email to isar-users+unsubscribe@googlegroups.com.
> > To view this discussion on the web visit
> > https://groups.google.com/d/msgid/isar-users/20240122132647.779902-1
> > -
> > felix.moessbauer%40siemens.com.
Uladzimir Bely Jan. 29, 2024, 10:50 a.m. UTC | #3
On Mon, 2024-01-22 at 14:26 +0100, 'Felix Moessbauer' via isar-users
wrote:
> This series fixes the build issues reported in "potential race
> condition in sbuild post commands" by not copying symlinks. This has
> been tested on various large layers and seems to do the job. However,
> it still remains unclear why packages are missing in the apt cache
> under some rebuild scenarios.
> 
> While investigating this, I also noticed that we AGAIN have a
> quadratic
> space blow-up in the isar-apt directory that is copied into the build
> folder of each package. This makes me wonder if the whole APT
> handling
> is properly understood. IMHO this needs another haul-over, which is
> also
> indicated by the number of apt related issues which were reported in
> the
> last months.
> 
> Best regards,
> Felix
> 
> Felix Moessbauer (1):
>   fix copy out of apt cache after sbuild
> 
>  meta/classes/dpkg.bbclass | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> -- 
> 2.39.2
> 

Applied to next, thanks.