[1/1] bugfix: isar-exclude-docs delete man folders but some packages expect that folders

Hi all,

bugfix: isar-exclude-docs delete man folders but some packages expect that
folders.
solution: adopts the same approach used for the doc folder.

https://github.com/robang74/isar/commit/c656e6df63aa76701e34d7ee3f99c81e96312b9d

-exec rm -rf {} \;

Here you are the error during the configuration of the package:

Setting up openjdk-11-jre-headless:amd64 (11.0.16+8-1~deb11u1) ...
update-alternatives: using /usr/lib/jvm/java-11-openjdk-amd64/bin/java to
provide /usr/bin/java (java) in auto mode
update-alternatives: error: error creating symbolic link
'/usr/share/man/man1/java.1.gz.dpkg-tmp': No such file or directory
dpkg: error processing package openjdk-11-jre-headless:amd64 (--configure):
 installed openjdk-11-jre-headless:amd64 package post-installation script
subprocess returned error exit status 2
dpkg: dependency problems prevent configuration of openjdk-11-jre:amd64:
 openjdk-11-jre:amd64 depends on openjdk-11-jre-headless (=
11.0.16+8-1~deb11u1); however:
  Package openjdk-11-jre-headless:amd64 is not configured yet.

Cheers, R-

Il giorno gio 22 set 2022 alle ore 22:36 Roberto A. Foglietta <
roberto.foglietta@gmail.com> ha scritto:
>
> Hi all,
>
> bugfix: isar-exclude-docs delete man folders but some packages expect
that folders.
> solution: adopts the same approach used for the doc folder.
>
>
https://github.com/robang74/isar/commit/c656e6df63aa76701e34d7ee3f99c81e96312b9d

bugfix: isar-exclude-docs delete man folders but some packages expect that
folders, p2
Instead of using "-type f", it is better to use "! -type d" in order to
remove also the links

find /usr/share/man/ ! -type d -exec rm -rf {} \;

https://github.com/robang74/isar/commit/7af338b06c142eb7a60a881aaf5a9870e375c1c2

Bests, R-

> From: isar-users@googlegroups.com <isar-users@googlegroups.com> On Behalf Of Roberto A. Foglietta
> Sent: Thursday, September 22, 2022 10:36 PM
> To: isar-users@googlegroups.com
> Subject: [PATCH 1/1] bugfix: isar-exclude-docs delete man folders but some packages expect that folders
> 
> Hi all,
> 
> bugfix: isar-exclude-docs delete man folders but some packages expect that folders. 
> solution: adopts the same approach used for the doc folder.

Hi Roberto,

There are two things to the removal of the docs:

1. no package should assume that some folders that are not provided by a direct dependency exists (=> packaging bug of openjdk-11-jre-headless)
2. I'm really wondering if simply clearing these folders is the right approach.
Instead, you could instruct dpkg via dpkg.conf to not install files into particular paths:

Simply add a drop-in to: /etc/dpkg/dpkg.cfg.d/ with dpkg options for path exclusions:

# Delete man pages
path-exclude=/usr/share/man/*

# Delete docs
path-exclude=/usr/share/doc/*
path-include=/usr/share/doc/*/copyright

Of course, this has to be done prior to any package installation and at best not from a package itself.
This would also solve the issue that docs are installed when installing packages into the live image (outside of ISAR).

Felix

> 
> https://github.com/robang74/isar/commit/c656e6df63aa76701e34d7ee3f99c81e96312b9d
> 
> diff --git a/meta/recipes-support/isar-exclude-docs/files/postinst b/meta/recipes-support/isar-exclude-docs/files/postinst
> index d34c04e..581cdc1 100644
> --- a/meta/recipes-support/isar-exclude-docs/files/postinst
> +++ b/meta/recipes-support/isar-exclude-docs/files/postinst
> @@ -3,5 +3,5 @@
>  # time and do not need to be "the first" package
>  # what we delete needs to be in sync with the dpkg configuration we ship
>  
> -rm -rf /usr/share/man/*
> +find /usr/share/man/ -type f -exec rm -rf {} \;
>  find /usr/share/doc/ -type f ! -name "copyright" ! -name "changelog.*" -exec rm -rf {} \;
> 
> Here you are the error during the configuration of the package:
> 
> Setting up openjdk-11-jre-headless:amd64 (11.0.16+8-1~deb11u1) ...
> update-alternatives: using /usr/lib/jvm/java-11-openjdk-amd64/bin/java to provide /usr/bin/java (java) in auto mode
> update-alternatives: error: error creating symbolic link '/usr/share/man/man1/java.1.gz.dpkg-tmp': No such file or directory
> dpkg: error processing package openjdk-11-jre-headless:amd64 (--configure):
>  installed openjdk-11-jre-headless:amd64 package post-installation script subprocess returned error exit status 2
> dpkg: dependency problems prevent configuration of openjdk-11-jre:amd64:
>  openjdk-11-jre:amd64 depends on openjdk-11-jre-headless (= 11.0.16+8-1~deb11u1); however:
>   Package openjdk-11-jre-headless:amd64 is not configured yet.
> 
> Cheers, R-

Hi,

you could simply not install "isar-exclude-docs", it is kind of a hack
which removes files that are very likely not needed. You found a case
where one file is needed after all.

Whatever we do we have to keep the postinst and the
/etc/dpkg/dpkg.conf.d/ snippet in sync. So you can not decide to keep
directories in one approach and drop them with the other.

The snippet is a one-to-one copy from ubuntu Docker containers, would
not want to fork it. So if we have to change it, we should also report
that to ubuntu.

Long story short, i think it will be a rather trivial fix in the
postinst of that java package. That very java package might show the
same problem in a ubuntu container ... which would be a nice repro to
motivate the maintainer to take a patch which deals with a file missing
... that in a plain debian would never be missing.

So i would say someone has to open an issue on salsa, maybe an MR.
Mention isar-exclude-docs and ubuntu docker images as examples where
the files and folders can be missing.

In the meantime you can not install isar-exlcude-docs, or install the
java package before it, or patch that postinst locally by rebuilding
the java package, or write a dummy package that does an "mkdir" in
postinst and install it after exclude and before java ... i think just
installing the docs is the easiest and will not cost you too much space.

Henning

Am Thu, 22 Sep 2022 22:36:22 +0200
schrieb "Roberto A. Foglietta" <roberto.foglietta@gmail.com>:

> Hi all,
> 
> bugfix: isar-exclude-docs delete man folders but some packages expect
> that folders.
> solution: adopts the same approach used for the doc folder.
> 
> https://github.com/robang74/isar/commit/c656e6df63aa76701e34d7ee3f99c81e96312b9d
> 
> diff --git a/meta/recipes-support/isar-exclude-docs/files/postinst
> b/meta/recipes-support/isar-exclude-docs/files/postinst
> index d34c04e..581cdc1 100644
> --- a/meta/recipes-support/isar-exclude-docs/files/postinst
> +++ b/meta/recipes-support/isar-exclude-docs/files/postinst
> @@ -3,5 +3,5 @@
>  # time and do not need to be "the first" package
>  # what we delete needs to be in sync with the dpkg configuration we
> ship
> 
> -rm -rf /usr/share/man/*
> +find /usr/share/man/ -type f -exec rm -rf {} \;
>  find /usr/share/doc/ -type f ! -name "copyright" ! -name
> "changelog.*" -exec rm -rf {} \;
> 
> Here you are the error during the configuration of the package:
> 
> Setting up openjdk-11-jre-headless:amd64 (11.0.16+8-1~deb11u1) ...
> update-alternatives: using
> /usr/lib/jvm/java-11-openjdk-amd64/bin/java to provide /usr/bin/java
> (java) in auto mode update-alternatives: error: error creating
> symbolic link '/usr/share/man/man1/java.1.gz.dpkg-tmp': No such file
> or directory dpkg: error processing package
> openjdk-11-jre-headless:amd64 (--configure): installed
> openjdk-11-jre-headless:amd64 package post-installation script
> subprocess returned error exit status 2 dpkg: dependency problems
> prevent configuration of openjdk-11-jre:amd64: openjdk-11-jre:amd64
> depends on openjdk-11-jre-headless (= 11.0.16+8-1~deb11u1); however:
>   Package openjdk-11-jre-headless:amd64 is not configured yet.
> 
> Cheers, R-
>

Am Fri, 23 Sep 2022 10:57:11 +0200
schrieb Henning Schild <henning.schild@siemens.com>:

> Hi,
> 
> you could simply not install "isar-exclude-docs", it is kind of a hack
> which removes files that are very likely not needed. You found a case
> where one file is needed after all.
> 
> Whatever we do we have to keep the postinst and the
> /etc/dpkg/dpkg.conf.d/ snippet in sync. So you can not decide to keep
> directories in one approach and drop them with the other.
> 
> The snippet is a one-to-one copy from ubuntu Docker containers, would
> not want to fork it. So if we have to change it, we should also report
> that to ubuntu.
> 
> Long story short, i think it will be a rather trivial fix in the
> postinst of that java package. That very java package might show the
> same problem in a ubuntu container ... which would be a nice repro to
> motivate the maintainer to take a patch which deals with a file
> missing ... that in a plain debian would never be missing.

Here is the related ubuntu code, or better a previous "collarboration
issue" between them and isar.

https://github.com/tianon/docker-brew-ubuntu-core/issues/216

Henning

> So i would say someone has to open an issue on salsa, maybe an MR.
> Mention isar-exclude-docs and ubuntu docker images as examples where
> the files and folders can be missing.
> 
> In the meantime you can not install isar-exlcude-docs, or install the
> java package before it, or patch that postinst locally by rebuilding
> the java package, or write a dummy package that does an "mkdir" in
> postinst and install it after exclude and before java ... i think just
> installing the docs is the easiest and will not cost you too much
> space.
> 
> Henning
> 
> Am Thu, 22 Sep 2022 22:36:22 +0200
> schrieb "Roberto A. Foglietta" <roberto.foglietta@gmail.com>:
> 
> > Hi all,
> > 
> > bugfix: isar-exclude-docs delete man folders but some packages
> > expect that folders.
> > solution: adopts the same approach used for the doc folder.
> > 
> > https://github.com/robang74/isar/commit/c656e6df63aa76701e34d7ee3f99c81e96312b9d
> > 
> > diff --git a/meta/recipes-support/isar-exclude-docs/files/postinst
> > b/meta/recipes-support/isar-exclude-docs/files/postinst
> > index d34c04e..581cdc1 100644
> > --- a/meta/recipes-support/isar-exclude-docs/files/postinst
> > +++ b/meta/recipes-support/isar-exclude-docs/files/postinst
> > @@ -3,5 +3,5 @@
> >  # time and do not need to be "the first" package
> >  # what we delete needs to be in sync with the dpkg configuration we
> > ship
> > 
> > -rm -rf /usr/share/man/*
> > +find /usr/share/man/ -type f -exec rm -rf {} \;
> >  find /usr/share/doc/ -type f ! -name "copyright" ! -name
> > "changelog.*" -exec rm -rf {} \;
> > 
> > Here you are the error during the configuration of the package:
> > 
> > Setting up openjdk-11-jre-headless:amd64 (11.0.16+8-1~deb11u1) ...
> > update-alternatives: using
> > /usr/lib/jvm/java-11-openjdk-amd64/bin/java to provide /usr/bin/java
> > (java) in auto mode update-alternatives: error: error creating
> > symbolic link '/usr/share/man/man1/java.1.gz.dpkg-tmp': No such file
> > or directory dpkg: error processing package
> > openjdk-11-jre-headless:amd64 (--configure): installed
> > openjdk-11-jre-headless:amd64 package post-installation script
> > subprocess returned error exit status 2 dpkg: dependency problems
> > prevent configuration of openjdk-11-jre:amd64: openjdk-11-jre:amd64
> > depends on openjdk-11-jre-headless (= 11.0.16+8-1~deb11u1); however:
> >   Package openjdk-11-jre-headless:amd64 is not configured yet.
> > 
> > Cheers, R-
> >   
>

Dear Felix and Henning,

 - I confirm that there is a bug into the openjdk-11-jre-headless package

 - I confirm that I can avoid to use isar-exclude-doc or doing it in my way

 - I did not changed the openjdk-11-jre-headless package but it is buggy by
itself

With these premises, every ISAR user can fall in my same condition: use the
ISAR doc exclusion and include a buggy package.

IMHO, the ISAR should be fault tollerant and should not break the building
just because someone took the assumption that nobody out there makes
mistakes or the generic ISAR user can solve in a minute any fault.

Thus, please consider to apply the patch and possible also the one about
depmod -a in a for loop. :-)

Thanks, R-

Am Fri, 23 Sep 2022 11:31:42 +0200
schrieb "Roberto A. Foglietta" <roberto.foglietta@gmail.com>:

> Dear Felix and Henning,
> 
>  - I confirm that there is a bug into the openjdk-11-jre-headless
> package
> 
>  - I confirm that I can avoid to use isar-exclude-doc or doing it in
> my way
> 
>  - I did not changed the openjdk-11-jre-headless package but it is
> buggy by itself
> 
> With these premises, every ISAR user can fall in my same condition:
> use the ISAR doc exclusion and include a buggy package.

Which would rather speak for making it more clear that the package can
cause trouble and by installing it one is leaving the "good debian
path".

> IMHO, the ISAR should be fault tollerant and should not break the
> building just because someone took the assumption that nobody out
> there makes mistakes or the generic ISAR user can solve in a minute
> any fault.

Including work arounds for bugs in other components is something that
can be done if really needed. But certainly not the first thing to do,
especially if the work-around can be done in a layer instead of the
core.

Such things are tricky and we try to help here when users come and ask.

> Thus, please consider to apply the patch and possible also the one
> about depmod -a in a for loop. :-)

The patch is wrong and only does half, if you turn the install order
around and call that update-alternative you will see the problem again.
Sorry ... no.

And i do not remember that depmod loop patch, but also a bug of another
component that can be worked around and does not need core support.

Thanks for pointing out the problems you found, but i am afraid those
two are not to be solved in Isar itself.

Please go ahead and patch that openjdk thingy in salsa, i bet they
might merge faster than we can discuss here. And the problem will be
solved in the right place.
If you can quickly point out 5 prominent packages that really need
their man-pages to function ... or wait no ... empty directories ... we
can think about patching the core.

Henning


> Thanks, R-

Am Thu, 22 Sep 2022 23:31:27 +0200
schrieb "Roberto A. Foglietta" <roberto.foglietta@gmail.com>:

> Il giorno gio 22 set 2022 alle ore 22:36 Roberto A. Foglietta <
> roberto.foglietta@gmail.com> ha scritto:
> >
> > Hi all,
> >
> > bugfix: isar-exclude-docs delete man folders but some packages
> > expect  
> that folders.
> > solution: adopts the same approach used for the doc folder.
> >
> >  
> https://github.com/robang74/isar/commit/c656e6df63aa76701e34d7ee3f99c81e96312b9d
> 
> bugfix: isar-exclude-docs delete man folders but some packages expect
> that folders, p2
> Instead of using "-type f", it is better to use "! -type d" in order
> to remove also the links
> 
> find /usr/share/man/ ! -type d -exec rm -rf {} \;
> 
> https://github.com/robang74/isar/commit/7af338b06c142eb7a60a881aaf5a9870e375c1c2

I double checked a ubuntu container and was expecting that jdk package
to not install there. But it worked.
Turns out the dpkg path-exclude in fact leaves directories in place.
Which means your patch is correct and fixes a problem where the
path-exclude and the rm do not lead to the same result ... not even
thinking about jdk.

Can you please send the patch with git send-email so it can be reviewed
and merged? I think i would make it two patches ... and that switches
to "find -type f exec rm", and a second one that switches to "! -type
d" ... but both finds not just one.

regards,
Henning

> Bests, R-
>

Il Ven 23 Set 2022, 14:02 Henning Schild <henning.schild@siemens.com> ha
scritto:

>
> Can you please send the patch with git send-email so it can be reviewed
> and merged? I think i would make it two patches ... and that switches
> to "find -type f exec rm", and a second one that switches to "! -type
> d" ... but both finds not just one.
>

Dear Henning,

 thanks for the consideration. Unfortunately at the moment, I can just
attach the file patch to this e-mail. It is the 0001.

 Plus, I am going to attach other two patches that I have sent in this list
in the last week. It is the best that I can provide you. If not enough, I
will try to do something better in the week end.

 Unfortunately, I gave a company win laptop and I run Linux on a unsecure
machine on my desktop thus any personal configuration would be not
acceptable. That's my life for now.

  Best, R-

> --
Roberto A. Foglietta
+49.176.274.75.661
+39.349.33.30.697

Am Fri, 23 Sep 2022 16:18:12 +0200
schrieb "Roberto A. Foglietta" <roberto.foglietta@gmail.com>:

> Il Ven 23 Set 2022, 14:02 Henning Schild <henning.schild@siemens.com>
> ha scritto:
> 
> >
> > Can you please send the patch with git send-email so it can be
> > reviewed and merged? I think i would make it two patches ... and
> > that switches to "find -type f exec rm", and a second one that
> > switches to "! -type d" ... but both finds not just one.
> >  
> 
> Dear Henning,
> 
>  thanks for the consideration. Unfortunately at the moment, I can just
> attach the file patch to this e-mail. It is the 0001.
> 
>  Plus, I am going to attach other two patches that I have sent in
> this list in the last week. It is the best that I can provide you. If
> not enough, I will try to do something better in the week end.
> 
>  Unfortunately, I gave a company win laptop and I run Linux on a
> unsecure machine on my desktop thus any personal configuration would
> be not acceptable. That's my life for now.

Well yes ... i know all that all too well. I bet if a security expert
looked at your Windows and compared it to that Linux the judgement of
which is less secure might be another than what your IT says.

Any machine on a network where you can smtp to gmail will work, it is
not hard just not so browser-GUI github style easy. I am not sure what
people will do with your attachments. They can not be reviewed as we
are used to, and also not applied as we are used to.

I will take the one that i approved and will take it further. Keeping
you as author, change it a bit and add my signed-off. You will be on CC
for all that. Speak up in case you at some point feel that you do not
want your name as author.

regards,
Henning

>   Best, R-
> 
> > --  
> Roberto A. Foglietta
> +49.176.274.75.661
> +39.349.33.30.697