Message ID | 20210219195719.29037-2-Vijaikumar_Kanagarajan@mentor.com |
---|---|
State | Superseded, archived |
Headers | show |
Series | Custom base-files | expand |
Am Sat, 20 Feb 2021 01:27:18 +0530 schrieb Vijai Kumar K <Vijaikumar_Kanagarajan@mentor.com>: > With the current do_apt_fetch implementation, it is not possible to > use a custom source directory(${S}). > > apt-get source by default extracts the contents of the debian source > into folder with name <pkg>_<version>. > > Add provision for specifying a custom source directory. I think this one is indeed worth being discussed seperately. Could you go into detail why a custom S is required in the first place? My guess is that we might not know PV and need to wait for that apt-unpack to finish so we can find it in the changelog. > Signed-off-by: Vijai Kumar K <Vijaikumar_Kanagarajan@mentor.com> > --- > meta/classes/dpkg-base.bbclass | 7 ++++++- > 1 file changed, 6 insertions(+), 1 deletion(-) > > diff --git a/meta/classes/dpkg-base.bbclass > b/meta/classes/dpkg-base.bbclass index 5c7bddc..1b94b76 100644 > --- a/meta/classes/dpkg-base.bbclass > +++ b/meta/classes/dpkg-base.bbclass > @@ -70,7 +70,12 @@ do_apt_fetch() { > sudo -E chroot --userspec=$( id -u ):$( id -g ) > ${BUILDCHROOT_DIR} \ sh -c 'mkdir -p /downloads/deb-src/"$1"/"$2" && > cd /downloads/deb-src/"$1"/"$2" && apt-get -y --download-only > --only-source source "$2"' my_script "${DISTRO}" "${uri}" sudo -E > chroot --userspec=$( id -u ):$( id -g ) ${BUILDCHROOT_DIR} \ > - sh -c 'cp /downloads/deb-src/"$1"/"$2"/* ${PP} && cd > ${PP} && apt-get -y --only-source source "$2"' my_script "${DISTRO}" > "${uri}" > + sh -c ' \ > + dscfile="$(apt-get -y -qq --print-uris source "${2}" > | cut -d " " -f2 | grep -E "*.dsc")" > + cd ${PP} > + cp /downloads/deb-src/"${1}"/"${2}"/* ${PP} > + dpkg-source -x "${dscfile}" "${PPS}"' \ > + my_script "${DISTRO}" "${uri}" You removed the && chaining, please try a "false" in there, i guess that sh might need a -e now I wonder if we can just symlink debians choice to PPS, or mv, or even "cp -a" And make that a new task we call apt_unpack Henning > done > > dpkg_undo_mounts
On Sat, Feb 20, 2021 at 1:47 PM Henning Schild <henning.schild@siemens.com> wrote: > > Am Sat, 20 Feb 2021 01:27:18 +0530 > schrieb Vijai Kumar K <Vijaikumar_Kanagarajan@mentor.com>: > > > With the current do_apt_fetch implementation, it is not possible to > > use a custom source directory(${S}). > > > > apt-get source by default extracts the contents of the debian source > > into folder with name <pkg>_<version>. > > > > Add provision for specifying a custom source directory. > > I think this one is indeed worth being discussed seperately. > > Could you go into detail why a custom S is required in the first place? > My guess is that we might not know PV and need to wait for that > apt-unpack to finish so we can find it in the changelog. Exactly. In some cases we might not know PV, we might need to wait for apt to see what version it is fetching. Change the APT sources and or preferences and you might get a completely new version of the package. We cannot effectively know ${S} for manipulating the source code from the recipe. > > > Signed-off-by: Vijai Kumar K <Vijaikumar_Kanagarajan@mentor.com> > > --- > > meta/classes/dpkg-base.bbclass | 7 ++++++- > > 1 file changed, 6 insertions(+), 1 deletion(-) > > > > diff --git a/meta/classes/dpkg-base.bbclass > > b/meta/classes/dpkg-base.bbclass index 5c7bddc..1b94b76 100644 > > --- a/meta/classes/dpkg-base.bbclass > > +++ b/meta/classes/dpkg-base.bbclass > > @@ -70,7 +70,12 @@ do_apt_fetch() { > > sudo -E chroot --userspec=$( id -u ):$( id -g ) > > ${BUILDCHROOT_DIR} \ sh -c 'mkdir -p /downloads/deb-src/"$1"/"$2" && > > cd /downloads/deb-src/"$1"/"$2" && apt-get -y --download-only > > --only-source source "$2"' my_script "${DISTRO}" "${uri}" sudo -E > > chroot --userspec=$( id -u ):$( id -g ) ${BUILDCHROOT_DIR} \ > > - sh -c 'cp /downloads/deb-src/"$1"/"$2"/* ${PP} && cd > > ${PP} && apt-get -y --only-source source "$2"' my_script "${DISTRO}" > > "${uri}" > > + sh -c ' \ > > + dscfile="$(apt-get -y -qq --print-uris source "${2}" > > | cut -d " " -f2 | grep -E "*.dsc")" > > + cd ${PP} > > + cp /downloads/deb-src/"${1}"/"${2}"/* ${PP} > > + dpkg-source -x "${dscfile}" "${PPS}"' \ > > + my_script "${DISTRO}" "${uri}" > > You removed the && chaining, please try a "false" in there, i guess > that sh might need a -e now Yes. Will add that in v2. > > I wonder if we can just symlink debians choice to PPS, or mv, or even > "cp -a" Wondering whether it brings in any added advantage? > And make that a new task we call apt_unpack I thought about this too. We could definitely have something like apt_unpack that makes things clear. When I first started looking around, I didn't expect the unpacking to happen in fetch. Thanks, Vijai Kumar K > > Henning > > > done > > > > dpkg_undo_mounts > > -- > You received this message because you are subscribed to the Google Groups "isar-users" group. > To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com. > To view this discussion on the web visit https://groups.google.com/d/msgid/isar-users/20210220090734.60a591c2%40md1za8fc.ad001.siemens.net.
Can we please get this one stand-alone on the fast-path? Just ran into exactly the problem and now have to work around. regards, Henning Am Mon, 22 Feb 2021 12:41:40 +0530 schrieb vijai kumar <vijaikumar.kanagarajan@gmail.com>: > On Sat, Feb 20, 2021 at 1:47 PM Henning Schild > <henning.schild@siemens.com> wrote: > > > > Am Sat, 20 Feb 2021 01:27:18 +0530 > > schrieb Vijai Kumar K <Vijaikumar_Kanagarajan@mentor.com>: > > > > > With the current do_apt_fetch implementation, it is not possible > > > to use a custom source directory(${S}). > > > > > > apt-get source by default extracts the contents of the debian > > > source into folder with name <pkg>_<version>. > > > > > > Add provision for specifying a custom source directory. > > > > I think this one is indeed worth being discussed seperately. > > > > Could you go into detail why a custom S is required in the first > > place? My guess is that we might not know PV and need to wait for > > that apt-unpack to finish so we can find it in the changelog. > > Exactly. In some cases we might not know PV, we might need to wait > for apt to see what version it is fetching. Change the APT sources > and or preferences and you might get a completely new version of the > package. We cannot effectively know ${S} for manipulating the source > code from the recipe. > > > > > > Signed-off-by: Vijai Kumar K <Vijaikumar_Kanagarajan@mentor.com> > > > --- > > > meta/classes/dpkg-base.bbclass | 7 ++++++- > > > 1 file changed, 6 insertions(+), 1 deletion(-) > > > > > > diff --git a/meta/classes/dpkg-base.bbclass > > > b/meta/classes/dpkg-base.bbclass index 5c7bddc..1b94b76 100644 > > > --- a/meta/classes/dpkg-base.bbclass > > > +++ b/meta/classes/dpkg-base.bbclass > > > @@ -70,7 +70,12 @@ do_apt_fetch() { > > > sudo -E chroot --userspec=$( id -u ):$( id -g ) > > > ${BUILDCHROOT_DIR} \ sh -c 'mkdir -p /downloads/deb-src/"$1"/"$2" > > > && cd /downloads/deb-src/"$1"/"$2" && apt-get -y --download-only > > > --only-source source "$2"' my_script "${DISTRO}" "${uri}" sudo -E > > > chroot --userspec=$( id -u ):$( id -g ) ${BUILDCHROOT_DIR} \ > > > - sh -c 'cp /downloads/deb-src/"$1"/"$2"/* ${PP} && cd > > > ${PP} && apt-get -y --only-source source "$2"' my_script > > > "${DISTRO}" "${uri}" > > > + sh -c ' \ > > > + dscfile="$(apt-get -y -qq --print-uris source > > > "${2}" | cut -d " " -f2 | grep -E "*.dsc")" > > > + cd ${PP} > > > + cp /downloads/deb-src/"${1}"/"${2}"/* ${PP} > > > + dpkg-source -x "${dscfile}" "${PPS}"' \ > > > + my_script "${DISTRO}" "${uri}" > > > > You removed the && chaining, please try a "false" in there, i guess > > that sh might need a -e now > > Yes. Will add that in v2. > > > > > I wonder if we can just symlink debians choice to PPS, or mv, or > > even "cp -a" > > Wondering whether it brings in any added advantage? > > > And make that a new task we call apt_unpack > > I thought about this too. We could definitely have something like > apt_unpack that makes things clear. When I first started looking > around, I didn't expect the > unpacking to happen in fetch. > > Thanks, > Vijai Kumar K > > > > > Henning > > > > > done > > > > > > dpkg_undo_mounts > > > > -- > > You received this message because you are subscribed to the Google > > Groups "isar-users" group. To unsubscribe from this group and stop > > receiving emails from it, send an email to > > isar-users+unsubscribe@googlegroups.com. To view this discussion on > > the web visit > > https://groups.google.com/d/msgid/isar-users/20210220090734.60a591c2%40md1za8fc.ad001.siemens.net. > > >
-----Original Message----- From: Henning Schild [mailto:henning.schild@siemens.com] Sent: 03 March 2021 18:19 To: vijai kumar <vijaikumar.kanagarajan@gmail.com> Cc: Kanagarajan, Vijaikumar <Vijaikumar_Kanagarajan@mentor.com>; isar-users <isar-users@googlegroups.com>; ibr@radix10.net; Silvano Cirujano Cuesta <silvano.cirujano-cuesta@siemens.com> Subject: Re: [RFC PATCH 1/2] dpkg-base: Handle custom source directory in do_apt_fetch Can we please get this one stand-alone on the fast-path? Just ran into exactly the problem and now have to work around. Sure Henning. Will take this out and send a v2 tonight. regards, Henning Am Mon, 22 Feb 2021 12:41:40 +0530 schrieb vijai kumar <vijaikumar.kanagarajan@gmail.com>: > On Sat, Feb 20, 2021 at 1:47 PM Henning Schild > <henning.schild@siemens.com> wrote: > > > > Am Sat, 20 Feb 2021 01:27:18 +0530 > > schrieb Vijai Kumar K <Vijaikumar_Kanagarajan@mentor.com>: > > > > > With the current do_apt_fetch implementation, it is not possible > > > to use a custom source directory(${S}). > > > > > > apt-get source by default extracts the contents of the debian > > > source into folder with name <pkg>_<version>. > > > > > > Add provision for specifying a custom source directory. > > > > I think this one is indeed worth being discussed seperately. > > > > Could you go into detail why a custom S is required in the first > > place? My guess is that we might not know PV and need to wait for > > that apt-unpack to finish so we can find it in the changelog. > > Exactly. In some cases we might not know PV, we might need to wait for > apt to see what version it is fetching. Change the APT sources and or > preferences and you might get a completely new version of the package. > We cannot effectively know ${S} for manipulating the source code from > the recipe. > > > > > > Signed-off-by: Vijai Kumar K <Vijaikumar_Kanagarajan@mentor.com> > > > --- > > > meta/classes/dpkg-base.bbclass | 7 ++++++- > > > 1 file changed, 6 insertions(+), 1 deletion(-) > > > > > > diff --git a/meta/classes/dpkg-base.bbclass > > > b/meta/classes/dpkg-base.bbclass index 5c7bddc..1b94b76 100644 > > > --- a/meta/classes/dpkg-base.bbclass > > > +++ b/meta/classes/dpkg-base.bbclass > > > @@ -70,7 +70,12 @@ do_apt_fetch() { > > > sudo -E chroot --userspec=$( id -u ):$( id -g ) > > > ${BUILDCHROOT_DIR} \ sh -c 'mkdir -p /downloads/deb-src/"$1"/"$2" > > > && cd /downloads/deb-src/"$1"/"$2" && apt-get -y --download-only > > > --only-source source "$2"' my_script "${DISTRO}" "${uri}" sudo -E > > > chroot --userspec=$( id -u ):$( id -g ) ${BUILDCHROOT_DIR} \ > > > - sh -c 'cp /downloads/deb-src/"$1"/"$2"/* ${PP} && cd > > > ${PP} && apt-get -y --only-source source "$2"' my_script > > > "${DISTRO}" "${uri}" > > > + sh -c ' \ > > > + dscfile="$(apt-get -y -qq --print-uris source > > > "${2}" | cut -d " " -f2 | grep -E "*.dsc")" > > > + cd ${PP} > > > + cp /downloads/deb-src/"${1}"/"${2}"/* ${PP} > > > + dpkg-source -x "${dscfile}" "${PPS}"' \ > > > + my_script "${DISTRO}" "${uri}" > > > > You removed the && chaining, please try a "false" in there, i guess > > that sh might need a -e now > > Yes. Will add that in v2. > > > > > I wonder if we can just symlink debians choice to PPS, or mv, or > > even "cp -a" > > Wondering whether it brings in any added advantage? > > > And make that a new task we call apt_unpack > > I thought about this too. We could definitely have something like > apt_unpack that makes things clear. When I first started looking > around, I didn't expect the unpacking to happen in fetch. > > Thanks, > Vijai Kumar K > > > > > Henning > > > > > done > > > > > > dpkg_undo_mounts > > > > -- > > You received this message because you are subscribed to the Google > > Groups "isar-users" group. To unsubscribe from this group and stop > > receiving emails from it, send an email to > > isar-users+unsubscribe@googlegroups.com. To view this discussion on > > the web visit > > https://groups.google.com/d/msgid/isar-users/20210220090734.60a591c2%40md1za8fc.ad001.siemens.net. > > >
Sweet, this is what i just placed in a recipe +## TODO hack, upstream will soon have a patch +do_rename_to_s() { + debs=$( find ${WORKDIR} -type d -iname "${P}*" | head -1 ) + rm -rf "${S}" + cp -a "$debs" "${S}" +} +addtask rename_to_s after do_apt_fetch symlink will not work, since that is mounted into buildchroot Henning Am Wed, 3 Mar 2021 13:49:26 +0000 schrieb "Kanagarajan, Vijaikumar" <Vijaikumar_Kanagarajan@mentor.com>: > -----Original Message----- > From: Henning Schild [mailto:henning.schild@siemens.com] > Sent: 03 March 2021 18:19 > To: vijai kumar <vijaikumar.kanagarajan@gmail.com> > Cc: Kanagarajan, Vijaikumar <Vijaikumar_Kanagarajan@mentor.com>; > isar-users <isar-users@googlegroups.com>; ibr@radix10.net; Silvano > Cirujano Cuesta <silvano.cirujano-cuesta@siemens.com> Subject: Re: > [RFC PATCH 1/2] dpkg-base: Handle custom source directory in > do_apt_fetch > > Can we please get this one stand-alone on the fast-path? Just ran > into exactly the problem and now have to work around. > > > Sure Henning. Will take this out and send a v2 tonight. > > regards, > Henning > > Am Mon, 22 Feb 2021 12:41:40 +0530 > schrieb vijai kumar <vijaikumar.kanagarajan@gmail.com>: > > > On Sat, Feb 20, 2021 at 1:47 PM Henning Schild > > <henning.schild@siemens.com> wrote: > > > > > > Am Sat, 20 Feb 2021 01:27:18 +0530 > > > schrieb Vijai Kumar K <Vijaikumar_Kanagarajan@mentor.com>: > > > > > > > With the current do_apt_fetch implementation, it is not > > > > possible to use a custom source directory(${S}). > > > > > > > > apt-get source by default extracts the contents of the debian > > > > source into folder with name <pkg>_<version>. > > > > > > > > Add provision for specifying a custom source directory. > > > > > > I think this one is indeed worth being discussed seperately. > > > > > > Could you go into detail why a custom S is required in the first > > > place? My guess is that we might not know PV and need to wait for > > > that apt-unpack to finish so we can find it in the changelog. > > > > Exactly. In some cases we might not know PV, we might need to wait > > for apt to see what version it is fetching. Change the APT sources > > and or preferences and you might get a completely new version of > > the package. We cannot effectively know ${S} for manipulating the > > source code from the recipe. > > > > > > > > > Signed-off-by: Vijai Kumar K <Vijaikumar_Kanagarajan@mentor.com> > > > > --- > > > > meta/classes/dpkg-base.bbclass | 7 ++++++- > > > > 1 file changed, 6 insertions(+), 1 deletion(-) > > > > > > > > diff --git a/meta/classes/dpkg-base.bbclass > > > > b/meta/classes/dpkg-base.bbclass index 5c7bddc..1b94b76 100644 > > > > --- a/meta/classes/dpkg-base.bbclass > > > > +++ b/meta/classes/dpkg-base.bbclass > > > > @@ -70,7 +70,12 @@ do_apt_fetch() { > > > > sudo -E chroot --userspec=$( id -u ):$( id -g ) > > > > ${BUILDCHROOT_DIR} \ sh -c 'mkdir -p > > > > /downloads/deb-src/"$1"/"$2" && cd /downloads/deb-src/"$1"/"$2" > > > > && apt-get -y --download-only --only-source source "$2"' > > > > my_script "${DISTRO}" "${uri}" sudo -E chroot --userspec=$( id > > > > -u ):$( id -g ) ${BUILDCHROOT_DIR} \ > > > > - sh -c 'cp /downloads/deb-src/"$1"/"$2"/* ${PP} && > > > > cd ${PP} && apt-get -y --only-source source "$2"' my_script > > > > "${DISTRO}" "${uri}" > > > > + sh -c ' \ > > > > + dscfile="$(apt-get -y -qq --print-uris source > > > > "${2}" | cut -d " " -f2 | grep -E "*.dsc")" > > > > + cd ${PP} > > > > + cp /downloads/deb-src/"${1}"/"${2}"/* ${PP} > > > > + dpkg-source -x "${dscfile}" "${PPS}"' \ > > > > + my_script "${DISTRO}" "${uri}" > > > > > > You removed the && chaining, please try a "false" in there, i > > > guess that sh might need a -e now > > > > Yes. Will add that in v2. > > > > > > > > I wonder if we can just symlink debians choice to PPS, or mv, or > > > even "cp -a" > > > > Wondering whether it brings in any added advantage? > > > > > And make that a new task we call apt_unpack > > > > I thought about this too. We could definitely have something like > > apt_unpack that makes things clear. When I first started looking > > around, I didn't expect the unpacking to happen in fetch. > > > > Thanks, > > Vijai Kumar K > > > > > > > > Henning > > > > > > > done > > > > > > > > dpkg_undo_mounts > > > > > > -- > > > You received this message because you are subscribed to the > > > Google Groups "isar-users" group. To unsubscribe from this group > > > and stop receiving emails from it, send an email to > > > isar-users+unsubscribe@googlegroups.com. To view this discussion > > > on the web visit > > > https://groups.google.com/d/msgid/isar-users/20210220090734.60a591c2%40md1za8fc.ad001.siemens.net. > > > > > >
diff --git a/meta/classes/dpkg-base.bbclass b/meta/classes/dpkg-base.bbclass index 5c7bddc..1b94b76 100644 --- a/meta/classes/dpkg-base.bbclass +++ b/meta/classes/dpkg-base.bbclass @@ -70,7 +70,12 @@ do_apt_fetch() { sudo -E chroot --userspec=$( id -u ):$( id -g ) ${BUILDCHROOT_DIR} \ sh -c 'mkdir -p /downloads/deb-src/"$1"/"$2" && cd /downloads/deb-src/"$1"/"$2" && apt-get -y --download-only --only-source source "$2"' my_script "${DISTRO}" "${uri}" sudo -E chroot --userspec=$( id -u ):$( id -g ) ${BUILDCHROOT_DIR} \ - sh -c 'cp /downloads/deb-src/"$1"/"$2"/* ${PP} && cd ${PP} && apt-get -y --only-source source "$2"' my_script "${DISTRO}" "${uri}" + sh -c ' \ + dscfile="$(apt-get -y -qq --print-uris source "${2}" | cut -d " " -f2 | grep -E "*.dsc")" + cd ${PP} + cp /downloads/deb-src/"${1}"/"${2}"/* ${PP} + dpkg-source -x "${dscfile}" "${PPS}"' \ + my_script "${DISTRO}" "${uri}" done dpkg_undo_mounts
With the current do_apt_fetch implementation, it is not possible to use a custom source directory(${S}). apt-get source by default extracts the contents of the debian source into folder with name <pkg>_<version>. Add provision for specifying a custom source directory. Signed-off-by: Vijai Kumar K <Vijaikumar_Kanagarajan@mentor.com> --- meta/classes/dpkg-base.bbclass | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-)