[yocto] [PATCH] bitbake: fetch2/gitsm: avoid live submodule fetching during unpack()
From: Matt Hoosier
Although the submodules' histories have been fetched during the
do_fetch() phase, the mechanics used to clone the workdir copy
of the repo haven't been transferring the actual .git/modules
directory from the repo fetched into downloads/ during the
fetch task.
Fix that, and for good measure also explicitly tell Git to avoid
hitting the network during do_unpack() of the submodules.
Ref:
https://bugzilla.yoctoproject.org/show_bug.cgi?id=12739
---
bitbake/lib/bb/fetch2/gitsm.py | 12 +++-
1 file changed, 11 insertions(+), 1 deletion(-)
diff --git a/bitbake/lib/bb/fetch2/gitsm.py b/bitbake/lib/bb/fetch2/gitsm.py
index 0aff1008e5..1f3fc44352 100644
--- a/bitbake/lib/bb/fetch2/gitsm.py
+++ b/bitbake/lib/bb/fetch2/gitsm.py
@@ -132,4 +132,14 @@ class GitSM(Git):
if self.uses_submodules(ud, d, ud.destdir):
runfetchcmd(ud.basecmd + " checkout " + ud.revisions[ud.names[0]],
d, workdir=ud.destdir)
-runfetchcmd(ud.basecmd + " submodule update --init --recursive",
d, workdir=ud.destdir)
+
+# Copy over the submodules' fetched histories too.
+if ud.bareclone:
+repo_conf = ud.destdir
+else:
+repo_conf = os.path.join(ud.destdir, '.git')
+runfetchcmd("cp -pr %s %s" % (os.path.join(ud.clonedir,
'modules'), repo_conf), d)
+
+# Careful not to hit the network during unpacking; all history
should already
+# be fetched.
+runfetchcmd(ud.basecmd + " submodule update --init --recursive
--no-fetch", d, workdir=ud.destdir)
--
2.13.6
--
___
yocto mailing list
yocto@yoctoproject.org
https://lists.yoctoproject.org/listinfo/yocto
Re: [yocto] [PATCH] bitbake: fetch2/gitsm: avoid live submodule fetching during unpack()
On Wed, May 9, 2018 at 2:48 PM, Matt Hoosier wrote:
> From: Matt Hoosier
>
> Although the submodules' histories have been fetched during the
> do_fetch() phase, the mechanics used to clone the workdir copy
> of the repo haven't been transferring the actual .git/modules
> directory from the repo fetched into downloads/ during the
> fetch task.
>
> Fix that, and for good measure also explicitly tell Git to avoid
> hitting the network during do_unpack() of the submodules.
>
> Ref:
> https://bugzilla.yoctoproject.org/show_bug.cgi?id=12739
Patches for bitbake should be sent to the bitbake-devel mailing list:
http://lists.openembedded.org/mailman/listinfo/bitbake-devel
> ---
> bitbake/lib/bb/fetch2/gitsm.py | 12 +++-
> 1 file changed, 11 insertions(+), 1 deletion(-)
>
> diff --git a/bitbake/lib/bb/fetch2/gitsm.py b/bitbake/lib/bb/fetch2/gitsm.py
> index 0aff1008e5..1f3fc44352 100644
> --- a/bitbake/lib/bb/fetch2/gitsm.py
> +++ b/bitbake/lib/bb/fetch2/gitsm.py
> @@ -132,4 +132,14 @@ class GitSM(Git):
>
> if self.uses_submodules(ud, d, ud.destdir):
> runfetchcmd(ud.basecmd + " checkout " +
> ud.revisions[ud.names[0]], d, workdir=ud.destdir)
> -runfetchcmd(ud.basecmd + " submodule update --init --recursive",
> d, workdir=ud.destdir)
> +
> +# Copy over the submodules' fetched histories too.
> +if ud.bareclone:
> +repo_conf = ud.destdir
> +else:
> +repo_conf = os.path.join(ud.destdir, '.git')
> +runfetchcmd("cp -pr %s %s" % (os.path.join(ud.clonedir,
> 'modules'), repo_conf), d)
> +
> +# Careful not to hit the network during unpacking; all history
> should already
> +# be fetched.
> +runfetchcmd(ud.basecmd + " submodule update --init --recursive
> --no-fetch", d, workdir=ud.destdir)
> --
> 2.13.6
>
> --
> ___
> yocto mailing list
> yocto@yoctoproject.org
> https://lists.yoctoproject.org/listinfo/yocto
--
___
yocto mailing list
yocto@yoctoproject.org
https://lists.yoctoproject.org/listinfo/yocto
Re: [yocto] [PATCH] bitbake: fetch2/gitsm: avoid live submodule fetching during unpack()
On Wed, May 9, 2018 at 6:27 PM, Andre McCurdy wrote:
> On Wed, May 9, 2018 at 2:48 PM, Matt Hoosier
> wrote:
> > From: Matt Hoosier
> >
> > Although the submodules' histories have been fetched during the
> > do_fetch() phase, the mechanics used to clone the workdir copy
> > of the repo haven't been transferring the actual .git/modules
> > directory from the repo fetched into downloads/ during the
> > fetch task.
> >
> > Fix that, and for good measure also explicitly tell Git to avoid
> > hitting the network during do_unpack() of the submodules.
> >
> > Ref:
> > https://bugzilla.yoctoproject.org/show_bug.cgi?id=12739
>
> Patches for bitbake should be sent to the bitbake-devel mailing list:
>
> http://lists.openembedded.org/mailman/listinfo/bitbake-devel
Right, okay. How's that meant to work for stuff developed in-tree with
poky? Do you manually want the 'bitbake/' leading directory stripped off
before emailing the patch?
>
>
> > ---
> > bitbake/lib/bb/fetch2/gitsm.py | 12 +++-
> > 1 file changed, 11 insertions(+), 1 deletion(-)
> >
> > diff --git a/bitbake/lib/bb/fetch2/gitsm.py
> b/bitbake/lib/bb/fetch2/gitsm.py
> > index 0aff1008e5..1f3fc44352 100644
> > --- a/bitbake/lib/bb/fetch2/gitsm.py
> > +++ b/bitbake/lib/bb/fetch2/gitsm.py
> > @@ -132,4 +132,14 @@ class GitSM(Git):
> >
> > if self.uses_submodules(ud, d, ud.destdir):
> > runfetchcmd(ud.basecmd + " checkout " +
> ud.revisions[ud.names[0]], d, workdir=ud.destdir)
> > -runfetchcmd(ud.basecmd + " submodule update --init
> --recursive", d, workdir=ud.destdir)
> > +
> > +# Copy over the submodules' fetched histories too.
> > +if ud.bareclone:
> > +repo_conf = ud.destdir
> > +else:
> > +repo_conf = os.path.join(ud.destdir, '.git')
> > +runfetchcmd("cp -pr %s %s" % (os.path.join(ud.clonedir,
> 'modules'), repo_conf), d)
> > +
> > +# Careful not to hit the network during unpacking; all
> history should already
> > +# be fetched.
> > +runfetchcmd(ud.basecmd + " submodule update --init
> --recursive --no-fetch", d, workdir=ud.destdir)
> > --
> > 2.13.6
> >
> > --
> > ___
> > yocto mailing list
> > yocto@yoctoproject.org
> > https://lists.yoctoproject.org/listinfo/yocto
>
--
___
yocto mailing list
yocto@yoctoproject.org
https://lists.yoctoproject.org/listinfo/yocto
Re: [yocto] [PATCH] bitbake: fetch2/gitsm: avoid live submodule fetching during unpack()
On Wed, May 9, 2018 at 5:40 PM, Matt Hoosier wrote:
> On Wed, May 9, 2018 at 6:27 PM, Andre McCurdy wrote:
>>
>> On Wed, May 9, 2018 at 2:48 PM, Matt Hoosier
>> wrote:
>> > From: Matt Hoosier
>> >
>> > Although the submodules' histories have been fetched during the
>> > do_fetch() phase, the mechanics used to clone the workdir copy
>> > of the repo haven't been transferring the actual .git/modules
>> > directory from the repo fetched into downloads/ during the
>> > fetch task.
>> >
>> > Fix that, and for good measure also explicitly tell Git to avoid
>> > hitting the network during do_unpack() of the submodules.
>> >
>> > Ref:
>> > https://bugzilla.yoctoproject.org/show_bug.cgi?id=12739
>>
>> Patches for bitbake should be sent to the bitbake-devel mailing list:
>>
>> http://lists.openembedded.org/mailman/listinfo/bitbake-devel
>
> Right, okay. How's that meant to work for stuff developed in-tree with poky?
> Do you manually want the 'bitbake/' leading directory stripped off before
> emailing the patch?
Yes, patches for bitbake should apply directly to a clone of the bitbake repo.
Manually editing a patch created from poky is certainly do-able, but
applying the patch to a genuine bikebake repo (with "git am -p2 ...")
and re-generating the patch from there may be safer. Whichever you
prefer.
>> > ---
>> > bitbake/lib/bb/fetch2/gitsm.py | 12 +++-
>> > 1 file changed, 11 insertions(+), 1 deletion(-)
>> >
>> > diff --git a/bitbake/lib/bb/fetch2/gitsm.py
>> > b/bitbake/lib/bb/fetch2/gitsm.py
>> > index 0aff1008e5..1f3fc44352 100644
>> > --- a/bitbake/lib/bb/fetch2/gitsm.py
>> > +++ b/bitbake/lib/bb/fetch2/gitsm.py
>> > @@ -132,4 +132,14 @@ class GitSM(Git):
>> >
>> > if self.uses_submodules(ud, d, ud.destdir):
>> > runfetchcmd(ud.basecmd + " checkout " +
>> > ud.revisions[ud.names[0]], d, workdir=ud.destdir)
>> > -runfetchcmd(ud.basecmd + " submodule update --init
>> > --recursive", d, workdir=ud.destdir)
>> > +
>> > +# Copy over the submodules' fetched histories too.
>> > +if ud.bareclone:
>> > +repo_conf = ud.destdir
>> > +else:
>> > +repo_conf = os.path.join(ud.destdir, '.git')
>> > +runfetchcmd("cp -pr %s %s" % (os.path.join(ud.clonedir,
>> > 'modules'), repo_conf), d)
>> > +
>> > +# Careful not to hit the network during unpacking; all
>> > history should already
>> > +# be fetched.
>> > +runfetchcmd(ud.basecmd + " submodule update --init
>> > --recursive --no-fetch", d, workdir=ud.destdir)
>> > --
>> > 2.13.6
>> >
>> > --
>> > ___
>> > yocto mailing list
>> > yocto@yoctoproject.org
>> > https://lists.yoctoproject.org/listinfo/yocto
>
>
--
___
yocto mailing list
yocto@yoctoproject.org
https://lists.yoctoproject.org/listinfo/yocto
Re: [yocto] [PATCH] bitbake: fetch2/gitsm: avoid live submodule fetching during unpack()
On 10 May 2018 at 01:40, Matt Hoosier wrote: > On Wed, May 9, 2018 at 6:27 PM, Andre McCurdy wrote: >> >> On Wed, May 9, 2018 at 2:48 PM, Matt Hoosier >> wrote: >> > From: Matt Hoosier >> > >> > Although the submodules' histories have been fetched during the >> > do_fetch() phase, the mechanics used to clone the workdir copy >> > of the repo haven't been transferring the actual .git/modules >> > directory from the repo fetched into downloads/ during the >> > fetch task. >> > >> > Fix that, and for good measure also explicitly tell Git to avoid >> > hitting the network during do_unpack() of the submodules. >> > >> > Ref: >> > https://bugzilla.yoctoproject.org/show_bug.cgi?id=12739 >> >> Patches for bitbake should be sent to the bitbake-devel mailing list: >> >> http://lists.openembedded.org/mailman/listinfo/bitbake-devel > > > > Right, okay. How's that meant to work for stuff developed in-tree with poky? > Do you manually want the 'bitbake/' leading directory stripped off before > emailing the patch? If you're feeling lazy you can just send the patch to bitbake@ from a poky repo and git will figure it out. The same works the other way: a bitbake patch applies fine to a poky repository. Ross -- ___ yocto mailing list yocto@yoctoproject.org https://lists.yoctoproject.org/listinfo/yocto
