Re: [OE-core] [PATCH] devtool: remove duplicate overrides
As Peter mentions the underlying issue is already mitigated when we read the
value of
DEVTOOL_EXTRA_OVERRIDES; for clarity I'd say let's revert this
(90f667db2219f04e6d61588cd61056d3d8da6d7d). My apologies, I wasn't
paying attention earlier.
Cheers,
Paul
On Wednesday, 23 January 2019 12:32:35 PM NZDT Peter Kjellerstedt wrote:
> This has already been merged, but see below.
>
> > -Original Message-
> > From: openembedded-core-boun...@lists.openembedded.org > core-boun...@lists.openembedded.org> On Behalf Of Dan Dedrick
> > Sent: den 21 januari 2019 17:05
> > To: openembedded-core@lists.openembedded.org
> > Subject: [OE-core] [PATCH] devtool: remove duplicate overrides
> >
> > DEVTOOL_EXTRA_OVERRIDES only needs one entry for each instance of
> > overrides. Previous to these changes it would find every override to
> > SRC_URI and add it to the list. This would duplicate instances where
> > SRC_URI is modified multiple times with the same override like:
> > SRC_URI_append_foo += "file://0001-foo.patch"
> > SRC_URI_append_foo += "file://0002-bar.patch"
> >
> > A bbappend might also overwrite a SRC_URI override, which would also
> > cause multiple instances to occur.
> >
> > When there are multiple instances of the same override in
> > DEVTOOL_EXTRA_OVERRIDES it causes devtool modify to fail when creating
> > override branches. The failure occurs when attempting to create the same
> > override branch a second time and looks like this:
> >
> > The stack trace of python calls that resulted in this exception/failure was:
> > File: 'exec_python_func() autogenerated', lineno: 2, function:
> > 0001:
> > *** 0002:devtool_post_patch(d)
> > 0003:
> > File: '/build/poky/meta/classes/devtool-source.bbclass', lineno: 202,
> > function: devtool_post_patch
> > 0198:
> > 0199:for override in extra_override_list:
>
> The reference to extra_override_list above indicates that you developed
> this patch for an older version of OE-Core, since that variable was
> renamed on master of OE-Core in October when this problem was fixed (see
> commit 95a92195). However, that fix was made to devtool-source.bbclass,
> where it uses the value of DEVTOOL_EXTRA_OVERRIDES, instead of to
> scripts/lib/devtool/standard.py where you made the change. This is
> probably why you did not notice that the problem had already been fixed.
>
> > 0200:localdata = bb.data.createCopy(d)
> > 0201:if override in default_overrides:
> > *** 0202:bb.process.run('git branch devtool-override-%s
> > %s' % (override, devbranch), cwd=srcsubdir)
> > 0203:else:
> > 0204:# Reset back to the initial commit on a new branch
> > 0205:bb.process.run('git checkout %s -b
> > devtool-override-%s' % (initial_rev, override), cwd=srcsubdir)
> > 0206:# Run do_patch function with the override applied
> > File: '/build/poky/bitbake/lib/bb/process.py', lineno: 178, function: run
> > 0174:if not stderr is None:
> > 0175:stderr = stderr.decode("utf-8")
> > 0176:
> > 0177:if pipe.returncode != 0:
> > *** 0178:raise ExecutionError(cmd, pipe.returncode, stdout, stderr)
> > 0179:return stdout, stderr
> > Exception: bb.process.ExecutionError: Execution of 'git branch
> > devtool-override-foo devtool' failed with exit code 128:
> > fatal: A branch named 'devtool-override-foo' already exists.
> >
> > Signed-off-by: Dan Dedrick
> > ---
> > scripts/lib/devtool/standard.py | 5 +
> > 1 file changed, 5 insertions(+)
> >
> > diff --git a/scripts/lib/devtool/standard.py
> > b/scripts/lib/devtool/standard.py
> > index d14b7a6543..a45ad36812 100644
> > --- a/scripts/lib/devtool/standard.py
> > +++ b/scripts/lib/devtool/standard.py
> > @@ -509,6 +509,11 @@ def _extract_source(srctree, keep_temp, devbranch,
> > sync, config, basepath, works
> > if not 'flag' in event:
> > if event['op'].startswith(('_append[', '_prepend[')):
> >
> > extra_overrides.append(event['op'].split('[')[1].split(']')[0])
> > +# We want to remove duplicate overrides. If a recipe had multiple
> > +# SRC_URI_override += values it would cause mulitple instances of
> > +# overrides. This doesn't play nicely with things like creating a
> > +# branch for every instance of DEVTOOL_EXTRA_OVERRIDES.
> > +extra_overrides = list(set(extra_overrides))
> > if extra_overrides:
> > logger.info('SRC_URI contains some conditional
> > appends/prepends - will create branches to represent these')
> >
> > --
> > 2.20.1
>
> //Peter
>
>
--
Paul Eggleton
Intel Open Source Technology Centre
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH] devtool: remove duplicate overrides
This has already been merged, but see below.
> -Original Message-
> From: openembedded-core-boun...@lists.openembedded.org core-boun...@lists.openembedded.org> On Behalf Of Dan Dedrick
> Sent: den 21 januari 2019 17:05
> To: openembedded-core@lists.openembedded.org
> Subject: [OE-core] [PATCH] devtool: remove duplicate overrides
>
> DEVTOOL_EXTRA_OVERRIDES only needs one entry for each instance of
> overrides. Previous to these changes it would find every override to
> SRC_URI and add it to the list. This would duplicate instances where
> SRC_URI is modified multiple times with the same override like:
> SRC_URI_append_foo += "file://0001-foo.patch"
> SRC_URI_append_foo += "file://0002-bar.patch"
>
> A bbappend might also overwrite a SRC_URI override, which would also
> cause multiple instances to occur.
>
> When there are multiple instances of the same override in
> DEVTOOL_EXTRA_OVERRIDES it causes devtool modify to fail when creating
> override branches. The failure occurs when attempting to create the same
> override branch a second time and looks like this:
>
> The stack trace of python calls that resulted in this exception/failure was:
> File: 'exec_python_func() autogenerated', lineno: 2, function:
> 0001:
> *** 0002:devtool_post_patch(d)
> 0003:
> File: '/build/poky/meta/classes/devtool-source.bbclass', lineno: 202,
> function: devtool_post_patch
> 0198:
> 0199:for override in extra_override_list:
The reference to extra_override_list above indicates that you developed
this patch for an older version of OE-Core, since that variable was
renamed on master of OE-Core in October when this problem was fixed (see
commit 95a92195). However, that fix was made to devtool-source.bbclass,
where it uses the value of DEVTOOL_EXTRA_OVERRIDES, instead of to
scripts/lib/devtool/standard.py where you made the change. This is
probably why you did not notice that the problem had already been fixed.
> 0200:localdata = bb.data.createCopy(d)
> 0201:if override in default_overrides:
> *** 0202:bb.process.run('git branch devtool-override-%s %s'
> % (override, devbranch), cwd=srcsubdir)
> 0203:else:
> 0204:# Reset back to the initial commit on a new branch
> 0205:bb.process.run('git checkout %s -b
> devtool-override-%s' % (initial_rev, override), cwd=srcsubdir)
> 0206:# Run do_patch function with the override applied
> File: '/build/poky/bitbake/lib/bb/process.py', lineno: 178, function: run
> 0174:if not stderr is None:
> 0175:stderr = stderr.decode("utf-8")
> 0176:
> 0177:if pipe.returncode != 0:
> *** 0178:raise ExecutionError(cmd, pipe.returncode, stdout, stderr)
> 0179:return stdout, stderr
> Exception: bb.process.ExecutionError: Execution of 'git branch
> devtool-override-foo devtool' failed with exit code 128:
> fatal: A branch named 'devtool-override-foo' already exists.
>
> Signed-off-by: Dan Dedrick
> ---
> scripts/lib/devtool/standard.py | 5 +
> 1 file changed, 5 insertions(+)
>
> diff --git a/scripts/lib/devtool/standard.py b/scripts/lib/devtool/standard.py
> index d14b7a6543..a45ad36812 100644
> --- a/scripts/lib/devtool/standard.py
> +++ b/scripts/lib/devtool/standard.py
> @@ -509,6 +509,11 @@ def _extract_source(srctree, keep_temp, devbranch, sync,
> config, basepath, works
> if not 'flag' in event:
> if event['op'].startswith(('_append[', '_prepend[')):
>
> extra_overrides.append(event['op'].split('[')[1].split(']')[0])
> +# We want to remove duplicate overrides. If a recipe had multiple
> +# SRC_URI_override += values it would cause mulitple instances of
> +# overrides. This doesn't play nicely with things like creating a
> +# branch for every instance of DEVTOOL_EXTRA_OVERRIDES.
> +extra_overrides = list(set(extra_overrides))
> if extra_overrides:
> logger.info('SRC_URI contains some conditional appends/prepends
> - will create branches to represent these')
>
> --
> 2.20.1
//Peter
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH] devtool: remove duplicate overrides
DEVTOOL_EXTRA_OVERRIDES only needs one entry for each instance of
overrides. Previous to these changes it would find every override to
SRC_URI and add it to the list. This would duplicate instances where
SRC_URI is modified multiple times with the same override like:
SRC_URI_append_foo += "file://0001-foo.patch"
SRC_URI_append_foo += "file://0002-bar.patch"
A bbappend might also overwrite a SRC_URI override, which would also
cause multiple instances to occur.
When there are multiple instances of the same override in
DEVTOOL_EXTRA_OVERRIDES it causes devtool modify to fail when creating
override branches. The failure occurs when attempting to create the same
override branch a second time and looks like this:
The stack trace of python calls that resulted in this exception/failure was:
File: 'exec_python_func() autogenerated', lineno: 2, function:
0001:
*** 0002:devtool_post_patch(d)
0003:
File: '/build/poky/meta/classes/devtool-source.bbclass', lineno: 202, function:
devtool_post_patch
0198:
0199:for override in extra_override_list:
0200:localdata = bb.data.createCopy(d)
0201:if override in default_overrides:
*** 0202:bb.process.run('git branch devtool-override-%s %s' %
(override, devbranch), cwd=srcsubdir)
0203:else:
0204:# Reset back to the initial commit on a new branch
0205:bb.process.run('git checkout %s -b
devtool-override-%s' % (initial_rev, override), cwd=srcsubdir)
0206:# Run do_patch function with the override applied
File: '/build/poky/bitbake/lib/bb/process.py', lineno: 178, function: run
0174:if not stderr is None:
0175:stderr = stderr.decode("utf-8")
0176:
0177:if pipe.returncode != 0:
*** 0178:raise ExecutionError(cmd, pipe.returncode, stdout, stderr)
0179:return stdout, stderr
Exception: bb.process.ExecutionError: Execution of 'git branch
devtool-override-foo devtool' failed with exit code 128:
fatal: A branch named 'devtool-override-foo' already exists.
Signed-off-by: Dan Dedrick
---
scripts/lib/devtool/standard.py | 5 +
1 file changed, 5 insertions(+)
diff --git a/scripts/lib/devtool/standard.py b/scripts/lib/devtool/standard.py
index d14b7a6543..a45ad36812 100644
--- a/scripts/lib/devtool/standard.py
+++ b/scripts/lib/devtool/standard.py
@@ -509,6 +509,11 @@ def _extract_source(srctree, keep_temp, devbranch, sync,
config, basepath, works
if not 'flag' in event:
if event['op'].startswith(('_append[', '_prepend[')):
extra_overrides.append(event['op'].split('[')[1].split(']')[0])
+# We want to remove duplicate overrides. If a recipe had multiple
+# SRC_URI_override += values it would cause mulitple instances of
+# overrides. This doesn't play nicely with things like creating a
+# branch for every instance of DEVTOOL_EXTRA_OVERRIDES.
+extra_overrides = list(set(extra_overrides))
if extra_overrides:
logger.info('SRC_URI contains some conditional appends/prepends -
will create branches to represent these')
--
2.20.1
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
