Re: [OE-core] [RFC v2 1/2] bitbake-layers: Add ability to update the reference of repositories
On Fri, 5 Jan 2024 at 12:14, Jermain Horsman wrote: > This leave the question of what to do if only the '--reference' option is > used? > Do nothing or update/use a custom revision for all repos? Updating all repos to the same reference is unlikely to work well with real world combinations of layer repositories. I would do nothing and print a message indicating so. Alex -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#193350): https://lists.openembedded.org/g/openembedded-core/message/193350 Mute This Topic: https://lists.openembedded.org/mt/103521079/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] [RFC v2 1/2] bitbake-layers: Add ability to update the reference of repositories
> On further thought, if done this way, then --update could instead > serve to update file that already exists, whereas the default would be I would prefer this, as I might have a config with repo A and B for which I have changed both locally, but I'd only want to apply the changes to B to the layers setup, writing a new file will update A, whereas using the old layers setup will leave A as is. > to overwrite the file with a new one. To clarify as it is not entirely clear this is what you suggested. Use '--update' as a boolean flag which determines whether or not to use the old layers setup or write a new one and use an additional '--use-custom-reference repo' option to determine for with repo to use the custom reference. This leave the question of what to do if only the '--reference' option is used? Do nothing or update/use a custom revision for all repos? An separate option like '--use-custom-reference-all' seems undesirable to me. Sincerely, Jermain Horsman -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#193349): https://lists.openembedded.org/g/openembedded-core/message/193349 Mute This Topic: https://lists.openembedded.org/mt/103521079/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] [RFC v2 1/2] bitbake-layers: Add ability to update the reference of repositories
On Thu, 4 Jan 2024 at 19:07, Alexander Kanavin via lists.openembedded.org wrote: > --update requires that the json file already exists. But why should > it? It's entirely possible to write a new json, but update the > references from command line arguments just before writing the file > out. No? > > I also wonder if we can name these two options in a better way. Maybe > '--use-custom-reference repo', instead of --update? On further thought, if done this way, then --update could instead serve to update file that already exists, whereas the default would be to overwrite the file with a new one. Alex -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#193334): https://lists.openembedded.org/g/openembedded-core/message/193334 Mute This Topic: https://lists.openembedded.org/mt/103521079/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] [RFC v2 1/2] bitbake-layers: Add ability to update the reference of repositories
On Thu, 4 Jan 2024 at 13:35, Jermain Horsman wrote:
> +parser_setup_layers.add_argument('--update', '-u',
> +action='append',
> +metavar='REPOSITORY',
> +help='Repository to update, this requires a reference to be
> specified.\nThis option can be used multiple times.')
> +parser_setup_layers.add_argument('--reference', '-r',
> +help="Reference to use when updating repositories.\nThe value
> can be any git reference, however it is up to the user to provide a valid
> value.\nThis option is only useful when using '--update'.")
> +
> self.plugins = []
These options should move to the plugin. If/when we have another
plugin, we can consider making them global, until then we can't assume
all plugins would want to handle these.
> def do_write(self, parent, args):
> """ Writes out a python script and a json config that replicate the
> directory structure and revisions of the layers in a current build. """
> -if not os.path.exists(args.destdir):
> -os.makedirs(args.destdir)
> -repos = parent.make_repo_config(args.destdir)
> -json = {"version":"1.0","sources":repos}
> -if not repos:
> -raise Exception("Could not determine layer sources")
> output = args.output_prefix or "setup-layers"
> -output = os.path.join(os.path.abspath(args.destdir),output)
> +output = os.path.join(os.path.abspath(args.destdir), output)
> +
> +if args.update is None:
> +if not os.path.exists(args.destdir):
> +os.makedirs(args.destdir)
> +repos = parent.make_repo_config(args.destdir)
> +json = {"version":"1.0","sources":repos}
> +if not repos:
> +raise Exception("Could not determine layer sources")
> +else:
> +json = self._read_repo_config(output + ".json")
> +if not 'sources' in json.keys():
> +raise Exception("File {}.json does not contain valid layer
> sources.".format(output))
> +self._modify_repo_config(json, args)
> +
> self._write_json(json, output + ".json")
> logger.info('Created {}.json'.format(output))
> if not args.json_only:
--update requires that the json file already exists. But why should
it? It's entirely possible to write a new json, but update the
references from command line arguments just before writing the file
out. No?
I also wonder if we can name these two options in a better way. Maybe
'--use-custom-reference repo', instead of --update?
Alex
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#193332):
https://lists.openembedded.org/g/openembedded-core/message/193332
Mute This Topic: https://lists.openembedded.org/mt/103521079/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
[OE-core] [RFC v2 1/2] bitbake-layers: Add ability to update the reference of repositories
From: Jermain Horsman
This uses an existing setup-layers configuration and modifies one
or more repositories using a reference provided by the user.
This is a very minimal implementation, no validation of this reference
is done and it is left to the user to provide a valid value.
Signed-off-by: Jermain Horsman
---
meta/lib/bblayers/makesetup.py| 11 +
.../bblayers/setupwriters/oe-setup-layers.py | 43 ---
2 files changed, 47 insertions(+), 7 deletions(-)
diff --git a/meta/lib/bblayers/makesetup.py b/meta/lib/bblayers/makesetup.py
index 99d5973760..df266c645d 100644
--- a/meta/lib/bblayers/makesetup.py
+++ b/meta/lib/bblayers/makesetup.py
@@ -78,6 +78,10 @@ class MakeSetupPlugin(LayerPlugin):
def do_make_setup(self, args):
""" Writes out a configuration file and/or a script that replicate the
directory structure and revisions of the layers in a current build. """
+if args.update is not None and args.reference is None:
+logger.error("No reference specified. Please provide one using
'--reference REFERENCE'.")
+return
+
for p in self.plugins:
if str(p) == args.writer:
p.do_write(self, args)
@@ -89,6 +93,13 @@ class MakeSetupPlugin(LayerPlugin):
parser_setup_layers.add_argument('--output-prefix', '-o',
help='File name prefix for the output files, if the default
(setup-layers) is undesirable.')
+parser_setup_layers.add_argument('--update', '-u',
+action='append',
+metavar='REPOSITORY',
+help='Repository to update, this requires a reference to be
specified.\nThis option can be used multiple times.')
+parser_setup_layers.add_argument('--reference', '-r',
+help="Reference to use when updating repositories.\nThe value can
be any git reference, however it is up to the user to provide a valid
value.\nThis option is only useful when using '--update'.")
+
self.plugins = []
for path in (self.tinfoil.config_data.getVar('BBPATH').split(':')):
diff --git a/meta/lib/bblayers/setupwriters/oe-setup-layers.py
b/meta/lib/bblayers/setupwriters/oe-setup-layers.py
index bd71ca1f51..fdc33d94d5 100644
--- a/meta/lib/bblayers/setupwriters/oe-setup-layers.py
+++ b/meta/lib/bblayers/setupwriters/oe-setup-layers.py
@@ -31,16 +31,45 @@ class OeSetupLayersWriter():
with open(output, 'w') as f:
json.dump(repos, f, sort_keys=True, indent=4)
+def _read_repo_config(self, json_path):
+with open(json_path) as f:
+json_config = json.load(f)
+
+supported_versions = ["1.0"]
+if json_config["version"] not in supported_versions:
+raise Exception("File {} has version {}, which is not in supported
versions: {}".format(json_path, json_config["version"], supported_versions))
+
+return json_config
+
+def _modify_repo_config(self, json_config, args):
+sources = json_config['sources']
+for repo in args.update:
+if not repo in sources.keys():
+raise Exception("Repository {} does not exist in setup-layers
config".format(repo))
+
+layer_remote = json_config['sources'][repo]['git-remote']
+layer_remote['rev'] = args.reference
+# Clear describe
+layer_remote['describe'] = ''
+
def do_write(self, parent, args):
""" Writes out a python script and a json config that replicate the
directory structure and revisions of the layers in a current build. """
-if not os.path.exists(args.destdir):
-os.makedirs(args.destdir)
-repos = parent.make_repo_config(args.destdir)
-json = {"version":"1.0","sources":repos}
-if not repos:
-raise Exception("Could not determine layer sources")
output = args.output_prefix or "setup-layers"
-output = os.path.join(os.path.abspath(args.destdir),output)
+output = os.path.join(os.path.abspath(args.destdir), output)
+
+if args.update is None:
+if not os.path.exists(args.destdir):
+os.makedirs(args.destdir)
+repos = parent.make_repo_config(args.destdir)
+json = {"version":"1.0","sources":repos}
+if not repos:
+raise Exception("Could not determine layer sources")
+else:
+json = self._read_repo_config(output + ".json")
+if not 'sources' in json.keys():
+raise Exception("File {}.json does not contain valid layer
sources.".format(output))
+self._modify_repo_config(json, args)
+
self._write_json(json, output + ".json")
logger.info('Created {}.json'.format(output))
if not args.json_only:
--
2.42.0.windows.2
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#193317):
