Re: [OE-Core][PATCH v3 0/4] recipetool: Add handler to create go recipes
Hi Lukas,
Sorry for the late reply.
I think I see what's going on.
The module I'm working on is already "vendored", which means there's no
modules.txt created for my project by recipetool, since it's already
provided in the repository itself.
I guess this has pros and cons, and I'm currently not sure which
approach is better. On one hand, offline vendoring (no by the
recipetool) has an advantage for SDK builds, when people don't use the
BSP directly. On the other hand, Yocto style vendoring (when modules.txt
is stored in the BSP) is more transparent for dependency management.
I.e. I would need to update the recipe when my dependencies change.
Slava
On 15.11.2023 08:34, Vyacheslav Yurkov via lists.openembedded.org wrote:
On 15.11.2023 08:30, Lukas Funke wrote:
Hi Slava,
No problem. Party is still going on. The 'modules.txt' is intendet to
be in your ${WORKDIR} as it is part of the SRC_URI, generated by
the recipetool. My guess would be, that there is some
missconfiguration in your SRC_URI?
Yes, I noticed that it should be there in recipetool tests. But
generated recipe didn't have It in SRC_URI. I tried to use generated
recipe as is. I'll try to double check later.
Slava
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#193015):
https://lists.openembedded.org/g/openembedded-core/message/193015
Mute This Topic: https://lists.openembedded.org/mt/102345308/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-Core][PATCH v3 0/4] recipetool: Add handler to create go recipes
On 15.11.2023 08:30, Lukas Funke wrote:
Hi Slava,
No problem. Party is still going on. The 'modules.txt' is intendet to
be in your ${WORKDIR} as it is part of the SRC_URI, generated by
the recipetool. My guess would be, that there is some
missconfiguration in your SRC_URI?
Yes, I noticed that it should be there in recipetool tests. But
generated recipe didn't have It in SRC_URI. I tried to use generated
recipe as is. I'll try to double check later.
Slava
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#190558):
https://lists.openembedded.org/g/openembedded-core/message/190558
Mute This Topic: https://lists.openembedded.org/mt/102345308/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-Core][PATCH v3 0/4] recipetool: Add handler to create go recipes
Hi Slava,
On 15.11.2023 06:59, Vyacheslav Yurkov wrote:
Hi Lukas,
Thanks for the v3. I know I'm late to the party, because it's already
merged, but my testing results are below anyway.
No problem. Party is still going on. The 'modules.txt' is intendet to be
in your ${WORKDIR} as it is part of the SRC_URI, generated by
the recipetool. My guess would be, that there is some missconfiguration
in your SRC_URI?
The series seems to have handled my ssh URL correctly.
I've got an error in do_go_vendor though.
File: ''/meta/classes/go-vendor.bbclass', lineno: 166, function:
do_go_vendor
0162:
0163: # Copy vendor manifest
0164: modules_txt_src = os.path.join(d.getVar('WORKDIR'),
"modules.txt")
0165: bb.debug(1, "cp %s --> %s" % (modules_txt_src, vendor_dir))
*** 0166: shutil.copy2(modules_txt_src, vendor_dir)
0167:
0168: # Clean up vendor dir
0169: # We only require the modules in the modules_txt file
0170: fetched_paths = set([os.path.relpath(x[0], vendor_dir)
for x in os.walk(vendor_dir)])
File: '/usr/lib64/python3.9/shutil.py', lineno: 444, function: copy2
0440: resembles GNU's "cp -P src dst".
0441: """
0442: if os.path.isdir(dst):
0443: dst = os.path.join(dst, os.path.basename(src))
*** 0444: copyfile(src, dst, follow_symlinks=follow_symlinks)
0445: copystat(src, dst, follow_symlinks=follow_symlinks)
0446: return dst
0447:
0448:def ignore_patterns(*patterns):
File: '/usr/lib64/python3.9/shutil.py', lineno: 264, function: copyfile
0260:
0261: if not follow_symlinks and _islink(src):
0262: os.symlink(os.readlink(src), dst)
0263: else:
*** 0264: with open(src, 'rb') as fsrc:
0265: try:
0266: with open(dst, 'wb') as fdst:
0267: # macOS
0268: if _HAS_FCOPYFILE:
Exception: FileNotFoundError: [Errno 2] No such file or directory:
'/modules.txt'
I've located "modules.txt" in my $WORKDIR/$GO_IMPORT/vendor directory
though, which seems to be consistent with
https://go.dev/ref/mod#go-mod-file-go . Am I missing something or I can
send a fixup?
Slava
On 02.11.2023 16:53, lukas.funke-...@weidmueller.com wrote:
From: Lukas Funke
This patch series adds a recipetool handler in order to create 'go'
recipes.
Each recipe contains a list of dependencies in their SRC_URI.
Dependencies are derived from the projects `go.mod` file. For each
dependency the corresponding license file uri/hash is added.
The recipe may not work ad-hoc, but is a good starting point to create
a working recipe and have a working offline-build.
In addition to the main recipe three additional files will be generated:
- $pn-modules.inc
- $pn-license.inc
- modules.txt
Changes from v2:
- Generate separate *.inc for go dependencies and licenses
- Adapted oe-selftest according to change above
- Incorparate community suggestions
Lukas Funke (4):
classes: go-vendor: Add go-vendor class
selftest: recipetool: Add test for go recipe handler
recipetool: Ignore *.go files while scanning for licenses
recipetool: Add handler to create go recipes
meta/classes/go-vendor.bbclass | 135
meta/lib/oeqa/selftest/cases/recipetool.py | 163 +
scripts/lib/recipetool/create.py | 2 +-
scripts/lib/recipetool/create_go.py | 730 +
4 files changed, 1029 insertions(+), 1 deletion(-)
create mode 100644 meta/classes/go-vendor.bbclass
create mode 100644 scripts/lib/recipetool/create_go.py
Best regards
Lukas
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#190557):
https://lists.openembedded.org/g/openembedded-core/message/190557
Mute This Topic: https://lists.openembedded.org/mt/102345308/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-Core][PATCH v3 0/4] recipetool: Add handler to create go recipes
Hi Lukas,
Thanks for the v3. I know I'm late to the party, because it's already
merged, but my testing results are below anyway.
The series seems to have handled my ssh URL correctly.
I've got an error in do_go_vendor though.
File: ''/meta/classes/go-vendor.bbclass', lineno: 166, function:
do_go_vendor
0162:
0163: # Copy vendor manifest
0164: modules_txt_src = os.path.join(d.getVar('WORKDIR'),
"modules.txt")
0165: bb.debug(1, "cp %s --> %s" % (modules_txt_src, vendor_dir))
*** 0166: shutil.copy2(modules_txt_src, vendor_dir)
0167:
0168: # Clean up vendor dir
0169: # We only require the modules in the modules_txt file
0170: fetched_paths = set([os.path.relpath(x[0], vendor_dir)
for x in os.walk(vendor_dir)])
File: '/usr/lib64/python3.9/shutil.py', lineno: 444, function: copy2
0440: resembles GNU's "cp -P src dst".
0441: """
0442: if os.path.isdir(dst):
0443: dst = os.path.join(dst, os.path.basename(src))
*** 0444: copyfile(src, dst, follow_symlinks=follow_symlinks)
0445: copystat(src, dst, follow_symlinks=follow_symlinks)
0446: return dst
0447:
0448:def ignore_patterns(*patterns):
File: '/usr/lib64/python3.9/shutil.py', lineno: 264, function: copyfile
0260:
0261: if not follow_symlinks and _islink(src):
0262: os.symlink(os.readlink(src), dst)
0263: else:
*** 0264: with open(src, 'rb') as fsrc:
0265: try:
0266: with open(dst, 'wb') as fdst:
0267: # macOS
0268: if _HAS_FCOPYFILE:
Exception: FileNotFoundError: [Errno 2] No such file or directory:
'/modules.txt'
I've located "modules.txt" in my $WORKDIR/$GO_IMPORT/vendor directory
though, which seems to be consistent with
https://go.dev/ref/mod#go-mod-file-go . Am I missing something or I can
send a fixup?
Slava
On 02.11.2023 16:53, lukas.funke-...@weidmueller.com wrote:
From: Lukas Funke
This patch series adds a recipetool handler in order to create 'go' recipes.
Each recipe contains a list of dependencies in their SRC_URI.
Dependencies are derived from the projects `go.mod` file. For each
dependency the corresponding license file uri/hash is added.
The recipe may not work ad-hoc, but is a good starting point to create
a working recipe and have a working offline-build.
In addition to the main recipe three additional files will be generated:
- $pn-modules.inc
- $pn-license.inc
- modules.txt
Changes from v2:
- Generate separate *.inc for go dependencies and licenses
- Adapted oe-selftest according to change above
- Incorparate community suggestions
Lukas Funke (4):
classes: go-vendor: Add go-vendor class
selftest: recipetool: Add test for go recipe handler
recipetool: Ignore *.go files while scanning for licenses
recipetool: Add handler to create go recipes
meta/classes/go-vendor.bbclass | 135
meta/lib/oeqa/selftest/cases/recipetool.py | 163 +
scripts/lib/recipetool/create.py | 2 +-
scripts/lib/recipetool/create_go.py| 730 +
4 files changed, 1029 insertions(+), 1 deletion(-)
create mode 100644 meta/classes/go-vendor.bbclass
create mode 100644 scripts/lib/recipetool/create_go.py
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#190555):
https://lists.openembedded.org/g/openembedded-core/message/190555
Mute This Topic: https://lists.openembedded.org/mt/102345308/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-Core][PATCH v3 0/4] recipetool: Add handler to create go recipes
On Sun, 2023-11-05 at 08:30 +, Richard Purdie via lists.openembedded.org wrote: > On Thu, 2023-11-02 at 16:53 +0100, Lukas Funke wrote: > > From: Lukas Funke > > > > This patch series adds a recipetool handler in order to create 'go' recipes. > > Each recipe contains a list of dependencies in their SRC_URI. > > Dependencies are derived from the projects `go.mod` file. For each > > dependency the corresponding license file uri/hash is added. > > > > The recipe may not work ad-hoc, but is a good starting point to create > > a working recipe and have a working offline-build. > > > > In addition to the main recipe three additional files will be generated: > > - $pn-modules.inc > > - $pn-license.inc > > - modules.txt > > > > Changes from v2: > > - Generate separate *.inc for go dependencies and licenses > > - Adapted oe-selftest according to change above > > - Incorparate community suggestions > > > > Lukas Funke (4): > > classes: go-vendor: Add go-vendor class > > selftest: recipetool: Add test for go recipe handler > > recipetool: Ignore *.go files while scanning for licenses > > recipetool: Add handler to create go recipes > > This looks to be getting there but the code failed in testing: > > https://autobuilder.yoctoproject.org/typhoon/#/builders/127/builds/2371/steps/14/logs/stdio > > since removeprefix is python 3.9 syntax and 3.8 is our minimum version > at present. Since these were through testing, there was no other negative review feedback, the patches appeared to do what I thought we'd all agreed was needed and I had the minor tweak needed for the python version issue I did merge them. I squashed the tweak in with a comment in the commit message. Cheers, Richard -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#190240): https://lists.openembedded.org/g/openembedded-core/message/190240 Mute This Topic: https://lists.openembedded.org/mt/102345308/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-Core][PATCH v3 0/4] recipetool: Add handler to create go recipes
On Thu, 2023-11-02 at 16:53 +0100, Lukas Funke wrote: > From: Lukas Funke > > This patch series adds a recipetool handler in order to create 'go' recipes. > Each recipe contains a list of dependencies in their SRC_URI. > Dependencies are derived from the projects `go.mod` file. For each > dependency the corresponding license file uri/hash is added. > > The recipe may not work ad-hoc, but is a good starting point to create > a working recipe and have a working offline-build. > > In addition to the main recipe three additional files will be generated: > - $pn-modules.inc > - $pn-license.inc > - modules.txt > > Changes from v2: > - Generate separate *.inc for go dependencies and licenses > - Adapted oe-selftest according to change above > - Incorparate community suggestions > > Lukas Funke (4): > classes: go-vendor: Add go-vendor class > selftest: recipetool: Add test for go recipe handler > recipetool: Ignore *.go files while scanning for licenses > recipetool: Add handler to create go recipes This looks to be getting there but the code failed in testing: https://autobuilder.yoctoproject.org/typhoon/#/builders/127/builds/2371/steps/14/logs/stdio since removeprefix is python 3.9 syntax and 3.8 is our minimum version at present. Cheers, Richard -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#190184): https://lists.openembedded.org/g/openembedded-core/message/190184 Mute This Topic: https://lists.openembedded.org/mt/102345308/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[OE-Core][PATCH v3 0/4] recipetool: Add handler to create go recipes
From: Lukas Funke This patch series adds a recipetool handler in order to create 'go' recipes. Each recipe contains a list of dependencies in their SRC_URI. Dependencies are derived from the projects `go.mod` file. For each dependency the corresponding license file uri/hash is added. The recipe may not work ad-hoc, but is a good starting point to create a working recipe and have a working offline-build. In addition to the main recipe three additional files will be generated: - $pn-modules.inc - $pn-license.inc - modules.txt Changes from v2: - Generate separate *.inc for go dependencies and licenses - Adapted oe-selftest according to change above - Incorparate community suggestions Lukas Funke (4): classes: go-vendor: Add go-vendor class selftest: recipetool: Add test for go recipe handler recipetool: Ignore *.go files while scanning for licenses recipetool: Add handler to create go recipes meta/classes/go-vendor.bbclass | 135 meta/lib/oeqa/selftest/cases/recipetool.py | 163 + scripts/lib/recipetool/create.py | 2 +- scripts/lib/recipetool/create_go.py| 730 + 4 files changed, 1029 insertions(+), 1 deletion(-) create mode 100644 meta/classes/go-vendor.bbclass create mode 100644 scripts/lib/recipetool/create_go.py -- 2.30.2 -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#190082): https://lists.openembedded.org/g/openembedded-core/message/190082 Mute This Topic: https://lists.openembedded.org/mt/102345308/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
