On 10.09.2023 23:26, Martin Jansa wrote:
On Sun, Mar 12, 2023 at 10:15 AM Martin Jansa via lists.openembedded.org
<Martin.Jansa=gmail....@lists.openembedded.org> wrote:

On Tue, Jan 3, 2023 at 5:03 PM Lukas Funke <
lukas.funke-...@weidmueller.com> wrote:

Martin,

first of all: thank you for testing the patches. As usual the corner
cases are the most tricky ones.

tl;dr: I'm working on an updated patch series to address the reported
issues.


Hi Lukas, Stefan,

and update on this series?

It doesn't have to be perfect, other people can help with remaining corner
cases and right now there is nothing for this in oe-core, so any version
will be big improvement :) and it will make it easier for others to submit
incremental improvements - I have some as well, but haven't sent them as I
don't know what you've already updated locally, so I'm waiting for v2.

Mickledore is already in feature freeze, but maybe RP will make an
exception as this is new important functionality which isn't likely to
break other existing code.


Hello Lukas,

any progress on updated patch series? Nanbield release is near and I don't
remember seeing the updated patch series in last couple months.

Is there anything I can do to help? The last version looked reasonably well
and IMHO could be used as starting point for everybody to help fixing those
corner cases where it didn't work well.

Regards,

Hello Martin,

I'm currently working on an update, but can't make any promises regarding the timeline.

Thanks for offering help! Reviewing and testing is always helpful when the updated series is on it's way.

Regards
Lukas




Kind Regards,


I've looked into the issues and would like to give some explanation.

1) the first module you mention ('gonum.org/v1/gonum
<http://gonum.org/v1/gonum>') does not obey the go documentation for
resolving the modules source-code repository (see
https://go.dev/ref/mod#go-mod-download , Section 'Version control
systems'). The documentation states that there should be a html page,
queried by "?go-get=1", with a <meta>-tag which contains the original
URL to the source-code repository. For 'gonum' there is only a 404-page.
This page contains javascript-code that redirects to the actual gonum
package page. Unnecessary hard to process. I'll open an issue for that.
The second module 'code.cloudfoundry.org/clock
<http://code.cloudfoundry.org/clock>' has a broken certificate. I
already opend an issue on github for this.

I addressed this issue above by maintaining a list of modules and their
actual repo URLs inside the go-recipetool as an absolute fallback. Would
do you think about this solution?


Yes, that's what I was starting to do in:

https://git.openembedded.org/openembedded-core-contrib/commit/?h=jansa/go&id=b973c7f17c8a613233a1a18de0bf6daa352c47d8

https://git.openembedded.org/openembedded-core-contrib/commit/?h=jansa/go&id=61c6c9962b3b7d4c1cf6e6823de6c32b09998b00


2) yes, my bad :)


as work around I was using

https://git.openembedded.org/openembedded-core-contrib/commit/?h=jansa/go&id=5767819bfe96de29ab751d4a37a431a324f7e547


3) The problem is that some repositories have a prefix in their tags.
The tags usually only contain the semantic version string. I'm curretly
working on this issue.

Best regards

Lukas

Am 22.12.2022 um 20:08 schrieb Martin Jansa via lists.openembedded.org:
On Thu, Dec 22, 2022 at 7:39 PM Vyacheslav Yurkov <uvv.m...@gmail.com>
wrote:

     On 22.12.2022 17:48, Bruce Ashfield wrote:
     > On Thu, Dec 22, 2022 at 11:20 AM Richard Purdie
     > <richard.pur...@linuxfoundation.org> wrote:
     >> This patch is not in master and is not a backport therefore not
     >> eligible for kirkstone/langdale.

     My bad, the intention was to have it in all three branches: master,
     kirkstone, and langdale.

     > 'nor should it go to master.
     >
     > If someone wants to allow go to fetch sources during builds, it
     needs
     > to be done in their own layers.
     >
     > Bruce

     Probably I misunderstood the outcome of the discussion in my first
     link.
     Bruce and Richard, you both suggested to have it in go-mod with the
     warning. I agree that it breaks reproducibility, and is considered
     a bad
     practice in general (I'm aware of a few more build systems with
     the same
     shortcoming).

     My point is, until a proper solution is in place, it should be at
     least
     documented somewhere that this should be done in own layer/recipe.
     Any
     suggestions where this can be documented if not in OE-Core?


Have you tried the changes submitted by Lukas/Stefan in:
https://lists.openembedded.org/g/openembedded-architecture/message/1539
?

It's not perfect, I was testing it on
https://github.com/influxdata/telegraf/blob/master/go.mod and I've
found some corner cases where it failed.

But it seems like very good start and we should work with Lukas/Stefan
to get it merged in master. Then all branches could consume recipes
created in master and only the exceptions would need to have network
access (instead of the bbclass enabling it for every go-mod user -
go-vendor inherits go-mod as well).

Lukas/Stefan: is there something we can do to help with v2 of your
patches?

I wanted to submit better review on submitted patches after more
debugging, but here it is:

1) to resolve 2 deps:
{'Path': 'gonum.org/v1/gonum <http://gonum.org/v1/gonum>', 'Version':
'v0.12.0'} due to 404 "ERROR: Error while fetching redirect page: HTTP
Error 404: Not Found"

{'Path': 'code.cloudfoundry.org/clock
<http://code.cloudfoundry.org/clock>', 'Version': 'v1.0.0',
'Indirect': True} due to this being redirect to github.com
<http://github.com> and failing with "ERROR: Error while fetching
redirect page: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED]
certificate verify failed: Hostname mismatch, certificate is not valid
for 'code.cloudfoundry.org <http://code.cloudfoundry.org>'.
(_ssl.c:996)>"
parse

2) some issues with LICENSEs:
INFO: Please add the following line for
'docs/LICENSE_OF_DEPENDENCIES.md' to a 'lib/recipetool/licenses.csv'
and replace `Unknown` with the license:
c4807c5b058a56c02f592207b4e056b1,Unknown
Traceback (most recent call last):
   File "/OE/build/oe-core/openembedded-core/scripts/recipetool", line
111, in <module>
     ret = main()
           ^^^^^^
   File "/OE/build/oe-core/openembedded-core/scripts/recipetool", line
100, in main
     ret = args.func(args)
           ^^^^^^^^^^^^^^^
   File
"/OE/layers/openembedded-core/scripts/lib/recipetool/create.py", line
746, in create_recipe
     handler.process(srctree_use, classes, lines_before, lines_after,
handled, extravalues)
   File
"/OE/layers/openembedded-core/scripts/lib/recipetool/create_go.py",
line 349, in process
     self._rewrite_lic_uri(lines_before)
   File
"/OE/layers/openembedded-core/scripts/lib/recipetool/create_go.py",
line 380, in _rewrite_lic_uri
     updated, newlines = bb.utils.edit_metadata(lines_before,
['LIC_FILES_CHKSUM'], varfunc)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
   File "/OE/build/oe-core/bitbake/lib/bb/utils.py", line 1382, in
edit_metadata
     if handle_var_end():
        ^^^^^^^^^^^^^^^^
   File "/OE/build/oe-core/bitbake/lib/bb/utils.py", line 1280, in
handle_var_end
     (newvalue, newop, indent, minbreak) = varfunc(in_var, full_value,
op, newlines)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
   File
"/OE/layers/openembedded-core/scripts/lib/recipetool/create_go.py",
line 372, in varfunc
     uri, chksum = license.split(';', 1)
     ^^^^^^^^^^^
ValueError: not enough values to unpack (expected 2, got 1)

(the license was just '/')

3) some SRCREVs set to None leading to parsing failure:

   File "/OE/layers/bitbake/lib/bb/fetch2/git.py", line 762, in
_latest_revision
     raise bb.fetch2.FetchError("Unable to resolve '%s' in upstream git
repository in git ls-remote output for %s" % \
bb.data_smart.ExpansionError: Failure expanding variable SRCPV,
expression was ${@bb.fetch2.get_srcrev(d)} which triggered exception
FetchError: Fetcher failure: Unable to resolve 'None' in upstream git
repository in git ls-remote output for
github.com/googleapis/google-cloud-go
<http://github.com/googleapis/google-cloud-go>
The variable dependency chain for the failure is: SRCPV -> PV -> BP ->
FILESPATH

ERROR: Parsing newly created recipe failed, moving recipe to

/OE/build/oe-core/workspace/recipes/telegraf/telegraf_git.bb.parsefailed
for reference. If this looks to be caused by the recipe itself, please
report this error.

It takes long time to fetch all telegraf dependencies, so I'll create
some test recipe to more easily debug these 3 issues in isolation.











-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#188320): 
https://lists.openembedded.org/g/openembedded-core/message/188320
Mute This Topic: https://lists.openembedded.org/mt/95827446/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Reply via email to