Re: [OE-core] [PATCH v5 06/13] go.bbclass: ptest cleanup and improvements
On 2019年06月26日 22:00, Matt Madison wrote:
On Wed, Jun 26, 2019 at 2:02 AM Yu, Mingli wrote:
On 2019年06月25日 20:23, Matt Madison wrote:
On Fri, Jun 21, 2019 at 2:08 AM Yu, Mingli wrote:
Hi Matt,
I noticed your commit is the latest update for go-dep ptest. But the
go-dep ptest doesn't work in my environment. I'm trying to figure out
what's wrong is here though I didn't know much about go.
I went back over the commits, and I don't think I did anything with
go-dep specifically. I can see that the tests are failing for it, and
it looks like it's because go-dep's test programs make some
assumptions about the environment. For one thing, it needs the go
compiler installed. It also looks like it's expecting some source
files to be present... in other words, it's not really designed for a
cross-build setup, with tests run on the cross-built target. It could
be messy to work around that, and I'm not sure how useful it would be
anyway, seeing as how go-dep is more of a build tool. Might be better
to just disable ptests for it completely.
Many thanks Matt for your information!
Did you ever run go-dep ptest?
Go through the run-ptest script for go-dep, it actually runs the
/usr/lib64/go-dep/ptest/github.com/golang/dep/cmd/dep/dep.test whose
source file is
https://github.com/golang/dep/blob/master/cmd/dep/dep_test.go.
Yes, I see that. That main program starts by rebuilding the dep
program from source, then runs the tests using that copy of the
program, so it's assuming that you're still in a development
environment where you can run a full go build. That's what I meant by
it not being designed for a cross-build setup.
I've patched the test program to create a symlink to the installed dep
program instead of rebuilding, and got further. You'll need to
include 'go', 'git', and 'git-perltools' in the image to get the tests
to run. Some of the test cases failed until I set GOCACHE in the
environment to point to a valid path; I'm not sure why. The tests
took a long time to run on my qemux86 build, though - at least 30
minutes.
Thanks very much for Matt's guide!
I have applied the patch you provide to create a symlink to the
installed dep program instead of rebuilding and also add go', 'git', and
'git-perltools' into the image, but there is no PASS case, all failed as
attached go-dep_ptest.log.
BTW, you said you set GOCACHE, what's the valid path do you mean?
Thanks,
Here's the patch I applied to use the already-built dep binary. An
alternative approach would be to put everything in place to allow the
test program to rebuild the dep binary itself, but that would drive up
the time for the test run even further.
Regards,
-Matt
---
cmd/dep/dep_test.go | 13 +
1 file changed, 9 insertions(+), 4 deletions(-)
diff --git a/src/github.com/golang/dep/cmd/dep/dep_test.go
b/src/github.com/golang/dep/cmd/dep/dep_test.go
index 687eef3a..535ca56d 100644
--- a/src/github.com/golang/dep/cmd/dep/dep_test.go
+++ b/src/github.com/golang/dep/cmd/dep/dep_test.go
@@ -18,13 +18,18 @@ import (
// deletes it after the tests have been run.
// Most of this is taken from
https://github.com/golang/go/blob/master/src/cmd/go/go_test.go and
reused here.
func TestMain(m *testing.M) {
- args := []string{"build", "-o", "testdep" + test.ExeSuffix}
- out, err := exec.Command("go", args...).CombinedOutput()
+ args := []string{"dep"}
+ out, err := exec.Command("which", args...).CombinedOutput()
if err != nil {
- fmt.Fprintf(os.Stderr, "building testdep failed: %v\n%s", err, out)
+ fmt.Fprintf(os.Stderr, "finding dep failed: %v\n%s", err, out)
+ os.Exit(2)
+ }
+ args = []string{"-s", string(out[:len(out)-1]), "testdep" + test.ExeSuffix}
+ out, err = exec.Command("ln", args...).CombinedOutput()
+ if err != nil {
+ fmt.Fprintf(os.Stderr, "symlinking dep failed: %v\n%s", err, out)
os.Exit(2)
}
-
// Don't let these environment variables confuse the test.
os.Unsetenv("GOPATH")
os.Unsetenv("GIT_ALLOW_PROTOCOL")
--- FAIL: TestIntegration (0.46s)
--- FAIL: TestIntegration/status/revision_constraint (0.00s)
--- FAIL: TestIntegration/status/revision_constraint/external (0.53s)
integration_test.go:194: cmd ensure raised an unexpected error: exit status 2
--- FAIL: TestIntegration/status/override_constraint (0.00s)
--- FAIL: TestIntegration/status/override_constraint/external (0.52s)
integration_test.go:194: cmd ensure raised an unexpected error: exit status 2
--- FAIL: TestIntegration/status/old_constraints (0.00s)
--- FAIL: TestIntegration/status/old_constraints/external (0.80s)
integration_test.go:194: cmd ensure raised an unexpected error: exit status 2
--- FAIL: TestIntegration/status/missing_pkgs_lock_mismatch (0.00s)
--- FAIL: TestIntegration/status/missing_pkgs_lock_mismatch/external (0.60s)
integration_test.go:199: exit status 1
testcase.go:165: error did not contain expected string:
Re: [OE-core] [PATCH v5 06/13] go.bbclass: ptest cleanup and improvements
On Wed, Jun 26, 2019 at 11:40 PM Yu, Mingli wrote:
>
>
>
> On 2019年06月26日 22:00, Matt Madison wrote:
> > On Wed, Jun 26, 2019 at 2:02 AM Yu, Mingli wrote:
> >>
> >>
> >>
> >> On 2019年06月25日 20:23, Matt Madison wrote:
> >>> On Fri, Jun 21, 2019 at 2:08 AM Yu, Mingli
> >>> wrote:
>
> Hi Matt,
>
> I noticed your commit is the latest update for go-dep ptest. But the
> go-dep ptest doesn't work in my environment. I'm trying to figure out
> what's wrong is here though I didn't know much about go.
> >>>
> >>> I went back over the commits, and I don't think I did anything with
> >>> go-dep specifically. I can see that the tests are failing for it, and
> >>> it looks like it's because go-dep's test programs make some
> >>> assumptions about the environment. For one thing, it needs the go
> >>> compiler installed. It also looks like it's expecting some source
> >>> files to be present... in other words, it's not really designed for a
> >>> cross-build setup, with tests run on the cross-built target. It could
> >>> be messy to work around that, and I'm not sure how useful it would be
> >>> anyway, seeing as how go-dep is more of a build tool. Might be better
> >>> to just disable ptests for it completely.
> >>
> >> Many thanks Matt for your information!
> >> Did you ever run go-dep ptest?
> >> Go through the run-ptest script for go-dep, it actually runs the
> >> /usr/lib64/go-dep/ptest/github.com/golang/dep/cmd/dep/dep.test whose
> >> source file is
> >> https://github.com/golang/dep/blob/master/cmd/dep/dep_test.go.
> >
> > Yes, I see that. That main program starts by rebuilding the dep
> > program from source, then runs the tests using that copy of the
> > program, so it's assuming that you're still in a development
> > environment where you can run a full go build. That's what I meant by
> > it not being designed for a cross-build setup.
> >
> > I've patched the test program to create a symlink to the installed dep
> > program instead of rebuilding, and got further. You'll need to
> > include 'go', 'git', and 'git-perltools' in the image to get the tests
> > to run. Some of the test cases failed until I set GOCACHE in the
> > environment to point to a valid path; I'm not sure why. The tests
> > took a long time to run on my qemux86 build, though - at least 30
> > minutes.
>
> Thanks very much for Matt's guide!
>
> I have applied the patch you provide to create a symlink to the
> installed dep program instead of rebuilding and also add go', 'git', and
> 'git-perltools' into the image, but there is no PASS case, all failed as
> attached go-dep_ptest.log.
The device under test needs Internet access to fetch repos from
Github. From the log, it looks like perhaps yours didn't?
>
> BTW, you said you set GOCACHE, what's the valid path do you mean?
Any directory that the user has write access to. The default is
$HOME/.cache/go-build .
-M
>
> Thanks,
>
> >
> > Here's the patch I applied to use the already-built dep binary. An
> > alternative approach would be to put everything in place to allow the
> > test program to rebuild the dep binary itself, but that would drive up
> > the time for the test run even further.
> >
> > Regards,
> > -Matt
> >
> > ---
> > cmd/dep/dep_test.go | 13 +
> > 1 file changed, 9 insertions(+), 4 deletions(-)
> >
> > diff --git a/src/github.com/golang/dep/cmd/dep/dep_test.go
> > b/src/github.com/golang/dep/cmd/dep/dep_test.go
> > index 687eef3a..535ca56d 100644
> > --- a/src/github.com/golang/dep/cmd/dep/dep_test.go
> > +++ b/src/github.com/golang/dep/cmd/dep/dep_test.go
> > @@ -18,13 +18,18 @@ import (
> > // deletes it after the tests have been run.
> > // Most of this is taken from
> > https://github.com/golang/go/blob/master/src/cmd/go/go_test.go and
> > reused here.
> > func TestMain(m *testing.M) {
> > - args := []string{"build", "-o", "testdep" + test.ExeSuffix}
> > - out, err := exec.Command("go", args...).CombinedOutput()
> > + args := []string{"dep"}
> > + out, err := exec.Command("which", args...).CombinedOutput()
> >if err != nil {
> > - fmt.Fprintf(os.Stderr, "building testdep failed: %v\n%s", err, out)
> > + fmt.Fprintf(os.Stderr, "finding dep failed: %v\n%s", err, out)
> > + os.Exit(2)
> > + }
> > + args = []string{"-s", string(out[:len(out)-1]), "testdep" +
> > test.ExeSuffix}
> > + out, err = exec.Command("ln", args...).CombinedOutput()
> > + if err != nil {
> > + fmt.Fprintf(os.Stderr, "symlinking dep failed: %v\n%s", err, out)
> >os.Exit(2)
> >}
> > -
> >// Don't let these environment variables confuse the test.
> >os.Unsetenv("GOPATH")
> >os.Unsetenv("GIT_ALLOW_PROTOCOL")
> >
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH v5 06/13] go.bbclass: ptest cleanup and improvements
On 2019年06月27日 19:34, Matt Madison wrote:
On Wed, Jun 26, 2019 at 11:40 PM Yu, Mingli wrote:
On 2019年06月26日 22:00, Matt Madison wrote:
On Wed, Jun 26, 2019 at 2:02 AM Yu, Mingli wrote:
On 2019年06月25日 20:23, Matt Madison wrote:
On Fri, Jun 21, 2019 at 2:08 AM Yu, Mingli wrote:
Hi Matt,
I noticed your commit is the latest update for go-dep ptest. But the
go-dep ptest doesn't work in my environment. I'm trying to figure out
what's wrong is here though I didn't know much about go.
I went back over the commits, and I don't think I did anything with
go-dep specifically. I can see that the tests are failing for it, and
it looks like it's because go-dep's test programs make some
assumptions about the environment. For one thing, it needs the go
compiler installed. It also looks like it's expecting some source
files to be present... in other words, it's not really designed for a
cross-build setup, with tests run on the cross-built target. It could
be messy to work around that, and I'm not sure how useful it would be
anyway, seeing as how go-dep is more of a build tool. Might be better
to just disable ptests for it completely.
Many thanks Matt for your information!
Did you ever run go-dep ptest?
Go through the run-ptest script for go-dep, it actually runs the
/usr/lib64/go-dep/ptest/github.com/golang/dep/cmd/dep/dep.test whose
source file is
https://github.com/golang/dep/blob/master/cmd/dep/dep_test.go.
Yes, I see that. That main program starts by rebuilding the dep
program from source, then runs the tests using that copy of the
program, so it's assuming that you're still in a development
environment where you can run a full go build. That's what I meant by
it not being designed for a cross-build setup.
I've patched the test program to create a symlink to the installed dep
program instead of rebuilding, and got further. You'll need to
include 'go', 'git', and 'git-perltools' in the image to get the tests
to run. Some of the test cases failed until I set GOCACHE in the
environment to point to a valid path; I'm not sure why. The tests
took a long time to run on my qemux86 build, though - at least 30
minutes.
Thanks very much for Matt's guide!
I have applied the patch you provide to create a symlink to the
installed dep program instead of rebuilding and also add go', 'git', and
'git-perltools' into the image, but there is no PASS case, all failed as
attached go-dep_ptest.log.
The device under test needs Internet access to fetch repos from
Github. From the log, it looks like perhaps yours didn't?
BTW, you said you set GOCACHE, what's the valid path do you mean?
Any directory that the user has write access to. The default is
$HOME/.cache/go-build .
There is no difference after set GOCACHE, still all the case failed.
Thanks,
-M
Thanks,
Here's the patch I applied to use the already-built dep binary. An
alternative approach would be to put everything in place to allow the
test program to rebuild the dep binary itself, but that would drive up
the time for the test run even further.
Regards,
-Matt
---
cmd/dep/dep_test.go | 13 +
1 file changed, 9 insertions(+), 4 deletions(-)
diff --git a/src/github.com/golang/dep/cmd/dep/dep_test.go
b/src/github.com/golang/dep/cmd/dep/dep_test.go
index 687eef3a..535ca56d 100644
--- a/src/github.com/golang/dep/cmd/dep/dep_test.go
+++ b/src/github.com/golang/dep/cmd/dep/dep_test.go
@@ -18,13 +18,18 @@ import (
// deletes it after the tests have been run.
// Most of this is taken from
https://github.com/golang/go/blob/master/src/cmd/go/go_test.go and
reused here.
func TestMain(m *testing.M) {
- args := []string{"build", "-o", "testdep" + test.ExeSuffix}
- out, err := exec.Command("go", args...).CombinedOutput()
+ args := []string{"dep"}
+ out, err := exec.Command("which", args...).CombinedOutput()
if err != nil {
- fmt.Fprintf(os.Stderr, "building testdep failed: %v\n%s", err, out)
+ fmt.Fprintf(os.Stderr, "finding dep failed: %v\n%s", err, out)
+ os.Exit(2)
+ }
+ args = []string{"-s", string(out[:len(out)-1]), "testdep" + test.ExeSuffix}
+ out, err = exec.Command("ln", args...).CombinedOutput()
+ if err != nil {
+ fmt.Fprintf(os.Stderr, "symlinking dep failed: %v\n%s", err, out)
os.Exit(2)
}
-
// Don't let these environment variables confuse the test.
os.Unsetenv("GOPATH")
os.Unsetenv("GIT_ALLOW_PROTOCOL")
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH v5 06/13] go.bbclass: ptest cleanup and improvements
Hi Matt,
I noticed your commit is the latest update for go-dep ptest. But the
go-dep ptest doesn't work in my environment. I'm trying to figure out
what's wrong is here though I didn't know much about go.
BTW, Could you help to check?
root@qemux86-64:~# cd /usr/lib64/go-dep/ptest/
root@qemux86-64:/usr/lib64/go-dep/ptest# cat run-ptest
#!/bin/sh
RC=0
run_test() (
cd "$1"
./$2 ; echo $? >&3) | sed -r -e"s,^(PASS|SKIP|FAIL)$,\1:
$1/$2," >&4) 3>&1) | (read rc; exit $rc)) 4>&1
exit $?)
run_test github.com/golang/dep/cmd/dep dep.test || RC=1
exit $RC
root@qemux86-64:/usr/lib64/go-dep/ptest# ./run-ptest
building testdep failed: exit status 1
can't load package: package .: no Go files in
/usr/lib64/go-dep/ptest/github.com/golang/dep/cmd/dep
root@qemux86-64:/usr/lib64/go-dep/ptest#
Thanks,
On 2018年03月05日 05:09, Matt Madison wrote:
* Don't enable verbose test output (-test.v)
by default, as it generates too much noise
for automated results parsing
* Override do_install_ptest_base in the bbclass,
so recipes can provide their own modifications
with do_install_ptest.
* Improve the generated run-ptest script to better
handle large numbers of tests, and to generate
'status: test name' output similar to Automake
tests.
* Install all non-vendored 'testdata' directories
from the source into the ptest package, as some
packages share test data among multiple tests.
Signed-off-by: Matt Madison
---
meta/classes/go.bbclass | 87 +
1 file changed, 51 insertions(+), 36 deletions(-)
diff --git a/meta/classes/go.bbclass b/meta/classes/go.bbclass
index afd68b5951..a51ba3e9f0 100644
--- a/meta/classes/go.bbclass
+++ b/meta/classes/go.bbclass
@@ -26,7 +26,7 @@ GO_LDFLAGS ?= '-ldflags="${GO_RPATH} ${GO_LINKMODE}
-extldflags '${GO_EXTLDFLAGS
export GOBUILDFLAGS ?= "-v ${GO_LDFLAGS}"
export GOPATH_OMIT_IN_ACTIONID ?= "1"
export GOPTESTBUILDFLAGS ?= "${GOBUILDFLAGS} -c"
-export GOPTESTFLAGS ?= "-test.v"
+export GOPTESTFLAGS ?= ""
GOBUILDFLAGS_prepend_task-compile = "${GO_PARALLEL_BUILD} "
export GO = "${HOST_PREFIX}go"
@@ -76,7 +76,7 @@ go_list_packages() {
}
go_list_package_tests() {
-${GO} list -f '{{.ImportPath}} {{.TestGoFiles}}' ${GOBUILDFLAGS}
${GO_INSTALL} | \
+ ${GO} list -f '{{.ImportPath}} {{.TestGoFiles}}' ${GOBUILDFLAGS}
${GO_INSTALL} | \
grep -v '\[\]$' | \
egrep -v '${GO_INSTALL_FILTEROUT}' | \
awk '{ print $1 }'
@@ -100,15 +100,16 @@ go_do_compile() {
do_compile[dirs] =+ "${GOTMPDIR}"
do_compile[cleandirs] = "${B}/bin ${B}/pkg"
-do_compile_ptest() {
+do_compile_ptest_base() {
export TMPDIR="${GOTMPDIR}"
-rm -f ${B}/.go_compiled_tests.list
+ rm -f ${B}/.go_compiled_tests.list
go_list_package_tests | while read pkg; do
cd ${B}/src/$pkg
${GO} test ${GOPTESTBUILDFLAGS} $pkg
find . -mindepth 1 -maxdepth 1 -type f -name '*.test' -exec
echo $pkg/{} \; | \
sed -e's,/\./,/,'>> ${B}/.go_compiled_tests.list
done
+ do_compile_ptest
}
do_compile_ptest_base[dirs] =+ "${GOTMPDIR}"
@@ -124,40 +125,54 @@ go_do_install() {
fi
}
-do_install_ptest_base() {
-test -f "${B}/.go_compiled_tests.list" || exit 0
-tests=""
-while read test; do
-tests="$tests${tests:+ }${test%.test}"
-testdir=`dirname $test`
-install -d ${D}${PTEST_PATH}/$testdir
-install -m 0755 ${B}/src/$test ${D}${PTEST_PATH}/$test
-if [ -d "${B}/src/$testdir/testdata" ]; then
-cp --preserve=mode,timestamps -R "${B}/src/$testdir/testdata"
${D}${PTEST_PATH}/$testdir
-fi
-done < ${B}/.go_compiled_tests.list
-if [ -n "$tests" ]; then
-install -d ${D}${PTEST_PATH}
-cat >${D}${PTEST_PATH}/run-ptest <${D}${PTEST_PATH}/run-ptest <&1; then
-ANYFAILED=1
-fi
-done
-if [ \$ANYFAILED -ne 0 ]; then
-echo "FAIL: ${PN}"
-exit 1
-fi
-echo "PASS: ${PN}"
-exit 0
+RC=0
+run_test() (
+cd "\$1"
+./\$2 ${GOPTESTFLAGS}; echo \$? >&3) | sed -r -e"s,^(PASS|SKIP|FAIL)\$,\\1: \$1/\$2,"
>&4) 3>&1) | (read rc; exit \$rc)) 4>&1
+exit \$?)
EOF
-chmod +x ${D}${PTEST_PATH}/run-ptest
-else
-rm -rf ${D}${PTEST_PATH}
-fi
+
+}
+
+go_stage_testdata() {
+ oldwd="$PWD"
+ cd ${S}/src
+ find ${GO_IMPORT} -depth -type d -name testdata | while read d; do
+ if echo "$d" | grep -q '/vendor/'; then
+ continue
+ fi
+ parent=`dirname $d`
+ install -d ${D}${PTEST_PATH}/$parent
+ cp --preserve=mode,timestamps -R $d ${D}${PTEST_PATH}/$parent/
+ done
+ cd "$oldwd"
+}
+
+do_install_ptest_base() {
+ test -f "${B}/.go_compiled_tests.list" || exit 0
+ install -d ${D}${PTEST_PATH}
+ go_stage_testdata
+ go_make_ptest_wrapp
Re: [OE-core] [PATCH v5 06/13] go.bbclass: ptest cleanup and improvements
On Fri, Jun 21, 2019 at 2:08 AM Yu, Mingli wrote:
>
> Hi Matt,
>
> I noticed your commit is the latest update for go-dep ptest. But the
> go-dep ptest doesn't work in my environment. I'm trying to figure out
> what's wrong is here though I didn't know much about go.
I went back over the commits, and I don't think I did anything with
go-dep specifically. I can see that the tests are failing for it, and
it looks like it's because go-dep's test programs make some
assumptions about the environment. For one thing, it needs the go
compiler installed. It also looks like it's expecting some source
files to be present... in other words, it's not really designed for a
cross-build setup, with tests run on the cross-built target. It could
be messy to work around that, and I'm not sure how useful it would be
anyway, seeing as how go-dep is more of a build tool. Might be better
to just disable ptests for it completely.
BTW, you can have the test programs generate verbose output by setting
GOPTESTFLAGS = "-test.v" in the recipe, which will add that flag to
their invocation in the wrapper script, or by manually running the
test program on the target and passing that flag (cd into the
directory where the test program is located, then run it with
-test.v).
Regards,
-Matt
>
> BTW, Could you help to check?
>
> root@qemux86-64:~# cd /usr/lib64/go-dep/ptest/
> root@qemux86-64:/usr/lib64/go-dep/ptest# cat run-ptest
> #!/bin/sh
> RC=0
> run_test() (
> cd "$1"
> ./$2 ; echo $? >&3) | sed -r -e"s,^(PASS|SKIP|FAIL)$,\1:
> $1/$2," >&4) 3>&1) | (read rc; exit $rc)) 4>&1
> exit $?)
> run_test github.com/golang/dep/cmd/dep dep.test || RC=1
> exit $RC
> root@qemux86-64:/usr/lib64/go-dep/ptest# ./run-ptest
> building testdep failed: exit status 1
> can't load package: package .: no Go files in
> /usr/lib64/go-dep/ptest/github.com/golang/dep/cmd/dep
> root@qemux86-64:/usr/lib64/go-dep/ptest#
>
> Thanks,
>
> On 2018年03月05日 05:09, Matt Madison wrote:
> > * Don't enable verbose test output (-test.v)
> >by default, as it generates too much noise
> >for automated results parsing
> >
> > * Override do_install_ptest_base in the bbclass,
> >so recipes can provide their own modifications
> >with do_install_ptest.
> >
> > * Improve the generated run-ptest script to better
> >handle large numbers of tests, and to generate
> >'status: test name' output similar to Automake
> >tests.
> >
> > * Install all non-vendored 'testdata' directories
> >from the source into the ptest package, as some
> >packages share test data among multiple tests.
> >
> > Signed-off-by: Matt Madison
> > ---
> > meta/classes/go.bbclass | 87
> > +
> > 1 file changed, 51 insertions(+), 36 deletions(-)
> >
> > diff --git a/meta/classes/go.bbclass b/meta/classes/go.bbclass
> > index afd68b5951..a51ba3e9f0 100644
> > --- a/meta/classes/go.bbclass
> > +++ b/meta/classes/go.bbclass
> > @@ -26,7 +26,7 @@ GO_LDFLAGS ?= '-ldflags="${GO_RPATH} ${GO_LINKMODE}
> > -extldflags '${GO_EXTLDFLAGS
> > export GOBUILDFLAGS ?= "-v ${GO_LDFLAGS}"
> > export GOPATH_OMIT_IN_ACTIONID ?= "1"
> > export GOPTESTBUILDFLAGS ?= "${GOBUILDFLAGS} -c"
> > -export GOPTESTFLAGS ?= "-test.v"
> > +export GOPTESTFLAGS ?= ""
> > GOBUILDFLAGS_prepend_task-compile = "${GO_PARALLEL_BUILD} "
> >
> > export GO = "${HOST_PREFIX}go"
> > @@ -76,7 +76,7 @@ go_list_packages() {
> > }
> >
> > go_list_package_tests() {
> > -${GO} list -f '{{.ImportPath}} {{.TestGoFiles}}' ${GOBUILDFLAGS}
> > ${GO_INSTALL} | \
> > + ${GO} list -f '{{.ImportPath}} {{.TestGoFiles}}' ${GOBUILDFLAGS}
> > ${GO_INSTALL} | \
> > grep -v '\[\]$' | \
> > egrep -v '${GO_INSTALL_FILTEROUT}' | \
> > awk '{ print $1 }'
> > @@ -100,15 +100,16 @@ go_do_compile() {
> > do_compile[dirs] =+ "${GOTMPDIR}"
> > do_compile[cleandirs] = "${B}/bin ${B}/pkg"
> >
> > -do_compile_ptest() {
> > +do_compile_ptest_base() {
> > export TMPDIR="${GOTMPDIR}"
> > -rm -f ${B}/.go_compiled_tests.list
> > + rm -f ${B}/.go_compiled_tests.list
> > go_list_package_tests | while read pkg; do
> > cd ${B}/src/$pkg
> > ${GO} test ${GOPTESTBUILDFLAGS} $pkg
> > find . -mindepth 1 -maxdepth 1 -type f -name '*.test' -exec
> > echo $pkg/{} \; | \
> > sed -e's,/\./,/,'>> ${B}/.go_compiled_tests.list
> > done
> > + do_compile_ptest
> > }
> > do_compile_ptest_base[dirs] =+ "${GOTMPDIR}"
> >
> > @@ -124,40 +125,54 @@ go_do_install() {
> > fi
> > }
> >
> > -do_install_ptest_base() {
> > -test -f "${B}/.go_compiled_tests.list" || exit 0
> > -tests=""
> > -while read test; do
> > -tests="$tests${tests:+ }${test%.test}"
> > -testdir=`dirname $test`
> > -install -d ${D}${PTEST_PATH}/$testdir
> > -install -m 0755 ${B}/src/$test ${D}${PTEST_PATH}/$test
> > -if [ -d "$
Re: [OE-core] [PATCH v5 06/13] go.bbclass: ptest cleanup and improvements
On 2019年06月25日 20:23, Matt Madison wrote:
On Fri, Jun 21, 2019 at 2:08 AM Yu, Mingli wrote:
Hi Matt,
I noticed your commit is the latest update for go-dep ptest. But the
go-dep ptest doesn't work in my environment. I'm trying to figure out
what's wrong is here though I didn't know much about go.
I went back over the commits, and I don't think I did anything with
go-dep specifically. I can see that the tests are failing for it, and
it looks like it's because go-dep's test programs make some
assumptions about the environment. For one thing, it needs the go
compiler installed. It also looks like it's expecting some source
files to be present... in other words, it's not really designed for a
cross-build setup, with tests run on the cross-built target. It could
be messy to work around that, and I'm not sure how useful it would be
anyway, seeing as how go-dep is more of a build tool. Might be better
to just disable ptests for it completely.
Many thanks Matt for your information!
Did you ever run go-dep ptest?
Go through the run-ptest script for go-dep, it actually runs the
/usr/lib64/go-dep/ptest/github.com/golang/dep/cmd/dep/dep.test whose
source file is
https://github.com/golang/dep/blob/master/cmd/dep/dep_test.go.
# pwd
/usr/lib64/go-dep/ptest/github.com/golang/dep/cmd/dep
# ./dep.test
building testdep failed: exit status 1
can't load package: package .: no Go files in
/usr/lib64/go-dep/ptest/github.com/golang/dep/cmd/dep
Check the logic for
https://github.com/golang/dep/blob/master/cmd/dep/dep_test.go, it try to
run "go build -o testdep" at its start. And the go compiler exist on our
target, but I don't know what go files the dep_test.go expects.
Hi Khem,
Do you have any suggestion?
Thanks,
BTW, you can have the test programs generate verbose output by setting
GOPTESTFLAGS = "-test.v" in the recipe, which will add that flag to
their invocation in the wrapper script, or by manually running the
test program on the target and passing that flag (cd into the
directory where the test program is located, then run it with
-test.v).
Regards,
-Matt
BTW, Could you help to check?
root@qemux86-64:~# cd /usr/lib64/go-dep/ptest/
root@qemux86-64:/usr/lib64/go-dep/ptest# cat run-ptest
#!/bin/sh
RC=0
run_test() (
cd "$1"
./$2 ; echo $? >&3) | sed -r -e"s,^(PASS|SKIP|FAIL)$,\1:
$1/$2," >&4) 3>&1) | (read rc; exit $rc)) 4>&1
exit $?)
run_test github.com/golang/dep/cmd/dep dep.test || RC=1
exit $RC
root@qemux86-64:/usr/lib64/go-dep/ptest# ./run-ptest
building testdep failed: exit status 1
can't load package: package .: no Go files in
/usr/lib64/go-dep/ptest/github.com/golang/dep/cmd/dep
root@qemux86-64:/usr/lib64/go-dep/ptest#
Thanks,
On 2018年03月05日 05:09, Matt Madison wrote:
* Don't enable verbose test output (-test.v)
by default, as it generates too much noise
for automated results parsing
* Override do_install_ptest_base in the bbclass,
so recipes can provide their own modifications
with do_install_ptest.
* Improve the generated run-ptest script to better
handle large numbers of tests, and to generate
'status: test name' output similar to Automake
tests.
* Install all non-vendored 'testdata' directories
from the source into the ptest package, as some
packages share test data among multiple tests.
Signed-off-by: Matt Madison
---
meta/classes/go.bbclass | 87
+
1 file changed, 51 insertions(+), 36 deletions(-)
diff --git a/meta/classes/go.bbclass b/meta/classes/go.bbclass
index afd68b5951..a51ba3e9f0 100644
--- a/meta/classes/go.bbclass
+++ b/meta/classes/go.bbclass
@@ -26,7 +26,7 @@ GO_LDFLAGS ?= '-ldflags="${GO_RPATH} ${GO_LINKMODE}
-extldflags '${GO_EXTLDFLAGS
export GOBUILDFLAGS ?= "-v ${GO_LDFLAGS}"
export GOPATH_OMIT_IN_ACTIONID ?= "1"
export GOPTESTBUILDFLAGS ?= "${GOBUILDFLAGS} -c"
-export GOPTESTFLAGS ?= "-test.v"
+export GOPTESTFLAGS ?= ""
GOBUILDFLAGS_prepend_task-compile = "${GO_PARALLEL_BUILD} "
export GO = "${HOST_PREFIX}go"
@@ -76,7 +76,7 @@ go_list_packages() {
}
go_list_package_tests() {
-${GO} list -f '{{.ImportPath}} {{.TestGoFiles}}' ${GOBUILDFLAGS}
${GO_INSTALL} | \
+ ${GO} list -f '{{.ImportPath}} {{.TestGoFiles}}' ${GOBUILDFLAGS}
${GO_INSTALL} | \
grep -v '\[\]$' | \
egrep -v '${GO_INSTALL_FILTEROUT}' | \
awk '{ print $1 }'
@@ -100,15 +100,16 @@ go_do_compile() {
do_compile[dirs] =+ "${GOTMPDIR}"
do_compile[cleandirs] = "${B}/bin ${B}/pkg"
-do_compile_ptest() {
+do_compile_ptest_base() {
export TMPDIR="${GOTMPDIR}"
-rm -f ${B}/.go_compiled_tests.list
+ rm -f ${B}/.go_compiled_tests.list
go_list_package_tests | while read pkg; do
cd ${B}/src/$pkg
${GO} test ${GOPTESTBUILDFLAGS} $pkg
find . -mindepth 1 -maxdepth 1 -type f -name '*.test' -exec echo
$pkg/{} \; | \
sed -e's,/\./,/,
Re: [OE-core] [PATCH v5 06/13] go.bbclass: ptest cleanup and improvements
On Wed, Jun 26, 2019 at 2:02 AM Yu, Mingli wrote:
>
>
>
> On 2019年06月25日 20:23, Matt Madison wrote:
> > On Fri, Jun 21, 2019 at 2:08 AM Yu, Mingli wrote:
> >>
> >> Hi Matt,
> >>
> >> I noticed your commit is the latest update for go-dep ptest. But the
> >> go-dep ptest doesn't work in my environment. I'm trying to figure out
> >> what's wrong is here though I didn't know much about go.
> >
> > I went back over the commits, and I don't think I did anything with
> > go-dep specifically. I can see that the tests are failing for it, and
> > it looks like it's because go-dep's test programs make some
> > assumptions about the environment. For one thing, it needs the go
> > compiler installed. It also looks like it's expecting some source
> > files to be present... in other words, it's not really designed for a
> > cross-build setup, with tests run on the cross-built target. It could
> > be messy to work around that, and I'm not sure how useful it would be
> > anyway, seeing as how go-dep is more of a build tool. Might be better
> > to just disable ptests for it completely.
>
> Many thanks Matt for your information!
> Did you ever run go-dep ptest?
> Go through the run-ptest script for go-dep, it actually runs the
> /usr/lib64/go-dep/ptest/github.com/golang/dep/cmd/dep/dep.test whose
> source file is
> https://github.com/golang/dep/blob/master/cmd/dep/dep_test.go.
Yes, I see that. That main program starts by rebuilding the dep
program from source, then runs the tests using that copy of the
program, so it's assuming that you're still in a development
environment where you can run a full go build. That's what I meant by
it not being designed for a cross-build setup.
I've patched the test program to create a symlink to the installed dep
program instead of rebuilding, and got further. You'll need to
include 'go', 'git', and 'git-perltools' in the image to get the tests
to run. Some of the test cases failed until I set GOCACHE in the
environment to point to a valid path; I'm not sure why. The tests
took a long time to run on my qemux86 build, though - at least 30
minutes.
Here's the patch I applied to use the already-built dep binary. An
alternative approach would be to put everything in place to allow the
test program to rebuild the dep binary itself, but that would drive up
the time for the test run even further.
Regards,
-Matt
---
cmd/dep/dep_test.go | 13 +
1 file changed, 9 insertions(+), 4 deletions(-)
diff --git a/src/github.com/golang/dep/cmd/dep/dep_test.go
b/src/github.com/golang/dep/cmd/dep/dep_test.go
index 687eef3a..535ca56d 100644
--- a/src/github.com/golang/dep/cmd/dep/dep_test.go
+++ b/src/github.com/golang/dep/cmd/dep/dep_test.go
@@ -18,13 +18,18 @@ import (
// deletes it after the tests have been run.
// Most of this is taken from
https://github.com/golang/go/blob/master/src/cmd/go/go_test.go and
reused here.
func TestMain(m *testing.M) {
- args := []string{"build", "-o", "testdep" + test.ExeSuffix}
- out, err := exec.Command("go", args...).CombinedOutput()
+ args := []string{"dep"}
+ out, err := exec.Command("which", args...).CombinedOutput()
if err != nil {
- fmt.Fprintf(os.Stderr, "building testdep failed: %v\n%s", err, out)
+ fmt.Fprintf(os.Stderr, "finding dep failed: %v\n%s", err, out)
+ os.Exit(2)
+ }
+ args = []string{"-s", string(out[:len(out)-1]), "testdep" + test.ExeSuffix}
+ out, err = exec.Command("ln", args...).CombinedOutput()
+ if err != nil {
+ fmt.Fprintf(os.Stderr, "symlinking dep failed: %v\n%s", err, out)
os.Exit(2)
}
-
// Don't let these environment variables confuse the test.
os.Unsetenv("GOPATH")
os.Unsetenv("GIT_ALLOW_PROTOCOL")
--
2.20.1
>
> # pwd
> /usr/lib64/go-dep/ptest/github.com/golang/dep/cmd/dep
> # ./dep.test
> building testdep failed: exit status 1
> can't load package: package .: no Go files in
> /usr/lib64/go-dep/ptest/github.com/golang/dep/cmd/dep
>
> Check the logic for
> https://github.com/golang/dep/blob/master/cmd/dep/dep_test.go, it try to
> run "go build -o testdep" at its start. And the go compiler exist on our
> target, but I don't know what go files the dep_test.go expects.
>
> Hi Khem,
>
> Do you have any suggestion?
>
> Thanks,
>
> >
> > BTW, you can have the test programs generate verbose output by setting
> > GOPTESTFLAGS = "-test.v" in the recipe, which will add that flag to
> > their invocation in the wrapper script, or by manually running the
> > test program on the target and passing that flag (cd into the
> > directory where the test program is located, then run it with
> > -test.v).
> >
> > Regards,
> > -Matt
> >
> >>
> >> BTW, Could you help to check?
> >>
> >> root@qemux86-64:~# cd /usr/lib64/go-dep/ptest/
> >> root@qemux86-64:/usr/lib64/go-dep/ptest# cat run-ptest
> >> #!/bin/sh
> >> RC=0
> >> run_test() (
> >> cd "$1"
> >> ./$2 ; echo $? >&3) | sed -r -e"s,^(PASS|SKIP|FAIL)$,\1:
> >> $1/$2," >&4) 3>&1) | (read rc; exit $rc)) 4>&1
> >> exit $?)
> >> run_test github.com/gola
