Bug#872935: FTBFS: Could not locate puppetlabs/kitchensink/testutils__init.class or puppetlabs/kitchensink/testutils.clj on classpath.

[Cyril Brulebois]

Control: tag -1 patch

Cyril Brulebois  (2018-12-30):
> I'll propose the same kind of patch as the other ones through salsa's
> MR system, as requested/suggested by maintainers. :)

There you go:
  https://salsa.debian.org/java-team/kitchensink-clojure/merge_requests/1

(You'll also find it attached.)
  

Cheers,
-- 
Cyril Brulebois (k...@debian.org)
D-I release manager -- Release team member -- Freelance Consultant
From 95edcea66dcd114592c4dcc67c806939bab68a9e Mon Sep 17 00:00:00 2001
From: Cyril Brulebois 
Date: Sun, 30 Dec 2018 22:21:52 +
Subject: [PATCH] Fix FTBFS by running tests in a specific order (Closes:
 #872935).

Instead of blindly sorting the results of the find command and trusting
it to be right, encode a suitable sequence of tests. Also make sure to
error out when tests are added or removed, to make sure this sequence
get refreshed when needed.
---
 debian/changelog | 10 ++
 debian/rules | 17 ++---
 2 files changed, 24 insertions(+), 3 deletions(-)

diff --git a/debian/changelog b/debian/changelog
index c123bad..927e644 100644
--- a/debian/changelog
+++ b/debian/changelog
@@ -1,3 +1,13 @@
+kitchensink-clojure (2.3.0-2) UNRELEASED; urgency=medium
+
+  * Fix FTBFS by running tests in a specific order. Instead of blindly
+sorting the results of the find command and trusting it to be right,
+encode a suitable sequence of tests (Closes: #872935). Also make sure
+to error out when tests are added or removed, to make sure this
+sequence get refreshed when needed.
+
+ -- Cyril Brulebois   Sun, 30 Dec 2018 22:18:30 +
+
 kitchensink-clojure (2.3.0-1) unstable; urgency=medium
 
   * Initial release (Closes: #855739)
diff --git a/debian/rules b/debian/rules
index 3711198..77fc244 100755
--- a/debian/rules
+++ b/debian/rules
@@ -26,10 +26,21 @@ override_jh_clean:
 override_jh_classpath:
 	jh_classpath $(PRODUCED_JAR)
 
+# Tests can't run in a random order (#872935):
+override_dh_auto_test: ORDERED_TESTS=puppetlabs/kitchensink/classpath_test.clj puppetlabs/kitchensink/core_test.clj puppetlabs/kitchensink/json_test.clj
+override_dh_auto_test: EXPECTED=$(shell echo "$(ORDERED_TESTS)" | sed 's/ /\n/g' | sort | xargs)
+override_dh_auto_test: FOUND=$(shell cd test && find -name '*_test.clj' | sed 's,^[.]/,,' | sort | xargs)
 override_dh_auto_test:
-	dh_auto_test
-	(cd test && find . -name "*_test.clj" | \
-		xargs --verbose clojure -cp $(CURDIR)/$(PRODUCED_JAR):$(CLASSPATH))
+	# Make sure to detect when tests appear or disappear:
+	@if [ "$(FOUND)" != "$(EXPECTED)" ]; then \
+		echo "E: sanity check failed, available tests don't match known tests:"; \
+		echo "- found:$(FOUND)"; \
+		echo "- expected: $(EXPECTED)"; \
+		echo "=> Please update ORDERED_TESTS in debian/rules accordingly"; \
+		exit 1; \
+	fi
+	# Run tests:
+	(cd test && clojure -cp $(CURDIR)/$(PRODUCED_JAR):$(CLASSPATH):$(TEST_CLASSPATH) $(ORDERED_TESTS))
 
 get-orig-source:
 	uscan --download-version $(DEB_VERSION_UPSTREAM) --force-download --rename
-- 
2.20.1



signature.asc
Description: PGP signature

Bug#872935: FTBFS: Could not locate puppetlabs/kitchensink/testutils__init.class or puppetlabs/kitchensink/testutils.clj on classpath.

[Cyril Brulebois]

Control: severity -1 serious

(Looping submitter back in.)

Santiago Vila  (2018-12-30):
> On Tue, 22 Aug 2017, Hans Joachim Desserud wrote:
> 
> > Source: kitchensink-clojure
> > Version: 2.3.0-1
> > Severity: important
> > 
> > Dear Maintainer,
> > 
> > kitchensink-clojure currently fails to build from source with the
> > following error message:
> > 
> > dh_auto_test
> > (cd test && find . -name "*_test.clj" | \
> > xargs --verbose clojure -cp
> > /build/kitchensink-clojure-2.3.0/kitchensink.jar:/usr/share/java/clojure.jar:/usr/share/java/tools.cli.jar:/usr/share/java/tools.logging.jar:/usr/share/java/clj-time.jar:/usr/share/java/raynes-fs.jar:/usr/share/java/slingshot.jar:/usr/share/java/cheshire.jar:/usr/share/java/ini4j.jar:/usr/share/java/dynapath.jar:/usr/share/java/clj-digest.jar)
> > clojure -cp
> > /build/kitchensink-clojure-2.3.0/kitchensink.jar:/usr/share/java/clojure.jar:/usr/share/java/tools.cli.jar:/usr/share/java/tools.logging.jar:/usr/share/java/clj-time.jar:/usr/share/java/raynes-fs.jar:/usr/share/java/slingshot.jar:/usr/share/java/cheshire.jar:/usr/share/java/ini4j.jar:/usr/share/java/dynapath.jar:/usr/share/java/clj-digest.jar
> > ./puppetlabs/kitchensink/core_test.clj 
> > ./puppetlabs/kitchensink/json_test.clj
> > ./puppetlabs/kitchensink/classpath_test.clj
> > Exception in thread "main" java.io.FileNotFoundException: Could not locate
> > puppetlabs/kitchensink/testutils__init.class or
> > puppetlabs/kitchensink/testutils.clj on classpath.,
> > compiling:(/build/kitchensink-clojure-2.3.0/test/./puppetlabs/kitchensink/core_test.clj:1:1)
> > at clojure.lang.Compiler.load(Compiler.java:7391)
> > at clojure.lang.Compiler.loadFile(Compiler.java:7317)
> > at clojure.main$load_script.invokeStatic(main.clj:275)
> > at clojure.main$load_script.invoke(main.clj:268)
> > at clojure.main$script_opt.invokeStatic(main.clj:337)
> > at clojure.main$script_opt.invoke(main.clj:330)
> > at clojure.main$main.invokeStatic(main.clj:421)
> > at clojure.main$main.doInvoke(main.clj:384)
> > at clojure.lang.RestFn.invoke(RestFn.java:436)
> > at clojure.lang.Var.invoke(Var.java:388)
> > at clojure.lang.AFn.applyToHelper(AFn.java:160)
> > at clojure.lang.Var.applyTo(Var.java:700)
> > at clojure.main.main(main.java:37)
> > Caused by: java.io.FileNotFoundException: Could not locate
> > puppetlabs/kitchensink/testutils__init.class or
> > puppetlabs/kitchensink/testutils.clj on classpath.
> 
> This seems to be the same problem which happens in honeysql-clojure or 
> ssl-utils-clojure.
> 
> There is a failed build log in reproducible-builds on armhf:
> 
> https://tests.reproducible-builds.org/debian/logs/buster/armhf/kitchensink-clojure_2.3.0-1.build2.log.gz
> 
> and I can reproduce it nearly 100% of the time on 1-XS instances (amd64) from 
> Scaleway.

You'll find attached a minimal patch that makes the issue go away by
sorting the tests before running them.

> I hope Cyril can team-upload this one as well.

I'll propose the same kind of patch as the other ones through salsa's MR
system, as requested/suggested by maintainers. :)


Cheers,
-- 
Cyril Brulebois (k...@debian.org)
D-I release manager -- Release team member -- Freelance Consultant
diff --git a/debian/rules b/debian/rules
index 3711198..4cdd942 100755
--- a/debian/rules
+++ b/debian/rules
@@ -28,7 +28,7 @@ override_jh_classpath:
 
 override_dh_auto_test:
 	dh_auto_test
-	(cd test && find . -name "*_test.clj" | \
+	(cd test && find . -name "*_test.clj" | LC_ALL=C sort | \
 		xargs --verbose clojure -cp $(CURDIR)/$(PRODUCED_JAR):$(CLASSPATH))
 
 get-orig-source:


signature.asc
Description: PGP signature

Bug#872935: FTBFS: Could not locate puppetlabs/kitchensink/testutils__init.class or puppetlabs/kitchensink/testutils.clj on classpath.

[Santiago Vila]

On Tue, 22 Aug 2017, Hans Joachim Desserud wrote:

> Source: kitchensink-clojure
> Version: 2.3.0-1
> Severity: important
> 
> Dear Maintainer,
> 
> kitchensink-clojure currently fails to build from source with the
> following error message:
> 
> dh_auto_test
> (cd test && find . -name "*_test.clj" | \
>   xargs --verbose clojure -cp
> /build/kitchensink-clojure-2.3.0/kitchensink.jar:/usr/share/java/clojure.jar:/usr/share/java/tools.cli.jar:/usr/share/java/tools.logging.jar:/usr/share/java/clj-time.jar:/usr/share/java/raynes-fs.jar:/usr/share/java/slingshot.jar:/usr/share/java/cheshire.jar:/usr/share/java/ini4j.jar:/usr/share/java/dynapath.jar:/usr/share/java/clj-digest.jar)
> clojure -cp
> /build/kitchensink-clojure-2.3.0/kitchensink.jar:/usr/share/java/clojure.jar:/usr/share/java/tools.cli.jar:/usr/share/java/tools.logging.jar:/usr/share/java/clj-time.jar:/usr/share/java/raynes-fs.jar:/usr/share/java/slingshot.jar:/usr/share/java/cheshire.jar:/usr/share/java/ini4j.jar:/usr/share/java/dynapath.jar:/usr/share/java/clj-digest.jar
> ./puppetlabs/kitchensink/core_test.clj ./puppetlabs/kitchensink/json_test.clj
> ./puppetlabs/kitchensink/classpath_test.clj
> Exception in thread "main" java.io.FileNotFoundException: Could not locate
> puppetlabs/kitchensink/testutils__init.class or
> puppetlabs/kitchensink/testutils.clj on classpath.,
> compiling:(/build/kitchensink-clojure-2.3.0/test/./puppetlabs/kitchensink/core_test.clj:1:1)
>   at clojure.lang.Compiler.load(Compiler.java:7391)
>   at clojure.lang.Compiler.loadFile(Compiler.java:7317)
>   at clojure.main$load_script.invokeStatic(main.clj:275)
>   at clojure.main$load_script.invoke(main.clj:268)
>   at clojure.main$script_opt.invokeStatic(main.clj:337)
>   at clojure.main$script_opt.invoke(main.clj:330)
>   at clojure.main$main.invokeStatic(main.clj:421)
>   at clojure.main$main.doInvoke(main.clj:384)
>   at clojure.lang.RestFn.invoke(RestFn.java:436)
>   at clojure.lang.Var.invoke(Var.java:388)
>   at clojure.lang.AFn.applyToHelper(AFn.java:160)
>   at clojure.lang.Var.applyTo(Var.java:700)
>   at clojure.main.main(main.java:37)
> Caused by: java.io.FileNotFoundException: Could not locate
> puppetlabs/kitchensink/testutils__init.class or
> puppetlabs/kitchensink/testutils.clj on classpath.

This seems to be the same problem which happens in honeysql-clojure or 
ssl-utils-clojure.

There is a failed build log in reproducible-builds on armhf:

https://tests.reproducible-builds.org/debian/logs/buster/armhf/kitchensink-clojure_2.3.0-1.build2.log.gz

and I can reproduce it nearly 100% of the time on 1-XS instances (amd64) from 
Scaleway.

I hope Cyril can team-upload this one as well.

Thanks.

Bug#872935: FTBFS: Could not locate puppetlabs/kitchensink/testutils__init.class or puppetlabs/kitchensink/testutils.clj on classpath.

[Hans Joachim Desserud]

Source: kitchensink-clojure
Version: 2.3.0-1
Severity: important

Dear Maintainer,

kitchensink-clojure currently fails to build from source with the
following error message:

dh_auto_test
(cd test && find . -name "*_test.clj" | \
	xargs --verbose clojure -cp 
/build/kitchensink-clojure-2.3.0/kitchensink.jar:/usr/share/java/clojure.jar:/usr/share/java/tools.cli.jar:/usr/share/java/tools.logging.jar:/usr/share/java/clj-time.jar:/usr/share/java/raynes-fs.jar:/usr/share/java/slingshot.jar:/usr/share/java/cheshire.jar:/usr/share/java/ini4j.jar:/usr/share/java/dynapath.jar:/usr/share/java/clj-digest.jar)
clojure -cp 
/build/kitchensink-clojure-2.3.0/kitchensink.jar:/usr/share/java/clojure.jar:/usr/share/java/tools.cli.jar:/usr/share/java/tools.logging.jar:/usr/share/java/clj-time.jar:/usr/share/java/raynes-fs.jar:/usr/share/java/slingshot.jar:/usr/share/java/cheshire.jar:/usr/share/java/ini4j.jar:/usr/share/java/dynapath.jar:/usr/share/java/clj-digest.jar 
./puppetlabs/kitchensink/core_test.clj 
./puppetlabs/kitchensink/json_test.clj 
./puppetlabs/kitchensink/classpath_test.clj
Exception in thread "main" java.io.FileNotFoundException: Could not 
locate puppetlabs/kitchensink/testutils__init.class or 
puppetlabs/kitchensink/testutils.clj on classpath., 
compiling:(/build/kitchensink-clojure-2.3.0/test/./puppetlabs/kitchensink/core_test.clj:1:1)

at clojure.lang.Compiler.load(Compiler.java:7391)
at clojure.lang.Compiler.loadFile(Compiler.java:7317)
at clojure.main$load_script.invokeStatic(main.clj:275)
at clojure.main$load_script.invoke(main.clj:268)
at clojure.main$script_opt.invokeStatic(main.clj:337)
at clojure.main$script_opt.invoke(main.clj:330)
at clojure.main$main.invokeStatic(main.clj:421)
at clojure.main$main.doInvoke(main.clj:384)
at clojure.lang.RestFn.invoke(RestFn.java:436)
at clojure.lang.Var.invoke(Var.java:388)
at clojure.lang.AFn.applyToHelper(AFn.java:160)
at clojure.lang.Var.applyTo(Var.java:700)
at clojure.main.main(main.java:37)
Caused by: java.io.FileNotFoundException: Could not locate 
puppetlabs/kitchensink/testutils__init.class or 
puppetlabs/kitchensink/testutils.clj on classpath.

at clojure.lang.RT.load(RT.java:456)
at clojure.lang.RT.load(RT.java:419)
at clojure.core$load$fn__1621.invoke(core.clj:5893)
at clojure.core$load.invokeStatic(core.clj:5892)
at clojure.core$load.doInvoke(core.clj:5876)
at clojure.lang.RestFn.invoke(RestFn.java:408)
at clojure.core$load_one.invokeStatic(core.clj:5697)
at clojure.core$load_one.invoke(core.clj:5692)
at clojure.core$load_lib$fn__1570.invoke(core.clj:5737)
at clojure.core$load_lib.invokeStatic(core.clj:5736)
at clojure.core$load_lib.doInvoke(core.clj:5717)
at clojure.lang.RestFn.applyTo(RestFn.java:142)
at clojure.core$apply.invokeStatic(core.clj:648)
at clojure.core$apply.invoke(core.clj:641)
at clojure.core$load_libs.invokeStatic(core.clj:5775)
at clojure.core$load_libs.doInvoke(core.clj:5758)
at clojure.lang.RestFn.applyTo(RestFn.java:137)
at clojure.core$apply.invokeStatic(core.clj:648)
at clojure.core$apply.invoke(core.clj:641)
at clojure.core$require.invokeStatic(core.clj:5858)
at clojure.core$require.doInvoke(core.clj:5796)
at clojure.lang.RestFn.invoke(RestFn.java:619)
	at 
puppetlabs.kitchensink.core_test$eval4329$loading__1497__auto4330.invoke(core_test.clj:1)
	at 
puppetlabs.kitchensink.core_test$eval4329.invokeStatic(core_test.clj:1)

at puppetlabs.kitchensink.core_test$eval4329.invoke(core_test.clj:1)
at clojure.lang.Compiler.eval(Compiler.java:6927)
at clojure.lang.Compiler.eval(Compiler.java:6916)
at clojure.lang.Compiler.load(Compiler.java:7379)
... 12 more
debian/rules:30: recipe for target 'override_dh_auto_test' failed
make[1]: *** [override_dh_auto_test] Error 123
make[1]: Leaving directory '/build/kitchensink-clojure-2.3.0'
debian/rules:14: recipe for target 'build' failed
make: *** [build] Error 2
dpkg-buildpackage: error: debian/rules build gave error exit status 2

(I originally spotted this when the package was automatically synced
to Ubuntu, but was able to reproduce it in a pbuilder environment
on my Debian sid system)


-- System Information:
Debian Release: buster/sid
  APT prefers unstable
  APT policy: (500, 'unstable')
Architecture: amd64 (x86_64)

Kernel: Linux 4.12.0-1-amd64 (SMP w/1 CPU core)
Locale: LANG=en_US.utf8, LC_CTYPE=en_US.utf8 (charmap=UTF-8), 
LANGUAGE=en_US:en (charmap=UTF-8)

Shell: /bin/sh linked to /bin/dash
Init: systemd (via /run/systemd/system)

--
mvh / best regards
Hans Joachim Desserud
http://desserud.org