Please specify '-DskipTests' on commandline.
Cheers
On Dec 5, 2014, at 3:52 AM, Emre Sevinc emre.sev...@gmail.com wrote:
Hello,
I'm currently developing a Spark Streaming application and trying to write my
first unit test. I've used Java for this application, and I also need use
Java (and JUnit) for writing unit tests.
I could not find any documentation that focuses on Spark Streaming unit
testing, all I could find was the Java based unit tests in Spark Streaming
source code:
https://github.com/apache/spark/blob/branch-1.1/streaming/src/test/java/org/apache/spark/streaming/JavaAPISuite.java
that depends on a Scala file:
https://github.com/apache/spark/blob/branch-1.1/streaming/src/test/java/org/apache/spark/streaming/JavaTestUtils.scala
which, in turn, depends on the Scala test files in
https://github.com/apache/spark/tree/branch-1.1/streaming/src/test/scala/org/apache/spark/streaming
So I thought that I could grab the Spark source code, switch to branch-1.1
branch and then only compile 'core' and 'streaming' modules, hopefully ending
up with the compiled classes (or jar files) of the Streaming test utilities,
so that I can import them in my Java based Spark Streaming application.
However, trying to build it via the following command line failed:
mvn -pl core,streaming package
You can see the full output at the end of this message.
Any ideas how to progress?
Full output of the build:
emre@emre-ubuntu:~/code/spark$ mvn -pl core,streaming package
[INFO] Scanning for projects...
[INFO]
[INFO] Reactor Build Order:
[INFO]
[INFO] Spark Project Core
[INFO] Spark Project Streaming
[INFO]
[INFO]
[INFO] Building Spark Project Core 1.1.2-SNAPSHOT
[INFO]
Downloading:
https://repo1.maven.org/maven2/org/apache/maven/plugins/maven-antrun-plugin/1.7/maven-antrun-plugin-1.7.pom
Downloaded:
https://repo1.maven.org/maven2/org/apache/maven/plugins/maven-antrun-plugin/1.7/maven-antrun-plugin-1.7.pom
(5 KB at 5.8 KB/sec)
Downloading:
https://repo1.maven.org/maven2/org/apache/maven/plugins/maven-antrun-plugin/1.7/maven-antrun-plugin-1.7.jar
Downloaded:
https://repo1.maven.org/maven2/org/apache/maven/plugins/maven-antrun-plugin/1.7/maven-antrun-plugin-1.7.jar
(31 KB at 200.4 KB/sec)
Downloading:
https://repo1.maven.org/maven2/org/apache/commons/commons-math3/3.3/commons-math3-3.3.pom
Downloaded:
https://repo1.maven.org/maven2/org/apache/commons/commons-math3/3.3/commons-math3-3.3.pom
(24 KB at 178.9 KB/sec)
Downloading:
https://repo1.maven.org/maven2/org/spark-project/akka/akka-testkit_2.10/2.2.3-shaded-protobuf/akka-testkit_2.10-2.2.3-shaded-protobuf.pom
Downloaded:
https://repo1.maven.org/maven2/org/spark-project/akka/akka-testkit_2.10/2.2.3-shaded-protobuf/akka-testkit_2.10-2.2.3-shaded-protobuf.pom
(3 KB at 22.5 KB/sec)
Downloading:
https://repo1.maven.org/maven2/org/scala-lang/scalap/2.10.4/scalap-2.10.4.pom
Downloaded:
https://repo1.maven.org/maven2/org/scala-lang/scalap/2.10.4/scalap-2.10.4.pom
(2 KB at 19.2 KB/sec)
Downloading:
https://repo1.maven.org/maven2/org/apache/derby/derby/10.4.2.0/derby-10.4.2.0.pom
Downloaded:
https://repo1.maven.org/maven2/org/apache/derby/derby/10.4.2.0/derby-10.4.2.0.pom
(2 KB at 14.9 KB/sec)
Downloading:
https://repo1.maven.org/maven2/org/mockito/mockito-all/1.9.0/mockito-all-1.9.0.pom
Downloaded:
https://repo1.maven.org/maven2/org/mockito/mockito-all/1.9.0/mockito-all-1.9.0.pom
(1010 B at 4.1 KB/sec)
Downloading:
https://repo1.maven.org/maven2/org/easymock/easymockclassextension/3.1/easymockclassextension-3.1.pom
Downloaded:
https://repo1.maven.org/maven2/org/easymock/easymockclassextension/3.1/easymockclassextension-3.1.pom
(5 KB at 42.9 KB/sec)
Downloading:
https://repo1.maven.org/maven2/org/easymock/easymock-parent/3.1/easymock-parent-3.1.pom
Downloaded:
https://repo1.maven.org/maven2/org/easymock/easymock-parent/3.1/easymock-parent-3.1.pom
(13 KB at 133.8 KB/sec)
Downloading:
https://repo1.maven.org/maven2/org/easymock/easymock/3.1/easymock-3.1.pom
Downloaded:
https://repo1.maven.org/maven2/org/easymock/easymock/3.1/easymock-3.1.pom (6
KB at 38.8 KB/sec)
Downloading:
https://repo1.maven.org/maven2/cglib/cglib-nodep/2.2.2/cglib-nodep-2.2.2.pom
Downloaded:
https://repo1.maven.org/maven2/cglib/cglib-nodep/2.2.2/cglib-nodep-2.2.2.pom
(2 KB at 9.9 KB/sec)
Downloading:
https://repo1.maven.org/maven2/org/apache/commons/commons-math3/3.3/commons-math3-3.3.jar
Downloading:
https://repo1.maven.org/maven2/org/spark-project/akka/akka-testkit_2.10/2.2.3-shaded-protobuf/akka-testkit_2.10-2.2.3-shaded-protobuf.jar
