Re: run JavaAPISuite with mavem
Running JavaAPISuite (master branch) on Linux, I got: testGuavaOptional(org.apache.spark.JavaAPISuite) Time elapsed: 32.945 sec ERROR! org.apache.spark.SparkException: Job aborted due to stage failure: Master removed our application: FAILED at org.apache.spark.scheduler.DAGScheduler.org $apache$spark$scheduler$DAGScheduler$$failJobAndIndependentStages(DAGScheduler.scala:1214) at org.apache.spark.scheduler.DAGScheduler$$anonfun$abortStage$1.apply(DAGScheduler.scala:1203) at org.apache.spark.scheduler.DAGScheduler$$anonfun$abortStage$1.apply(DAGScheduler.scala:1202) at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59) at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:47) at org.apache.spark.scheduler.DAGScheduler.abortStage(DAGScheduler.scala:1202) at org.apache.spark.scheduler.DAGScheduler$$anonfun$handleTaskSetFailed$1.apply(DAGScheduler.scala:696) at org.apache.spark.scheduler.DAGScheduler$$anonfun$handleTaskSetFailed$1.apply(DAGScheduler.scala:696) at scala.Option.foreach(Option.scala:236) at org.apache.spark.scheduler.DAGScheduler.handleTaskSetFailed(DAGScheduler.scala:696) at org.apache.spark.scheduler.DAGSchedulerEventProcessActor$$anonfun$receive$2.applyOrElse(DAGScheduler.scala:1420) at akka.actor.Actor$class.aroundReceive(Actor.scala:465) at org.apache.spark.scheduler.DAGSchedulerEventProcessActor.aroundReceive(DAGScheduler.scala:1375) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:516) at akka.actor.ActorCell.invoke(ActorCell.scala:487) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:238) at akka.dispatch.Mailbox.run(Mailbox.scala:220) at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:393) at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260) at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339) at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979) at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107) FYI On Sun, Dec 7, 2014 at 3:25 PM, Sean Owen so...@cloudera.com wrote: I think it's a known issue: https://issues.apache.org/jira/browse/SPARK-4159 https://issues.apache.org/jira/browse/SPARK-661 I got bit by this too recently and meant to look into it. On Sun, Dec 7, 2014 at 4:50 PM, Koert Kuipers ko...@tresata.com wrote: so as part of the official build the java api does not get tested then? i am sure there is a good reason for it, but thats surprising to me. On Sun, Dec 7, 2014 at 12:19 PM, Ted Yu yuzhih...@gmail.com wrote: Looking at the pom.xml, I think I found the reason - scalatest is used. With the following diff: diff --git a/pom.xml b/pom.xml index b7df53d..b0da893 100644 --- a/pom.xml +++ b/pom.xml @@ -947,7 +947,7 @@ version2.17/version configuration !-- Uses scalatest instead -- -skipTeststrue/skipTests +skipTestsfalse/skipTests /configuration /plugin plugin I was able to run JavaAPISuite using: mvn test -pl core -Dtest=JavaAPISuite But it takes a long time ... Cheers On Sun, Dec 7, 2014 at 8:56 AM, Koert Kuipers ko...@tresata.com wrote: hey guys, i was able to run the test just fine with: $ sbt project core testOnly org.apache.spark.JavaAPISuite however i found it strange that it didnt run when i do mvn test -pl core, or at least didnt seem like it ran to me. this would mean that when someone tests/publishes with maven the japa api does not get tested at all. On Sun, Dec 7, 2014 at 1:52 AM, Michael Armbrust mich...@databricks.com wrote: Not sure about maven, but you can run that test with sbt: sbt/sbt sql/test-only org.apache.spark.sql.api.java.JavaAPISuite On Sat, Dec 6, 2014 at 9:59 PM, Ted Yu yuzhih...@gmail.com wrote: I tried to run tests for core but there were failures. e.g. : ^[[32mExternalAppendOnlyMapSuite:^[[0m ^[[32m- simple insert^[[0m ^[[32m- insert with collision^[[0m ^[[32m- ordering^[[0m ^[[32m- null keys and values^[[0m ^[[32m- simple aggregator^[[0m ^[[32m- simple cogroup^[[0m Spark assembly has been built with Hive, including Datanucleus jars on classpath ^[[31m- spilling *** FAILED ***^[[0m ^[[31m org.apache.spark.SparkException: Job aborted due to stage failure: Task 0 in stage 0.0 failed 4 times, most recent failure: Lost task 0.3 in stage 0.0 (TID 6, localhost): java.lang.ClassNotFoundException: org.apache.spark.rdd.RDD$$anonfun$map$1^[[0m ^[[31m at java.net.URLClassLoader$1.run(URLClassLoader.java:366)^[[0m ^[[31m at java.net.URLClassLoader$1.run(URLClassLoader.java:355)^[[0m ^[[31m at java.security.AccessController.doPrivileged(Native Method)^[[0m ^[[31m at java.net.URLClassLoader.findClass(URLClassLoader.java:354)^[[0m ^[[31m at java.lang.ClassLoader.loadClass(ClassLoader.java:424)^[[0m ^[[31m
Re: run JavaAPISuite with mavem
hey guys,
i was able to run the test just fine with:
$ sbt
project core
testOnly org.apache.spark.JavaAPISuite
however i found it strange that it didnt run when i do mvn test -pl core,
or at least didnt seem like it ran to me. this would mean that when someone
tests/publishes with maven the japa api does not get tested at all.
On Sun, Dec 7, 2014 at 1:52 AM, Michael Armbrust mich...@databricks.com
wrote:
Not sure about maven, but you can run that test with sbt:
sbt/sbt sql/test-only org.apache.spark.sql.api.java.JavaAPISuite
On Sat, Dec 6, 2014 at 9:59 PM, Ted Yu yuzhih...@gmail.com wrote:
I tried to run tests for core but there were failures. e.g. :
^[[32mExternalAppendOnlyMapSuite:^[[0m
^[[32m- simple insert^[[0m
^[[32m- insert with collision^[[0m
^[[32m- ordering^[[0m
^[[32m- null keys and values^[[0m
^[[32m- simple aggregator^[[0m
^[[32m- simple cogroup^[[0m
Spark assembly has been built with Hive, including Datanucleus jars on
classpath
^[[31m- spilling *** FAILED ***^[[0m
^[[31m org.apache.spark.SparkException: Job aborted due to stage
failure: Task 0 in stage 0.0 failed 4 times, most recent failure: Lost task
0.3 in stage 0.0 (TID 6, localhost): java.lang.ClassNotFoundException:
org.apache.spark.rdd.RDD$$anonfun$map$1^[[0m
^[[31m at java.net.URLClassLoader$1.run(URLClassLoader.java:366)^[[0m
^[[31m at java.net.URLClassLoader$1.run(URLClassLoader.java:355)^[[0m
^[[31m at java.security.AccessController.doPrivileged(Native Method)^[[0m
^[[31m at java.net.URLClassLoader.findClass(URLClassLoader.java:354)^[[0m
^[[31m at java.lang.ClassLoader.loadClass(ClassLoader.java:424)^[[0m
^[[31m at java.lang.ClassLoader.loadClass(ClassLoader.java:357)^[[0m
^[[31m at java.lang.Class.forName0(Native Method)^[[0m
^[[31m at java.lang.Class.forName(Class.java:270)^[[0m
BTW I didn't find JavaAPISuite in test output either.
Cheers
On Sat, Dec 6, 2014 at 9:12 PM, Koert Kuipers ko...@tresata.com wrote:
Ted,
i mean
core/src/test/java/org/apache/spark/JavaAPISuite.java
On Sat, Dec 6, 2014 at 9:27 PM, Ted Yu yuzhih...@gmail.com wrote:
Pardon me, the test is here:
sql/core/src/test/java/org/apache/spark/sql/api/java/JavaAPISuite.java
You can run 'mvn test' under sql/core
Cheers
On Sat, Dec 6, 2014 at 5:55 PM, Ted Yu yuzhih...@gmail.com wrote:
In master branch, I only found JavaAPISuite in comment:
spark tyu$ find . -name '*.scala' -exec grep JavaAPISuite {} \; -print
* For usage example, see test case JavaAPISuite.testJavaJdbcRDD.
* converted into a `Object` array. For usage example, see test case
JavaAPISuite.testJavaJdbcRDD.
./core/src/main/scala/org/apache/spark/rdd/JdbcRDD.scala
FYI
On Sat, Dec 6, 2014 at 5:43 PM, Koert Kuipers ko...@tresata.com
wrote:
when i run mvn test -pl core, i dont see JavaAPISuite being run. or
if it is, its being very very quiet about it. is this by design?
Re: run JavaAPISuite with mavem
Looking at the pom.xml, I think I found the reason - scalatest is used.
With the following diff:
diff --git a/pom.xml b/pom.xml
index b7df53d..b0da893 100644
--- a/pom.xml
+++ b/pom.xml
@@ -947,7 +947,7 @@
version2.17/version
configuration
!-- Uses scalatest instead --
-skipTeststrue/skipTests
+skipTestsfalse/skipTests
/configuration
/plugin
plugin
I was able to run JavaAPISuite using:
mvn test -pl core -Dtest=JavaAPISuite
But it takes a long time ...
Cheers
On Sun, Dec 7, 2014 at 8:56 AM, Koert Kuipers ko...@tresata.com wrote:
hey guys,
i was able to run the test just fine with:
$ sbt
project core
testOnly org.apache.spark.JavaAPISuite
however i found it strange that it didnt run when i do mvn test -pl
core, or at least didnt seem like it ran to me. this would mean that when
someone tests/publishes with maven the japa api does not get tested at all.
On Sun, Dec 7, 2014 at 1:52 AM, Michael Armbrust mich...@databricks.com
wrote:
Not sure about maven, but you can run that test with sbt:
sbt/sbt sql/test-only org.apache.spark.sql.api.java.JavaAPISuite
On Sat, Dec 6, 2014 at 9:59 PM, Ted Yu yuzhih...@gmail.com wrote:
I tried to run tests for core but there were failures. e.g. :
^[[32mExternalAppendOnlyMapSuite:^[[0m
^[[32m- simple insert^[[0m
^[[32m- insert with collision^[[0m
^[[32m- ordering^[[0m
^[[32m- null keys and values^[[0m
^[[32m- simple aggregator^[[0m
^[[32m- simple cogroup^[[0m
Spark assembly has been built with Hive, including Datanucleus jars on
classpath
^[[31m- spilling *** FAILED ***^[[0m
^[[31m org.apache.spark.SparkException: Job aborted due to stage
failure: Task 0 in stage 0.0 failed 4 times, most recent failure: Lost task
0.3 in stage 0.0 (TID 6, localhost): java.lang.ClassNotFoundException:
org.apache.spark.rdd.RDD$$anonfun$map$1^[[0m
^[[31m at java.net.URLClassLoader$1.run(URLClassLoader.java:366)^[[0m
^[[31m at java.net.URLClassLoader$1.run(URLClassLoader.java:355)^[[0m
^[[31m at java.security.AccessController.doPrivileged(Native
Method)^[[0m
^[[31m at
java.net.URLClassLoader.findClass(URLClassLoader.java:354)^[[0m
^[[31m at java.lang.ClassLoader.loadClass(ClassLoader.java:424)^[[0m
^[[31m at java.lang.ClassLoader.loadClass(ClassLoader.java:357)^[[0m
^[[31m at java.lang.Class.forName0(Native Method)^[[0m
^[[31m at java.lang.Class.forName(Class.java:270)^[[0m
BTW I didn't find JavaAPISuite in test output either.
Cheers
On Sat, Dec 6, 2014 at 9:12 PM, Koert Kuipers ko...@tresata.com wrote:
Ted,
i mean
core/src/test/java/org/apache/spark/JavaAPISuite.java
On Sat, Dec 6, 2014 at 9:27 PM, Ted Yu yuzhih...@gmail.com wrote:
Pardon me, the test is here:
sql/core/src/test/java/org/apache/spark/sql/api/java/JavaAPISuite.java
You can run 'mvn test' under sql/core
Cheers
On Sat, Dec 6, 2014 at 5:55 PM, Ted Yu yuzhih...@gmail.com wrote:
In master branch, I only found JavaAPISuite in comment:
spark tyu$ find . -name '*.scala' -exec grep JavaAPISuite {} \; -print
* For usage example, see test case JavaAPISuite.testJavaJdbcRDD.
* converted into a `Object` array. For usage example, see test
case JavaAPISuite.testJavaJdbcRDD.
./core/src/main/scala/org/apache/spark/rdd/JdbcRDD.scala
FYI
On Sat, Dec 6, 2014 at 5:43 PM, Koert Kuipers ko...@tresata.com
wrote:
when i run mvn test -pl core, i dont see JavaAPISuite being run.
or if it is, its being very very quiet about it. is this by design?
Re: run JavaAPISuite with mavem
so as part of the official build the java api does not get tested then?
i am sure there is a good reason for it, but thats surprising to me.
On Sun, Dec 7, 2014 at 12:19 PM, Ted Yu yuzhih...@gmail.com wrote:
Looking at the pom.xml, I think I found the reason - scalatest is used.
With the following diff:
diff --git a/pom.xml b/pom.xml
index b7df53d..b0da893 100644
--- a/pom.xml
+++ b/pom.xml
@@ -947,7 +947,7 @@
version2.17/version
configuration
!-- Uses scalatest instead --
-skipTeststrue/skipTests
+skipTestsfalse/skipTests
/configuration
/plugin
plugin
I was able to run JavaAPISuite using:
mvn test -pl core -Dtest=JavaAPISuite
But it takes a long time ...
Cheers
On Sun, Dec 7, 2014 at 8:56 AM, Koert Kuipers ko...@tresata.com wrote:
hey guys,
i was able to run the test just fine with:
$ sbt
project core
testOnly org.apache.spark.JavaAPISuite
however i found it strange that it didnt run when i do mvn test -pl
core, or at least didnt seem like it ran to me. this would mean that when
someone tests/publishes with maven the japa api does not get tested at all.
On Sun, Dec 7, 2014 at 1:52 AM, Michael Armbrust mich...@databricks.com
wrote:
Not sure about maven, but you can run that test with sbt:
sbt/sbt sql/test-only org.apache.spark.sql.api.java.JavaAPISuite
On Sat, Dec 6, 2014 at 9:59 PM, Ted Yu yuzhih...@gmail.com wrote:
I tried to run tests for core but there were failures. e.g. :
^[[32mExternalAppendOnlyMapSuite:^[[0m
^[[32m- simple insert^[[0m
^[[32m- insert with collision^[[0m
^[[32m- ordering^[[0m
^[[32m- null keys and values^[[0m
^[[32m- simple aggregator^[[0m
^[[32m- simple cogroup^[[0m
Spark assembly has been built with Hive, including Datanucleus jars on
classpath
^[[31m- spilling *** FAILED ***^[[0m
^[[31m org.apache.spark.SparkException: Job aborted due to stage
failure: Task 0 in stage 0.0 failed 4 times, most recent failure: Lost task
0.3 in stage 0.0 (TID 6, localhost): java.lang.ClassNotFoundException:
org.apache.spark.rdd.RDD$$anonfun$map$1^[[0m
^[[31m at java.net.URLClassLoader$1.run(URLClassLoader.java:366)^[[0m
^[[31m at java.net.URLClassLoader$1.run(URLClassLoader.java:355)^[[0m
^[[31m at java.security.AccessController.doPrivileged(Native
Method)^[[0m
^[[31m at
java.net.URLClassLoader.findClass(URLClassLoader.java:354)^[[0m
^[[31m at java.lang.ClassLoader.loadClass(ClassLoader.java:424)^[[0m
^[[31m at java.lang.ClassLoader.loadClass(ClassLoader.java:357)^[[0m
^[[31m at java.lang.Class.forName0(Native Method)^[[0m
^[[31m at java.lang.Class.forName(Class.java:270)^[[0m
BTW I didn't find JavaAPISuite in test output either.
Cheers
On Sat, Dec 6, 2014 at 9:12 PM, Koert Kuipers ko...@tresata.com
wrote:
Ted,
i mean
core/src/test/java/org/apache/spark/JavaAPISuite.java
On Sat, Dec 6, 2014 at 9:27 PM, Ted Yu yuzhih...@gmail.com wrote:
Pardon me, the test is here:
sql/core/src/test/java/org/apache/spark/sql/api/java/JavaAPISuite.java
You can run 'mvn test' under sql/core
Cheers
On Sat, Dec 6, 2014 at 5:55 PM, Ted Yu yuzhih...@gmail.com wrote:
In master branch, I only found JavaAPISuite in comment:
spark tyu$ find . -name '*.scala' -exec grep JavaAPISuite {} \;
-print
* For usage example, see test case JavaAPISuite.testJavaJdbcRDD.
* converted into a `Object` array. For usage example, see test
case JavaAPISuite.testJavaJdbcRDD.
./core/src/main/scala/org/apache/spark/rdd/JdbcRDD.scala
FYI
On Sat, Dec 6, 2014 at 5:43 PM, Koert Kuipers ko...@tresata.com
wrote:
when i run mvn test -pl core, i dont see JavaAPISuite being run.
or if it is, its being very very quiet about it. is this by design?
Re: run JavaAPISuite with mavem
I think it's a known issue:
https://issues.apache.org/jira/browse/SPARK-4159
https://issues.apache.org/jira/browse/SPARK-661
I got bit by this too recently and meant to look into it.
On Sun, Dec 7, 2014 at 4:50 PM, Koert Kuipers ko...@tresata.com wrote:
so as part of the official build the java api does not get tested then?
i am sure there is a good reason for it, but thats surprising to me.
On Sun, Dec 7, 2014 at 12:19 PM, Ted Yu yuzhih...@gmail.com wrote:
Looking at the pom.xml, I think I found the reason - scalatest is used.
With the following diff:
diff --git a/pom.xml b/pom.xml
index b7df53d..b0da893 100644
--- a/pom.xml
+++ b/pom.xml
@@ -947,7 +947,7 @@
version2.17/version
configuration
!-- Uses scalatest instead --
-skipTeststrue/skipTests
+skipTestsfalse/skipTests
/configuration
/plugin
plugin
I was able to run JavaAPISuite using:
mvn test -pl core -Dtest=JavaAPISuite
But it takes a long time ...
Cheers
On Sun, Dec 7, 2014 at 8:56 AM, Koert Kuipers ko...@tresata.com wrote:
hey guys,
i was able to run the test just fine with:
$ sbt
project core
testOnly org.apache.spark.JavaAPISuite
however i found it strange that it didnt run when i do mvn test -pl
core, or at least didnt seem like it ran to me. this would mean that when
someone tests/publishes with maven the japa api does not get tested at all.
On Sun, Dec 7, 2014 at 1:52 AM, Michael Armbrust mich...@databricks.com
wrote:
Not sure about maven, but you can run that test with sbt:
sbt/sbt sql/test-only org.apache.spark.sql.api.java.JavaAPISuite
On Sat, Dec 6, 2014 at 9:59 PM, Ted Yu yuzhih...@gmail.com wrote:
I tried to run tests for core but there were failures. e.g. :
^[[32mExternalAppendOnlyMapSuite:^[[0m
^[[32m- simple insert^[[0m
^[[32m- insert with collision^[[0m
^[[32m- ordering^[[0m
^[[32m- null keys and values^[[0m
^[[32m- simple aggregator^[[0m
^[[32m- simple cogroup^[[0m
Spark assembly has been built with Hive, including Datanucleus jars on
classpath
^[[31m- spilling *** FAILED ***^[[0m
^[[31m org.apache.spark.SparkException: Job aborted due to stage
failure: Task 0 in stage 0.0 failed 4 times, most recent failure: Lost
task
0.3 in stage 0.0 (TID 6, localhost): java.lang.ClassNotFoundException:
org.apache.spark.rdd.RDD$$anonfun$map$1^[[0m
^[[31m at java.net.URLClassLoader$1.run(URLClassLoader.java:366)^[[0m
^[[31m at java.net.URLClassLoader$1.run(URLClassLoader.java:355)^[[0m
^[[31m at java.security.AccessController.doPrivileged(Native
Method)^[[0m
^[[31m at
java.net.URLClassLoader.findClass(URLClassLoader.java:354)^[[0m
^[[31m at java.lang.ClassLoader.loadClass(ClassLoader.java:424)^[[0m
^[[31m at java.lang.ClassLoader.loadClass(ClassLoader.java:357)^[[0m
^[[31m at java.lang.Class.forName0(Native Method)^[[0m
^[[31m at java.lang.Class.forName(Class.java:270)^[[0m
BTW I didn't find JavaAPISuite in test output either.
Cheers
On Sat, Dec 6, 2014 at 9:12 PM, Koert Kuipers ko...@tresata.com
wrote:
Ted,
i mean
core/src/test/java/org/apache/spark/JavaAPISuite.java
On Sat, Dec 6, 2014 at 9:27 PM, Ted Yu yuzhih...@gmail.com wrote:
Pardon me, the test is here:
sql/core/src/test/java/org/apache/spark/sql/api/java/JavaAPISuite.java
You can run 'mvn test' under sql/core
Cheers
On Sat, Dec 6, 2014 at 5:55 PM, Ted Yu yuzhih...@gmail.com wrote:
In master branch, I only found JavaAPISuite in comment:
spark tyu$ find . -name '*.scala' -exec grep JavaAPISuite {} \;
-print
* For usage example, see test case JavaAPISuite.testJavaJdbcRDD.
* converted into a `Object` array. For usage example, see test
case JavaAPISuite.testJavaJdbcRDD.
./core/src/main/scala/org/apache/spark/rdd/JdbcRDD.scala
FYI
On Sat, Dec 6, 2014 at 5:43 PM, Koert Kuipers ko...@tresata.com
wrote:
when i run mvn test -pl core, i dont see JavaAPISuite being run.
or if it is, its being very very quiet about it. is this by design?
-
To unsubscribe, e-mail: user-unsubscr...@spark.apache.org
For additional commands, e-mail: user-h...@spark.apache.org
Re: run JavaAPISuite with mavem
thanks. that makes sense. i searched the mailing list but couldnt find any
mention of it. i should have searched jira instead...
On Sun, Dec 7, 2014 at 6:25 PM, Sean Owen so...@cloudera.com wrote:
I think it's a known issue:
https://issues.apache.org/jira/browse/SPARK-4159
https://issues.apache.org/jira/browse/SPARK-661
I got bit by this too recently and meant to look into it.
On Sun, Dec 7, 2014 at 4:50 PM, Koert Kuipers ko...@tresata.com wrote:
so as part of the official build the java api does not get tested then?
i am sure there is a good reason for it, but thats surprising to me.
On Sun, Dec 7, 2014 at 12:19 PM, Ted Yu yuzhih...@gmail.com wrote:
Looking at the pom.xml, I think I found the reason - scalatest is used.
With the following diff:
diff --git a/pom.xml b/pom.xml
index b7df53d..b0da893 100644
--- a/pom.xml
+++ b/pom.xml
@@ -947,7 +947,7 @@
version2.17/version
configuration
!-- Uses scalatest instead --
-skipTeststrue/skipTests
+skipTestsfalse/skipTests
/configuration
/plugin
plugin
I was able to run JavaAPISuite using:
mvn test -pl core -Dtest=JavaAPISuite
But it takes a long time ...
Cheers
On Sun, Dec 7, 2014 at 8:56 AM, Koert Kuipers ko...@tresata.com
wrote:
hey guys,
i was able to run the test just fine with:
$ sbt
project core
testOnly org.apache.spark.JavaAPISuite
however i found it strange that it didnt run when i do mvn test -pl
core, or at least didnt seem like it ran to me. this would mean that
when
someone tests/publishes with maven the japa api does not get tested at
all.
On Sun, Dec 7, 2014 at 1:52 AM, Michael Armbrust
mich...@databricks.com
wrote:
Not sure about maven, but you can run that test with sbt:
sbt/sbt sql/test-only org.apache.spark.sql.api.java.JavaAPISuite
On Sat, Dec 6, 2014 at 9:59 PM, Ted Yu yuzhih...@gmail.com wrote:
I tried to run tests for core but there were failures. e.g. :
^[[32mExternalAppendOnlyMapSuite:^[[0m
^[[32m- simple insert^[[0m
^[[32m- insert with collision^[[0m
^[[32m- ordering^[[0m
^[[32m- null keys and values^[[0m
^[[32m- simple aggregator^[[0m
^[[32m- simple cogroup^[[0m
Spark assembly has been built with Hive, including Datanucleus jars
on
classpath
^[[31m- spilling *** FAILED ***^[[0m
^[[31m org.apache.spark.SparkException: Job aborted due to stage
failure: Task 0 in stage 0.0 failed 4 times, most recent failure:
Lost task
0.3 in stage 0.0 (TID 6, localhost):
java.lang.ClassNotFoundException:
org.apache.spark.rdd.RDD$$anonfun$map$1^[[0m
^[[31m at
java.net.URLClassLoader$1.run(URLClassLoader.java:366)^[[0m
^[[31m at
java.net.URLClassLoader$1.run(URLClassLoader.java:355)^[[0m
^[[31m at java.security.AccessController.doPrivileged(Native
Method)^[[0m
^[[31m at
java.net.URLClassLoader.findClass(URLClassLoader.java:354)^[[0m
^[[31m at java.lang.ClassLoader.loadClass(ClassLoader.java:424)^[[0m
^[[31m at java.lang.ClassLoader.loadClass(ClassLoader.java:357)^[[0m
^[[31m at java.lang.Class.forName0(Native Method)^[[0m
^[[31m at java.lang.Class.forName(Class.java:270)^[[0m
BTW I didn't find JavaAPISuite in test output either.
Cheers
On Sat, Dec 6, 2014 at 9:12 PM, Koert Kuipers ko...@tresata.com
wrote:
Ted,
i mean
core/src/test/java/org/apache/spark/JavaAPISuite.java
On Sat, Dec 6, 2014 at 9:27 PM, Ted Yu yuzhih...@gmail.com wrote:
Pardon me, the test is here:
sql/core/src/test/java/org/apache/spark/sql/api/java/JavaAPISuite.java
You can run 'mvn test' under sql/core
Cheers
On Sat, Dec 6, 2014 at 5:55 PM, Ted Yu yuzhih...@gmail.com
wrote:
In master branch, I only found JavaAPISuite in comment:
spark tyu$ find . -name '*.scala' -exec grep JavaAPISuite {} \;
-print
* For usage example, see test case
JavaAPISuite.testJavaJdbcRDD.
* converted into a `Object` array. For usage example, see test
case JavaAPISuite.testJavaJdbcRDD.
./core/src/main/scala/org/apache/spark/rdd/JdbcRDD.scala
FYI
On Sat, Dec 6, 2014 at 5:43 PM, Koert Kuipers ko...@tresata.com
wrote:
when i run mvn test -pl core, i dont see JavaAPISuite being
run.
or if it is, its being very very quiet about it. is this by
design?
run JavaAPISuite with mavem
when i run mvn test -pl core, i dont see JavaAPISuite being run. or if it is, its being very very quiet about it. is this by design?
Re: run JavaAPISuite with mavem
In master branch, I only found JavaAPISuite in comment:
spark tyu$ find . -name '*.scala' -exec grep JavaAPISuite {} \; -print
* For usage example, see test case JavaAPISuite.testJavaJdbcRDD.
* converted into a `Object` array. For usage example, see test case
JavaAPISuite.testJavaJdbcRDD.
./core/src/main/scala/org/apache/spark/rdd/JdbcRDD.scala
FYI
On Sat, Dec 6, 2014 at 5:43 PM, Koert Kuipers ko...@tresata.com wrote:
when i run mvn test -pl core, i dont see JavaAPISuite being run. or if
it is, its being very very quiet about it. is this by design?
Re: run JavaAPISuite with mavem
Pardon me, the test is here:
sql/core/src/test/java/org/apache/spark/sql/api/java/JavaAPISuite.java
You can run 'mvn test' under sql/core
Cheers
On Sat, Dec 6, 2014 at 5:55 PM, Ted Yu yuzhih...@gmail.com wrote:
In master branch, I only found JavaAPISuite in comment:
spark tyu$ find . -name '*.scala' -exec grep JavaAPISuite {} \; -print
* For usage example, see test case JavaAPISuite.testJavaJdbcRDD.
* converted into a `Object` array. For usage example, see test case
JavaAPISuite.testJavaJdbcRDD.
./core/src/main/scala/org/apache/spark/rdd/JdbcRDD.scala
FYI
On Sat, Dec 6, 2014 at 5:43 PM, Koert Kuipers ko...@tresata.com wrote:
when i run mvn test -pl core, i dont see JavaAPISuite being run. or if
it is, its being very very quiet about it. is this by design?
Re: run JavaAPISuite with mavem
Ted,
i mean
core/src/test/java/org/apache/spark/JavaAPISuite.java
On Sat, Dec 6, 2014 at 9:27 PM, Ted Yu yuzhih...@gmail.com wrote:
Pardon me, the test is here:
sql/core/src/test/java/org/apache/spark/sql/api/java/JavaAPISuite.java
You can run 'mvn test' under sql/core
Cheers
On Sat, Dec 6, 2014 at 5:55 PM, Ted Yu yuzhih...@gmail.com wrote:
In master branch, I only found JavaAPISuite in comment:
spark tyu$ find . -name '*.scala' -exec grep JavaAPISuite {} \; -print
* For usage example, see test case JavaAPISuite.testJavaJdbcRDD.
* converted into a `Object` array. For usage example, see test case
JavaAPISuite.testJavaJdbcRDD.
./core/src/main/scala/org/apache/spark/rdd/JdbcRDD.scala
FYI
On Sat, Dec 6, 2014 at 5:43 PM, Koert Kuipers ko...@tresata.com wrote:
when i run mvn test -pl core, i dont see JavaAPISuite being run. or if
it is, its being very very quiet about it. is this by design?
Re: run JavaAPISuite with mavem
I tried to run tests for core but there were failures. e.g. :
^[[32mExternalAppendOnlyMapSuite:^[[0m
^[[32m- simple insert^[[0m
^[[32m- insert with collision^[[0m
^[[32m- ordering^[[0m
^[[32m- null keys and values^[[0m
^[[32m- simple aggregator^[[0m
^[[32m- simple cogroup^[[0m
Spark assembly has been built with Hive, including Datanucleus jars on
classpath
^[[31m- spilling *** FAILED ***^[[0m
^[[31m org.apache.spark.SparkException: Job aborted due to stage failure:
Task 0 in stage 0.0 failed 4 times, most recent failure: Lost task 0.3 in
stage 0.0 (TID 6, localhost): java.lang.ClassNotFoundException:
org.apache.spark.rdd.RDD$$anonfun$map$1^[[0m
^[[31m at java.net.URLClassLoader$1.run(URLClassLoader.java:366)^[[0m
^[[31m at java.net.URLClassLoader$1.run(URLClassLoader.java:355)^[[0m
^[[31m at java.security.AccessController.doPrivileged(Native Method)^[[0m
^[[31m at java.net.URLClassLoader.findClass(URLClassLoader.java:354)^[[0m
^[[31m at java.lang.ClassLoader.loadClass(ClassLoader.java:424)^[[0m
^[[31m at java.lang.ClassLoader.loadClass(ClassLoader.java:357)^[[0m
^[[31m at java.lang.Class.forName0(Native Method)^[[0m
^[[31m at java.lang.Class.forName(Class.java:270)^[[0m
BTW I didn't find JavaAPISuite in test output either.
Cheers
On Sat, Dec 6, 2014 at 9:12 PM, Koert Kuipers ko...@tresata.com wrote:
Ted,
i mean
core/src/test/java/org/apache/spark/JavaAPISuite.java
On Sat, Dec 6, 2014 at 9:27 PM, Ted Yu yuzhih...@gmail.com wrote:
Pardon me, the test is here:
sql/core/src/test/java/org/apache/spark/sql/api/java/JavaAPISuite.java
You can run 'mvn test' under sql/core
Cheers
On Sat, Dec 6, 2014 at 5:55 PM, Ted Yu yuzhih...@gmail.com wrote:
In master branch, I only found JavaAPISuite in comment:
spark tyu$ find . -name '*.scala' -exec grep JavaAPISuite {} \; -print
* For usage example, see test case JavaAPISuite.testJavaJdbcRDD.
* converted into a `Object` array. For usage example, see test case
JavaAPISuite.testJavaJdbcRDD.
./core/src/main/scala/org/apache/spark/rdd/JdbcRDD.scala
FYI
On Sat, Dec 6, 2014 at 5:43 PM, Koert Kuipers ko...@tresata.com wrote:
when i run mvn test -pl core, i dont see JavaAPISuite being run. or
if it is, its being very very quiet about it. is this by design?
Re: run JavaAPISuite with mavem
Not sure about maven, but you can run that test with sbt:
sbt/sbt sql/test-only org.apache.spark.sql.api.java.JavaAPISuite
On Sat, Dec 6, 2014 at 9:59 PM, Ted Yu yuzhih...@gmail.com wrote:
I tried to run tests for core but there were failures. e.g. :
^[[32mExternalAppendOnlyMapSuite:^[[0m
^[[32m- simple insert^[[0m
^[[32m- insert with collision^[[0m
^[[32m- ordering^[[0m
^[[32m- null keys and values^[[0m
^[[32m- simple aggregator^[[0m
^[[32m- simple cogroup^[[0m
Spark assembly has been built with Hive, including Datanucleus jars on
classpath
^[[31m- spilling *** FAILED ***^[[0m
^[[31m org.apache.spark.SparkException: Job aborted due to stage failure:
Task 0 in stage 0.0 failed 4 times, most recent failure: Lost task 0.3 in
stage 0.0 (TID 6, localhost): java.lang.ClassNotFoundException:
org.apache.spark.rdd.RDD$$anonfun$map$1^[[0m
^[[31m at java.net.URLClassLoader$1.run(URLClassLoader.java:366)^[[0m
^[[31m at java.net.URLClassLoader$1.run(URLClassLoader.java:355)^[[0m
^[[31m at java.security.AccessController.doPrivileged(Native Method)^[[0m
^[[31m at java.net.URLClassLoader.findClass(URLClassLoader.java:354)^[[0m
^[[31m at java.lang.ClassLoader.loadClass(ClassLoader.java:424)^[[0m
^[[31m at java.lang.ClassLoader.loadClass(ClassLoader.java:357)^[[0m
^[[31m at java.lang.Class.forName0(Native Method)^[[0m
^[[31m at java.lang.Class.forName(Class.java:270)^[[0m
BTW I didn't find JavaAPISuite in test output either.
Cheers
On Sat, Dec 6, 2014 at 9:12 PM, Koert Kuipers ko...@tresata.com wrote:
Ted,
i mean
core/src/test/java/org/apache/spark/JavaAPISuite.java
On Sat, Dec 6, 2014 at 9:27 PM, Ted Yu yuzhih...@gmail.com wrote:
Pardon me, the test is here:
sql/core/src/test/java/org/apache/spark/sql/api/java/JavaAPISuite.java
You can run 'mvn test' under sql/core
Cheers
On Sat, Dec 6, 2014 at 5:55 PM, Ted Yu yuzhih...@gmail.com wrote:
In master branch, I only found JavaAPISuite in comment:
spark tyu$ find . -name '*.scala' -exec grep JavaAPISuite {} \; -print
* For usage example, see test case JavaAPISuite.testJavaJdbcRDD.
* converted into a `Object` array. For usage example, see test case
JavaAPISuite.testJavaJdbcRDD.
./core/src/main/scala/org/apache/spark/rdd/JdbcRDD.scala
FYI
On Sat, Dec 6, 2014 at 5:43 PM, Koert Kuipers ko...@tresata.com
wrote:
when i run mvn test -pl core, i dont see JavaAPISuite being run. or
if it is, its being very very quiet about it. is this by design?
