Spark test error in ProactiveClosureSerializationSuite.scala

2017-02-25 Thread ??????????
hello all, I am building Spark1.6.2 and I meet a problem when doing mvn test


The command is mvn -e -Pyarn  -Phive -Phive-thriftserver  
-DwildcardSuites=org.apache.spark.serializer.ProactiveClosureSerializationSuite 
test
and the test error is
ProactiveClosureSerializationSuite:
- throws expected serialization exceptions on actions
- mapPartitions transformations throw proactive serialization exceptions *** 
FAILED ***
  Expected exception org.apache.spark.SparkException to be thrown, but no 
exception was thrown. (ProactiveClosureSerializationSuite.scala:58)
- map transformations throw proactive serialization exceptions
- filter transformations throw proactive serialization exceptions
- flatMap transformations throw proactive serialization exceptions
- mapPartitionsWithIndex transformations throw proactive serialization 
exceptions *** FAILED ***
  Expected exception org.apache.spark.SparkException to be thrown, but no 
exception was thrown. (ProactiveClosureSerializationSuite.scala:58)



I think this test is about task not serializable, but why do I only get test 
error on mapPartitions and mapPartitionsWithIndex?


Thanks.

Spark test error

2017-01-03 Thread Yanwei Wayne Zhang
I tried to run the tests in 'GeneralizedLinearRegressionSuite', and all tests 
passed except for test("read/write") which yielded the following error message. 
Any suggestion on why this happened and how to fix it? Thanks. BTW, I ran the 
test in IntelliJ.


The default jsonEncode only supports string and vector. 
org.apache.spark.ml.param.Param must override jsonEncode for java.lang.Double.
scala.NotImplementedError: The default jsonEncode only supports string and 
vector. org.apache.spark.ml.param.Param must override jsonEncode for 
java.lang.Double.
at org.apache.spark.ml.param.Param.jsonEncode(params.scala:98)
at 
org.apache.spark.ml.util.DefaultParamsWriter$$anonfun$1$$anonfun$2.apply(ReadWrite.scala:293)
at 
org.apache.spark.ml.util.DefaultParamsWriter$$anonfun$1$$anonfun$2.apply(ReadWrite.scala:292)


Regards,
Wayne