Re: [VOTE] Apache Spark 2.1.0 (RC2)

Another failing test is "ReplSuite:should clone and clean line object in ClosureCleaner". It never passes for me, just keeps spinning until the JVM eventually starts throwing OOM errors. Anyone seeing that? On Thu, Dec 8, 2016 at 12:39 AM, Reynold Xin wrote: > Please vote on

Re: [VOTE] Apache Spark 2.1.0 (RC2)

-1 I hit https://issues.apache.org/jira/browse/SPARK-18816, which prevents executor page from showing the log links if an application does not have executors initially. On Mon, Dec 12, 2016 at 3:02 PM, Marcelo Vanzin wrote: > Actually this is not a simple pom change. The

Re: [VOTE] Apache Spark 2.1.0 (RC2)

Actually this is not a simple pom change. The code in UDFRegistration.scala calls this method: if (returnType == null) { returnType = JavaTypeInference.inferDataType(TypeToken.of(udfReturnType))._1 } Because we shade guava, it's generally not very safe to call

Re: [VOTE] Apache Spark 2.1.0 (RC2)

I'm running into this when building / testing on 1.7 (haven't tried 1.8): udf3Test(test.org.apache.spark.sql.JavaUDFSuite) Time elapsed: 0.079 sec <<< ERROR! java.lang.NoSuchMethodError: org.apache.spark.sql.catalyst.JavaTypeInference$.inferDataType(Lcom/google/common/reflect/TypeToken;)Lsc

Re: Aggregating over sorted data

Hi, SPARK-18591 might be a solution to your problem but making assuming in your UDAF logic on how Spark will process the aggregation is really a risky thing. Is there a way to do it using Windows function with ORDER BY clause to enforce the