[GitHub] spark pull request: [SPARK-8335] [MLLIB] DecisionTreeModel.predict...

2015-06-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6854#issuecomment-112721004 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-8395] [DOCS] start-slave.sh docs incorr...

2015-06-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6855#issuecomment-112720975 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not h

[GitHub] spark pull request: [SPARK-8335] [MLLIB] DecisionTreeModel.predict...

2015-06-17 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/6854#issuecomment-112721117 [Test build #35031 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/35031/consoleFull) for PR 6854 at commit [`2c8f925`](https://gith

[GitHub] spark pull request: [SPARK-8395] [DOCS] start-slave.sh docs incorr...

2015-06-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6855#issuecomment-112721005 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-8335] [MLLIB] DecisionTreeModel.predict...

2015-06-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6854#issuecomment-112720982 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not h

[GitHub] spark pull request: [SPARK-7199][SQL] Add date and timestamp suppo...

2015-06-17 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5984#issuecomment-112720408 [Test build #35029 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/35029/consoleFull) for PR 5984 at commit [`7f21ce9`](https://gith

[GitHub] spark pull request: [SPARK-8395] [DOCS] start-slave.sh docs incorr...

2015-06-17 Thread srowen
GitHub user srowen opened a pull request: https://github.com/apache/spark/pull/6855 [SPARK-8395] [DOCS] start-slave.sh docs incorrect start-slave.sh no longer takes a worker # param in 1.4+ You can merge this pull request into a Git repository by running: $ git pull https://git

[GitHub] spark pull request: [SPARK-8240] [SPARK-8241] [SQL] string functio...

2015-06-17 Thread adrian-wang
Github user adrian-wang commented on the pull request: https://github.com/apache/spark/pull/6775#issuecomment-112719572 @rxin --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature ena

[GitHub] spark pull request: [SPARK-8335] [MLLIB] DecisionTreeModel.predict...

2015-06-17 Thread srowen
GitHub user srowen opened a pull request: https://github.com/apache/spark/pull/6854 [SPARK-8335] [MLLIB] DecisionTreeModel.predict() return type not convenient Make Java-friendly predict return a JavaRDD of java.lang.Double instead of Scala Double, in the way ClassificationModel et

[GitHub] spark pull request: [SPARK-7199][SQL] Add date and timestamp suppo...

2015-06-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/5984#issuecomment-112718287 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not h

[GitHub] spark pull request: [SPARK-7199][SQL] Add date and timestamp suppo...

2015-06-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/5984#issuecomment-112718380 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-8240] [SPARK-8241] [SQL] string functio...

2015-06-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6775#issuecomment-112717485 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-8240] [SPARK-8241] [SQL] string functio...

2015-06-17 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/6775#issuecomment-112717437 [Test build #35026 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/35026/console) for PR 6775 at commit [`282e958`](https://github.

[GitHub] spark pull request: [SQL][SPARK-7088] Fix analysis for 3rd party l...

2015-06-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6853#issuecomment-112716410 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your pr

[GitHub] spark pull request: [SQL][SPARK-7088] Fix analysis for 3rd party l...

2015-06-17 Thread smola
GitHub user smola opened a pull request: https://github.com/apache/spark/pull/6853 [SQL][SPARK-7088] Fix analysis for 3rd party logical plan. ResolveReferences analysis rule now does not throw when it cannot resolve references in a self-join. You can merge this pull request into a

[GitHub] spark pull request: [SQL][SPARK-7088] Fix analysis for 3rd party l...

2015-06-17 Thread smola
Github user smola commented on the pull request: https://github.com/apache/spark/pull/6853#issuecomment-112716142 This is still a work in progress, since I have not tested yet. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as we

[GitHub] spark pull request: [SPARK-7799][Streaming]Add "streaming-akka" pr...

2015-06-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6841#issuecomment-112715278 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-7799][Streaming]Add "streaming-akka" pr...

2015-06-17 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/6841#issuecomment-112715229 [Test build #35024 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/35024/console) for PR 6841 at commit [`2be2090`](https://github.

[GitHub] spark pull request: [SPARK-6157][CORE]Unroll unsuccessful memory_a...

2015-06-17 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4887#issuecomment-112714878 [Test build #35025 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/35025/console) for PR 4887 at commit [`f364f07`](https://github.

[GitHub] spark pull request: [SPARK-6157][CORE]Unroll unsuccessful memory_a...

2015-06-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4887#issuecomment-112714933 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-6157][CORE]Unroll unsuccessful memory_a...

2015-06-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4887#issuecomment-112713787 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-7050][build] Keep maven build consisten...

2015-06-17 Thread srowen
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/5632#issuecomment-112713765 OK, then this should happen for flume-sink too? And update the titles here. --- If your project is set up for it, you can reply to this email and have your reply appear o

[GitHub] spark pull request: [SPARK-6157][CORE]Unroll unsuccessful memory_a...

2015-06-17 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4887#issuecomment-112713758 [Test build #35023 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/35023/console) for PR 4887 at commit [`26c4e86`](https://github.

[GitHub] spark pull request: [SPARK-8404][Streaming][Tests] Use thread-safe...

2015-06-17 Thread srowen
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/6852#issuecomment-112712207 Yes I understand that argument, though have you observed a failure as a result? There are certainly some memory barriers in between the writes and reads without this.

[GitHub] spark pull request: [SPARK-8404][Streaming][Tests] Use thread-safe...

2015-06-17 Thread zsxwing
Github user zsxwing commented on the pull request: https://github.com/apache/spark/pull/6852#issuecomment-112707759 > In all cases foreachRDD executes serially on the driver; what's the thread safety issue? The codes in foreachRDD run in `JobScheduler.jobExecutor`. But the che

[GitHub] spark pull request: [SPARK-8213][SQL][WIP]Add function factorial

2015-06-17 Thread zhichao-li
Github user zhichao-li commented on a diff in the pull request: https://github.com/apache/spark/pull/6822#discussion_r32603052 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/math.scala --- @@ -160,6 +161,42 @@ case class Expm1(child: Expression) ex

[GitHub] spark pull request: [SPARK-8213][SQL][WIP]Add function factorial

2015-06-17 Thread zhichao-li
Github user zhichao-li commented on a diff in the pull request: https://github.com/apache/spark/pull/6822#discussion_r32602793 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/math.scala --- @@ -160,6 +161,42 @@ case class Expm1(child: Expression) ex

[GitHub] spark pull request: [SPARK-8404][Streaming][Tests] Use thread-safe...

2015-06-17 Thread srowen
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/6852#issuecomment-112705568 In all cases `foreachRDD` executes serially on the driver; what's the thread safety issue? --- If your project is set up for it, you can reply to this email and have you

[GitHub] spark pull request: [SPARK-8213][SQL][WIP]Add function factorial

2015-06-17 Thread zhichao-li
Github user zhichao-li commented on a diff in the pull request: https://github.com/apache/spark/pull/6822#discussion_r32602667 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/math.scala --- @@ -160,6 +161,42 @@ case class Expm1(child: Expression) ex

[GitHub] spark pull request: [SPARK-8404][Streaming][Tests] Use thread-safe...

2015-06-17 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/6852#discussion_r32602651 --- Diff: external/kafka/src/test/java/org/apache/spark/streaming/kafka/JavaDirectKafkaStreamSuite.java --- @@ -116,7 +114,7 @@ public String call(MessageAndM

[GitHub] spark pull request: [SPARK-8404][Streaming][Tests] Use thread-safe...

2015-06-17 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/6852#issuecomment-112703034 [Test build #35028 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/35028/consoleFull) for PR 6852 at commit [`d464211`](https://gith

[GitHub] spark pull request: [SPARK-8404][Streaming][Tests] Use thread-safe...

2015-06-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6852#issuecomment-112701815 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not h

[GitHub] spark pull request: [SPARK-8283][SQL] Resolve udf_struct test fail...

2015-06-17 Thread yijieshen
Github user yijieshen commented on a diff in the pull request: https://github.com/apache/spark/pull/6828#discussion_r32602451 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/complexTypes.scala --- @@ -62,9 +62,14 @@ case class CreateStruct(children:

[GitHub] spark pull request: [SPARK-8404][Streaming][Tests] Use thread-safe...

2015-06-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6852#issuecomment-112701962 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-8404][Streaming][Tests] Use thread-safe...

2015-06-17 Thread zsxwing
Github user zsxwing commented on the pull request: https://github.com/apache/spark/pull/6852#issuecomment-112700126 cc @tdas --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabl

[GitHub] spark pull request: [SPARK-8404][Streaming][Tests] Use thread-safe...

2015-06-17 Thread zsxwing
GitHub user zsxwing opened a pull request: https://github.com/apache/spark/pull/6852 [SPARK-8404][Streaming][Tests] Use thread-safe collections to make the tests more reliable KafkaStreamSuite, DirectKafkaStreamSuite, JavaKafkaStreamSuite and JavaDirectKafkaStreamSuite use non-thre

[GitHub] spark pull request: [SPARK-8203] [SPARK-8204] [WIP] [SQL] conditio...

2015-06-17 Thread chenghao-intel
Github user chenghao-intel commented on the pull request: https://github.com/apache/spark/pull/6851#issuecomment-112695568 Don't forget to implement the `def foldable`. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If y

[GitHub] spark pull request: [SPARK-8203] [SPARK-8204] [WIP] [SQL] conditio...

2015-06-17 Thread chenghao-intel
Github user chenghao-intel commented on a diff in the pull request: https://github.com/apache/spark/pull/6851#discussion_r32601604 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/conditionals.scala --- @@ -312,3 +312,63 @@ case class CaseKeyWhen(key

[GitHub] spark pull request: [SPARK-8203] [SPARK-8204] [WIP] [SQL] conditio...

2015-06-17 Thread chenghao-intel
Github user chenghao-intel commented on a diff in the pull request: https://github.com/apache/spark/pull/6851#discussion_r32601495 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/conditionals.scala --- @@ -312,3 +312,63 @@ case class CaseKeyWhen(key

[GitHub] spark pull request: [SPARK-8203] [SPARK-8204] [WIP] [SQL] conditio...

2015-06-17 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/6851#issuecomment-112694311 [Test build #35027 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/35027/consoleFull) for PR 6851 at commit [`3e25c6c`](https://gith

[GitHub] spark pull request: [SPARK-8283][SQL] Resolve udf_struct test fail...

2015-06-17 Thread chenghao-intel
Github user chenghao-intel commented on the pull request: https://github.com/apache/spark/pull/6828#issuecomment-112692973 `named_struct` is not implemented in Spark SQL right? At least, it should be registered in `FunctionRegistry`. Of course we can leave it for the future. --- If y

[GitHub] spark pull request: [SPARK-8203] [SPARK-8204] [WIP] [SQL] conditio...

2015-06-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6851#issuecomment-112692996 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-8203] [SPARK-8204] [WIP] [SQL] conditio...

2015-06-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6851#issuecomment-112692907 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not h

[GitHub] spark pull request: [SPARK-8203] [SPARK-8204] [WIP] [SQL] conditio...

2015-06-17 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/6851#discussion_r32601187 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/ConditionalExpressionSuite.scala --- @@ -134,4 +134,21 @@ class ConditionalExpre

[GitHub] spark pull request: Spark 5659 Flaky test: o.a.s.streaming.Receive...

2015-06-17 Thread kevinpetersavage
Github user kevinpetersavage closed the pull request at: https://github.com/apache/spark/pull/4957 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the featu

[GitHub] spark pull request: [Streaming][Kafka][SPARK-8389] Example of gett...

2015-06-17 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/6846#discussion_r32601142 --- Diff: external/kafka/src/test/java/org/apache/spark/streaming/kafka/JavaDirectKafkaStreamSuite.java --- @@ -89,6 +90,16 @@ public void testKafkaStream() t

[GitHub] spark pull request: Fix break introduced by backport

2015-06-17 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/6850 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[GitHub] spark pull request: Fix break introduced by backport

2015-06-17 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/6850#issuecomment-112692419 Actually never mind I'm going to push a commit to close this. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as wel

[GitHub] spark pull request: [SPARK-8283][SQL] Resolve udf_struct test fail...

2015-06-17 Thread chenghao-intel
Github user chenghao-intel commented on a diff in the pull request: https://github.com/apache/spark/pull/6828#discussion_r32601130 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/complexTypes.scala --- @@ -62,9 +62,14 @@ case class CreateStruct(chil

[GitHub] spark pull request: [SPARK-8203] [SPARK-8204] [WIP] [SQL] conditio...

2015-06-17 Thread adrian-wang
GitHub user adrian-wang opened a pull request: https://github.com/apache/spark/pull/6851 [SPARK-8203] [SPARK-8204] [WIP] [SQL] conditional function: least/greatest @chenghao-intel @zhichao-li @qiansl127 You can merge this pull request into a Git repository by running: $ git pul

[GitHub] spark pull request: [SPARK-8307] [SQL] improve timestamp from parq...

2015-06-17 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/6759#issuecomment-112692322 @JoshRosen can you review this? Thanks. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project doe

[GitHub] spark pull request: [SPARK-7050][build] Keep maven build consisten...

2015-06-17 Thread jerryshao
Github user jerryshao commented on the pull request: https://github.com/apache/spark/pull/5632#issuecomment-112692394 Hi @srowen , that's what I original meaning, move kafka-assembly into scala-version folder. --- If your project is set up for it, you can reply to this email and have

[GitHub] spark pull request: Update Java dependency to 1.7

2015-06-17 Thread tellison
Github user tellison closed the pull request at: https://github.com/apache/spark/pull/6264 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is en

[GitHub] spark pull request: [SPARK-8401] [Build] Scala version switching b...

2015-06-17 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/6832#discussion_r32600697 --- Diff: dev/change-scala-version.sh --- @@ -0,0 +1,63 @@ +#!/usr/bin/env bash + +# +# Licensed to the Apache Software Foundation (ASF) under o

[GitHub] spark pull request: [SPARK-8401] [Build] Scala version switching b...

2015-06-17 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/6832#discussion_r32600616 --- Diff: dev/change-scala-version.sh --- @@ -0,0 +1,63 @@ +#!/usr/bin/env bash + +# +# Licensed to the Apache Software Foundation (ASF) under o

[GitHub] spark pull request: [SPARK-7050][build] Keep maven build consisten...

2015-06-17 Thread srowen
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/5632#issuecomment-112687299 I think the destination of the assembly JARs has been overridden in Maven because it needs to include the Scala version in order to not get those mixed up easily, and sam

[GitHub] spark pull request: [SPARK-5259][CORE]Make sure mapStage.pendingta...

2015-06-17 Thread suyanNone
Github user suyanNone commented on a diff in the pull request: https://github.com/apache/spark/pull/4055#discussion_r32600352 --- Diff: core/src/main/scala/org/apache/spark/scheduler/ResultTask.scala --- @@ -65,4 +65,6 @@ private[spark] class ResultTask[T, U]( override def p

[GitHub] spark pull request: [SPARK-8283][SQL] Resolve udf_struct test fail...

2015-06-17 Thread yijieshen
Github user yijieshen commented on a diff in the pull request: https://github.com/apache/spark/pull/6828#discussion_r32599952 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/complexTypes.scala --- @@ -62,9 +62,14 @@ case class CreateStruct(children:

<    3   4   5   6   7   8