[GitHub] spark pull request: [SPARK-5338][MESOS] Add cluster mode support f...

2015-04-02 Thread tnachen
Github user tnachen commented on a diff in the pull request: https://github.com/apache/spark/pull/5144#discussion_r27717930 --- Diff: core/src/main/scala/org/apache/spark/deploy/mesos/MesosClusterDispatcher.scala --- @@ -0,0 +1,207 @@ +/* + * Licensed to the Apache Softwar

[GitHub] spark pull request: [SPARK-6211][Streaming] Add Python Kafka API u...

2015-04-02 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/4961#discussion_r27717848 --- Diff: external/kafka/src/main/scala/org/apache/spark/streaming/kafka/KafkaTestUtils.scala --- @@ -0,0 +1,272 @@ +/* + * Licensed to the Apache Softw

[GitHub] spark pull request: [Spark-6693][MLlib]add tostring with max lines...

2015-04-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/5344#issuecomment-89194299 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/29

[GitHub] spark pull request: [Spark-6693][MLlib]add tostring with max lines...

2015-04-02 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5344#issuecomment-89194293 [Test build #29655 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29655/consoleFull) for PR 5344 at commit [`64a9e0f`](https://gith

[GitHub] spark pull request: [Spark-6693][MLlib]add tostring with max lines...

2015-04-02 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5344#issuecomment-89193994 [Test build #29655 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29655/consoleFull) for PR 5344 at commit [`64a9e0f`](https://githu

[GitHub] spark pull request: [SPARK-5972] [MLlib] Cache residuals and gradi...

2015-04-02 Thread MechCoder
Github user MechCoder commented on a diff in the pull request: https://github.com/apache/spark/pull/5330#discussion_r27717768 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/tree/GradientBoostedTrees.scala --- @@ -195,17 +195,24 @@ object GradientBoostedTrees extends Logging

[GitHub] spark pull request: [Spark-6693][MLlib]add to string with max line...

2015-04-02 Thread hhbyyh
GitHub user hhbyyh opened a pull request: https://github.com/apache/spark/pull/5344 [Spark-6693][MLlib]add to string with max lines and width for matrix jira: https://issues.apache.org/jira/browse/SPARK-6693 It's kind of annoying when debugging and found you cannot print out

[GitHub] spark pull request: [SPARK-6692][YARN] Make it possible to kill AM...

2015-04-02 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5343#issuecomment-89193013 [Test build #29654 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29654/consoleFull) for PR 5343 at commit [`ef2793c`](https://githu

[GitHub] spark pull request: [SPARK-6211][Streaming] Add Python Kafka API u...

2015-04-02 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/4961#discussion_r27717725 --- Diff: python/pyspark/streaming/tests.py --- @@ -556,5 +559,42 @@ def check_output(n): check_output(3) +class KafkaStreamTest

[GitHub] spark pull request: [SPARK-5523][Core][Streaming] Add a cache for ...

2015-04-02 Thread tdas
Github user tdas commented on the pull request: https://github.com/apache/spark/pull/5064#issuecomment-89191467 @JoshRosen Can you take a look at this? I had observed a lot of string objects with hostnames once in the stack, and this might help reduce it. --- If your project is set

[GitHub] spark pull request: [SPARK-6211][Streaming] Add Python Kafka API u...

2015-04-02 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/4961#discussion_r27717631 --- Diff: python/pyspark/streaming/tests.py --- @@ -556,5 +559,42 @@ def check_output(n): check_output(3) +class KafkaStreamTest

[GitHub] spark pull request: [SPARK-6692][YARN] Make it possible to kill AM...

2015-04-02 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/5343#issuecomment-89190428 Jenkins, test this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [SPARK-6692][YARN] Make it possible to kill AM...

2015-04-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/5343#issuecomment-89190287 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your pro

[GitHub] spark pull request: [SPARK-5556][MLLib][WIP] Gibbs LDA, Refactor L...

2015-04-02 Thread EntilZha
Github user EntilZha commented on the pull request: https://github.com/apache/spark/pull/4807#issuecomment-89189465 Just double checking, your suggestion would be to rebase from master, implement those general changes, then commit/push the modified branch? Primary reason I did

[GitHub] spark pull request: [SPARK-4233] [SQL] WIP:Simplify the UDAF API (...

2015-04-02 Thread chenghao-intel
Github user chenghao-intel commented on a diff in the pull request: https://github.com/apache/spark/pull/3247#discussion_r27717574 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregates.scala --- @@ -17,285 +17,159 @@ package org.apac

[GitHub] spark pull request: [SPARK-4233] [SQL] WIP:Simplify the UDAF API (...

2015-04-02 Thread chenghao-intel
Github user chenghao-intel commented on a diff in the pull request: https://github.com/apache/spark/pull/3247#discussion_r27717553 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregates.scala --- @@ -17,285 +17,159 @@ package org.apac

[GitHub] spark pull request: [SPARK-4233] [SQL] WIP:Simplify the UDAF API (...

2015-04-02 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/3247#discussion_r27717539 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/FunctionRegistry.scala --- @@ -26,9 +26,12 @@ trait FunctionRegistry {

[GitHub] spark pull request: [SPARK-6428] Turn on explicit type checking fo...

2015-04-02 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5342#issuecomment-89188173 [Test build #29653 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29653/consoleFull) for PR 5342 at commit [`7b531ab`](https://githu

[GitHub] spark pull request: [SPARK-6692][YARN] Make it possible to kill AM...

2015-04-02 Thread piaozhexiu
GitHub user piaozhexiu opened a pull request: https://github.com/apache/spark/pull/5343 [SPARK-6692][YARN] Make it possible to kill AM in YARN cluster mode I understand that the yarn-cluster mode is designed for fire-and-forget model; therefore, terminating the yarn client doesn't k

[GitHub] spark pull request: [SPARK-6211][Streaming] Add Python Kafka API u...

2015-04-02 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/4961#discussion_r27717482 --- Diff: python/run-tests --- @@ -96,6 +98,21 @@ function run_ml_tests() { function run_streaming_tests() { echo "Run streaming tests ..

[GitHub] spark pull request: [SPARK-6428] Turn on explicit type checking fo...

2015-04-02 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5342#issuecomment-89186810 [Test build #29652 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29652/consoleFull) for PR 5342 at commit [`9b9e119`](https://githu

[GitHub] spark pull request: [SPARK-4233] [SQL] WIP:Simplify the UDAF API (...

2015-04-02 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/3247#discussion_r27717430 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/Aggregate.scala --- @@ -17,181 +17,461 @@ package org.apache.spark.sql.execution

[GitHub] spark pull request: [SPARK-6428] Turn on explicit type checking fo...

2015-04-02 Thread mengxr
Github user mengxr commented on the pull request: https://github.com/apache/spark/pull/5342#issuecomment-89186702 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark pull request: [SPARK-6428] Turn on explicit type checking fo...

2015-04-02 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/5342#discussion_r27717373 --- Diff: yarn/src/main/scala/org/apache/spark/deploy/yarn/ExecutorRunnable.scala --- @@ -59,7 +59,7 @@ class ExecutorRunnable( val yarnConf: YarnConfig

[GitHub] spark pull request: [SPARK-6428] Turn on explicit type checking fo...

2015-04-02 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/5342#discussion_r27717375 --- Diff: core/src/main/scala/org/apache/spark/api/java/JavaRDDLike.scala --- @@ -17,6 +17,7 @@ package org.apache.spark.api.java +import

[GitHub] spark pull request: [SPARK-4233] [SQL] WIP:Simplify the UDAF API (...

2015-04-02 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/3247#discussion_r27717364 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregates.scala --- @@ -17,285 +17,159 @@ package org.apache.spark

[GitHub] spark pull request: [SPARK-6211][Streaming] Add Python Kafka API u...

2015-04-02 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/4961#discussion_r27717263 --- Diff: external/kafka/src/main/scala/org/apache/spark/streaming/kafka/KafkaTestUtils.scala --- @@ -0,0 +1,272 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request: [SPARK-4233] [SQL] WIP:Simplify the UDAF API (...

2015-04-02 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/3247#discussion_r27717257 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregates.scala --- @@ -17,285 +17,159 @@ package org.apache.spark

[GitHub] spark pull request: [SPARK-6130] [SQL] support if not exists for i...

2015-04-02 Thread adrian-wang
Github user adrian-wang commented on the pull request: https://github.com/apache/spark/pull/4865#issuecomment-89184362 The updated PR test of 29646 has passed... 29645 is an invalid rebase, let's just ignore it. --- If your project is set up for it, you can reply to this email and

[GitHub] spark pull request: [SPARK-6211][Streaming] Add Python Kafka API u...

2015-04-02 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/4961#discussion_r27717233 --- Diff: external/kafka/src/main/scala/org/apache/spark/streaming/kafka/KafkaTestUtils.scala --- @@ -0,0 +1,272 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request: [SPARK-6428] Turn on explicit type checking fo...

2015-04-02 Thread ankurdave
Github user ankurdave commented on the pull request: https://github.com/apache/spark/pull/5342#issuecomment-89184349 LGTM for the GraphX changes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not ha

[GitHub] spark pull request: [SPARK-6428] Turn on explicit type checking fo...

2015-04-02 Thread ankurdave
Github user ankurdave commented on a diff in the pull request: https://github.com/apache/spark/pull/5342#discussion_r27717210 --- Diff: graphx/src/main/scala/org/apache/spark/graphx/lib/ConnectedComponents.scala --- @@ -37,7 +37,7 @@ object ConnectedComponents { */ d

[GitHub] spark pull request: [SPARK-6428] Turn on explicit type checking fo...

2015-04-02 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5342#issuecomment-89184332 [Test build #29651 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29651/consoleFull) for PR 5342 at commit [`82e0cf5`](https://githu

[GitHub] spark pull request: [SPARK-4233] [SQL] WIP:Simplify the UDAF API (...

2015-04-02 Thread chenghao-intel
Github user chenghao-intel commented on a diff in the pull request: https://github.com/apache/spark/pull/3247#discussion_r27717209 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregates.scala --- @@ -17,285 +17,159 @@ package org.apac

[GitHub] spark pull request: [SPARK-6211][Streaming] Add Python Kafka API u...

2015-04-02 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/4961#discussion_r27717198 --- Diff: external/kafka/src/main/scala/org/apache/spark/streaming/kafka/KafkaTestUtils.scala --- @@ -0,0 +1,272 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request: [SPARK-6428] Turn on explicit type checking fo...

2015-04-02 Thread rxin
GitHub user rxin opened a pull request: https://github.com/apache/spark/pull/5342 [SPARK-6428] Turn on explicit type checking for public methods. You can merge this pull request into a Git repository by running: $ git pull https://github.com/rxin/spark SPARK-6428 Alternativel

[GitHub] spark pull request: [SPARK-3862] [SQL] [WIP] MultiWayBroadcastJoin...

2015-04-02 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5326#issuecomment-89183925 [Test build #29650 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29650/consoleFull) for PR 5326 at commit [`e62be3b`](https://githu

[GitHub] spark pull request: [SPARK-4233] [SQL] WIP:Simplify the UDAF API (...

2015-04-02 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/3247#discussion_r27717124 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregates.scala --- @@ -17,285 +17,159 @@ package org.apache.spark

[GitHub] spark pull request: [SPARK-4233] [SQL] WIP:Simplify the UDAF API (...

2015-04-02 Thread chenghao-intel
Github user chenghao-intel commented on a diff in the pull request: https://github.com/apache/spark/pull/3247#discussion_r27717120 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/Row.scala --- @@ -305,6 +305,49 @@ trait Row extends Serializable { */ def get

[GitHub] spark pull request: [SPARK-4233] [SQL] WIP:Simplify the UDAF API (...

2015-04-02 Thread chenghao-intel
Github user chenghao-intel commented on a diff in the pull request: https://github.com/apache/spark/pull/3247#discussion_r27717103 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/FunctionRegistry.scala --- @@ -26,9 +26,12 @@ trait FunctionRegistry {

[GitHub] spark pull request: [MLLIB] Remove println in LogisticRegression.s...

2015-04-02 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5338#issuecomment-89183760 [Test build #29647 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29647/consoleFull) for PR 5338 at commit [`3f3dbf4`](https://gith

[GitHub] spark pull request: [MLLIB] Remove println in LogisticRegression.s...

2015-04-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/5338#issuecomment-89183767 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/29

[GitHub] spark pull request: [SPARK-4233] [SQL] WIP:Simplify the UDAF API (...

2015-04-02 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/3247#discussion_r27717014 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/FunctionRegistry.scala --- @@ -26,9 +26,12 @@ trait FunctionRegistry {

[GitHub] spark pull request: [SPARK-6130] [SQL] support if not exists for i...

2015-04-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4865#issuecomment-89183381 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/29

[GitHub] spark pull request: [SPARK-6211][Streaming] Add Python Kafka API u...

2015-04-02 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/4961#discussion_r27716952 --- Diff: python/run-tests --- @@ -27,6 +27,8 @@ cd "$FWDIR/python" FAILED=0 LOG_FILE=unit-tests.log +. "$FWDIR"/bin/load-spark-env.sh

[GitHub] spark pull request: [SPARK-6211][Streaming] Add Python Kafka API u...

2015-04-02 Thread tdas
Github user tdas commented on the pull request: https://github.com/apache/spark/pull/4961#issuecomment-89182308 Overall this looks good to me, just a few minor comments. I also asked @JoshRosen to take a look at the Python changes. --- If your project is set up for it, you can reply

[GitHub] spark pull request: [SPARK-4233] [SQL] WIP:Simplify the UDAF API (...

2015-04-02 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/3247#discussion_r27716889 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/rows.scala --- @@ -37,6 +35,21 @@ trait MutableRow extends Row { def se

[GitHub] spark pull request: [SPARK-4233] [SQL] WIP:Simplify the UDAF API (...

2015-04-02 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/3247#discussion_r27716819 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/Row.scala --- @@ -305,6 +305,49 @@ trait Row extends Serializable { */ def getAs[T](i:

[GitHub] spark pull request: [SPARK-6211][Streaming] Add Python Kafka API u...

2015-04-02 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/4961#discussion_r27716721 --- Diff: python/run-tests --- @@ -27,6 +27,8 @@ cd "$FWDIR/python" FAILED=0 LOG_FILE=unit-tests.log +. "$FWDIR"/bin/load-spark-env.sh ---

[GitHub] spark pull request: [SPARK-6621][Core] Fix the bug that calling Ev...

2015-04-02 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/5280#issuecomment-89177379 LGTM, so I'm going to merge this into `master` (1.4.0) and `branch-1.3` (1.3.1). Thanks! --- If your project is set up for it, you can reply to this email and have yo

[GitHub] spark pull request: [WIP][SPARK-3468] WebUI Timeline-View feature

2015-04-02 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2342#issuecomment-89177222 [Test build #29649 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29649/consoleFull) for PR 2342 at commit [`d05f2c2`](https://githu

[GitHub] spark pull request: [SPARK-6575][SQL] Converted Parquet Metastore ...

2015-04-02 Thread liancheng
Github user liancheng commented on the pull request: https://github.com/apache/spark/pull/5339#issuecomment-89174022 Merging to master and branch-1.3. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-6130] [SQL] support if not exists for i...

2015-04-02 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4865#issuecomment-89173730 [Test build #29646 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29646/consoleFull) for PR 4865 at commit [`5fbec76`](https://gith

[GitHub] spark pull request: [SPARK-6517][mllib] Implement the Algorithm of...

2015-04-02 Thread yu-iskw
Github user yu-iskw commented on a diff in the pull request: https://github.com/apache/spark/pull/5267#discussion_r27716372 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/clustering/HierarchicalClustering.scala --- @@ -0,0 +1,610 @@ +/* + * Licensed to the Apache So

[GitHub] spark pull request: [SPARK-6130] [SQL] support if not exists for i...

2015-04-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4865#issuecomment-89173754 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/29

[GitHub] spark pull request: [SPARK-6602][Core] Replace direct use of Akka ...

2015-04-02 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/5268#issuecomment-89172812 The changes mostly lgtm. Let's add the ThreadSafeRpcEndpoint interface for differentiation. --- If your project is set up for it, you can reply to this email and have your

[GitHub] spark pull request: [SPARK-6690] spark-sql script ends up throwing...

2015-04-02 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5341#issuecomment-89171715 [Test build #29648 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29648/consoleFull) for PR 5341 at commit [`d05f4d1`](https://githu

[GitHub] spark pull request: spark-sql script ends up throwing Exception wh...

2015-04-02 Thread sarutak
GitHub user sarutak opened a pull request: https://github.com/apache/spark/pull/5341 spark-sql script ends up throwing Exception when event logging is enabled. When event logging is enabled, spark-sql script ends up throwing Exception like as follows. ``` 15/04/03 13:51

[GitHub] spark pull request: [SPARK-6602][Core] Replace direct use of Akka ...

2015-04-02 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/5268#issuecomment-89170592 I like the idea of ThreadSafeRpcEndpoint. I need to think a little bit more about send/sendWithReply. I do find it error prone as is. --- If your project is set u

[GitHub] spark pull request: [SPARK-6638] [SQL] Improve performance of Stri...

2015-04-02 Thread davies
Github user davies commented on the pull request: https://github.com/apache/spark/pull/5303#issuecomment-89169864 @rxin I will create a new PR after fixing the rest tests. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. I

[GitHub] spark pull request: [SPARK-6211][Streaming] Add Python Kafka API u...

2015-04-02 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/4961#discussion_r27716148 --- Diff: external/kafka/src/main/scala/org/apache/spark/streaming/kafka/KafkaTestUtils.scala --- @@ -0,0 +1,272 @@ +/* + * Licensed to the Apache Softw

[GitHub] spark pull request: [SPARK-6211][Streaming] Add Python Kafka API u...

2015-04-02 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/4961#discussion_r27716103 --- Diff: external/kafka/src/main/scala/org/apache/spark/streaming/kafka/KafkaTestUtils.scala --- @@ -0,0 +1,272 @@ +/* + * Licensed to the Apache Softw

[GitHub] spark pull request: [SPARK-6211][Streaming] Add Python Kafka API u...

2015-04-02 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/4961#discussion_r27715991 --- Diff: external/kafka/src/main/scala/org/apache/spark/streaming/kafka/KafkaTestUtils.scala --- @@ -0,0 +1,272 @@ +/* + * Licensed to the Apache Softw

[GitHub] spark pull request: [SPARK-6211][Streaming] Add Python Kafka API u...

2015-04-02 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/4961#discussion_r27715996 --- Diff: external/kafka/src/main/scala/org/apache/spark/streaming/kafka/KafkaTestUtils.scala --- @@ -0,0 +1,272 @@ +/* + * Licensed to the Apache Softw

[GitHub] spark pull request: [SPARK-6638] [SQL] Improve performance of Stri...

2015-04-02 Thread davies
Github user davies commented on a diff in the pull request: https://github.com/apache/spark/pull/5303#discussion_r27715679 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/ScalaReflection.scala --- @@ -80,6 +82,24 @@ trait ScalaReflection { case (other,

[GitHub] spark pull request: [SPARK-6345][STREAMING][MLLIB] Fix for trainin...

2015-04-02 Thread freeman-lab
Github user freeman-lab commented on the pull request: https://github.com/apache/spark/pull/5037#issuecomment-89164396 Great thanks @tdas @mengxr ! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-4346][SPARK-3596][YARN] Commonize the m...

2015-04-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/5305#issuecomment-89164379 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/29

[GitHub] spark pull request: [SPARK-4346][SPARK-3596][YARN] Commonize the m...

2015-04-02 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5305#issuecomment-89164373 [Test build #29644 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29644/consoleFull) for PR 5305 at commit [`ee2b2fd`](https://gith

[GitHub] spark pull request: [SPARK-4346][SPARK-3596][YARN] Commonize the m...

2015-04-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/5305#issuecomment-89164248 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/29

[GitHub] spark pull request: [SPARK-4346][SPARK-3596][YARN] Commonize the m...

2015-04-02 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5305#issuecomment-89164239 [Test build #29643 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29643/consoleFull) for PR 5305 at commit [`6483a2a`](https://gith

[GitHub] spark pull request: [SPARK-3591][YARN]fire and forget for YARN clu...

2015-04-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/5297#issuecomment-89163033 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/29

[GitHub] spark pull request: [SPARK-3591][YARN]fire and forget for YARN clu...

2015-04-02 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5297#issuecomment-89163014 [Test build #29642 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29642/consoleFull) for PR 5297 at commit [`fea390d`](https://gith

[GitHub] spark pull request: [SPARK-6575][SQL] Converted Parquet Metastore ...

2015-04-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/5339#issuecomment-89162147 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/29

[GitHub] spark pull request: [SPARK-6575][SQL] Converted Parquet Metastore ...

2015-04-02 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5339#issuecomment-89162142 [Test build #29641 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29641/consoleFull) for PR 5339 at commit [`b0e1a42`](https://gith

[GitHub] spark pull request: [MLLIB] Remove println in LogisticRegression.s...

2015-04-02 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5338#issuecomment-89161761 [Test build #29647 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29647/consoleFull) for PR 5338 at commit [`3f3dbf4`](https://githu

[GitHub] spark pull request: [MLLIB] Remove println in LogisticRegression.s...

2015-04-02 Thread mengxr
Github user mengxr commented on the pull request: https://github.com/apache/spark/pull/5338#issuecomment-89161142 ok to test --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabl

[GitHub] spark pull request: [SPARK-6345][STREAMING][MLLIB] Fix for trainin...

2015-04-02 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/5037#discussion_r27715247 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/regression/StreamingLinearRegressionWithSGD.scala --- @@ -59,6 +59,8 @@ class StreamingLinearRegressionW

[GitHub] spark pull request: [SPARK-6345][STREAMING][MLLIB] Fix for trainin...

2015-04-02 Thread mengxr
Github user mengxr commented on the pull request: https://github.com/apache/spark/pull/5037#issuecomment-89160576 @tdas Thanks for the reminder! LGTM. I've merged it into both master and branch-1.3. --- If your project is set up for it, you can reply to this email and have your reply

[GitHub] spark pull request: [SPARK-6621][Core] Fix the bug that calling Ev...

2015-04-02 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/5280#issuecomment-89158963 @JoshRosen does this look good? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-6130] [SQL] support if not exists for i...

2015-04-02 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4865#issuecomment-89157982 [Test build #29646 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29646/consoleFull) for PR 4865 at commit [`5fbec76`](https://githu

[GitHub] spark pull request: [SPARK-6662][YARN] Allow variable substitution...

2015-04-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/5321#issuecomment-89156160 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/29

[GitHub] spark pull request: [SPARK-6662][YARN] Allow variable substitution...

2015-04-02 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5321#issuecomment-89156138 [Test build #29640 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29640/consoleFull) for PR 5321 at commit [`10e2917`](https://gith

[GitHub] spark pull request: [SPARK-6130] [SQL] support if not exists for i...

2015-04-02 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4865#issuecomment-89155246 [Test build #29645 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29645/consoleFull) for PR 4865 at commit [`dc911cd`](https://githu

[GitHub] spark pull request: [SPARK-6640][Core] Fix the race condition of c...

2015-04-02 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/5306#issuecomment-89150977 @zsxwing this looks fine, though a little funky because we send `TaskScheduler` and all of its pointers through Akka, though I don't see another simpler alternative. I

[GitHub] spark pull request: [SPARK-6640][Core] Fix the race condition of c...

2015-04-02 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/5306#discussion_r27714109 --- Diff: core/src/main/scala/org/apache/spark/HeartbeatReceiver.scala --- @@ -71,12 +75,22 @@ private[spark] class HeartbeatReceiver(sc: SparkContext, sch

[GitHub] spark pull request: [SPARK-6640][Core] Fix the race condition of c...

2015-04-02 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/5306#discussion_r27714088 --- Diff: core/src/main/scala/org/apache/spark/SparkContext.scala --- @@ -356,11 +356,17 @@ class SparkContext(config: SparkConf) extends Logging with Exec

[GitHub] spark pull request: [SPARK-4346][SPARK-3596][YARN] Commonize the m...

2015-04-02 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5305#issuecomment-89149972 [Test build #29644 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29644/consoleFull) for PR 5305 at commit [`ee2b2fd`](https://githu

[GitHub] spark pull request: [SPARK-4346][SPARK-3596][YARN] Commonize the m...

2015-04-02 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5305#issuecomment-89148868 [Test build #29643 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29643/consoleFull) for PR 5305 at commit [`6483a2a`](https://githu

[GitHub] spark pull request: [SPARK-6640][Core] Fix the race condition of c...

2015-04-02 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/5306#discussion_r27713951 --- Diff: core/src/main/scala/org/apache/spark/HeartbeatReceiver.scala --- @@ -71,12 +75,22 @@ private[spark] class HeartbeatReceiver(sc: SparkContext, sch

[GitHub] spark pull request: [SPARK-3591][YARN]fire and forget for YARN clu...

2015-04-02 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5297#issuecomment-89148335 [Test build #29642 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29642/consoleFull) for PR 5297 at commit [`fea390d`](https://githu

[GitHub] spark pull request: [SPARK-3591][YARN]fire and forget for YARN clu...

2015-04-02 Thread WangTaoTheTonic
Github user WangTaoTheTonic commented on the pull request: https://github.com/apache/spark/pull/5297#issuecomment-89147994 @tgravescs IIUC the output diagnostics message should be: >15/04/03 11:18:47 INFO YarnClientImpl: Submitted application application_1427257505534_0029 15/0

[GitHub] spark pull request: [SPARK-6575][SQL] Converted Parquet Metastore ...

2015-04-02 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5339#issuecomment-89146980 [Test build #29641 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29641/consoleFull) for PR 5339 at commit [`b0e1a42`](https://githu

[GitHub] spark pull request: [SPARK-6345][STREAMING][MLLIB] Fix for trainin...

2015-04-02 Thread tdas
Github user tdas commented on the pull request: https://github.com/apache/spark/pull/5037#issuecomment-89145346 @mengxr Shouldnt this be merged for 1.3.1? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project do

[GitHub] spark pull request: [SPARK-6575][SQL] Converted Parquet Metastore ...

2015-04-02 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/5339#issuecomment-89144498 Given these comments were mostly stylistic and we want to make the 1.3.1 release ASAP, I'm going to merge and let the cleanup be done in a follow-up. Thanks! --- If y

[GitHub] spark pull request: [CORE] The descriptionof jobHistory config sho...

2015-04-02 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/5332#issuecomment-89144361 Oops my mistake! LGTM I'm merging this into master and 1.2 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as we

[GitHub] spark pull request: SPARK-6414: Spark driver failed with NPE on jo...

2015-04-02 Thread hunglin
Github user hunglin commented on the pull request: https://github.com/apache/spark/pull/5124#issuecomment-89140693 @JoshRosen, no problem. I'm happy to contribute. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-6688] [core] Always use resolved URIs i...

2015-04-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/5340#issuecomment-89139464 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/29

[GitHub] spark pull request: [SPARK-6688] [core] Always use resolved URIs i...

2015-04-02 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5340#issuecomment-89139445 [Test build #29639 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29639/consoleFull) for PR 5340 at commit [`20d2a34`](https://gith

[GitHub] spark pull request: [SPARK-6575][SQL] Converted Parquet Metastore ...

2015-04-02 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5339#issuecomment-89137738 [Test build #29638 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29638/consoleFull) for PR 5339 at commit [`83d9846`](https://gith

[GitHub] spark pull request: [SPARK-6575][SQL] Converted Parquet Metastore ...

2015-04-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/5339#issuecomment-89137743 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/29

  1   2   3   4   5   >