[GitHub] spark issue #18101: [SPARK-20874][Examples]Add Structured Streaming Kafka So...

2017-05-25 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/18101 Thanks. Merging to master, 2.2 and 2.1. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature ena

[GitHub] spark issue #18051: [SPARK-18825][SPARKR][DOCS][WIP] Eliminate duplicate lin...

2017-05-25 Thread actuaryzhang
Github user actuaryzhang commented on the issue: https://github.com/apache/spark/pull/18051 That makes sense! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark pull request #18101: [SPARK-20874][Examples]Add Structured Streaming K...

2017-05-25 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/18101 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[GitHub] spark issue #18094: [Spark-20775][SQL] Added scala support from_json

2017-05-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18094 **[Test build #77379 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/77379/testReport)** for PR 18094 at commit [`a2f99ec`](https://github.com/apache/spark/commit/a2

[GitHub] spark issue #18105: [SPARK-20881] [SQL] Use Hive's stats in metastore when c...

2017-05-25 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/18105 If users have not analyzed the table in Spark yet, we should respect the stats from hive metastore. But if users have already run the analyze table command in Spark, I think it's fair to ask them

[GitHub] spark pull request #18112: [SPARK-20888][SQL][DOCS] Document change of defau...

2017-05-25 Thread mallman
GitHub user mallman opened a pull request: https://github.com/apache/spark/pull/18112 [SPARK-20888][SQL][DOCS] Document change of default setting of spark.sql.hive.caseSensitiveInferenceMode (Link to Jira: https://issues.apache.org/jira/browse/SPARK-20888) ## What changes w

[GitHub] spark issue #18094: [Spark-20775][SQL] Added scala support from_json

2017-05-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18094 **[Test build #77379 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/77379/testReport)** for PR 18094 at commit [`a2f99ec`](https://github.com/apache/spark/commit/a

[GitHub] spark issue #18094: [Spark-20775][SQL] Added scala support from_json

2017-05-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18094 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/77379/ Test FAILed. ---

[GitHub] spark pull request #11746: [SPARK-13602][CORE] Add shutdown hook to DriverRu...

2017-05-25 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/11746#discussion_r118544204 --- Diff: core/src/main/scala/org/apache/spark/deploy/worker/DriverRunner.scala --- @@ -53,9 +53,11 @@ private[deploy] class DriverRunner( @volatile

[GitHub] spark issue #18094: [Spark-20775][SQL] Added scala support from_json

2017-05-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18094 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #18112: [SPARK-20888][SQL][DOCS] Document change of default sett...

2017-05-25 Thread mallman
Github user mallman commented on the issue: https://github.com/apache/spark/pull/18112 @budde Can you please review (urgently) for inclusion as a migration note for 2.2? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If

[GitHub] spark issue #18112: [SPARK-20888][SQL][DOCS] Document change of default sett...

2017-05-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18112 **[Test build #77380 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/77380/testReport)** for PR 18112 at commit [`037e7d3`](https://github.com/apache/spark/commit/03

[GitHub] spark issue #18094: [Spark-20775][SQL] Added scala support from_json

2017-05-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18094 **[Test build #77381 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/77381/testReport)** for PR 18094 at commit [`27a8c26`](https://github.com/apache/spark/commit/27

[GitHub] spark issue #18092: [SPARK-20640][CORE]Make rpc timeout and retry for shuffl...

2017-05-25 Thread sitalkedia
Github user sitalkedia commented on the issue: https://github.com/apache/spark/pull/18092 >> I can not think of meaningful test cases, are there any suggestions? How about just "unit tests" ? --- If your project is set up for it, you can reply to this email and have your repl

[GitHub] spark issue #18110: [SPARK-20887][CORE] support alternative keys in ConfigBu...

2017-05-25 Thread JoshRosen
Github user JoshRosen commented on the issue: https://github.com/apache/spark/pull/18110 @cloud-fan, what about `SparkConf`'s `configsWithAlternatives `: https://github.com/apache/spark/blob/master/core/src/main/scala/org/apache/spark/SparkConf.scala#L596 --- If your project is set

[GitHub] spark pull request #11974: [SPARK-14174][ML] Accelerate KMeans via Mini-Batc...

2017-05-25 Thread MLnick
Github user MLnick commented on a diff in the pull request: https://github.com/apache/spark/pull/11974#discussion_r118546602 --- Diff: mllib/src/test/scala/org/apache/spark/ml/clustering/KMeansSuite.scala --- @@ -89,6 +92,9 @@ class KMeansSuite extends SparkFunSuite with MLlibTest

[GitHub] spark issue #17471: [SPARK-3577] Report Spill size on disk for UnsafeExterna...

2017-05-25 Thread sitalkedia
Github user sitalkedia commented on the issue: https://github.com/apache/spark/pull/17471 @sameeragarwal - Thanks for taking a look. I will update the PR adding test case soon. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as

[GitHub] spark issue #18110: [SPARK-20887][CORE] support alternative keys in ConfigBu...

2017-05-25 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/18110 It's only used in the `SparkCong.get(key: String)` code path, not `SparkConf.get(entry: ConfigEntry[T])` code path. That's why we only support alternative keys if users get conf value by hard-code

[GitHub] spark pull request #18092: [SPARK-20640][CORE]Make rpc timeout and retry for...

2017-05-25 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/18092#discussion_r118547246 --- Diff: core/src/main/scala/org/apache/spark/storage/BlockManager.scala --- @@ -170,11 +170,17 @@ private[spark] class BlockManager( // service,

[GitHub] spark issue #16578: [SPARK-4502][SQL] Parquet nested column pruning

2017-05-25 Thread mallman
Github user mallman commented on the issue: https://github.com/apache/spark/pull/16578 Also, I'm confused about something—who has jenkins retest privileges? And can I get them? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as

[GitHub] spark issue #18112: [SPARK-20888][SQL][DOCS] Document change of default sett...

2017-05-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18112 **[Test build #77380 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/77380/testReport)** for PR 18112 at commit [`037e7d3`](https://github.com/apache/spark/commit/0

[GitHub] spark issue #18112: [SPARK-20888][SQL][DOCS] Document change of default sett...

2017-05-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18112 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/77380/ Test PASSed. ---

[GitHub] spark issue #18112: [SPARK-20888][SQL][DOCS] Document change of default sett...

2017-05-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18112 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark pull request #11974: [SPARK-14174][ML] Accelerate KMeans via Mini-Batc...

2017-05-25 Thread MLnick
Github user MLnick commented on a diff in the pull request: https://github.com/apache/spark/pull/11974#discussion_r118548583 --- Diff: mllib/src/main/scala/org/apache/spark/ml/clustering/KMeans.scala --- @@ -85,6 +85,20 @@ private[clustering] trait KMeansParams extends Params with

[GitHub] spark issue #18110: [SPARK-20887][CORE] support alternative keys in ConfigBu...

2017-05-25 Thread JoshRosen
Github user JoshRosen commented on the issue: https://github.com/apache/spark/pull/18110 Ahhh, makes sense. Thanks for the clarification. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request #11974: [SPARK-14174][ML] Accelerate KMeans via Mini-Batc...

2017-05-25 Thread MLnick
Github user MLnick commented on a diff in the pull request: https://github.com/apache/spark/pull/11974#discussion_r118548684 --- Diff: mllib/src/main/scala/org/apache/spark/ml/clustering/KMeans.scala --- @@ -85,6 +85,20 @@ private[clustering] trait KMeansParams extends Params with

[GitHub] spark pull request #11974: [SPARK-14174][ML] Accelerate KMeans via Mini-Batc...

2017-05-25 Thread MLnick
Github user MLnick commented on a diff in the pull request: https://github.com/apache/spark/pull/11974#discussion_r118548792 --- Diff: mllib/src/main/scala/org/apache/spark/ml/clustering/KMeans.scala --- @@ -85,6 +85,20 @@ private[clustering] trait KMeansParams extends Params with

[GitHub] spark issue #18064: [SPARK-20213][SQL] Fix DataFrameWriter operations in SQL...

2017-05-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18064 **[Test build #77374 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/77374/testReport)** for PR 18064 at commit [`def0878`](https://github.com/apache/spark/commit/d

[GitHub] spark issue #18064: [SPARK-20213][SQL] Fix DataFrameWriter operations in SQL...

2017-05-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18064 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/77374/ Test FAILed. ---

[GitHub] spark issue #18064: [SPARK-20213][SQL] Fix DataFrameWriter operations in SQL...

2017-05-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18064 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #11974: [SPARK-14174][ML] Accelerate KMeans via Mini-Batch EM

2017-05-25 Thread MLnick
Github user MLnick commented on the issue: https://github.com/apache/spark/pull/11974 cc @srowen @setha also --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] spark issue #18051: [SPARK-18825][SPARKR][DOCS][WIP] Eliminate duplicate lin...

2017-05-25 Thread zero323
Github user zero323 commented on the issue: https://github.com/apache/spark/pull/18051 Exactly my point. Run examples internally ([it is not hard to patch knitr](https://github.com/zero323/knitr/commit/7a0d8f9ddb9d77a9c235f25aca26131e83c1f6cc) or even `tools::Rd2ex`) to validate examp

[GitHub] spark pull request #18051: [SPARK-18825][SPARKR][DOCS][WIP] Eliminate duplic...

2017-05-25 Thread zero323
Github user zero323 closed the pull request at: https://github.com/apache/spark/pull/18051 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is en

[GitHub] spark issue #18112: [SPARK-20888][SQL][DOCS] Document change of default sett...

2017-05-25 Thread mallman
Github user mallman commented on the issue: https://github.com/apache/spark/pull/18112 CC @cloud-fan @ericl --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] spark issue #18083: [SPARK-20863] Add metrics/instrumentation to LiveListene...

2017-05-25 Thread JoshRosen
Github user JoshRosen commented on the issue: https://github.com/apache/spark/pull/18083 > I am not sure that monitoring (with real metrics) the number of dropped events really worth it. You just want to know if messages have been dropped (and having the number in the log is fine).

[GitHub] spark pull request #18098: [SPARK-16944][Mesos] Improve data locality when l...

2017-05-25 Thread mgummelt
Github user mgummelt commented on a diff in the pull request: https://github.com/apache/spark/pull/18098#discussion_r118550928 --- Diff: resource-managers/mesos/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosCoarseGrainedSchedulerBackend.scala --- @@ -393,7 +409,30 @

[GitHub] spark issue #17094: [SPARK-19762][ML] Hierarchy for consolidating ML aggrega...

2017-05-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17094 **[Test build #77376 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/77376/testReport)** for PR 17094 at commit [`29052d3`](https://github.com/apache/spark/commit/2

[GitHub] spark issue #17094: [SPARK-19762][ML] Hierarchy for consolidating ML aggrega...

2017-05-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17094 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/77376/ Test PASSed. ---

[GitHub] spark issue #17094: [SPARK-19762][ML] Hierarchy for consolidating ML aggrega...

2017-05-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17094 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #17343: [SPARK-20014] Optimize mergeSpillsWithFileStream method

2017-05-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17343 **[Test build #77382 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/77382/testReport)** for PR 17343 at commit [`d4f09c2`](https://github.com/apache/spark/commit/d4

[GitHub] spark issue #17864: [SPARK-20604][ML] Allow imputer to handle numeric types

2017-05-25 Thread MLnick
Github user MLnick commented on the issue: https://github.com/apache/spark/pull/17864 Originally the idea behind only supporting double was as @sethah posted above - there could be some issues with handling of int casting etc. As mentioned originally, we did consider "always cast to d

[GitHub] spark issue #18064: [SPARK-20213][SQL] Fix DataFrameWriter operations in SQL...

2017-05-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18064 **[Test build #77383 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/77383/testReport)** for PR 18064 at commit [`59ec0e9`](https://github.com/apache/spark/commit/59

[GitHub] spark issue #18025: [SPARK-20889][SparkR][WIP] Grouped documentation for agg...

2017-05-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18025 **[Test build #77378 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/77378/testReport)** for PR 18025 at commit [`21e54c0`](https://github.com/apache/spark/commit/2

[GitHub] spark issue #18025: [SPARK-20889][SparkR][WIP] Grouped documentation for agg...

2017-05-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18025 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #18025: [SPARK-20889][SparkR][WIP] Grouped documentation for agg...

2017-05-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18025 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/77378/ Test PASSed. ---

[GitHub] spark pull request #17343: [SPARK-20014] Optimize mergeSpillsWithFileStream ...

2017-05-25 Thread sitalkedia
Github user sitalkedia commented on a diff in the pull request: https://github.com/apache/spark/pull/17343#discussion_r118553894 --- Diff: core/src/main/java/org/apache/spark/shuffle/sort/UnsafeShuffleWriter.java --- @@ -339,23 +355,26 @@ void forceSorterToSpill() throws IOExceptio

[GitHub] spark issue #17343: [SPARK-20014] Optimize mergeSpillsWithFileStream method

2017-05-25 Thread sitalkedia
Github user sitalkedia commented on the issue: https://github.com/apache/spark/pull/17343 Thanks @sameeragarwal, addressed the check style issues. Yes, the exisiting unit tests in UnsafeShuffleWriter#mergeSpillsWithTransferToAndLZF covers this code. --- If your project is set up fo

[GitHub] spark issue #18064: [SPARK-20213][SQL] Fix DataFrameWriter operations in SQL...

2017-05-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18064 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/77383/ Test FAILed. ---

[GitHub] spark issue #18064: [SPARK-20213][SQL] Fix DataFrameWriter operations in SQL...

2017-05-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18064 **[Test build #77383 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/77383/testReport)** for PR 18064 at commit [`59ec0e9`](https://github.com/apache/spark/commit/5

[GitHub] spark issue #18064: [SPARK-20213][SQL] Fix DataFrameWriter operations in SQL...

2017-05-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18064 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #18064: [SPARK-20213][SQL] Fix DataFrameWriter operations in SQL...

2017-05-25 Thread rdblue
Github user rdblue commented on the issue: https://github.com/apache/spark/pull/18064 @cloud-fan, can you summarize how this differs from the original PR #17540? I have time to pick this up again, but I thought that the other PR only needed two changes: * Merge your c

[GitHub] spark issue #18110: [SPARK-20887][CORE] support alternative keys in ConfigBu...

2017-05-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18110 **[Test build #77372 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/77372/testReport)** for PR 18110 at commit [`cc51dd0`](https://github.com/apache/spark/commit/c

[GitHub] spark pull request #18065: [SPARK-20844] Remove experimental from Structured...

2017-05-25 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/18065#discussion_r118555796 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/streaming/DataStreamReader.scala --- @@ -35,7 +35,6 @@ import org.apache.spark.sql.types.StructType

[GitHub] spark issue #18110: [SPARK-20887][CORE] support alternative keys in ConfigBu...

2017-05-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18110 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #18110: [SPARK-20887][CORE] support alternative keys in ConfigBu...

2017-05-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18110 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/77372/ Test PASSed. ---

[GitHub] spark issue #18111: [SPARK-20886][CORE] HadoopMapReduceCommitProtocol to fai...

2017-05-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18111 **[Test build #77373 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/77373/testReport)** for PR 18111 at commit [`02eb7bf`](https://github.com/apache/spark/commit/0

[GitHub] spark issue #18111: [SPARK-20886][CORE] HadoopMapReduceCommitProtocol to fai...

2017-05-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18111 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #18111: [SPARK-20886][CORE] HadoopMapReduceCommitProtocol to fai...

2017-05-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18111 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/77373/ Test PASSed. ---

[GitHub] spark pull request #18098: [SPARK-16944][Mesos] Improve data locality when l...

2017-05-25 Thread mgummelt
Github user mgummelt commented on a diff in the pull request: https://github.com/apache/spark/pull/18098#discussion_r118558695 --- Diff: resource-managers/mesos/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosCoarseGrainedSchedulerBackend.scala --- @@ -393,7 +410,30 @

[GitHub] spark issue #18014: [SPARK-20783][SQL] Enhance ColumnVector to keep UnsafeAr...

2017-05-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18014 **[Test build #77384 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/77384/testReport)** for PR 18014 at commit [`bf6ab20`](https://github.com/apache/spark/commit/bf

[GitHub] spark issue #18098: [SPARK-16944][Mesos] Improve data locality when launchin...

2017-05-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18098 **[Test build #77375 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/77375/testReport)** for PR 18098 at commit [`3e656e5`](https://github.com/apache/spark/commit/3

[GitHub] spark issue #18098: [SPARK-16944][Mesos] Improve data locality when launchin...

2017-05-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18098 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #18098: [SPARK-16944][Mesos] Improve data locality when launchin...

2017-05-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18098 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/77375/ Test PASSed. ---

[GitHub] spark issue #18014: [SPARK-20783][SQL] Enhance ColumnVector to keep UnsafeAr...

2017-05-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18014 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #18014: [SPARK-20783][SQL] Enhance ColumnVector to keep UnsafeAr...

2017-05-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18014 **[Test build #77384 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/77384/testReport)** for PR 18014 at commit [`bf6ab20`](https://github.com/apache/spark/commit/b

[GitHub] spark issue #18014: [SPARK-20783][SQL] Enhance ColumnVector to keep UnsafeAr...

2017-05-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18014 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/77384/ Test FAILed. ---

[GitHub] spark issue #18097: [Spark-20873][SQL] Improve the error message for unsuppo...

2017-05-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18097 **[Test build #77377 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/77377/testReport)** for PR 18097 at commit [`90433a9`](https://github.com/apache/spark/commit/9

[GitHub] spark issue #18097: [Spark-20873][SQL] Improve the error message for unsuppo...

2017-05-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18097 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/77377/ Test PASSed. ---

[GitHub] spark issue #18097: [Spark-20873][SQL] Improve the error message for unsuppo...

2017-05-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18097 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #18014: [SPARK-20783][SQL] Enhance ColumnVector to keep UnsafeAr...

2017-05-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18014 **[Test build #77385 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/77385/testReport)** for PR 18014 at commit [`9954d6b`](https://github.com/apache/spark/commit/99

[GitHub] spark issue #18107: [SPARK-20883][SPARK-20376][SS] Refactored StateStore API...

2017-05-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18107 **[Test build #77386 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/77386/testReport)** for PR 18107 at commit [`324fc24`](https://github.com/apache/spark/commit/32

[GitHub] spark pull request #14269: [SPARK-15703] [Scheduler][Core][WebUI] Make Liste...

2017-05-25 Thread dhruve
Github user dhruve commented on a diff in the pull request: https://github.com/apache/spark/pull/14269#discussion_r118573078 --- Diff: core/src/main/scala/org/apache/spark/scheduler/LiveListenerBus.scala --- @@ -32,18 +33,24 @@ import org.apache.spark.util.Utils * has started

[GitHub] spark pull request #14269: [SPARK-15703] [Scheduler][Core][WebUI] Make Liste...

2017-05-25 Thread dhruve
Github user dhruve commented on a diff in the pull request: https://github.com/apache/spark/pull/14269#discussion_r118573206 --- Diff: core/src/main/scala/org/apache/spark/internal/config/package.scala --- @@ -103,4 +103,9 @@ package object config { .stringConf .c

[GitHub] spark issue #18094: [Spark-20775][SQL] Added scala support from_json

2017-05-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18094 **[Test build #77381 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/77381/testReport)** for PR 18094 at commit [`27a8c26`](https://github.com/apache/spark/commit/2

[GitHub] spark issue #18094: [Spark-20775][SQL] Added scala support from_json

2017-05-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18094 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #18094: [Spark-20775][SQL] Added scala support from_json

2017-05-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18094 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/77381/ Test PASSed. ---

[GitHub] spark issue #11974: [SPARK-14174][ML] Accelerate KMeans via Mini-Batch EM

2017-05-25 Thread sethah
Github user sethah commented on the issue: https://github.com/apache/spark/pull/11974 Mini-batching in Spark generally isn't that efficient, since to extract a mini-batch you still need to iterate over the entire dataset - and that means reading it from disk if it doesn't fit into mem

[GitHub] spark pull request #18078: [SPARK-10643] Make spark-submit download remote f...

2017-05-25 Thread jiangxb1987
Github user jiangxb1987 commented on a diff in the pull request: https://github.com/apache/spark/pull/18078#discussion_r118580623 --- Diff: core/src/test/scala/org/apache/spark/deploy/SparkSubmitSuite.scala --- @@ -535,7 +538,7 @@ class SparkSubmitSuite test("resolves

[GitHub] spark pull request #18078: [SPARK-10643] Make spark-submit download remote f...

2017-05-25 Thread jiangxb1987
Github user jiangxb1987 commented on a diff in the pull request: https://github.com/apache/spark/pull/18078#discussion_r118580006 --- Diff: core/src/main/scala/org/apache/spark/deploy/SparkSubmit.scala --- @@ -308,6 +311,15 @@ object SparkSubmit extends CommandLineUtils {

[GitHub] spark pull request #18098: [SPARK-16944][Mesos] Improve data locality when l...

2017-05-25 Thread gpang
Github user gpang commented on a diff in the pull request: https://github.com/apache/spark/pull/18098#discussion_r118581400 --- Diff: resource-managers/mesos/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosCoarseGrainedSchedulerBackend.scala --- @@ -393,7 +409,30 @@ p

[GitHub] spark pull request #18098: [SPARK-16944][Mesos] Improve data locality when l...

2017-05-25 Thread gpang
Github user gpang commented on a diff in the pull request: https://github.com/apache/spark/pull/18098#discussion_r118581450 --- Diff: resource-managers/mesos/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosCoarseGrainedSchedulerBackend.scala --- @@ -393,7 +410,30 @@ p

[GitHub] spark issue #18078: [SPARK-10643] Make spark-submit download remote files to...

2017-05-25 Thread jiangxb1987
Github user jiangxb1987 commented on the issue: https://github.com/apache/spark/pull/18078 Could you also add "[Core]" tag in the title? @loneknightpy Also cc @cloud-fan @gatorsmile --- If your project is set up for it, you can reply to this email and have your reply appear on G

[GitHub] spark issue #18098: [SPARK-16944][Mesos] Improve data locality when launchin...

2017-05-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18098 **[Test build #77387 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/77387/testReport)** for PR 18098 at commit [`fa2daff`](https://github.com/apache/spark/commit/fa

[GitHub] spark pull request #18078: [SPARK-10643] Make spark-submit download remote f...

2017-05-25 Thread loneknightpy
Github user loneknightpy commented on a diff in the pull request: https://github.com/apache/spark/pull/18078#discussion_r118583624 --- Diff: core/src/test/scala/org/apache/spark/deploy/SparkSubmitSuite.scala --- @@ -535,7 +538,7 @@ class SparkSubmitSuite test("resolves

[GitHub] spark pull request #18098: [SPARK-16944][Mesos] Improve data locality when l...

2017-05-25 Thread mgummelt
Github user mgummelt commented on a diff in the pull request: https://github.com/apache/spark/pull/18098#discussion_r118586090 --- Diff: resource-managers/mesos/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosCoarseGrainedSchedulerBackend.scala --- @@ -502,6 +521,25 @

[GitHub] spark issue #17343: [SPARK-20014] Optimize mergeSpillsWithFileStream method

2017-05-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17343 **[Test build #77382 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/77382/testReport)** for PR 17343 at commit [`d4f09c2`](https://github.com/apache/spark/commit/d

[GitHub] spark issue #17343: [SPARK-20014] Optimize mergeSpillsWithFileStream method

2017-05-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17343 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #17343: [SPARK-20014] Optimize mergeSpillsWithFileStream method

2017-05-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17343 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/77382/ Test PASSed. ---

[GitHub] spark pull request #18113: [SPARK-20890][SQL] Added min and max typed aggreg...

2017-05-25 Thread setjet
GitHub user setjet opened a pull request: https://github.com/apache/spark/pull/18113 [SPARK-20890][SQL] Added min and max typed aggregation functions ## What changes were proposed in this pull request? Typed Min and Max functions are missing for aggregations done on dataset. The

[GitHub] spark issue #18113: [SPARK-20890][SQL] Added min and max typed aggregation f...

2017-05-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18113 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feat

[GitHub] spark issue #18078: [SPARK-10643] [Core] Make spark-submit download remote f...

2017-05-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18078 **[Test build #77388 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/77388/testReport)** for PR 18078 at commit [`e5171ca`](https://github.com/apache/spark/commit/e5

[GitHub] spark pull request #18098: [SPARK-16944][Mesos] Improve data locality when l...

2017-05-25 Thread gpang
Github user gpang commented on a diff in the pull request: https://github.com/apache/spark/pull/18098#discussion_r118590493 --- Diff: resource-managers/mesos/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosCoarseGrainedSchedulerBackend.scala --- @@ -502,6 +521,25 @@ p

[GitHub] spark issue #18078: [SPARK-10643] [Core] Make spark-submit download remote f...

2017-05-25 Thread jiangxb1987
Github user jiangxb1987 commented on the issue: https://github.com/apache/spark/pull/18078 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the f

[GitHub] spark pull request #18078: [SPARK-10643] [Core] Make spark-submit download r...

2017-05-25 Thread jiangxb1987
Github user jiangxb1987 commented on a diff in the pull request: https://github.com/apache/spark/pull/18078#discussion_r118593252 --- Diff: core/src/test/scala/org/apache/spark/deploy/SparkSubmitSuite.scala --- @@ -535,7 +538,7 @@ class SparkSubmitSuite test("resolves

[GitHub] spark issue #18078: [SPARK-10643] [Core] Make spark-submit download remote f...

2017-05-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18078 **[Test build #77389 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/77389/testReport)** for PR 18078 at commit [`62e57df`](https://github.com/apache/spark/commit/62

[GitHub] spark pull request #18107: [SPARK-20883][SPARK-20376][SS] Refactored StateSt...

2017-05-25 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/18107#discussion_r118595928 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/streaming/FlatMapGroupsWithStateSuite.scala --- @@ -508,22 +508,6 @@ class FlatMapGroupsWithStateSuite

[GitHub] spark issue #18014: [SPARK-20783][SQL] Enhance ColumnVector to keep UnsafeAr...

2017-05-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18014 **[Test build #77385 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/77385/testReport)** for PR 18014 at commit [`9954d6b`](https://github.com/apache/spark/commit/9

[GitHub] spark issue #18014: [SPARK-20783][SQL] Enhance ColumnVector to keep UnsafeAr...

2017-05-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18014 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #18014: [SPARK-20783][SQL] Enhance ColumnVector to keep UnsafeAr...

2017-05-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18014 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/77385/ Test PASSed. ---

[GitHub] spark issue #18107: [SPARK-20883][SPARK-20376][SS] Refactored StateStore API...

2017-05-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18107 **[Test build #77386 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/77386/testReport)** for PR 18107 at commit [`324fc24`](https://github.com/apache/spark/commit/3

<    1   2   3   4   5   6   >