[GitHub] spark pull request #13939: [SPARK-16248][SQL] Whitelist the list of Hive fal...

2016-06-28 Thread yhuai
Github user yhuai commented on a diff in the pull request: https://github.com/apache/spark/pull/13939#discussion_r68843139 --- Diff: sql/hive/compatibility/src/test/scala/org/apache/spark/sql/hive/execution/HiveWindowFunctionQuerySuite.scala --- @@ -569,6 +572,7 @@ class HiveWindo

[GitHub] spark pull request #13945: [SPARK-16256][SQL][STREAMING] Added Structured St...

2016-06-28 Thread koeninger
Github user koeninger commented on a diff in the pull request: https://github.com/apache/spark/pull/13945#discussion_r68842976 --- Diff: docs/structured-streaming-programming-guide.md --- @@ -0,0 +1,888 @@ +--- +layout: global +displayTitle: Structured Streaming Programm

[GitHub] spark pull request #13945: [SPARK-16256][SQL][STREAMING] Added Structured St...

2016-06-28 Thread koeninger
Github user koeninger commented on a diff in the pull request: https://github.com/apache/spark/pull/13945#discussion_r68842804 --- Diff: docs/structured-streaming-programming-guide.md --- @@ -0,0 +1,888 @@ +--- +layout: global +displayTitle: Structured Streaming Programm

[GitHub] spark pull request #13950: [SPARK-15487] [Web UI] Spark Master UI to reverse...

2016-06-28 Thread ajbozarth
Github user ajbozarth commented on a diff in the pull request: https://github.com/apache/spark/pull/13950#discussion_r68842668 --- Diff: core/src/main/scala/org/apache/spark/deploy/master/Master.scala --- @@ -127,7 +128,14 @@ private[deploy] class Master( logInfo(s"Running

[GitHub] spark pull request #13945: [SPARK-16256][SQL][STREAMING] Added Structured St...

2016-06-28 Thread koeninger
Github user koeninger commented on a diff in the pull request: https://github.com/apache/spark/pull/13945#discussion_r68842633 --- Diff: docs/structured-streaming-programming-guide.md --- @@ -0,0 +1,888 @@ +--- +layout: global +displayTitle: Structured Streaming Programm

[GitHub] spark issue #13921: [SPARK-16140][MLlib][SparkR][Docs] Group k-means method ...

2016-06-28 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13921 **[Test build #61410 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61410/consoleFull)** for PR 13921 at commit [`402026c`](https://github.com/apache/spark/commit/4

[GitHub] spark issue #13878: [SPARK-16175] [PYSPARK] handle None for UDT

2016-06-28 Thread JoshRosen
Github user JoshRosen commented on the issue: https://github.com/apache/spark/pull/13878 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the fea

[GitHub] spark pull request #13950: [SPARK-15487] [Web UI] Spark Master UI to reverse...

2016-06-28 Thread ajbozarth
Github user ajbozarth commented on a diff in the pull request: https://github.com/apache/spark/pull/13950#discussion_r68842364 --- Diff: core/src/main/scala/org/apache/spark/deploy/master/Master.scala --- @@ -127,7 +128,14 @@ private[deploy] class Master( logInfo(s"Running

[GitHub] spark pull request #13945: [SPARK-16256][SQL][STREAMING] Added Structured St...

2016-06-28 Thread koeninger
Github user koeninger commented on a diff in the pull request: https://github.com/apache/spark/pull/13945#discussion_r68842326 --- Diff: docs/structured-streaming-programming-guide.md --- @@ -0,0 +1,888 @@ +--- +layout: global +displayTitle: Structured Streaming Programm

[GitHub] spark issue #13824: [SPARK-16110][YARN][PYSPARK] Fix allowing python version...

2016-06-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13824 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #13824: [SPARK-16110][YARN][PYSPARK] Fix allowing python version...

2016-06-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13824 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/61407/ Test PASSed. ---

[GitHub] spark issue #13824: [SPARK-16110][YARN][PYSPARK] Fix allowing python version...

2016-06-28 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13824 **[Test build #61407 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61407/consoleFull)** for PR 13824 at commit [`4ad7904`](https://github.com/apache/spark/commit/

[GitHub] spark pull request #13945: [SPARK-16256][SQL][STREAMING] Added Structured St...

2016-06-28 Thread koeninger
Github user koeninger commented on a diff in the pull request: https://github.com/apache/spark/pull/13945#discussion_r68841970 --- Diff: docs/structured-streaming-programming-guide.md --- @@ -0,0 +1,888 @@ +--- +layout: global +displayTitle: Structured Streaming Programm

[GitHub] spark issue #13953: [SPARK-16265][SUBMIT] Addition of --driver-jre Flag to S...

2016-06-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13953 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feat

[GitHub] spark pull request #13945: [SPARK-16256][SQL][STREAMING] Added Structured St...

2016-06-28 Thread koeninger
Github user koeninger commented on a diff in the pull request: https://github.com/apache/spark/pull/13945#discussion_r68841644 --- Diff: docs/structured-streaming-programming-guide.md --- @@ -0,0 +1,888 @@ +--- +layout: global +displayTitle: Structured Streaming Programm

[GitHub] spark pull request #13939: [SPARK-16248][SQL] Whitelist the list of Hive fal...

2016-06-28 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/13939#discussion_r68841421 --- Diff: sql/hive/compatibility/src/test/scala/org/apache/spark/sql/hive/execution/HiveWindowFunctionQuerySuite.scala --- @@ -569,6 +572,7 @@ class HiveWi

[GitHub] spark pull request #13945: [SPARK-16256][SQL][STREAMING] Added Structured St...

2016-06-28 Thread koeninger
Github user koeninger commented on a diff in the pull request: https://github.com/apache/spark/pull/13945#discussion_r68841415 --- Diff: docs/structured-streaming-programming-guide.md --- @@ -0,0 +1,888 @@ +--- +layout: global +displayTitle: Structured Streaming Programm

[GitHub] spark pull request #13953: [SPARK-16265][SUBMIT] Addition of --driver-jre Fl...

2016-06-28 Thread andreweduffy
GitHub user andreweduffy opened a pull request: https://github.com/apache/spark/pull/13953 [SPARK-16265][SUBMIT] Addition of --driver-jre Flag to SparkSubmit ## What changes were proposed in this pull request? 1. Addition of `--driver-jre` flag to `SparkSubmitArguments`

[GitHub] spark issue #13924: [SPARK-15643][DOC][ML] Add breaking changes to ML migrat...

2016-06-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13924 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/61408/ Test PASSed. ---

[GitHub] spark issue #13924: [SPARK-15643][DOC][ML] Add breaking changes to ML migrat...

2016-06-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13924 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #13924: [SPARK-15643][DOC][ML] Add breaking changes to ML migrat...

2016-06-28 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13924 **[Test build #61408 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61408/consoleFull)** for PR 13924 at commit [`6ef09a3`](https://github.com/apache/spark/commit/

[GitHub] spark issue #13906: [SPARK-16208][SQL] Add `CollapseEmptyPlan` optimizer

2016-06-28 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13906 **[Test build #61409 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61409/consoleFull)** for PR 13906 at commit [`bd79a14`](https://github.com/apache/spark/commit/b

[GitHub] spark pull request #13939: [SPARK-16248][SQL] Whitelist the list of Hive fal...

2016-06-28 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/13939#discussion_r68840940 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveSessionCatalog.scala --- @@ -196,6 +185,10 @@ private[sql] class HiveSessionCatalog(

[GitHub] spark issue #13906: [SPARK-16208][SQL] Add `CollapseEmptyPlan` optimizer

2016-06-28 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/13906 Yep. It's done! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] spark pull request #13945: [SPARK-16256][SQL][STREAMING] Added Structured St...

2016-06-28 Thread koeninger
Github user koeninger commented on a diff in the pull request: https://github.com/apache/spark/pull/13945#discussion_r68840717 --- Diff: docs/structured-streaming-programming-guide.md --- @@ -0,0 +1,888 @@ +--- +layout: global +displayTitle: Structured Streaming Programm

[GitHub] spark pull request #13945: [SPARK-16256][SQL][STREAMING] Added Structured St...

2016-06-28 Thread koeninger
Github user koeninger commented on a diff in the pull request: https://github.com/apache/spark/pull/13945#discussion_r68840643 --- Diff: docs/structured-streaming-programming-guide.md --- @@ -0,0 +1,888 @@ +--- +layout: global +displayTitle: Structured Streaming Programm

[GitHub] spark issue #13765: [SPARK-16052][SQL] Improve `CollapseRepartition` optimiz...

2016-06-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13765 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #13765: [SPARK-16052][SQL] Improve `CollapseRepartition` optimiz...

2016-06-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13765 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/61398/ Test PASSed. ---

[GitHub] spark issue #13915: [SPARK-16081][BUILD] Disallow using `l` as variable name

2016-06-28 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/13915 BTW it is not just I. It's also the number "1" that can get mixed up with l. I don't think we need a rule to exclude "I", since nobody uses a single capital I as identifier. So perhaps we can j

[GitHub] spark issue #13740: [SPARK-16020][SQL]Fix complete mode aggregation with con...

2016-06-28 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/13740 @zsxwing great I've left the similar comment on the that PR --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not h

[GitHub] spark issue #13765: [SPARK-16052][SQL] Improve `CollapseRepartition` optimiz...

2016-06-28 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13765 **[Test build #61398 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61398/consoleFull)** for PR 13765 at commit [`f1ee771`](https://github.com/apache/spark/commit/

[GitHub] spark pull request #13945: [SPARK-16256][SQL][STREAMING] Added Structured St...

2016-06-28 Thread koeninger
Github user koeninger commented on a diff in the pull request: https://github.com/apache/spark/pull/13945#discussion_r68840069 --- Diff: docs/structured-streaming-programming-guide.md --- @@ -0,0 +1,888 @@ +--- +layout: global +displayTitle: Structured Streaming Programm

[GitHub] spark pull request #13945: [SPARK-16256][SQL][STREAMING] Added Structured St...

2016-06-28 Thread holdenk
Github user holdenk commented on a diff in the pull request: https://github.com/apache/spark/pull/13945#discussion_r68840104 --- Diff: docs/structured-streaming-programming-guide.md --- @@ -0,0 +1,888 @@ +--- +layout: global +displayTitle: Structured Streaming Programmin

[GitHub] spark pull request #13945: [SPARK-16256][SQL][STREAMING] Added Structured St...

2016-06-28 Thread koeninger
Github user koeninger commented on a diff in the pull request: https://github.com/apache/spark/pull/13945#discussion_r68839964 --- Diff: docs/structured-streaming-programming-guide.md --- @@ -0,0 +1,888 @@ +--- +layout: global +displayTitle: Structured Streaming Programm

[GitHub] spark pull request #13945: [SPARK-16256][SQL][STREAMING] Added Structured St...

2016-06-28 Thread koeninger
Github user koeninger commented on a diff in the pull request: https://github.com/apache/spark/pull/13945#discussion_r68839851 --- Diff: docs/structured-streaming-programming-guide.md --- @@ -0,0 +1,888 @@ +--- +layout: global +displayTitle: Structured Streaming Programm

[GitHub] spark pull request #13948: [SPARK-16259] [PYSPARK] cleanup options in DataFr...

2016-06-28 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/13948 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[GitHub] spark pull request #13906: [SPARK-16208][SQL] Add `CollapseEmptyPlan` optimi...

2016-06-28 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/13906#discussion_r68839609 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -1053,6 +1055,49 @@ object PruneFilters extends Rule[Logic

[GitHub] spark pull request #13945: [SPARK-16256][SQL][STREAMING] Added Structured St...

2016-06-28 Thread koeninger
Github user koeninger commented on a diff in the pull request: https://github.com/apache/spark/pull/13945#discussion_r68839530 --- Diff: docs/structured-streaming-programming-guide.md --- @@ -0,0 +1,888 @@ +--- +layout: global +displayTitle: Structured Streaming Programm

[GitHub] spark pull request #13945: [SPARK-16256][SQL][STREAMING] Added Structured St...

2016-06-28 Thread koeninger
Github user koeninger commented on a diff in the pull request: https://github.com/apache/spark/pull/13945#discussion_r68839308 --- Diff: docs/structured-streaming-programming-guide.md --- @@ -0,0 +1,888 @@ +--- +layout: global +displayTitle: Structured Streaming Programm

[GitHub] spark pull request #13945: [SPARK-16256][SQL][STREAMING] Added Structured St...

2016-06-28 Thread koeninger
Github user koeninger commented on a diff in the pull request: https://github.com/apache/spark/pull/13945#discussion_r68839249 --- Diff: docs/structured-streaming-programming-guide.md --- @@ -0,0 +1,888 @@ +--- +layout: global +displayTitle: Structured Streaming Programm

[GitHub] spark issue #13948: [SPARK-16259] [PYSPARK] cleanup options in DataFrame rea...

2016-06-28 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/13948 LGTM - merging in master/2.0. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark pull request #13945: [SPARK-16256][SQL][STREAMING] Added Structured St...

2016-06-28 Thread koeninger
Github user koeninger commented on a diff in the pull request: https://github.com/apache/spark/pull/13945#discussion_r68838818 --- Diff: docs/structured-streaming-programming-guide.md --- @@ -0,0 +1,888 @@ +--- +layout: global +displayTitle: Structured Streaming Programm

[GitHub] spark pull request #13906: [SPARK-16208][SQL] Add `CollapseEmptyPlan` optimi...

2016-06-28 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/13906#discussion_r68838472 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -1053,6 +1055,49 @@ object PruneFilters extends R

[GitHub] spark pull request #13906: [SPARK-16208][SQL] Add `CollapseEmptyPlan` optimi...

2016-06-28 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/13906#discussion_r68838412 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -1053,6 +1055,49 @@ object PruneFilters extends R

[GitHub] spark pull request #13858: [SPARK-16148] [Scheduler] Allow for underscores i...

2016-06-28 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/13858 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[GitHub] spark pull request #13906: [SPARK-16208][SQL] Add `CollapseEmptyPlan` optimi...

2016-06-28 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/13906#discussion_r68838170 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -1053,6 +1055,49 @@ object PruneFilters extends R

[GitHub] spark issue #13858: [SPARK-16148] [Scheduler] Allow for underscores in TaskL...

2016-06-28 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/13858 LGTM. Merging to master and 2.0. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature en

[GitHub] spark issue #13924: [SPARK-15643][DOC][ML] Add breaking changes to ML migrat...

2016-06-28 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13924 **[Test build #61408 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61408/consoleFull)** for PR 13924 at commit [`6ef09a3`](https://github.com/apache/spark/commit/6

[GitHub] spark pull request #13906: [SPARK-16208][SQL] Add `CollapseEmptyPlan` optimi...

2016-06-28 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/13906#discussion_r68837197 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -1053,6 +1055,49 @@ object PruneFilters extends Rule[Logic

[GitHub] spark issue #13906: [SPARK-16208][SQL] Add `CollapseEmptyPlan` optimizer

2016-06-28 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/13906 cc @cloud-fan to take a look too. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark pull request #13906: [SPARK-16208][SQL] Add `CollapseEmptyPlan` optimi...

2016-06-28 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/13906#discussion_r68836963 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -1053,6 +1055,49 @@ object PruneFilters extends Rule[Logic

[GitHub] spark issue #13924: [SPARK-15643][DOC][ML] Add breaking changes to ML migrat...

2016-06-28 Thread MLnick
Github user MLnick commented on the issue: https://github.com/apache/spark/pull/13924 @yanboliang @jkbradley @mengxr updated. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enab

[GitHub] spark issue #13824: [SPARK-16110][YARN][PYSPARK] Fix allowing python version...

2016-06-28 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13824 **[Test build #61407 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61407/consoleFull)** for PR 13824 at commit [`4ad7904`](https://github.com/apache/spark/commit/4

[GitHub] spark issue #13824: [SPARK-16110][YARN][PYSPARK] Fix allowing python version...

2016-06-28 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/13824 ok to test --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] spark issue #13824: [SPARK-16110][YARN][PYSPARK] Fix allowing python version...

2016-06-28 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/13824 @KevinGrealish you can read env vars through `SparkConf.getenv` and that allows you to override them in tests somewhat easily (look for `SparkConfWithEnv`). --- If your project is set up for it, yo

[GitHub] spark issue #13921: [SPARK-16140][MLlib][SparkR][Docs] Group k-means method ...

2016-06-28 Thread keypointt
Github user keypointt commented on the issue: https://github.com/apache/spark/pull/13921 sure, I tried ``` #' @seealso \link{predict}, \link{read.ml}, \link{write.ml} ``` and it's like below for `spark.kmeans` page, is this what you mean? ![screen shot 2016-06-

[GitHub] spark issue #13670: [SPARK-15951] Change Executors Page to use datatables to...

2016-06-28 Thread ajbozarth
Github user ajbozarth commented on the issue: https://github.com/apache/spark/pull/13670 I'll re-checkout later today and see if I can reproduce the error for you --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your p

[GitHub] spark pull request #13670: [SPARK-15951] Change Executors Page to use datata...

2016-06-28 Thread ajbozarth
Github user ajbozarth commented on a diff in the pull request: https://github.com/apache/spark/pull/13670#discussion_r68835621 --- Diff: docs/monitoring.md --- @@ -288,7 +288,11 @@ where `[base-app-id]` is the YARN application ID. /applications/[app-id]/exec

[GitHub] spark pull request #13945: [SPARK-16256][SQL][STREAMING] Added Structured St...

2016-06-28 Thread koeninger
Github user koeninger commented on a diff in the pull request: https://github.com/apache/spark/pull/13945#discussion_r68835454 --- Diff: docs/structured-streaming-programming-guide.md --- @@ -0,0 +1,888 @@ +--- +layout: global +displayTitle: Structured Streaming Programm

[GitHub] spark pull request #13945: [SPARK-16256][SQL][STREAMING] Added Structured St...

2016-06-28 Thread koeninger
Github user koeninger commented on a diff in the pull request: https://github.com/apache/spark/pull/13945#discussion_r68835399 --- Diff: docs/structured-streaming-programming-guide.md --- @@ -0,0 +1,888 @@ +--- +layout: global +displayTitle: Structured Streaming Programm

[GitHub] spark issue #13670: [SPARK-15951] Change Executors Page to use datatables to...

2016-06-28 Thread kishorvpatil
Github user kishorvpatil commented on the issue: https://github.com/apache/spark/pull/13670 @ajbozarth, I do not see how I could produce the exception you see. Please help me reproduce this so that I can address it. --- If your project is set up for it, you can reply to this email an

[GitHub] spark issue #13670: [SPARK-15951] Change Executors Page to use datatables to...

2016-06-28 Thread kishorvpatil
Github user kishorvpatil commented on the issue: https://github.com/apache/spark/pull/13670 @zhuoliu , i have removed unwanted, unused javascript functions in the executorspage.js. There are no duplicate functions now between historyserver.js and this script.. --- If your project is

[GitHub] spark issue #13831: [SPARK-16119][sql] Support PURGE option to drop table / ...

2016-06-28 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13831 **[Test build #61406 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61406/consoleFull)** for PR 13831 at commit [`3dca099`](https://github.com/apache/spark/commit/3

[GitHub] spark issue #13740: [SPARK-16020][SQL]Fix complete mode aggregation with con...

2016-06-28 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/13740 @holdenk it will be documented here: https://github.com/apache/spark/pull/13945/files#diff-21245da8f8dbfef6401c5500f559f0bcR531 --- If your project is set up for it, you can reply to this email and

[GitHub] spark issue #13921: [SPARK-16140][MLlib][SparkR][Docs] Group k-means method ...

2016-06-28 Thread mengxr
Github user mengxr commented on the issue: https://github.com/apache/spark/pull/13921 @keypointt Could you add `seealso` links to the generic `predict` and `write.ml` doc? @felixcheung I think it is useful to have a page for the generic `predict` and `write.ml`. We just need

[GitHub] spark issue #13730: [SPARK-16006][SQL] Attemping to write empty DataFrame wi...

2016-06-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13730 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/61397/ Test PASSed. ---

[GitHub] spark issue #13730: [SPARK-16006][SQL] Attemping to write empty DataFrame wi...

2016-06-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13730 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #13951: [WIP][SPARK-8425]

2016-06-28 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13951 **[Test build #61405 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61405/consoleFull)** for PR 13951 at commit [`17e1e77`](https://github.com/apache/spark/commit/1

[GitHub] spark issue #13730: [SPARK-16006][SQL] Attemping to write empty DataFrame wi...

2016-06-28 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13730 **[Test build #61397 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61397/consoleFull)** for PR 13730 at commit [`7d38003`](https://github.com/apache/spark/commit/

[GitHub] spark issue #13952: [MINOR][DOCS][Structured Streaming] Minor doc fixes arou...

2016-06-28 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/13952 LGTM pending tests. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so,

[GitHub] spark issue #13921: [SPARK-16140][MLlib][SparkR][Docs] Group k-means method ...

2016-06-28 Thread keypointt
Github user keypointt commented on the issue: https://github.com/apache/spark/pull/13921 if we are to have an actual @rdname write.ml block in mllib.R, then it has to be enforced to all documentation stories under https://issues.apache.org/jira/browse/SPARK-16090, to remove individual

[GitHub] spark issue #13952: [MINOR][DOCS][Structured Streaming] Minor doc fixes arou...

2016-06-28 Thread brkyvz
Github user brkyvz commented on the issue: https://github.com/apache/spark/pull/13952 cc @tdas @zsxwing --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or

[GitHub] spark issue #13860: [SPARK-16157] [SQL] Add New Methods for comments in Stru...

2016-06-28 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/13860 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] spark issue #13952: [MINOR][DOCS][Structured Streaming] Minor doc fixes arou...

2016-06-28 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13952 **[Test build #61404 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61404/consoleFull)** for PR 13952 at commit [`3fa4776`](https://github.com/apache/spark/commit/3

[GitHub] spark issue #11105: [SPARK-12469][CORE] Data Property accumulators for Spark

2016-06-28 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/11105 **[Test build #61403 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61403/consoleFull)** for PR 11105 at commit [`b44a9ff`](https://github.com/apache/spark/commit/b

[GitHub] spark issue #13860: [SPARK-16157] [SQL] Add New Methods for comments in Stru...

2016-06-28 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13860 **[Test build #61402 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61402/consoleFull)** for PR 13860 at commit [`b8e0511`](https://github.com/apache/spark/commit/b

[GitHub] spark issue #13860: [SPARK-16157] [SQL] Add New Methods for comments in Stru...

2016-06-28 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/13860 Great! Thank you! @yhuai @cloud-fan @hvanhovell --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark issue #13951: [WIP][SPARK-8425]

2016-06-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13951 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/61400/ Test FAILed. ---

[GitHub] spark issue #13951: [WIP][SPARK-8425]

2016-06-28 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13951 **[Test build #61400 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61400/consoleFull)** for PR 13951 at commit [`168de23`](https://github.com/apache/spark/commit/

[GitHub] spark issue #13951: [WIP][SPARK-8425]

2016-06-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13951 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #13952: [MINOR][DOCS][Structured Streaming] Minor doc fixes arou...

2016-06-28 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13952 **[Test build #61401 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61401/consoleFull)** for PR 13952 at commit [`186deb8`](https://github.com/apache/spark/commit/1

[GitHub] spark pull request #13952: [MINOR][DOCS][Structured Streaming] Minor doc fix...

2016-06-28 Thread brkyvz
GitHub user brkyvz opened a pull request: https://github.com/apache/spark/pull/13952 [MINOR][DOCS][Structured Streaming] Minor doc fixes around `DataFrameWriter` and `DataStreamWriter` ## What changes were proposed in this pull request? Fixes a couple old references to `Dat

[GitHub] spark pull request #13950: [SPARK-15487] [Web UI] Spark Master UI to reverse...

2016-06-28 Thread gurvindersingh
Github user gurvindersingh commented on a diff in the pull request: https://github.com/apache/spark/pull/13950#discussion_r68826828 --- Diff: core/src/main/scala/org/apache/spark/ui/JettyUtils.scala --- @@ -186,6 +188,67 @@ private[spark] object JettyUtils extends Logging {

[GitHub] spark issue #13951: [WIP][SPARK-8425]

2016-06-28 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13951 **[Test build #61400 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61400/consoleFull)** for PR 13951 at commit [`168de23`](https://github.com/apache/spark/commit/1

[GitHub] spark pull request #13950: [SPARK-15487] [Web UI] Spark Master UI to reverse...

2016-06-28 Thread gurvindersingh
Github user gurvindersingh commented on a diff in the pull request: https://github.com/apache/spark/pull/13950#discussion_r68826228 --- Diff: core/src/main/scala/org/apache/spark/deploy/master/ui/ApplicationPage.scala --- @@ -77,7 +77,15 @@ private[ui] class ApplicationPage(parent:

[GitHub] spark issue #13740: [SPARK-16020][SQL]Fix complete mode aggregation with con...

2016-06-28 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/13740 Would it make sense to follow up with a warning in ConsoleSink that it is only for debugging and collects the entire microbatch to the driver? This seems like some people might use it for logging in

[GitHub] spark pull request #13951: [WIP][SPARK-8425]

2016-06-28 Thread squito
GitHub user squito opened a pull request: https://github.com/apache/spark/pull/13951 [WIP][SPARK-8425] just running tests, not ready for review just yet. Still need to do some cleanup. You can merge this pull request into a Git repository by running: $ git pull https://github

[GitHub] spark pull request #13950: [SPARK-15487] [Web UI] Spark Master UI to reverse...

2016-06-28 Thread prb
Github user prb commented on a diff in the pull request: https://github.com/apache/spark/pull/13950#discussion_r68825733 --- Diff: core/src/main/scala/org/apache/spark/ui/JettyUtils.scala --- @@ -186,6 +188,67 @@ private[spark] object JettyUtils extends Logging { contextHan

[GitHub] spark issue #13737: [SPARK-15954][SQL][PySpark][TEST] Fix TestHiveContext in...

2016-06-28 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/13737 Can you check what files the python tests are using? My understanding is that there should only be a few number of places in Python that use these test files. --- If your project is set up for it, y

[GitHub] spark pull request #13950: [SPARK-15487] [Web UI] Spark Master UI to reverse...

2016-06-28 Thread prb
Github user prb commented on a diff in the pull request: https://github.com/apache/spark/pull/13950#discussion_r68824409 --- Diff: core/src/main/scala/org/apache/spark/deploy/master/ui/ApplicationPage.scala --- @@ -77,7 +77,15 @@ private[ui] class ApplicationPage(parent: MasterWebU

[GitHub] spark issue #13950: [SPARK-15487] [Web UI] Spark Master UI to reverse proxy ...

2016-06-28 Thread ajbozarth
Github user ajbozarth commented on the issue: https://github.com/apache/spark/pull/13950 This looks good and I really like this added functionality, I'll check out your code and give it a test run as soon as I have time. --- If your project is set up for it, you can reply to this em

[GitHub] spark pull request #13946: [MINOR][SparkR] update sparkR DataFrame.R comment

2016-06-28 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/13946 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[GitHub] spark issue #13946: [MINOR][SparkR] update sparkR DataFrame.R comment

2016-06-28 Thread shivaram
Github user shivaram commented on the issue: https://github.com/apache/spark/pull/13946 merging this to master and branch-2.0 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enab

[GitHub] spark issue #13948: [SPARK-16259] [PYSPARK] cleanup options in DataFrame rea...

2016-06-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13948 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #13948: [SPARK-16259] [PYSPARK] cleanup options in DataFrame rea...

2016-06-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13948 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/61396/ Test PASSed. ---

[GitHub] spark issue #13948: [SPARK-16259] [PYSPARK] cleanup options in DataFrame rea...

2016-06-28 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13948 **[Test build #61396 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61396/consoleFull)** for PR 13948 at commit [`2e7b6b9`](https://github.com/apache/spark/commit/

[GitHub] spark issue #13950: [SPARK-15487] [Web UI] Spark Master UI to reverse proxy ...

2016-06-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13950 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feat

[GitHub] spark pull request #13950: [SPARK-15487] [Web UI] Spark Master UI to reverse...

2016-06-28 Thread gurvindersingh
GitHub user gurvindersingh opened a pull request: https://github.com/apache/spark/pull/13950 [SPARK-15487] [Web UI] Spark Master UI to reverse proxy Application and Workers UI ## What changes were proposed in this pull request? This pull request adds the functionality to en

[GitHub] spark issue #13603: [SPARK-15865][CORE] Blacklist should not result in job h...

2016-06-28 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13603 **[Test build #61399 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61399/consoleFull)** for PR 13603 at commit [`9665029`](https://github.com/apache/spark/commit/9

[GitHub] spark pull request #13378: [SPARK-15643] [Doc] [ML] Update spark.ml and spar...

2016-06-28 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/13378 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

<    1   2   3   4   5   6   7   8   >