[GitHub] spark pull request #15475: [SPARK-17991] [SQL] Enable metastore partition pr...

2016-10-19 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/15475 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[GitHub] spark pull request #15539: [SPARK-17994] [SQL] Add back a file status cache ...

2016-10-19 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/15539#discussion_r84222074 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/FileStatusCache.scala --- @@ -0,0 +1,82 @@ +/* + * Licensed to the Apache

[GitHub] spark issue #15475: [SPARK-17991] [SQL] Enable metastore partition pruning b...

2016-10-19 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/15475 Merging in master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or

[GitHub] spark issue #15172: [SPARK-13331] AES support for over-the-wire encryption

2016-10-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15172 **[Test build #67245 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67245/consoleFull)** for PR 15172 at commit [`8d7532f`](https://github.com/apache/spark/commit/8

[GitHub] spark issue #15563: [SPARK-16759][CORE] Add a configuration property to pass...

2016-10-19 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/15563 Can't this just be a normal sparkconf? Why do we need to specialize the spark-submit arguments for this? --- If your project is set up for it, you can reply to this email and have your reply appear o

[GitHub] spark issue #15562: [SPARK-18021][SQL] Refactor file name specification for ...

2016-10-19 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/15562 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the fea

[GitHub] spark pull request #15550: [SPARK-18003][Spark Core] Fix bug of RDD zipWithI...

2016-10-19 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/15550 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[GitHub] spark issue #15562: [SPARK-18021][SQL] Refactor file name specification for ...

2016-10-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15562 **[Test build #67244 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67244/consoleFull)** for PR 15562 at commit [`34b3cb1`](https://github.com/apache/spark/commit/3

[GitHub] spark issue #15563: [SPARK-16759][CORE] Add a configuration property to pass...

2016-10-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15563 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feat

[GitHub] spark issue #15550: [SPARK-18003][Spark Core] Fix bug of RDD zipWithIndex & ...

2016-10-19 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/15550 Merging in master / branch-2.0. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and w

[GitHub] spark issue #15544: [SPARK-17997] [SQL] Add an aggregation function for coun...

2016-10-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15544 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67235/ Test FAILed. ---

[GitHub] spark issue #15544: [SPARK-17997] [SQL] Add an aggregation function for coun...

2016-10-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15544 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark pull request #15563: [SPARK-16759][CORE] Add a configuration property ...

2016-10-19 Thread weiqingy
GitHub user weiqingy opened a pull request: https://github.com/apache/spark/pull/15563 [SPARK-16759][CORE] Add a configuration property to pass in caller contexts of upstream applications into Spark ## What changes were proposed in this pull request? Many applications take Spark

[GitHub] spark issue #15544: [SPARK-17997] [SQL] Add an aggregation function for coun...

2016-10-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15544 **[Test build #67235 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67235/consoleFull)** for PR 15544 at commit [`90a17f6`](https://github.com/apache/spark/commit/

[GitHub] spark pull request #15562: [SPARK-18021][SQL] Refactor file name specificati...

2016-10-19 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/15562#discussion_r84220232 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetOutputWriter.scala --- @@ -122,47 +122,29 @@ private[parquet]

[GitHub] spark issue #15562: [SPARK-18021][SQL] Refactor file name specification for ...

2016-10-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15562 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67241/ Test FAILed. ---

[GitHub] spark issue #15562: [SPARK-18021][SQL] Refactor file name specification for ...

2016-10-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15562 **[Test build #67241 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67241/consoleFull)** for PR 15562 at commit [`6b79d88`](https://github.com/apache/spark/commit/

[GitHub] spark issue #15562: [SPARK-18021][SQL] Refactor file name specification for ...

2016-10-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15562 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #15505: [WIP][SPARK-17931][CORE] taskScheduler has some unneeded...

2016-10-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15505 **[Test build #67242 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67242/consoleFull)** for PR 15505 at commit [`07b0581`](https://github.com/apache/spark/commit/0

[GitHub] spark issue #15297: [WIP][SPARK-9862]Handling data skew

2016-10-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15297 **[Test build #67243 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67243/consoleFull)** for PR 15297 at commit [`6c943d1`](https://github.com/apache/spark/commit/6

[GitHub] spark issue #15562: [SPARK-18021][SQL] Refactor file name specification for ...

2016-10-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15562 **[Test build #67241 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67241/consoleFull)** for PR 15562 at commit [`6b79d88`](https://github.com/apache/spark/commit/6

[GitHub] spark pull request #15561: [SPARK-18012][SQL] Simplify WriterContainer follo...

2016-10-19 Thread rxin
Github user rxin closed the pull request at: https://github.com/apache/spark/pull/15561 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabl

[GitHub] spark issue #15561: [SPARK-18012][SQL] Simplify WriterContainer follow-up

2016-10-19 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/15561 Actually this is subsumed by https://github.com/apache/spark/pull/15562 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark issue #15562: [SPARK-18021][SQL] Refactor file name specification for ...

2016-10-19 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/15562 cc @liancheng @cloud-fan @tejasapatil also @marmbrus -- you have always wanted this to happen. --- If your project is set up for it, you can reply to this email and have your reply appear on

[GitHub] spark pull request #15562: Spark 18021

2016-10-19 Thread rxin
GitHub user rxin opened a pull request: https://github.com/apache/spark/pull/15562 Spark 18021 ## What changes were proposed in this pull request? Currently each data source OutputWriter is responsible for specifying the entire file name for each file output. This, however, does

[GitHub] spark pull request #15516: [SPARK-17961][SparkR][SQL] Add storageLevel to Da...

2016-10-19 Thread WeichenXu123
Github user WeichenXu123 commented on a diff in the pull request: https://github.com/apache/spark/pull/15516#discussion_r84218510 --- Diff: R/pkg/R/utils.R --- @@ -385,6 +385,41 @@ getStorageLevel <- function(newLevel = c("DISK_ONLY", "OFF_HEAP" = callJ

[GitHub] spark issue #15554: [SPARK-16078] [SQL] Backport: from_utc_timestamp/to_utc_...

2016-10-19 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/15554 Merged in. Can you close this? Thanks. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feat

[GitHub] spark issue #15558: [SPARK-17357][SPARK-6624][SQL] Convert filter predicate ...

2016-10-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15558 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #15558: [SPARK-17357][SPARK-6624][SQL] Convert filter predicate ...

2016-10-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15558 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67234/ Test PASSed. ---

[GitHub] spark issue #15558: [SPARK-17357][SPARK-6624][SQL] Convert filter predicate ...

2016-10-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15558 **[Test build #67234 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67234/consoleFull)** for PR 15558 at commit [`5343947`](https://github.com/apache/spark/commit/

[GitHub] spark issue #15560: [SPARKR] fix warnings

2016-10-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15560 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #15560: [SPARKR] fix warnings

2016-10-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15560 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67238/ Test PASSed. ---

[GitHub] spark issue #15560: [SPARKR] fix warnings

2016-10-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15560 **[Test build #67238 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67238/consoleFull)** for PR 15560 at commit [`b25b277`](https://github.com/apache/spark/commit/

[GitHub] spark issue #15559: [SPARK-18013][SPARKR] add crossJoin API

2016-10-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15559 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67236/ Test PASSed. ---

[GitHub] spark issue #15559: [SPARK-18013][SPARKR] add crossJoin API

2016-10-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15559 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #15559: [SPARK-18013][SPARKR] add crossJoin API

2016-10-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15559 **[Test build #67236 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67236/consoleFull)** for PR 15559 at commit [`c43d169`](https://github.com/apache/spark/commit/

[GitHub] spark issue #15551: [SPARK-18012][SQL] Simplify WriterContainer

2016-10-19 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/15551 cc @tejasapatil fyi on the change --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #15561: [SPARK-18012][SQL] Simplify WriterContainer follow-up

2016-10-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15561 **[Test build #67240 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67240/consoleFull)** for PR 15561 at commit [`426ed1f`](https://github.com/apache/spark/commit/4

[GitHub] spark issue #15428: [SPARK-17219][ML] enhanced NaN value handling in Bucketi...

2016-10-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15428 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67233/ Test PASSed. ---

[GitHub] spark issue #15428: [SPARK-17219][ML] enhanced NaN value handling in Bucketi...

2016-10-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15428 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark pull request #15561: [SPARK-18012][SQL] Simplify WriterContainer follo...

2016-10-19 Thread rxin
GitHub user rxin opened a pull request: https://github.com/apache/spark/pull/15561 [SPARK-18012][SQL] Simplify WriterContainer follow-up ## What changes were proposed in this pull request? This patch is a follow-up to https://github.com/apache/spark/pull/15551 and did a few cosm

[GitHub] spark issue #15428: [SPARK-17219][ML] enhanced NaN value handling in Bucketi...

2016-10-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15428 **[Test build #67233 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67233/consoleFull)** for PR 15428 at commit [`d1dd840`](https://github.com/apache/spark/commit/

[GitHub] spark pull request #15551: [SPARK-18012][SQL] Simplify WriterContainer

2016-10-19 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/15551 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[GitHub] spark issue #15551: [SPARK-18012][SQL] Simplify WriterContainer

2016-10-19 Thread liancheng
Github user liancheng commented on the issue: https://github.com/apache/spark/pull/15551 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the fea

[GitHub] spark issue #15377: [SPARK-17802] Improved caller context logging.

2016-10-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15377 **[Test build #67239 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67239/consoleFull)** for PR 15377 at commit [`9cffcf3`](https://github.com/apache/spark/commit/9

[GitHub] spark issue #15560: [SPARKR] fix warnings

2016-10-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15560 **[Test build #67238 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67238/consoleFull)** for PR 15560 at commit [`b25b277`](https://github.com/apache/spark/commit/b

[GitHub] spark pull request #15560: [SPARKR] fix warnings

2016-10-19 Thread felixcheung
GitHub user felixcheung opened a pull request: https://github.com/apache/spark/pull/15560 [SPARKR] fix warnings ## What changes were proposed in this pull request? Fix for a bunch of test warnings that were added recently. We need to investigate why warnings are not turn

[GitHub] spark issue #15423: [SPARK-17860][SQL] SHOW COLUMN's database conflict check...

2016-10-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15423 **[Test build #67237 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67237/consoleFull)** for PR 15423 at commit [`15c568f`](https://github.com/apache/spark/commit/1

[GitHub] spark issue #15559: [SPARK-18013][SPARKR] add crossJoin API

2016-10-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15559 **[Test build #67236 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67236/consoleFull)** for PR 15559 at commit [`c43d169`](https://github.com/apache/spark/commit/c

[GitHub] spark pull request #15559: [SPARK-18013][SPARKR] add crossJoin API

2016-10-19 Thread felixcheung
GitHub user felixcheung opened a pull request: https://github.com/apache/spark/pull/15559 [SPARK-18013][SPARKR] add crossJoin API ## What changes were proposed in this pull request? Add crossJoin and do not default to cross join if joinExpr is left out ## How was th

[GitHub] spark issue #15544: [SPARK-17997] [SQL] Add an aggregation function for coun...

2016-10-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15544 **[Test build #67235 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67235/consoleFull)** for PR 15544 at commit [`90a17f6`](https://github.com/apache/spark/commit/9

[GitHub] spark pull request #15377: [SPARK-17802] Improved caller context logging.

2016-10-19 Thread jerryshao
Github user jerryshao commented on a diff in the pull request: https://github.com/apache/spark/pull/15377#discussion_r84212380 --- Diff: core/src/main/scala/org/apache/spark/util/Utils.scala --- @@ -2432,6 +2432,26 @@ private[spark] object Utils extends Logging { } }

[GitHub] spark issue #15550: [SPARK-18003][Spark Core] Fix bug of RDD zipWithIndex & ...

2016-10-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15550 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67228/ Test PASSed. ---

[GitHub] spark issue #15550: [SPARK-18003][Spark Core] Fix bug of RDD zipWithIndex & ...

2016-10-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15550 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #15550: [SPARK-18003][Spark Core] Fix bug of RDD zipWithIndex & ...

2016-10-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15550 **[Test build #67228 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67228/consoleFull)** for PR 15550 at commit [`b83a606`](https://github.com/apache/spark/commit/

[GitHub] spark issue #15523: [SPARK-17981] [SPARK-17957] [SQL] Fix Incorrect Nullabil...

2016-10-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15523 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #15523: [SPARK-17981] [SPARK-17957] [SQL] Fix Incorrect Nullabil...

2016-10-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15523 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67229/ Test PASSed. ---

[GitHub] spark issue #15523: [SPARK-17981] [SPARK-17957] [SQL] Fix Incorrect Nullabil...

2016-10-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15523 **[Test build #67229 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67229/consoleFull)** for PR 15523 at commit [`52cb8fb`](https://github.com/apache/spark/commit/

[GitHub] spark pull request #15552: [SPARK-18007][SparkR][ML] update SparkR MLP - add...

2016-10-19 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/15552#discussion_r84210373 --- Diff: mllib/src/main/scala/org/apache/spark/ml/r/MultilayerPerceptronClassifierWrapper.scala --- @@ -73,6 +75,8 @@ private[r] object MultilayerPerce

[GitHub] spark issue #15428: [SPARK-17219][ML] enhanced NaN value handling in Bucketi...

2016-10-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15428 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67232/ Test PASSed. ---

[GitHub] spark issue #15428: [SPARK-17219][ML] enhanced NaN value handling in Bucketi...

2016-10-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15428 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #15428: [SPARK-17219][ML] enhanced NaN value handling in Bucketi...

2016-10-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15428 **[Test build #67232 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67232/consoleFull)** for PR 15428 at commit [`0fb8d38`](https://github.com/apache/spark/commit/

[GitHub] spark issue #15515: [SPARK-17970][SQL][WIP] store partition spec in metastor...

2016-10-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15515 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #15515: [SPARK-17970][SQL][WIP] store partition spec in metastor...

2016-10-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15515 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67230/ Test FAILed. ---

[GitHub] spark issue #15515: [SPARK-17970][SQL][WIP] store partition spec in metastor...

2016-10-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15515 **[Test build #67230 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67230/consoleFull)** for PR 15515 at commit [`4d93f48`](https://github.com/apache/spark/commit/

[GitHub] spark issue #15551: [SPARK-18012][SQL] Simplify WriterContainer

2016-10-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15551 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67227/ Test PASSed. ---

[GitHub] spark issue #15558: [SPARK-17357][SPARK-6624][SQL] Convert filter predicate ...

2016-10-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15558 **[Test build #67234 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67234/consoleFull)** for PR 15558 at commit [`5343947`](https://github.com/apache/spark/commit/5

[GitHub] spark issue #15551: [SPARK-18012][SQL] Simplify WriterContainer

2016-10-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15551 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #15551: [SPARK-18012][SQL] Simplify WriterContainer

2016-10-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15551 **[Test build #67227 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67227/consoleFull)** for PR 15551 at commit [`0ca0c81`](https://github.com/apache/spark/commit/

[GitHub] spark pull request #15558: [SPARK-17357][SPARK-6624][SQL] Convert filter pre...

2016-10-19 Thread viirya
GitHub user viirya opened a pull request: https://github.com/apache/spark/pull/15558 [SPARK-17357][SPARK-6624][SQL] Convert filter predicate to CNF in Optimizer for pushdown ## What changes were proposed in this pull request? This PR is proposed to solve the problem #14912

[GitHub] spark issue #15421: [SPARK-17811] SparkR cannot parallelize data.frame with ...

2016-10-19 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/15421 @falaki I mean for the case the exception is raised, but it sounds it is the same test just on different R version LGTM. --- If your project is set up for it, you can reply to this email a

[GitHub] spark issue #15428: [SPARK-17219][ML] enhanced NaN value handling in Bucketi...

2016-10-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15428 **[Test build #67233 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67233/consoleFull)** for PR 15428 at commit [`d1dd840`](https://github.com/apache/spark/commit/d

[GitHub] spark issue #15009: [SPARK-17443][SPARK-11035] Stop Spark Application if lau...

2016-10-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15009 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67225/ Test PASSed. ---

[GitHub] spark issue #15009: [SPARK-17443][SPARK-11035] Stop Spark Application if lau...

2016-10-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15009 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #15009: [SPARK-17443][SPARK-11035] Stop Spark Application if lau...

2016-10-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15009 **[Test build #67225 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67225/consoleFull)** for PR 15009 at commit [`92e4445`](https://github.com/apache/spark/commit/

[GitHub] spark pull request #15546: [SPARK-17982][SQL] SQLBuilder should wrap the gen...

2016-10-19 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/15546#discussion_r84205834 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/catalyst/SQLBuilder.scala --- @@ -138,6 +138,9 @@ class SQLBuilder private ( case g: Ge

[GitHub] spark issue #15428: [SPARK-17219][ML] enhanced NaN value handling in Bucketi...

2016-10-19 Thread VinceShieh
Github user VinceShieh commented on the issue: https://github.com/apache/spark/pull/15428 Thanks for your valuable suggestions. @jkbradley @srowen --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request #15428: [SPARK-17219][ML] enhanced NaN value handling in ...

2016-10-19 Thread VinceShieh
Github user VinceShieh commented on a diff in the pull request: https://github.com/apache/spark/pull/15428#discussion_r84205458 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/QuantileDiscretizer.scala --- @@ -66,11 +67,13 @@ private[feature] trait QuantileDiscretizerBas

[GitHub] spark issue #15428: [SPARK-17219][ML] enhanced NaN value handling in Bucketi...

2016-10-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15428 **[Test build #67232 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67232/consoleFull)** for PR 15428 at commit [`0fb8d38`](https://github.com/apache/spark/commit/0

[GitHub] spark pull request #15428: [SPARK-17219][ML] enhanced NaN value handling in ...

2016-10-19 Thread VinceShieh
Github user VinceShieh commented on a diff in the pull request: https://github.com/apache/spark/pull/15428#discussion_r84205467 --- Diff: python/pyspark/ml/feature.py --- @@ -1157,9 +1157,11 @@ class QuantileDiscretizer(JavaEstimator, HasInputCol, HasOutputCol, JavaMLReadab

[GitHub] spark pull request #15428: [SPARK-17219][ML] enhanced NaN value handling in ...

2016-10-19 Thread VinceShieh
Github user VinceShieh commented on a diff in the pull request: https://github.com/apache/spark/pull/15428#discussion_r84205414 --- Diff: docs/ml-features.md --- @@ -1104,9 +1104,11 @@ for more details on the API. `QuantileDiscretizer` takes a column with continuous features an

[GitHub] spark issue #15428: [SPARK-17219][ML] enhanced NaN value handling in Bucketi...

2016-10-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15428 **[Test build #67231 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67231/consoleFull)** for PR 15428 at commit [`c350e9f`](https://github.com/apache/spark/commit/

[GitHub] spark issue #15428: [SPARK-17219][ML] enhanced NaN value handling in Bucketi...

2016-10-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15428 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67231/ Test FAILed. ---

[GitHub] spark issue #15428: [SPARK-17219][ML] enhanced NaN value handling in Bucketi...

2016-10-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15428 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #15551: [SPARK-18012][SQL] Simplify WriterContainer

2016-10-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15551 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67226/ Test FAILed. ---

[GitHub] spark issue #15551: [SPARK-18012][SQL] Simplify WriterContainer

2016-10-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15551 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #15428: [SPARK-17219][ML] enhanced NaN value handling in Bucketi...

2016-10-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15428 **[Test build #67231 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67231/consoleFull)** for PR 15428 at commit [`c350e9f`](https://github.com/apache/spark/commit/c

[GitHub] spark issue #15551: [SPARK-18012][SQL] Simplify WriterContainer

2016-10-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15551 **[Test build #67226 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67226/consoleFull)** for PR 15551 at commit [`cfba7bb`](https://github.com/apache/spark/commit/

[GitHub] spark pull request #15555: [Minor][ML] Refactor clustering summary.

2016-10-19 Thread yanboliang
Github user yanboliang commented on a diff in the pull request: https://github.com/apache/spark/pull/1#discussion_r84204673 --- Diff: mllib/src/main/scala/org/apache/spark/ml/clustering/KMeans.scala --- @@ -354,21 +354,41 @@ object KMeans extends DefaultParamsReadable[KMeans] {

[GitHub] spark pull request #15532: [SPARK-17989][SQL] Check ascendingOrder type in s...

2016-10-19 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/15532 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[GitHub] spark issue #15423: [SPARK-17860][SQL] SHOW COLUMN's database conflict check...

2016-10-19 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/15423 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the fea

[GitHub] spark pull request #15423: [SPARK-17860][SQL] SHOW COLUMN's database conflic...

2016-10-19 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/15423#discussion_r84204534 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/command/DDLSuite.scala --- @@ -1713,4 +1713,21 @@ class DDLSuite extends QueryTest with

[GitHub] spark pull request #15546: [SPARK-17982][SQL] SQLBuilder should wrap the gen...

2016-10-19 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/15546#discussion_r84204517 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/catalyst/SQLBuilder.scala --- @@ -138,6 +138,9 @@ class SQLBuilder private ( case g: Generate =>

[GitHub] spark issue #15532: [SPARK-17989][SQL] Check ascendingOrder type in sort_arr...

2016-10-19 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/15532 Thanks - merging in master/branch-2.0. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enable

[GitHub] spark issue #15493: [SPARK-17946][PYSPARK] Python crossJoin API similar to S...

2016-10-19 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/15493 Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the featu

[GitHub] spark issue #15550: [SPARK-18003][Spark Core] Fix bug of RDD zipWithIndex & ...

2016-10-19 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/15550 Cool - LGTM! (I will merge once Jenkins comes back positive) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not hav

[GitHub] spark issue #15493: [SPARK-17946][PYSPARK] Python crossJoin API similar to S...

2016-10-19 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/15493 I will take that and add my note. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #15539: [SPARK-17994] [SQL] Add back a file status cache ...

2016-10-19 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/15539#discussion_r84203885 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/HiveTablePerfStatsSuite.scala --- @@ -103,11 +92,103 @@ class HiveDataFrameSuite extends Query

[GitHub] spark pull request #15539: [SPARK-17994] [SQL] Add back a file status cache ...

2016-10-19 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/15539#discussion_r84203441 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveMetastoreCatalog.scala --- @@ -231,11 +231,16 @@ private[hive] class HiveMetastoreCatalog(

[GitHub] spark issue #15550: [SPARK-18003][Spark Core] Fix bug of RDD zipWithIndex & ...

2016-10-19 Thread WeichenXu123
Github user WeichenXu123 commented on the issue: https://github.com/apache/spark/pull/15550 @tejasapatil check other reference to scala's `zipWithIndex` and fix similar case in `RDD.zipWithUniqueId`, thanks! --- If your project is set up for it, you can reply to this email and have y

  1   2   3   4   5   6   >