[GitHub] spark issue #9183: [SPARK-11215] [ML] Add multiple columns support to String...

2016-08-22 Thread MLnick
Github user MLnick commented on the issue: https://github.com/apache/spark/pull/9183 @yanboliang will you be reviving this PR? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature ena

[GitHub] spark pull request #14749: [SPARK-17182][SQL] Mark Collect as non-determinis...

2016-08-22 Thread liancheng
GitHub user liancheng opened a pull request: https://github.com/apache/spark/pull/14749 [SPARK-17182][SQL] Mark Collect as non-deterministic ## What changes were proposed in this pull request? This PR marks the abstract class `Collect` as non-deterministic since the results

[GitHub] spark issue #9183: [SPARK-11215] [ML] Add multiple columns support to String...

2016-08-22 Thread yanboliang
Github user yanboliang commented on the issue: https://github.com/apache/spark/pull/9183 @MLnick I think this is an important feature to make ML pipeline handle large datasets elegantly. I will update/send a new PR soon and looking forward that you can help to review. Thanks! --- If

[GitHub] spark issue #14748: [SPARK-16781] [PYSPARK] java launched by PySpark as gate...

2016-08-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14748 **[Test build #64187 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64187/consoleFull)** for PR 14748 at commit [`f19fc5e`](https://github.com/apache/spark/commit/

[GitHub] spark issue #14748: [SPARK-16781] [PYSPARK] java launched by PySpark as gate...

2016-08-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14748 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/64187/ Test PASSed. ---

[GitHub] spark issue #14748: [SPARK-16781] [PYSPARK] java launched by PySpark as gate...

2016-08-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14748 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #14567: [SPARK-16992][PYSPARK] Python Pep8 formatting and import...

2016-08-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14567 **[Test build #64189 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64189/consoleFull)** for PR 14567 at commit [`015028f`](https://github.com/apache/spark/commit/0

[GitHub] spark issue #14747: [SPARK-17086][ML] Fix an issue in QuantileDiscretizer

2016-08-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14747 **[Test build #64188 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64188/consoleFull)** for PR 14747 at commit [`ea9146c`](https://github.com/apache/spark/commit/e

[GitHub] spark issue #14749: [SPARK-17182][SQL] Mark Collect as non-deterministic

2016-08-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14749 **[Test build #64195 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64195/consoleFull)** for PR 14749 at commit [`0045128`](https://github.com/apache/spark/commit/0

[GitHub] spark issue #14524: [SPARK-16832] [ML] [WIP] CrossValidator and TrainValidat...

2016-08-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14524 **[Test build #64193 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64193/consoleFull)** for PR 14524 at commit [`ceebf7c`](https://github.com/apache/spark/commit/c

[GitHub] spark issue #14688: [SPARK-17095] [Documentation] [Latex and Scala doc do no...

2016-08-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14688 **[Test build #64192 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64192/consoleFull)** for PR 14688 at commit [`d358569`](https://github.com/apache/spark/commit/d

[GitHub] spark issue #14567: [SPARK-16992][PYSPARK] Python Pep8 formatting and import...

2016-08-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14567 **[Test build #64190 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64190/consoleFull)** for PR 14567 at commit [`0801573`](https://github.com/apache/spark/commit/0

[GitHub] spark issue #14747: [SPARK-17086][ML] Fix an issue in QuantileDiscretizer

2016-08-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14747 **[Test build #64196 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64196/consoleFull)** for PR 14747 at commit [`5f414b7`](https://github.com/apache/spark/commit/5

[GitHub] spark issue #14744: [SPARKR][SPARKSUBMIT] Allow to set sparkr shell command ...

2016-08-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14744 **[Test build #64191 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64191/consoleFull)** for PR 14744 at commit [`0a24b2d`](https://github.com/apache/spark/commit/0

[GitHub] spark issue #14688: [SPARK-17095] [Documentation] [Latex and Scala doc do no...

2016-08-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14688 **[Test build #64194 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64194/consoleFull)** for PR 14688 at commit [`d358569`](https://github.com/apache/spark/commit/d

[GitHub] spark issue #14683: [SPARK-16968]Document additional options in jdbc Writer

2016-08-22 Thread GraceH
Github user GraceH commented on the issue: https://github.com/apache/spark/pull/14683 Thanks @srowen --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if

[GitHub] spark issue #7927: [SPARK-9591][CORE]Job may fail for exception during getti...

2016-08-22 Thread GraceH
Github user GraceH commented on the issue: https://github.com/apache/spark/pull/7927 @sprite331. According to my understanding, this patch tries to catch certain exceptions when the user introducing dynamic allocation. One quick solution is to disable dynamic allocation if possible, w

[GitHub] spark issue #14537: [SPARK-16948][SQL] Querying empty partitioned orc tables...

2016-08-22 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/14537 why do we infer schema for tables? Table schema should be persisted to metastore when it was created. --- If your project is set up for it, you can reply to this email and have your reply appear

[GitHub] spark pull request #14531: [SPARK-16943] [SPARK-16942] [SQL] Fix multiple bu...

2016-08-22 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/14531#discussion_r75660683 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/tables.scala --- @@ -75,18 +81,52 @@ case class CreateTableLikeCommand(

[GitHub] spark pull request #14597: [SPARK-17017][MLLIB][ML] add a chiSquare Selector...

2016-08-22 Thread mpjlu
Github user mpjlu commented on a diff in the pull request: https://github.com/apache/spark/pull/14597#discussion_r75661562 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/feature/ChiSqSelector.scala --- @@ -189,11 +227,20 @@ class ChiSqSelector @Since("1.3.0") ( */

[GitHub] spark pull request #14735: [SPARK-17173][SPARKR] R MLlib refactor, cleanup, ...

2016-08-22 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/14735#discussion_r75661714 --- Diff: R/pkg/R/mllib.R --- @@ -1027,7 +1009,7 @@ setMethod("spark.gaussianMixture", signature(data = "SparkDataFrame", formula = #' @export

[GitHub] spark pull request #14735: [SPARK-17173][SPARKR] R MLlib refactor, cleanup, ...

2016-08-22 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/14735#discussion_r75661739 --- Diff: R/pkg/R/mllib.R --- @@ -499,11 +505,11 @@ setMethod("predict", signature(object = "IsotonicRegressionModel"), #' @export #' @note sum

[GitHub] spark issue #14734: [SPARK-16508][SPARKR] doc updates and more CRAN check fi...

2016-08-22 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/14734 @shivaram any more thought? I'll merge since we should have this to be ready for CRAN. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as

[GitHub] spark issue #14743: [SparkR][Minor] Fix Cache Folder Path in Windows

2016-08-22 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/14743 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the f

[GitHub] spark issue #14663: [SPARK-17001] [ML] Enable standardScaler to standardize ...

2016-08-22 Thread MLnick
Github user MLnick commented on the issue: https://github.com/apache/spark/pull/14663 Ah right, good point. Actually I realised that the doc in `ml.feature.StandardScaler` needs updating for `withMean`: ``` /** * Whether to center the data with mean before scaling

[GitHub] spark pull request #14735: [SPARK-17173][SPARKR] R MLlib refactor, cleanup, ...

2016-08-22 Thread yanboliang
Github user yanboliang commented on a diff in the pull request: https://github.com/apache/spark/pull/14735#discussion_r75664196 --- Diff: R/pkg/R/mllib.R --- @@ -1027,7 +1009,7 @@ setMethod("spark.gaussianMixture", signature(data = "SparkDataFrame", formula = #' @export #

[GitHub] spark pull request #14744: [SPARKR][SPARKSUBMIT] Allow to set sparkr shell c...

2016-08-22 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/14744#discussion_r75664285 --- Diff: docs/configuration.md --- @@ -1752,6 +1752,13 @@ showDF(properties, numRows = 200, truncate = FALSE) Executable for executing R scripts

[GitHub] spark issue #14744: [SPARKR][SPARKSUBMIT] Allow to set sparkr shell command ...

2016-08-22 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/14744 Could you open a JIRA on this and add more info on why this is needed and can't use `SPARKR_DRIVER_R`? --- If your project is set up for it, you can reply to this email and have your reply appe

[GitHub] spark issue #14747: [SPARK-17086][ML] Fix an issue in QuantileDiscretizer

2016-08-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14747 **[Test build #64197 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64197/consoleFull)** for PR 14747 at commit [`348436f`](https://github.com/apache/spark/commit/3

[GitHub] spark issue #14747: [SPARK-17086][ML] Fix InvalidArgumentException issue in ...

2016-08-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14747 **[Test build #64196 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64196/consoleFull)** for PR 14747 at commit [`5f414b7`](https://github.com/apache/spark/commit/

[GitHub] spark issue #14747: [SPARK-17086][ML] Fix InvalidArgumentException issue in ...

2016-08-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14747 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #14524: [SPARK-16832] [ML] [WIP] CrossValidator and TrainValidat...

2016-08-22 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/14524 + @yanboliang for R `spark.gaussianMixture` --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this fea

[GitHub] spark issue #14747: [SPARK-17086][ML] Fix InvalidArgumentException issue in ...

2016-08-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14747 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/64196/ Test PASSed. ---

[GitHub] spark issue #14688: [SPARK-17095] [Documentation] [Latex and Scala doc do no...

2016-08-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14688 **[Test build #64194 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64194/consoleFull)** for PR 14688 at commit [`d358569`](https://github.com/apache/spark/commit/

[GitHub] spark issue #14688: [SPARK-17095] [Documentation] [Latex and Scala doc do no...

2016-08-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14688 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #14747: [SPARK-17086][ML] Fix InvalidArgumentException issue in ...

2016-08-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14747 **[Test build #64188 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64188/consoleFull)** for PR 14747 at commit [`ea9146c`](https://github.com/apache/spark/commit/

[GitHub] spark issue #14688: [SPARK-17095] [Documentation] [Latex and Scala doc do no...

2016-08-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14688 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #14747: [SPARK-17086][ML] Fix InvalidArgumentException issue in ...

2016-08-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14747 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/64188/ Test PASSed. ---

[GitHub] spark issue #14688: [SPARK-17095] [Documentation] [Latex and Scala doc do no...

2016-08-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14688 **[Test build #64192 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64192/consoleFull)** for PR 14688 at commit [`d358569`](https://github.com/apache/spark/commit/

[GitHub] spark issue #14688: [SPARK-17095] [Documentation] [Latex and Scala doc do no...

2016-08-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14688 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/64194/ Test PASSed. ---

[GitHub] spark issue #14688: [SPARK-17095] [Documentation] [Latex and Scala doc do no...

2016-08-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14688 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/64192/ Test PASSed. ---

[GitHub] spark issue #14747: [SPARK-17086][ML] Fix InvalidArgumentException issue in ...

2016-08-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14747 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #14735: [SPARK-17173][SPARKR] R MLlib refactor, cleanup, reforma...

2016-08-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14735 **[Test build #64198 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64198/consoleFull)** for PR 14735 at commit [`d727093`](https://github.com/apache/spark/commit/d

[GitHub] spark pull request #10896: [SPARK-12978][SQL] Skip unnecessary final group-b...

2016-08-22 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/10896#discussion_r75666430 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/aggregate/SortAggregateExec.scala --- @@ -121,3 +121,6 @@ case class SortAggregateExec(

[GitHub] spark pull request #10896: [SPARK-12978][SQL] Skip unnecessary final group-b...

2016-08-22 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/10896#discussion_r75666478 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/PlannerSuite.scala --- @@ -37,36 +37,58 @@ class PlannerSuite extends SharedSQLContext {

[GitHub] spark issue #10896: [SPARK-12978][SQL] Skip unnecessary final group-by when ...

2016-08-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/10896 **[Test build #64199 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64199/consoleFull)** for PR 10896 at commit [`e37ef6a`](https://github.com/apache/spark/commit/e

[GitHub] spark issue #14524: [SPARK-16832] [ML] [WIP] CrossValidator and TrainValidat...

2016-08-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14524 **[Test build #64193 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64193/consoleFull)** for PR 14524 at commit [`ceebf7c`](https://github.com/apache/spark/commit/

[GitHub] spark issue #14524: [SPARK-16832] [ML] [WIP] CrossValidator and TrainValidat...

2016-08-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14524 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #14524: [SPARK-16832] [ML] [WIP] CrossValidator and TrainValidat...

2016-08-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14524 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/64193/ Test FAILed. ---

[GitHub] spark issue #14666: [SPARK-16578][SparkR] Enable SparkR to connect to a remo...

2016-08-22 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/14666 Does this work with Spark Standalone? "yarn-client" is actually deprecated: https://github.com/apache/spark/blob/9f37d4eac28dd179dd523fa7d645be97bb52af9c/core/src/main/scala/org/apac

[GitHub] spark issue #14735: [SPARK-17173][SPARKR] R MLlib refactor, cleanup, reforma...

2016-08-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14735 **[Test build #64198 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64198/consoleFull)** for PR 14735 at commit [`d727093`](https://github.com/apache/spark/commit/

[GitHub] spark issue #14735: [SPARK-17173][SPARKR] R MLlib refactor, cleanup, reforma...

2016-08-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14735 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #14735: [SPARK-17173][SPARKR] R MLlib refactor, cleanup, reforma...

2016-08-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14735 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/64198/ Test PASSed. ---

[GitHub] spark pull request #10896: [SPARK-12978][SQL] Skip unnecessary final group-b...

2016-08-22 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/10896#discussion_r75670643 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/aggregate/AggUtils.scala --- @@ -27,26 +27,87 @@ import org.apache.spark.sql.execution.stre

[GitHub] spark pull request #10896: [SPARK-12978][SQL] Skip unnecessary final group-b...

2016-08-22 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/10896#discussion_r75670619 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/aggregate/AggUtils.scala --- @@ -27,26 +27,87 @@ import org.apache.spark.sql.execution.stre

[GitHub] spark pull request #14745: [SPARK-16896][SQL] Handle duplicated field names ...

2016-08-22 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/14745#discussion_r75671104 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/csv/CSVFileFormat.scala --- @@ -57,28 +57,45 @@ class CSVFileFormat extends

[GitHub] spark issue #14747: [SPARK-17086][ML] Fix InvalidArgumentException issue in ...

2016-08-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14747 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/64197/ Test PASSed. ---

[GitHub] spark issue #14747: [SPARK-17086][ML] Fix InvalidArgumentException issue in ...

2016-08-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14747 **[Test build #64197 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64197/consoleFull)** for PR 14747 at commit [`348436f`](https://github.com/apache/spark/commit/

[GitHub] spark issue #14747: [SPARK-17086][ML] Fix InvalidArgumentException issue in ...

2016-08-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14747 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark pull request #14745: [SPARK-16896][SQL] Handle duplicated field names ...

2016-08-22 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/14745#discussion_r75671454 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/csv/CSVFileFormat.scala --- @@ -57,28 +57,45 @@ class CSVFileFormat extends

[GitHub] spark issue #14745: [SPARK-16896][SQL] Handle duplicated field names in head...

2016-08-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14745 **[Test build #64201 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64201/consoleFull)** for PR 14745 at commit [`94620ca`](https://github.com/apache/spark/commit/9

[GitHub] spark issue #14745: [SPARK-16896][SQL] Handle duplicated field names in head...

2016-08-22 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/14745 looks good to me. do we need to consider case? is "a1" the same as "A1"? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If yo

[GitHub] spark issue #14729: [SPARK-17167] [SQL] Issue Exceptions when Analyze Table ...

2016-08-22 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/14729 @hvanhovell as I know, a temporary table will be resolved as arbitrary logical plan, instead of `LeafNode` that the statistics of a query plan is based on. I think it will cause problem, doesn't it?

[GitHub] spark issue #10896: [SPARK-12978][SQL] Skip unnecessary final group-by when ...

2016-08-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/10896 **[Test build #64200 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64200/consoleFull)** for PR 10896 at commit [`8b64305`](https://github.com/apache/spark/commit/8

[GitHub] spark pull request #14750: [SPARK-17183][SQL] put hive serde table schema to...

2016-08-22 Thread cloud-fan
GitHub user cloud-fan opened a pull request: https://github.com/apache/spark/pull/14750 [SPARK-17183][SQL] put hive serde table schema to table properties like data source table ## What changes were proposed in this pull request? For data source tables, we will put its tabl

[GitHub] spark issue #14743: [SparkR][Minor] Fix Cache Folder Path in Windows

2016-08-22 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/14743 [Here](https://gist.github.com/HyukjinKwon/4a7e0848173d045fd6faf3e3030f62db) before this PR and [here](https://gist.github.com/HyukjinKwon/1a5be83ae77633550e2ab15a2f6883a3) after this PR. It s

[GitHub] spark issue #14750: [SPARK-17183][SQL] put hive serde table schema to table ...

2016-08-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14750 **[Test build #64202 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64202/consoleFull)** for PR 14750 at commit [`167fd43`](https://github.com/apache/spark/commit/1

[GitHub] spark pull request #14733: [SPARK-17170] [SQL] InMemoryTableScanExec driver-...

2016-08-22 Thread pwoody
Github user pwoody commented on a diff in the pull request: https://github.com/apache/spark/pull/14733#discussion_r75672675 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/columnar/InMemoryTableScanExec.scala --- @@ -125,12 +129,37 @@ case class InMemoryTableScanE

[GitHub] spark issue #14750: [SPARK-17183][SQL] put hive serde table schema to table ...

2016-08-22 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/14750 cc @yhuai @gatorsmile --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark issue #14745: [SPARK-16896][SQL] Handle duplicated field names in head...

2016-08-22 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/14745 Hm, yea, I think we should take that into account as `spark.sql.caseSensitive` is `false` by default. I will take a look at R as well and will fix this up tomorrow. Thank you for reviewing @feli

[GitHub] spark issue #14743: [SparkR][Minor] Fix Cache Folder Path in Windows

2016-08-22 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/14743 ah, thanks for running the tests. yea, I don't think we fare well on Windows. I don't know if the Spark Jenkins support Windows or that we check it works before releasing. We should

[GitHub] spark issue #14735: [SPARK-17173][SPARKR] R MLlib refactor, cleanup, reforma...

2016-08-22 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/14735 any more thought? I'd like to merge to facilitate more R mllib work. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your proj

[GitHub] spark issue #14749: [SPARK-17182][SQL] Mark Collect as non-deterministic

2016-08-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14749 **[Test build #64195 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64195/consoleFull)** for PR 14749 at commit [`0045128`](https://github.com/apache/spark/commit/

[GitHub] spark issue #14749: [SPARK-17182][SQL] Mark Collect as non-deterministic

2016-08-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14749 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/64195/ Test PASSed. ---

[GitHub] spark issue #14749: [SPARK-17182][SQL] Mark Collect as non-deterministic

2016-08-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14749 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #14738: [SPARK-17090][FOLLOW-UP][ML]Add expert param support to ...

2016-08-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14738 **[Test build #64203 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64203/consoleFull)** for PR 14738 at commit [`f9377ae`](https://github.com/apache/spark/commit/f

[GitHub] spark issue #14744: [SPARKR][SPARKSUBMIT] Allow to set sparkr shell command ...

2016-08-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14744 **[Test build #64191 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64191/consoleFull)** for PR 14744 at commit [`0a24b2d`](https://github.com/apache/spark/commit/

[GitHub] spark issue #14744: [SPARKR][SPARKSUBMIT] Allow to set sparkr shell command ...

2016-08-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14744 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/64191/ Test PASSed. ---

[GitHub] spark issue #14744: [SPARKR][SPARKSUBMIT] Allow to set sparkr shell command ...

2016-08-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14744 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #14567: [SPARK-16992][PYSPARK] Python Pep8 formatting and import...

2016-08-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14567 **[Test build #64189 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64189/consoleFull)** for PR 14567 at commit [`015028f`](https://github.com/apache/spark/commit/

[GitHub] spark issue #14567: [SPARK-16992][PYSPARK] Python Pep8 formatting and import...

2016-08-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14567 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #14567: [SPARK-16992][PYSPARK] Python Pep8 formatting and import...

2016-08-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14567 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/64189/ Test FAILed. ---

[GitHub] spark issue #14567: [SPARK-16992][PYSPARK] Python Pep8 formatting and import...

2016-08-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14567 **[Test build #64190 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64190/consoleFull)** for PR 14567 at commit [`0801573`](https://github.com/apache/spark/commit/

[GitHub] spark issue #14567: [SPARK-16992][PYSPARK] Python Pep8 formatting and import...

2016-08-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14567 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark pull request #14751: [WIP][SPARK-17184][[CORE]]Replace ByteBuf with In...

2016-08-22 Thread witgo
GitHub user witgo opened a pull request: https://github.com/apache/spark/pull/14751 [WIP][SPARK-17184][[CORE]]Replace ByteBuf with InputStream ## What changes were proposed in this pull request? The size of ByteBuf can not be greater than 2G, should be replaced by InputStre

[GitHub] spark issue #14567: [SPARK-16992][PYSPARK] Python Pep8 formatting and import...

2016-08-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14567 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/64190/ Test PASSed. ---

[GitHub] spark issue #14751: [WIP][SPARK-17184][[CORE]]Replace ByteBuf with InputStre...

2016-08-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14751 **[Test build #64204 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64204/consoleFull)** for PR 14751 at commit [`7ab9ba5`](https://github.com/apache/spark/commit/7

[GitHub] spark issue #14751: [WIP][SPARK-17184][[CORE]]Replace ByteBuf with InputStre...

2016-08-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14751 **[Test build #64204 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64204/consoleFull)** for PR 14751 at commit [`7ab9ba5`](https://github.com/apache/spark/commit/

[GitHub] spark issue #14751: [WIP][SPARK-17184][[CORE]]Replace ByteBuf with InputStre...

2016-08-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14751 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #14751: [WIP][SPARK-17184][[CORE]]Replace ByteBuf with InputStre...

2016-08-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14751 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/64204/ Test FAILed. ---

[GitHub] spark issue #14718: [SPARK-16711] YarnShuffleService doesn't re-init properl...

2016-08-22 Thread tgravescs
Github user tgravescs commented on the issue: https://github.com/apache/spark/pull/14718 No, it all gets including into one assembly jar used by the nodemanagers (/spark-${project.version}-yarn-shuffle.jar) --- If your project is set up for it, you can reply to this email and have yo

[GitHub] spark pull request #14752: [SPARK-17186][SQL] remove catalog table type INDE...

2016-08-22 Thread cloud-fan
GitHub user cloud-fan opened a pull request: https://github.com/apache/spark/pull/14752 [SPARK-17186][SQL] remove catalog table type INDEX ## What changes were proposed in this pull request? Actually Spark SQL doesn't support index, the catalog table type `INDEX` is from Hi

[GitHub] spark issue #14738: [SPARK-17090][FOLLOW-UP][ML]Add expert param support to ...

2016-08-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14738 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/64203/ Test PASSed. ---

[GitHub] spark issue #14738: [SPARK-17090][FOLLOW-UP][ML]Add expert param support to ...

2016-08-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14738 **[Test build #64203 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64203/consoleFull)** for PR 14738 at commit [`f9377ae`](https://github.com/apache/spark/commit/

[GitHub] spark issue #14738: [SPARK-17090][FOLLOW-UP][ML]Add expert param support to ...

2016-08-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14738 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #14752: [SPARK-17186][SQL] remove catalog table type INDEX

2016-08-22 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/14752 cc @yhuai @gatorsmile --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark issue #14752: [SPARK-17186][SQL] remove catalog table type INDEX

2016-08-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14752 **[Test build #64205 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64205/consoleFull)** for PR 14752 at commit [`d2bc794`](https://github.com/apache/spark/commit/d

[GitHub] spark issue #10896: [SPARK-12978][SQL] Skip unnecessary final group-by when ...

2016-08-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/10896 **[Test build #64199 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64199/consoleFull)** for PR 10896 at commit [`e37ef6a`](https://github.com/apache/spark/commit/

[GitHub] spark issue #14752: [SPARK-17186][SQL] remove catalog table type INDEX

2016-08-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14752 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #10896: [SPARK-12978][SQL] Skip unnecessary final group-by when ...

2016-08-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/10896 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

<    1   2   3   4   5   6   7   8   >