[GitHub] spark issue #14432: [SPARK-16765] [Example] Added example for k-means to pre...

2016-07-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14432 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request #14432: [SPARK-16765] [Example] Added example for k-means...

2016-07-31 Thread manishatGit
GitHub user manishatGit opened a pull request: https://github.com/apache/spark/pull/14432 [SPARK-16765] [Example] Added example for k-means to predict with categorical features ## What changes were proposed in this pull request? Added example for K-Means as Pipeline API

[GitHub] spark issue #14065: [SPARK-14743][YARN] Add a configurable credential manage...

2016-07-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14065 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/63068/ Test PASSed. ---

[GitHub] spark issue #14065: [SPARK-14743][YARN] Add a configurable credential manage...

2016-07-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14065 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #14065: [SPARK-14743][YARN] Add a configurable credential manage...

2016-07-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14065 **[Test build #63068 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63068/consoleFull)** for PR 14065 at commit

[GitHub] spark issue #14363: [SPARK-16731][SQL] use StructType in CatalogTable and re...

2016-07-31 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/14363 Oh sorry I misread our parser rules. `varchar(length)` is supported but the `length` is ignored. I checked with hive again, looks like the only unsupported data type is `UNIONTYPE` --- If your

[GitHub] spark issue #14363: [SPARK-16731][SQL] use StructType in CatalogTable and re...

2016-07-31 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/14363 ```SCALA TestHive.sessionState.metadataHive.runSqlHive("CREATE TABLE test (id varchar(50))") TestHive.sessionState.metadataHive.runSqlHive("INSERT INTO TABLE test VALUES ('4')")

[GitHub] spark issue #14430: [SPARK-16825] [SQL] Replace hive.default.fileformat by s...

2016-07-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14430 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #14430: [SPARK-16825] [SQL] Replace hive.default.fileformat by s...

2016-07-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14430 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/63067/ Test PASSed. ---

[GitHub] spark issue #14430: [SPARK-16825] [SQL] Replace hive.default.fileformat by s...

2016-07-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14430 **[Test build #63067 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63067/consoleFull)** for PR 14430 at commit

[GitHub] spark pull request #14368: [SPARK-16734][EXAMPLES][SQL] Revise examples of a...

2016-07-31 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/14368#discussion_r72923711 --- Diff: examples/src/main/r/RSparkSQLExample.R --- @@ -18,31 +18,43 @@ library(SparkR) # $example on:init_session$

[GitHub] spark issue #14355: [SPARK-16726][SQL] Improve `Union/Intersect/Except` erro...

2016-07-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14355 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/63066/ Test PASSed. ---

[GitHub] spark issue #14355: [SPARK-16726][SQL] Improve `Union/Intersect/Except` erro...

2016-07-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14355 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #14368: [SPARK-16734][EXAMPLES][SQL] Revise examples of all lang...

2016-07-31 Thread yhuai
Github user yhuai commented on the issue: https://github.com/apache/spark/pull/14368 @liancheng after second thought, I think it makes sense to also merge it to branch 2.0 to avoid potential conflicts on doc fixes. --- If your project is set up for it, you can reply to this email

[GitHub] spark issue #14355: [SPARK-16726][SQL] Improve `Union/Intersect/Except` erro...

2016-07-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14355 **[Test build #63066 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63066/consoleFull)** for PR 14355 at commit

[GitHub] spark issue #14363: [SPARK-16731][SQL] use StructType in CatalogTable and re...

2016-07-31 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/14363 for a hive table(created by hive) with `varchar(length)` column, we can describe it but can't read data from it before this PR. Now we can't describe it. Do you think we should fix it? BTW there

[GitHub] spark issue #14431: [SPARK-16258][SparkR] Automatically append the grouping ...

2016-07-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14431 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/63061/ Test PASSed. ---

[GitHub] spark issue #14431: [SPARK-16258][SparkR] Automatically append the grouping ...

2016-07-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14431 Build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark issue #14431: [SPARK-16258][SparkR] Automatically append the grouping ...

2016-07-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14431 **[Test build #63061 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63061/consoleFull)** for PR 14431 at commit

[GitHub] spark issue #14431: [SPARK-16258][SparkR] Automatically append the grouping ...

2016-07-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14431 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #14431: [SPARK-16258][SparkR] Automatically append the grouping ...

2016-07-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14431 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/63065/ Test PASSed. ---

[GitHub] spark issue #11105: [SPARK-12469][CORE] Data Property accumulators for Spark

2016-07-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/11105 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/63063/ Test PASSed. ---

[GitHub] spark issue #11105: [SPARK-12469][CORE] Data Property accumulators for Spark

2016-07-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/11105 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #14431: [SPARK-16258][SparkR] Automatically append the grouping ...

2016-07-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14431 **[Test build #63065 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63065/consoleFull)** for PR 14431 at commit

[GitHub] spark issue #11105: [SPARK-12469][CORE] Data Property accumulators for Spark

2016-07-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/11105 **[Test build #63063 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63063/consoleFull)** for PR 11105 at commit

[GitHub] spark issue #14431: [SPARK-16258][SparkR] Automatically append the grouping ...

2016-07-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14431 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/63064/ Test PASSed. ---

[GitHub] spark issue #14431: [SPARK-16258][SparkR] Automatically append the grouping ...

2016-07-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14431 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #14431: [SPARK-16258][SparkR] Automatically append the grouping ...

2016-07-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14431 **[Test build #63064 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63064/consoleFull)** for PR 14431 at commit

[GitHub] spark issue #14419: [SPARK-16814][SQL] Fix deprecated parquet constructor us...

2016-07-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14419 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/63062/ Test PASSed. ---

[GitHub] spark issue #14419: [SPARK-16814][SQL] Fix deprecated parquet constructor us...

2016-07-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14419 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #14419: [SPARK-16814][SQL] Fix deprecated parquet constructor us...

2016-07-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14419 **[Test build #63062 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63062/consoleFull)** for PR 14419 at commit

[GitHub] spark issue #14065: [SPARK-14743][YARN] Add a configurable credential manage...

2016-07-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14065 **[Test build #63068 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63068/consoleFull)** for PR 14065 at commit

[GitHub] spark issue #14363: [SPARK-16731][SQL] use StructType in CatalogTable and re...

2016-07-31 Thread yhuai
Github user yhuai commented on the issue: https://github.com/apache/spark/pull/14363 Thanks. But, what are specific cases are not supported? If there is any case, we should make change to support that, right? --- If your project is set up for it, you can reply to this email and have

[GitHub] spark issue #14430: [SPARK-16825] [SQL] Replace hive.default.fileformat by s...

2016-07-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14430 **[Test build #63067 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63067/consoleFull)** for PR 14430 at commit

[GitHub] spark issue #11105: [SPARK-12469][CORE] Data Property accumulators for Spark

2016-07-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/11105 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/63060/ Test FAILed. ---

[GitHub] spark issue #11105: [SPARK-12469][CORE] Data Property accumulators for Spark

2016-07-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/11105 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #11105: [SPARK-12469][CORE] Data Property accumulators for Spark

2016-07-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/11105 **[Test build #63060 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63060/consoleFull)** for PR 11105 at commit

[GitHub] spark issue #14431: [SPARK-16258][SparkR] Automatically append the grouping ...

2016-07-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14431 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/63059/ Test PASSed. ---

[GitHub] spark issue #14431: [SPARK-16258][SparkR] Automatically append the grouping ...

2016-07-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14431 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #14431: [SPARK-16258][SparkR] Automatically append the grouping ...

2016-07-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14431 **[Test build #63059 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63059/consoleFull)** for PR 14431 at commit

[GitHub] spark pull request #11209: [SPARK-13325][SQL] Create a 64-bit hashcode expre...

2016-07-31 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/11209#discussion_r72918850 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/HashBenchmark.scala --- @@ -119,11 +146,12 @@ object HashBenchmark { .add("map",

[GitHub] spark issue #14355: [SPARK-16726][SQL] Improve `Union/Intersect/Except` erro...

2016-07-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14355 **[Test build #63066 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63066/consoleFull)** for PR 14355 at commit

[GitHub] spark issue #14363: [SPARK-16731][SQL] use StructType in CatalogTable and re...

2016-07-31 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/14363 > Do we know which hive type strings cannot be parsed by spark? `varchar(length)` and `char(length)`. see https://github.com/apache/spark/pull/14363#discussion_r72735867 for what we

[GitHub] spark pull request #14355: [SPARK-16726][SQL] Improve `Union/Intersect/Excep...

2016-07-31 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/14355#discussion_r72917596 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/CheckAnalysis.scala --- @@ -280,6 +267,32 @@ trait CheckAnalysis extends

[GitHub] spark issue #14397: [SPARK-16771][SQL] WITH clause should not fall into infi...

2016-07-31 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/14397 Thank you for review, @hvanhovell . For the recursive CTE queries, traditional DBMS supports optional `RECURSIVE` keyword which computing **closures**, i.e., it iterates the subquery

[GitHub] spark issue #14420: [SPARK-14204] [SQL] register driverClass rather than use...

2016-07-31 Thread zzcclp
Github user zzcclp commented on the issue: https://github.com/apache/spark/pull/14420 @srowen will this pr be meged into branch-2.0? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark issue #14431: [SPARK-16258][SparkR] Automatically append the grouping ...

2016-07-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14431 **[Test build #63065 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63065/consoleFull)** for PR 14431 at commit

[GitHub] spark issue #14431: [SPARK-16258][SparkR] Automatically append the grouping ...

2016-07-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14431 **[Test build #63064 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63064/consoleFull)** for PR 14431 at commit

[GitHub] spark issue #12896: [SPARK-14489][ML][PYSPARK] ALS unknown user/item predict...

2016-07-31 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/12896 This looks good to me pending since annotation bump in Python. Personally I think a docs update could be a good follow up - but since were tagging it as an expert param for now I think this makes

[GitHub] spark pull request #12896: [SPARK-14489][ML][PYSPARK] ALS unknown user/item ...

2016-07-31 Thread holdenk
Github user holdenk commented on a diff in the pull request: https://github.com/apache/spark/pull/12896#discussion_r72916851 --- Diff: python/pyspark/ml/recommendation.py --- @@ -332,6 +338,20 @@ def getFinalStorageLevel(self): """ return

[GitHub] spark issue #11105: [SPARK-12469][CORE] Data Property accumulators for Spark

2016-07-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/11105 **[Test build #63063 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63063/consoleFull)** for PR 11105 at commit

[GitHub] spark issue #14419: [SPARK-16814][SQL] Fix deprecated parquet constructor us...

2016-07-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14419 **[Test build #63062 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63062/consoleFull)** for PR 14419 at commit

[GitHub] spark issue #11105: [SPARK-12469][CORE] Data Property accumulators for Spark

2016-07-31 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/11105 Kafka failure seems unlikely to be related, jenkins retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark issue #14431: [SPARK-16258][SparkR] Automatically append the grouping ...

2016-07-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14431 **[Test build #63061 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63061/consoleFull)** for PR 14431 at commit

[GitHub] spark pull request #14431: [SPARK-16258][SparkR] Automatically append the gr...

2016-07-31 Thread NarineK
Github user NarineK commented on a diff in the pull request: https://github.com/apache/spark/pull/14431#discussion_r72916339 --- Diff: docs/sparkr.md --- @@ -429,19 +431,19 @@ result <- gapplyCollect( df, "waiting", function(key, x) { -y <-

[GitHub] spark pull request #14431: [SPARK-16258][SparkR] Automatically append the gr...

2016-07-31 Thread NarineK
Github user NarineK commented on a diff in the pull request: https://github.com/apache/spark/pull/14431#discussion_r72916310 --- Diff: docs/sparkr.md --- @@ -398,23 +398,25 @@ and Spark. {% highlight r %} # Determine six waiting times with the largest eruption time

[GitHub] spark issue #11105: [SPARK-12469][CORE] Data Property accumulators for Spark

2016-07-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/11105 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/63058/ Test FAILed. ---

[GitHub] spark issue #11105: [SPARK-12469][CORE] Data Property accumulators for Spark

2016-07-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/11105 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #11105: [SPARK-12469][CORE] Data Property accumulators for Spark

2016-07-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/11105 **[Test build #63058 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63058/consoleFull)** for PR 11105 at commit

[GitHub] spark pull request #13704: [SPARK-15985][SQL] Eliminate redundant cast from ...

2016-07-31 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/13704#discussion_r72915873 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -1441,6 +1441,12 @@ object PushPredicateThroughJoin

[GitHub] spark pull request #14419: [SPARK-16814][SQL] Fix deprecated parquet constru...

2016-07-31 Thread holdenk
Github user holdenk commented on a diff in the pull request: https://github.com/apache/spark/pull/14419#discussion_r72915861 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetCompatibilityTest.scala --- @@ -119,8 +119,18 @@ private[sql]

[GitHub] spark pull request #14413: [SPARK-16805][SQL] Log timezone when query result...

2016-07-31 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/14413 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request #14363: [SPARK-16731][SQL] use StructType in CatalogTable...

2016-07-31 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/14363 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #14413: [SPARK-16805][SQL] Log timezone when query result does n...

2016-07-31 Thread yhuai
Github user yhuai commented on the issue: https://github.com/apache/spark/pull/14413 LGTM. Merging to master and branch 2.0. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #14279: [SPARK-16216][SQL] Write Timestamp and Date in ISO 8601 ...

2016-07-31 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/14279 I might have to make a follow-up for.. - making `DatatypeConverter.parseDateTime` to be safe with early dates - enabling `TimestampType` tests in `JsonHadoopFsRelationSuite`

[GitHub] spark issue #14386: [SPARK-16761][DOC][ML] Fix doc link in docs/ml-guide.md

2016-07-31 Thread sundapeng
Github user sundapeng commented on the issue: https://github.com/apache/spark/pull/14386 Thank @BryanCutler and @srowen for your review --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark issue #14430: [SPARK-16825] [SQL] Replace hive.default.fileformat by s...

2016-07-31 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/14430 @rxin That is definitely true! I had the same concern when changing the code. Now, we are having two types of tables: one is Data source tables and the other is Hive tables. The default

[GitHub] spark issue #14363: [SPARK-16731][SQL] use StructType in CatalogTable and re...

2016-07-31 Thread yhuai
Github user yhuai commented on the issue: https://github.com/apache/spark/pull/14363 LGTM. Thanks. Merging to master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #11105: [SPARK-12469][CORE] Data Property accumulators for Spark

2016-07-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/11105 **[Test build #63060 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63060/consoleFull)** for PR 11105 at commit

[GitHub] spark issue #14279: [SPARK-16216][SQL] Write Timestamp and Date in ISO 8601 ...

2016-07-31 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/14279 I think it is okay. Just to make sure, - **This writes date and timestamps fine**: ```scala val path = "/tmp" val date = java.sql.Date.valueOf("1970-01-01")

[GitHub] spark issue #14431: [SPARK-16258][SparkR][WIP] Gapply add key attach option

2016-07-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14431 **[Test build #63059 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63059/consoleFull)** for PR 14431 at commit

[GitHub] spark pull request #14431: [SPARK-16258][SparkR][WIP] Gapply add key attach ...

2016-07-31 Thread NarineK
GitHub user NarineK opened a pull request: https://github.com/apache/spark/pull/14431 [SPARK-16258][SparkR][WIP] Gapply add key attach option ## What changes were proposed in this pull request? The following pull request addresses the new feature request described in

[GitHub] spark issue #14413: [SPARK-16805][SQL] Log timezone when query result does n...

2016-07-31 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/14413 cc @yhuai --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] spark issue #14430: [SPARK-16825] [SQL] Replace hive.default.fileformat by s...

2016-07-31 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/14430 We already have default sources. It is very confusing to have default format on top of that. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as

[GitHub] spark pull request #14419: [SPARK-16814][SQL] Fix deprecated parquet constru...

2016-07-31 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/14419#discussion_r72914240 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetCompatibilityTest.scala --- @@ -119,8 +119,18 @@

[GitHub] spark pull request #14419: [SPARK-16814][SQL] Fix deprecated parquet constru...

2016-07-31 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/14419#discussion_r72914197 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetCompatibilityTest.scala --- @@ -119,8 +119,18 @@

[GitHub] spark issue #11105: [SPARK-12469][CORE] Data Property accumulators for Spark

2016-07-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/11105 **[Test build #63058 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63058/consoleFull)** for PR 11105 at commit

[GitHub] spark issue #11105: [SPARK-12469][CORE] Data Property accumulators for Spark

2016-07-31 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/11105 Now that the 2.0 release is fully out (release artifacts and all) - maybe @rxin would have a chance to take another look at this (and @techaddict who did some AccumulatorV2 work as well)? --- If

[GitHub] spark issue #14419: [SPARK-16814][SQL][WIP] Fix deprecated parquet construct...

2016-07-31 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/14419 @HyukjinKwon no worries - just wanted to make sure we weren't accidentally duplicating eachothers efforts :) --- If your project is set up for it, you can reply to this email and have your reply

[GitHub] spark issue #14398: [SPARK-16774][SQL][WIP] Fix use of deprecated timestamp ...

2016-07-31 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/14398 Just for good measure lets re-run the jenkins tests. Jenkins retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well.

[GitHub] spark issue #14411: [SPARK-16804][SQL] Correlated subqueries containing LIMI...

2016-07-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14411 **[Test build #3199 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3199/consoleFull)** for PR 14411 at commit

[GitHub] spark issue #14430: [SPARK-16825] [SQL] Replace hive.default.fileformat by s...

2016-07-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14430 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #14430: [SPARK-16825] [SQL] Replace hive.default.fileformat by s...

2016-07-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14430 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/63057/ Test PASSed. ---

[GitHub] spark issue #14430: [SPARK-16825] [SQL] Replace hive.default.fileformat by s...

2016-07-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14430 **[Test build #63057 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63057/consoleFull)** for PR 14430 at commit

[GitHub] spark issue #14430: [SPARK-16825] [SQL] Replace hive.default.fileformat by s...

2016-07-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14430 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/63056/ Test PASSed. ---

[GitHub] spark issue #14430: [SPARK-16825] [SQL] Replace hive.default.fileformat by s...

2016-07-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14430 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #14430: [SPARK-16825] [SQL] Replace hive.default.fileformat by s...

2016-07-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14430 **[Test build #63056 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63056/consoleFull)** for PR 14430 at commit

[GitHub] spark pull request #14355: [SPARK-16726][SQL] Improve `Union/Intersect/Excep...

2016-07-31 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/14355#discussion_r72912491 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/CheckAnalysis.scala --- @@ -280,6 +267,32 @@ trait CheckAnalysis extends

[GitHub] spark pull request #14397: [SPARK-16771][SQL] WITH clause should not fall in...

2016-07-31 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/14397#discussion_r72912387 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala --- @@ -131,17 +131,17 @@ class Analyzer( def

[GitHub] spark issue #14397: [SPARK-16771][SQL] WITH clause should not fall into infi...

2016-07-31 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/14397 @dongjoon-hyun I think this has merit. I do have one question, what do other databases do? Like postgresql, mysql, sqlserver and others? --- If your project is set up for it, you can reply to

[GitHub] spark issue #14411: [SPARK-16804][SQL] Correlated subqueries containing LIMI...

2016-07-31 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/14411 Ok, this looks pretty good. One overall comment: We are basically blacklisting operators here (which is fine IMO), should we check if there any other operators we should about? If there are we

[GitHub] spark issue #14411: [SPARK-16804][SQL] Correlated subqueries containing LIMI...

2016-07-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14411 **[Test build #3199 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3199/consoleFull)** for PR 14411 at commit

[GitHub] spark issue #14430: [SPARK-16825] [SQL] Replace hive.default.fileformat by s...

2016-07-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14430 **[Test build #63057 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63057/consoleFull)** for PR 14430 at commit

[GitHub] spark issue #14428: [SPARK-16810] Refactor registerSinks with multiple const...

2016-07-31 Thread lovexi
Github user lovexi commented on the issue: https://github.com/apache/spark/pull/14428 Oh, my bad. Added unit test in `MetricsSystemSuite ` for this already. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark issue #14363: [SPARK-16731][SQL] use StructType in CatalogTable and re...

2016-07-31 Thread yhuai
Github user yhuai commented on the issue: https://github.com/apache/spark/pull/14363 Do we know which hive type strings cannot be parsed by spark? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark issue #14430: [SPARK-16825] [SQL] Replace hive.default.fileformat by s...

2016-07-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14430 **[Test build #63056 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63056/consoleFull)** for PR 14430 at commit

[GitHub] spark pull request #14430: [SPARK-16825] [SQL] Replace hive.default.fileform...

2016-07-31 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/14430#discussion_r72910955 --- Diff: sql/hive/compatibility/src/test/scala/org/apache/spark/sql/hive/execution/HiveWindowFunctionQuerySuite.scala --- @@ -89,15 +89,6 @@ class

[GitHub] spark pull request #14430: [SPARK-16825] [SQL] Replace hive.default.fileform...

2016-07-31 Thread gatorsmile
GitHub user gatorsmile opened a pull request: https://github.com/apache/spark/pull/14430 [SPARK-16825] [SQL] Replace hive.default.fileformat by spark.sql.default.fileformat ### What changes were proposed in this pull request? Currently, we are using `hive.default.fileformat`,

[GitHub] spark pull request #14363: [SPARK-16731][SQL] use StructType in CatalogTable...

2016-07-31 Thread yhuai
Github user yhuai commented on a diff in the pull request: https://github.com/apache/spark/pull/14363#discussion_r72910844 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/interface.scala --- @@ -78,28 +78,6 @@ object CatalogStorageFormat { }

[GitHub] spark issue #14279: [SPARK-16216][SQL] Write Timestamp and Date in ISO 8601 ...

2016-07-31 Thread deanchen
Github user deanchen commented on the issue: https://github.com/apache/spark/pull/14279 @srowen ltgm, thanks for chiming in! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

  1   2   >