[GitHub] spark issue #14569: Fixed typo

2016-08-09 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/14569 Merging in master/2.0. Thanks. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #14576: [SPARK-16391][SQL] ReduceAggregator and partial aggregat...

2016-08-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14576 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #14577: [SPARK-16986][WEB UI] Make 'Started' time, 'Completed' t...

2016-08-09 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/14577 So this change can be really annoying in debugging, when we try to correlate time shown on the UI with time in logs. --- If your project is set up for it, you can reply to this email and have your

[GitHub] spark issue #14576: [SPARK-16391][SQL] ReduceAggregator and partial aggregat...

2016-08-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14576 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/63505/ Test FAILed. ---

[GitHub] spark issue #14576: [SPARK-16391][SQL] ReduceAggregator and partial aggregat...

2016-08-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14576 **[Test build #63505 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63505/consoleFull)** for PR 14576 at commit

[GitHub] spark pull request #14472: [SPARK-16866][SQL] Infrastructure for file-based ...

2016-08-09 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/14472#discussion_r74189204 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/SQLQueryTestSuite.scala --- @@ -0,0 +1,214 @@ +/* + * Licensed to the Apache Software

[GitHub] spark issue #14577: [SPARK-16986][WEB UI] Make 'Started' time, 'Completed' t...

2016-08-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14577 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request #14577: [SPARK-16986][WEB UI] Make 'Started' time, 'Compl...

2016-08-09 Thread Sherry302
GitHub user Sherry302 opened a pull request: https://github.com/apache/spark/pull/14577 [SPARK-16986][WEB UI] Make 'Started' time, 'Completed' time and 'Last… ## What changes were proposed in this pull request? In historypage.js, format 'Started' time, 'Completed' time and

[GitHub] spark issue #14431: [SPARK-16258][SparkR] Automatically append the grouping ...

2016-08-09 Thread NarineK
Github user NarineK commented on the issue: https://github.com/apache/spark/pull/14431 My point is the following: Let's say we have the following: `var relationalGroupedDataset = df.groupBy("col1", "col2");` Now, having `relationalGroupedDataset` how can I find out the grouping

[GitHub] spark issue #14576: [SPARK-16391][SQL] ReduceAggregator and partial aggregat...

2016-08-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14576 **[Test build #63505 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63505/consoleFull)** for PR 14576 at commit

[GitHub] spark pull request #14559: [SPARK-16968]Add additional options in jdbc when ...

2016-08-09 Thread GraceH
Github user GraceH commented on a diff in the pull request: https://github.com/apache/spark/pull/14559#discussion_r74188842 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/DataFrameWriter.scala --- @@ -447,7 +447,11 @@ final class DataFrameWriter[T] private[sql](ds:

[GitHub] spark issue #14527: [SPARK-16938][SQL] `dropDuplicate` should not raise exce...

2016-08-09 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/14527 Oh, I see. I'll check `drop`, too. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #14576: [SPARK-16391][SQL] ReduceAggregator and partial a...

2016-08-09 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/14576#discussion_r74188692 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/expressions/ReduceAggregator.scala --- @@ -0,0 +1,79 @@ +/* + * Licensed to the Apache Software

[GitHub] spark issue #14527: [SPARK-16938][SQL] `dropDuplicate` should not raise exce...

2016-08-09 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/14527 This is consistent with `drop`, if we wanna fix it, we should also fix `drop`. But one thing is, `drop` has a version that can take `Column`, so we can work around this case by

[GitHub] spark issue #14222: [SPARK-16391][SQL] KeyValueGroupedDataset.reduceGroups s...

2016-08-09 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/14222 I've created a pr here for discussion, based on my playing with the API: https://github.com/apache/spark/pull/14576 --- If your project is set up for it, you can reply to this email and have your

[GitHub] spark pull request #14576: [SPARK-16391][SQL] ReduceAggregator and partial a...

2016-08-09 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/14576#discussion_r74188646 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/expressions/ReduceAggregator.scala --- @@ -0,0 +1,79 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request #14576: [SPARK-16391][SQL] ReduceAggregator and partial a...

2016-08-09 Thread rxin
GitHub user rxin opened a pull request: https://github.com/apache/spark/pull/14576 [SPARK-16391][SQL] ReduceAggregator and partial aggregation for reduceGroups function - WIP ## What changes were proposed in this pull request? This patch introduces a new ReduceAggregator

[GitHub] spark issue #14571: [SPARK-16983][SQL] Add `prettyName` for row_number, dens...

2016-08-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14571 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/63499/ Test FAILed. ---

[GitHub] spark issue #14571: [SPARK-16983][SQL] Add `prettyName` for row_number, dens...

2016-08-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14571 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #14571: [SPARK-16983][SQL] Add `prettyName` for row_number, dens...

2016-08-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14571 **[Test build #63499 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63499/consoleFull)** for PR 14571 at commit

[GitHub] spark issue #14452: [SPARK-16849][SQL] Improve subquery execution in CTE by ...

2016-08-09 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/14452 It is just bound to its own `children`, right? If you have two CTE, their children might have different expression IDs after the deduplication logics on conflicting attributes in Analyzer.

[GitHub] spark pull request #14556: [SPARK-16966][Core] Make App Name to the valid na...

2016-08-09 Thread Sherry302
Github user Sherry302 closed the pull request at: https://github.com/apache/spark/pull/14556 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request #14562: [SPARK-16973][SQL] remove the buffer offsets in I...

2016-08-09 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/14562#discussion_r74187449 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/slicedRows.scala --- @@ -0,0 +1,108 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request #14527: [SPARK-16938][SQL] `dropDuplicate` should not rai...

2016-08-09 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/14527#discussion_r74187346 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala --- @@ -1865,7 +1865,7 @@ class Dataset[T] private[sql]( val resolver =

[GitHub] spark pull request #14527: [SPARK-16938][SQL] `dropDuplicate` should not rai...

2016-08-09 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/14527#discussion_r74187361 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala --- @@ -1865,7 +1865,7 @@ class Dataset[T] private[sql]( val resolver =

[GitHub] spark issue #13701: [SPARK-15639][SPARK-16321][SQL] Push down filter at RowG...

2016-08-09 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/13701 @davies The accumulator could be released by JVM early for optimization. I made new change to prevent it. --- If your project is set up for it, you can reply to this email and have your reply

[GitHub] spark pull request #14559: [SPARK-16968]Add additional options in jdbc when ...

2016-08-09 Thread GraceH
Github user GraceH commented on a diff in the pull request: https://github.com/apache/spark/pull/14559#discussion_r74187289 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/DataFrameWriter.scala --- @@ -447,7 +447,11 @@ final class DataFrameWriter[T] private[sql](ds:

[GitHub] spark issue #14575: [SPARK-16522][MESOS] Spark application throws exception ...

2016-08-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14575 **[Test build #63504 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63504/consoleFull)** for PR 14575 at commit

[GitHub] spark pull request #14527: [SPARK-16938][SQL] `dropDuplicate` should not rai...

2016-08-09 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/14527#discussion_r74187122 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala --- @@ -1865,7 +1865,7 @@ class Dataset[T] private[sql]( val resolver =

[GitHub] spark issue #14531: [SPARK-16943] [SPARK-16942] [SQL] Fix multiple bugs in C...

2016-08-09 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/14531 @rxin Hive does not copy the table comments but it copy the column comments. The [document of Postgres](https://www.postgresql.org/docs/9.1/static/sql-createtable.html) shows it does not do it

[GitHub] spark issue #14575: [SPARK-16522][MESOS] Spark application throws exception ...

2016-08-09 Thread sun-rui
Github user sun-rui commented on the issue: https://github.com/apache/spark/pull/14575 @mgummelt, @srowen --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so,

[GitHub] spark pull request #14575: [SPARK-16522][MESOS] Spark application throws exc...

2016-08-09 Thread sun-rui
GitHub user sun-rui opened a pull request: https://github.com/apache/spark/pull/14575 [SPARK-16522][MESOS] Spark application throws exception on exit. This is backport of https://github.com/apache/spark/pull/14175 to branch 2.0 You can merge this pull request into a Git repository

[GitHub] spark issue #14452: [SPARK-16849][SQL] Improve subquery execution in CTE by ...

2016-08-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14452 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #14452: [SPARK-16849][SQL] Improve subquery execution in CTE by ...

2016-08-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14452 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/63496/ Test PASSed. ---

[GitHub] spark issue #14452: [SPARK-16849][SQL] Improve subquery execution in CTE by ...

2016-08-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14452 **[Test build #63496 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63496/consoleFull)** for PR 14452 at commit

[GitHub] spark pull request #14562: [SPARK-16973][SQL] remove the buffer offsets in I...

2016-08-09 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/14562#discussion_r74186888 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/slicedRows.scala --- @@ -0,0 +1,108 @@ +/* + * Licensed to the

[GitHub] spark issue #14452: [SPARK-16849][SQL] Improve subquery execution in CTE by ...

2016-08-09 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/14452 Yea, but other attribute references will be bound before comparing in `sameResult`. So the expression id differences shouldl be no effect. --- If your project is set up for it, you can reply to

[GitHub] spark pull request #14531: [SPARK-16943] [SPARK-16942] [SQL] Fix multiple bu...

2016-08-09 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/14531#discussion_r74186426 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/tables.scala --- @@ -80,13 +83,49 @@ case class CreateTableLikeCommand(

[GitHub] spark issue #14531: [SPARK-16943] [SPARK-16942] [SQL] Fix multiple bugs in C...

2016-08-09 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/14531 Why can't the source table be a temp table? Also why not copy the table comment? Is it the same behavior in Hive / Postgres? --- If your project is set up for it, you can reply to this email and

[GitHub] spark pull request #14531: [SPARK-16943] [SPARK-16942] [SQL] Fix multiple bu...

2016-08-09 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/14531#discussion_r74186326 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/tables.scala --- @@ -56,7 +56,14 @@ case class

[GitHub] spark issue #14452: [SPARK-16849][SQL] Improve subquery execution in CTE by ...

2016-08-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14452 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/63494/ Test PASSed. ---

[GitHub] spark issue #14452: [SPARK-16849][SQL] Improve subquery execution in CTE by ...

2016-08-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14452 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #14531: [SPARK-16943] [SPARK-16942] [SQL] Fix multiple bu...

2016-08-09 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/14531#discussion_r74186248 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/tables.scala --- @@ -80,13 +87,45 @@ case class CreateTableLikeCommand(

[GitHub] spark issue #14452: [SPARK-16849][SQL] Improve subquery execution in CTE by ...

2016-08-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14452 **[Test build #63494 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63494/consoleFull)** for PR 14452 at commit

[GitHub] spark pull request #14550: [SPARK-16959] [SQL] Rebuild Table Comment when Re...

2016-08-09 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/14550#discussion_r74186144 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/HiveDDLSuite.scala --- @@ -135,7 +135,9 @@ class HiveDDLSuite

[GitHub] spark pull request #14531: [SPARK-16943] [SPARK-16942] [SQL] Fix multiple bu...

2016-08-09 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/14531#discussion_r74186158 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/tables.scala --- @@ -56,7 +56,14 @@ case class

[GitHub] spark pull request #14531: [SPARK-16943] [SPARK-16942] [SQL] Fix multiple bu...

2016-08-09 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/14531#discussion_r74186142 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/tables.scala --- @@ -56,7 +56,14 @@ case class

[GitHub] spark issue #13701: [SPARK-15639][SPARK-16321][SQL] Push down filter at RowG...

2016-08-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13701 **[Test build #63503 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63503/consoleFull)** for PR 13701 at commit

[GitHub] spark pull request #14550: [SPARK-16959] [SQL] Rebuild Table Comment when Re...

2016-08-09 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/14550#discussion_r74185996 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/HiveDDLSuite.scala --- @@ -135,7 +135,9 @@ class HiveDDLSuite

[GitHub] spark issue #13886: [SPARK-16185] [SQL] Better Error Messages When Creating ...

2016-08-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13886 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/63495/ Test PASSed. ---

[GitHub] spark issue #13886: [SPARK-16185] [SQL] Better Error Messages When Creating ...

2016-08-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13886 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #13886: [SPARK-16185] [SQL] Better Error Messages When Creating ...

2016-08-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13886 **[Test build #63495 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63495/consoleFull)** for PR 13886 at commit

[GitHub] spark issue #14118: [SPARK-16462][SPARK-16460][SPARK-15144][SQL] Make CSV ca...

2016-08-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14118 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/63493/ Test PASSed. ---

[GitHub] spark issue #14118: [SPARK-16462][SPARK-16460][SPARK-15144][SQL] Make CSV ca...

2016-08-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14118 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #14452: [SPARK-16849][SQL] Improve subquery execution in CTE by ...

2016-08-09 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/14452 If `sameResult` does not consider the expression id, will it return `true` if two tables have the same name columns? If my understanding is not wrong, it only tolerates `Alias` with the

[GitHub] spark issue #14118: [SPARK-16462][SPARK-16460][SPARK-15144][SQL] Make CSV ca...

2016-08-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14118 **[Test build #63493 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63493/consoleFull)** for PR 14118 at commit

[GitHub] spark pull request #14155: [SPARK-16498][SQL] move hive hack for data source...

2016-08-09 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/14155#discussion_r74184835 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveExternalCatalog.scala --- @@ -363,3 +503,82 @@ private[spark] class

[GitHub] spark pull request #14155: [SPARK-16498][SQL] move hive hack for data source...

2016-08-09 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/14155#discussion_r74184777 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveExternalCatalog.scala --- @@ -81,6 +86,19 @@ private[spark] class

[GitHub] spark pull request #14155: [SPARK-16498][SQL] move hive hack for data source...

2016-08-09 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/14155#discussion_r74184724 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/command/DDLSuite.scala --- @@ -93,7 +92,7 @@ class DDLSuite extends QueryTest with

[GitHub] spark pull request #14571: [SPARK-16983][SQL] Add `prettyName` for row_numbe...

2016-08-09 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/14571#discussion_r74184702 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/windowExpressions.scala --- @@ -675,6 +675,7 @@ case class

[GitHub] spark issue #14571: [SPARK-16983][SQL] Add `prettyName` for row_number, dens...

2016-08-09 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/14571 Finally, * Add `prettyName` and remove `sql` for `row_number` and `cume_dist`. * Add `prettyName` for `dense_rank` and `percent_rank`. * `LogicalPlanToSQLSuite` does not changed.

[GitHub] spark pull request #14571: [SPARK-16983][SQL] Add `prettyName` for row_numbe...

2016-08-09 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/14571#discussion_r74184622 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/windowExpressions.scala --- @@ -675,6 +675,7 @@ case class DenseRank(children:

[GitHub] spark issue #14571: [SPARK-16983][SQL] Add `prettyName` for row_number, dens...

2016-08-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14571 **[Test build #63502 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63502/consoleFull)** for PR 14571 at commit

[GitHub] spark issue #13146: [SPARK-13081][PYSPARK][SPARK_SUBMIT]. Allow set pythonEx...

2016-08-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13146 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/63490/ Test PASSed. ---

[GitHub] spark issue #13146: [SPARK-13081][PYSPARK][SPARK_SUBMIT]. Allow set pythonEx...

2016-08-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13146 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #13701: [SPARK-15639][SPARK-16321][SQL] Push down filter at RowG...

2016-08-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13701 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/63492/ Test FAILed. ---

[GitHub] spark issue #13701: [SPARK-15639][SPARK-16321][SQL] Push down filter at RowG...

2016-08-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13701 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #13146: [SPARK-13081][PYSPARK][SPARK_SUBMIT]. Allow set pythonEx...

2016-08-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13146 **[Test build #63490 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63490/consoleFull)** for PR 13146 at commit

[GitHub] spark issue #13701: [SPARK-15639][SPARK-16321][SQL] Push down filter at RowG...

2016-08-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13701 **[Test build #63492 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63492/consoleFull)** for PR 13701 at commit

[GitHub] spark issue #14561: [SPARK-16972][CORE] Move DriverEndpoint out of CoarseGra...

2016-08-09 Thread lshmouse
Github user lshmouse commented on the issue: https://github.com/apache/spark/pull/14561 @jerryshao The patch doesn't just change the code structure. It distinguish the responsibility of these two classes. What's more, it's the first step to refactor the schedule path, because

[GitHub] spark issue #14571: [SPARK-16983][SQL] Add `prettyName` for row_number, dens...

2016-08-09 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/14571 Hm, interesting. Sorry, @rxin. I missed that the window functions need to hide their children. The default behavior should be overrided. I'll revert the last commit. Please refer the

[GitHub] spark issue #14547: [SPARK-16718][MLlib] gbm-style treeboost [WIP]

2016-08-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14547 **[Test build #63500 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63500/consoleFull)** for PR 14547 at commit

[GitHub] spark issue #14571: [SPARK-16983][SQL] Add `prettyName` for row_number, dens...

2016-08-09 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/14571 Oh, @rxin . It will break `LogicalPlanToSQLSuite` since the upper case function names like `ROW_NUMBER` is changed into `row_number`. I need to update `LogicalPlanToSQLSuite`, too. I will

[GitHub] spark issue #14155: [SPARK-16498][SQL] move hive hack for data source table ...

2016-08-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14155 **[Test build #63501 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63501/consoleFull)** for PR 14155 at commit

[GitHub] spark issue #14431: [SPARK-16258][SparkR] Automatically append the grouping ...

2016-08-09 Thread shivaram
Github user shivaram commented on the issue: https://github.com/apache/spark/pull/14431 I'm not sure I understand the question. Also some of the SQL committers like @liancheng might be able to answer this better --- If your project is set up for it, you can reply to this email and

[GitHub] spark issue #14571: [SPARK-16983][SQL] Add `prettyName` for row_number, dens...

2016-08-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14571 **[Test build #63499 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63499/consoleFull)** for PR 14571 at commit

[GitHub] spark issue #13775: [SPARK-16060][SQL] Vectorized Orc reader

2016-08-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13775 **[Test build #63498 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63498/consoleFull)** for PR 13775 at commit

[GitHub] spark pull request #13775: [SPARK-16060][SQL] Vectorized Orc reader

2016-08-09 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/13775#discussion_r74183317 --- Diff: sql/hive/src/main/java/org/apache/hadoop/hive/ql/io/orc/VectorizedSparkOrcNewRecordReader.java --- @@ -0,0 +1,317 @@ +/* + * Licensed to

[GitHub] spark pull request #14571: [SPARK-16983][SQL] Add `prettyName` for row_numbe...

2016-08-09 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/14571#discussion_r74183323 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/windowExpressions.scala --- @@ -478,6 +478,7 @@ object

[GitHub] spark pull request #14566: Make logDir easily copy/paste-able

2016-08-09 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/14566 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request #14571: [SPARK-16983][SQL] Add `prettyName` for row_numbe...

2016-08-09 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/14571#discussion_r74183057 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/windowExpressions.scala --- @@ -478,6 +478,7 @@ object SizeBasedWindowFunction

[GitHub] spark pull request #14571: [SPARK-16983][SQL] Add `prettyName` for row_numbe...

2016-08-09 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/14571#discussion_r74183002 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/windowExpressions.scala --- @@ -478,6 +478,7 @@ object

[GitHub] spark issue #14566: Make logDir easily copy/paste-able

2016-08-09 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/14566 Merging in master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or

[GitHub] spark pull request #14571: [SPARK-16983][SQL] Add `prettyName` for row_numbe...

2016-08-09 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/14571#discussion_r74182897 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/windowExpressions.scala --- @@ -478,6 +478,7 @@ object

[GitHub] spark pull request #14562: [SPARK-16973][SQL] remove the buffer offsets in I...

2016-08-09 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/14562#discussion_r74182507 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/slicedRows.scala --- @@ -0,0 +1,108 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request #14562: [SPARK-16973][SQL] remove the buffer offsets in I...

2016-08-09 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/14562#discussion_r74182501 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/slicedRows.scala --- @@ -0,0 +1,108 @@ +/* + * Licensed to the Apache

[GitHub] spark issue #14573: [SPARK-16984][SQL] don't try whole dataset immediately w...

2016-08-09 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/14573 Actually while you are at this, can you make the ramp-up configurable? Add an entry to SQLConf; something like spark.sql.limit.scaleUpFactor --- If your project is set up for it, you can

[GitHub] spark issue #14574: [SPARK-16985] Change dataFormat from yyyyMMddHHmm to yyy...

2016-08-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14574 **[Test build #63497 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63497/consoleFull)** for PR 14574 at commit

[GitHub] spark issue #14572: [SPARK-16552] [FOLLOW-UP] [SQL] Store the Inferred Schem...

2016-08-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14572 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #14574: [SPARK-16985] Change dataFormat from yyyyMMddHHmm...

2016-08-09 Thread shenh062326
GitHub user shenh062326 opened a pull request: https://github.com/apache/spark/pull/14574 [SPARK-16985] Change dataFormat from MMddHHmm to MMddHHmmss ## What changes were proposed in this pull request? In our cluster, sometimes the sql output maybe overrided. When I

[GitHub] spark issue #14572: [SPARK-16552] [FOLLOW-UP] [SQL] Store the Inferred Schem...

2016-08-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14572 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/63489/ Test PASSed. ---

[GitHub] spark issue #14572: [SPARK-16552] [FOLLOW-UP] [SQL] Store the Inferred Schem...

2016-08-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14572 **[Test build #63489 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63489/consoleFull)** for PR 14572 at commit

[GitHub] spark pull request #14571: [SPARK-16983][SQL] Add `prettyName` for row_numbe...

2016-08-09 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/14571#discussion_r74182139 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/windowExpressions.scala --- @@ -478,6 +478,7 @@ object SizeBasedWindowFunction

[GitHub] spark issue #14452: [SPARK-16849][SQL] Improve subquery execution in CTE by ...

2016-08-09 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/14452 I think the deduplicating the conflicting attributes in Analyzer is generating new expression ids. `sameResult` is tolerant of expression id differences. --- If your project is set up for it, you

[GitHub] spark issue #14452: [SPARK-16849][SQL] Improve subquery execution in CTE by ...

2016-08-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14452 **[Test build #63496 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63496/consoleFull)** for PR 14452 at commit

[GitHub] spark pull request #13775: [SPARK-16060][SQL] Vectorized Orc reader

2016-08-09 Thread dafrista
Github user dafrista commented on a diff in the pull request: https://github.com/apache/spark/pull/13775#discussion_r74180849 --- Diff: sql/hive/src/main/java/org/apache/hadoop/hive/ql/io/orc/VectorizedSparkOrcNewRecordReader.java --- @@ -0,0 +1,317 @@ +/* + * Licensed to

[GitHub] spark issue #14534: [SPARK-16941]Use concurrentHashMap instead of scala Map ...

2016-08-09 Thread SaintBacchus
Github user SaintBacchus commented on the issue: https://github.com/apache/spark/pull/14534 any other comment? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark issue #13886: [SPARK-16185] [SQL] Better Error Messages When Creating ...

2016-08-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13886 **[Test build #63495 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63495/consoleFull)** for PR 13886 at commit

[GitHub] spark issue #14452: [SPARK-16849][SQL] Improve subquery execution in CTE by ...

2016-08-09 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/14452 Will it affect `sameResult`? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark issue #14452: [SPARK-16849][SQL] Improve subquery execution in CTE by ...

2016-08-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14452 **[Test build #63494 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63494/consoleFull)** for PR 14452 at commit

  1   2   3   4   5   6   7   8   >