Github user yhuai commented on a diff in the pull request:
https://github.com/apache/spark/pull/14155#discussion_r75431775
--- Diff:
sql/hive/src/test/scala/org/apache/spark/sql/hive/MetastoreDataSourcesSuite.scala
---
@@ -49,6 +49,8 @@ class MetastoreDataSourcesSuite extends
Github user yhuai commented on a diff in the pull request:
https://github.com/apache/spark/pull/14155#discussion_r75431528
--- Diff:
sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveExternalCatalog.scala ---
@@ -200,22 +348,73 @@ private[spark] class HiveExternalCatalog
Github user yhuai commented on a diff in the pull request:
https://github.com/apache/spark/pull/14155#discussion_r75431369
--- Diff:
sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveExternalCatalog.scala ---
@@ -144,16 +163,172 @@ private[spark] class HiveExternalCatalog
Github user yhuai commented on a diff in the pull request:
https://github.com/apache/spark/pull/14155#discussion_r75431206
--- Diff:
sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveExternalCatalog.scala ---
@@ -144,16 +161,147 @@ private[spark] class HiveExternalCatalog
Github user yhuai commented on a diff in the pull request:
https://github.com/apache/spark/pull/14155#discussion_r75250883
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/command/createDataSourceTables.scala
---
@@ -97,16 +92,17 @@ case class
Github user yhuai commented on a diff in the pull request:
https://github.com/apache/spark/pull/14155#discussion_r75232851
--- Diff:
sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveExternalCatalog.scala ---
@@ -81,6 +86,18 @@ private[spark] class HiveExternalCatalog(client
Repository: spark
Updated Branches:
refs/heads/branch-1.6 5c34029b8 -> 60de30faf
[SPARK-17102][SQL] bypass UserDefinedGenerator for json format check
We use reflection to convert `TreeNode` to json string, and currently don't
support arbitrary object. `UserDefinedGenerator` takes a function
Repository: spark
Updated Branches:
refs/heads/master 0b0c8b95e -> 928ca1c6d
[SPARK-17102][SQL] bypass UserDefinedGenerator for json format check
## What changes were proposed in this pull request?
We use reflection to convert `TreeNode` to json string, and currently don't
support arbitrary
Repository: spark
Updated Branches:
refs/heads/branch-2.0 22c7660a8 -> 394d59866
[SPARK-17102][SQL] bypass UserDefinedGenerator for json format check
## What changes were proposed in this pull request?
We use reflection to convert `TreeNode` to json string, and currently don't
support
Github user yhuai commented on the issue:
https://github.com/apache/spark/pull/14679
I am merging this to master, branch 2.0, and branch 1.6. So, we will not
hit this issue. I have created
https://issues.apache.org/jira/browse/SPARK-17109 to investigate the root cause.
---
If your
Github user yhuai commented on the issue:
https://github.com/apache/spark/pull/14679
Looks like it is caused by `Seq[(DataType, Boolean, String)]` in
`UserDefinedGenerator`? I think it is fine to bypass it for now. But, it will
be good to address issues that causing toJson crash
Github user yhuai commented on the issue:
https://github.com/apache/spark/pull/14155
I have took a close look at `HiveExternalCatalog`. My overall feeling is
that the current version still not very clear and people may have a hard time
to understand the code. Let me also think about
Github user yhuai commented on a diff in the pull request:
https://github.com/apache/spark/pull/14155#discussion_r75064724
--- Diff:
sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveExternalCatalog.scala ---
@@ -81,6 +86,18 @@ private[spark] class HiveExternalCatalog(client
Github user yhuai commented on a diff in the pull request:
https://github.com/apache/spark/pull/14155#discussion_r75064560
--- Diff:
sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveExternalCatalog.scala ---
@@ -200,22 +348,73 @@ private[spark] class HiveExternalCatalog
Github user yhuai commented on a diff in the pull request:
https://github.com/apache/spark/pull/14155#discussion_r75064362
--- Diff:
sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveExternalCatalog.scala ---
@@ -200,22 +348,73 @@ private[spark] class HiveExternalCatalog
Github user yhuai commented on a diff in the pull request:
https://github.com/apache/spark/pull/14155#discussion_r75063920
--- Diff:
sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveExternalCatalog.scala ---
@@ -144,16 +161,147 @@ private[spark] class HiveExternalCatalog
Github user yhuai commented on a diff in the pull request:
https://github.com/apache/spark/pull/14155#discussion_r75063736
--- Diff:
sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveExternalCatalog.scala ---
@@ -144,16 +161,147 @@ private[spark] class HiveExternalCatalog
Github user yhuai commented on a diff in the pull request:
https://github.com/apache/spark/pull/14155#discussion_r75063676
--- Diff:
sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveExternalCatalog.scala ---
@@ -144,16 +161,147 @@ private[spark] class HiveExternalCatalog
Github user yhuai commented on a diff in the pull request:
https://github.com/apache/spark/pull/14155#discussion_r75063449
--- Diff:
sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveExternalCatalog.scala ---
@@ -144,16 +161,147 @@ private[spark] class HiveExternalCatalog
Github user yhuai commented on a diff in the pull request:
https://github.com/apache/spark/pull/14155#discussion_r75063156
--- Diff:
sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveExternalCatalog.scala ---
@@ -144,16 +161,147 @@ private[spark] class HiveExternalCatalog
Github user yhuai commented on a diff in the pull request:
https://github.com/apache/spark/pull/14155#discussion_r75063094
--- Diff:
sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveExternalCatalog.scala ---
@@ -144,16 +161,147 @@ private[spark] class HiveExternalCatalog
Github user yhuai commented on a diff in the pull request:
https://github.com/apache/spark/pull/14155#discussion_r75062850
--- Diff:
sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveExternalCatalog.scala ---
@@ -144,16 +161,147 @@ private[spark] class HiveExternalCatalog
Github user yhuai commented on a diff in the pull request:
https://github.com/apache/spark/pull/14155#discussion_r75062846
--- Diff:
sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveExternalCatalog.scala ---
@@ -144,16 +161,147 @@ private[spark] class HiveExternalCatalog
Github user yhuai commented on a diff in the pull request:
https://github.com/apache/spark/pull/14155#discussion_r75062743
--- Diff:
sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveExternalCatalog.scala ---
@@ -144,16 +161,147 @@ private[spark] class HiveExternalCatalog
Github user yhuai closed the pull request at:
https://github.com/apache/spark/pull/14668
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature
Github user yhuai commented on the issue:
https://github.com/apache/spark/pull/14668
I have merged this PR to branch 1.6. Closing it.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
ems it is a flaky test. This PR tries to
make this test more stable.
Author: Yin Huai <yh...@databricks.com>
Closes #14668 from yhuai/SPARK-16656-branch1.6.
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/5c34029b
Tree: http:
Github user yhuai commented on the issue:
https://github.com/apache/spark/pull/14668
I am merging this to branch 1.6.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled
GitHub user yhuai opened a pull request:
https://github.com/apache/spark/pull/14668
[SPARK-16656][SQL][BRANCH-1.6] Try to make CreateTableAsSelectSuite more
stable
## What changes were proposed in this pull request?
This PR backports #14289 to branch 1.6
https
Github user yhuai commented on the issue:
https://github.com/apache/spark/pull/14506
Thanks. Merging to master.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled
Github user yhuai commented on the issue:
https://github.com/apache/spark/pull/14580
Can you explain `isnotnull(coalesce(b#227, c#238)) does not filter out
NULL!!!`?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well
Github user yhuai commented on a diff in the pull request:
https://github.com/apache/spark/pull/14155#discussion_r74852722
--- Diff:
sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/HiveDDLSuite.scala
---
@@ -689,4 +689,38 @@ class HiveDDLSuite
Github user yhuai commented on the issue:
https://github.com/apache/spark/pull/14506
LGTM pending jenkins.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so
Github user yhuai commented on the issue:
https://github.com/apache/spark/pull/14506
test this please
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so
Repository: spark
Updated Branches:
refs/heads/branch-2.0 8f4cacd3a -> 45036327f
[SPARK-17065][SQL] Improve the error message when encountering an incompatible
DataSourceRegister
## What changes were proposed in this pull request?
Add an instruction to ask the user to remove or upgrade the
Repository: spark
Updated Branches:
refs/heads/master fffb0c0d1 -> 268b71d0d
[SPARK-17065][SQL] Improve the error message when encountering an incompatible
DataSourceRegister
## What changes were proposed in this pull request?
Add an instruction to ask the user to remove or upgrade the
Github user yhuai commented on the issue:
https://github.com/apache/spark/pull/14651
Merging to master and branch 2.0.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled
Github user yhuai commented on the issue:
https://github.com/apache/spark/pull/14651
LGTM. Thanks!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so
Github user yhuai commented on the issue:
https://github.com/apache/spark/pull/14634
Sorry. What's the necessity to make this change?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user yhuai commented on a diff in the pull request:
https://github.com/apache/spark/pull/14155#discussion_r74665317
--- Diff:
sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveExternalCatalog.scala ---
@@ -207,15 +310,52 @@ private[spark] class HiveExternalCatalog
Github user yhuai commented on a diff in the pull request:
https://github.com/apache/spark/pull/14155#discussion_r74665106
--- Diff:
sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveExternalCatalog.scala ---
@@ -81,6 +86,19 @@ private[spark] class HiveExternalCatalog(client
Github user yhuai commented on a diff in the pull request:
https://github.com/apache/spark/pull/14155#discussion_r74665091
--- Diff:
sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveExternalCatalog.scala ---
@@ -81,6 +86,19 @@ private[spark] class HiveExternalCatalog(client
Github user yhuai commented on a diff in the pull request:
https://github.com/apache/spark/pull/14155#discussion_r74665033
--- Diff:
sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveExternalCatalog.scala ---
@@ -363,3 +503,82 @@ private[spark] class HiveExternalCatalog(client
Github user yhuai commented on a diff in the pull request:
https://github.com/apache/spark/pull/14155#discussion_r74664610
--- Diff:
sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveExternalCatalog.scala ---
@@ -207,15 +310,52 @@ private[spark] class HiveExternalCatalog
Github user yhuai commented on a diff in the pull request:
https://github.com/apache/spark/pull/14155#discussion_r74664134
--- Diff:
sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveExternalCatalog.scala ---
@@ -144,16 +162,101 @@ private[spark] class HiveExternalCatalog
Github user yhuai commented on a diff in the pull request:
https://github.com/apache/spark/pull/14155#discussion_r74664001
--- Diff:
sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveExternalCatalog.scala ---
@@ -81,6 +86,19 @@ private[spark] class HiveExternalCatalog(client
Github user yhuai commented on a diff in the pull request:
https://github.com/apache/spark/pull/14155#discussion_r74663776
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/execution/command/DDLSuite.scala
---
@@ -396,40 +393,6 @@ class DDLSuite extends QueryTest
Github user yhuai closed the pull request at:
https://github.com/apache/spark/pull/14586
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature
tps://issues.apache.org/jira/browse/SPARK-8013). So, let's publish scala
2.11 artifacts with the flag of `-Phive-thfitserver`. I am also fixing the doc.
Author: Yin Huai <yh...@databricks.com>
Closes #14586 from yhuai/SPARK-16453-branch-1.6.
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Com
Github user yhuai commented on the issue:
https://github.com/apache/spark/pull/14586
merging to branch 1.6.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so
Github user yhuai commented on the issue:
https://github.com/apache/spark/pull/14586
I am merging this.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so
Github user yhuai commented on the issue:
https://github.com/apache/spark/pull/14546
@dongjoon-hyun Seems this issue has been fixed as a by-product of
https://github.com/apache/spark/pull/14595. How about we close this? Also, feel
free to look at @clockfly's follow-up pr
Github user yhuai commented on the issue:
https://github.com/apache/spark/pull/14586
test this please
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so
Github user yhuai commented on the issue:
https://github.com/apache/spark/pull/14586
Done
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so
Github user yhuai commented on the issue:
https://github.com/apache/spark/pull/14603
With this chnage, I think we can use encoder to serialize it.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does
Github user yhuai commented on the issue:
https://github.com/apache/spark/pull/14603
LGTM. Merging to master.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes
Github user yhuai commented on the issue:
https://github.com/apache/spark/pull/14456
Thanks! Seems https://github.com/apache/spark/pull/12464 introduced
avgMetrics to CrossValidator model.
---
If your project is set up for it, you can reply to this email and have your
reply appear
Github user yhuai commented on the issue:
https://github.com/apache/spark/pull/14456
Sorry. I think this pr breaks 1.6 build.
```
**
File
"/home/jenkins/workspace/NewSparkPullRequestBuilder/p
Github user yhuai commented on the issue:
https://github.com/apache/spark/pull/13775
for the benchmark, how about we just test the scan operation?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does
Github user yhuai commented on the issue:
https://github.com/apache/spark/pull/14586
test this please
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so
Github user yhuai commented on the issue:
https://github.com/apache/spark/pull/14586
test this please
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so
GitHub user yhuai opened a pull request:
https://github.com/apache/spark/pull/14586
[SPARK-16453] [BUILD] [BRANCH-1.6] release-build.sh is missing
hive-thriftserver for scala 2.10
## What changes were proposed in this pull request?
hive-thriftserver works with Scala 2.11
Github user yhuai commented on the issue:
https://github.com/apache/spark/pull/14155
Thank you for working on this! It's great to see we are moving those hacks
into `HiveExternalCatalog`. It will be very helpful if we can have two diagrams
to show how we use CatalogTable before
Github user yhuai commented on a diff in the pull request:
https://github.com/apache/spark/pull/14155#discussion_r74131369
--- Diff:
sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/HiveCommandSuite.scala
---
@@ -18,21 +18,32 @@
package
Github user yhuai commented on a diff in the pull request:
https://github.com/apache/spark/pull/14155#discussion_r74131072
--- Diff:
sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveMetastoreCatalog.scala ---
@@ -70,64 +69,16 @@ private[hive] class HiveMetastoreCatalog
Github user yhuai commented on a diff in the pull request:
https://github.com/apache/spark/pull/14155#discussion_r74130831
--- Diff:
sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveExternalCatalog.scala ---
@@ -363,3 +503,82 @@ private[spark] class HiveExternalCatalog(client
Github user yhuai commented on a diff in the pull request:
https://github.com/apache/spark/pull/14155#discussion_r74130452
--- Diff:
sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveExternalCatalog.scala ---
@@ -144,16 +162,101 @@ private[spark] class HiveExternalCatalog
Github user yhuai commented on a diff in the pull request:
https://github.com/apache/spark/pull/14155#discussion_r74130034
--- Diff:
sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveExternalCatalog.scala ---
@@ -81,6 +86,19 @@ private[spark] class HiveExternalCatalog(client
Github user yhuai commented on a diff in the pull request:
https://github.com/apache/spark/pull/14155#discussion_r74129714
--- Diff:
sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveExternalCatalog.scala ---
@@ -81,6 +86,19 @@ private[spark] class HiveExternalCatalog(client
Github user yhuai commented on a diff in the pull request:
https://github.com/apache/spark/pull/14155#discussion_r74129515
--- Diff:
sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/HiveDDLSuite.scala
---
@@ -689,4 +689,38 @@ class HiveDDLSuite
Github user yhuai commented on a diff in the pull request:
https://github.com/apache/spark/pull/14155#discussion_r74128927
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/execution/command/DDLSuite.scala
---
@@ -93,7 +92,7 @@ class DDLSuite extends QueryTest
Github user yhuai commented on a diff in the pull request:
https://github.com/apache/spark/pull/14155#discussion_r74127228
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/command/ddl.scala ---
@@ -229,10 +230,8 @@ case class AlterTableSetPropertiesCommand
Github user yhuai commented on a diff in the pull request:
https://github.com/apache/spark/pull/14500#discussion_r74123952
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/command/ddl.scala ---
@@ -425,6 +430,111 @@ case class AlterTableDropPartitionCommand
Github user yhuai commented on the issue:
https://github.com/apache/spark/pull/14500
@liancheng Can you do a post-hoc review?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user yhuai commented on the issue:
https://github.com/apache/spark/pull/14492
That's a good point. Let me close this.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user yhuai closed the pull request at:
https://github.com/apache/spark/pull/14492
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature
Repository: spark
Updated Branches:
refs/heads/master 53d1c7877 -> df1065883
[SPARK-16749][SQL] Simplify processing logic in LEAD/LAG processing.
## What changes were proposed in this pull request?
The logic for LEAD/LAG processing is more complex that it needs to be. This PR
fixes that.
##
Github user yhuai commented on the issue:
https://github.com/apache/spark/pull/14376
Merging to master.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so
Github user yhuai commented on the issue:
https://github.com/apache/spark/pull/14531
We do not support index tables at all (you can not create such a table).
Let's not add the support right now.
---
If your project is set up for it, you can reply to this email and have your
reply
Github user yhuai commented on the issue:
https://github.com/apache/spark/pull/14207
@gatorsmile
Where is change for the following description?
```
This PR is to store the inferred schema in the external catalog when
creating the table. When users intend to refresh
Github user yhuai commented on a diff in the pull request:
https://github.com/apache/spark/pull/14207#discussion_r73941472
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/command/createDataSourceTables.scala
---
@@ -95,17 +95,39 @@ case class
Github user yhuai commented on a diff in the pull request:
https://github.com/apache/spark/pull/14207#discussion_r73940895
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/command/ddl.scala ---
@@ -521,31 +521,29 @@ object DDLUtils
Github user yhuai commented on a diff in the pull request:
https://github.com/apache/spark/pull/14207#discussion_r73940848
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/command/ddl.scala ---
@@ -521,31 +521,29 @@ object DDLUtils
Github user yhuai commented on a diff in the pull request:
https://github.com/apache/spark/pull/14207#discussion_r73940468
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/command/createDataSourceTables.scala
---
@@ -95,17 +95,39 @@ case class
Github user yhuai commented on a diff in the pull request:
https://github.com/apache/spark/pull/14207#discussion_r73940350
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/command/createDataSourceTables.scala
---
@@ -95,17 +95,39 @@ case class
Github user yhuai commented on the issue:
https://github.com/apache/spark/pull/14541
lgtm
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so
Github user yhuai commented on a diff in the pull request:
https://github.com/apache/spark/pull/14376#discussion_r73917166
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/WindowExec.scala ---
@@ -565,7 +566,7 @@ private[execution] abstract class
Github user yhuai commented on the issue:
https://github.com/apache/spark/pull/14518
I think it is fine that `compression` takes precedence. btw, is this flag
used by other data sources?
---
If your project is set up for it, you can reply to this email and have your
reply appear
Github user yhuai commented on a diff in the pull request:
https://github.com/apache/spark/pull/14518#discussion_r73914641
--- Diff:
sql/hive/src/main/scala/org/apache/spark/sql/hive/orc/OrcOptions.scala ---
@@ -31,7 +30,8 @@ private[orc] class OrcOptions(
* Acceptable
Github user yhuai commented on a diff in the pull request:
https://github.com/apache/spark/pull/14518#discussion_r73914816
--- Diff:
sql/hive/src/test/scala/org/apache/spark/sql/hive/orc/OrcQuerySuite.scala ---
@@ -161,6 +161,29 @@ class OrcQuerySuite extends QueryTest
Github user yhuai commented on a diff in the pull request:
https://github.com/apache/spark/pull/14518#discussion_r73811203
--- Diff:
sql/hive/src/main/scala/org/apache/spark/sql/hive/orc/OrcOptions.scala ---
@@ -17,27 +17,40 @@
package org.apache.spark.sql.hive.orc
Github user yhuai commented on the issue:
https://github.com/apache/spark/pull/12872
Can you be more specific on the inconsistency? Seems `ALTER VIEW view_name`
is the only inconsistent command?
---
If your project is set up for it, you can reply to this email and have your
reply
ich is
jdbc:derby:;databaseName=metastore_db;create=true.
This issue only shows up when `spark.sql.hive.metastore.jars` is not set to
builtin.
## How was this patch tested?
New test in HiveSparkSubmitSuite.
Author: Yin Huai <yh...@databricks.com>
Closes #14497 from yhuai/SPARK-16901.
(cher
ich is
jdbc:derby:;databaseName=metastore_db;create=true.
This issue only shows up when `spark.sql.hive.metastore.jars` is not set to
builtin.
## How was this patch tested?
New test in HiveSparkSubmitSuite.
Author: Yin Huai <yh...@databricks.com>
Closes #14497 from yhuai/SPARK-16901.
Proj
Github user yhuai commented on the issue:
https://github.com/apache/spark/pull/14497
Thanks for reviewing! I am merging this to master and branch 2.0.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does
Github user yhuai commented on a diff in the pull request:
https://github.com/apache/spark/pull/14500#discussion_r73730807
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/command/ddl.scala ---
@@ -425,6 +431,96 @@ case class AlterTableDropPartitionCommand
Github user yhuai commented on a diff in the pull request:
https://github.com/apache/spark/pull/14500#discussion_r73730136
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/SparkSqlParser.scala ---
@@ -409,6 +409,18 @@ class SparkSqlAstBuilder(conf: SQLConf) extends
Github user yhuai commented on a diff in the pull request:
https://github.com/apache/spark/pull/14500#discussion_r73729927
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/execution/command/DDLSuite.scala
---
@@ -827,6 +827,45 @@ class DDLSuite extends QueryTest
Github user yhuai commented on a diff in the pull request:
https://github.com/apache/spark/pull/14500#discussion_r73728488
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/SparkSqlParser.scala ---
@@ -409,6 +409,18 @@ class SparkSqlAstBuilder(conf: SQLConf) extends
Github user yhuai commented on a diff in the pull request:
https://github.com/apache/spark/pull/14155#discussion_r73727982
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/command/ddl.scala ---
@@ -301,9 +298,6 @@ case class AlterTableSerDePropertiesCommand
701 - 800 of 5990 matches
Mail list logo
