[GitHub] spark pull request: [SPARK-14184][SQL] Support native execution of...
Github user dilipbiswal commented on the pull request: https://github.com/apache/spark/pull/11991#issuecomment-204474413 @hvanhovell Thank you, Herman. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14184][SQL] Support native execution of...
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/11991 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14184][SQL] Support native execution of...
Github user hvanhovell commented on the pull request: https://github.com/apache/spark/pull/11991#issuecomment-204460100 Merging to master. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14184][SQL] Support native execution of...
Github user hvanhovell commented on the pull request: https://github.com/apache/spark/pull/11991#issuecomment-204128228 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14184][SQL] Support native execution of...
Github user dilipbiswal commented on the pull request: https://github.com/apache/spark/pull/11991#issuecomment-204127378 cc @andrewor14 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14184][SQL] Support native execution of...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11991#issuecomment-20401 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14184][SQL] Support native execution of...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11991#issuecomment-20403 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/54651/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14184][SQL] Support native execution of...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11991#issuecomment-204099670 **[Test build #54651 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/54651/consoleFull)** for PR 11991 at commit [`bafca92`](https://github.com/apache/spark/commit/bafca929060a62d255f8067e11531d4609da). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14184][SQL] Support native execution of...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11991#issuecomment-204064388 **[Test build #54651 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/54651/consoleFull)** for PR 11991 at commit [`bafca92`](https://github.com/apache/spark/commit/bafca929060a62d255f8067e11531d4609da). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14184][SQL] Support native execution of...
Github user dilipbiswal commented on the pull request: https://github.com/apache/spark/pull/11991#issuecomment-204064015 @andrewor14 Hi Andrew, just rebased. - FYI --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14184][SQL] Support native execution of...
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/11991#issuecomment-204046002 @dilipbiswal can you rebase? I think `SparkQl` is now deleted. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14184][SQL] Support native execution of...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11991#issuecomment-203708616 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14184][SQL] Support native execution of...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11991#issuecomment-203708619 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/54565/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14184][SQL] Support native execution of...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11991#issuecomment-203707994 **[Test build #54565 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/54565/consoleFull)** for PR 11991 at commit [`2e16ce4`](https://github.com/apache/spark/commit/2e16ce44ff11bcdc588383cd00175482ccce). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14184][SQL] Support native execution of...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11991#issuecomment-203688154 **[Test build #54565 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/54565/consoleFull)** for PR 11991 at commit [`2e16ce4`](https://github.com/apache/spark/commit/2e16ce44ff11bcdc588383cd00175482ccce). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14184][SQL] Support native execution of...
Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/11991#discussion_r57981279
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/SparkQl.scala ---
@@ -370,13 +370,33 @@ private[sql] class SparkQl(conf: ParserConf =
SimpleParserConf()) extends Cataly
case Token("TOK_CLEARCACHE", Nil) =>
ClearCacheCommand
+ // SHOW TABLES [(IN|FROM) database_name]
['identifier_with_wildcards']
--- End diff --
I see... never mind then. It seems that we can't avoid the conflicts after
all.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14184][SQL] Support native execution of...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11991#issuecomment-203677174 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14184][SQL] Support native execution of...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11991#issuecomment-203677175 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/54554/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14184][SQL] Support native execution of...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11991#issuecomment-203676949 **[Test build #54554 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/54554/consoleFull)** for PR 11991 at commit [`9a4f356`](https://github.com/apache/spark/commit/9a4f3564c3a44db237d05f5fe26d2d2aa043cb69). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14184][SQL] Support native execution of...
Github user dilipbiswal commented on a diff in the pull request:
https://github.com/apache/spark/pull/11991#discussion_r57977077
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/SparkQl.scala ---
@@ -370,13 +370,33 @@ private[sql] class SparkQl(conf: ParserConf =
SimpleParserConf()) extends Cataly
case Token("TOK_CLEARCACHE", Nil) =>
ClearCacheCommand
+ // SHOW TABLES [(IN|FROM) database_name]
['identifier_with_wildcards']
--- End diff --
@andrewor14 Hi Andrew, is it ok if i have a very minor change in this file
just to make sure compilation goes through. Or it will still cause issue with
merging ?
ShowTablesCommand(databaseName, None) => we introduced a new parameter to
the command.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14184][SQL] Support native execution of...
Github user dilipbiswal commented on a diff in the pull request:
https://github.com/apache/spark/pull/11991#discussion_r57970378
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/SparkQl.scala ---
@@ -370,13 +370,33 @@ private[sql] class SparkQl(conf: ParserConf =
SimpleParserConf()) extends Cataly
case Token("TOK_CLEARCACHE", Nil) =>
ClearCacheCommand
+ // SHOW TABLES [(IN|FROM) database_name]
['identifier_with_wildcards']
--- End diff --
ok. Sure.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14184][SQL] Support native execution of...
Github user dilipbiswal commented on a diff in the pull request:
https://github.com/apache/spark/pull/11991#discussion_r57970436
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/command/commands.scala
---
@@ -322,18 +322,17 @@ case class DescribeCommand(
* If a databaseName is not given, the current database will be used.
* The syntax of using this command in SQL is:
* {{{
- *SHOW TABLES [IN databaseName]
+ * SHOW TABLES [(IN|FROM) database_name] ['identifier_with_wildcards'];
--- End diff --
Thank you @andrewor14 . Will add it now.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14184][SQL] Support native execution of...
Github user dilipbiswal commented on a diff in the pull request:
https://github.com/apache/spark/pull/11991#discussion_r57970312
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/SparkSqlParser.scala ---
@@ -70,12 +70,26 @@ class SparkSqlAstBuilder extends AstBuilder {
/**
* Create a [[ShowTablesCommand]] logical plan.
+ * Example SQL :
+ * {{{
+ * SHOW TABLES [(IN|FROM) database_name] ['identifier_with_wildcards'];
--- End diff --
Thank you @andrewor14 . Will add it now.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14184][SQL] Support native execution of...
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/11991#issuecomment-203649240 Looks great! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14184][SQL] Support native execution of...
Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/11991#discussion_r57969533
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/command/commands.scala
---
@@ -322,18 +322,17 @@ case class DescribeCommand(
* If a databaseName is not given, the current database will be used.
* The syntax of using this command in SQL is:
* {{{
- *SHOW TABLES [IN databaseName]
+ * SHOW TABLES [(IN|FROM) database_name] ['identifier_with_wildcards'];
--- End diff --
same here
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14184][SQL] Support native execution of...
Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/11991#discussion_r57969388
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/SparkSqlParser.scala ---
@@ -70,12 +70,26 @@ class SparkSqlAstBuilder extends AstBuilder {
/**
* Create a [[ShowTablesCommand]] logical plan.
+ * Example SQL :
+ * {{{
+ * SHOW TABLES [(IN|FROM) database_name] ['identifier_with_wildcards'];
--- End diff --
can you add an optional `LIKE` here
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14184][SQL] Support native execution of...
Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/11991#discussion_r57969324
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/SparkQl.scala ---
@@ -370,13 +370,33 @@ private[sql] class SparkQl(conf: ParserConf =
SimpleParserConf()) extends Cataly
case Token("TOK_CLEARCACHE", Nil) =>
ClearCacheCommand
+ // SHOW TABLES [(IN|FROM) database_name]
['identifier_with_wildcards']
--- End diff --
FYI to avoid conflicts with #12071 let's revert all the changes in this
file since we're not using this parser anymore anyway.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14184][SQL] Support native execution of...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11991#issuecomment-203645241 **[Test build #54554 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/54554/consoleFull)** for PR 11991 at commit [`9a4f356`](https://github.com/apache/spark/commit/9a4f3564c3a44db237d05f5fe26d2d2aa043cb69). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14184][SQL] Support native execution of...
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/11991#discussion_r57959149 --- Diff: sql/catalyst/src/main/antlr4/org/apache/spark/sql/catalyst/parser/ng/SqlBase.g4 --- @@ -838,6 +839,8 @@ OUTPUTFORMAT: 'OUTPUTFORMAT'; INPUTDRIVER: 'INPUTDRIVER'; OUTPUTDRIVER: 'OUTPUTDRIVER'; DATABASE: 'DATABASE' | 'SCHEMA'; +DATABASES: 'DATABASES'; --- End diff -- Yeah we can. It would be the same as the `DATABASE` lexer rule. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14184][SQL] Support native execution of...
Github user dilipbiswal commented on a diff in the pull request: https://github.com/apache/spark/pull/11991#discussion_r57955257 --- Diff: sql/catalyst/src/main/antlr4/org/apache/spark/sql/catalyst/parser/ng/SqlBase.g4 --- @@ -838,6 +839,8 @@ OUTPUTFORMAT: 'OUTPUTFORMAT'; INPUTDRIVER: 'INPUTDRIVER'; OUTPUTDRIVER: 'OUTPUTDRIVER'; DATABASE: 'DATABASE' | 'SCHEMA'; +DATABASES: 'DATABASES'; --- End diff -- @hvanhovell Herman, yeah.. they are synonyms. I was not sure, If we just define it as one Token as say "DATABASES" , we can still refer to it from the nonReserved rule as "DATABASES" ? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14184][SQL] Support native execution of...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11991#issuecomment-203597169 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14184][SQL] Support native execution of...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11991#issuecomment-203597175 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/54539/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14184][SQL] Support native execution of...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11991#issuecomment-203596517 **[Test build #54539 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/54539/consoleFull)** for PR 11991 at commit [`9a11daa`](https://github.com/apache/spark/commit/9a11daae879ab595505d6869739020c104141254). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14184][SQL] Support native execution of...
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/11991#discussion_r57946967 --- Diff: sql/catalyst/src/main/antlr4/org/apache/spark/sql/catalyst/parser/ng/SqlBase.g4 --- @@ -838,6 +839,8 @@ OUTPUTFORMAT: 'OUTPUTFORMAT'; INPUTDRIVER: 'INPUTDRIVER'; OUTPUTDRIVER: 'OUTPUTDRIVER'; DATABASE: 'DATABASE' | 'SCHEMA'; +DATABASES: 'DATABASES'; --- End diff -- Do we need to seperate tokens here? I thought they were synonyms? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14184][SQL] Support native execution of...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11991#issuecomment-203546843 **[Test build #54539 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/54539/consoleFull)** for PR 11991 at commit [`9a11daa`](https://github.com/apache/spark/commit/9a11daae879ab595505d6869739020c104141254). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14184][SQL] Support native execution of...
Github user dilipbiswal commented on a diff in the pull request:
https://github.com/apache/spark/pull/11991#discussion_r57929969
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/execution/command/DDLSuite.scala
---
@@ -31,9 +31,11 @@ class DDLSuite extends QueryTest with SharedSQLContext {
*/
private def withDatabase(dbNames: String*)(f: => Unit): Unit = {
try f finally {
+ val currentDb = sqlContext.sessionState.catalog.getCurrentDatabase
dbNames.foreach { name =>
sqlContext.sql(s"DROP DATABASE IF EXISTS $name CASCADE")
}
+ sqlContext.sessionState.catalog.setCurrentDatabase(currentDb)
--- End diff --
@gatorsmile Ok.. I will make the change.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14184][SQL] Support native execution of...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11991#issuecomment-203538215 **[Test build #54527 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/54527/consoleFull)** for PR 11991 at commit [`fe60df8`](https://github.com/apache/spark/commit/fe60df84f7f5bbce9a485381531043ef1fe8db33). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14184][SQL] Support native execution of...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11991#issuecomment-203539275 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/54527/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14184][SQL] Support native execution of...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11991#issuecomment-203539266 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14184][SQL] Support native execution of...
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/11991#discussion_r57919394
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/execution/command/DDLSuite.scala
---
@@ -31,9 +31,11 @@ class DDLSuite extends QueryTest with SharedSQLContext {
*/
private def withDatabase(dbNames: String*)(f: => Unit): Unit = {
try f finally {
+ val currentDb = sqlContext.sessionState.catalog.getCurrentDatabase
dbNames.foreach { name =>
sqlContext.sql(s"DROP DATABASE IF EXISTS $name CASCADE")
}
+ sqlContext.sessionState.catalog.setCurrentDatabase(currentDb)
--- End diff --
IMO, we do not need nested cases now. If multiple databases are used, we
can do something like,
```Scala
withDatabase("db1", "db2")
```
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14184][SQL] Support native execution of...
Github user dilipbiswal commented on a diff in the pull request:
https://github.com/apache/spark/pull/11991#discussion_r57918497
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/execution/command/DDLSuite.scala
---
@@ -31,9 +31,11 @@ class DDLSuite extends QueryTest with SharedSQLContext {
*/
private def withDatabase(dbNames: String*)(f: => Unit): Unit = {
try f finally {
+ val currentDb = sqlContext.sessionState.catalog.getCurrentDatabase
dbNames.foreach { name =>
sqlContext.sql(s"DROP DATABASE IF EXISTS $name CASCADE")
}
+ sqlContext.sessionState.catalog.setCurrentDatabase(currentDb)
--- End diff --
@gatorsmile Yeah.,initially i was thinking of resetting to "default". I
thought a restore semantics might be better here. Is it possible for us to do
nesting in withDatabase ? Like
WithDatabase("foo") {
// create db
// sql
// withDatabase("bar") {
sql
}
sql
sql
}
Let me know if you want to reset it to "default" instead of restoring ?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14184][SQL] Support native execution of...
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/11991#discussion_r57910069
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/execution/command/DDLSuite.scala
---
@@ -31,9 +31,11 @@ class DDLSuite extends QueryTest with SharedSQLContext {
*/
private def withDatabase(dbNames: String*)(f: => Unit): Unit = {
try f finally {
+ val currentDb = sqlContext.sessionState.catalog.getCurrentDatabase
dbNames.foreach { name =>
sqlContext.sql(s"DROP DATABASE IF EXISTS $name CASCADE")
}
+ sqlContext.sessionState.catalog.setCurrentDatabase(currentDb)
--- End diff --
How about doing it directly? `setCurrentDatabase("default")`
Then, you do not need to first get the value for `currentDb` since
`currentDb` could be any database name here.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14184][SQL] Support native execution of...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11991#issuecomment-203490158 **[Test build #54527 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/54527/consoleFull)** for PR 11991 at commit [`fe60df8`](https://github.com/apache/spark/commit/fe60df84f7f5bbce9a485381531043ef1fe8db33). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14184][SQL] Support native execution of...
Github user dilipbiswal commented on a diff in the pull request:
https://github.com/apache/spark/pull/11991#discussion_r57907320
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/execution/command/DDLSuite.scala
---
@@ -31,9 +31,11 @@ class DDLSuite extends QueryTest with SharedSQLContext {
*/
private def withDatabase(dbNames: String*)(f: => Unit): Unit = {
try f finally {
+ val currentDb = sqlContext.sessionState.catalog.getCurrentDatabase
dbNames.foreach { name =>
sqlContext.sql(s"DROP DATABASE IF EXISTS $name CASCADE")
}
+ sqlContext.sessionState.catalog.setCurrentDatabase(currentDb)
--- End diff --
default
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14184][SQL] Support native execution of...
Github user dilipbiswal commented on a diff in the pull request:
https://github.com/apache/spark/pull/11991#discussion_r57907169
--- Diff: sql/core/src/test/scala/org/apache/spark/sql/SQLQuerySuite.scala
---
@@ -2376,4 +2376,47 @@ class SQLQuerySuite extends QueryTest with
SharedSQLContext {
Row("r3c1x", "r3c2", "t1r3c3", "r3c2", "t1r3c3") :: Nil)
}
}
+
+ test("show tables") {
--- End diff --
OK
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14184][SQL] Support native execution of...
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/11991#discussion_r57878137
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/execution/command/DDLSuite.scala
---
@@ -31,9 +31,11 @@ class DDLSuite extends QueryTest with SharedSQLContext {
*/
private def withDatabase(dbNames: String*)(f: => Unit): Unit = {
try f finally {
+ val currentDb = sqlContext.sessionState.catalog.getCurrentDatabase
dbNames.foreach { name =>
sqlContext.sql(s"DROP DATABASE IF EXISTS $name CASCADE")
}
+ sqlContext.sessionState.catalog.setCurrentDatabase(currentDb)
--- End diff --
The value of `currentDb` is?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14184][SQL] Support native execution of...
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/11991#discussion_r57877844
--- Diff: sql/core/src/test/scala/org/apache/spark/sql/SQLQuerySuite.scala
---
@@ -2376,4 +2376,47 @@ class SQLQuerySuite extends QueryTest with
SharedSQLContext {
Row("r3c1x", "r3c2", "t1r3c3", "r3c2", "t1r3c3") :: Nil)
}
}
+
+ test("show tables") {
--- End diff --
Can you move this test case into `DDLSuite.scala`? `SQLQuerySuite` is
already very big now.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14184][SQL] Support native execution of...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11991#issuecomment-203318997 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14184][SQL] Support native execution of...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11991#issuecomment-203319004 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/54500/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14184][SQL] Support native execution of...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11991#issuecomment-203318486 **[Test build #54500 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/54500/consoleFull)** for PR 11991 at commit [`034af13`](https://github.com/apache/spark/commit/034af13270b4999a8f2b5dc064b0c268fd8db802). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14184][SQL] Support native execution of...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11991#issuecomment-203272610 **[Test build #54500 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/54500/consoleFull)** for PR 11991 at commit [`034af13`](https://github.com/apache/spark/commit/034af13270b4999a8f2b5dc064b0c268fd8db802). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14184][SQL] Support native execution of...
Github user dilipbiswal commented on a diff in the pull request:
https://github.com/apache/spark/pull/11991#discussion_r57841828
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/ng/ParserUtils.scala
---
@@ -24,6 +24,7 @@ import
org.apache.spark.sql.catalyst.parser.ParseUtils.unescapeSQLString
import org.apache.spark.sql.catalyst.plans.logical.LogicalPlan
import org.apache.spark.sql.catalyst.trees.{CurrentOrigin, Origin}
+
--- End diff --
Fixed. Thanks !!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14184][SQL] Support native execution of...
Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/11991#discussion_r57832569
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/ng/ParserUtils.scala
---
@@ -24,6 +24,7 @@ import
org.apache.spark.sql.catalyst.parser.ParseUtils.unescapeSQLString
import org.apache.spark.sql.catalyst.plans.logical.LogicalPlan
import org.apache.spark.sql.catalyst.trees.{CurrentOrigin, Origin}
+
--- End diff --
?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14184][SQL] Support native execution of...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11991#issuecomment-203181669 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14184][SQL] Support native execution of...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11991#issuecomment-203181672 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/54475/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14184][SQL] Support native execution of...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11991#issuecomment-203181120 **[Test build #54475 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/54475/consoleFull)** for PR 11991 at commit [`3e6fb2e`](https://github.com/apache/spark/commit/3e6fb2e30712c423339170d5c6fbeeb9c19bdec8). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14184][SQL] Support native execution of...
Github user gatorsmile commented on the pull request: https://github.com/apache/spark/pull/11991#issuecomment-203175485 @dilipbiswal The PR https://github.com/apache/spark/pull/12009 has been resolved. Could you add a test case for show database? Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14184][SQL] Support native execution of...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11991#issuecomment-203159026 **[Test build #54475 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/54475/consoleFull)** for PR 11991 at commit [`3e6fb2e`](https://github.com/apache/spark/commit/3e6fb2e30712c423339170d5c6fbeeb9c19bdec8). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14184][SQL] Support native execution of...
Github user dilipbiswal commented on a diff in the pull request:
https://github.com/apache/spark/pull/11991#discussion_r57802500
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/SparkSqlParser.scala ---
@@ -70,12 +70,28 @@ class SparkSqlAstBuilder extends AstBuilder {
/**
* Create a [[ShowTablesCommand]] logical plan.
+ * Example SQL :
+ * SHOW TABLES [(IN|FROM) database_name] ['identifier_with_wildcards']
*/
override def visitShowTables(ctx: ShowTablesContext): LogicalPlan =
withOrigin(ctx) {
-if (ctx.LIKE != null) {
- logWarning("SHOW TABLES LIKE option is ignored.")
+if (ctx.qualifiedName() != null) {
+ logWarning("SHOW TABLES qualifiedName is ignored.")
--- End diff --
@hvanhovell Yeah.. its supported in their parser. But for every input i see
a empty result set returned. Need to check if we break some test-case which
expects an empty result set on a input qualified name. Let me run the test
suites now :-)
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14184][SQL] Support native execution of...
Github user hvanhovell commented on a diff in the pull request:
https://github.com/apache/spark/pull/11991#discussion_r57801096
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/SparkSqlParser.scala ---
@@ -70,12 +70,28 @@ class SparkSqlAstBuilder extends AstBuilder {
/**
* Create a [[ShowTablesCommand]] logical plan.
+ * Example SQL :
+ * SHOW TABLES [(IN|FROM) database_name] ['identifier_with_wildcards']
*/
override def visitShowTables(ctx: ShowTablesContext): LogicalPlan =
withOrigin(ctx) {
-if (ctx.LIKE != null) {
- logWarning("SHOW TABLES LIKE option is ignored.")
+if (ctx.qualifiedName() != null) {
+ logWarning("SHOW TABLES qualifiedName is ignored.")
--- End diff --
The only reason for qualifiedName to be used was Hive. If Hive doesn't
support it (it is supported in the parser BTW), then we should drop it.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14184][SQL] Support native execution of...
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/11991#discussion_r57800827 --- Diff: sql/catalyst/src/main/antlr4/org/apache/spark/sql/catalyst/parser/ng/SqlBase.g4 --- @@ -107,7 +107,8 @@ statement | DROP TEMPORARY? FUNCTION (IF EXISTS)? qualifiedName #dropFunction | EXPLAIN explainOption* statement #explain | SHOW TABLES ((FROM | IN) db=identifier)? --- End diff -- Lets keep it then. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14184][SQL] Support native execution of...
Github user dilipbiswal commented on a diff in the pull request: https://github.com/apache/spark/pull/11991#discussion_r57800444 --- Diff: sql/catalyst/src/main/antlr4/org/apache/spark/sql/catalyst/parser/ng/SqlBase.g4 --- @@ -107,7 +107,8 @@ statement | DROP TEMPORARY? FUNCTION (IF EXISTS)? qualifiedName #dropFunction | EXPLAIN explainOption* statement #explain | SHOW TABLES ((FROM | IN) db=identifier)? --- End diff -- @hvanhovell Actually Herman, i tried the following in Hive. ```SQL hive> show tables from default; OK default__t1_t1_index__ ``` should we keep the from option ? There are also some tests which use from option. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14184][SQL] Support native execution of...
Github user dilipbiswal commented on a diff in the pull request:
https://github.com/apache/spark/pull/11991#discussion_r57799009
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/command/commands.scala
---
@@ -350,6 +354,34 @@ case class ShowTablesCommand(databaseName:
Option[String]) extends RunnableComma
}
/**
+ * A command for users to list the databases/schemas.
+ * If a databasePattern is supplied then the databases that only matches
the
+ * pattern would be listed.
+ * The syntax of using this command in SQL is:
+ * {{{
+ *SHOW (DATABASES|SCHEMAS) [LIKE 'identifier_with_wildcards'];
+ * }}}
+ */
+case class ShowDatabasesCommand(databasePattern: Option[String]) extends
RunnableCommand {
+
+ // The result of SHOW DATABASES has one column called 'result'
+ override val output: Seq[Attribute] = {
+val schema = StructType(
+ StructField("result", StringType, false) :: Nil)
+
+schema.toAttributes
--- End diff --
OK.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14184][SQL] Support native execution of...
Github user dilipbiswal commented on a diff in the pull request:
https://github.com/apache/spark/pull/11991#discussion_r57795477
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/command/commands.scala
---
@@ -350,6 +354,34 @@ case class ShowTablesCommand(databaseName:
Option[String]) extends RunnableComma
}
/**
+ * A command for users to list the databases/schemas.
+ * If a databasePattern is supplied then the databases that only matches
the
+ * pattern would be listed.
+ * The syntax of using this command in SQL is:
+ * {{{
+ *SHOW (DATABASES|SCHEMAS) [LIKE 'identifier_with_wildcards'];
+ * }}}
+ */
+case class ShowDatabasesCommand(databasePattern: Option[String]) extends
RunnableCommand {
+
+ // The result of SHOW DATABASES has one column called 'result'
+ override val output: Seq[Attribute] = {
+val schema = StructType(
+ StructField("result", StringType, false) :: Nil)
+
+schema.toAttributes
+ }
+
+ override def run(sqlContext: SQLContext): Seq[Row] = {
+val catalog = sqlContext.sessionState.catalog
+val databases =
+
databasePattern.map(catalog.listDatabases(_)).getOrElse(catalog.listDatabases())
+val rows = databases.map{d => Row(d)}
--- End diff --
Will make the change.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14184][SQL] Support native execution of...
Github user dilipbiswal commented on a diff in the pull request:
https://github.com/apache/spark/pull/11991#discussion_r57795331
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/SparkSqlParser.scala ---
@@ -70,12 +70,28 @@ class SparkSqlAstBuilder extends AstBuilder {
/**
* Create a [[ShowTablesCommand]] logical plan.
+ * Example SQL :
+ * SHOW TABLES [(IN|FROM) database_name] ['identifier_with_wildcards']
*/
override def visitShowTables(ctx: ShowTablesContext): LogicalPlan =
withOrigin(ctx) {
-if (ctx.LIKE != null) {
- logWarning("SHOW TABLES LIKE option is ignored.")
+if (ctx.qualifiedName() != null) {
+ logWarning("SHOW TABLES qualifiedName is ignored.")
--- End diff --
@hvanhovell @rxin
qualifiedName does not seem to work in hive. Only the table name pattern
works. Not sure if its a defect in hive...
Just for reference i looked at cloudera documentation on how they support
SHOW commands. They just take
a pattern string literal. Should we simplify our grammer and drop the
qualifiedName ? Please let me know.
http://www.cloudera.com/documentation/archive/impala/2-x/2-1-x/topics/impala_show.html
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14184][SQL] Support native execution of...
Github user dilipbiswal commented on a diff in the pull request:
https://github.com/apache/spark/pull/11991#discussion_r57793590
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/SparkSqlParser.scala ---
@@ -70,12 +70,28 @@ class SparkSqlAstBuilder extends AstBuilder {
/**
* Create a [[ShowTablesCommand]] logical plan.
+ * Example SQL :
+ * SHOW TABLES [(IN|FROM) database_name] ['identifier_with_wildcards']
*/
override def visitShowTables(ctx: ShowTablesContext): LogicalPlan =
withOrigin(ctx) {
-if (ctx.LIKE != null) {
- logWarning("SHOW TABLES LIKE option is ignored.")
+if (ctx.qualifiedName() != null) {
+ logWarning("SHOW TABLES qualifiedName is ignored.")
}
-ShowTablesCommand(Option(ctx.db).map(_.getText))
+ShowTablesCommand(
+ Option(ctx.db).map(_.getText),
+ Option(ctx.pattern).map(p => unquoteString(p.getText)))
--- End diff --
@hvanhovell Yeah.. i had seen your comment. I was running test with exact
same change :-)
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14184][SQL] Support native execution of...
Github user dilipbiswal commented on a diff in the pull request: https://github.com/apache/spark/pull/11991#discussion_r57793412 --- Diff: sql/catalyst/src/main/antlr4/org/apache/spark/sql/catalyst/parser/ng/SqlBase.g4 --- @@ -107,7 +107,8 @@ statement | DROP TEMPORARY? FUNCTION (IF EXISTS)? qualifiedName #dropFunction | EXPLAIN explainOption* statement #explain | SHOW TABLES ((FROM | IN) db=identifier)? -(LIKE (qualifiedName | pattern=STRING))? #showTables +(LIKE? (qualifiedName | pattern=STRING))? #showTables --- End diff -- Will fix it. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14184][SQL] Support native execution of...
Github user dilipbiswal commented on a diff in the pull request: https://github.com/apache/spark/pull/11991#discussion_r57793349 --- Diff: sql/catalyst/src/main/antlr4/org/apache/spark/sql/catalyst/parser/ng/SqlBase.g4 --- @@ -107,7 +107,8 @@ statement | DROP TEMPORARY? FUNCTION (IF EXISTS)? qualifiedName #dropFunction | EXPLAIN explainOption* statement #explain | SHOW TABLES ((FROM | IN) db=identifier)? --- End diff -- Sure. Will do --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14184][SQL] Support native execution of...
Github user hvanhovell commented on a diff in the pull request:
https://github.com/apache/spark/pull/11991#discussion_r57789745
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/command/commands.scala
---
@@ -350,6 +354,34 @@ case class ShowTablesCommand(databaseName:
Option[String]) extends RunnableComma
}
/**
+ * A command for users to list the databases/schemas.
+ * If a databasePattern is supplied then the databases that only matches
the
+ * pattern would be listed.
+ * The syntax of using this command in SQL is:
+ * {{{
+ *SHOW (DATABASES|SCHEMAS) [LIKE 'identifier_with_wildcards'];
+ * }}}
+ */
+case class ShowDatabasesCommand(databasePattern: Option[String]) extends
RunnableCommand {
+
+ // The result of SHOW DATABASES has one column called 'result'
+ override val output: Seq[Attribute] = {
+val schema = StructType(
+ StructField("result", StringType, false) :: Nil)
+
+schema.toAttributes
+ }
+
+ override def run(sqlContext: SQLContext): Seq[Row] = {
+val catalog = sqlContext.sessionState.catalog
+val databases =
+
databasePattern.map(catalog.listDatabases(_)).getOrElse(catalog.listDatabases())
+val rows = databases.map{d => Row(d)}
--- End diff --
Nit space after map, and why not return this directly?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14184][SQL] Support native execution of...
Github user rxin commented on a diff in the pull request:
https://github.com/apache/spark/pull/11991#discussion_r57788225
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/SparkSqlParser.scala ---
@@ -70,12 +70,28 @@ class SparkSqlAstBuilder extends AstBuilder {
/**
* Create a [[ShowTablesCommand]] logical plan.
+ * Example SQL :
+ * SHOW TABLES [(IN|FROM) database_name] ['identifier_with_wildcards']
*/
override def visitShowTables(ctx: ShowTablesContext): LogicalPlan =
withOrigin(ctx) {
-if (ctx.LIKE != null) {
- logWarning("SHOW TABLES LIKE option is ignored.")
+if (ctx.qualifiedName() != null) {
+ logWarning("SHOW TABLES qualifiedName is ignored.")
--- End diff --
yea end users would have no idea what qualified names mean
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14184][SQL] Support native execution of...
Github user hvanhovell commented on a diff in the pull request:
https://github.com/apache/spark/pull/11991#discussion_r57788212
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/command/commands.scala
---
@@ -350,6 +354,34 @@ case class ShowTablesCommand(databaseName:
Option[String]) extends RunnableComma
}
/**
+ * A command for users to list the databases/schemas.
+ * If a databasePattern is supplied then the databases that only matches
the
+ * pattern would be listed.
+ * The syntax of using this command in SQL is:
+ * {{{
+ *SHOW (DATABASES|SCHEMAS) [LIKE 'identifier_with_wildcards'];
+ * }}}
+ */
+case class ShowDatabasesCommand(databasePattern: Option[String]) extends
RunnableCommand {
+
+ // The result of SHOW DATABASES has one column called 'result'
+ override val output: Seq[Attribute] = {
+val schema = StructType(
+ StructField("result", StringType, false) :: Nil)
+
+schema.toAttributes
--- End diff --
Why not just create the AttributeReference?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14184][SQL] Support native execution of...
Github user hvanhovell commented on a diff in the pull request:
https://github.com/apache/spark/pull/11991#discussion_r57787998
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/SparkSqlParser.scala ---
@@ -70,12 +70,28 @@ class SparkSqlAstBuilder extends AstBuilder {
/**
* Create a [[ShowTablesCommand]] logical plan.
+ * Example SQL :
+ * SHOW TABLES [(IN|FROM) database_name] ['identifier_with_wildcards']
*/
override def visitShowTables(ctx: ShowTablesContext): LogicalPlan =
withOrigin(ctx) {
-if (ctx.LIKE != null) {
- logWarning("SHOW TABLES LIKE option is ignored.")
+if (ctx.qualifiedName() != null) {
+ logWarning("SHOW TABLES qualifiedName is ignored.")
}
-ShowTablesCommand(Option(ctx.db).map(_.getText))
+ShowTablesCommand(
+ Option(ctx.db).map(_.getText),
+ Option(ctx.pattern).map(p => unquoteString(p.getText)))
+ }
+
+ /**
+ * Create a [[ShowDatabasesCommand]] logical plan.
+ * Example SQL:
+ * SHOW (DATABASES|SCHEMAS) [LIKE 'identifier_with_wildcards']
+ */
+ override def visitShowDatabases(ctx: ShowDatabasesContext): LogicalPlan
= withOrigin(ctx) {
+if (ctx.qualifiedName() != null) {
+ logWarning("SHOW DATABASES qualifiedName is ignored.")
--- End diff --
I am not sure any one (besides us) will understand this. Any idea how hive
treats these?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14184][SQL] Support native execution of...
Github user hvanhovell commented on a diff in the pull request:
https://github.com/apache/spark/pull/11991#discussion_r57787715
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/SparkSqlParser.scala ---
@@ -70,12 +70,28 @@ class SparkSqlAstBuilder extends AstBuilder {
/**
* Create a [[ShowTablesCommand]] logical plan.
+ * Example SQL :
+ * SHOW TABLES [(IN|FROM) database_name] ['identifier_with_wildcards']
*/
override def visitShowTables(ctx: ShowTablesContext): LogicalPlan =
withOrigin(ctx) {
-if (ctx.LIKE != null) {
- logWarning("SHOW TABLES LIKE option is ignored.")
+if (ctx.qualifiedName() != null) {
+ logWarning("SHOW TABLES qualifiedName is ignored.")
}
-ShowTablesCommand(Option(ctx.db).map(_.getText))
+ShowTablesCommand(
+ Option(ctx.db).map(_.getText),
+ Option(ctx.pattern).map(p => unquoteString(p.getText)))
--- End diff --
`Option(ctx.pattern).map(string)` also see my comments in your branch.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14184][SQL] Support native execution of...
Github user hvanhovell commented on a diff in the pull request:
https://github.com/apache/spark/pull/11991#discussion_r57787597
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/SparkSqlParser.scala ---
@@ -70,12 +70,28 @@ class SparkSqlAstBuilder extends AstBuilder {
/**
* Create a [[ShowTablesCommand]] logical plan.
+ * Example SQL :
+ * SHOW TABLES [(IN|FROM) database_name] ['identifier_with_wildcards']
*/
override def visitShowTables(ctx: ShowTablesContext): LogicalPlan =
withOrigin(ctx) {
-if (ctx.LIKE != null) {
- logWarning("SHOW TABLES LIKE option is ignored.")
+if (ctx.qualifiedName() != null) {
+ logWarning("SHOW TABLES qualifiedName is ignored.")
--- End diff --
I am not sure any one (besides us) will understand this.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14184][SQL] Support native execution of...
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/11991#discussion_r57786921 --- Diff: sql/catalyst/src/main/antlr4/org/apache/spark/sql/catalyst/parser/ng/SqlBase.g4 --- @@ -107,7 +107,8 @@ statement | DROP TEMPORARY? FUNCTION (IF EXISTS)? qualifiedName #dropFunction | EXPLAIN explainOption* statement #explain | SHOW TABLES ((FROM | IN) db=identifier)? -(LIKE (qualifiedName | pattern=STRING))? #showTables +(LIKE? (qualifiedName | pattern=STRING))? #showTables --- End diff -- Nit `#showTables` is not alligned anymore. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14184][SQL] Support native execution of...
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/11991#discussion_r57786838 --- Diff: sql/catalyst/src/main/antlr4/org/apache/spark/sql/catalyst/parser/ng/SqlBase.g4 --- @@ -107,7 +107,8 @@ statement | DROP TEMPORARY? FUNCTION (IF EXISTS)? qualifiedName #dropFunction | EXPLAIN explainOption* statement #explain | SHOW TABLES ((FROM | IN) db=identifier)? --- End diff -- You can remove the 'from' option now. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14184][SQL] Support native execution of...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11991#issuecomment-203017554 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/54443/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14184][SQL] Support native execution of...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11991#issuecomment-203017546 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14184][SQL] Support native execution of...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11991#issuecomment-203016207 **[Test build #54443 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/54443/consoleFull)** for PR 11991 at commit [`b66f16e`](https://github.com/apache/spark/commit/b66f16e25d1208150e0f0fd507d98b51328cd485). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14184][SQL] Support native execution of...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11991#issuecomment-202968892 **[Test build #54443 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/54443/consoleFull)** for PR 11991 at commit [`b66f16e`](https://github.com/apache/spark/commit/b66f16e25d1208150e0f0fd507d98b51328cd485). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14184][SQL] Support native execution of...
Github user dilipbiswal commented on the pull request: https://github.com/apache/spark/pull/11991#issuecomment-202769968 @rxin Yes.. I am working on using the new ANTLR4 parser. Thank you. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14184][SQL] Support native execution of...
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/11991#issuecomment-202687185 I just merged https://github.com/apache/spark/pull/12015 Can you update to use the new ANTLR4 parser instead? We are going to remove the ANTLR3 one in the next day or two. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14184][SQL] Support native execution of...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11991#issuecomment-202680740 Build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14184][SQL] Support native execution of...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11991#issuecomment-202680741 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/54395/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14184][SQL] Support native execution of...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11991#issuecomment-202680527 **[Test build #54395 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/54395/consoleFull)** for PR 11991 at commit [`92fe39f`](https://github.com/apache/spark/commit/92fe39f2e9f7363c7b7bd915c494ffca2350b52f). * This patch **fails Spark unit tests**. * This patch **does not merge cleanly**. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14184][SQL] Support native execution of...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11991#issuecomment-202653602 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/54380/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14184][SQL] Support native execution of...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11991#issuecomment-202653600 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14184][SQL] Support native execution of...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11991#issuecomment-202653469 **[Test build #54380 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/54380/consoleFull)** for PR 11991 at commit [`92fe39f`](https://github.com/apache/spark/commit/92fe39f2e9f7363c7b7bd915c494ffca2350b52f). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14184][SQL] Support native execution of...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11991#issuecomment-202648258 **[Test build #54395 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/54395/consoleFull)** for PR 11991 at commit [`92fe39f`](https://github.com/apache/spark/commit/92fe39f2e9f7363c7b7bd915c494ffca2350b52f). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14184][SQL] Support native execution of...
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/11991#issuecomment-202647611 add to whitelist --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14184][SQL] Support native execution of...
Github user viirya commented on the pull request: https://github.com/apache/spark/pull/11991#issuecomment-202647358 @hvanhovell Can you please add @dilipbiswal into whitelist? Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14184][SQL] Support native execution of...
Github user dilipbiswal commented on the pull request: https://github.com/apache/spark/pull/11991#issuecomment-202638668 @hvanhovell Sure Herman. I will give it a try and get back to you with questions :-) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14184][SQL] Support native execution of...
Github user hvanhovell commented on the pull request: https://github.com/apache/spark/pull/11991#issuecomment-202629267 @dilipbiswal Could you also add support for the ANTLR4 parser? You'll need to a rule to SqlBase.q4 and do the parsing in SparkSqlParser.scala. Let me know if you need help. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14184][SQL] Support native execution of...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11991#issuecomment-202624967 **[Test build #54380 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/54380/consoleFull)** for PR 11991 at commit [`92fe39f`](https://github.com/apache/spark/commit/92fe39f2e9f7363c7b7bd915c494ffca2350b52f). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14184][SQL] Support native execution of...
Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/11991#discussion_r57648372
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/SparkQl.scala ---
@@ -369,13 +369,33 @@ private[sql] class SparkQl(conf: ParserConf =
SimpleParserConf()) extends Cataly
case Token("TOK_CLEARCACHE", Nil) =>
ClearCacheCommand
+ // SHOW TABLES [(IN|FROM) database_name]
['identifier_with_wildcards']
case Token("TOK_SHOWTABLES", args) =>
-val databaseName = args match {
- case Nil => None
- case Token("TOK_FROM", Token(dbName, Nil) :: Nil) :: Nil =>
Option(dbName)
+// Parses the following conditions
+// 1. SHOW TABLES
+// 2. SHOW TABLES FROM database 'table_pattern'
+// 3. SHOW TABLES FROM database
--- End diff --
I see, that's too bad. Let's just keep it
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14184][SQL] Support native execution of...
Github user dilipbiswal commented on a diff in the pull request:
https://github.com/apache/spark/pull/11991#discussion_r57646589
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/SparkQl.scala ---
@@ -369,13 +369,33 @@ private[sql] class SparkQl(conf: ParserConf =
SimpleParserConf()) extends Cataly
case Token("TOK_CLEARCACHE", Nil) =>
ClearCacheCommand
+ // SHOW TABLES [(IN|FROM) database_name]
['identifier_with_wildcards']
case Token("TOK_SHOWTABLES", args) =>
-val databaseName = args match {
- case Nil => None
- case Token("TOK_FROM", Token(dbName, Nil) :: Nil) :: Nil =>
Option(dbName)
+// Parses the following conditions
+// 1. SHOW TABLES
+// 2. SHOW TABLES FROM database 'table_pattern'
+// 3. SHOW TABLES FROM database
--- End diff --
@andrewor14 Hello, we always transform to TOK_FROM for both TOK_IN and
TOK_FROM in the grammer file.
https://github.com/apache/spark/blob/637a78f1d3dff00658324de3887d75c5ccd857be/sql/catalyst/src/main/antlr3/org/apache/spark/sql/catalyst/parser/SparkSqlParser.g#L1473
So now, both form works.
```SQL
show tables from default
show tables in default
```
I will fix up the comments to use 'IN' as opposed to 'FROM'. Did you want
to disallow this syntax in the grammer file ? Would there be a
backward-compatibility issue if we do that ?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14184][SQL] Support native execution of...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11991#issuecomment-202611071 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14184][SQL] Support native execution of...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11991#issuecomment-202611076 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/54373/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14184][SQL] Support native execution of...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11991#issuecomment-202611059 **[Test build #54373 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/54373/consoleFull)** for PR 11991 at commit [`1e15bc2`](https://github.com/apache/spark/commit/1e15bc216db893c680565246c31fe68e9ce215fd). * This patch **fails to build**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14184][SQL] Support native execution of...
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/11991#issuecomment-202605622 ok to test --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
