[GitHub] spark pull request: [SPARK-15312] [SQL] Detect Duplicate Key in Pa...
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/13095 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15312] [SQL] Detect Duplicate Key in Pa...
Github user cloud-fan commented on the pull request: https://github.com/apache/spark/pull/13095#issuecomment-220817564 thanks, merging to master and 2.0! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15312] [SQL] Detect Duplicate Key in Pa...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/13095#issuecomment-220816499 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/59089/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15312] [SQL] Detect Duplicate Key in Pa...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/13095#issuecomment-220816498 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15312] [SQL] Detect Duplicate Key in Pa...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/13095#issuecomment-220816465 **[Test build #59089 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/59089/consoleFull)** for PR 13095 at commit [`d7c2420`](https://github.com/apache/spark/commit/d7c2420cd21e812e08bdea7aa27adf42fe534b98). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15312] [SQL] Detect Duplicate Key in Pa...
Github user gatorsmile commented on the pull request: https://github.com/apache/spark/pull/13095#issuecomment-220814326 Thank you, @cloud-fan ! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15312] [SQL] Detect Duplicate Key in Pa...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/13095#issuecomment-220814047 **[Test build #59089 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/59089/consoleFull)** for PR 13095 at commit [`d7c2420`](https://github.com/apache/spark/commit/d7c2420cd21e812e08bdea7aa27adf42fe534b98). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15312] [SQL] Detect Duplicate Key in Pa...
Github user cloud-fan commented on the pull request: https://github.com/apache/spark/pull/13095#issuecomment-220813962 LGTM, pending jenkins --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15312] [SQL] Detect Duplicate Key in Pa...
Github user cloud-fan commented on the pull request: https://github.com/apache/spark/pull/13095#issuecomment-220813959 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15312] [SQL] Detect Duplicate Key in Pa...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/13095#issuecomment-220657917 **[Test build #59000 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/59000/consoleFull)** for PR 13095 at commit [`d7c2420`](https://github.com/apache/spark/commit/d7c2420cd21e812e08bdea7aa27adf42fe534b98). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15312] [SQL] Detect Duplicate Key in Pa...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/13095#issuecomment-220658173 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15312] [SQL] Detect Duplicate Key in Pa...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/13095#issuecomment-220658177 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/59000/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15312] [SQL] Detect Duplicate Key in Pa...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/13095#issuecomment-220636065 **[Test build #59000 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/59000/consoleFull)** for PR 13095 at commit [`d7c2420`](https://github.com/apache/spark/commit/d7c2420cd21e812e08bdea7aa27adf42fe534b98). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15312] [SQL] Detect Duplicate Key in Pa...
Github user gatorsmile commented on the pull request: https://github.com/apache/spark/pull/13095#issuecomment-220635823 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15312] [SQL] Detect Duplicate Key in Pa...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/13095#issuecomment-220548134 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/58964/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15312] [SQL] Detect Duplicate Key in Pa...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/13095#issuecomment-220548132 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15312] [SQL] Detect Duplicate Key in Pa...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/13095#issuecomment-220547970 **[Test build #58964 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/58964/consoleFull)** for PR 13095 at commit [`d7c2420`](https://github.com/apache/spark/commit/d7c2420cd21e812e08bdea7aa27adf42fe534b98). * This patch **fails PySpark unit tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15312] [SQL] Detect Duplicate Key in Pa...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/13095#issuecomment-220533402 **[Test build #58964 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/58964/consoleFull)** for PR 13095 at commit [`d7c2420`](https://github.com/apache/spark/commit/d7c2420cd21e812e08bdea7aa27adf42fe534b98). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15312] [SQL] Detect Duplicate Key in Pa...
Github user gatorsmile commented on the pull request: https://github.com/apache/spark/pull/13095#issuecomment-220532545 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15312] [SQL] Detect Duplicate Key in Pa...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/13095#issuecomment-220532100 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/58955/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15312] [SQL] Detect Duplicate Key in Pa...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/13095#issuecomment-220532097 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15312] [SQL] Detect Duplicate Key in Pa...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/13095#issuecomment-220532067 **[Test build #58955 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/58955/consoleFull)** for PR 13095 at commit [`d7c2420`](https://github.com/apache/spark/commit/d7c2420cd21e812e08bdea7aa27adf42fe534b98). * This patch **fails MiMa tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15312] [SQL] Detect Duplicate Key in Pa...
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/13095#issuecomment-220530579 cc @cloud-fan --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15312] [SQL] Detect Duplicate Key in Pa...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/13095#issuecomment-220530398 **[Test build #58955 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/58955/consoleFull)** for PR 13095 at commit [`d7c2420`](https://github.com/apache/spark/commit/d7c2420cd21e812e08bdea7aa27adf42fe534b98). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15312] [SQL] Detect Duplicate Key in Pa...
Github user gatorsmile commented on the pull request: https://github.com/apache/spark/pull/13095#issuecomment-220529713 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15312] [SQL] Detect Duplicate Key in Pa...
Github user gatorsmile commented on the pull request: https://github.com/apache/spark/pull/13095#issuecomment-220529704 Is this ready to merge? @hvanhovell @rxin Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15312] [SQL] Detect Duplicate Key in Pa...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/13095#issuecomment-220101530 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/58790/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15312] [SQL] Detect Duplicate Key in Pa...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/13095#issuecomment-220101527 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15312] [SQL] Detect Duplicate Key in Pa...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/13095#issuecomment-220101213 **[Test build #58790 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/58790/consoleFull)** for PR 13095 at commit [`d7c2420`](https://github.com/apache/spark/commit/d7c2420cd21e812e08bdea7aa27adf42fe534b98). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15312] [SQL] Detect Duplicate Key in Pa...
Github user hvanhovell commented on the pull request: https://github.com/apache/spark/pull/13095#issuecomment-220077433 LGTM pending Jenkins. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15312] [SQL] Detect Duplicate Key in Pa...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/13095#issuecomment-220076276 **[Test build #58790 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/58790/consoleFull)** for PR 13095 at commit [`d7c2420`](https://github.com/apache/spark/commit/d7c2420cd21e812e08bdea7aa27adf42fe534b98). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15312] [SQL] Detect Duplicate Key in Pa...
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/13095#discussion_r63731185
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/ParserUtils.scala
---
@@ -43,6 +44,14 @@ object ParserUtils {
new ParseException(s"Operation not allowed: $message", ctx)
}
+ /** Check if duplicate keys exist in a set of key-value pairs. */
+ def checkDuplicateKeys(
+ keyPairs: mutable.Buffer[_ <: (String, Any)], ctx:
ParserRuleContext): Unit = {
--- End diff --
Great idea! Let me change it now
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15312] [SQL] Detect Duplicate Key in Pa...
Github user hvanhovell commented on a diff in the pull request:
https://github.com/apache/spark/pull/13095#discussion_r63715298
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/ParserUtils.scala
---
@@ -43,6 +44,14 @@ object ParserUtils {
new ParseException(s"Operation not allowed: $message", ctx)
}
+ /** Check if duplicate keys exist in a set of key-value pairs. */
+ def checkDuplicateKeys(
+ keyPairs: mutable.Buffer[_ <: (String, Any)], ctx:
ParserRuleContext): Unit = {
--- End diff --
Style: Parameter per line.
`mutable.Buffer[_ <: (String, Any)]` is it possible to just check a `Seq`
or `Traversable`? Do you think it is possible to improve the signature (maybe
move the type of Any into a generic type)? It is kinda messy now.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15312] [SQL] Detect Duplicate Key in Pa...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/13095#issuecomment-219639672 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15312] [SQL] Detect Duplicate Key in Pa...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/13095#issuecomment-219639674 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/58667/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15312] [SQL] Detect Duplicate Key in Pa...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/13095#issuecomment-219639437 **[Test build #58667 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/58667/consoleFull)** for PR 13095 at commit [`06f89d5`](https://github.com/apache/spark/commit/06f89d511233373e3d3bca1cb424da8f77976f7d). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15312] [SQL] Detect Duplicate Key in Pa...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/13095#issuecomment-219626498 **[Test build #58667 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/58667/consoleFull)** for PR 13095 at commit [`06f89d5`](https://github.com/apache/spark/commit/06f89d511233373e3d3bca1cb424da8f77976f7d). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15312] [SQL] Detect Duplicate Key in Pa...
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/13095#discussion_r63428168
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/AstBuilder.scala
---
@@ -127,10 +127,8 @@ class AstBuilder extends SqlBaseBaseVisitor[AnyRef]
with Logging {
}
// Check for duplicate names.
- ctes.groupBy(_._1).filter(_._2.size > 1).foreach {
-case (name, _) =>
- throw new ParseException(
-s"Name '$name' is used for multiple common table expressions",
ctx)
+ ctes.groupBy(_._1).filter(_._2.size > 1).foreach { case (name, _) =>
--- End diff --
Great idea! Will do it. Thanks!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15312] [SQL] Detect Duplicate Key in Pa...
Github user hvanhovell commented on the pull request: https://github.com/apache/spark/pull/13095#issuecomment-219548364 @gatorsmile Looks good, one smallish comment. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15312] [SQL] Detect Duplicate Key in Pa...
Github user hvanhovell commented on a diff in the pull request:
https://github.com/apache/spark/pull/13095#discussion_r63425710
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/AstBuilder.scala
---
@@ -127,10 +127,8 @@ class AstBuilder extends SqlBaseBaseVisitor[AnyRef]
with Logging {
}
// Check for duplicate names.
- ctes.groupBy(_._1).filter(_._2.size > 1).foreach {
-case (name, _) =>
- throw new ParseException(
-s"Name '$name' is used for multiple common table expressions",
ctx)
+ ctes.groupBy(_._1).filter(_._2.size > 1).foreach { case (name, _) =>
--- End diff --
Lets just add a `checkDuplicateKeys` method to ParseUtils (or something
similar), and not call the same piece of code 3 times. . What do you think?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15312] [SQL] Detect Duplicate Key in Pa...
Github user gatorsmile commented on the pull request: https://github.com/apache/spark/pull/13095#issuecomment-219099341 @hvanhovell Could you please review this PR? Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15312] [SQL] Detect Duplicate Key in Pa...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/13095#issuecomment-219097172 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15312] [SQL] Detect Duplicate Key in Pa...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/13095#issuecomment-219097176 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/58578/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15312] [SQL] Detect Duplicate Key in Pa...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/13095#issuecomment-219096955 **[Test build #58578 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/58578/consoleFull)** for PR 13095 at commit [`5a15c13`](https://github.com/apache/spark/commit/5a15c13f2e26a0d29a62b0549068a4413166fb2e). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15312] [SQL] Detect Duplicate Key in Pa...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/13095#issuecomment-219075266 **[Test build #58578 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/58578/consoleFull)** for PR 13095 at commit [`5a15c13`](https://github.com/apache/spark/commit/5a15c13f2e26a0d29a62b0549068a4413166fb2e). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15312] [SQL] Detect Duplicate Key in Pa...
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/13095#discussion_r63200541
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/AstBuilder.scala
---
@@ -220,11 +220,18 @@ class AstBuilder extends SqlBaseBaseVisitor[AnyRef]
with Logging {
*/
override def visitPartitionSpec(
ctx: PartitionSpecContext): Map[String, Option[String]] =
withOrigin(ctx) {
-ctx.partitionVal.asScala.map { pVal =>
+val parts = ctx.partitionVal.asScala.map { pVal =>
val name = pVal.identifier.getText.toLowerCase
val value = Option(pVal.constant).map(visitStringConstant)
name -> value
-}.toMap
+}
+// Check for duplicate partition columns in one spec.
+parts.groupBy(_._1).filter(_._2.size > 1).foreach {
+ case (name, _) =>
--- End diff --
Sure, let me change it. Thanks!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15312] [SQL] Detect Duplicate Key in Pa...
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/13095#discussion_r63151860
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/AstBuilder.scala
---
@@ -220,11 +220,18 @@ class AstBuilder extends SqlBaseBaseVisitor[AnyRef]
with Logging {
*/
override def visitPartitionSpec(
ctx: PartitionSpecContext): Map[String, Option[String]] =
withOrigin(ctx) {
-ctx.partitionVal.asScala.map { pVal =>
+val parts = ctx.partitionVal.asScala.map { pVal =>
val name = pVal.identifier.getText.toLowerCase
val value = Option(pVal.constant).map(visitStringConstant)
name -> value
-}.toMap
+}
+// Check for duplicate partition columns in one spec.
+parts.groupBy(_._1).filter(_._2.size > 1).foreach {
+ case (name, _) =>
--- End diff --
(it seems putting the case in the same line is preferred `foreach { case
(name, _) =>`,
[scala-style-guide#pattern-matching](https://github.com/databricks/scala-style-guide#pattern-matching))
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15312] [SQL] Detect Duplicate Key in Pa...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/13095#issuecomment-218985277 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/58550/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15312] [SQL] Detect Duplicate Key in Pa...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/13095#issuecomment-218985274 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15312] [SQL] Detect Duplicate Key in Pa...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/13095#issuecomment-218985070 **[Test build #58550 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/58550/consoleFull)** for PR 13095 at commit [`23f0cfb`](https://github.com/apache/spark/commit/23f0cfb994178e13df84a5c4904f1c148b199f48). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15312] [SQL] Detect Duplicate Key in Pa...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/13095#issuecomment-218969269 **[Test build #58550 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/58550/consoleFull)** for PR 13095 at commit [`23f0cfb`](https://github.com/apache/spark/commit/23f0cfb994178e13df84a5c4904f1c148b199f48). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15312] [SQL] Detect Duplicate Key in Pa...
GitHub user gatorsmile opened a pull request: https://github.com/apache/spark/pull/13095 [SPARK-15312] [SQL] Detect Duplicate Key in Partition Spec and Table Properties What changes were proposed in this pull request? When there are duplicate keys in the partition specs or table properties, we always use the last value and ignore all the previous values. This is caused by toMap. partition specs or table properties are widely used in multiple DDL statements. This PR is to detect the duplicates and issue an exception if found. How was this patch tested? Added test cases in DDLSuite You can merge this pull request into a Git repository by running: $ git pull https://github.com/gatorsmile/spark detectDuplicate Alternatively you can review and apply these changes as the patch at: https://github.com/apache/spark/pull/13095.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #13095 commit 23f0cfb994178e13df84a5c4904f1c148b199f48 Author: gatorsmile Date: 2016-05-13T07:09:23Z detect duplicates before toMap --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
