[GitHub] spark pull request: [SPARK-5135][SQL] Add support for describe tab...
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/4227#issuecomment-73128709 Thanks. Merging now. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5135][SQL] Add support for describe tab...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4227#issuecomment-73127984 [Test build #26855 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26855/consoleFull) for PR 4227 at commit [`053826f`](https://github.com/apache/spark/commit/053826f19efd41c5805e64d4c5818204f5198aab). * This patch **passes all tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5135][SQL] Add support for describe tab...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4227#issuecomment-73127998 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/26855/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5135][SQL] Add support for describe tab...
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/4227 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5135][SQL] Add support for describe tab...
Github user OopsOutOfMemory commented on the pull request: https://github.com/apache/spark/pull/4227#issuecomment-73062533 Hi, @rxin Thanks for the advice. I refined the code according to your advice now I think it's ready to go. yeah, this PR is a little conflict with https://github.com/apache/spark/pull/4249/files. They all modified `logical DescribeCommand`. But I think it's not a big deal. This two PRs (this and #4249). No matter which one will be merged first, I'll bring the other one up-to-date quickly. And I don't want to resolve the two issues in `One PR` which make issues untraceable. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5135][SQL] Add support for describe tab...
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/4227#issuecomment-73113359 Jenkins, retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5135][SQL] Add support for describe tab...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4227#issuecomment-73114302 [Test build #26855 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26855/consoleFull) for PR 4227 at commit [`053826f`](https://github.com/apache/spark/commit/053826f19efd41c5805e64d4c5818204f5198aab). * This patch merges cleanly. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5135][SQL] Add support for describe tab...
Github user OopsOutOfMemory commented on the pull request: https://github.com/apache/spark/pull/4227#issuecomment-72819162 May AmplabJenkins test this please ? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5135][SQL] Add support for describe tab...
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/4227#issuecomment-72819767 Jenkins, test this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5135][SQL] Add support for describe tab...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4227#issuecomment-72819908 [Test build #26743 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26743/consoleFull) for PR 4227 at commit [`a3061a5`](https://github.com/apache/spark/commit/a3061a5a99159c5983abdec346da3cd9a715eda7). * This patch merges cleanly. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5135][SQL] Add support for describe tab...
Github user rxin commented on a diff in the pull request:
https://github.com/apache/spark/pull/4227#discussion_r24147649
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/types/dataTypes.scala ---
@@ -227,8 +227,9 @@ abstract class DataType {
def json: String = compact(render(jsonValue))
def prettyJson: String = pretty(render(jsonValue))
-}
+ def toSimpleString: String = typeName
--- End diff --
can you rename this simpleString?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5135][SQL] Add support for describe tab...
Github user rxin commented on a diff in the pull request:
https://github.com/apache/spark/pull/4227#discussion_r24147699
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/commands.scala ---
@@ -174,9 +175,19 @@ case class UncacheTableCommand(tableName: String)
extends RunnableCommand {
@DeveloperApi
case class DescribeCommand(
child: SparkPlan,
-override val output: Seq[Attribute]) extends RunnableCommand {
+override val output: Seq[Attribute],
+isExtended: Boolean) extends RunnableCommand {
override def run(sqlContext: SQLContext) = {
-child.output.map(field = Row(field.name, field.dataType.toString,
null))
+val rows = new ArrayBuffer[Row]()
+rows ++= child.schema.fields.map{field =
--- End diff --
why not just
```scala
child.schema.fields.map { field =
val comment = if (field.metadata.contains(comment))
field.metadata.get(comment) else
Row(field.name, field.dataType.toSimpleString, comment)
}
```
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5135][SQL] Add support for describe tab...
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/4227#issuecomment-73007162 I'd like to merge this into 1.3.0. Can you address the two comments I have and make sure it still works with other changes? btw does this one conflict with your other change? https://github.com/apache/spark/pull/4249/files --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5135][SQL] Add support for describe tab...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4227#issuecomment-72833970 [Test build #26743 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26743/consoleFull) for PR 4227 at commit [`a3061a5`](https://github.com/apache/spark/commit/a3061a5a99159c5983abdec346da3cd9a715eda7). * This patch **passes all tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5135][SQL] Add support for describe tab...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4227#issuecomment-72833983 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/26743/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5135][SQL] Add support for describe tab...
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/4227#issuecomment-72792300 @marmbrus this looks good to me. Any more comments? @OopsOutOfMemory would be great to bring this up to date with master. Thanks. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5135][SQL] Add support for describe tab...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4227#issuecomment-71708164 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5135][SQL] Add support for describe tab...
GitHub user OopsOutOfMemory opened a pull request: https://github.com/apache/spark/pull/4227 [SPARK-5135][SQL] Add support for describe table to DDL in SQLContext Hi, @rxin @marmbrus I considered your suggestion and now re-write it. This is now up-to-date. Could u please review it ? You can merge this pull request into a Git repository by running: $ git pull https://github.com/OopsOutOfMemory/spark describe Alternatively you can review and apply these changes as the patch at: https://github.com/apache/spark/pull/4227.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #4227 commit 5b7ae19dfdc4410f1018193e0b1701abf799c439 Author: OopsOutOfMemory victorshen...@126.com Date: 2015-01-16T10:45:13Z patch commit d1689e2fdfed67decabe6c696a3d7f051138a7ad Author: OopsOutOfMemory victorshen...@126.com Date: 2015-01-16T15:49:50Z refine imports commit 5b56286c7df0e721496478ab72d56a41a72d9fd6 Author: OopsOutOfMemory victorshen...@126.com Date: 2015-01-16T10:45:13Z patch commit d70b699bf391f7011540510a22cdcf3f6317945f Author: OopsOutOfMemory victorshen...@126.com Date: 2015-01-16T15:49:50Z refine imports commit 5abfbc0fbfd62c7ab0ab33f99619f7c2b6fb6ee6 Author: OopsOutOfMemory victorshen...@126.com Date: 2015-01-27T16:17:19Z refine commit 6537b16011e18a51648e98cf3674b7d334a467b2 Author: OopsOutOfMemory victorshen...@126.com Date: 2015-01-27T16:23:40Z refine commit 1b85c73bcb35cc162cd2ad678927d664015b9ce6 Author: OopsOutOfMemory victorshen...@126.com Date: 2015-01-27T16:27:36Z refine commit 88ee78f6072843be0cfe3c559017ab381ba78d5a Author: OopsOutOfMemory victorshen...@126.com Date: 2015-01-27T16:34:08Z style refine commit a083cc5bc31e4e762095953d16f21790d763d495 Author: OopsOutOfMemory victorshen...@126.com Date: 2015-01-27T16:36:55Z refine commit 8e1be4935d286681cb71ccbe64c0b0e3c9a48352 Author: OopsOutOfMemory victorshen...@126.com Date: 2015-01-27T19:01:27Z refine commit 5d1b54fa47a04bacc0651d4617a9f38d2c4db983 Author: OopsOutOfMemory victorshen...@126.com Date: 2015-01-27T19:04:36Z style fix commit b2e30a01555c40dfec6d7d72f926424b8f66fd81 Author: OopsOutOfMemory victorshen...@126.com Date: 2015-01-27T19:05:36Z refine import --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5135][SQL] Add support for describe tab...
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/4227#issuecomment-71769478 Thanks for submitting the new version. Are these two PRs working on the same thing? https://github.com/apache/spark/pull/4207 Would be great if you two can chime in on each other's PR. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5135][SQL] Add support for describe tab...
Github user OopsOutOfMemory commented on the pull request: https://github.com/apache/spark/pull/4227#issuecomment-71770856 yeah, @rxin, would you like to talk with @marmbrus for `what we'd like to show in describe extended table` in SQLContext and then file a `JIRA` issues? So that we can do it separately but not in this PR ? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
