[GitHub] spark pull request #22458: [SPARK-25459] Add viewOriginalText back to Catalo...

2018-09-25 Thread zheyuan28
Github user zheyuan28 commented on a diff in the pull request:

https://github.com/apache/spark/pull/22458#discussion_r220257132
  
--- Diff: 
sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/HiveDDLSuite.scala 
---
@@ -2348,4 +2348,17 @@ class HiveDDLSuite
   }
 }
   }
+
+  test("desc formatted table should also show viewOriginalText for views") 
{
+withView("v1") {
+  sql("CREATE VIEW v1 AS SELECT 1 AS value")
+  assert(sql("DESC FORMATTED v1").collect().containsSlice(
+Seq(
+  Row("Type", "VIEW", ""),
+  Row("View Text", "SELECT 1 AS value", ""),
+  Row("View Original Text:", "SELECT 1 AS value", "")
--- End diff --

@MaxGekk @gatorsmile Sorry for late, I was trying to use hive client to 
create the older view. But I find the 
[toHiveTable](https://github.com/apache/spark/blob/master/sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveClientImpl.scala#L918)
 method will always use the expanded view text when I create view: 
https://github.com/apache/spark/blob/master/sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveClientImpl.scala#L955.
 Is that reasonable we change that to:
```scala
table.viewText.foreach { t => hiveTable.setViewExpandedText(t) }
table.viewOriginalText.foreach { t => hiveTable.setViewOriginalText(t) }
```
Here is my new test case:
```scala
test("SPARK-25459 desc formatted table for views created by older Spark") {
withTable("hive_table") {
  withView("old_view") {
spark.sql("CREATE TABLE hive_table AS SELECT 1 AS a, 2 AS b")
val expandedView = "SELECT `gen_attr_0` AS `a`, `gen_attr_1` AS `b` 
FROM (SELECT " +
  "`gen_attr_0`, `gen_attr_1` FROM (SELECT `a` AS `gen_attr_0`, `b` 
AS " +
  "`gen_attr_1` FROM hive_table) AS gen_subquery_0) AS hive_table"
val view = CatalogTable(
  identifier = TableIdentifier("old_view"),
  tableType = CatalogTableType.VIEW,
  storage = CatalogStorageFormat.empty,
  schema = new StructType().add("a", "int").add("b", "int"),
  viewText = Some(expandedView),
  viewOriginalText = Some("SELECT 1 AS a, 2 AS b")
)
hiveContext.sessionState.catalog.createTable(view, ignoreIfExists = 
false)
// Check the output rows.
assert(sql("DESC FORMATTED old_view").collect().containsSlice(
  Seq(
Row("Type", "VIEW", ""),
Row("View Text", expandedView, ""),
Row("View Original Text", "SELECT 1 AS a, 2 AS b", "")
  )
))
  }
}
  }
```


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] spark issue #22458: [SPARK-25459] Add viewOriginalText back to CatalogTable

2018-09-18 Thread zheyuan28
Github user zheyuan28 commented on the issue:

https://github.com/apache/spark/pull/22458
  
cc @gatorsmile 


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] spark pull request #22458: add viewOriginalText back to CatalogTable

2018-09-18 Thread zheyuan28
GitHub user zheyuan28 opened a pull request:

https://github.com/apache/spark/pull/22458

add viewOriginalText back to CatalogTable

## What changes were proposed in this pull request?

The `show create table` will show a lot of generated attributes for views 
that created by older Spark version. This PR will basically revert 
https://issues.apache.org/jira/browse/SPARK-19272 back, so when you `DESC 
[FORMATTED|EXTENDED] view` will show the original view DDL text.

## How was this patch tested?
Unit test.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/zheyuan28/spark-1 testbranch

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/spark/pull/22458.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #22458


commit 9f8b889cade55fbc8ee4068413e319f97739a939
Author: Chris Zhao 
Date:   2018-09-18T23:34:57Z

add viewOriginalText back to CatalogTable




---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org