[GitHub] spark pull request #17394: [SPARK-20067] [SQL] Unify and Clean Up Desc Comma...

2017-04-04 Thread asfgit 
Github user asfgit closed the pull request at:

https://github.com/apache/spark/pull/17394


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #17394: [SPARK-20067] [SQL] Unify and Clean Up Desc Comma...

2017-04-03 Thread gatorsmile 
Github user gatorsmile commented on a diff in the pull request:

https://github.com/apache/spark/pull/17394#discussion_r109558929
  
--- Diff: sql/core/src/test/resources/sql-tests/results/describe.sql.out ---
@@ -1,205 +1,259 @@
 -- Automatically generated by SQLQueryTestSuite
--- Number of queries: 14
+-- Number of queries: 31
 
 
 -- !query 0
-CREATE TABLE t (a STRING, b INT, c STRING, d STRING) USING parquet 
PARTITIONED BY (c, d) COMMENT 'table_comment'
+CREATE TABLE t (a STRING, b INT, c STRING, d STRING) USING parquet
+  PARTITIONED BY (c, d) CLUSTERED BY (a) SORTED BY (b ASC) INTO 2 BUCKETS
+  COMMENT 'table_comment'
 -- !query 0 schema
 struct<>
 -- !query 0 output
 
 
 
 -- !query 1
-ALTER TABLE t ADD PARTITION (c='Us', d=1)
+CREATE TEMPORARY VIEW temp_v AS SELECT * FROM t
 -- !query 1 schema
 struct<>
 -- !query 1 output
 
 
 
 -- !query 2
-DESCRIBE t
+CREATE TEMPORARY VIEW temp_Data_Source_View
+  USING org.apache.spark.sql.sources.DDLScanSource
+  OPTIONS (
+From '1',
+To '10',
+Table 'test1')
 -- !query 2 schema
-struct
+struct<>
 -- !query 2 output
-# Partition Information
+
+
+
+-- !query 3
+CREATE VIEW v AS SELECT * FROM t
+-- !query 3 schema
+struct<>
+-- !query 3 output
+
+
+
+-- !query 4
+ALTER TABLE t ADD PARTITION (c='Us', d=1)
+-- !query 4 schema
+struct<>
+-- !query 4 output
+
+
+
+-- !query 5
+DESCRIBE t
+-- !query 5 schema
+struct
+-- !query 5 output
 # col_name data_type   comment 
 a  string  
 b  int 
 c  string  
-c  string  
 d  string  
+# Partition Information
+# col_name data_type   comment 
+c  string  
 d  string
 
 
--- !query 3
-DESC t
--- !query 3 schema
+-- !query 6
+DESC default.t
+-- !query 6 schema
 struct
--- !query 3 output
-# Partition Information
+-- !query 6 output
 # col_name data_type   comment 
 a  string  
 b  int 
 c  string  
-c  string  
 d  string  
+# Partition Information
+# col_name data_type   comment 
+c  string  
 d  string
 
 
--- !query 4
+-- !query 7
 DESC TABLE t
--- !query 4 schema
+-- !query 7 schema
 struct
--- !query 4 output
-# Partition Information
+-- !query 7 output
 # col_name data_type   comment 
 a  string  
 b  int 
 c  string  
-c  string  
 d  string  
+# Partition Information
+# col_name data_type   comment 
+c  string  
 d  string
 
 
--- !query 5
+-- !query 8
 DESC FORMATTED t
--- !query 5 schema
+-- !query 8 schema
 struct
--- !query 5 output
-# Detailed Table Information   

-# Partition Information
-#

[GitHub] spark pull request #17394: [SPARK-20067] [SQL] Unify and Clean Up Desc Comma...

2017-04-03 Thread cloud-fan 
Github user cloud-fan commented on a diff in the pull request:

https://github.com/apache/spark/pull/17394#discussion_r109375300
  
--- Diff: sql/core/src/test/resources/sql-tests/results/describe.sql.out ---
@@ -1,205 +1,259 @@
 -- Automatically generated by SQLQueryTestSuite
--- Number of queries: 14
+-- Number of queries: 31
 
 
 -- !query 0
-CREATE TABLE t (a STRING, b INT, c STRING, d STRING) USING parquet 
PARTITIONED BY (c, d) COMMENT 'table_comment'
+CREATE TABLE t (a STRING, b INT, c STRING, d STRING) USING parquet
+  PARTITIONED BY (c, d) CLUSTERED BY (a) SORTED BY (b ASC) INTO 2 BUCKETS
+  COMMENT 'table_comment'
 -- !query 0 schema
 struct<>
 -- !query 0 output
 
 
 
 -- !query 1
-ALTER TABLE t ADD PARTITION (c='Us', d=1)
+CREATE TEMPORARY VIEW temp_v AS SELECT * FROM t
 -- !query 1 schema
 struct<>
 -- !query 1 output
 
 
 
 -- !query 2
-DESCRIBE t
+CREATE TEMPORARY VIEW temp_Data_Source_View
+  USING org.apache.spark.sql.sources.DDLScanSource
+  OPTIONS (
+From '1',
+To '10',
+Table 'test1')
 -- !query 2 schema
-struct
+struct<>
 -- !query 2 output
-# Partition Information
+
+
+
+-- !query 3
+CREATE VIEW v AS SELECT * FROM t
+-- !query 3 schema
+struct<>
+-- !query 3 output
+
+
+
+-- !query 4
+ALTER TABLE t ADD PARTITION (c='Us', d=1)
+-- !query 4 schema
+struct<>
+-- !query 4 output
+
+
+
+-- !query 5
+DESCRIBE t
+-- !query 5 schema
+struct
+-- !query 5 output
 # col_name data_type   comment 
 a  string  
 b  int 
 c  string  
-c  string  
 d  string  
+# Partition Information
+# col_name data_type   comment 
+c  string  
 d  string
 
 
--- !query 3
-DESC t
--- !query 3 schema
+-- !query 6
+DESC default.t
+-- !query 6 schema
 struct
--- !query 3 output
-# Partition Information
+-- !query 6 output
 # col_name data_type   comment 
 a  string  
 b  int 
 c  string  
-c  string  
 d  string  
+# Partition Information
+# col_name data_type   comment 
+c  string  
 d  string
 
 
--- !query 4
+-- !query 7
 DESC TABLE t
--- !query 4 schema
+-- !query 7 schema
 struct
--- !query 4 output
-# Partition Information
+-- !query 7 output
 # col_name data_type   comment 
 a  string  
 b  int 
 c  string  
-c  string  
 d  string  
+# Partition Information
+# col_name data_type   comment 
+c  string  
 d  string
 
 
--- !query 5
+-- !query 8
 DESC FORMATTED t
--- !query 5 schema
+-- !query 8 schema
 struct
--- !query 5 output
-# Detailed Table Information   

-# Partition Information
-#

[GitHub] spark pull request #17394: [SPARK-20067] [SQL] Unify and Clean Up Desc Comma...

2017-04-02 Thread gatorsmile 
Github user gatorsmile commented on a diff in the pull request:

https://github.com/apache/spark/pull/17394#discussion_r109311325
  
--- Diff: sql/core/src/test/resources/sql-tests/results/describe.sql.out ---
@@ -1,205 +1,259 @@
 -- Automatically generated by SQLQueryTestSuite
--- Number of queries: 14
+-- Number of queries: 31
 
 
 -- !query 0
-CREATE TABLE t (a STRING, b INT, c STRING, d STRING) USING parquet 
PARTITIONED BY (c, d) COMMENT 'table_comment'
+CREATE TABLE t (a STRING, b INT, c STRING, d STRING) USING parquet
+  PARTITIONED BY (c, d) CLUSTERED BY (a) SORTED BY (b ASC) INTO 2 BUCKETS
+  COMMENT 'table_comment'
 -- !query 0 schema
 struct<>
 -- !query 0 output
 
 
 
 -- !query 1
-ALTER TABLE t ADD PARTITION (c='Us', d=1)
+CREATE TEMPORARY VIEW temp_v AS SELECT * FROM t
 -- !query 1 schema
 struct<>
 -- !query 1 output
 
 
 
 -- !query 2
-DESCRIBE t
+CREATE TEMPORARY VIEW temp_Data_Source_View
+  USING org.apache.spark.sql.sources.DDLScanSource
+  OPTIONS (
+From '1',
+To '10',
+Table 'test1')
 -- !query 2 schema
-struct
+struct<>
 -- !query 2 output
-# Partition Information
+
+
+
+-- !query 3
+CREATE VIEW v AS SELECT * FROM t
+-- !query 3 schema
+struct<>
+-- !query 3 output
+
+
+
+-- !query 4
+ALTER TABLE t ADD PARTITION (c='Us', d=1)
+-- !query 4 schema
+struct<>
+-- !query 4 output
+
+
+
+-- !query 5
+DESCRIBE t
+-- !query 5 schema
+struct
+-- !query 5 output
 # col_name data_type   comment 
 a  string  
 b  int 
 c  string  
-c  string  
 d  string  
+# Partition Information
+# col_name data_type   comment 
+c  string  
 d  string
 
 
--- !query 3
-DESC t
--- !query 3 schema
+-- !query 6
+DESC default.t
+-- !query 6 schema
 struct
--- !query 3 output
-# Partition Information
+-- !query 6 output
 # col_name data_type   comment 
 a  string  
 b  int 
 c  string  
-c  string  
 d  string  
+# Partition Information
+# col_name data_type   comment 
+c  string  
 d  string
 
 
--- !query 4
+-- !query 7
 DESC TABLE t
--- !query 4 schema
+-- !query 7 schema
 struct
--- !query 4 output
-# Partition Information
+-- !query 7 output
 # col_name data_type   comment 
 a  string  
 b  int 
 c  string  
-c  string  
 d  string  
+# Partition Information
+# col_name data_type   comment 
+c  string  
 d  string
 
 
--- !query 5
+-- !query 8
 DESC FORMATTED t
--- !query 5 schema
+-- !query 8 schema
 struct
--- !query 5 output
-# Detailed Table Information   

-# Partition Information
-#

[GitHub] spark pull request #17394: [SPARK-20067] [SQL] Unify and Clean Up Desc Comma...

2017-04-02 Thread cloud-fan 
Github user cloud-fan commented on a diff in the pull request:

https://github.com/apache/spark/pull/17394#discussion_r109307263
  
--- Diff: sql/core/src/test/resources/sql-tests/results/describe.sql.out ---
@@ -1,205 +1,259 @@
 -- Automatically generated by SQLQueryTestSuite
--- Number of queries: 14
+-- Number of queries: 31
 
 
 -- !query 0
-CREATE TABLE t (a STRING, b INT, c STRING, d STRING) USING parquet 
PARTITIONED BY (c, d) COMMENT 'table_comment'
+CREATE TABLE t (a STRING, b INT, c STRING, d STRING) USING parquet
+  PARTITIONED BY (c, d) CLUSTERED BY (a) SORTED BY (b ASC) INTO 2 BUCKETS
+  COMMENT 'table_comment'
 -- !query 0 schema
 struct<>
 -- !query 0 output
 
 
 
 -- !query 1
-ALTER TABLE t ADD PARTITION (c='Us', d=1)
+CREATE TEMPORARY VIEW temp_v AS SELECT * FROM t
 -- !query 1 schema
 struct<>
 -- !query 1 output
 
 
 
 -- !query 2
-DESCRIBE t
+CREATE TEMPORARY VIEW temp_Data_Source_View
+  USING org.apache.spark.sql.sources.DDLScanSource
+  OPTIONS (
+From '1',
+To '10',
+Table 'test1')
 -- !query 2 schema
-struct
+struct<>
 -- !query 2 output
-# Partition Information
+
+
+
+-- !query 3
+CREATE VIEW v AS SELECT * FROM t
+-- !query 3 schema
+struct<>
+-- !query 3 output
+
+
+
+-- !query 4
+ALTER TABLE t ADD PARTITION (c='Us', d=1)
+-- !query 4 schema
+struct<>
+-- !query 4 output
+
+
+
+-- !query 5
+DESCRIBE t
+-- !query 5 schema
+struct
+-- !query 5 output
 # col_name data_type   comment 
 a  string  
 b  int 
 c  string  
-c  string  
 d  string  
+# Partition Information
+# col_name data_type   comment 
+c  string  
 d  string
 
 
--- !query 3
-DESC t
--- !query 3 schema
+-- !query 6
+DESC default.t
+-- !query 6 schema
 struct
--- !query 3 output
-# Partition Information
+-- !query 6 output
 # col_name data_type   comment 
 a  string  
 b  int 
 c  string  
-c  string  
 d  string  
+# Partition Information
+# col_name data_type   comment 
+c  string  
 d  string
 
 
--- !query 4
+-- !query 7
 DESC TABLE t
--- !query 4 schema
+-- !query 7 schema
 struct
--- !query 4 output
-# Partition Information
+-- !query 7 output
 # col_name data_type   comment 
 a  string  
 b  int 
 c  string  
-c  string  
 d  string  
+# Partition Information
+# col_name data_type   comment 
+c  string  
 d  string
 
 
--- !query 5
+-- !query 8
 DESC FORMATTED t
--- !query 5 schema
+-- !query 8 schema
 struct
--- !query 5 output
-# Detailed Table Information   

-# Partition Information
-#

[GitHub] spark pull request #17394: [SPARK-20067] [SQL] Unify and Clean Up Desc Comma...

2017-04-01 Thread gatorsmile 
Github user gatorsmile commented on a diff in the pull request:

https://github.com/apache/spark/pull/17394#discussion_r109300476
  
--- Diff: sql/core/src/test/resources/sql-tests/results/describe.sql.out ---
@@ -1,205 +1,259 @@
 -- Automatically generated by SQLQueryTestSuite
--- Number of queries: 14
+-- Number of queries: 31
 
 
 -- !query 0
-CREATE TABLE t (a STRING, b INT, c STRING, d STRING) USING parquet 
PARTITIONED BY (c, d) COMMENT 'table_comment'
+CREATE TABLE t (a STRING, b INT, c STRING, d STRING) USING parquet
+  PARTITIONED BY (c, d) CLUSTERED BY (a) SORTED BY (b ASC) INTO 2 BUCKETS
+  COMMENT 'table_comment'
 -- !query 0 schema
 struct<>
 -- !query 0 output
 
 
 
 -- !query 1
-ALTER TABLE t ADD PARTITION (c='Us', d=1)
+CREATE TEMPORARY VIEW temp_v AS SELECT * FROM t
 -- !query 1 schema
 struct<>
 -- !query 1 output
 
 
 
 -- !query 2
-DESCRIBE t
+CREATE TEMPORARY VIEW temp_Data_Source_View
+  USING org.apache.spark.sql.sources.DDLScanSource
+  OPTIONS (
+From '1',
+To '10',
+Table 'test1')
 -- !query 2 schema
-struct
+struct<>
 -- !query 2 output
-# Partition Information
+
+
+
+-- !query 3
+CREATE VIEW v AS SELECT * FROM t
+-- !query 3 schema
+struct<>
+-- !query 3 output
+
+
+
+-- !query 4
+ALTER TABLE t ADD PARTITION (c='Us', d=1)
+-- !query 4 schema
+struct<>
+-- !query 4 output
+
+
+
+-- !query 5
+DESCRIBE t
+-- !query 5 schema
+struct
+-- !query 5 output
 # col_name data_type   comment 
 a  string  
 b  int 
 c  string  
-c  string  
 d  string  
+# Partition Information
+# col_name data_type   comment 
+c  string  
 d  string
 
 
--- !query 3
-DESC t
--- !query 3 schema
+-- !query 6
+DESC default.t
+-- !query 6 schema
 struct
--- !query 3 output
-# Partition Information
+-- !query 6 output
 # col_name data_type   comment 
 a  string  
 b  int 
 c  string  
-c  string  
 d  string  
+# Partition Information
+# col_name data_type   comment 
+c  string  
 d  string
 
 
--- !query 4
+-- !query 7
 DESC TABLE t
--- !query 4 schema
+-- !query 7 schema
 struct
--- !query 4 output
-# Partition Information
+-- !query 7 output
 # col_name data_type   comment 
 a  string  
 b  int 
 c  string  
-c  string  
 d  string  
+# Partition Information
+# col_name data_type   comment 
+c  string  
 d  string
 
 
--- !query 5
+-- !query 8
 DESC FORMATTED t
--- !query 5 schema
+-- !query 8 schema
 struct
--- !query 5 output
-# Detailed Table Information   

-# Partition Information
-#

[GitHub] spark pull request #17394: [SPARK-20067] [SQL] Unify and Clean Up Desc Comma...

2017-04-01 Thread gatorsmile 
Github user gatorsmile commented on a diff in the pull request:

https://github.com/apache/spark/pull/17394#discussion_r109300438
  
--- Diff: sql/core/src/test/resources/sql-tests/results/describe.sql.out ---
@@ -1,205 +1,259 @@
 -- Automatically generated by SQLQueryTestSuite
--- Number of queries: 14
+-- Number of queries: 31
 
 
 -- !query 0
-CREATE TABLE t (a STRING, b INT, c STRING, d STRING) USING parquet 
PARTITIONED BY (c, d) COMMENT 'table_comment'
+CREATE TABLE t (a STRING, b INT, c STRING, d STRING) USING parquet
+  PARTITIONED BY (c, d) CLUSTERED BY (a) SORTED BY (b ASC) INTO 2 BUCKETS
+  COMMENT 'table_comment'
 -- !query 0 schema
 struct<>
 -- !query 0 output
 
 
 
 -- !query 1
-ALTER TABLE t ADD PARTITION (c='Us', d=1)
+CREATE TEMPORARY VIEW temp_v AS SELECT * FROM t
 -- !query 1 schema
 struct<>
 -- !query 1 output
 
 
 
 -- !query 2
-DESCRIBE t
+CREATE TEMPORARY VIEW temp_Data_Source_View
+  USING org.apache.spark.sql.sources.DDLScanSource
+  OPTIONS (
+From '1',
+To '10',
+Table 'test1')
 -- !query 2 schema
-struct
+struct<>
 -- !query 2 output
-# Partition Information
+
+
+
+-- !query 3
+CREATE VIEW v AS SELECT * FROM t
+-- !query 3 schema
+struct<>
+-- !query 3 output
+
+
+
+-- !query 4
+ALTER TABLE t ADD PARTITION (c='Us', d=1)
+-- !query 4 schema
+struct<>
+-- !query 4 output
+
+
+
+-- !query 5
+DESCRIBE t
+-- !query 5 schema
+struct
+-- !query 5 output
 # col_name data_type   comment 
 a  string  
 b  int 
 c  string  
-c  string  
 d  string  
+# Partition Information
+# col_name data_type   comment 
+c  string  
 d  string
 
 
--- !query 3
-DESC t
--- !query 3 schema
+-- !query 6
+DESC default.t
+-- !query 6 schema
 struct
--- !query 3 output
-# Partition Information
+-- !query 6 output
 # col_name data_type   comment 
 a  string  
 b  int 
 c  string  
-c  string  
 d  string  
+# Partition Information
+# col_name data_type   comment 
+c  string  
 d  string
 
 
--- !query 4
+-- !query 7
 DESC TABLE t
--- !query 4 schema
+-- !query 7 schema
 struct
--- !query 4 output
-# Partition Information
+-- !query 7 output
 # col_name data_type   comment 
 a  string  
 b  int 
 c  string  
-c  string  
 d  string  
+# Partition Information
+# col_name data_type   comment 
+c  string  
 d  string
 
 
--- !query 5
+-- !query 8
 DESC FORMATTED t
--- !query 5 schema
+-- !query 8 schema
 struct
--- !query 5 output
-# Detailed Table Information   

-# Partition Information
-#

[GitHub] spark pull request #17394: [SPARK-20067] [SQL] Unify and Clean Up Desc Comma...

2017-04-01 Thread cloud-fan 
Github user cloud-fan commented on a diff in the pull request:

https://github.com/apache/spark/pull/17394#discussion_r109299664
  
--- Diff: sql/core/src/test/resources/sql-tests/results/describe.sql.out ---
@@ -1,205 +1,259 @@
 -- Automatically generated by SQLQueryTestSuite
--- Number of queries: 14
+-- Number of queries: 31
 
 
 -- !query 0
-CREATE TABLE t (a STRING, b INT, c STRING, d STRING) USING parquet 
PARTITIONED BY (c, d) COMMENT 'table_comment'
+CREATE TABLE t (a STRING, b INT, c STRING, d STRING) USING parquet
+  PARTITIONED BY (c, d) CLUSTERED BY (a) SORTED BY (b ASC) INTO 2 BUCKETS
+  COMMENT 'table_comment'
 -- !query 0 schema
 struct<>
 -- !query 0 output
 
 
 
 -- !query 1
-ALTER TABLE t ADD PARTITION (c='Us', d=1)
+CREATE TEMPORARY VIEW temp_v AS SELECT * FROM t
 -- !query 1 schema
 struct<>
 -- !query 1 output
 
 
 
 -- !query 2
-DESCRIBE t
+CREATE TEMPORARY VIEW temp_Data_Source_View
+  USING org.apache.spark.sql.sources.DDLScanSource
+  OPTIONS (
+From '1',
+To '10',
+Table 'test1')
 -- !query 2 schema
-struct
+struct<>
 -- !query 2 output
-# Partition Information
+
+
+
+-- !query 3
+CREATE VIEW v AS SELECT * FROM t
+-- !query 3 schema
+struct<>
+-- !query 3 output
+
+
+
+-- !query 4
+ALTER TABLE t ADD PARTITION (c='Us', d=1)
+-- !query 4 schema
+struct<>
+-- !query 4 output
+
+
+
+-- !query 5
+DESCRIBE t
+-- !query 5 schema
+struct
+-- !query 5 output
 # col_name data_type   comment 
 a  string  
 b  int 
 c  string  
-c  string  
 d  string  
+# Partition Information
+# col_name data_type   comment 
+c  string  
 d  string
 
 
--- !query 3
-DESC t
--- !query 3 schema
+-- !query 6
+DESC default.t
+-- !query 6 schema
 struct
--- !query 3 output
-# Partition Information
+-- !query 6 output
 # col_name data_type   comment 
 a  string  
 b  int 
 c  string  
-c  string  
 d  string  
+# Partition Information
+# col_name data_type   comment 
+c  string  
 d  string
 
 
--- !query 4
+-- !query 7
 DESC TABLE t
--- !query 4 schema
+-- !query 7 schema
 struct
--- !query 4 output
-# Partition Information
+-- !query 7 output
 # col_name data_type   comment 
 a  string  
 b  int 
 c  string  
-c  string  
 d  string  
+# Partition Information
+# col_name data_type   comment 
+c  string  
 d  string
 
 
--- !query 5
+-- !query 8
 DESC FORMATTED t
--- !query 5 schema
+-- !query 8 schema
 struct
--- !query 5 output
-# Detailed Table Information   

-# Partition Information
-#

[GitHub] spark pull request #17394: [SPARK-20067] [SQL] Unify and Clean Up Desc Comma...

2017-04-01 Thread cloud-fan 
Github user cloud-fan commented on a diff in the pull request:

https://github.com/apache/spark/pull/17394#discussion_r109299543
  
--- Diff: sql/core/src/test/resources/sql-tests/results/describe.sql.out ---
@@ -1,205 +1,259 @@
 -- Automatically generated by SQLQueryTestSuite
--- Number of queries: 14
+-- Number of queries: 31
 
 
 -- !query 0
-CREATE TABLE t (a STRING, b INT, c STRING, d STRING) USING parquet 
PARTITIONED BY (c, d) COMMENT 'table_comment'
+CREATE TABLE t (a STRING, b INT, c STRING, d STRING) USING parquet
+  PARTITIONED BY (c, d) CLUSTERED BY (a) SORTED BY (b ASC) INTO 2 BUCKETS
+  COMMENT 'table_comment'
 -- !query 0 schema
 struct<>
 -- !query 0 output
 
 
 
 -- !query 1
-ALTER TABLE t ADD PARTITION (c='Us', d=1)
+CREATE TEMPORARY VIEW temp_v AS SELECT * FROM t
 -- !query 1 schema
 struct<>
 -- !query 1 output
 
 
 
 -- !query 2
-DESCRIBE t
+CREATE TEMPORARY VIEW temp_Data_Source_View
+  USING org.apache.spark.sql.sources.DDLScanSource
+  OPTIONS (
+From '1',
+To '10',
+Table 'test1')
 -- !query 2 schema
-struct
+struct<>
 -- !query 2 output
-# Partition Information
+
+
+
+-- !query 3
+CREATE VIEW v AS SELECT * FROM t
+-- !query 3 schema
+struct<>
+-- !query 3 output
+
+
+
+-- !query 4
+ALTER TABLE t ADD PARTITION (c='Us', d=1)
+-- !query 4 schema
+struct<>
+-- !query 4 output
+
+
+
+-- !query 5
+DESCRIBE t
+-- !query 5 schema
+struct
+-- !query 5 output
 # col_name data_type   comment 
 a  string  
 b  int 
 c  string  
-c  string  
 d  string  
+# Partition Information
+# col_name data_type   comment 
+c  string  
 d  string
 
 
--- !query 3
-DESC t
--- !query 3 schema
+-- !query 6
+DESC default.t
+-- !query 6 schema
 struct
--- !query 3 output
-# Partition Information
+-- !query 6 output
 # col_name data_type   comment 
 a  string  
 b  int 
 c  string  
-c  string  
 d  string  
+# Partition Information
+# col_name data_type   comment 
+c  string  
 d  string
 
 
--- !query 4
+-- !query 7
 DESC TABLE t
--- !query 4 schema
+-- !query 7 schema
 struct
--- !query 4 output
-# Partition Information
+-- !query 7 output
 # col_name data_type   comment 
 a  string  
 b  int 
 c  string  
-c  string  
 d  string  
+# Partition Information
+# col_name data_type   comment 
+c  string  
 d  string
 
 
--- !query 5
+-- !query 8
 DESC FORMATTED t
--- !query 5 schema
+-- !query 8 schema
 struct
--- !query 5 output
-# Detailed Table Information   

-# Partition Information
-#

[GitHub] spark pull request #17394: [SPARK-20067] [SQL] Unify and Clean Up Desc Comma...

2017-04-01 Thread gatorsmile 
Github user gatorsmile commented on a diff in the pull request:

https://github.com/apache/spark/pull/17394#discussion_r109297793
  
--- Diff: 
sql/core/src/test/scala/org/apache/spark/sql/sources/DDLTestSuite.scala ---
@@ -1,123 +0,0 @@
-/*
-* Licensed to the Apache Software Foundation (ASF) under one or more
-* contributor license agreements.  See the NOTICE file distributed with
-* this work for additional information regarding copyright ownership.
-* The ASF licenses this file to You under the Apache License, Version 2.0
-* (the "License"); you may not use this file except in compliance with
-* the License.  You may obtain a copy of the License at
-*
-*http://www.apache.org/licenses/LICENSE-2.0
-*
-* Unless required by applicable law or agreed to in writing, software
-* distributed under the License is distributed on an "AS IS" BASIS,
-* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-* See the License for the specific language governing permissions and
-* limitations under the License.
-*/
-
-package org.apache.spark.sql.sources
-
-import org.apache.spark.rdd.RDD
-import org.apache.spark.sql._
-import org.apache.spark.sql.catalyst.InternalRow
-import org.apache.spark.sql.test.SharedSQLContext
-import org.apache.spark.sql.types._
-import org.apache.spark.unsafe.types.UTF8String
-
-class DDLScanSource extends RelationProvider {
-  override def createRelation(
-  sqlContext: SQLContext,
-  parameters: Map[String, String]): BaseRelation = {
-SimpleDDLScan(
-  parameters("from").toInt,
-  parameters("TO").toInt,
-  parameters("Table"))(sqlContext.sparkSession)
-  }
-}
-
-case class SimpleDDLScan(
--- End diff --

These two classes are moved to `DataSourceTest.scala`


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #17394: [SPARK-20067] [SQL] Unify and Clean Up Desc Comma...

2017-04-01 Thread gatorsmile 
Github user gatorsmile commented on a diff in the pull request:

https://github.com/apache/spark/pull/17394#discussion_r109289677
  
--- Diff: sql/core/src/test/resources/sql-tests/results/describe.sql.out ---
@@ -1,205 +1,248 @@
 -- Automatically generated by SQLQueryTestSuite
--- Number of queries: 14
+-- Number of queries: 28
 
 
 -- !query 0
-CREATE TABLE t (a STRING, b INT, c STRING, d STRING) USING parquet 
PARTITIONED BY (c, d) COMMENT 'table_comment'
+CREATE TABLE t (a STRING, b INT, c STRING, d STRING) USING parquet
+  PARTITIONED BY (c, d) CLUSTERED BY (a) SORTED BY (b ASC) INTO 2 BUCKETS
+  COMMENT 'table_comment'
 -- !query 0 schema
 struct<>
 -- !query 0 output
 
 
 
 -- !query 1
-ALTER TABLE t ADD PARTITION (c='Us', d=1)
+CREATE TEMPORARY VIEW temp_v AS SELECT * FROM t
 -- !query 1 schema
 struct<>
 -- !query 1 output
 
 
 
 -- !query 2
-DESCRIBE t
+CREATE VIEW v AS SELECT * FROM t
 -- !query 2 schema
-struct
+struct<>
 -- !query 2 output
-# Partition Information
-# col_name data_type   comment 
-a  string  
-b  int 
-c  string  
-c  string  
-d  string  
-d  string
+
 
 
 -- !query 3
-DESC t
+ALTER TABLE t ADD PARTITION (c='Us', d=1)
 -- !query 3 schema
-struct
+struct<>
 -- !query 3 output
-# Partition Information
-# col_name data_type   comment 
-a  string  
-b  int 
-c  string  
-c  string  
-d  string  
-d  string
+
 
 
 -- !query 4
-DESC TABLE t
+DESCRIBE t
 -- !query 4 schema
 struct
 -- !query 4 output
-# Partition Information
-# col_name data_type   comment 
 a  string  
 b  int 
 c  string  
-c  string  
 d  string  
+# Partition Information
+# col_name data_type   comment 
+c  string  
 d  string
 
 
 -- !query 5
-DESC FORMATTED t
+DESC t
 -- !query 5 schema
 struct
 -- !query 5 output
-# Detailed Table Information   

-# Partition Information
-# Storage Information  
-# col_name data_type   comment 
-Comment:   table_comment   
-Compressed:No  
-Created: 
-Database:  default 
-Last Access: 
-Location: sql/core/spark-warehouse/t   
-Owner: 
-Partition Provider:Catalog 
-Storage Desc Parameters:   
-Table Parameters:  
-Table Type:MANAGED 
 a  string  
 b  int 
 c  string  
-c  string

[GitHub] spark pull request #17394: [SPARK-20067] [SQL] Unify and Clean Up Desc Comma...

2017-04-01 Thread gatorsmile 
Github user gatorsmile commented on a diff in the pull request:

https://github.com/apache/spark/pull/17394#discussion_r109289263
  
--- Diff: 
sql/core/src/main/scala/org/apache/spark/sql/execution/command/tables.scala ---
@@ -637,21 +570,7 @@ case class DescribeTableCommand(
 }
 DDLUtils.verifyPartitionProviderIsHive(spark, metadata, "DESC 
PARTITION")
 val partition = catalog.getPartition(table, partitionSpec)
-if (isExtended) {
-  describeExtendedDetailedPartitionInfo(table, metadata, partition, 
result)
-} else if (isFormatted) {
-  describeFormattedDetailedPartitionInfo(table, metadata, partition, 
result)
-  describeStorageInfo(metadata, result)
-}
-  }
-
-  private def describeExtendedDetailedPartitionInfo(
-  tableIdentifier: TableIdentifier,
-  table: CatalogTable,
-  partition: CatalogTablePartition,
-  buffer: ArrayBuffer[Row]): Unit = {
-append(buffer, "", "", "")
-append(buffer, "Detailed Partition Information " + partition.toString, 
"", "")
+if (isExtended) describeFormattedDetailedPartitionInfo(table, 
metadata, partition, result)
--- End diff --

This function `describeDetailedPartitionInfo ` will only be called for the 
DDL command
```SQL
  DESCRIBE [EXTENDED|FORMATTED] table_name PARTITION (partitionVal*)
```


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #17394: [SPARK-20067] [SQL] Unify and Clean Up Desc Comma...

2017-04-01 Thread gatorsmile 
Github user gatorsmile commented on a diff in the pull request:

https://github.com/apache/spark/pull/17394#discussion_r109289119
  
--- Diff: sql/core/src/test/resources/sql-tests/results/describe.sql.out ---
@@ -1,205 +1,248 @@
 -- Automatically generated by SQLQueryTestSuite
--- Number of queries: 14
+-- Number of queries: 28
 
 
 -- !query 0
-CREATE TABLE t (a STRING, b INT, c STRING, d STRING) USING parquet 
PARTITIONED BY (c, d) COMMENT 'table_comment'
+CREATE TABLE t (a STRING, b INT, c STRING, d STRING) USING parquet
+  PARTITIONED BY (c, d) CLUSTERED BY (a) SORTED BY (b ASC) INTO 2 BUCKETS
+  COMMENT 'table_comment'
 -- !query 0 schema
 struct<>
 -- !query 0 output
 
 
 
 -- !query 1
-ALTER TABLE t ADD PARTITION (c='Us', d=1)
+CREATE TEMPORARY VIEW temp_v AS SELECT * FROM t
 -- !query 1 schema
 struct<>
 -- !query 1 output
 
 
 
 -- !query 2
-DESCRIBE t
+CREATE VIEW v AS SELECT * FROM t
 -- !query 2 schema
-struct
+struct<>
 -- !query 2 output
-# Partition Information
-# col_name data_type   comment 
-a  string  
-b  int 
-c  string  
-c  string  
-d  string  
-d  string
+
 
 
 -- !query 3
-DESC t
+ALTER TABLE t ADD PARTITION (c='Us', d=1)
 -- !query 3 schema
-struct
+struct<>
 -- !query 3 output
-# Partition Information
-# col_name data_type   comment 
-a  string  
-b  int 
-c  string  
-c  string  
-d  string  
-d  string
+
 
 
 -- !query 4
-DESC TABLE t
+DESCRIBE t
 -- !query 4 schema
 struct
 -- !query 4 output
-# Partition Information
-# col_name data_type   comment 
 a  string  
 b  int 
 c  string  
-c  string  
 d  string  
+# Partition Information
+# col_name data_type   comment 
+c  string  
 d  string
 
 
 -- !query 5
-DESC FORMATTED t
+DESC t
 -- !query 5 schema
 struct
 -- !query 5 output
-# Detailed Table Information   

-# Partition Information
-# Storage Information  
-# col_name data_type   comment 
-Comment:   table_comment   
-Compressed:No  
-Created: 
-Database:  default 
-Last Access: 
-Location: sql/core/spark-warehouse/t   
-Owner: 
-Partition Provider:Catalog 
-Storage Desc Parameters:   
-Table Parameters:  
-Table Type:MANAGED 
 a  string  
 b  int 
 c  string  
-c  string

[GitHub] spark pull request #17394: [SPARK-20067] [SQL] Unify and Clean Up Desc Comma...

2017-04-01 Thread gatorsmile 
Github user gatorsmile commented on a diff in the pull request:

https://github.com/apache/spark/pull/17394#discussion_r109289117
  
--- Diff: sql/core/src/test/resources/sql-tests/results/describe.sql.out ---
@@ -1,205 +1,248 @@
 -- Automatically generated by SQLQueryTestSuite
--- Number of queries: 14
+-- Number of queries: 28
 
 
 -- !query 0
-CREATE TABLE t (a STRING, b INT, c STRING, d STRING) USING parquet 
PARTITIONED BY (c, d) COMMENT 'table_comment'
+CREATE TABLE t (a STRING, b INT, c STRING, d STRING) USING parquet
+  PARTITIONED BY (c, d) CLUSTERED BY (a) SORTED BY (b ASC) INTO 2 BUCKETS
+  COMMENT 'table_comment'
 -- !query 0 schema
 struct<>
 -- !query 0 output
 
 
 
 -- !query 1
-ALTER TABLE t ADD PARTITION (c='Us', d=1)
+CREATE TEMPORARY VIEW temp_v AS SELECT * FROM t
 -- !query 1 schema
 struct<>
 -- !query 1 output
 
 
 
 -- !query 2
-DESCRIBE t
+CREATE VIEW v AS SELECT * FROM t
 -- !query 2 schema
-struct
+struct<>
 -- !query 2 output
-# Partition Information
-# col_name data_type   comment 
-a  string  
-b  int 
-c  string  
-c  string  
-d  string  
-d  string
+
 
 
 -- !query 3
-DESC t
+ALTER TABLE t ADD PARTITION (c='Us', d=1)
 -- !query 3 schema
-struct
+struct<>
 -- !query 3 output
-# Partition Information
-# col_name data_type   comment 
-a  string  
-b  int 
-c  string  
-c  string  
-d  string  
-d  string
+
 
 
 -- !query 4
-DESC TABLE t
+DESCRIBE t
 -- !query 4 schema
 struct
 -- !query 4 output
-# Partition Information
-# col_name data_type   comment 
 a  string  
 b  int 
 c  string  
-c  string  
 d  string  
+# Partition Information
+# col_name data_type   comment 
+c  string  
 d  string
 
 
 -- !query 5
-DESC FORMATTED t
+DESC t
 -- !query 5 schema
 struct
 -- !query 5 output
-# Detailed Table Information   

-# Partition Information
-# Storage Information  
-# col_name data_type   comment 
-Comment:   table_comment   
-Compressed:No  
-Created: 
-Database:  default 
-Last Access: 
-Location: sql/core/spark-warehouse/t   
-Owner: 
-Partition Provider:Catalog 
-Storage Desc Parameters:   
-Table Parameters:  
-Table Type:MANAGED 
 a  string  
 b  int 
 c  string  
-c  string

[GitHub] spark pull request #17394: [SPARK-20067] [SQL] Unify and Clean Up Desc Comma...

2017-04-01 Thread gatorsmile 
Github user gatorsmile commented on a diff in the pull request:

https://github.com/apache/spark/pull/17394#discussion_r109289114
  
--- Diff: sql/core/src/test/resources/sql-tests/results/describe.sql.out ---
@@ -1,205 +1,248 @@
 -- Automatically generated by SQLQueryTestSuite
--- Number of queries: 14
+-- Number of queries: 28
 
 
 -- !query 0
-CREATE TABLE t (a STRING, b INT, c STRING, d STRING) USING parquet 
PARTITIONED BY (c, d) COMMENT 'table_comment'
+CREATE TABLE t (a STRING, b INT, c STRING, d STRING) USING parquet
+  PARTITIONED BY (c, d) CLUSTERED BY (a) SORTED BY (b ASC) INTO 2 BUCKETS
+  COMMENT 'table_comment'
 -- !query 0 schema
 struct<>
 -- !query 0 output
 
 
 
 -- !query 1
-ALTER TABLE t ADD PARTITION (c='Us', d=1)
+CREATE TEMPORARY VIEW temp_v AS SELECT * FROM t
 -- !query 1 schema
 struct<>
 -- !query 1 output
 
 
 
 -- !query 2
-DESCRIBE t
+CREATE VIEW v AS SELECT * FROM t
 -- !query 2 schema
-struct
+struct<>
 -- !query 2 output
-# Partition Information
-# col_name data_type   comment 
-a  string  
-b  int 
-c  string  
-c  string  
-d  string  
-d  string
+
 
 
 -- !query 3
-DESC t
+ALTER TABLE t ADD PARTITION (c='Us', d=1)
 -- !query 3 schema
-struct
+struct<>
 -- !query 3 output
-# Partition Information
-# col_name data_type   comment 
-a  string  
-b  int 
-c  string  
-c  string  
-d  string  
-d  string
+
 
 
 -- !query 4
-DESC TABLE t
+DESCRIBE t
 -- !query 4 schema
 struct
 -- !query 4 output
-# Partition Information
-# col_name data_type   comment 
 a  string  
 b  int 
 c  string  
-c  string  
 d  string  
+# Partition Information
+# col_name data_type   comment 
+c  string  
 d  string
 
 
 -- !query 5
-DESC FORMATTED t
+DESC t
 -- !query 5 schema
 struct
 -- !query 5 output
-# Detailed Table Information   

-# Partition Information
-# Storage Information  
-# col_name data_type   comment 
-Comment:   table_comment   
-Compressed:No  
-Created: 
-Database:  default 
-Last Access: 
-Location: sql/core/spark-warehouse/t   
-Owner: 
-Partition Provider:Catalog 
-Storage Desc Parameters:   
-Table Parameters:  
-Table Type:MANAGED 
 a  string  
 b  int 
 c  string  
-c  string

[GitHub] spark pull request #17394: [SPARK-20067] [SQL] Unify and Clean Up Desc Comma...

2017-04-01 Thread gatorsmile 
Github user gatorsmile commented on a diff in the pull request:

https://github.com/apache/spark/pull/17394#discussion_r109289107
  
--- Diff: sql/core/src/test/resources/sql-tests/results/describe.sql.out ---
@@ -1,205 +1,248 @@
 -- Automatically generated by SQLQueryTestSuite
--- Number of queries: 14
+-- Number of queries: 28
 
 
 -- !query 0
-CREATE TABLE t (a STRING, b INT, c STRING, d STRING) USING parquet 
PARTITIONED BY (c, d) COMMENT 'table_comment'
+CREATE TABLE t (a STRING, b INT, c STRING, d STRING) USING parquet
+  PARTITIONED BY (c, d) CLUSTERED BY (a) SORTED BY (b ASC) INTO 2 BUCKETS
+  COMMENT 'table_comment'
 -- !query 0 schema
 struct<>
 -- !query 0 output
 
 
 
 -- !query 1
-ALTER TABLE t ADD PARTITION (c='Us', d=1)
+CREATE TEMPORARY VIEW temp_v AS SELECT * FROM t
 -- !query 1 schema
 struct<>
 -- !query 1 output
 
 
 
 -- !query 2
-DESCRIBE t
+CREATE VIEW v AS SELECT * FROM t
 -- !query 2 schema
-struct
+struct<>
 -- !query 2 output
-# Partition Information
-# col_name data_type   comment 
-a  string  
-b  int 
-c  string  
-c  string  
-d  string  
-d  string
+
 
 
 -- !query 3
-DESC t
+ALTER TABLE t ADD PARTITION (c='Us', d=1)
 -- !query 3 schema
-struct
+struct<>
 -- !query 3 output
-# Partition Information
-# col_name data_type   comment 
-a  string  
-b  int 
-c  string  
-c  string  
-d  string  
-d  string
+
 
 
 -- !query 4
-DESC TABLE t
+DESCRIBE t
 -- !query 4 schema
 struct
 -- !query 4 output
-# Partition Information
-# col_name data_type   comment 
 a  string  
 b  int 
 c  string  
-c  string  
 d  string  
+# Partition Information
+# col_name data_type   comment 
+c  string  
 d  string
 
 
 -- !query 5
-DESC FORMATTED t
+DESC t
 -- !query 5 schema
 struct
 -- !query 5 output
-# Detailed Table Information   

-# Partition Information
-# Storage Information  
-# col_name data_type   comment 
-Comment:   table_comment   
-Compressed:No  
-Created: 
-Database:  default 
-Last Access: 
-Location: sql/core/spark-warehouse/t   
-Owner: 
-Partition Provider:Catalog 
-Storage Desc Parameters:   
-Table Parameters:  
-Table Type:MANAGED 
 a  string  
 b  int 
 c  string  
-c  string

[GitHub] spark pull request #17394: [SPARK-20067] [SQL] Unify and Clean Up Desc Comma...

2017-04-01 Thread cloud-fan 
Github user cloud-fan commented on a diff in the pull request:

https://github.com/apache/spark/pull/17394#discussion_r109285879
  
--- Diff: sql/core/src/test/resources/sql-tests/results/describe.sql.out ---
@@ -1,205 +1,248 @@
 -- Automatically generated by SQLQueryTestSuite
--- Number of queries: 14
+-- Number of queries: 28
 
 
 -- !query 0
-CREATE TABLE t (a STRING, b INT, c STRING, d STRING) USING parquet 
PARTITIONED BY (c, d) COMMENT 'table_comment'
+CREATE TABLE t (a STRING, b INT, c STRING, d STRING) USING parquet
+  PARTITIONED BY (c, d) CLUSTERED BY (a) SORTED BY (b ASC) INTO 2 BUCKETS
+  COMMENT 'table_comment'
 -- !query 0 schema
 struct<>
 -- !query 0 output
 
 
 
 -- !query 1
-ALTER TABLE t ADD PARTITION (c='Us', d=1)
+CREATE TEMPORARY VIEW temp_v AS SELECT * FROM t
 -- !query 1 schema
 struct<>
 -- !query 1 output
 
 
 
 -- !query 2
-DESCRIBE t
+CREATE VIEW v AS SELECT * FROM t
 -- !query 2 schema
-struct
+struct<>
 -- !query 2 output
-# Partition Information
-# col_name data_type   comment 
-a  string  
-b  int 
-c  string  
-c  string  
-d  string  
-d  string
+
 
 
 -- !query 3
-DESC t
+ALTER TABLE t ADD PARTITION (c='Us', d=1)
 -- !query 3 schema
-struct
+struct<>
 -- !query 3 output
-# Partition Information
-# col_name data_type   comment 
-a  string  
-b  int 
-c  string  
-c  string  
-d  string  
-d  string
+
 
 
 -- !query 4
-DESC TABLE t
+DESCRIBE t
 -- !query 4 schema
 struct
 -- !query 4 output
-# Partition Information
-# col_name data_type   comment 
 a  string  
 b  int 
 c  string  
-c  string  
 d  string  
+# Partition Information
+# col_name data_type   comment 
+c  string  
 d  string
 
 
 -- !query 5
-DESC FORMATTED t
+DESC t
 -- !query 5 schema
 struct
 -- !query 5 output
-# Detailed Table Information   

-# Partition Information
-# Storage Information  
-# col_name data_type   comment 
-Comment:   table_comment   
-Compressed:No  
-Created: 
-Database:  default 
-Last Access: 
-Location: sql/core/spark-warehouse/t   
-Owner: 
-Partition Provider:Catalog 
-Storage Desc Parameters:   
-Table Parameters:  
-Table Type:MANAGED 
 a  string  
 b  int 
 c  string  
-c  string

[GitHub] spark pull request #17394: [SPARK-20067] [SQL] Unify and Clean Up Desc Comma...

2017-04-01 Thread cloud-fan 
Github user cloud-fan commented on a diff in the pull request:

https://github.com/apache/spark/pull/17394#discussion_r109285811
  
--- Diff: 
sql/core/src/main/scala/org/apache/spark/sql/execution/command/tables.scala ---
@@ -637,21 +570,7 @@ case class DescribeTableCommand(
 }
 DDLUtils.verifyPartitionProviderIsHive(spark, metadata, "DESC 
PARTITION")
 val partition = catalog.getPartition(table, partitionSpec)
-if (isExtended) {
-  describeExtendedDetailedPartitionInfo(table, metadata, partition, 
result)
-} else if (isFormatted) {
-  describeFormattedDetailedPartitionInfo(table, metadata, partition, 
result)
-  describeStorageInfo(metadata, result)
-}
-  }
-
-  private def describeExtendedDetailedPartitionInfo(
-  tableIdentifier: TableIdentifier,
-  table: CatalogTable,
-  partition: CatalogTablePartition,
-  buffer: ArrayBuffer[Row]): Unit = {
-append(buffer, "", "", "")
-append(buffer, "Detailed Partition Information " + partition.toString, 
"", "")
+if (isExtended) describeFormattedDetailedPartitionInfo(table, 
metadata, partition, result)
--- End diff --

not related to this PR, but it looks weird that `DESC tbl` and `DESC tbl 
PARTITION (xxx)` has the same result.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #17394: [SPARK-20067] [SQL] Unify and Clean Up Desc Comma...

2017-04-01 Thread cloud-fan 
Github user cloud-fan commented on a diff in the pull request:

https://github.com/apache/spark/pull/17394#discussion_r109285540
  
--- Diff: sql/core/src/test/resources/sql-tests/results/describe.sql.out ---
@@ -1,205 +1,248 @@
 -- Automatically generated by SQLQueryTestSuite
--- Number of queries: 14
+-- Number of queries: 28
 
 
 -- !query 0
-CREATE TABLE t (a STRING, b INT, c STRING, d STRING) USING parquet 
PARTITIONED BY (c, d) COMMENT 'table_comment'
+CREATE TABLE t (a STRING, b INT, c STRING, d STRING) USING parquet
+  PARTITIONED BY (c, d) CLUSTERED BY (a) SORTED BY (b ASC) INTO 2 BUCKETS
+  COMMENT 'table_comment'
 -- !query 0 schema
 struct<>
 -- !query 0 output
 
 
 
 -- !query 1
-ALTER TABLE t ADD PARTITION (c='Us', d=1)
+CREATE TEMPORARY VIEW temp_v AS SELECT * FROM t
 -- !query 1 schema
 struct<>
 -- !query 1 output
 
 
 
 -- !query 2
-DESCRIBE t
+CREATE VIEW v AS SELECT * FROM t
 -- !query 2 schema
-struct
+struct<>
 -- !query 2 output
-# Partition Information
-# col_name data_type   comment 
-a  string  
-b  int 
-c  string  
-c  string  
-d  string  
-d  string
+
 
 
 -- !query 3
-DESC t
+ALTER TABLE t ADD PARTITION (c='Us', d=1)
 -- !query 3 schema
-struct
+struct<>
 -- !query 3 output
-# Partition Information
-# col_name data_type   comment 
-a  string  
-b  int 
-c  string  
-c  string  
-d  string  
-d  string
+
 
 
 -- !query 4
-DESC TABLE t
+DESCRIBE t
 -- !query 4 schema
 struct
 -- !query 4 output
-# Partition Information
-# col_name data_type   comment 
 a  string  
 b  int 
 c  string  
-c  string  
 d  string  
+# Partition Information
+# col_name data_type   comment 
+c  string  
 d  string
 
 
 -- !query 5
-DESC FORMATTED t
+DESC t
 -- !query 5 schema
 struct
 -- !query 5 output
-# Detailed Table Information   

-# Partition Information
-# Storage Information  
-# col_name data_type   comment 
-Comment:   table_comment   
-Compressed:No  
-Created: 
-Database:  default 
-Last Access: 
-Location: sql/core/spark-warehouse/t   
-Owner: 
-Partition Provider:Catalog 
-Storage Desc Parameters:   
-Table Parameters:  
-Table Type:MANAGED 
 a  string  
 b  int 
 c  string  
-c  string

[GitHub] spark pull request #17394: [SPARK-20067] [SQL] Unify and Clean Up Desc Comma...

2017-04-01 Thread cloud-fan 
Github user cloud-fan commented on a diff in the pull request:

https://github.com/apache/spark/pull/17394#discussion_r109285423
  
--- Diff: sql/core/src/test/resources/sql-tests/results/describe.sql.out ---
@@ -1,205 +1,248 @@
 -- Automatically generated by SQLQueryTestSuite
--- Number of queries: 14
+-- Number of queries: 28
 
 
 -- !query 0
-CREATE TABLE t (a STRING, b INT, c STRING, d STRING) USING parquet 
PARTITIONED BY (c, d) COMMENT 'table_comment'
+CREATE TABLE t (a STRING, b INT, c STRING, d STRING) USING parquet
+  PARTITIONED BY (c, d) CLUSTERED BY (a) SORTED BY (b ASC) INTO 2 BUCKETS
+  COMMENT 'table_comment'
 -- !query 0 schema
 struct<>
 -- !query 0 output
 
 
 
 -- !query 1
-ALTER TABLE t ADD PARTITION (c='Us', d=1)
+CREATE TEMPORARY VIEW temp_v AS SELECT * FROM t
 -- !query 1 schema
 struct<>
 -- !query 1 output
 
 
 
 -- !query 2
-DESCRIBE t
+CREATE VIEW v AS SELECT * FROM t
 -- !query 2 schema
-struct
+struct<>
 -- !query 2 output
-# Partition Information
-# col_name data_type   comment 
-a  string  
-b  int 
-c  string  
-c  string  
-d  string  
-d  string
+
 
 
 -- !query 3
-DESC t
+ALTER TABLE t ADD PARTITION (c='Us', d=1)
 -- !query 3 schema
-struct
+struct<>
 -- !query 3 output
-# Partition Information
-# col_name data_type   comment 
-a  string  
-b  int 
-c  string  
-c  string  
-d  string  
-d  string
+
 
 
 -- !query 4
-DESC TABLE t
+DESCRIBE t
 -- !query 4 schema
 struct
 -- !query 4 output
-# Partition Information
-# col_name data_type   comment 
 a  string  
 b  int 
 c  string  
-c  string  
 d  string  
+# Partition Information
+# col_name data_type   comment 
+c  string  
 d  string
 
 
 -- !query 5
-DESC FORMATTED t
+DESC t
 -- !query 5 schema
 struct
 -- !query 5 output
-# Detailed Table Information   

-# Partition Information
-# Storage Information  
-# col_name data_type   comment 
-Comment:   table_comment   
-Compressed:No  
-Created: 
-Database:  default 
-Last Access: 
-Location: sql/core/spark-warehouse/t   
-Owner: 
-Partition Provider:Catalog 
-Storage Desc Parameters:   
-Table Parameters:  
-Table Type:MANAGED 
 a  string  
 b  int 
 c  string  
-c  string

[GitHub] spark pull request #17394: [SPARK-20067] [SQL] Unify and Clean Up Desc Comma...

2017-04-01 Thread cloud-fan 
Github user cloud-fan commented on a diff in the pull request:

https://github.com/apache/spark/pull/17394#discussion_r109285410
  
--- Diff: sql/core/src/test/resources/sql-tests/results/describe.sql.out ---
@@ -1,205 +1,248 @@
 -- Automatically generated by SQLQueryTestSuite
--- Number of queries: 14
+-- Number of queries: 28
 
 
 -- !query 0
-CREATE TABLE t (a STRING, b INT, c STRING, d STRING) USING parquet 
PARTITIONED BY (c, d) COMMENT 'table_comment'
+CREATE TABLE t (a STRING, b INT, c STRING, d STRING) USING parquet
+  PARTITIONED BY (c, d) CLUSTERED BY (a) SORTED BY (b ASC) INTO 2 BUCKETS
+  COMMENT 'table_comment'
 -- !query 0 schema
 struct<>
 -- !query 0 output
 
 
 
 -- !query 1
-ALTER TABLE t ADD PARTITION (c='Us', d=1)
+CREATE TEMPORARY VIEW temp_v AS SELECT * FROM t
 -- !query 1 schema
 struct<>
 -- !query 1 output
 
 
 
 -- !query 2
-DESCRIBE t
+CREATE VIEW v AS SELECT * FROM t
 -- !query 2 schema
-struct
+struct<>
 -- !query 2 output
-# Partition Information
-# col_name data_type   comment 
-a  string  
-b  int 
-c  string  
-c  string  
-d  string  
-d  string
+
 
 
 -- !query 3
-DESC t
+ALTER TABLE t ADD PARTITION (c='Us', d=1)
 -- !query 3 schema
-struct
+struct<>
 -- !query 3 output
-# Partition Information
-# col_name data_type   comment 
-a  string  
-b  int 
-c  string  
-c  string  
-d  string  
-d  string
+
 
 
 -- !query 4
-DESC TABLE t
+DESCRIBE t
 -- !query 4 schema
 struct
 -- !query 4 output
-# Partition Information
-# col_name data_type   comment 
 a  string  
 b  int 
 c  string  
-c  string  
 d  string  
+# Partition Information
+# col_name data_type   comment 
+c  string  
 d  string
 
 
 -- !query 5
-DESC FORMATTED t
+DESC t
 -- !query 5 schema
 struct
 -- !query 5 output
-# Detailed Table Information   

-# Partition Information
-# Storage Information  
-# col_name data_type   comment 
-Comment:   table_comment   
-Compressed:No  
-Created: 
-Database:  default 
-Last Access: 
-Location: sql/core/spark-warehouse/t   
-Owner: 
-Partition Provider:Catalog 
-Storage Desc Parameters:   
-Table Parameters:  
-Table Type:MANAGED 
 a  string  
 b  int 
 c  string  
-c  string

[GitHub] spark pull request #17394: [SPARK-20067] [SQL] Unify and Clean Up Desc Comma...

2017-04-01 Thread cloud-fan 
Github user cloud-fan commented on a diff in the pull request:

https://github.com/apache/spark/pull/17394#discussion_r109285396
  
--- Diff: sql/core/src/test/resources/sql-tests/results/describe.sql.out ---
@@ -1,205 +1,248 @@
 -- Automatically generated by SQLQueryTestSuite
--- Number of queries: 14
+-- Number of queries: 28
 
 
 -- !query 0
-CREATE TABLE t (a STRING, b INT, c STRING, d STRING) USING parquet 
PARTITIONED BY (c, d) COMMENT 'table_comment'
+CREATE TABLE t (a STRING, b INT, c STRING, d STRING) USING parquet
+  PARTITIONED BY (c, d) CLUSTERED BY (a) SORTED BY (b ASC) INTO 2 BUCKETS
+  COMMENT 'table_comment'
 -- !query 0 schema
 struct<>
 -- !query 0 output
 
 
 
 -- !query 1
-ALTER TABLE t ADD PARTITION (c='Us', d=1)
+CREATE TEMPORARY VIEW temp_v AS SELECT * FROM t
 -- !query 1 schema
 struct<>
 -- !query 1 output
 
 
 
 -- !query 2
-DESCRIBE t
+CREATE VIEW v AS SELECT * FROM t
 -- !query 2 schema
-struct
+struct<>
 -- !query 2 output
-# Partition Information
-# col_name data_type   comment 
-a  string  
-b  int 
-c  string  
-c  string  
-d  string  
-d  string
+
 
 
 -- !query 3
-DESC t
+ALTER TABLE t ADD PARTITION (c='Us', d=1)
 -- !query 3 schema
-struct
+struct<>
 -- !query 3 output
-# Partition Information
-# col_name data_type   comment 
-a  string  
-b  int 
-c  string  
-c  string  
-d  string  
-d  string
+
 
 
 -- !query 4
-DESC TABLE t
+DESCRIBE t
 -- !query 4 schema
 struct
 -- !query 4 output
-# Partition Information
-# col_name data_type   comment 
 a  string  
 b  int 
 c  string  
-c  string  
 d  string  
+# Partition Information
+# col_name data_type   comment 
+c  string  
 d  string
 
 
 -- !query 5
-DESC FORMATTED t
+DESC t
 -- !query 5 schema
 struct
 -- !query 5 output
-# Detailed Table Information   

-# Partition Information
-# Storage Information  
-# col_name data_type   comment 
-Comment:   table_comment   
-Compressed:No  
-Created: 
-Database:  default 
-Last Access: 
-Location: sql/core/spark-warehouse/t   
-Owner: 
-Partition Provider:Catalog 
-Storage Desc Parameters:   
-Table Parameters:  
-Table Type:MANAGED 
 a  string  
 b  int 
 c  string  
-c  string

[GitHub] spark pull request #17394: [SPARK-20067] [SQL] Unify and Clean Up Desc Comma...

2017-04-01 Thread cloud-fan 
Github user cloud-fan commented on a diff in the pull request:

https://github.com/apache/spark/pull/17394#discussion_r109285367
  
--- Diff: sql/core/src/test/resources/sql-tests/results/describe.sql.out ---
@@ -1,205 +1,248 @@
 -- Automatically generated by SQLQueryTestSuite
--- Number of queries: 14
+-- Number of queries: 28
 
 
 -- !query 0
-CREATE TABLE t (a STRING, b INT, c STRING, d STRING) USING parquet 
PARTITIONED BY (c, d) COMMENT 'table_comment'
+CREATE TABLE t (a STRING, b INT, c STRING, d STRING) USING parquet
+  PARTITIONED BY (c, d) CLUSTERED BY (a) SORTED BY (b ASC) INTO 2 BUCKETS
+  COMMENT 'table_comment'
 -- !query 0 schema
 struct<>
 -- !query 0 output
 
 
 
 -- !query 1
-ALTER TABLE t ADD PARTITION (c='Us', d=1)
+CREATE TEMPORARY VIEW temp_v AS SELECT * FROM t
 -- !query 1 schema
 struct<>
 -- !query 1 output
 
 
 
 -- !query 2
-DESCRIBE t
+CREATE VIEW v AS SELECT * FROM t
 -- !query 2 schema
-struct
+struct<>
 -- !query 2 output
-# Partition Information
-# col_name data_type   comment 
-a  string  
-b  int 
-c  string  
-c  string  
-d  string  
-d  string
+
 
 
 -- !query 3
-DESC t
+ALTER TABLE t ADD PARTITION (c='Us', d=1)
 -- !query 3 schema
-struct
+struct<>
 -- !query 3 output
-# Partition Information
-# col_name data_type   comment 
-a  string  
-b  int 
-c  string  
-c  string  
-d  string  
-d  string
+
 
 
 -- !query 4
-DESC TABLE t
+DESCRIBE t
 -- !query 4 schema
 struct
 -- !query 4 output
-# Partition Information
-# col_name data_type   comment 
 a  string  
 b  int 
 c  string  
-c  string  
 d  string  
+# Partition Information
+# col_name data_type   comment 
+c  string  
 d  string
 
 
 -- !query 5
-DESC FORMATTED t
+DESC t
 -- !query 5 schema
 struct
 -- !query 5 output
-# Detailed Table Information   

-# Partition Information
-# Storage Information  
-# col_name data_type   comment 
-Comment:   table_comment   
-Compressed:No  
-Created: 
-Database:  default 
-Last Access: 
-Location: sql/core/spark-warehouse/t   
-Owner: 
-Partition Provider:Catalog 
-Storage Desc Parameters:   
-Table Parameters:  
-Table Type:MANAGED 
 a  string  
 b  int 
 c  string  
-c  string

[GitHub] spark pull request #17394: [SPARK-20067] [SQL] Unify and Clean Up Desc Comma...

2017-04-01 Thread cloud-fan 
Github user cloud-fan commented on a diff in the pull request:

https://github.com/apache/spark/pull/17394#discussion_r109285298
  
--- Diff: 
sql/core/src/test/scala/org/apache/spark/sql/SQLQueryTestSuite.scala ---
@@ -214,6 +215,7 @@ class SQLQueryTestSuite extends QueryTest with 
SharedSQLContext {
 // Returns true if the plan is supposed to be sorted.
 def isSorted(plan: LogicalPlan): Boolean = plan match {
--- End diff --

maybe call it `needSort`?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #17394: [SPARK-20067] [SQL] Unify and Clean Up Desc Comma...

2017-03-31 Thread gatorsmile 
Github user gatorsmile commented on a diff in the pull request:

https://github.com/apache/spark/pull/17394#discussion_r109265858
  
--- Diff: 
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/interface.scala
 ---
@@ -91,15 +98,27 @@ case class CatalogTablePartition(
 storage: CatalogStorageFormat,
 parameters: Map[String, String] = Map.empty) {
 
-  override def toString: String = {
+  def toLinkedHashMap: mutable.LinkedHashMap[String, String] = {
+val map = new mutable.LinkedHashMap[String, String]()
 val specString = spec.map { case (k, v) => s"$k=$v" }.mkString(", ")
-val output =
-  Seq(
-s"Partition Values: [$specString]",
-s"$storage",
-s"Partition Parameters:{${parameters.map(p => p._1 + "=" + 
p._2).mkString(", ")}}")
+map.put("Partition Values", s"[$specString]")
+map ++= storage.toLinkedHashMap
+if (parameters.nonEmpty) {
+  map.put("Partition Parameters", s"{${parameters.map(p => p._1 + "=" 
+ p._2).mkString(", ")}}")
+}
+map
+  }
 
-output.filter(_.nonEmpty).mkString("CatalogPartition(\n\t", "\n\t", 
")")
+  override def toString: String = {
+toLinkedHashMap.map { case ((key, value)) =>
+  if (value.isEmpty) key else s"$key: $value"
+}.mkString("CatalogPartition(\n\t", "\n\t", ")")
+  }
+
+  def simpleString: String = {
--- End diff --

`Show Table` command will call it.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #17394: [SPARK-20067] [SQL] Unify and Clean Up Desc Comma...

2017-03-31 Thread gatorsmile 
Github user gatorsmile commented on a diff in the pull request:

https://github.com/apache/spark/pull/17394#discussion_r109265870
  
--- Diff: 
sql/core/src/main/scala/org/apache/spark/sql/execution/command/tables.scala ---
@@ -500,7 +500,6 @@ case class TruncateTableCommand(
 case class DescribeTableCommand(
 table: TableIdentifier,
 partitionSpec: TablePartitionSpec,
-isExtended: Boolean,
--- End diff --

Sure, let me make the change.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #17394: [SPARK-20067] [SQL] Unify and Clean Up Desc Comma...

2017-03-31 Thread cloud-fan 
Github user cloud-fan commented on a diff in the pull request:

https://github.com/apache/spark/pull/17394#discussion_r109205777
  
--- Diff: 
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/interface.scala
 ---
@@ -91,15 +98,27 @@ case class CatalogTablePartition(
 storage: CatalogStorageFormat,
 parameters: Map[String, String] = Map.empty) {
 
-  override def toString: String = {
+  def toLinkedHashMap: mutable.LinkedHashMap[String, String] = {
+val map = new mutable.LinkedHashMap[String, String]()
 val specString = spec.map { case (k, v) => s"$k=$v" }.mkString(", ")
-val output =
-  Seq(
-s"Partition Values: [$specString]",
-s"$storage",
-s"Partition Parameters:{${parameters.map(p => p._1 + "=" + 
p._2).mkString(", ")}}")
+map.put("Partition Values", s"[$specString]")
+map ++= storage.toLinkedHashMap
+if (parameters.nonEmpty) {
+  map.put("Partition Parameters", s"{${parameters.map(p => p._1 + "=" 
+ p._2).mkString(", ")}}")
+}
+map
+  }
 
-output.filter(_.nonEmpty).mkString("CatalogPartition(\n\t", "\n\t", 
")")
+  override def toString: String = {
+toLinkedHashMap.map { case ((key, value)) =>
+  if (value.isEmpty) key else s"$key: $value"
+}.mkString("CatalogPartition(\n\t", "\n\t", ")")
+  }
+
+  def simpleString: String = {
--- End diff --

when shall we call `simpleString`?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #17394: [SPARK-20067] [SQL] Unify and Clean Up Desc Comma...

2017-03-31 Thread cloud-fan 
Github user cloud-fan commented on a diff in the pull request:

https://github.com/apache/spark/pull/17394#discussion_r109205166
  
--- Diff: 
sql/core/src/main/scala/org/apache/spark/sql/execution/command/tables.scala ---
@@ -500,7 +500,6 @@ case class TruncateTableCommand(
 case class DescribeTableCommand(
 table: TableIdentifier,
 partitionSpec: TablePartitionSpec,
-isExtended: Boolean,
--- End diff --

seems `DESC EXTENDED` is better, normal `DESC tbl` is also formated


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #17394: [SPARK-20067] [SQL] Unify and Clean Up Desc Comma...

2017-03-27 Thread viirya 
Github user viirya commented on a diff in the pull request:

https://github.com/apache/spark/pull/17394#discussion_r108338016
  
--- Diff: 
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/interface.scala
 ---
@@ -21,6 +21,7 @@ import java.net.URI
 import java.util.Date
 
 import com.google.common.base.Objects
+import scala.collection.mutable
--- End diff --

Usually, the import order is:

import java.util.Date

import scala.collection.mutable

import com.google.common.base.Objects

`Dataset.scala` is an example.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #17394: [SPARK-20067] [SQL] Unify and Clean Up Desc Comma...

2017-03-27 Thread gatorsmile 
Github user gatorsmile commented on a diff in the pull request:

https://github.com/apache/spark/pull/17394#discussion_r108333203
  
--- Diff: 
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/interface.scala
 ---
@@ -21,6 +21,7 @@ import java.net.URI
 import java.util.Date
 
 import com.google.common.base.Objects
+import scala.collection.mutable
--- End diff --

If we change the order, can scalastyle pass?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #17394: [SPARK-20067] [SQL] Unify and Clean Up Desc Comma...

2017-03-26 Thread viirya 
Github user viirya commented on a diff in the pull request:

https://github.com/apache/spark/pull/17394#discussion_r108082510
  
--- Diff: 
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/interface.scala
 ---
@@ -21,6 +21,7 @@ import java.net.URI
 import java.util.Date
 
 import com.google.common.base.Objects
+import scala.collection.mutable
--- End diff --

Usually we put the imports of scala library above third-party.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org