[ https://issues.apache.org/jira/browse/SPARK-28930?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Dongjoon Hyun updated SPARK-28930: ---------------------------------- Component/s: (was: Spark Shell) > Spark DESC FORMATTED TABLENAME information display issues > --------------------------------------------------------- > > Key: SPARK-28930 > URL: https://issues.apache.org/jira/browse/SPARK-28930 > Project: Spark > Issue Type: Bug > Components: SQL > Affects Versions: 2.4.3 > Reporter: jobit mathew > Priority: Minor > > Spark DESC FORMATTED TABLENAME information display issues.Showing incorrect > *Last Access time and* feeling some information displays can make it better. > Test steps: > 1. Open spark sql > 2. Create table with partition > CREATE EXTERNAL TABLE IF NOT EXISTS employees_info_extended ( id INT, name > STRING, usd_flag STRING, salary DOUBLE, deductions MAP<STRING, DOUBLE>, > address STRING ) PARTITIONED BY (entrytime STRING) STORED AS TEXTFILE > location 'hdfs://hacluster/user/sparkhive/warehouse'; > 3. from spark sql check the table description > desc formatted tablename; > 4. From scala shell check the table description > sql("desc formatted tablename").show() > *Issue1:* > If there is no comment for spark scala shell shows *"null" in small letters* > but all other places Hive beeline/Spark beeline/Spark SQL it is showing in > *CAPITAL "NULL*". Better to show same in all places. > > {code} > *scala>* sql("desc formatted employees_info_extended").show(false); > +-----------------------------+---------------------------++------- > |col_name|data_type|*comment*| > +-----------------------------+---------------------------++------- > |id|int|*null*| > |name|string|*null*| > |usd_flag|string|*null*| > |salary|double|*null*| > |deductions|map<string,double>|*null*| > |address|string|null| > |entrytime|string|null| > | # Partition Information| | | > | # col_name|data_type|comment| > |entrytime|string|null| > | | | | > | # Detailed Table Information| | | > |Database|sparkdb__| | > |Table|employees_info_extended| | > |Owner|root| | > *|Created Time |Tue Aug 20 13:42:06 CST 2019| |* > *|Last Access |Thu Jan 01 08:00:00 CST 1970| |* > |Created By|Spark 2.4.3| | > |Type|EXTERNAL| | > |Provider|hive| | > +-----------------------------+---------------------------++------- > only showing top 20 rows > *scala>* > {code} > *Issue 2:* > Spark SQL "desc formatted tablename" is not showing the header [# > col_name,data_type,comment|#col_name,data_type,comment] in the top of the > query result.But header is showing on top of partition description. For > Better understanding show the header on Top of the query result. > {code} > *spark-sql>* desc formatted employees_info_extended1; > id int *NULL* > name string *NULL* > usd_flag string NULL > salary double NULL > deductions map<string,double> NULL > address string NULL > entrytime string NULL > * > ## Partition Information* > ## col_name data_type comment* > entrytime string *NULL* > # Detailed Table Information > Database sparkdb__ > Table employees_info_extended1 > Owner spark > *Created Time Tue Aug 20 14:50:37 CST 2019* > *Last Access Thu Jan 01 08:00:00 CST 1970* > Created By Spark 2.3.2.0201 > Type EXTERNAL > Provider hive > Table Properties [transient_lastDdlTime=1566286655] > Location hdfs://hacluster/user/sparkhive/warehouse > Serde Library org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe > InputFormat org.apache.hadoop.mapred.TextInputFormat > OutputFormat org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat > Storage Properties [serialization.format=1] > Partition Provider Catalog > Time taken: 0.477 seconds, Fetched 27 row(s) > *spark-sql>* > {code} > > *Issue 3:* > I created the table on Aug 20.So it is showing created time correct .*But > Last access time showing 1970 Jan 01*. It is not good to show Last access > time earlier time than the created time.Better to show the correct date and > time else show UNKNOWN. > *[Created Time,Tue Aug 20 13:42:06 CST 2019,]* > *[Last Access,Thu Jan 01 08:00:00 CST 1970,]* -- This message was sent by Atlassian Jira (v8.3.2#803003) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org