[jira] [Commented] (SPARK-19241) remove hive generated table properties if they are not useful in Spark
[ https://issues.apache.org/jira/browse/SPARK-19241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16314081#comment-16314081 ] Sameer Agarwal commented on SPARK-19241: re-targeting this for 2.4.0 > remove hive generated table properties if they are not useful in Spark > -- > > Key: SPARK-19241 > URL: https://issues.apache.org/jira/browse/SPARK-19241 > Project: Spark > Issue Type: Improvement > Components: SQL >Reporter: Wenchen Fan > > When we save a table into hive metastore, hive will generate some table > properties automatically. e.g. transient_lastDdlTime, last_modified_by, > rawDataSize, etc. Some of them are useless in Spark SQL, we should remove > them. > It will be good if we can get the list of Hive-generated table properties via > Hive API, so that we don't need to hardcode them. > We can take a look at Hive code to see how it excludes these auto-generated > table properties when describe table. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-19241) remove hive generated table properties if they are not useful in Spark
[ https://issues.apache.org/jira/browse/SPARK-19241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15823654#comment-15823654 ] Song Jun commented on SPARK-19241: -- I am working on this~ > remove hive generated table properties if they are not useful in Spark > -- > > Key: SPARK-19241 > URL: https://issues.apache.org/jira/browse/SPARK-19241 > Project: Spark > Issue Type: Improvement > Components: SQL >Reporter: Wenchen Fan > > When we save a table into hive metastore, hive will generate some table > properties automatically. e.g. transient_lastDdlTime, last_modified_by, > rawDataSize, etc. Some of them are useless in Spark SQL, we should remove > them. > It will be good if we can get the list of Hive-generated table properties via > Hive API, so that we don't need to hardcode them. > We can take a look at Hive code to see how it excludes these auto-generated > table properties when describe table. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
