ulysses-you opened a new issue #1070: URL: https://github.com/apache/incubator-kyuubi/issues/1070
### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) ### Search before asking - [X] I have searched in the [issues](https://github.com/apache/incubator-kyuubi/issues?q=is%3Aissue) and found no similar issues. ### Describe the feature Add zorder before writing dynamically. This feature support: - insert into datasource table - create datasource table as select - insert into hive table - create hive table as select ### Motivation Zorder is a popular approach of data skipping. We have supported `OPTIMIZE <table> [WHERE <predicate>] ZORDER BY col1, col2, ...` but it's not enough for user. Let's say, If we have a hot table that adds a partition every day so we decide to apply zorder to optimize it. The table properties become to a good place to store the zorder info. When you realize that this is marked as a zorder table, we can optimize it. ### Describe the solution Support parse two keys in table properties: - kyuubi.zorder.enabled decide if we should add zorder before writing this table - kyuubi.zorder.cols which columns should be included with zorder And we also add a global config `spark.sql.optimizer.insertZorderBeforeWriting.enabled`. ### Additional context _No response_ ### Are you willing to submit PR? - [X] Yes I am willing to submit a PR! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
