Thanks Rui for starting this discussion. I can see the benefit that we improve hive compatibility further, as quite some users are asking for this feature in mailing lists [1][2][3] and some online chatting tools such as DingTalk.
I have 3 comments regarding to the design doc: a) Could you add a section to describe the typical use case you want to support after this feature is introduced? In that way, users can also have an impression how to use this feature and what the behavior and outcome will be. b) Regarding the naming: "BlinkParserFactory", I suggest renaming it to "FlinkParserFactory". c) About the two limitations you mentioned: 1. Only works with Hive tables and the current catalog needs to be a HiveCatalog. 2. Queries cannot involve tables/views from multiple catalogs. I assume this is because hive parser and analyzer doesn't support referring to a name with "x.y.z" fashion? Since we can control all the behaviors by leveraging the codes hive currently use. Is it possible that we can remove such limitations? The reason is I'm not sure if users can make the whole story work purely depending on hive catalog (that's the reason why I gave comment #a). If multiple catalogs are involved, with this limitation I don't think any meaningful pipeline could be built. For example, users want to stream data from Kafka to Hive, fully use hive's dialect including query part. The kafka table could be a temporary table or saved in default memory catalog. [1] http://apache-flink.147419.n8.nabble.com/calcite-td9059.html#a9118 [2] http://apache-flink.147419.n8.nabble.com/hive-sql-flink-11-td9116.html [3] http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/How-to-to-in-Flink-to-support-below-HIVE-SQL-td34162.html Best, Kurt On Wed, Dec 2, 2020 at 10:02 PM Rui Li <lirui.fu...@gmail.com> wrote: > Hi guys, > > I'd like to start a discussion about providing HiveQL compatibility for > users connecting to a hive warehouse. FLIP-123 has already covered most > DDLs. So now it's time to complement the other big missing part -- queries. > With FLIP-152, the hive dialect covers more scenarios and makes it even > easier for users to migrate to Flink. More details are in the FLIP wiki > page [1]. Looking forward to your feedback! > > [1] > > https://cwiki.apache.org/confluence/display/FLINK/FLIP-152%3A+Hive+Query+Syntax+Compatibility > > -- > Best regards! > Rui Li >