Hi, I would like to explain a bit more about the current dissusion[1] for the ways to decouple Hive connector with Flink Planner.
The background is to support Hive dialect, the Hive connector is dependent on Flink Planner for the current implementation is generate RelNode and then deliver the RelNode to Flink. But it also brings much complexity and maintenance burden, so we want to decouple Hive connector with Flink planner. There're two ways to do that: 1. Make the hive parser just generate an Operation tree like Table API currently does. 2. Introduce a slim module called table-planner-spl which provide Calcite dependency and expose limit public intefaces. Then, still generating Calcite RelNode, the Hive connector will only require the slim module. The first way is the ideal way and we should go in that direction. But it will take much effort for it requires rewriting all the code about Hive dialect and it's hard to do it in one shot. And given we want to move out Hive connector in 1.16, it's more pratical to decouple first, and then migrate it to operation tree. So, the FLIP-216[2] is for the second way. It explains the public interfaces to be exposed, all of which has been implemented by PlannerContext. [1] https://docs.google.com/document/d/1LMQ_mWfB_mkYkEBCUa2DgCO2YdtiZV7YRs2mpXyjdP4/edit?usp=sharing [2] https://cwiki.apache.org/confluence/display/FLINK/FLIP-216%3A+Decouple+Hive+connector+with+Flink+planner Best regards, Yuxia------------------------------------------------------------------ 发件人：罗宇侠(莫辞)<[email protected]> 日 期：2022年03月25日 20:41:15 收件人：[email protected]<[email protected]> 主 题：[DISCUSS] FLIP-216 Decouple Hive connector with Flink planner Hi, everyone I would like to open a discussion about decoupling Hive connector with Flink table planner. It's a follow-up discussion after Hive syntax discussion[1], but only focus on how to decouple Hive connector. The origin doc is here[2], from which you can see the details work and heated discussion about exposing Calcite API or reuse Operation tree to decouple. I have created FLIP-216: Decouple Hive connector with Flink planner[3] for it. Thanks and looking forward to a lively discussion! [1] https://lists.apache.org/thread/2w046dwl46tf2wy750gzmt0qrcz17z8t [2] https://docs.google.com/document/d/1LMQ_mWfB_mkYkEBCUa2DgCO2YdtiZV7YRs2mpXyjdP4/edit?usp=sharing [3] https://cwiki.apache.org/confluence/display/FLINK/FLIP-216%3A+Decouple+Hive+connector+with+Flink+planner Best regards, Yuxia
