Thanks Rui for starting this discussion.
I can see the benefit that we improve hive compatibility further, as quite
some users are asking for this
feature in mailing lists [1][2][3] and some online chatting tools such as
DingTalk.
I have 3 comments regarding to the design doc:
a) Could you add a section to describe the typical use case you want to
support after this feature is introduced?
In that way, users can also have an impression how to use this feature and
what the behavior and outcome will be.
b) Regarding the naming: "BlinkParserFactory", I suggest renaming it to
"FlinkParserFactory".
c) About the two limitations you mentioned:
1. Only works with Hive tables and the current catalog needs to be a
HiveCatalog.
2. Queries cannot involve tables/views from multiple catalogs.
I assume this is because hive parser and analyzer doesn't support
referring to a name with "x.y.z" fashion? Since
we can control all the behaviors by leveraging the codes hive currently
use. Is it possible that we can remove such
limitations? The reason is I'm not sure if users can make the whole story
work purely depending on hive catalog (that's
the reason why I gave comment #a). If multiple catalogs are involved, with
this limitation I don't think any meaningful
pipeline could be built. For example, users want to stream data from Kafka
to Hive, fully use hive's dialect including
query part. The kafka table could be a temporary table or saved in default
memory catalog.
[1] http://apache-flink.147419.n8.nabble.com/calcite-td9059.html#a9118
[2] http://apache-flink.147419.n8.nabble.com/hive-sql-flink-11-td9116.html
[3]
http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/How-to-to-in-Flink-to-support-below-HIVE-SQL-td34162.html
Best,
Kurt
On Wed, Dec 2, 2020 at 10:02 PM Rui Li <lirui.fu...@gmail.com> wrote:
> Hi guys,
>
> I'd like to start a discussion about providing HiveQL compatibility for
> users connecting to a hive warehouse. FLIP-123 has already covered most
> DDLs. So now it's time to complement the other big missing part -- queries.
> With FLIP-152, the hive dialect covers more scenarios and makes it even
> easier for users to migrate to Flink. More details are in the FLIP wiki
> page [1]. Looking forward to your feedback!
>
> [1]
>
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-152%3A+Hive+Query+Syntax+Compatibility
>
> --
> Best regards!
> Rui Li
>
