Re: [DISCUSS] FLIP-152: Hive Query Syntax Compatibility

2021-02-03 Thread Rui Li
Hi Jingsong, Godfrey and Jark, I have updated the FLIP to incorporate your suggestions. Please let me know if you have further comments. Thank you. On Wed, Feb 3, 2021 at 4:51 PM Rui Li wrote: > Thanks Godfrey & Jark for your comments! > > Since both of you mentioned the naming of the parser fa

Re: [DISCUSS] FLIP-152: Hive Query Syntax Compatibility

2021-02-03 Thread Rui Li
Thanks Godfrey & Jark for your comments! Since both of you mentioned the naming of the parser factory, I'll answer this question first. I only intend to introduce the pluggable Parser for blink planner. So the BlinkParserFactory will be added to the flink-table-planner-blink module. We already ha

Re: [DISCUSS] FLIP-152: Hive Query Syntax Compatibility

2021-02-02 Thread Jark Wu
Thanks Rui for the great proposal, I believe this can be very attractive for many Hive users. The FLIP looks good to me in general, I only have some minor comments: 1) BlinkParserFactory IIUC, BlinkParserFactory is located in the flink-table-api-java module with the Parser interface there. I sugg

Re: [DISCUSS] FLIP-152: Hive Query Syntax Compatibility

2021-02-02 Thread godfrey he
Thanks for bringing up the discussion, Rui! Regarding the DDL part in the "Introduce HiveParser" section, I would like to choose the second option. Because if we could use one hive parser to parse all hive SQLs, we need not to copy Calcite parser code, and the framework and the code will be very s

Re: [DISCUSS] FLIP-152: Hive Query Syntax Compatibility

2021-02-02 Thread Rui Li
Hi Jingsong, Thanks for your comments and they're very good questions. Regarding # Version, we need to do some tradeoff here. Choosing the latest 3.x will cover all the features we want to support. But as you said, 3.x and 2.x can have some differences and requires more efforts to support lower v

Re: [DISCUSS] FLIP-152: Hive Query Syntax Compatibility

2021-02-02 Thread Jingsong Li
Thanks Rui for the proposal, I think this FLIP is required by many users, and it is very good to traditional Hive users. I have some confusion: # Version Which Hive version do you want to choose? Maybe, Hive 3.X and Hive 2.X have some differences? # Hive Codes Can you evaluate how much code we

Re: [DISCUSS] FLIP-152: Hive Query Syntax Compatibility

2020-12-10 Thread Zhijiang
Thanks for the further info and explanations! I have no other concerns. Best, Zhijiang -- From:Rui Li Send Time:2020年12月10日(星期四) 20:35 To:dev ; Zhijiang Subject:Re: [DISCUSS] FLIP-152: Hive Query Syntax Compatibility Hi Zhijiang,

Re: [DISCUSS] FLIP-152: Hive Query Syntax Compatibility

2020-12-10 Thread Rui Li
Hi Zhijiang, Glad to know you're interested in this FLIP. I wouldn't claim 100% compatibility with this FLIP. That's because Flink doesn't have the functionalities to support all Hive's features. To list a few examples: 1. Hive allows users to process data with shell scripts -- very similar

Re: [DISCUSS] FLIP-152: Hive Query Syntax Compatibility

2020-12-10 Thread Zhijiang
Thanks for launching the discussion and the FLIP, Rui! It is really nice to see our continuous efforts for compatibility with Hive and benefiting users in this area. I am only curious that are there any other compatible limitations for Hive users after this FLIP? Or can I say that the Hive compa

Re: [DISCUSS] FLIP-152: Hive Query Syntax Compatibility

2020-12-10 Thread Rui Li
Thanks Kurt for your inputs! I agree we should extend Hive code to support non-Hive tables. I have updated the wiki page to remove the limitations you mentioned, and add typical use cases in the "Motivation" section. Regarding comment #b, the interface is defined in flink-table-planner-blink and

Re: [DISCUSS] FLIP-152: Hive Query Syntax Compatibility

2020-12-06 Thread Kurt Young
Thanks Rui for starting this discussion. I can see the benefit that we improve hive compatibility further, as quite some users are asking for this feature in mailing lists [1][2][3] and some online chatting tools such as DingTalk. I have 3 comments regarding to the design doc: a) Could you add a