Hi Jingsong,
Thanks a lot for reporting this issue.
IIRC, we added [INSERT OVERWRITE] and [PARTITION] clauses to support Hive
integration before FLIP-63 was proposed to introduce generic partition
support to Flink. Thus when we added these syntax, we were intentionally
conservative and limited their scope to Hive dialect. @Rui may help to
confirm that. I'm a bit surprised about it as well. As core APIs of FLIP-63
were done, I don't see why we would limit these syntax to Hive dialect
alone. It's just unfortunately that we may have forgot to revisit this
topic and we apparently missed some test cases on SQL CLI side. Sorry for
that.
>From a product perspective, SQL CLI is super critical for Flink-Hive
integration and Flink SQL iteself. INSERT OVERWRITE and PARTITION are two
of the most commonly used syntax in Hive, the product wouldn't be useable
without such proper support.
Thus, I think this is a *bug*, we can stop limiting them in Hive dialect,
and should fix it in 1.10. We should also add more test coverage for SQL
CLI to avoid such surprise.
Cheers,
Bowen
On Wed, Dec 11, 2019 at 1:29 AM Jingsong Li <jingsongl...@gmail.com> wrote:
> Hi Dev,
>
> After cutting out the branch of 1.10, I tried the following functions of
> SQL-CLI and found that it does not support:
> - insert overwrite
> - PARTITION (partcol1=val1, partcol2=val2 ...)
> The SQL pattern is:
> INSERT { INTO | OVERWRITE } TABLE tablename1 [PARTITION (partcol1=val1,
> partcol2=val2 ...) select_statement1 FROM from_statement;
> It is a surprise to me.
> The reason is that we only allow these two grammars in hive dialect. And
> SQL-CLI does not have an interface to switch dialects.
>
> Because it directly hinders the SQL-CLI's insert syntax in hive integration
> and seriously hinders the practicability of SQL-CLI.
> And we have introduced these two grammars in FLIP-63 [1] to Flink.
> Here are my question:
> 1.Should we remove hive dialect limitation for these two grammars?
> 2.Should we fix this in 1.10?
>
> [1]
>
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-63%3A+Rework+table+partition+support
>
> Best,
> Jingsong Lee
>
