Hi Ran,
adding additional filter and selection capabilities makes sense.
However, I would recommend to not only check Spark or Hive but also
focus on bigger commercial players such as Snowflake, Oracle, SQL Server.
In any case, maybe we should also investigate more thoughts on a Flink
INFORMATION_SCHEMA as the SQL standard defines it. Having a queryable
table that contains metadata on which we can run arbitrary SQL logic not
only filtering but aggregations like counting or projections.
This would avoid touching the parser everytime and introduce more and
more SQL keywords.
What do you think?
Regards,
Timo
On 21.02.23 14:32, Martijn Visser wrote:
Hi Ran,
When you're creating your FLIP, please also have a look at Calcite SQL
parser, since that's used by Flink [1]
Best regards,
Martijn
[1] https://calcite.apache.org/docs/reference.html
On Tue, Feb 21, 2023 at 2:10 PM Ran Tao <chucheng...@gmail.com> wrote:
Yes, I totally agree with your point. The current flink sql syntax is tiled
together without classification.
Let me try to summarize these four categories, not just Auxiliary
Statements. And compare it with other popular engines.
Thank you, Jing !
Best Regards,
Ran Tao
https://github.com/chucheng92
Jing Ge <j...@ververica.com.invalid> 于2023年2月21日周二 20:50写道：
+ 1 for a new FLIP
It would be great if we could take this opportunity to go through all
current Flink SQL syntax and define a feasible classification in the
FLIP.
In my opinion, the 4 categories, i.e. DDL, DML, DQL, DAL , used in Spark
doc [1] make sense.
We will then have a better big picture to know the gap between Flink and
other big data engines like Spark and what should be done.
Best regards,
Jing
[1]
https://spark.apache.org/docs/latest/sql-ref-syntax.html#auxiliary-statements
On Tue, Feb 21, 2023 at 1:15 PM Ran Tao <chucheng...@gmail.com> wrote:
Thanks. I will create a FLIP to illustrate some details.
Best Regards,
Ran Tao
https://github.com/chucheng92
Jark Wu <imj...@gmail.com> 于2023年2月21日周二 20:03写道：
Thank you,
I think this is worth a FLIP design doc to discuss the detailed
syntax.
Could you prepare a FLIP for that?
Best,
Jark
On Tue, 21 Feb 2023 at 19:37, Ran Tao <chucheng...@gmail.com> wrote:
Hi, Jark. thanks. I have added a google doc.
https://docs.google.com/document/d/1hAiOfPx14VTBTOlpyxG7FA2mB1k5M31VnKYad2XpJ1I/edit?usp=sharing
Jark Wu <imj...@gmail.com> 于2023年2月21日周二 19:27写道：
Hi Ran,
I think it’s always nice to support new syntax if it’s useful for
the
users.
From my side, your syntax table is broken. Could you share it
with
a
Google doc or create a JIRA issue?
Best,
Jark
2023年2月21日 17:51，Ran Tao <chucheng...@gmail.com> 写道：
Hi guys. When I recently used flink sql to manage internal
metadata
(catalog/databases/table/functions),
I found that many current flink sql statements do not support
filtering
or
some advanced syntax, however these abilities are very useful
to
end-users.
These are some statements I have collected so far, which are
supported
on
other big data engines, such as spark, hive or presto. I wonder
if
we
can
support these abilities?
In addition, the subject of this email is named 'Auxiliary
Statements'
mainly because the alignment of these statements will not have
much
impact
on the core SQL runtime.
Support or Not
With Advanced Syntax Or Not (in/from or like)
show create table
Yes
Yes
show tables
Yes
Yes
show columns
Yes
Yes
show catalogs
Yes
without filter
show databases
Yes
without filter
show functions
Yes
without filter
show views
Yes
without filter
show modules
Yes
without filter
show jars
Yes
without filter
show jobs
Yes
without filter
We can see current flink many sql statements only support
showing
with
full
datas, without 'FROM/IN' or 'LIKE' filter clause.
Support or Not
describe database
No
describe table
Yes
describe function
No
describe query
No
current flink only supports describing tables.
Also, please let me know if there is a mistake. Looking forward
to
your
reply.
Best Regards,
Ran Tao
https://github.com/chucheng92
