Hi everyone,
some of you might have already noticed the JIRA issue that I opened
recently [1] about introducing a proper Java expression DSL for the
Table API. Instead of using string-based expressions, we should aim for
a unified, maintainable, programmatic Java DSL.
Some background: The Blink merging efforts and the big refactorings as
part of FLIP-32 have revealed many shortcomings in the current Table &
SQL API design. Most of these legacy issues cause problems nowadays in
making the Table API a first-class API next to the DataStream API. An
example is the ExpressionParser class[2]. It was implemented in the
early days of the Table API using Scala parser combinators. During the
last years, this parser caused many JIRA issues and user confusion on
the mailing list. Because the exceptions and syntax might not be
straight forward.
For FLINK-11908, we added a temporary bridge instead of reimplementing
the parser in Java for FLIP-32. However, this is only a intermediate
solution until we made a final decision.
I would like to propose a new, parser-free version of the Java Table API:
https://docs.google.com/document/d/1r3bfR9R6q5Km0wXKcnhfig2XQ4aMiLG5h2MTx960Fg8/edit?usp=sharing
I already implemented an early protoype that shows that such a DSL is
not much implementation effort and integrates nicely with all existing
API methods.
What do you think?
Thanks for your feedback,
Timo
[1] https://issues.apache.org/jira/browse/FLINK-11890
[2]
https://github.com/apache/flink/blob/master/flink-table/flink-table-planner/src/main/scala/org/apache/flink/table/expressions/PlannerExpressionParserImpl.scala
