Thanks Jark and Timo for your explain. +1 to use SQL strings. Just took a look to calcite unparse, it really complex.
Jark, yeah, The only thing need by ExpressionConverter is RelBuilder. Best, Jingsong Lee On Thu, Oct 24, 2019 at 5:09 PM Timo Walther <twal...@apache.org> wrote: > Hi Jark, > > +1 for your suggestion. I think it will simplify the design a lot if we > serialize all expressions as SQL strings and will avoid duplicate parser > code. Initially, I had concerns that there might be expressions in the > future that cannot be represented in SQL. But currently I cannot come up > with a counter example. > > Table operations will be a different topic that will require a custom > string syntax. But expressions as SQL expressions sounds good to me. > > @Jingsong: Jark is right. Table API expression strings are outdated and > error-prone. They will be removed at some point. > > Regards, > Timo > > > On 24.10.19 10:50, Jark Wu wrote: > > Thanks Jingsong, > > > > As discussed in Java Expression DSL, we are planning to drop the Java > Expression string API. > > So I think we don’t have a plan to unify #1 and #2. And in the future, > we may only have SQL parser to parse a string expression. > > The only thing to consider is, whether can all the resolved expression > be converted to SqlNode. > > AFAIK, currently, after expression resolving, all the expressions can be > converted to SqlNodes. > > > > Best, > > Jark > > > > [1]: > http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Introduction-of-a-Table-API-Java-Expression-DSL-td27787.html > < > http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Introduction-of-a-Table-API-Java-Expression-DSL-td27787.html > > > > > >> 在 2019年10月24日，13:30，Jingsong Li <jingsongl...@gmail.com> 写道： > >> > >> Thanks Jark for your proposal. > >> > >> If we introduce a new kind of string presentation for expression, we > will > >> have 3 string presentation now: > >> 1. Java expression string api. We have PlannerExpressionParser to parse > >> string to Expressions. > >> 2. Sql string presentation, as you said, we can use calcite classes to > >> parse and unparse. > >> 3. New kind of string presentation for serialize. > >> > >> From this point of view, I prefer not to introduce a new kind of string > >> presentation to reduce the complexity. > >> > >> There are some differences between #1 and #2: > >> - method invoking: "f0.substring(1, f7)" and "SUBSTRING(f0, 1, f7)" > >> - bigint literal: "1L" and "cast(1 as BIGINT)" > >> > >> Now it is two completely independent sets., Whether we can unify #1 and > #2 > >> into one set, and we all use one parser? > >> > >> Best, > >> Jingsong Lee > >> > >> On Tue, Oct 22, 2019 at 7:57 PM Jark Wu <imj...@gmail.com> wrote: > >> > >>> Hi Timo, > >>> > >>> I think it's a good idea to use `SqlParser#parseExpression()` to parse > >>> literals. > >>> That means the string format of literal is SQL compatible. > >>> After some discussion with Kurt, we think why not one more step > forward, > >>> i.e. convert the whole expression to SQL format. > >>> > >>> For example, the above expression will be converted to: > >>> > >>> `cat`.`db`.`func`(`cat`.`db`.`f0`, TIMESTAMP '2019-10-21 12:12:12') > >>> > >>> There are some benefits from this: > >>> 0) the string representation is more readable, and can be manually > typed > >>> more easily. > >>> 1) the string format is SQL syntax, not customized, which means it can > be > >>> integrated by third party projects. > >>> 2) we can reuse Calcite's SqlParser to parse string and > SqlNode#unparse to > >>> generate string, this can avoid introducing duplicate code and a custom > >>> parser. > >>> 3) no compatible problems. > >>> > >>> Regarding to how Expression can be converted into a SQL string, I > think we > >>> can leverage some Calcite utils: > >>> > >>> ResolvedExpression ---(ExpressionConverter)---> RexNode > >>> ----(RexToSqlNodeConverter)---> SqlNode --> SqlNode#unparse() > >>> > >>> What do you think? > >>> > >>> Best, > >>> Jark > >>> > >>> On Mon, 21 Oct 2019 at 22:08, Timo Walther <twal...@apache.org> wrote: > >>> > >>>> Hi Jark, > >>>> > >>>> thanks for the proposal. This is a great effort to finalize the new > API > >>>> design. > >>>> > >>>> I'm wondering if we could simply use the SQL parser like > >>>> `org.apache.calcite.sql.parser.SqlParser#parseExpression(..)` to parse > >>>> an expression that contain only literals. This would avoid any > >>>> discussion as the syntax is already defined by the SQL standard. And > it > >>>> would also be very unlikely to have a need for a version. > >>>> > >>>> For example: > >>>> > >>>> CALL('FUNC', FIELD('f0'), VALUE('TIMESTAMP(3)', TIMESTAMP '2019-10-21 > >>>> 12:12:12')) > >>>> > >>>> Or even further if the SQL parser allows that: > >>>> > >>>> CALL('FUNC', `cat`.`db`.`f0`, TIMESTAMP '2019-10-21 12:12:12') > >>>> > >>>> I would find it confusing if we use different representation for > >>>> literals such as intervals and timestamps in the properties. This > would > >>>> also reduce code duplication as we reuse logic for parsing identifiers > >>> etc. > >>>> > >>>> What do you think? > >>>> > >>>> Regards, > >>>> Timo > >>>> > >>>> > >>>> On 18.10.19 12:28, Jark Wu wrote: > >>>>> Hi everyone, > >>>>> > >>>>> I would like to start a discussion[1] about how to make Expression > >>> string > >>>>> serializable and deserializable. Expression is the general interface > >>> for > >>>>> all kinds of expressions in Flink Table API & SQL, it represents a > >>>> logical > >>>>> tree for producing a computation result. In FLIP-66[2] and > FLIP-70[3], > >>> we > >>>>> introduced watermark and computed column syntax in DDL. The watermark > >>>>> strategy and computed column are both represented in Expression. In > >>> order > >>>>> to persist watermark and computed column information in catalog, we > >>> need > >>>> to > >>>>> figure out how to persist and restore Expression. > >>>>> > >>>>> FLIP-80: > >>>>> > >>>> > >>> > https://docs.google.com/document/d/1LxPEzbPuEVWNixb1L_USv0gFgjRMgoZuMsAecS_XvdE/edit?usp=sharing > >>>>> > >>>>> Thanks for any feedback! > >>>>> > >>>>> Best, > >>>>> Jark > >>>>> > >>>>> [1]: > >>>>> > >>>> > >>> > https://docs.google.com/document/d/1LxPEzbPuEVWNixb1L_USv0gFgjRMgoZuMsAecS_XvdE/edit?usp=sharing > >>>>> [2]: > >>>>> > >>>> > >>> > https://cwiki.apache.org/confluence/display/FLINK/FLIP-66%3A+Support+time+attribute+in+SQL+DDL > >>>>> [3]: > >>>>> > >>>> > >>> > https://cwiki.apache.org/confluence/display/FLINK/FLIP-70%3A+Flink+SQL+Computed+Column+Design > >>>>> > >>>> > >>>> > >>> > >> > >> > >> -- > >> Best, Jingsong Lee > > > > > > -- Best, Jingsong Lee
