[ https://issues.apache.org/jira/browse/FLINK-26764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17562047#comment-17562047 ]
Jark Wu edited comment on FLINK-26764 at 7/5/22 7:57 AM: --------------------------------------------------------- I checked some resources[1][2][3], and it seems the default behavior of first_value is "respect nulls"[3]: > The SQL standard defines a RESPECT NULLS or IGNORE NULLS option for lead, > lag, first_value, last_value, and nth_value. This is not implemented in > PostgreSQL: the behavior is always the same as the standard's default, namely > RESPECT NULLS. I think we should follow SQL standards but keep compatibility. Therefore, I think we should change the default behavior to respect nulls and we can provide an option for users to switch back to the previous behavior. Besides, [~luoyuxia] could you help to check the null behavior of LEAD and LAG in Flink SQL? We should also fix them if they ignore nulls. Regarding the config option name, I would suggest {{table.exec.navigation-functions.null-treatment=respect_nulls/ignore_nulls}} or {{table.exec.first-last-value.null-treatment=respect_nulls/ignore_nulls}} in case of only need to fix first_value and last_value. [1]: https://modern-sql.com/caniuse/T617 [2]: https://modern-sql.com/caniuse/first_value [3]: https://www.postgresql.org/docs/current/functions-window.html was (Author: jark): I checked some resources[1][2][3], and it seems the default behavior of first_value is "respect nulls"[3]: > The SQL standard defines a RESPECT NULLS or IGNORE NULLS option for lead, > lag, first_value, last_value, and nth_value. This is not implemented in > PostgreSQL: the behavior is always the same as the standard's default, namely > RESPECT NULLS. I think we should follow SQL standards but keep compatibility. Therefore, I agree with [~godfreyhe] that adding a config option to respect nulls (default ignore nulls). Besides, [~luoyuxia] could you help to check the null behavior of LEAD and LAG in Flink SQL? We should also fix them if they ignore nulls. Regarding the config option name, I would suggest {{table.exec.navigation-functions.null-treatment=respect_nulls/ignore_nulls}} or {{table.exec.first-last-value.null-treatment=respect_nulls/ignore_nulls}} in case of only need to fix first_value and last_value. [1]: https://modern-sql.com/caniuse/T617 [2]: https://modern-sql.com/caniuse/first_value [3]: https://www.postgresql.org/docs/current/functions-window.html > Support RESPECT NULLS for FIRST_VALUE/LAST_VALUE > ------------------------------------------------- > > Key: FLINK-26764 > URL: https://issues.apache.org/jira/browse/FLINK-26764 > Project: Flink > Issue Type: New Feature > Components: Table SQL / API, Table SQL / Planner > Reporter: luoyuxia > Assignee: luoyuxia > Priority: Major > Labels: pull-request-available > Fix For: 1.16.0 > > > Flink supports fucntion FIRST_VALUE/LAST_VALUE, but the behavior is always > ignore null value. > But the > [Spark|https://spark.apache.org/docs/2.4.2/api/sql/index.html#first_value], > [Hive|https://cwiki.apache.org/confluence/display/hive/languagemanual+windowingandanalytics], > > [Oracle|https://docs.oracle.com/cd/B19306_01/server.102/b14200/functions057.htm], > > [Snowflake|https://docs.snowflake.com/en/sql-reference/functions/first_value.html], > etc, also support to respect null for FIRST_VALUE/LAST_VALUE. > Should we also support to allow users to specifc whether to ignore null? -- This message was sent by Atlassian Jira (v8.20.10#820010)