beyond1920 opened a new pull request #17038:
URL: https://github.com/apache/flink/pull/17038
## What is the purpose of the change
There is a syntax mistake in session Window TVF in FLINK-23543.
For example, the following SQL has syntax mistake.
"""
|SELECT
| a,
| window_start,
| window_end,
| count(*),
| sum(d),
| max(d) filter (where b > 1000),
| count(distinct c) AS uv
|FROM TABLE(
| SESSION(
| TABLE MyTable,
| DESCRIPTOR(proctime),
| INTERVAL '5' MINUTE))
|GROUP BY a, window_start, window_end
""".stripMargin
It should updated to the following SQL, while partition key (a) should be
moved into SESSION
Window TVF based on Calcite SESSION window TVF.
val sql =
"""
|SELECT
| a,
| window_start,
| window_end,
| count(*),
| sum(d),
| max(d) filter (where b > 1000),
| count(distinct c) AS uv
|FROM TABLE(
| SESSION(
| TABLE MyTable,
| DESCRIPTOR(proctime),
| DESCRIPTOR(a)
| INTERVAL '5' MINUTE))
|GROUP BY a, window_start, window_end
""".stripMargin
To fix the bug, we only need update Session Window TVF syntax, we don't need
update the operator part.
Besides, we should check group keys of window aggregate should only contain
window_start, window_end, partition_key. group keys could not contain other
fields.
## Brief change log
* Update `SqlSessionTableFunction` to add partition key parameter and add
check logical for new added parameter
* Update `SessionWindowSpec` to add partition key indices
* Update `WindowSpec#toSummaryString` to add second parameter
inputFieldNames in order to make session window spec string more friendly to
read. and update it's related classes.
* Update `ProjectWindowTableFunctionTransposeRule`,
`ExpandWindowTableFunctionTransposeRule`,
`PullUpWindowTableFunctionIntoWindowAggregateRule` to adjust session window
partition key indices.
## Verifying this change
- Add test in `WindowAggregateTest` to test group key of window aggregate
should contain and only contain window_start, window_end and partition keys of
session window.
- Add test in `WindowAggregateJsonITCase`
- Update test in `WindowAggregateITCase`
## Does this pull request potentially affect one of the following parts:
- Dependencies (does it add or upgrade a dependency): (no)
- The public API, i.e., is any changed class annotated with
`@Public(Evolving)`: (no)
- The serializers: (no)
- The runtime per-record code paths (performance sensitive): (no)
- Anything that affects deployment or recovery: JobManager (and its
components), Checkpointing, Kubernetes/Yarn, ZooKeeper: (no)
- The S3 file system connector: (no)
## Documentation
- Does this pull request introduce a new feature? (no)
- If yes, how is the feature documented? (not applicable)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
