[ https://issues.apache.org/jira/browse/FLINK-19896?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17225815#comment-17225815 ]
Jun Zhang commented on FLINK-19896: ----------------------------------- We introduce a lightweight variant of AppendOnlyTopNFunction to handle first-n-row fetching, which only store a counter in the state to record the occurrence of partition key. > Improve first-n-rows fetching in the rank operator > -------------------------------------------------- > > Key: FLINK-19896 > URL: https://issues.apache.org/jira/browse/FLINK-19896 > Project: Flink > Issue Type: Improvement > Components: Table SQL / Planner, Table SQL / Runtime > Reporter: Jun Zhang > Priority: Major > Attachments: flink-19896.patch > > > Currently Deduplicate operator only supports first-row deduplication (ordered > by proc-time). In scenario of first-n-rows deduplication, the planner has to > resort to Rank operator. However, Rank operator is less efficient than > Deduplicate due to larger state and more state access. > This issue proposes to extend DeduplicateKeepFirstRowFunction to support > first-n-rows deduplication. And the original first-row deduplication would be > a special case of first-n-rows deduplication. -- This message was sent by Atlassian Jira (v8.3.4#803005)