[
https://issues.apache.org/jira/browse/FLINK-37741?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Leonard Xu reassigned FLINK-37741:
----------------------------------
Assignee: yux
> Severe performance regression of transform operator node
> --------------------------------------------------------
>
> Key: FLINK-37741
> URL: https://issues.apache.org/jira/browse/FLINK-37741
> Project: Flink
> Issue Type: Bug
> Components: Flink CDC
> Reporter: yux
> Assignee: yux
> Priority: Major
> Labels: pull-request-available
> Attachments: image-2025-04-28-15-31-18-308.png
>
>
> FLINK-37104 refactored transform module with various code cleanup and logic
> simplification. However, a List<DataType> object is repeatedly created from
> pre/post transformed schemas, which significantly affects the RPS throughput.
> !image-2025-04-28-15-31-18-308.png!
> Based on my local testing, even a simple transform rule would cause a ~40%
> performance regression.
> |{color:#000000}*Version*{color}|*{color:#000000}Transform
> Rule{color}*|*{color:#000000}Time Elapsed{color}*|{color:#172b4d}*RPS*{color}|
> |{color:#000000}{*}3.4-SNAPSHOT{*}{*}{*}{color}|{color:#000000}Complex{color}|{color:#000000}4m
> 21s 86ms{color}|{color:#a30003}{*}38,314.18{*}{*}{*}{color}|
> |{color:#000000}{*}3.4-SNAPSHOT{*}{*}{*}{color}|{color:#000000}Simple{color}|{color:#000000}2m
> 45s 488ms{color}|{color:#a30003}{*}60,606.06{*}{*}{*}{color}|
> |{color:#000000}{*}3.4-SNAPSHOT{*}{*}{*}{color}|{color:#000000}Empty{color}|{color:#000000}1m
> 31s 869ms{color}|{color:#000000}109,890.11{color}|
> |{color:#000000}{*}3.4-SNAPSHOT{*}{*}{*}{color}|{color:#000000}Off{color}|{color:#000000}1m
> 45s 587ms{color}|{color:#000000}95,238.10{color}|
> |{color:#000000}{*}3.3{*}{*}{*}{color}|{color:#000000}Complex{color}|{color:#000000}1m
> 46s 54ms{color}|{color:#000000}94,339.62{color}|
> |{color:#000000}{*}3.3{*}{*}{*}{color}|{color:#000000}Simple{color}|{color:#000000}1m
> 34s 971ms{color}|{color:#000000}106,382.98{color}|
> |{color:#000000}{*}3.3{*}{*}{*}{color}|{color:#000000}Empty{color}|{color:#000000}1m
> 21s 546ms{color}|{color:#000000}123,456.79{color}|
> |{color:#000000}{*}3.3{*}{*}{*}{color}|{color:#000000}Off{color}|{color:#000000}1m
> 34s 5ms{color}|{color:#000000}106,382.98{color}|
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
