[ https://issues.apache.org/jira/browse/FLINK-30556?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17654349#comment-17654349 ]
Wencong Liu commented on FLINK-30556: ------------------------------------- Hello [~luoyuxia] , I'm quite interested in this issue. Could you provide the code location where HiveSource enumerate all splits? > Improve the logic for enumerating splits for Hive source to avoid potential > OOM > ------------------------------------------------------------------------------- > > Key: FLINK-30556 > URL: https://issues.apache.org/jira/browse/FLINK-30556 > Project: Flink > Issue Type: Sub-task > Components: Connectors / Hive > Affects Versions: 1.16.0 > Reporter: luoyuxia > Priority: Major > > Currently, when read hive source in batch mode, it'll first enumerate all > split for the hive table. But when the table is large, the split will be too > many which may well cause OOM. Some commuity users has also reported this > problem. > We need to optimize the logic for enumerating splits for hive table source to > avoid potential OOM. -- This message was sent by Atlassian Jira (v8.20.10#820010)