Weijie Guo created FLINK-30332: ---------------------------------- Summary: HsFileDataIndex should supports spill to file Key: FLINK-30332 URL: https://issues.apache.org/jira/browse/FLINK-30332 Project: Flink Issue Type: Sub-task Reporter: Weijie Guo Assignee: Weijie Guo
`HsFileDataIndex` is maintains in a list of treeMap(i.e. in memory), there is no problem in our 10T TPC-DS test. However, with the increase of job's data volume, the index will continue to grow, and there is a risk of OOM. We should support index data spilling and loading to avoid this problem. -- This message was sent by Atlassian Jira (v8.20.10#820010)