Yangyang ZHANG created FLINK-33127:
--------------------------------------
Summary: HeapKeyedStateBackend: use buffered I/O to speed up local
recovery
Key: FLINK-33127
URL: https://issues.apache.org/jira/browse/FLINK-33127
Project: Flink
Issue Type: Improvement
Components: Runtime / State Backends
Reporter: Yangyang ZHANG
Attachments: thread_dump.png
Recently, I observed a slow restore case in local recovery using hashmap
statebackend.
It took 147 seconds to restore from a 467MB snapshot, 9 times slower than that
(16s) when restore from remote fs.
The thread dump show that It read local snapshot file directly by unbuffered
FileInputStream / fs.local.LocalDataInputStream.
!thread_dump.png!
Maybe we can wrap with BufferInputStream to speed up local recovery.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
