Re: Problems with the state.backend.fs.memory-threshold parameter

2023-10-23 Thread rui chen
Hi,Zakelly Thank you for your answer. Best, rui Zakelly Lan 于2023年10月13日周五 19:12写道: > Hi rui, > > The 'state.backend.fs.memory-threshold' configures the threshold below > which state is stored as part of the metadata, rather than in separate > files. So as a result the JM will use its memory t

Re: Problems with the state.backend.fs.memory-threshold parameter

2023-10-13 Thread Zakelly Lan
Hi rui, The 'state.backend.fs.memory-threshold' configures the threshold below which state is stored as part of the metadata, rather than in separate files. So as a result the JM will use its memory to merge small checkpoint files and write them into one file. Currently the FLIP-306[1][2] is propo