Hi,
I have already implemented both issues [1][2]. Timed flush memtable [3] is used to flush inactive memtables periodically according to their created time. For example, if we set seq_memtable_flush_interval_in_ms to 1000, memtable A, created at time 2000, can be flushed after time 3000. Timed close TsFileProcessor [4] is used to close inactive TsFileProcessors periodically according to how long they don't process write operations. For example, if we set close_tsfile_interval_after_flushing_in_ms to 1000, TsFileProcessor A, whose last workMemtable was flush at time 2000 and current workMemtable is null (i.e., no more wirite operations after last flush), can be closed after time 3000. But, i am confused whether to open timed flush seq/unseq memtable and timed close TsFileProcessor by default, because they can save memory in some user cases while they just waste cpu in other user cases. Look forward to your reply. [1] https://github.com/apache/iotdb/pull/3776 [2] https://github.com/apache/iotdb/pull/3777 [3] https://issues.apache.org/jira/projects/IOTDB/issues/IOTDB-1496 [4] https://issues.apache.org/jira/projects/IOTDB/issues/IOTDB-1569 Thanks, ?????????????????????????????????? Haiming Zhu ------------------ ???????? ------------------ ??????: "dev" <[email protected]>; ????????: 2021??8??16??(??????) ????5:18 ??????: "IoTDB????????"<[email protected]>; ????: Timed flush memtable and timed close TsFileProcessor Hi, If we open a TsFileProcessor and then never write to it, this TsFileProcessor will become a zombie because nothing can trigger a flush or close operation. So, i am implementing timed flush memtable [1] and timed close TsFileProcessor [2]. Timed flush memtable is used to flush inactive memtables periodically according to their created time. Timed close TsFileProcessor is used to close inactive TsFileProcessors&nbsp;periodically according to how long they don't process write operations. I add 9 params to control above two functions, and these 9 params all can be hot loaded by 'load configuration' command. 1. enable_timed_flush_seq_memtable: whether to timed flush sequence tsfiles' memtables. 2. seq_memtable_flush_interval_in_ms: when a sequence memTable's created time is older than current time minus this, the memtable is flushed to disk. 3. seq_memtable_flush_check_interval_in_ms: the interval to check whether the sequence&nbsp;memtables need flushing. 4. enable_timed_flush_unseq_memtable: whether to timed flush unsequence tsfiles' memtables. 5. unseq_memtable_flush_interval_in_ms: when an unsequence memTable's created time is older than current time minus this, the memtable is flushed to disk. 6. unseq_memtable_flush_check_interval_in_ms: the interval to check whether the unsequence&nbsp;memtables need flushing. 7. enable_timed_close_tsfile: whether to timed close TsfileProcessors. 8.&nbsp;close_tsfile_interval_after_flushing_in_ms: when TsfileProcessor's last working memtable flush time is older than current time minus this and its working memtable is null, this TsfileProcessor will be closed. 9.&nbsp;close_tsfile_check_interval_in_ms: the interval to check whether the TsfileProcessors&nbsp;need closing. What do you think? [1]&nbsp;https://issues.apache.org/jira/projects/IOTDB/issues/IOTDB-1496 [2] https://issues.apache.org/jira/projects/IOTDB/issues/IOTDB-1569 Thanks, ?????????????????????????????????? Haiming Zhu
