Hi,
Param seq_memtable_flush_check_interval_in_ms and unseq_memtable_flush_check_interval_in_ms aim to make timed flush memtable configuration more flexible. If users don't have this requirement, i will merge both params into one param memtable_flush_check_interval_in_ms. Thanks, ?????????????????????????????????? Haiming Zhu ------------------ ???????? ------------------ ??????: "dev" <hhao...@outlook.com>; ????????: 2021??8??19??(??????) ????11:05 ??????: "dev-iotdb"<dev@iotdb.apache.org>; ????: Re: Timed flush memtable and timed close TsFileProcessor Hi, I think the param number 3 and number 6 can be merged into one parameter. The checking interval can be the same value for both sequence and unsequence memtables. I don??t think any users will try to set different value for them. How do you think? Thanks, Haonan Hou > On Aug 19, 2021, at 6:04 PM, Jialin Qiao <qiaojia...@apache.org> wrote: > > Hi, > >> I am confused whether to open timed flush seq/unseq memtable and timed > close TsFileProcessor by default, because they can save memory in some user > cases while they just waste cpu in other user cases. > > It could be enabled by default, both seq and unseq. > The 12 hours for memtable flush interval and close file interval is fine. > If it really kept 12 hours in memory, the CPU load should be very low, no > need to worry. > > Thanks, > ?????????????????????????????????? > Jialin Qiao > School of Software, Tsinghua University > > ?????? > ???????? ???????? > > > Haiming Zhu <zhuh...@qq.com.invalid> ??2021??8??19?????? ????4:37?????? > >> Hi, >> >> >> I have already implemented both issues [1][2]. >> >> >> Timed flush memtable [3] is used to flush inactive memtables periodically >> according to their created time. For example, if we set >> seq_memtable_flush_interval_in_ms to 1000, memtable A, created at time >> 2000, can be flushed after time 3000. >> >> >> Timed close TsFileProcessor [4] is used to close inactive TsFileProcessors >> periodically according to how long they don't process write operations. For >> example, if we set close_tsfile_interval_after_flushing_in_ms to 1000, >> TsFileProcessor A, whose last workMemtable was flush at time 2000 and >> current workMemtable is null (i.e., no more wirite operations after last >> flush), can be closed after time 3000. >> >> >> But, i am confused whether to open timed flush seq/unseq memtable and >> timed close TsFileProcessor by default, because they can save memory in >> some user cases while they just waste cpu in other user cases. >> >> >> Look forward to your reply. >> >> >> [1]&nbsp;https://github.com/apache/iotdb/pull/3776 >> [2]&nbsp;https://github.com/apache/iotdb/pull/3777 >> >> [3]&nbsp;https://issues.apache.org/jira/projects/IOTDB/issues/IOTDB-1496 >> [4]&nbsp;https://issues.apache.org/jira/projects/IOTDB/issues/IOTDB-1569 >> >> >> Thanks, >> ?????????????????????????????????? >> Haiming Zhu >> >> >> >> >> ------------------&nbsp;????????&nbsp;------------------ >> ??????: >> "dev" >> >> <zhuh...@qq.com.INVALID&gt;; >> ????????:&nbsp;2021??8??16??(??????) ????5:18 >> ??????:&nbsp;"IoTDB????????"<dev@iotdb.apache.org&gt;; >> >> ????:&nbsp;Timed flush memtable and timed close TsFileProcessor >> >> >> >> Hi, >> >> >> If we open a TsFileProcessor and then never write to it, this >> TsFileProcessor will become a zombie because nothing can trigger a flush or >> close operation. So, i am implementing timed flush memtable [1] and timed >> close TsFileProcessor [2]. Timed flush memtable is used to flush inactive >> memtables periodically according to their created time. Timed close >> TsFileProcessor is used to close inactive >> TsFileProcessors&amp;nbsp;periodically according to how long they don't >> process write operations. >> >> >> I add 9 params to control above two functions, and these 9 params all can >> be hot loaded by 'load configuration' command. >> >> >> 1. enable_timed_flush_seq_memtable: whether to timed flush sequence >> tsfiles' memtables. >> 2. seq_memtable_flush_interval_in_ms: when a sequence memTable's created >> time is older than current time minus this, the memtable is flushed to disk. >> 3. seq_memtable_flush_check_interval_in_ms: the interval to check whether >> the sequence&amp;nbsp;memtables need flushing. >> 4. enable_timed_flush_unseq_memtable: whether to timed flush unsequence >> tsfiles' memtables. >> 5. unseq_memtable_flush_interval_in_ms: when an unsequence memTable's >> created time is older than current time minus this, the memtable is flushed >> to disk. >> 6. unseq_memtable_flush_check_interval_in_ms: the interval to check >> whether the unsequence&amp;nbsp;memtables need flushing. >> 7. enable_timed_close_tsfile: whether to timed close TsfileProcessors. >> 8.&amp;nbsp;close_tsfile_interval_after_flushing_in_ms: when >> TsfileProcessor's last working memtable flush time is older than current >> time minus this and its working memtable is null, this TsfileProcessor will >> be closed. >> 9.&amp;nbsp;close_tsfile_check_interval_in_ms: the interval to check >> whether the TsfileProcessors&amp;nbsp;need closing. >> >> >> What do you think? >> >> >> [1]&amp;nbsp; >> https://issues.apache.org/jira/projects/IOTDB/issues/IOTDB-1496 >> [2] https://issues.apache.org/jira/projects/IOTDB/issues/IOTDB-1569 >> >> >> Thanks, >> ?????????????????????????????????? >> Haiming Zhu >