Hi, I have almost finished refactor the TsFileProcessor module. ( https://issues.apache.org/jira/browse/IOTDB-58)
TsFileProcessor is aim to replace BufferWriteProcessor and some functions of FileNodeProcessor. TsFileProcessor manages the ordered data in memtable and all complete TsFiles. it only provide insert/update/delete/close/flush and query APIs. - For Insert, if it returns false, then it means that the inserted data should not be managed by TsFileProcessor, which means it is an overflowed data. - For query, the function will query data not only from the memtable but also from TsFiles. - The parameter of Insert() is InsertPlan, rather than (deviceID, time, measurements, values). It is to reduce the class transformation... (Another way to optimize it is do not use InsertPlan in WAL..., which may be better) WAL is not enabled in this module. I will add the function when all related codes is refactored. Now the codes are not used in IoTDB's process, you can try it by run TsFileProcessorTest.java. When the overflowProcessor and FileNodeProcessor is refactored, we can enable this part of codes. The branch is: refactor_bufferwrite, I will submit a PR later to let interested guys review it. Then I will refactor the module of OverflowProcessor. Best, ----------------------------------- Xiangdong Huang School of Software, Tsinghua University 黄向东 清华大学 软件学院 Willem Jiang <[email protected]> 于2019年3月9日周六 上午10:31写道： > Hi Xiangdong, > > I think you can keep everyone posted by creating a refactoring breach > in the repo. > In this way it will save you some energy when you merge the PR from > other developer, as other developer can create their PR which is based > on your latest change. > > Willem Jiang > > Twitter: willemjiang > Weibo: 姜宁willem > > On Sat, Mar 9, 2019 at 12:09 AM Xiangdong Huang <[email protected]> > wrote: > > > > Hi, > > > > The two class, FileNodeManager and FileNodeProcessor, are too large (more > > than 3000 lines totally), and the logic is complex. But these two classes > > play very very important roles in IoTDB. > > > > Now, I decide to refactor the two classes to try to make it more clear > and > > guarantee the logic is optimized. It may spend about 1 ~ 2 weeks. > > > > So, > > > > 1) I will close PR #77 (rename FileNodeManager to StorageGroupManager) > > without merge. > > 2) If some new coming PR will modify the logic of the two classes, please > > let me know. I can merge your PR to avoid heavy conflict of codes... > > > > Best, > > > > ----------------------------------- > > Xiangdong Huang > > School of Software, Tsinghua University > > > > 黄向东 > > 清华大学 软件学院 >
