Merge PR #82 and #85 notification

2019-03-05 Thread Xiangdong Huang
Hi, PR #82 is for fixing some issues reported by Sonar. PR #85 is for add a tool for checking whether all WAL files are readable. They are opened more than 3 days and someone has reviewed them on Github, I will merge them tomorrow if there is no more questions. Best,

AW: Operation and robustness of iotDB

2019-03-05 Thread Julian Feinauer
Hi Xiangdong, Great work! I'll try to go through your code to understand the internals better and perhaps do a simulation test with hard jvm exits. If everything goes fine we will incorporate that asap in test code and give you feedback! Thank you! Julian Von meinem Mobiltelefon gesendet

Re: Operation and robustness of iotDB

2019-03-05 Thread Xiangdong Huang
Hi, I have added a new TsFileIOWriter, which supports to recover data from a broken TsFile (I mean, a TsFile that is not closed correctly). The PR is https://github.com/apache/incubator-iotdb/pull/87. You can use the new feature by: ``` NativeRestorableIOWriter rWriter = new

Re: Operation and robustness of iotDB

2019-03-05 Thread Julian Feinauer
Hey, thank you fort he link... I did not know of this.. this is exactly what I was looking for! Julian PS.: Looking forward to your PR : ) Am 05.03.19, 12:26 schrieb "Xiangdong Huang" : Hi, 1. We have a document to introduce that:

Re: Operation and robustness of iotDB

2019-03-05 Thread Xiangdong Huang
Hi, 1. We have a document to introduce that: https://cwiki.apache.org/confluence/display/IOTDB/TsFile+Format 2. The new API for recovering data is almost done. I am writing the UTs now. Maybe I can submit a PR tonight (if everything is fine...) Best, ---

Re: Operation and robustness of iotDB

2019-03-05 Thread Julian Feinauer
Hi Xiangdong, that sounds excellent. Do you have a short overview of how the file format is designed on disk? I know that its somewhat similar to parquet but I did not find more details. Basically what would suffice for us would be something like skipping an invalid column group (or how you name