Hi Xiangzhi, Thank you for sharing your work on the load tsfile feature. I am currently working on the Load function as well, and I have a question for you. Have you considered enhancing the Load function directly within the core code, rather than relying on the script to load tsfiles one by one? Additionally , do you think there would be a significant performance difference between the original batch load method and your script of loading files individually?
Your insights on this matter would be very valuable, and I believe it could lead to further optimizations! Best regards, Itami Sho > 2024年5月30日 11:11，孟祥志 <[email protected]> 写道： > > Hello everyone, > > I am Xiangzhi Meng, a new contributor to Apache IoTDB. I am excited to share > with you a feature that I have been working on for the past few months. > > load tsfile has always been an important feature for IoTDB, but due to the > original design, he has some flaws in it, when load a folder fails to import > some tsfile, it may cause the whole task fails, which is obviously a not very > user-friendly design, so I decided to modify his. > > I'll modify the original load tsfile script to traverse the entire folder > first, find all the files, and then use the LOAD function to import > individual files; and I've added a few interesting changes along the way, > modifying the successful operation parameter to make it easier for people to > deal with successful files, and added a parameter for failed files to make it > easier for people to find those failed files, and finally added a new number > of threads to make the LOAD function work more quickly. > > > I hope you are interested in this feature and would like to participate in > the development and testing. You can also leave your comments and suggestions > in this thread. Appreciate any suggestion/feedback & contribution. > > Thank you for your attention and support. > > Best regards, > > Xiangzhi Meng > > Reference: > https://github.com/apache/iotdb/pull/12541
