Re: Concurrent data loading issues

2019-11-12 Thread Chin Wei
Thanks for your response. I am running manual compaction, basically the flow are: 1. Multiple (4) threads that running the load data command (each load is 100k rows). I am loading data continuously up to 10,000 csv files. 2. For every 10 load data command, calling the compaction command in another

Re: Concurrent data loading issues

2019-11-12 Thread xuchuanyin
Hi, concurrent load will not cause the problem and I've tried that months ago. Seen from the log, it seems that the problem lies in Compation that automatically triggered after loading. To solve the problem, I think you can: 1. firstly turn off auto-compaction to increase loading performance, and