Improve carbondata CDC performance

2021-02-07 Thread Akash r
Hi community, The Carbondata’s CDC is an important feature as CDC is an important use case in data analytics of merging the source data changes to target table. With the current design of CDC, the performance is not good when the data is huge in the target table and input source data also. This ma

Re: [DISCUSSION] Display the segment ID when carbondata load is successful

2021-02-07 Thread akashrn5
hi, i think still the auto compaction after load is not async, plan is there to make it async. But according to me, we should give back current segment ID and if its merged to some segment we should say that , "X" is the segment ID loaded and its been merged to "Y" segment, so that user can take d