Re: [DISCUSSION]Merge index property and operations improvement.

2020-11-27 Thread akashrn5
Hi,

final points to be considered are:

1. make merge index by default enable and fail the compaction, load if the
merge index operation fails.
2. Merge index property can be removed completely, if any developer wants to
check something code will be simpler to add some check to skip the merge
index.
3. update table status file and the segment file only if the merge index
operation succeeds.
4. Keep retry count max for table status updation of success and have retry
count logic for merge index also. 
5. As david suggested, merge index listener will be removed and cab be added
as a logic in load and compaction transactions
6. Alter compact command for merge index will not do anything for new
tables. It will merge for old tables only and do not delete the old index
files immediately

These will be considered in PRs, please see if i had missed something.

Thanks

Regards
Akash R 



--
Sent from: 
http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/


Re: [DISCUSSION]Improve Simple updates and delete performance in carbondata

2020-11-27 Thread David CaiQiang
hi Akash, for the simple update case, can you do a test to confirm your
inference after a fast change?



-
Best Regards
David Cai
--
Sent from: 
http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/