This mechanism will work fine for LOCAL_SORT loading of big data and the small cluster with big executor.
If it doesn't match these conditions, better consider a new solution to adapter the generic scenario. I suggest re-factoring NO_SORT, maybe we can check and improve the global_sort solution. The solution should support both NO_SORT and GLOBAL_SORT, and automatically determines the number of partitions to avoid small file issue. ----- Best Regards David Cai -- Sent from: http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/
