[GitHub] [carbondata] ajantha-bhat commented on a change in pull request #3847: [CARBONDATA-3906] Optimize sort performance in writting file

[GitBox]

ajantha-bhat commented on a change in pull request #3847:
URL: https://github.com/apache/carbondata/pull/3847#discussion_r460765555



##########
File path: 
processing/src/main/java/org/apache/carbondata/processing/loading/sort/impl/UnsafeParallelReadMergeSorterWithColumnRangeImpl.java
##########
@@ -99,6 +101,8 @@ public void initialize(SortParameters sortParameters) {
     UnsafeSortDataRows[] sortDataRows = new 
UnsafeSortDataRows[columnRangeInfo.getNumOfRanges()];
     intermediateFileMergers = new 
UnsafeIntermediateMerger[columnRangeInfo.getNumOfRanges()];
     SortParameters[] sortParameterArray = new 
SortParameters[columnRangeInfo.getNumOfRanges()];
+    this.writeService = 
Executors.newFixedThreadPool(originSortParameters.getNumberOfCores(),

Review comment:
       @kevinjmh : Yes, If cores are available adding threads horizontally can 
speedup not just sort, but other steps in data loading also.
   If cores are not available, adding threads vertically also no use as they 
will end up waiting for cpu. 
   
   so, I felt. This PR changes not required and user can increase 
`carbon.number.of.cores.while.loading`




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org