[GitHub] [hbase] anoopsjohn commented on a change in pull request #1783: HBASE-24436 The store file open and close thread pool should be share…

[GitBox]

anoopsjohn commented on a change in pull request #1783:
URL: https://github.com/apache/hbase/pull/1783#discussion_r430912909



##########
File path: 
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HStore.java
##########
@@ -555,15 +555,24 @@ void setDataBlockEncoderInTest(HFileDataBlockEncoder 
blockEncoder) {
     }
     // initialize the thread pool for opening store files in parallel..
     ThreadPoolExecutor storeFileOpenerThreadPool =
-      this.region.getStoreFileOpenAndCloseThreadPool("StoreFileOpener-"
-        + this.region.getRegionInfo().getEncodedName() + "-" + 
this.getColumnFamilyName());
+      this.region.getStoreFileOpenAndCloseThreadPool();
     CompletionService<HStoreFile> completionService =
       new ExecutorCompletionService<>(storeFileOpenerThreadPool);
-
+    String executingThreadNamePrefix = "StoreFileOpener-"
+      + this.region.getRegionInfo().getEncodedName() + "-" + 
this.getColumnFamilyName();
+    AtomicInteger threadId = new AtomicInteger(0);
     int totalValidStoreFile = 0;
     for (StoreFileInfo storeFileInfo : files) {
       // open each store file in parallel
-      completionService.submit(() -> 
this.createStoreFileAndReader(storeFileInfo));
+      completionService.submit(() -> {
+        Thread t = Thread.currentThread();
+        String name = t.getName();
+        Thread.currentThread().setName(executingThreadNamePrefix + "-"

Review comment:
       This is done to dynamically change the thread name as per the Store name?

##########
File path: 
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
##########
@@ -1769,7 +1771,7 @@ public void setTimeoutForWriteLock(long 
timeoutForWriteLock) {
           storeCloserThreadPool.shutdownNow();
         }
       }
-
+      storeFileOpenAndCloseThreadPool.shutdownNow();

Review comment:
       This means keeping this 'storeFileOpenAndCloseThreadPool' active till 
the Region is closed. 
   previously we create the pool at the region open time and once the stores 
are opened, the pool is shutdown. Keeping it longer this way not looks good

##########
File path: 
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HStore.java
##########
@@ -955,19 +964,24 @@ private void bulkLoadHFile(HStoreFile sf) throws 
IOException {
       if (!result.isEmpty()) {
         // initialize the thread pool for closing store files in parallel.
         ThreadPoolExecutor storeFileCloserThreadPool = this.region
-            .getStoreFileOpenAndCloseThreadPool("StoreFileCloser-"
-              + this.region.getRegionInfo().getEncodedName() + "-" + 
this.getColumnFamilyName());
-
+            .getStoreFileOpenAndCloseThreadPool();
+        String executingThreadNamePrefix = "StoreFileCloser-"
+          + this.region.getRegionInfo().getEncodedName() + "-" + 
this.getColumnFamilyName();
+        AtomicInteger threadId = new AtomicInteger(0);
         // close each store file in parallel
         CompletionService<Void> completionService =
           new ExecutorCompletionService<>(storeFileCloserThreadPool);
         for (HStoreFile f : result) {
           completionService.submit(new Callable<Void>() {
             @Override
             public Void call() throws IOException {
+              Thread t = Thread.currentThread();

Review comment:
       All these looks bit hacky




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org