saintstack commented on a change in pull request #2352:
URL: https://github.com/apache/hbase/pull/2352#discussion_r485151085
##########
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/AssignmentManager.java
##########
@@ -1556,6 +1556,7 @@ public RegionInfo loadRegionFromMeta(String
regionEncodedName) throws UnknownReg
}
private void loadMeta() throws IOException {
+ master.getTableStateManager().asyncLoadTableState();
Review comment:
We are doing a full async scan of meta on the table state column
followed by another full scan of the meta table?
Why not just do one table scan that gets both table state and whatever is
done below in visitMeta visitor? If the table state is in the first row found
when we cross into a new table, and at this time, we save the table state to
the TableStateManager cache, won't this have us go to cache on all subsequent
table lookups: i.e. less machinery to obtain your objective above?
##########
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/master/TableStateManager.java
##########
@@ -318,4 +326,39 @@ protected void deleteZooKeeper(TableName tableName) {
LOG.warn("Failed deleting table state from zookeeper", e);
}
}
+
+ public void asyncLoadTableState() {
+ new Thread(() -> {
+ LOG.info("load table state...");
+ ThreadFactoryBuilder builder = new
ThreadFactoryBuilder().setDaemon(true).setNameFormat("TableStateFetcher" +
"-%1$d");
+ ExecutorService
+ executorService =
Executors.newFixedThreadPool(master.getConfiguration().getInt("hbase.table.state.fetcher.threads",10),
builder.build());
+
+ try {
+ List<Result> results =
MetaTableAccessor.fullScan(master.getConnection(),
ClientMetaTableAccessor.QueryType.TABLE);
+ List<Future<Void>> futures = new ArrayList<>();
+ results.forEach(r -> futures.add(executorService.submit(new
Callable<Void>() {
Review comment:
This is a Scan of the Table column in meta? Why we have an executor w/
pool of ten threads to process the result of the Read?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
