runzhiwang commented on a change in pull request #709: HDDS-3244. Improve write efficiency by opening RocksDB only once URL: https://github.com/apache/hadoop-ozone/pull/709#discussion_r397568527
########## File path: hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/keyvalue/helpers/KeyValueContainerUtil.java ########## @@ -73,14 +71,6 @@ public static void createContainerMetaData(File containerMetaDataPath, File throw new IOException("Unable to create directory for metadata storage." + " Path: " + containerMetaDataPath); } - MetadataStore store = MetadataStoreBuilder.newBuilder().setConf(conf) - .setCreateIfMissing(true).setDbFile(dbFile).build(); Review comment: @bshashikant @mukul1987 1. From the code, the [future thread](https://github.com/apache/hadoop-ozone/blob/master/hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/transport/server/ratis/ContainerStateMachine.java#L672) do `getCachedStateMachineData ` in `readStateMachineData `and the [future thread](https://github.com/apache/hadoop-ozone/blob/master/hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/transport/server/ratis/ContainerStateMachine.java#L459) do `createContainer` in `writeStateMachineData `are the same [thread](https://github.com/apache/hadoop-ozone/blob/master/hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/transport/server/ratis/ContainerStateMachine.java#L505). Because `writeStateMachineData `called before `readStateMachineData`. So leader must wait `createContainer `finish then `getCachedStateMachineData `and append logs to the follower, so leader and follower are not independent, follower must wait leader finish `createContainer`. **How to improve it:** I think this order can be improved by distinguishing the thread used by `getCachedStateMachineData ` and `createContainer `, and if [stateMachineDataCache.get(logIndex)](https://github.com/apache/hadoop-ozone/blob/master/hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/transport/server/ratis/ContainerStateMachine.java#L617) does not return null, leader can get stateMachineData from cache and need not wait createContainer finish, thus leader and follower can be independent. But if it return null, leader must finish createContaer than apennd logs to the follower. So I think [data = readStateMachineData(requestProto, term, logIndex)](https://github.com/apache/hadoop-ozone/blob/master/hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/transport/server/ratis/ContainerStateMachine.java#L619) should use the same thread with `createContainer` rather than the whole [getCachedStateMachineData](https://github.com/apache/hadoop-ozone/blob/master/hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/transport/server/ratis/ContainerStateMachine.java#L614). What do you think ? 2. From the jaeger UI, you can also see follower create container after leader finishing it. ![image](https://user-images.githubusercontent.com/51938049/77489346-dbbac900-6e72-11ea-89e7-70b9f313e991.png) ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org