JacksonYao287 commented on a change in pull request #3024:
URL: https://github.com/apache/ozone/pull/3024#discussion_r822582889
##########
File path:
hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/OMConfigKeys.java
##########
@@ -292,4 +292,8 @@ private OMConfigKeys() {
public static final long OZONE_OM_ADMIN_PROTOCOL_WAIT_BETWEEN_RETRIES_DEFAULT
= 1000;
+ public static final String OZONE_OM_MAX_PENDING_REQ_COUNT =
+ "ozone.om.max.pending.req.count";
+ public static final int OZONE_OM_MAX_PENDING_REQ_COUNT_DEFAULT = 10000;
Review comment:
in fact , the object cached in the `current queue` and `ready queue` of
`ozoneDoubleBuffer` is `DoubleBufferEntry` and it has a fixed size 24 bytes.
```
2022-03-09 17:37:01,670 [OM StateMachine ApplyTransaction Thread - 0] INFO
org.apache.hadoop.ozone.om.ratis.OzoneManagerDoubleBuffer: DoubleBufferEntry
size: org.apache.hadoop.ozone.om.ratis.helpers.DoubleBufferEntry object
internals:
OFFSET SIZE TYPE
DESCRIPTION VALUE
0 4 (object
header) 05 00 00 00 (00000101 00000000 00000000
00000000) (5)
4 4 (object
header) 00 00 00 00 (00000000 00000000 00000000
00000000) (0)
8 4 (object
header) 58 b9 9c 02 (01011000 10111001 10011100
00000010) (43825496)
12 4 org.apache.hadoop.ozone.om.response.OMClientResponse
DoubleBufferEntry.response (object)
16 8 long
DoubleBufferEntry.trxLogIndex 25
Instance size: 24 bytes\
```
so , 10000 DoubleBufferEntry will take about 256MB memory, it is small
enough for almost all the physical machine.
i have test the size from 10000 to 300000, and i find the performance almost
does not change. what is more, i find a case that the size of double buffer
flush batch is 200000, and the flush option take about 10 senconds. thus i
think this value should not be too big and i think 10000 maybe a suitable
value. do you have any suggesstion about this?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
