[jira] [Updated] (HDDS-14379) Implement basic Hadoop OM client proxy provider to read from followers
[ https://issues.apache.org/jira/browse/HDDS-14379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz-wo Sze updated HDDS-14379: -- Fix Version/s: 2.2.0 Resolution: Fixed Status: Resolved (was: Patch Available) The pull request is now merged. Thanks, [~ivanandika]! > Implement basic Hadoop OM client proxy provider to read from followers > -- > > Key: HDDS-14379 > URL: https://issues.apache.org/jira/browse/HDDS-14379 > Project: Apache Ozone > Issue Type: Sub-task > Components: OM HA, Ozone Client >Reporter: Ivan Andika >Assignee: Ivan Andika >Priority: Major > Labels: pull-request-available > Fix For: 2.2.0 > > > This task is to come up with the basic implementation of follower read client > proxy as a baseline before further performance improvements. The idea is to > simply pick a OM node in random (which be leader or follower / listener) and > use it to submit read requests. If the chosen OM node is a follower, the read > requests need to keep sending to that OM node unless the OM is down which > triggers failover. Write requests should be sent to the OM leader directly. > Further improvements such as followers priority or picking OM based on > latency, applied index, etc will be implemented in follow up tasks. > The implementation is to introduce HadoopRpcOMFollowerReadProxyProvider which > wraps > HadoopRpcOMFailoverProxyProvider. FollowerReadProxyProvider tracks a > different currentOmNodeId from HadoopRpcOMFailoverProxyProvider. > FollowerReadInvocationHandler will check whether the request is a read > request (using OmUtils#isReadOnly) and if so forwards it to its current > proxy. If it's a write request, the request if forwarded to > HadoopRpcOMFailoverProxyProvider to be sent to the leader. > So the proxy hierarchy (each with its own InvocationHandler) is > * TracingUtil's proxy (InvocationHandler: TraceAllMethod) > ** RetryProxy (InvocationHandler: RetryInvocationHandler) > *** *HadoopRpcOMFollowerReadProxyProvider* (InvocationHandler: > FollowerReadInvocationHandler) > * > ** > *** > > * ProtocolProxy (which is created in > OMFailoverProxyProviderBase#createOMProxy): ProtobufRpcEngine.Invoker > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
[jira] [Updated] (HDDS-14379) Implement basic Hadoop OM client proxy provider to read from followers
[ https://issues.apache.org/jira/browse/HDDS-14379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chu Cheng Li updated HDDS-14379: Status: Patch Available (was: Open) > Implement basic Hadoop OM client proxy provider to read from followers > -- > > Key: HDDS-14379 > URL: https://issues.apache.org/jira/browse/HDDS-14379 > Project: Apache Ozone > Issue Type: Sub-task > Components: OM HA, Ozone Client >Reporter: Ivan Andika >Assignee: Ivan Andika >Priority: Major > Labels: pull-request-available > > This task is to come up with the basic implementation of follower read client > proxy as a baseline before further performance improvements. The idea is to > simply pick a OM node in random (which be leader or follower / listener) and > use it to submit read requests. If the chosen OM node is a follower, the read > requests need to keep sending to that OM node unless the OM is down which > triggers failover. Write requests should be sent to the OM leader directly. > Further improvements such as followers priority or picking OM based on > latency, applied index, etc will be implemented in follow up tasks. > The implementation is to introduce HadoopRpcOMFollowerReadProxyProvider which > wraps > HadoopRpcOMFailoverProxyProvider. FollowerReadProxyProvider tracks a > different currentOmNodeId from HadoopRpcOMFailoverProxyProvider. > FollowerReadInvocationHandler will check whether the request is a read > request (using OmUtils#isReadOnly) and if so forwards it to its current > proxy. If it's a write request, the request if forwarded to > HadoopRpcOMFailoverProxyProvider to be sent to the leader. > So the proxy hierarchy (each with its own InvocationHandler) is > * TracingUtil's proxy (InvocationHandler: TraceAllMethod) > ** RetryProxy (InvocationHandler: RetryInvocationHandler) > *** *HadoopRpcOMFollowerReadProxyProvider* (InvocationHandler: > FollowerReadInvocationHandler) > * > ** > *** > > * ProtocolProxy (which is created in > OMFailoverProxyProviderBase#createOMProxy): ProtobufRpcEngine.Invoker > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
[jira] [Updated] (HDDS-14379) Implement basic Hadoop OM client proxy provider to read from followers
[ https://issues.apache.org/jira/browse/HDDS-14379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HDDS-14379: -- Labels: pull-request-available (was: ) > Implement basic Hadoop OM client proxy provider to read from followers > -- > > Key: HDDS-14379 > URL: https://issues.apache.org/jira/browse/HDDS-14379 > Project: Apache Ozone > Issue Type: Sub-task > Components: OM HA, Ozone Client >Reporter: Ivan Andika >Assignee: Ivan Andika >Priority: Major > Labels: pull-request-available > > This task is to come up with the basic implementation of follower read client > proxy as a baseline before further performance improvements. The idea is to > simply pick a OM node in random (which be leader or follower / listener) and > use it to submit read requests. If the chosen OM node is a follower, the read > requests need to keep sending to that OM node unless the OM is down which > triggers failover. Write requests should be sent to the OM leader directly. > Further improvements such as followers priority or picking OM based on > latency, applied index, etc will be implemented in follow up tasks. > The implementation is to introduce HadoopRpcOMFollowerReadProxyProvider which > wraps > HadoopRpcOMFailoverProxyProvider. FollowerReadProxyProvider tracks a > different currentOmNodeId from HadoopRpcOMFailoverProxyProvider. > FollowerReadInvocationHandler will check whether the request is a read > request (using OmUtils#isReadOnly) and if so forwards it to its current > proxy. If it's a write request, the request if forwarded to > HadoopRpcOMFailoverProxyProvider to be sent to the leader. > So the proxy hierarchy (each with its own InvocationHandler) is > * TracingUtil's proxy (InvocationHandler: TraceAllMethod) > ** RetryProxy (InvocationHandler: RetryInvocationHandler) > *** *HadoopRpcOMFollowerReadProxyProvider* (InvocationHandler: > FollowerReadInvocationHandler) > * > ** > *** > > * ProtocolProxy (which is created in > OMFailoverProxyProviderBase#createOMProxy): ProtobufRpcEngine.Invoker > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
[jira] [Updated] (HDDS-14379) Implement basic Hadoop OM client proxy provider to read from followers
[ https://issues.apache.org/jira/browse/HDDS-14379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Andika updated HDDS-14379: --- Parent: HDDS-14424 Issue Type: Sub-task (was: New Feature) > Implement basic Hadoop OM client proxy provider to read from followers > -- > > Key: HDDS-14379 > URL: https://issues.apache.org/jira/browse/HDDS-14379 > Project: Apache Ozone > Issue Type: Sub-task > Components: OM HA, Ozone Client >Reporter: Ivan Andika >Assignee: Ivan Andika >Priority: Major > > This task is to come up with the basic implementation of follower read client > proxy as a baseline before further performance improvements. The idea is to > simply pick a OM node in random (which be leader or follower / listener) and > use it to submit read requests. If the chosen OM node is a follower, the read > requests need to keep sending to that OM node unless the OM is down which > triggers failover. Write requests should be sent to the OM leader directly. > Further improvements such as followers priority or picking OM based on > latency, applied index, etc will be implemented in follow up tasks. > The implementation is to introduce HadoopRpcOMFollowerReadProxyProvider which > wraps > HadoopRpcOMFailoverProxyProvider. FollowerReadProxyProvider tracks a > different currentOmNodeId from HadoopRpcOMFailoverProxyProvider. > FollowerReadInvocationHandler will check whether the request is a read > request (using OmUtils#isReadOnly) and if so forwards it to its current > proxy. If it's a write request, the request if forwarded to > HadoopRpcOMFailoverProxyProvider to be sent to the leader. > So the proxy hierarchy (each with its own InvocationHandler) is > * TracingUtil's proxy (InvocationHandler: TraceAllMethod) > ** RetryProxy (InvocationHandler: RetryInvocationHandler) > *** *HadoopRpcOMFollowerReadProxyProvider* (InvocationHandler: > FollowerReadInvocationHandler) > * > ** > *** > > * ProtocolProxy (which is created in > OMFailoverProxyProviderBase#createOMProxy): ProtobufRpcEngine.Invoker > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
[jira] [Updated] (HDDS-14379) Implement basic Hadoop OM client proxy provider to read from followers
[ https://issues.apache.org/jira/browse/HDDS-14379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Andika updated HDDS-14379: --- Description: This task is to come up with the basic implementation of follower read client proxy as a baseline before further performance improvements. The idea is to simply pick a OM node in random (which be leader or follower / listener) and use it to submit read requests. If the chosen OM node is a follower, the read requests need to keep sending to that OM node unless the OM is down which triggers failover. Write requests should be sent to the OM leader directly. Further improvements such as adding OM probe logic will be implemented in follow up tasks. The implementation is to introduce HadoopRpcOMFollowerReadProxyProvider which wraps HadoopRpcOMFailoverProxyProvider. FollowerReadProxyProvider tracks a different currentOmNodeId from HadoopRpcOMFailoverProxyProvider. FollowerReadInvocationHandler will check whether the request is a read request (using OmUtils#isReadOnly) and if so forwards it to its current proxy. If it's a write request, the request if forwarded to HadoopRpcOMFailoverProxyProvider to be sent to the leader. So the proxy hierarchy (each with its own InvocationHandler) is * TracingUtil's proxy (InvocationHandler: TraceAllMethod) ** RetryProxy (InvocationHandler: RetryInvocationHandler) *** *HadoopRpcOMFollowerReadProxyProvider* (InvocationHandler: FollowerReadInvocationHandler) * ProtocolProxy (which is created in OMFailoverProxyProviderBase#createOMProxy): ProtobufRpcEngine.Invoker was: This task is to come up with the basic implementation of follower read client proxy as a baseline before further performance improvements. The idea is to simply pick a OM node in random (which be leader or follower / listener) and use it to submit read requests. If the chosen OM node is a follower, the read requests need to keep sending to that OM node unless the OM is down which triggers failover. Write requests should be sent to the OM leader directly. Further improvements such as adding OM probe logic will be implemented in follow up tasks. The implementation is to introduce HadoopRpcOMFollowerReadProxyProvider which wraps HadoopRpcOMFailoverProxyProvider. FollowerReadProxyProvider tracks a different currentOmNodeId from HadoopRpcOMFailoverProxyProvider. FollowerReadInvocationHandler will check whether the request is a read request (using OmUtils#isReadOnly) and if so forwards it to its current proxy. If it's a write request, the request if forwarded to HadoopRpcOMFailoverProxyProvider to be sent to the leader. So the proxy hierarchy is * TracingProxy ** RetryProxy *** HadoopRpcOMFollowerReadProxyProvider HadoopRpcOMFailoverProxyProvider * ProtocolProxy (from ProtobufRpcEngine) > Implement basic Hadoop OM client proxy provider to read from followers > -- > > Key: HDDS-14379 > URL: https://issues.apache.org/jira/browse/HDDS-14379 > Project: Apache Ozone > Issue Type: New Feature > Components: OM HA, Ozone Client >Reporter: Ivan Andika >Assignee: Ivan Andika >Priority: Major > > This task is to come up with the basic implementation of follower read client > proxy as a baseline before further performance improvements. The idea is to > simply pick a OM node in random (which be leader or follower / listener) and > use it to submit read requests. If the chosen OM node is a follower, the read > requests need to keep sending to that OM node unless the OM is down which > triggers failover. Write requests should be sent to the OM leader directly. > Further improvements such as adding OM probe logic will be implemented in > follow up tasks. > The implementation is to introduce HadoopRpcOMFollowerReadProxyProvider which > wraps > HadoopRpcOMFailoverProxyProvider. FollowerReadProxyProvider tracks a > different currentOmNodeId from HadoopRpcOMFailoverProxyProvider. > FollowerReadInvocationHandler will check whether the request is a read > request (using OmUtils#isReadOnly) and if so forwards it to its current > proxy. If it's a write request, the request if forwarded to > HadoopRpcOMFailoverProxyProvider to be sent to the leader. > So the proxy hierarchy (each with its own InvocationHandler) is > * TracingUtil's proxy (InvocationHandler: TraceAllMethod) > ** RetryProxy (InvocationHandler: RetryInvocationHandler) > *** *HadoopRpcOMFollowerReadProxyProvider* (InvocationHandler: > FollowerReadInvocationHandler) > > * ProtocolProxy (which is created in > OMFailoverProxyProviderBase#createOMProxy): ProtobufRpcEngine.Invoker > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HDDS-14379) Implement basic Hadoop OM client proxy provider to read from followers
[ https://issues.apache.org/jira/browse/HDDS-14379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Andika updated HDDS-14379: --- Description: This task is to come up with the basic implementation of follower read client proxy as a baseline before further performance improvements. The idea is to simply pick a OM node in random (which be leader or follower / listener) and use it to submit read requests. If the chosen OM node is a follower, the read requests need to keep sending to that OM node unless the OM is down which triggers failover. Write requests should be sent to the OM leader directly. Further improvements such as followers priority or picking OM based on latency, applied index, etc will be implemented in follow up tasks. The implementation is to introduce HadoopRpcOMFollowerReadProxyProvider which wraps HadoopRpcOMFailoverProxyProvider. FollowerReadProxyProvider tracks a different currentOmNodeId from HadoopRpcOMFailoverProxyProvider. FollowerReadInvocationHandler will check whether the request is a read request (using OmUtils#isReadOnly) and if so forwards it to its current proxy. If it's a write request, the request if forwarded to HadoopRpcOMFailoverProxyProvider to be sent to the leader. So the proxy hierarchy (each with its own InvocationHandler) is * TracingUtil's proxy (InvocationHandler: TraceAllMethod) ** RetryProxy (InvocationHandler: RetryInvocationHandler) *** *HadoopRpcOMFollowerReadProxyProvider* (InvocationHandler: FollowerReadInvocationHandler) * ** *** * ProtocolProxy (which is created in OMFailoverProxyProviderBase#createOMProxy): ProtobufRpcEngine.Invoker was: This task is to come up with the basic implementation of follower read client proxy as a baseline before further performance improvements. The idea is to simply pick a OM node in random (which be leader or follower / listener) and use it to submit read requests. If the chosen OM node is a follower, the read requests need to keep sending to that OM node unless the OM is down which triggers failover. Write requests should be sent to the OM leader directly. Further improvements such as adding OM probe logic will be implemented in follow up tasks. The implementation is to introduce HadoopRpcOMFollowerReadProxyProvider which wraps HadoopRpcOMFailoverProxyProvider. FollowerReadProxyProvider tracks a different currentOmNodeId from HadoopRpcOMFailoverProxyProvider. FollowerReadInvocationHandler will check whether the request is a read request (using OmUtils#isReadOnly) and if so forwards it to its current proxy. If it's a write request, the request if forwarded to HadoopRpcOMFailoverProxyProvider to be sent to the leader. So the proxy hierarchy (each with its own InvocationHandler) is * TracingUtil's proxy (InvocationHandler: TraceAllMethod) ** RetryProxy (InvocationHandler: RetryInvocationHandler) *** *HadoopRpcOMFollowerReadProxyProvider* (InvocationHandler: FollowerReadInvocationHandler) * ProtocolProxy (which is created in OMFailoverProxyProviderBase#createOMProxy): ProtobufRpcEngine.Invoker > Implement basic Hadoop OM client proxy provider to read from followers > -- > > Key: HDDS-14379 > URL: https://issues.apache.org/jira/browse/HDDS-14379 > Project: Apache Ozone > Issue Type: New Feature > Components: OM HA, Ozone Client >Reporter: Ivan Andika >Assignee: Ivan Andika >Priority: Major > > This task is to come up with the basic implementation of follower read client > proxy as a baseline before further performance improvements. The idea is to > simply pick a OM node in random (which be leader or follower / listener) and > use it to submit read requests. If the chosen OM node is a follower, the read > requests need to keep sending to that OM node unless the OM is down which > triggers failover. Write requests should be sent to the OM leader directly. > Further improvements such as followers priority or picking OM based on > latency, applied index, etc will be implemented in follow up tasks. > The implementation is to introduce HadoopRpcOMFollowerReadProxyProvider which > wraps > HadoopRpcOMFailoverProxyProvider. FollowerReadProxyProvider tracks a > different currentOmNodeId from HadoopRpcOMFailoverProxyProvider. > FollowerReadInvocationHandler will check whether the request is a read > request (using OmUtils#isReadOnly) and if so forwards it to its current > proxy. If it's a write request, the request if forwarded to > HadoopRpcOMFailoverProxyProvider to be sent to the leader. > So the proxy hierarchy (each with its own InvocationHandler) is > * TracingUtil's proxy (InvocationHandler: TraceAllMethod) > ** RetryProxy (InvocationHandler: RetryInvocationHandler) > *** *HadoopRpcOMFollowerReadProxyP
[jira] [Updated] (HDDS-14379) Implement basic Hadoop OM client proxy provider to read from followers
[ https://issues.apache.org/jira/browse/HDDS-14379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Andika updated HDDS-14379: --- Description: This task is to come up with the basic implementation of follower read client proxy as a baseline before further performance improvements. The idea is to simply pick a OM node in random (which be leader or follower / listener) and use it to submit read requests. If the chosen OM node is a follower, the read requests need to keep sending to that OM node unless the OM is down which triggers failover. Write requests should be sent to the OM leader directly. Further improvements such as adding OM probe logic will be implemented in follow up tasks. The implementation is to introduce HadoopRpcOMFollowerReadProxyProvider which wraps HadoopRpcOMFailoverProxyProvider. FollowerReadProxyProvider tracks a different currentOmNodeId from HadoopRpcOMFailoverProxyProvider. FollowerReadInvocationHandler will check whether the request is a read request (using OmUtils#isReadOnly) and if so forwards it to its current proxy. If it's a write request, the request if forwarded to HadoopRpcOMFailoverProxyProvider to be sent to the leader. So the proxy hierarchy is * TracingProxy ** RetryProxy *** HadoopRpcOMFollowerReadProxyProvider HadoopRpcOMFailoverProxyProvider * ProtocolProxy (from ProtobufRpcEngine) was: This task is to come up with the basic implementation of follower read client proxy as a baseline before further performance improvements. The idea is to simply pick a OM node in random (which be leader or follower / listener) and use it to submit read requests. If the chosen OM node is a follower, the read requests need to keep sending to that OM node unless the OM is down which triggers failover. Write requests should be sent to the OM leader directly. Further improvements such as adding OM probe logic will be implemented in follow up tasks. The implementation is to introduce HadoopRpcOMFollowerReadProxyProvider which wraps HadoopRpcOMFailoverProxyProvider. FollowerReadProxyProvider tracks a different currentOmNodeId from HadoopRpcOMFailoverProxyProvider. FollowerReadInvocationHandler will check whether the request is a read request (using OmUtils#isReadOnly) and if so forwards it to its current proxy. If it's a write request, the request if forwarded to HadoopRpcOMFailoverProxyProvider to be sent to the leader. So the proxy hierarchy is * TracingProxy ** RetryProxy *** HadoopRpcOMFollowerReadProxyProvider HadoopRpcOMFailoverProxyProvider > Implement basic Hadoop OM client proxy provider to read from followers > -- > > Key: HDDS-14379 > URL: https://issues.apache.org/jira/browse/HDDS-14379 > Project: Apache Ozone > Issue Type: New Feature > Components: OM HA, Ozone Client >Reporter: Ivan Andika >Assignee: Ivan Andika >Priority: Major > > This task is to come up with the basic implementation of follower read client > proxy as a baseline before further performance improvements. The idea is to > simply pick a OM node in random (which be leader or follower / listener) and > use it to submit read requests. If the chosen OM node is a follower, the read > requests need to keep sending to that OM node unless the OM is down which > triggers failover. Write requests should be sent to the OM leader directly. > Further improvements such as adding OM probe logic will be implemented in > follow up tasks. > The implementation is to introduce HadoopRpcOMFollowerReadProxyProvider which > wraps > HadoopRpcOMFailoverProxyProvider. FollowerReadProxyProvider tracks a > different currentOmNodeId from HadoopRpcOMFailoverProxyProvider. > FollowerReadInvocationHandler will check whether the request is a read > request (using OmUtils#isReadOnly) and if so forwards it to its current > proxy. If it's a write request, the request if forwarded to > HadoopRpcOMFailoverProxyProvider to be sent to the leader. > So the proxy hierarchy is > * TracingProxy > ** RetryProxy > *** HadoopRpcOMFollowerReadProxyProvider > HadoopRpcOMFailoverProxyProvider > * ProtocolProxy (from ProtobufRpcEngine) > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
[jira] [Updated] (HDDS-14379) Implement basic Hadoop OM client proxy provider to read from followers
[ https://issues.apache.org/jira/browse/HDDS-14379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Andika updated HDDS-14379: --- Description: This task is to come up with the basic implementation of follower read client proxy as a baseline before further performance improvements. The idea is to simply pick a OM node in random (which be leader or follower / listener) and use it to submit read requests. If the chosen OM node is a follower, the read requests need to keep sending to that OM node unless the OM is down which triggers failover. Write requests should be sent to the OM leader directly. Further improvements such as adding OM probe logic will be implemented in follow up tasks. The implementation is to introduce HadoopRpcOMFollowerReadProxyProvider which wraps HadoopRpcOMFailoverProxyProvider. FollowerReadProxyProvider tracks a different currentOmNodeId from HadoopRpcOMFailoverProxyProvider. FollowerReadInvocationHandler will check whether the request is a read request (using OmUtils#isReadOnly) and if so forwards it to its current proxy. If it's a write request, the request if forwarded to HadoopRpcOMFailoverProxyProvider to be sent to the leader. So the proxy hierarchy is * TracingProxy ** RetryProxy *** HadoopRpcOMFollowerReadProxyProvider HadoopRpcOMFailoverProxyProvider was: This task is to come up with the basic implementation of follower read client proxy as a baseline before further performance improvements. The idea is to simply pick a OM node in random (which be leader or follower / listener) and use it to submit read requests. If the chosen OM node is a follower, the read requests need to keep sending to that OM node unless the OM is down which triggers failover. Write requests should be sent to the OM leader directly. Further improvements such as adding OM probe logic will be implemented in follow up tasks. The implementation is to introduce HadoopRpcOMFollowerReadProxyProvider which wraps HadoopRpcOMFailoverProxyProvider. FollowerReadProxyProvider tracks a different currentOmNodeId from HadoopRpcOMFailoverProxyProvider. FollowerReadInvocationHandler will check whether the request is a read request (using OmUtils#isReadOnly) and if so forwards it to its current proxy. If it's a write request, the request if forwarded to HadoopRpcOMFailoverProxyProvider to be sent to the leader. So the proxy hierarchy is * RetryProxy ** HadoopRpcOMFollowerReadProxyProvider *** HadoopRpcOMFailoverProxyProvider > Implement basic Hadoop OM client proxy provider to read from followers > -- > > Key: HDDS-14379 > URL: https://issues.apache.org/jira/browse/HDDS-14379 > Project: Apache Ozone > Issue Type: New Feature > Components: OM HA, Ozone Client >Reporter: Ivan Andika >Assignee: Ivan Andika >Priority: Major > > This task is to come up with the basic implementation of follower read client > proxy as a baseline before further performance improvements. The idea is to > simply pick a OM node in random (which be leader or follower / listener) and > use it to submit read requests. If the chosen OM node is a follower, the read > requests need to keep sending to that OM node unless the OM is down which > triggers failover. Write requests should be sent to the OM leader directly. > Further improvements such as adding OM probe logic will be implemented in > follow up tasks. > The implementation is to introduce HadoopRpcOMFollowerReadProxyProvider which > wraps > HadoopRpcOMFailoverProxyProvider. FollowerReadProxyProvider tracks a > different currentOmNodeId from HadoopRpcOMFailoverProxyProvider. > FollowerReadInvocationHandler will check whether the request is a read > request (using OmUtils#isReadOnly) and if so forwards it to its current > proxy. If it's a write request, the request if forwarded to > HadoopRpcOMFailoverProxyProvider to be sent to the leader. > So the proxy hierarchy is > * TracingProxy > ** RetryProxy > *** HadoopRpcOMFollowerReadProxyProvider > HadoopRpcOMFailoverProxyProvider > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
[jira] [Updated] (HDDS-14379) Implement basic Hadoop OM client proxy provider to read from followers
[ https://issues.apache.org/jira/browse/HDDS-14379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Andika updated HDDS-14379: --- Component/s: OM HA Ozone Client > Implement basic Hadoop OM client proxy provider to read from followers > -- > > Key: HDDS-14379 > URL: https://issues.apache.org/jira/browse/HDDS-14379 > Project: Apache Ozone > Issue Type: New Feature > Components: OM HA, Ozone Client >Reporter: Ivan Andika >Assignee: Ivan Andika >Priority: Major > > This task is to come up with the basic implementation of follower read client > proxy as a baseline before further performance improvements. The idea is to > simply pick a OM node in random (which be leader or follower / listener) and > use it to submit read requests. If the chosen OM node is a follower, the read > requests need to keep sending to that OM node unless the OM is down which > triggers failover. Write requests should be sent to the OM leader directly. > Further improvements such as adding OM probe logic will be implemented in > follow up tasks. > The implementation is to introduce HadoopRpcOMFollowerReadProxyProvider which > wraps > HadoopRpcOMFailoverProxyProvider. FollowerReadProxyProvider tracks a > different currentOmNodeId from HadoopRpcOMFailoverProxyProvider. > FollowerReadInvocationHandler will check whether the request is a read > request (using OmUtils#isReadOnly) and if so forwards it to its current > proxy. If it's a write request, the request if forwarded to > HadoopRpcOMFailoverProxyProvider to be sent to the leader. > So the proxy hierarchy is > * RetryProxy > ** HadoopRpcOMFollowerReadProxyProvider > *** HadoopRpcOMFailoverProxyProvider > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
[jira] [Updated] (HDDS-14379) Implement basic Hadoop OM client proxy provider to read from followers
[ https://issues.apache.org/jira/browse/HDDS-14379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Andika updated HDDS-14379: --- Summary: Implement basic Hadoop OM client proxy provider to read from followers (was: Implement basic client proxy provider to read from followers) > Implement basic Hadoop OM client proxy provider to read from followers > -- > > Key: HDDS-14379 > URL: https://issues.apache.org/jira/browse/HDDS-14379 > Project: Apache Ozone > Issue Type: New Feature >Reporter: Ivan Andika >Assignee: Ivan Andika >Priority: Major > > This task is to come up with the basic implementation of follower read client > proxy as a baseline before further performance improvements. The idea is to > simply pick a OM node in random (which be leader or follower / listener) and > use it to submit read requests. If the chosen OM node is a follower, the read > requests need to keep sending to that OM node unless the OM is down which > triggers failover. Write requests should be sent to the OM leader directly. > Further improvements such as adding OM probe logic will be implemented in > follow up tasks. > The implementation is to introduce HadoopRpcOMFollowerReadProxyProvider which > wraps > HadoopRpcOMFailoverProxyProvider. FollowerReadProxyProvider tracks a > different currentOmNodeId from HadoopRpcOMFailoverProxyProvider. > FollowerReadInvocationHandler will check whether the request is a read > request (using OmUtils#isReadOnly) and if so forwards it to its current > proxy. If it's a write request, the request if forwarded to > HadoopRpcOMFailoverProxyProvider to be sent to the leader. > So the proxy hierarchy is > * RetryProxy > ** HadoopRpcOMFollowerReadProxyProvider > *** HadoopRpcOMFailoverProxyProvider > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
