errose28 commented on PR #7988:
URL: https://github.com/apache/ozone/pull/7988#issuecomment-2770846705
Thanks for working on this @ivandika3. I only briefly looked at the code,
but can we implement this such that the client/server API required for this
change can be reused if/when we add general purpose Ratis based read from
follower?
For example, with Ratis read-from-follower, I imagine the implementation
would look something like this:
- Client gives request to leader OM with an option that says it is ok with
reading from a follower for this request.
- OM leader either services the request itself, or load balances by
returning options for other followers which have applied all requests.
- If given a list of other OMs, client will try to have them service its
request by contacting them directly.
In this implementation, since we are not yet using Ratis to determine who
has the snapshots, the same request flow could be used:
- Client gives request to leader OM with an option that says it is ok with
reading from a follower for this request.
- Leader OM only acknowledges this option for snapshot read requests.
- OM leader returns addresses of both followers for the client to try, since
we don't have read-from-follower implemented.
- Client tries each OM returned until it finds one that has the snapshot,
and then does the read there.
The only snapshot specific aspect here would be the ability to read snapdiff
from a specific OM. I would assume the snapdiff request outputs the host that
the job is running on so that the user can feed this back into requests for
their snapdiff. I don't think general snapshot list/info CLIs would need to be
updated since they could use the same general flow as above. Since snapdiff is
already OM specific, we should probably implement this part of the change on
its own (to account for leader changes) and then come back to this PR as an
enhancement on top of that.
It would be good to avoid adding anything to the client/server code that
would just get deprecated later, because it adds complexity to our cross
compatibility support. I imagine general read-from-follower would be able to
replace a lot of the snapshot specific changes here, so we should probably
think in that direction.
Alternatively, given that this change is almost 2k lines long, we could look
at the level of effort for supporting general purpose Ratis read-from-follower,
and only enabling it for snapshot use cases to start out. I would imagine it is
about the same amount of changes or less assuming Ratis has already laid the
groundwork to help us out.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
