gaoyajun02 opened a new pull request, #3452:
URL: https://github.com/apache/celeborn/pull/3452
### What changes were proposed in this pull request?
This PR enhances the Ratis peer add operation to support clientAddress and
adminAddress parameters with RESTful api, allowing these critical RPC endpoints
to be properly configured when adding new peers to the Celeborn master cluster.
### Why are the changes needed?
Currently, when expanding the Celeborn master cluster using the ratis peer
add operation, newly added peers lack clientAddress and adminAddress settings.
If a newly added peer becomes the Leader, all Followers will return empty
addresses to clients, causing them to attempt connections to an incorrect
Leader address (127.0.0.1:0). This change ensures proper client request routing
in expanded clusters.
### Does this PR introduce _any_ user-facing change?
Yes, this PR extends the API for adding Ratis peers by adding support for
clientAddress and adminAddress parameters. Users will now be able to specify
these addresses when adding new peers to the cluster.
### How was this patch tested?
Manual testing of cluster expansion scenarios to ensure clients can
correctly connect to the Leader regardless of which peer holds leadership
```
➜ curl -sX POST
zw06-data-k8s-sparktest-node007.mt:9098/api/v1/ratis/peer/add \
-H "Content-Type: application/json" \
-d '{ "peers": [{"id": "2", "address":
"zw06-data-k8s-sparktest-node009.mt:9872", "clientAddress":
"zw06-data-k8s-sparktest-node009.mt:9097", "adminAddress":
"zw06-data-k8s-sparktest-node009.mt:9097" }] }' | jq
{
"success": true,
"message": "Successfully added peers
ArrayBuffer(2|zw06-data-k8s-sparktest-node009.mt:9872) to group
GroupInfoReply:client-3E7C9CE679B2->0@group-47BEDE733167, cid=1031, SUCCESS,
logIndex=0, commits[0:c224, 1:c224]."
}
➜ curl -s zw06-data-k8s-sparktest-node009.mt:9098/masterGroupInfo
====================== Master Group INFO ==============================
group id: c5196f6d-2c34-3ed3-8b8a-47bede733167
leader info: 0(zw06-data-k8s-sparktest-node007.mt:9872)
[server {
id: "2"
address: "zw06-data-k8s-sparktest-node009.mt:9872"
clientAddress: "zw06-data-k8s-sparktest-node009.mt:9097"
adminAddress: "zw06-data-k8s-sparktest-node009.mt:9097"
startupRole: FOLLOWER
}
commitIndex: 228
, server {
id: "0"
address: "zw06-data-k8s-sparktest-node007.mt:9872"
clientAddress: "zw06-data-k8s-sparktest-node007.mt:9097"
adminAddress: "zw06-data-k8s-sparktest-node007.mt:9097"
startupRole: FOLLOWER
}
commitIndex: 228
, server {
id: "1"
address: "zw06-data-k8s-sparktest-node008.mt:9872"
clientAddress: "zw06-data-k8s-sparktest-node008.mt:9097"
adminAddress: "zw06-data-k8s-sparktest-node008.mt:9097"
startupRole: FOLLOWER
}
commitIndex: 228
]
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
