wangyum commented on PR #57525: URL: https://github.com/apache/spark/pull/57525#issuecomment-5654309727
@sunchao I've revised this to use a **non-secret driver-instance ID**, rather than an application ID or bearer token. The mechanism is now simpler: - Each `CoarseGrainedSchedulerBackend` generates a UUID during construction, before launchers capture executor startup configuration. This removes the application-ID readiness races, fallback IDs, and retry/wait logic. - The existing startup-property paths propagate the ID in Standalone, YARN, and Kubernetes. No new Driver/AM handshake is needed. - The driver validates the launch-time ID both before returning bootstrap credentials and before accepting registration. Missing/mismatched IDs and legacy bootstrap requests are rejected without an authentication-based bypass. - `RegisterExecutor` retains its eight-field shape, and the original bootstrap request retains its one-field shape. The migration guide explicitly documents the behavioral incompatibility for external launchers. Tests exercise the actual registration sender in plain and SSL modes, reject stale identities after a replacement driver binds the same address with the same application ID, and verify launch-option propagation. The previously broken synthetic registration callers are also updated. One important distinction concerns your disclosure comments: **this revision does not implement the requested confidentiality or malicious-replay protection**. The ID can remain visible in configuration, Pod specifications, and executor attributes. Its purpose is to prevent an honest, stale executor from silently attaching to the wrong driver, not to authenticate an adversarial caller. Renaming the value does not close those replay paths; RPC authentication and appropriate cluster isolation remain necessary. Would you be comfortable treating this PR as the narrowly scoped stale-executor/port-reuse correctness fix, with stronger cross-application isolation handled separately? I don't want to mark the disclosure concerns resolved without agreement on that scope. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
