konstantinb commented on PR #8426: URL: https://github.com/apache/hadoop/pull/8426#issuecomment-4784666120
@steveloughran @mukund-thakur — even though it *appears* safe (running in a privately deployed production fork with the pool enabled for ~2.5 months with no evident issues), I'm still hesitant to flip the default. My worry isn't whether the code works — that evidence covers one workload and the test paths. It's the **blast radius of the flip itself**: every S3A deployment would silently inherit the new shared-executor model on upgrade, often unknowingly, across workloads none of us has exercised. Opt-in is a conscious choice; a default change is inherited — we'd be changing everyone's behavior to fix a problem only some may have hit. The current approval is also on the opt-in design — @ajfabbri noted that defaulting to disabled "mitigates a lot of the risk … around qualifying different workloads" — so I believe that a default flip should have his and @ahmarsuhail's sign-off. To help us settle the on/off question with data, I've kicked off a full apples-to-apples comparison on the current revision: the entire `hadoop-aws` integration suite with all four shared pools **off vs on**, `-Dscale=true`, mirroring the `cloud_aws` CI invocation, against both LocalStack and real S3 (`us-east-1`). It's running now; I'll report the results back here so we can weigh the default with the numbers in front of us. My proposal : keep it opt-in for this PR, and flip the default on in a follow-up once it's had real-world exposure across a bigger deployment base (open to alternatives here) — same end state, lower risk. Happy to go straight to default-on now if that's the consensus. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
