hunnguye commented on PR #71949: URL: https://github.com/apache/airflow/pull/71949#issuecomment-5412132840
Although `_StandaloneSparkSubmitBackend.get_job_status()` supports multiple master URLs in connection["master"], I chose to follow the hook’s current assumption that connection["master"] contains a single URL and constructed connection["rest_endpoint"] accordingly. I could also have added support for rest_endpoint to handle either a single URL or a comma-separated list of master urls, but that would conflict with the assumption above. I think doing this properly would also require refactoring the usages of connection["master"], either always assume a comma separated list of master url of datatype string or already providing a list of master urls (which results in just one element if not in HA). However, that feels outside the scope of this PR. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
