amoghrajesh commented on PR #65991: URL: https://github.com/apache/airflow/pull/65991#issuecomment-4532984527
@nailo2c Thanks for the refactor, I think this is moving in the right direction. Re auth, the `LivyHook` pattern is the right call. No objection there. On the key name: `yarn_resourcemanager_webapp_address` mirrors Hadoop's own config key (`yarn.resourcemanager.webapp.address` IIRC) which means cluster operators will recognise it immediately. So, I'd keep it cos verbosity ensures clarity here. Re coordination, I plan to add YARN support once https://github.com/apache/airflow/pull/67118 is in, and it would call into the same RM REST API for `get_job_status()`. The two PRs are complementary, yours frees the JVM at submission time, mine handles reconnecting to the running app on worker crash. Worth keeping in sync so we don't end up with two separate RM client implementations. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
