ColtenOuO commented on PR #72133: URL: https://github.com/apache/airflow/pull/72133#issuecomment-5463010917
Hi Jarek! I've updated the PR description. It now includes the benchmark results from running on a live Airflow instance, along with a real-world ML training pipeline use case. Beyond releasing worker slots, WebSockets solve the core drawback of airflow-provider-http sensors constantly polling the remote service. Because WebSocket pushes the result the moment the job finishes, it eliminates the polling latency inherent to HTTP sensors, allowing the DAG workflow to proceed to downstream tasks immediately and far more responsively. Please feel free to let me know if anything in the description needs clarification or further improvement. Thanks a lot! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
