ArvidMartensRenson commented on issue #50185: URL: https://github.com/apache/airflow/issues/50185#issuecomment-2891834967
I created an async client from the sync client, using httpx.AsyncClient as the base (like httpx.Client is for the sync client). Inherited the init client from the sync task client. Only problem I had was that the operations had to be rewritten as an async method, so I copied the code from the sync client and changed the method definition to async. I found this implementation nicer than using plain api calls with the async httpx client. Traffic is over localhost so even then a token is required, I reused the methods that are used in the creation of the task workload to get the token. I considered dropping the token requirement but it seemed safer and whenever the trigger service and api server are separated it will be needed. I implemented this solution last Thursday and for now the trigger service has not been stuck and the external workload triggers all completed. So for now I would conclude that the solution works. If it is desirable to have an async client I am happy to help on creating the client, however I am a little worried with duplicate code I have now, same code for the sync and async client. If someone knows a solution for this or a way to handle that problem, that would be nice. For the record, I am using a custom implementation of the workload trigger. I was an early adopter of the trigger paradigm and the external workload trigger did not exist at that time, so we had to implement our own trigger. We never came around converting to the standard provider implementation. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
