ashb commented on issue #50185:
URL: https://github.com/apache/airflow/issues/50185#issuecomment-2967206575
Actually, how about this @x42005e1f:
- We remove the lock on CommsDecoder.
- We create a new subclass called TriggererCommsDecoder (so we know when we
are operating in this hybrid sync/async env)
- On that we have two queues - one for requests, the other for responses.
- The code that pulls off of the requests Q and writes to requests/reads
from stdin is 100% async code
- The sync send code (in my Pr I combined `send_msg() + get_response()` into
a single function as it's a nicer API anyway) does something like this:
```python
req_id = next(self.id_counter)
self._q_requests.put(Frame(..., id=req_id)
while True:
resp = self._q_responses.peek()
if resp.id != req_id:
# Response to someone else's message. Try again
continue
self._q_responses.get() # Though this feels racy? Maybe we need a
sanity check and to requeue it if the req_id doesn't match?
```
Is that actually any better (clearer/easier to understand/more performant)
than just using a n=1 Thread pool?
WDYT as well @gopidesupavan?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
