firasbouzazi opened a new pull request, #74299: URL: https://github.com/apache/airflow/pull/74299
Follow-up to #74211. `SSHTunnel._serve_forever` waited with `select.select()`, which cannot watch a descriptor numbered 1024 or higher. The `ValueError` was caught and ended the forwarding thread, so with that many descriptors open `SSHHook.get_tunnel()` left a local port that accepted connections but never forwarded them. The loop now uses `selectors.DefaultSelector`. Since connections come and go, the listening and shutdown sockets are registered once, and each forwarded connection's socket and channel are registered when accepted and unregistered before they are closed. Unregistering uses the descriptor recorded at registration: a closed socket has no descriptor, and `paramiko.Channel.fileno()` on a closed channel would create a new pipe instead of returning the old one. Connections whose channel the remote end closed are now closed and unregistered rather than only dropped from the list. Tests: the in-process paramiko server fixtures from #74211 move to a shared `conftest.py` and now echo forwarded (`direct-tcpip`) channels. New tests forward data through `SSHTunnel` sequentially and concurrently while the process holds over 1024 descriptors (no data gets through before the fix), and check that connections closed by either end are unregistered. Against a local sshd, `SSHHook.get_tunnel()` with 1,100 extra descriptors open went from 0/10 to 10/10 round trips. The SSH provider unit tests (209) pass. related: #74205 --- ##### Was generative AI tooling used to co-author this PR? - [X] Yes — Claude Code Generated-by: Claude Code following [the guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
