sunildataengineer opened a new issue, #65475:
URL: https://github.com/apache/airflow/issues/65475
### Description
The SFTPOperator currently runs in synchronous mode only, blocking
a worker slot for the entire duration of every file transfer.
For large file transfers, this wastes worker resources unnecessarily.
Proposed Solution
Add deferrable=True parameter to SFTPOperator following the same
pattern used in other Airflow providers. When deferrable=True,
the operator will defer execution to a trigger, freeing the worker
slot immediately and allowing the Triggerer to poll for completion.
Implementation Plan
- Add SFTPOperatorTrigger class in triggers/sftp.py
- Add deferrable param to SFTPOperator.__init__()
- Add defer() call in execute() when deferrable=True
- Add execute_complete() callback method
- Add unit tests
Related
Similar to deferrable mode already implemented in SFTPSensor.
I am willing to submit a PR for this.
### Use case/motivation
_No response_
### Related issues
_No response_
### Are you willing to submit a PR?
- [x] Yes I am willing to submit a PR!
### Code of Conduct
- [x] I agree to follow this project's [Code of
Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
