nathadfield opened a new pull request, #61132:
URL: https://github.com/apache/airflow/pull/61132

   ## Description                                                               
                                                                                
                                                                                
                                                                          
                                                                                
                                                                                
                                                                                
                                                                          
   Fixes a regression in Airflow 3.x where deferrable sensors with 
`soft_fail=True` fail with `AirflowSensorTimeout` instead of being marked as 
SKIPPED when they timeout. This breaks compatibility with Airflow 2.x behavior 
and user expectations.                                                          
           
                                                                                
                                                                                
                                                                                
                                                                          
   ## Root Cause                                                                
                                                                                
                                                                                
                                                                          
                                                                                
                                                                                
                                                                                
                                                                          
   In `BaseSensorOperator.resume_execution()`, the exception handling order was 
incorrect:                                                                      
                                                                                
                                                                          
                                                                                
                                                                                
                                                                                
                                                                          
   1. `TaskDeferralTimeout` was caught and immediately converted to 
`AirflowSensorTimeout`                                                          
                                                                                
                                                                                
      
   2. The `soft_fail`/`never_fail` checks only caught `AirflowException` and 
`TaskDeferralError`                                                             
                                                                                
                                                                             
   3. By the time these checks ran, the timeout was already converted, causing 
the flags to be ignored                                                         
                                                                                
                                                                           
                                                                                
                                                                                
                                                                                
                                                                          
   ## Solution                                                                  
                                                                                
                                                                                
                                                                          
                                                                                
                                                                                
                                                                                
                                                                          
   - Use nested try-except blocks to convert `TaskDeferralTimeout` → 
`AirflowSensorTimeout` while ensuring both exceptions are caught by the outer 
handler                                                                         
                                                                                
       
   - Check `soft_fail` and `never_fail` flags before raising 
`AirflowSensorTimeout`                                                          
                                                                                
                                                                                
             
   - Align error messages with the `execute()` method for consistency           
                                                                                
                                                                                
                                                                          
                                                                                
                                                                                
                                                                                
                                                                          
   ## Changes                                                                   
                                                                                
                                                                                
                                                                          
                                                                                
                                                                                
                                                                                
                                                                          
   **`task-sdk/src/airflow/sdk/bases/sensor.py`:**                              
                                                                                
                                                                                
                                                                          
   - Modified `resume_execution()` to use nested exception handling             
                                                                                
                                                                                
                                                                          
   - Changed to separate `soft_fail` and `never_fail` checks with explicit 
error messages                                                                  
                                                                                
                                                                               
                                                                                
                                                                                
                                                                                
                                                                          
   **`task-sdk/tests/task_sdk/bases/test_sensor.py`:**                          
                                                                                
                                                                                
                                                                          
   - Added tests for timeout scenarios with `soft_fail=True/False`              
                                                                                
                                                                                
                                                                          
   - Added tests for timeout scenarios with `never_fail=True`                   
                                                                                
                                                                                
                                                                          
   - Added tests for trigger failure scenarios with `soft_fail=True/False`      
                                                                                
                                                                                
                                                                          
   - Added tests for trigger failure scenarios with `never_fail=True`           
                                                                                
                                                                                
                                                                          
   - Fixed existing tests to use `TriggerFailureReason` enum values
   ##### Was generative AI tooling used to co-author this PR?                   
                                                                                
                                                                                
                                                                          
                                                                                
                                                                                
                                                                                
                                                                          
   - [X] Yes (please specify the tool below)                                    
                                                                                
                                                                                
                                                                          
                                                                                
                                                                                
                                                                                
                                                                          
   <!--                                                                         
                                                                                
                                                                                
                                                                          
   Generated-by: Claude Code following [the 
guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to