SameerMesiah97 opened a new pull request, #61830:
URL: https://github.com/apache/airflow/pull/61830

   **Description**
   
   This change enhances `ComputeEngineInsertInstanceOperator` to detect 
configuration differences when an instance already exists.
   
   Previously, the operator treated instance presence as success and returned 
without validating that the existing resource matched the requested 
configuration. As a result, changes to fields such as `machine_type` or disk 
settings were not detected on subsequent DAG runs.
   
   This update introduces configuration comparison logic when an instance is 
found. Detected differences are logged. An optional `recreate_if_different` 
flag allows users to explicitly request deletion and recreation of the instance 
when configuration drift is detected.
   
   To support this behavior, two helper methods were introduced: 
`_detect_instance_drift`, which compares the existing instance with the 
requested body, and `_create_instance`, which encapsulates instance creation 
logic used by both the initial and recreation paths.
   
   **Rationale**
   
   The previous behavior relied solely on presence-based idempotence and did 
not validate configuration consistency across DAG runs.
   
   This change surfaces configuration differences and provides an opt-in 
mechanism for reconciliation, while preserving default behavior.
   
   **Notes**
   
   * The `execute` method has been refactored to accommodate configuration 
drift logging and the new `recreate_if_different` flag while preserving default 
behavior. The refactor extracts instance creation into a helper and introduces 
structured comparison logic without altering presence-based idempotence unless 
the new flag is explicitly enabled. 
   * Minor corrections have been made to comments where applicable with 
redundant comments removed. 
   
   **Tests**
   
   * Added a unit test verifying that configuration drift is detected and 
logged.
   * Added a unit test verifying that the instance is deleted and recreated 
when `recreate_if_different=True`.
   
   **Documentation**
   
   * Added documentation for the new `recreate_if_different` parameter in the 
operator docstring.
   * Updated the `execute` method docstring to clarify presence-based 
idempotence and drift handling behavior.
   
   **Backwards Compatibility**
   
   There is a behavioral difference in that configuration drift is now logged 
by default when detected. Additionally, users may opt into reconciliation 
behavior via `recreate_if_different=True`, which will delete and recreate the 
instance when differences are found. Existing DAGs will otherwise continue to 
behave as before unless the new flag is explicitly enabled.
   
   Closes: #61829 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to