MalteNiederstadt opened a new pull request, #69689:
URL: https://github.com/apache/airflow/pull/69689
Summary
azure-mgmt-datafactory 10.0.0 (released 2026-07-08) introduced hybrid
models (https://aka.ms/azsdk/python/migrate/hybrid-models) that change the
internal __dict__ representation of response objects. This
breaks vars(response)["run_id"] with a KeyError, even though
response.run_id still works.
Root Cause Chain
1. The provider's pyproject.toml declares azure-mgmt-datafactory>=2.0.0
with no upper bound (source
(https://github.com/apache/airflow/blob/main/providers/microsoft/azure/pyproject.toml))
2. When a managed Airflow environment (e.g., Cloud Composer) rebuilds, pip
resolves azure-mgmt-datafactory==10.0.0 — which was released on 2026-07-08
3. SDK v10 uses new "hybrid models" where attributes are exposed via
properties/descriptors rather than stored directly in __dict__
4. The operator accesses the response via vars(response)["run_id"], which
reads __dict__ directly — this now raises KeyError since the attribute is no
longer in __dict__
Error
ERROR - Task failed with exception
Traceback (most recent call last):
File
"/opt/python3.11/lib/python3.11/site-packages/airflow/sdk/execution_time/task_runner.py",
line 1068, in run
result = _execute_task(context=context, ti=ti, log=log)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File
"/opt/python3.11/lib/python3.11/site-packages/airflow/sdk/execution_time/task_runner.py",
line 1477, in _execute_task
result = ctx.run(execute, context=context)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File
"/opt/python3.11/lib/python3.11/site-packages/airflow/sdk/bases/operator.py",
line 417, in wrapper
return func(self, *args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File
"/opt/python3.11/lib/python3.11/site-packages/airflow/providers/microsoft/azure/operators/data_factory.py",
line 184, in execute
self.run_id = vars(response)["run_id"]
~~~~~~~~~~~~~~^^^^^^^^^^
KeyError: 'run_id'
Affected operators
• AzureDataFactoryRunPipelineOperator (data_factory.py:184)
• AzureSynapseRunSparkBatchOperator (synapse.py:107)
• AzureSynapseRunPipelineOperator (synapse.py:260)
Fix
Replace vars(response)["key"] with direct attribute access response.key,
which works with both old and new SDK versions (attribute access is part of the
public API contract for both classic and hybrid
models).
Ref
•
https://github.com/Azure/azure-sdk-for-python/blob/main/sdk/datafactory/azure-mgmt-datafactory/CHANGELOG.md#1000-2026-07-07
• Breaking change: "This version introduces new hybrid models which have
dual dictionary and model nature"
• Migration guide: https://aka.ms/azsdk/python/migrate/hybrid-models
────────────────────────────────────────
Was generative AI tooling used to co-author this PR?
• [x] Yes — Cortex Code (Snowflake)
<!--
Generated-by: Cortex Code following [the
guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions)
-->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
