DG47 opened a new pull request, #74120:
URL: https://github.com/apache/airflow/pull/74120
Since provider 7.16 (#66613) `DatabricksSubmitRunOperator` forwards the
operator's `params` into `python_wheel_task.named_parameters` whenever that
field is empty. The guard only looked at `named_parameters`, so a Python wheel
task that already carries positional `parameters` was submitted with both
fields populated. The Databricks `runs/submit` API treats the two as mutually
exclusive ("Leave it empty if `named_parameters` is not null") and rejects the
run, so adding a Dag-level param to such a Dag breaks every wheel task that
uses positional arguments.
**Fix**
* Add `_TASK_PARAM_SLOTS_CONFLICTING_WITH_DICT_PARAM_FIELD` mapping each
task type to the parameter slots that cannot coexist with the dict-shaped
injection target (currently `python_wheel_task.parameters`), mirroring the
existing `_RUN_NOW_PARAM_SLOTS_CONFLICTING_WITH_JOB_PARAMETERS` guard in
`DatabricksRunNowOperator`.
* `_inject_airflow_params_into_task` skips injection when one of those slots
is populated. An empty `parameters` list still counts as unset, consistent with
how an empty `named_parameters` is treated. Other tasks in the same payload are
still filled in.
* Docstring and `submit_run.rst` updated to document the skip.
**Testing**
New
`test_submit_run_skips_param_injection_when_wheel_task_has_positional_parameters`
(parametrized: populated list -> no injection; empty list -> injection) fails
on `main` and passes with this change.
```
pytest
providers/databricks/tests/unit/databricks/operators/test_databricks.py
267 passed
```
closes: #74095
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
