shubham22 commented on issue #35490:
URL: https://github.com/apache/airflow/issues/35490#issuecomment-1858568177
> if you want to just update one field for a particular task or DAG (like in
this example, add some tags or update the memory for a particularly heavy task)
you may wipe out 10s or even ~100 lines of config settings
Yeah, that's fair critique. To be on the same page, I was suggesting
replacing at 1st level field. However, guessing by how this might be used, a
user may want to change just memory to accommodate a given task, while keeping
everything same. So, instead of replacing a field at the 1st level, we can
replace a field at lowest level. In that case as well we should follow same
behavior for arrays or strings.
```
RUN_TASK_KWARGS = {
"tags": [{"key": "fish", "value": "banana"}, {"key": "peanut", "value":
"sauce"}],
"startedBy": "Niko",
"overrides": {
"containerOverrides": [{"memory": 500, "environment":[{"name":
"FOO", "value": "BAR"}]}],
"cpu": "5",
}
}
```
```
executor_config={
"tags": [{"key": "FOO", "value": "BAR"}],
"overrides": {
"containerOverrides": [{"memory": 740}]
},
}
```
Final config for task:
```
Config = {
"tags": [{"key": "FOO", "value": "BAR"}],
"startedBy": "Niko",
"overrides": {
"containerOverrides": [{"memory": 740}],
"cpu": "5",
}
}
```
Yes, you'd still need to provide entire containerOverrides set.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
