kaxil commented on code in PR #67842:
URL: https://github.com/apache/airflow/pull/67842#discussion_r3336353344
##########
task-sdk/src/airflow/sdk/execution_time/context.py:
##########
@@ -550,6 +552,9 @@ def set(self, key: str, value: JsonValue, *, retention:
timedelta | None = None)
from airflow.sdk.execution_time.comms import SetTaskState
from airflow.sdk.execution_time.task_runner import SUPERVISOR_COMMS
+ if value is None:
+ raise ValueError("Cannot set value as None")
Review Comment:
Rejecting `None` here changes behavior for the one in-tree caller,
`ResumableMixin`. At
[`resumablemixin.py:127-130`](https://github.com/apache/airflow/blob/cb02ce2ed7f0178dcf23965aabaaa0c3a0d86e0b/task-sdk/src/airflow/sdk/bases/resumablemixin.py#L127-L130)
it does `external_id = self.submit_job(context)` then
`task_state.set(self.external_id_key, external_id)`. `submit_job` is typed `->
JsonValue`, so a subclass can legitimately return `None`. Previously a `None`
id was stored silently and the next run hit `if external_id:` as falsy and
resubmitted, so the task still completed. Now it raises mid-execution.
Was that break intended? If not, options are to skip the write when the
value is `None`, guard the call in the mixin (the existing `if task_state is
not None:` guard at line 129 only covers `task_state`, not `external_id`), or
at minimum document that `submit_job` must never return `None`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
