nagasrisai opened a new pull request, #71150: URL: https://github.com/apache/airflow/pull/71150
Fixes #65366. ### What happened Under `KubernetesExecutor`, failed retry attempts can end up with `hostname=''` and `start_date=NULL` in `task_instance_history`. This causes the served-log URL builder to construct `http://:8793/log/...`, which Python's urllib rejects with: ``` Could not read served logs: Invalid URL 'http://:8793/...': No host supplied ``` Only the final successful attempt is unaffected — the problem is confined to the earlier failed-retry rows in history. ### Root cause `adopt_or_reset_orphaned_tasks()` queries `TaskInstance` with a `load_only()` clause that covers only seven columns: ```python load_only(TI.id, TI.dag_id, TI.task_id, TI.run_id, TI.map_index, TI.state, TI.external_executor_id) ``` All other columns — including `hostname`, `start_date`, `end_date`, and `duration` — are deferred. When the orphan-reset path then calls `prepare_db_for_next_try()` → `TaskInstanceHistory.record_ti()`, the history constructor iterates every column of `task_instance_history` and does `getattr(ti, column_name)` for each one. For deferred attributes this fires a separate lazy-SELECT round-trip per column. For tasks whose pod was killed before the task-sdk could reach the execution API, `hostname` remains `""` in the database (the Python-level default set in `TaskInstance.__init__`). The lazy-SELECT returns that empty string, and it is written verbatim into history. An empty-string hostname — not SQL NULL — then gets embedded in the served-log URL, producing the `http://:8793` error. ### Fix **`scheduler_job_runner.py`** — add `hostname`, `start_date`, `end_date`, `duration`, and `try_number` to the `load_only()` clause in `adopt_or_reset_orphaned_tasks()`. This loads all fields that `record_ti()` needs in the same query as the `FOR UPDATE` row-lock, eliminating the N+1 lazy-SELECT round-trips and guaranteeing that we read from the already-locked row. **`taskinstancehistory.py`** — in `TaskInstanceHistory.__init__`, convert `hostname == ""` to `None` before storing in the history row. `TaskInstance` initialises `hostname` to `""` rather than `None`, so tasks that were never reached by the task-sdk leave an empty string in the DB. Storing `NULL` in history lets callers (including the log URL builder) distinguish "hostname was never reported" from a real hostname, and prevents the broken URL construction. ### Tests Two new tests in `test_scheduler_job.py`: - `test_adopt_or_reset_resettable_tasks_preserves_execution_metadata_in_history` — verifies that a task in RUNNING state with a real hostname has that hostname preserved correctly in the history row after the orphan reset. - `test_adopt_or_reset_resettable_tasks_stores_null_hostname_when_never_reported` — verifies that a task whose pod was killed before the task-sdk could report back (hostname stays `""` in DB) produces a history row with `hostname=NULL` rather than `hostname=""`. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
