PrakshiGoyal10 commented on PR #69998: URL: https://github.com/apache/airflow/pull/69998#issuecomment-5294456127
Thanks for the careful review and for validating it against a live workspace — the duplicate-key bug is exactly right, and I confirmed it the same way. Pushed a fix. **Duplicate task keys.** `get_run_failed_task_keys` now keeps only the latest attempt per `task_key` before judging state, using the sort-by-`start_time` idiom from `_get_current_databricks_task` as you suggested. That removes the duplicates that Databricks rejects and also fixes the stale-attempt symptom (a key whose newest attempt succeeded is no longer reported failed). Added a hook test shaped like a live run — multiple `flaky` attempts (FAILED, FAILED, then SUCCESS) plus a task whose latest attempt failed — asserting each key appears once and is judged by its latest attempt. **A second, related defect I found while fixing that.** Reconstructing the Databricks `task_key` on the API-server side as `md5(dag_id__task_id)` is only correct for tasks that use the default key. An explicit `databricks_task_key` does not survive Dag serialization (I verified the attribute is absent from the deserialized operator), so for such tasks single-task repair targeted a non-existent key and repair-all silently skipped clearing that task's instance. The launch task now records the `task_id` to `task_key` map in its run-metadata XCom, and the endpoint reads it server-side (so no request-supplied keys, consistent with the trusted-identifier model); runs launched before this fall back to the md5 derivation. Covered by tests for both the explicit-key and legacy-fallback paths. All provider unit tests pass locally. On the still-open CodeQL redirect finding: the redirect target is a same-site relative path built with `quote(..., safe='')`, so it can't leave the origin — I believe it's a false positive, but happy to refactor if you'd prefer it read differently. --- Drafted-by: Claude Code (Opus 4.8); reviewed by @PrakshiGoyal10 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
