ramitkataria commented on code in PR #69985:
URL: https://github.com/apache/airflow/pull/69985#discussion_r4624711379


##########
airflow-core/src/airflow/migrations/versions/0094_3_2_0_replace_deadline_inline_callback_with_fkey.py:
##########
@@ -323,7 +425,7 @@ def _downgrade_postgresql(conn, batch_size):
                     SET callback = json_build_object(
                             '__data__', json_build_object(
                                 'path', c.data::jsonb->'__var'->>'path',
-                                'kwargs', c.data::jsonb->'__var'->'kwargs'
+                                'kwargs', 
pg_temp.decode_extended(c.data::jsonb->'__var'->'kwargs')

Review Comment:
   Not from this PR (it goes back to my original version of the migration), but 
since these lines now decode `kwargs`, this might be a good place to fix it: 
the downgrade copies `kwargs["context"]` back into the inline callback.
   
   On 3.2+ `handle_miss` stores the runtime context in 
`callback.data["kwargs"]["context"]`. After a downgrade, a missed deadline 
whose callback was still queued or running gets `callback_state = NULL`, so the 
3.1 scheduler picks it up again and `Callback.__init__` raises 
`ValueError("context is a reserved kwarg for this class")`.
   
   Dropping `context` from the decoded kwargs in both downgrade paths should be 
enough I think



##########
airflow-core/src/airflow/migrations/versions/0094_3_2_0_replace_deadline_inline_callback_with_fkey.py:
##########
@@ -54,12 +54,102 @@
 _ASYNC_CALLBACK_CLASSNAME = "airflow.sdk.definitions.deadline.AsyncCallback"
 
 
+def _serialize_extended(value):

Review Comment:
   +1 to a repair revision, and to revisiting the "out of band" plan.
   
   Some context from writing the original version of this migration: the 
`ExtendedJSON` encoding was replaced by the SQL rewrite in #63628 before 3.2.0 
was cut, so the broken encoding is the only one that has shipped. Anyone who 
had deadline rows on 3.1 and upgraded has them.
   
   Two things make me uneasy about leaving the repair to operators:
   
   - The SQL here and on the issue is Postgres-only. MySQL/SQLite rows have no 
`__var` envelope, so there is nothing for it to match.
   - `repair_node` on the issue still has the empty-array `jsonb_agg` NULL 
problem that was fixed in this PR, so copying it turns `[]` into `null`.
   
   A small follow-up revision with the same lenient logic (wrap bare objects, 
leave wrapped ones alone) is idempotent, so it could run over every `callback` 
row. Fine as a separate PR, and I'm happy to help with it or review.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to