KidAmnesiac1 opened a new pull request, #74445:
URL: https://github.com/apache/airflow/pull/74445

   ### Description
   
   Changes the `edge_job.command` column from `varchar(2048)` to `Text`, so the 
`EdgeExecutor` can store workloads of any size.
   
   ### Reason for the change
   
   The `EdgeExecutor` stores each workload as JSON in `edge_job.command`. A 
`SyncCallback` deadline workload carries a snapshot of the Dag run, including 
its `conf`, so it can easily exceed 2048 characters. Ours was about 2.3 KB 
(Airflow 3.3.2, edge3 4.3.2, PostgreSQL), and the insert failed:
   
   ```
   psycopg2.errors.StringDataRightTruncation: value too long for type character 
varying(2048)
   ```
   
   The error escapes the scheduler loop and the transaction rolls back, so the 
deadline stays pending and every restarted scheduler crashes on it again. 
Scheduling stops for the whole deployment until the Dag run is deleted.
   
   ### Fix
   
   - New edge3 migration `0007` changes `edge_job.command` to `Text`. On 
PostgreSQL, converting `varchar` to `text` only changes metadata, so the 
upgrade doesn't rewrite the table.
   - `EdgeJobModel.command` is now `Text`, and the `5.0.0` revision head points 
at the new migration.
   - A new test queues a callback workload over 2048 characters and checks it's 
stored unchanged. On PostgreSQL it fails with the error above without this 
change. SQLite doesn't enforce `varchar` lengths, so the test only catches the 
bug on PostgreSQL and MySQL.
   
   The migration is labelled `5.0.0` to match `0006`, which landed after 
`5.0.0rc2` was cut. Both may need relabelling to the next edge3 version.
   
   Related: #SCHED_PR makes the scheduler survive a callback that fails to 
queue for any reason. The Dag run snapshot itself comes from 
`get_simple_context()` in `Deadline.handle_miss`, whose `TODO` covers fetching 
it at runtime instead (attempted in #66608, reverted in #68909).
   
   ---
   
   ##### Was generative AI tooling used to co-author this PR?
   
   - [X] Yes — Claude Code (Opus 5.5)
   
   Generated-by: Claude Code (Opus 5.5) following [the 
guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to