villebro opened a new pull request, #43678: URL: https://github.com/apache/superset/pull/43678
### SUMMARY Async-path hardening for the Global Task Framework, from the pre-merge review + performance investigation. Targets `gaq-to-gtf`. **Landed in this PR so far:** 1. **Abort-listener teardown no longer delays task completion.** A task that registers an `on_abort` handler (e.g. chart-data query cancellation) starts a background abort listener parked in a blocking Redis `XREAD` (`_LISTEN_BLOCK_MS = 1000`). `SignalListener.stop()` joined that thread, so it waited out the full block — adding ~1s to every such task at teardown (Celery reported ~1.1s while the task body, `started_at`→`ended_at`, was ~0.1s). `stop()` now nudges the stream so the blocked read returns at once. **Abort detection is unchanged**: a real signal still wakes the read, and the loop checks the stop flag *before* the predicate, so the wake nudge can never fire a handler. 2. **Internal task state moved into a `private` properties bucket.** Task `properties` are returned by the Task REST API, so internal runtime state must not leak. A reserved `properties["private"]` sub-dict (seeded empty on every task) holds framework plumbing — `celery_task_id`, `cancel_query_id`, `cancel_database_id` — and the API strips it wholesale. Everything top-level (`progress_*`, `is_abortable`, `error_message`, `timeout`, …) stays public. `Task.update_private_properties()` merges into the bucket so independent writers (job-id at pickup, cancel-handle during execution) don't clobber each other. Addresses the review's "public/private metadata boundary" finding. **Planned on this branch (rolling into this PR):** - **High — 202 status-poll cursor precision/skew.** `async_queries.py` captures the 202 cursor with an unfloored `datetime.now()`, while the status-changes read floors to whole seconds for MySQL's second-precision `changed_on` — so a same-second task transition can be missed under the `>=` bound. Fix the cursor floor (polling-mode correctness). - **WebSocket as a first-class transport.** Stop steady-state `status_changes` polling while the socket is healthy; rely on push + a task-id-aware resync on (re)connect (which also sidesteps the cursor entirely on the ws path). - **`related_subscribers` scoped to visible tasks** (currently global). - **WebSocket origin/cookie hardening docs** (ALLOWED_ORIGINS, secure cookies, SameSite, connection caps). ### TESTING INSTRUCTIONS - **Abort teardown:** with a Redis coordination backend + a cancellable engine, run an async chart query; the Celery task wall-time should now match the query time (no ~1s tail). A user abort still cancels the running query. - **Private properties:** `GET /api/v1/task/<uuid>` — `properties` must not contain `celery_task_id`/`cancel_query_id`/`cancel_database_id`; `progress_*`/`error_message` remain. - **Automated:** `pytest tests/unit_tests/tasks/ tests/unit_tests/coordination/`. ### ADDITIONAL INFORMATION - [ ] Has associated issue: - [ ] Required feature flags: `GLOBAL_TASK_FRAMEWORK` (+ `GLOBAL_ASYNC_QUERIES` for the chart-data path) - [ ] Changes UI - [ ] Includes DB Migration - [ ] Introduces new feature or API - [ ] Removes existing feature or API -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
