aminghadersohi commented on issue #29839:
URL: https://github.com/apache/superset/issues/29839#issuecomment-3815402536
Hey Ville, great proposal! A couple things came to mind:
Deduplication race conditions - The "check if exists, then insert" pattern
for deduplication can race under load. Two requests could both see no entry
and both try to schedule. Might be worth specifying the locking strategy -
something like INSERT ... ON CONFLICT DO NOTHING or Redis SETNX. The current
GAQ code uses Redis for exactly this reason.
Retry semantics - Alerts & Reports has pretty sophisticated retry logic
(autoretry_for, max_retries=3, exponential backoff). The AsyncTask model
doesn't have any retry-related fields. Will GATF support configurable retry
policies per task type, or would tasks need to handle their own retries
internally?
Also been working on MCP background tasks using FastMCP's task protocol -
could be interesting to have MCP tools create GATF entries so AI-initiated
operations get the same deduplication/visibility as web-initiated ones. Happy
to chat if useful.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
