[
https://issues.apache.org/jira/browse/ATLAS-5317?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sheetal Shah updated ATLAS-5317:
--------------------------------
Summary: Make Atlas Purge API more resilient (was: Make Atlas Purge API
more resilient by continuing on per-entity errors and returning structured
success/failure response.)
> Make Atlas Purge API more resilient
> -----------------------------------
>
> Key: ATLAS-5317
> URL: https://issues.apache.org/jira/browse/ATLAS-5317
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
> Reporter: Sheetal Shah
> Assignee: Sheetal Shah
> Priority: Major
>
> h2. Problem Statement
> Atlas exposes a purge API ({{{}PUT /api/atlas/admin/purge{}}}) to hard-delete
> entities. The API accepts a batch of GUIDs but fails the entire request if
> any single entity delete throws an exception. This all-or-nothing behavior
> blocks large clean-up jobs.
> Key issues:
> * One corrupt, missing, or locked GUID causes the entire batch to roll back
> with HTTP 500
> * No structured failure reporting — bad GUIDs are only logged as
> {{{}WARN{}}}; callers cannot identify which GUIDs failed
> * Audit entry stores all input GUIDs in a single row, which can exceed safe
> size limits and cause transaction rollbacks
> * REST purge and background {{PurgeService}} cron can run concurrently on
> the same GUIDs, causing {{PermanentLockingException}}
> * No input validation — non-GUID strings passed to the API cause unexpected
> failures
> ----
> h2. Requirements
> # Resilient purge — Continue purging remaining entities on per-entity error.
> Return {{failedEntities}} (guid, error code, message) alongside successfully
> purged entities. Return HTTP 207 on partial success.
> # Bounded audit — Write one audit entry per mini-batch instead of one
> oversized entry per request.
> # Fix purge logic — Process in mini-batches (default 50 GUIDs per
> transaction); retry on locking errors; isolate corrupt/missing GUIDs as
> skippable instead of failing the whole batch. Validate input and reject
> non-GUID strings with HTTP 400. Prevent concurrent REST + scheduled purge
> conflicts.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
