[
https://issues.apache.org/jira/browse/ATLAS-5312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18103236#comment-18103236
]
ASF subversion and git services commented on ATLAS-5312:
--------------------------------------------------------
Commit eda0c19765d2a61c1e59cf46d54aea76b0ca3f6d in atlas's branch
refs/heads/master from bhor-sanket
[ https://gitbox.apache.org/repos/asf?p=atlas.git;h=eda0c1976 ]
ATLAS-5312: Add delete propagation support via propagateDelete flag o… (#670)
> Handle Delete Propogation between Related entities
> --------------------------------------------------
>
> Key: ATLAS-5312
> URL: https://issues.apache.org/jira/browse/ATLAS-5312
> Project: Atlas
> Issue Type: New Feature
> Reporter: sanket bhor
> Assignee: sanket bhor
> Priority: Major
> Time Spent: 40m
> Remaining Estimate: 0h
>
> *Problem Statement*
>
>
>
> When
> a container entity is deleted in Atlas, entities linked via AGGREGATION or
> ASSOCIATION relationships are NOT deleted — they become orphaned stale
> metadata visible in UI, search, lineage, and
> governance policies.
> *Two gaps exist:*
>
>
> 1. Delete Cascade (AGGREGATION): When a trino_schema is deleted (e.g., DROP
> SCHEMA sales CASCADE), trino_table entities linked via AGGREGATION
> (trino_table_schema) are NOT deleted. Only
> COMPOSITION-owned children are cascaded today via
> DeleteHandlerV1.getOwnedVertices().
>
> 2. Delete Propagation (Cross-system aliases): When a source entity is
> deleted (e.g., hive_table or hive_db), alias entities in other systems (e.g.,
> trino_table via trino_table_hive_table, trino_schema
> via trino_schema_hive_db) remain as stale metadata pointing to non-existent
> source entities.
>
>
>
> *Current behavior:*
>
>
> - DeleteHandlerV1.getOwnedVertices() only follows isOwnedRef=true
> attributes (injected only for COMPOSITION relationships)
>
> - AGGREGATION/ASSOCIATION edges: only the relationship edge is removed; the
> child/alias entity persists
>
> - Result: orphaned tables, columns, schemas visible in Atlas after source
> deletion
> *Proposed Solution (High Level)*
>
>
>
>
>
> Add a typedef-driven propagateDelete boolean flag on
> AtlasRelationshipEndDef (mirrors existing propagateRename pattern):
>
>
>
>
> - Flag is configured via model patches (SET_PROPAGATE_DELETE action) — no
> hook-side changes required
>
> - At typedef resolution time, AtlasEntityType pre-computes
> deletePropagationTargets list
>
> - At runtime, DeleteHandlerV1 traverses propagateDelete-marked edges after
> getOwnedVertices() and adds connected entities to the deletion set
>
> - Multi-hop propagation supported via recursion (e.g., hive_db →
> trino_schema → trino_table → trino_column)
>
> - Idempotent: skip already-DELETED entities; visited-set prevents cycles
>
>
> - All propagated deletes happen within the same @GraphTransaction — atomic
> commit/rollback
>
> - Supports both soft delete and hard delete (propagation targets inherit
> parent's delete type)
>
> *Steps to Reproduce*
> Scenario A — AGGREGATION orphan (trino_schema → trino_table):
> 1. Create trino_schema entity (qualifiedName=cat1.sales@inst1) with 3
> trino_table entities linked via trino_table_schema relationship
> 2. Send ENTITY_DELETE_V2 event for trino_schema:
> {"type":"ENTITY_DELETE_V2","user":"trino","entities":[\{"typeName":"trino_schema","uniqueAttributes":{"qualifiedName":"cat1.sales@inst1"}}]}
> 3. Observe: trino_schema is deleted, but all 3 trino_table entities and their
> trino_column entities remain in Atlas (orphaned)
> Scenario B — Cross-system alias orphan (hive_table → trino_table):
> 1. Create hive_table entity (qualifiedName=default.orders@cluster) linked to
> trino_table (qualifiedName=cat1.schema1.orders@inst1) via
> trino_table_hive_table relationship
> 2. Send ENTITY_DELETE_V2 event for hive_table:
> {"type":"ENTITY_DELETE_V2","user":"hive","entities":[\{"typeName":"hive_table","uniqueAttributes":{"qualifiedName":"default.orders@cluster"}}]}
> 3. Observe: hive_table and its hive_column entities are deleted, but
> trino_table and its trino_column entities remain (stale alias)
> Scenario C — Cross-system schema orphan (hive_db → trino_schema):
> 1. Create hive_db (qualifiedName=sales@cluster) linked to trino_schema
> (qualifiedName=cat1.sales@inst1) via trino_schema_hive_db relationship
> 2. Send ENTITY_DELETE_V2 event for hive_db:
> {"type":"ENTITY_DELETE_V2","user":"hive","entities":[\{"typeName":"hive_db","uniqueAttributes":{"qualifiedName":"sales@cluster"}}]}
> 3. Observe: hive_db deleted, but trino_schema, its trino_table entities, and
> trino_column entities all remain (stale)
> *Acceptance Criteria*
> Functional
> - [ ] Deleting a trino_schema cascades deletion to all trino_table entities
> linked via trino_table_schema (AGGREGATION) and their trino_column entities
> (COMPOSITION)
> - [ ] Deleting a hive_table propagates deletion to linked trino_table (via
> trino_table_hive_table) and its trino_column entities
> - [ ] Deleting a hive_db propagates deletion to linked trino_schema (via
> trino_schema_hive_db), which further cascades to all its trino_table and
> trino_column entities
> - [ ] Propagation is unidirectional: deleting trino_table does NOT delete
> hive_table; deleting trino_schema does NOT delete hive_db
> - [ ] Multi-hop propagation works: hive_db → trino_schema → trino_table →
> trino_column (full chain)
> - [ ] Both soft delete and hard delete modes are supported (propagation
> targets inherit parent's delete type)
> - [ ] Feature is opt-in via model patches — no behavior change without
> explicit SET_PROPAGATE_DELETE patch enablement
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
