[ https://issues.apache.org/jira/browse/ATLAS-4348?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17382369#comment-17382369 ]
ASF subversion and git services commented on ATLAS-4348: -------------------------------------------------------- Commit 8fb29d30baf9328687916cb6cdf4981a7915d805 in atlas's branch refs/heads/branch-2.0 from Radhika Kundam [ https://gitbox.apache.org/repos/asf?p=atlas.git;h=8fb29d3 ] ATLAS-4348: Atlas-Kafka Hook : When a producer publishes messages to multiple topics, the latest relationship is marked ACTIVE , rest are marked DELETED Signed-off-by: Sarath Subramanian <sar...@apache.org> (cherry picked from commit 8725f31af9821907ced99a829b9e0dbbf3f6de2b) > Atlas-Kafka Hook : When a producer publishes messages to multiple topics, the > latest relationship is marked ACTIVE , rest are marked DELETED > -------------------------------------------------------------------------------------------------------------------------------------------- > > Key: ATLAS-4348 > URL: https://issues.apache.org/jira/browse/ATLAS-4348 > Project: Atlas > Issue Type: Task > Components: atlas-core > Affects Versions: 2.1.0 > Reporter: Radhika Kundam > Assignee: Radhika Kundam > Priority: Major > Attachments: > 0001-ATLAS-4348-Atlas-Kafka-Hook-appendRelationshipsOnPar.patch > > > *Steps to reproduce the issue*: > Created 2 topics : test1, test2 > Using console consumer produced messages to test1 and test2. > Relationship between console-producer-lineage and test1 is ACTIVE now. > Relationship between console-producer-lineage and test1 is set to DELETED > when messages are published to test2. > *Root cause of issue:* > When messages published on topic, Atlas hook receives partial_update request > with only one topic as output and by default partial update deletes > relationships between previous topics by considering the only topic(which is > in partial_update request) as the active. > *Proposed Solution:* > Introducing typedef option "appendRelationshipsOnPartialUpdate" which will > hold list of relationship attribute names. While Atlas startup, it will > resolve entity type definition and all the specified relationship attributes > definitions will be updated with typedef property "isAppendOnPartialUpdate" > only for that entity type. > With this solution, we can keep all the previous relationship entries active > along with latest relationship entry for the specific entity by providing > typedef options as below. > "typeDefOptions": { "appendRelationshipsOnPartialUpdate": "[\"inputs\", > \"outputs\"]"} > > This solution can be used for any entity type to consider all the existing > relationship entries as active along with new entries from partial_update. > -- This message was sent by Atlassian Jira (v8.3.4#803005)