[ https://issues.apache.org/jira/browse/ATLAS-3198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16865889#comment-16865889 ]
ASF subversion and git services commented on ATLAS-3198: -------------------------------------------------------- Commit 4e1eeb4cdcd8bc0601564434176eb200b2811c2e in atlas's branch refs/heads/master from Yu-Hsin Shih [ https://gitbox.apache.org/repos/asf?p=atlas.git;h=4e1eeb4 ] ATLAS-3198: Stop pushing message to ATLAS_HOOK for Truncate Table/Delete data/Update data Both ATLAS_HOOK and ATLAS_ENTITIES should stop receiving message from Truncate Table/Delete data/Update data Signed-off-by: Sarath Subramanian <sar...@apache.org> > update/delete/truncate operations ATLAS_HOOK and ATLAS_ENTITIES improvements > ----------------------------------------------------------------------------- > > Key: ATLAS-3198 > URL: https://issues.apache.org/jira/browse/ATLAS-3198 > Project: Atlas > Issue Type: Improvement > Affects Versions: 2.0.0 > Reporter: Umesh Padashetty > Assignee: Yu-Hsin Chang > Priority: Critical > Fix For: trunk > > Attachments: ATLAS-3198.patch, ATLAS_HOOK and ATLAS_ENTITIES > messages.xlsx > > > With the recent changes, we have now stopped ingesting simple insert > operations since with this operation, there is no change in the metadata. > Ideally, we should be doing similar improvements w.r.t. update, delete and > truncate operations too. > Currently, for update/delete/truncate, we are pushing messages in both > ATLAS_HOOK as well as ATLAS_ENTITIES topic, which can be avoided. > If we stop ingesting updates/deletes/truncates, it will further help us > reduce the burden on Kafka and save CPU cycles > I have created and attached an excel sheet, with the list of operations and > the corresponding messages generated in ATLAS_HOOK and ATLAS_ENTITIES topic, > along with suitable comments. -- This message was sent by Atlassian JIRA (v7.6.3#76005)