[ https://issues.apache.org/jira/browse/ATLAS-3198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Yu-Hsin Chang updated ATLAS-3198: --------------------------------- Code review link: [https://reviews.apache.org/r/70784/] > update/delete/truncate operations ATLAS_HOOK and ATLAS_ENTITIES improvements > ----------------------------------------------------------------------------- > > Key: ATLAS-3198 > URL: https://issues.apache.org/jira/browse/ATLAS-3198 > Project: Atlas > Issue Type: Improvement > Affects Versions: 2.0.0 > Reporter: Umesh Padashetty > Assignee: Yu-Hsin Chang > Priority: Critical > Fix For: trunk > > Attachments: ATLAS-3198.patch, ATLAS_HOOK and ATLAS_ENTITIES > messages.xlsx > > > With the recent changes, we have now stopped ingesting simple insert > operations since with this operation, there is no change in the metadata. > Ideally, we should be doing similar improvements w.r.t. update, delete and > truncate operations too. > Currently, for update/delete/truncate, we are pushing messages in both > ATLAS_HOOK as well as ATLAS_ENTITIES topic, which can be avoided. > If we stop ingesting updates/deletes/truncates, it will further help us > reduce the burden on Kafka and save CPU cycles > I have created and attached an excel sheet, with the list of operations and > the corresponding messages generated in ATLAS_HOOK and ATLAS_ENTITIES topic, > along with suitable comments. -- This message was sent by Atlassian JIRA (v7.6.3#76005)