[ https://issues.apache.org/jira/browse/ATLAS-158?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14988893#comment-14988893 ]
Shwetha G S commented on ATLAS-158: ----------------------------------- Attaching the patch here as I won't have permission to update the patch on review board. Changes done: 1. Removed dependency of repository module in notification. Moved listeners to common module 2. Added atlas.kafka.hook.group.id in distro/src/conf/application.properties - required for atlas deployment. The consumer group id changed to 'atlas' as the value should represent the consumer so that any debugging from kafka end can relate to the actual consumer. Also, removed the property atlas.kafka.entities.group.id to force users to set this value 3. Replaced TraitInfo with IStruct 4. Changed EntityNotification interface. I think the new interface makes sense for the following reasons: a. The entity returned IReferenceableInstance already has getTraits(). So, EntityNotification.getAllTraits() can represent the flattened traits(including super types) b. Traits as map doesn't work - Assume traits t1, t2, t3. traits t2 and t3 extend t1. If traits t2 and t3 are associated with the entity, getAllTraits() should return t1, t2, t1, t3. So, it has to be a list and not map c. For trait_add/delete operation, the traits added/deleted won't be a single trait, it will be multiple traits because of inheritance. And just the trait name is not enough as the same super trait may be shared across multiple traits. In general, its better to just give the whole entity and all its latest traits even in case of entity/trait add/update/delete. This will work even for ranger as ranger maintains resource-tag mapping and they can just replace all entity mappings. 5. Deleted AbstractNotificationConsumerTest and EntityNotificationImplTest as they weren't compiling with the new changes. They didn't really test anything much. If you want, you can add them back The patch compiles and all the tests pass except for NotificationHookConsumerIT. Looks like kafka doesn't stop/start clean. Pending are: 1. Debug NotificationHookConsumerIT failure 2. Add UT for EntityNotificationImpl.getAllTraits() 3. There are no tests at all for the end to end flow of entity notification. Please add ITs for all operation types 4. Add documentation I think without the ITs, its risky to commit the patch 2. > Provide Atlas Entity Change Notification > ---------------------------------------- > > Key: ATLAS-158 > URL: https://issues.apache.org/jira/browse/ATLAS-158 > Project: Atlas > Issue Type: Task > Reporter: Tom Beerbower > Assignee: Tom Beerbower > Fix For: 0.6-incubating > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)