[ https://issues.apache.org/jira/browse/ATLAS-4358?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ashutosh Mestry reassigned ATLAS-4358: -------------------------------------- Assignee: Ashutosh Mestry > Mapping for some internal Atlas attributes ( like __patch.type , __timestamp, > etc) does not exist in Elasticsearch > ------------------------------------------------------------------------------------------------------------------ > > Key: ATLAS-4358 > URL: https://issues.apache.org/jira/browse/ATLAS-4358 > Project: Atlas > Issue Type: Bug > Reporter: Anshul Mehta > Assignee: Ashutosh Mestry > Priority: Major > Attachments: patch-manager-fix.patch > > Time Spent: 1h 40m > Remaining Estimate: 0h > > *Impact of the problem -* > * Atlas pod taking much longer to become active and this time keep > increasing as the assets increase. This basically means a downtime on every > Atlas release. > * Not able to filter via basic search on attributes like {{__timestamp}} , > {{__modificationTimestamp}} , {{createdBy}} and {{modifiedBy}} . > *Issue -* > So just before creating the mapping in the mixed index (ES index) Atlas > creates something called {{propertyKey}} and this propertyKey is used to > create the mapping. The code is written in a way that checks if propertyKey > for the current property is null or not. If it is null it creates the > propertyKey and then adds it to mixed index. If it is not Null it assumes > that the property has already been added to the index and so skips adding it. > Now in our case when Atlas checked the propertyKey it was not null (which > should not have been the case) therefore Atlas skipped adding it to the mixed > index and so these properties never got added to the mixed index. This > basically meant propertyKey for these properties were getting created > somewhere else. We looked into the entire codebase but could not find the use > of makePropertyKey method ( which is used to create propertyKey) or any other > similar method. > Then I saw certain java patch vertices getting created even before these > internal attributes are added to various indices. Though these patches were > applied later once all internal attributes were added to all the indices. > Now, these patch vertices have 9 attributes and we releaized these 9 > attributes are the only attributes missing from ES. So basically when patch > vertices got created and these vertices with their attributes got added to > cassandra via janusgraph, janusgraph automatically created propertyKey for > all these attributes (the janusgraph's makePropertyKey method is not called > during this process anywhere in the Atlas code). And because internal > attributes were getting added to indices in another thread at the same time, > when code checked for propertyKey, it was not null and so it did not add the > property to the mixed index. -- This message was sent by Atlassian Jira (v8.3.4#803005)