[
https://issues.apache.org/jira/browse/ATLAS-1114?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jeffrey Hagelberg updated ATLAS-1114:
-
Description:
General Atlas Perforance Improvements
Apply performance fixes for create/update entities from IBM fork to Atlas.
During our performance profiling, we found a number of performance hotspots in
JProfiler. Our main findings were
- multiple queries were being executed for each instance being
created/updated to find matches by unique attribute.
- one query was being executed for each instance being created/updated to
find the corresponding vertex if there is one
- Calculating the value of the full text property was taking a significant
portion of the time to create/update entities, mainly due to its calls to
getVertexForGUID
The changes we put in do the following:
- batch lookups by guid when create/update entities. Execute one
AtlasGraphQuery to find them all.
- batch lookups by unique attribute when create/update entities. Execute one
AtlasGraphQuery per class to find unique attribute matches.
- find all existing vertices up front during create/update entity. Use those
vertices during the graph mapping process to avoid running unnecessary graph
queries
- reuse reference vertices from instance to graph mapping when computing full
text property
was:
General Atlas Perforance Improvements
Apply performance fixes for create/update entities from IBM fork to Atlas.
During our performance profiling, we found a number of performance hotspots in
JProfiler. Note that our profiling was done with the IBM Graph implementation,
but the performance optimizations are very general in nature and help with
performance on Titan as well. Our main findings were
- multiple queries were being executed for each instance being
created/updated to find matches by unique attribute.
- one query was being executed for each instance being created/updated to
find the corresponding vertex if there is one
- Calculating the value of the full text property was taking a significant
portion of the time to create/update entities, mainly due to its calls to
getVertexForGUID
The changes we put in do the following:
- batch lookups by guid when create/update entities. Execute one
AtlasGraphQuery to find them all.
- batch lookups by unique attribute when create/update entities. Execute one
AtlasGraphQuery per class to find unique attribute matches.
- find all existing vertices up front during create/update entity. Use those
vertices during the graph mapping process to avoid running unnecessary graph
queries
- reuse reference vertices from instance to graph mapping when computing full
text property
> Performance improvements for create/update entities
> ---
>
> Key: ATLAS-1114
> URL: https://issues.apache.org/jira/browse/ATLAS-1114
> Project: Atlas
> Issue Type: Bug
>Reporter: Jeffrey Hagelberg
>Assignee: Jeffrey Hagelberg
>
> General Atlas Perforance Improvements
> Apply performance fixes for create/update entities from IBM fork to Atlas.
> During our performance profiling, we found a number of performance hotspots
> in JProfiler. Our main findings were
>- multiple queries were being executed for each instance being
> created/updated to find matches by unique attribute.
> - one query was being executed for each instance being created/updated to
> find the corresponding vertex if there is one
> - Calculating the value of the full text property was taking a significant
> portion of the time to create/update entities, mainly due to its calls to
> getVertexForGUID
> The changes we put in do the following:
> - batch lookups by guid when create/update entities. Execute one
> AtlasGraphQuery to find them all.
> - batch lookups by unique attribute when create/update entities. Execute one
> AtlasGraphQuery per class to find unique attribute matches.
> - find all existing vertices up front during create/update entity. Use those
> vertices during the graph mapping process to avoid running unnecessary graph
> queries
> - reuse reference vertices from instance to graph mapping when computing full
> text property
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
