[ https://issues.apache.org/jira/browse/ATLAS-2816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16581936#comment-16581936 ]
ASF subversion and git services commented on ATLAS-2816: -------------------------------------------------------- Commit 24830e6bc8624ae6d4abbe4f9037f0de8ad57cd0 in atlas's branch refs/heads/master from [~chengbing.liu] [ https://git-wip-us.apache.org/repos/asf?p=atlas.git;h=24830e6 ] ATLAS-2816: Allow ignoring relationship in EntityGraphRetriever for FullTextMapperV2 Signed-off-by: apoorvnaik <apoorvn...@apache.org> > Allow ignoring relationship in EntityGraphRetriever for FullTextMapperV2 > ------------------------------------------------------------------------ > > Key: ATLAS-2816 > URL: https://issues.apache.org/jira/browse/ATLAS-2816 > Project: Atlas > Issue Type: Bug > Affects Versions: 1.0.0 > Reporter: Chengbing Liu > Assignee: Apoorv Naik > Priority: Major > Attachments: ATLAS-2816.01.patch, ATLAS-2816.02.patch > > > We encountered a problem when using Hive bridge in production. One database > has 5000+ tables. Importing the first table costs only tens of milliseconds, > and then it becomes slower with more tables. In the end, it costs 1~2 seconds > to import one table. > After investigation, we realized that it is not necessary for the > {{FullTextMapperV2}} to retrieve all the relationship of the database each > time a table is imported. The time complexity of importing a whole database > actually goes to O(n^2) (n is number of tables). > We propose to add a parameter to the constructor of {{EntityGraphRetriever}}: > {{ignoreRelationship}}. When set to true, {{mapVertexToAtlasEntity}} will > skip the {{mapRelationshipAttributes}} call. Since {{FullTextMapperV2}} will > not use relationship attributes of the entity, this can save plenty of time > when importing entities with a large number of relations. -- This message was sent by Atlassian JIRA (v7.6.3#76005)