Ashutosh Mestry created ATLAS-4121: -------------------------------------- Summary: Import Service: Improve Speed of Ingest for replicatedTo Option Key: ATLAS-4121 URL: https://issues.apache.org/jira/browse/ATLAS-4121 Project: Atlas Issue Type: Improvement Components: atlas-core Affects Versions: 2.1.0 Reporter: Ashutosh Mestry Assignee: Ashutosh Mestry Fix For: trunk
*Background* For Import replication options that includes _skipLineage_ and _replicatedTo_, __ current implementation uses one entity-at-time import approach. This approach can be end up taking long time to import for large payloads. *Solution* * Use an approach that allows for entity of a type to be concurrently created. * This assumes that the incoming stream of entities to be imported is topologically sorted (where parent entities appear before the child entities). * Create entities of a type. * Once all entities of a type are created, fetch next type. -- This message was sent by Atlassian Jira (v8.3.4#803005)