[
https://issues.apache.org/jira/browse/ATLAS-1851?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ashutosh Mestry updated ATLAS-1851:
-----------------------------------
Description:
h3. Background & Scenarios
Import operations have a potential to go on for several hours. It is likely
that the the operation does not complete and the server crashes.
To improve on this situation, it would be beneficial to be able to specify the
last successfully imported entity using either the entity guid or the position
in the entity creation order. That way, the import operation could resume from
a later point rather than starting from scratch.
This improvement will thus allow Import API to resume where it last stopped.
h3. Implementation
_AtlasImportRequest_ should support 2 additional options:
* _startGuid_: Entity GUID that should be used to resume import operation.
* _startPosition_: Starting position of the entity within the _creationOrder_.
Within _ZipSource_ this option would be used to set the internal iterator to a
position from which the import can start. _ImportService_ could read do the
needful before starting to process entities.
h3. Additional Implementation Guidance
Logs generated during import operations do not have the necessary information
necessary to help the consumer with information needed to use these new options.
It would be worth while to include the entity index within _creationOrder_ and
the guid of the entity that was successfully imported.
was:
h3. Background & Scenarios
Import operations have a potential to go on for several hours. It is likely
that the the operation does not complete and the server crashes.
To improve on this situation, it would be beneficial to be able to specify the
last successfully imported entity using either the entity guid or the position
in the entity creation order. That way, the import operation could resume from
a later point rather than starting from scratch.
h3. Implementation
_AtlasImportRequest_ should support 2 additional options:
* _startGuid_: Entity GUID that should be used to resume import operation.
* _startPosition_: Starting position of the entity within the _creationOrder_.
Within _ZipSource_ this option would be used to set the internal iterator to a
position from which the import can start. _ImportService_ could read do the
needful before starting to process entities.
h3. Additional Implementation Guidance
Logs generated during import operations do not have the necessary information
necessary to help the consumer with information needed to use these new options.
It would be worth while to include the entity index within _creationOrder_ and
the guid of the entity that was successfully imported.
> Import API: Ability to Specify Starting Entity for Import
> ---------------------------------------------------------
>
> Key: ATLAS-1851
> URL: https://issues.apache.org/jira/browse/ATLAS-1851
> Project: Atlas
> Issue Type: Improvement
> Components: atlas-core
> Affects Versions: trunk
> Reporter: Ashutosh Mestry
> Assignee: Ashutosh Mestry
> Fix For: trunk
>
>
> h3. Background & Scenarios
> Import operations have a potential to go on for several hours. It is likely
> that the the operation does not complete and the server crashes.
> To improve on this situation, it would be beneficial to be able to specify
> the last successfully imported entity using either the entity guid or the
> position in the entity creation order. That way, the import operation could
> resume from a later point rather than starting from scratch.
> This improvement will thus allow Import API to resume where it last stopped.
> h3. Implementation
> _AtlasImportRequest_ should support 2 additional options:
> * _startGuid_: Entity GUID that should be used to resume import operation.
> * _startPosition_: Starting position of the entity within the _creationOrder_.
> Within _ZipSource_ this option would be used to set the internal iterator to
> a position from which the import can start. _ImportService_ could read do the
> needful before starting to process entities.
> h3. Additional Implementation Guidance
> Logs generated during import operations do not have the necessary information
> necessary to help the consumer with information needed to use these new
> options.
> It would be worth while to include the entity index within _creationOrder_
> and the guid of the entity that was successfully imported.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
