-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/66184/#review199992
-----------------------------------------------------------


Ship it!




Ship It!

- Madhan Neethiraj


On March 26, 2018, 7:38 p.m., Ashutosh Mestry wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/66184/
> -----------------------------------------------------------
> 
> (Updated March 26, 2018, 7:38 p.m.)
> 
> 
> Review request for atlas, Madhan Neethiraj, Ruchi Solani, and Sarath 
> Subramanian.
> 
> 
> Bugs: ATLAS-2461
>     https://issues.apache.org/jira/browse/ATLAS-2461
> 
> 
> Repository: atlas
> 
> 
> Description
> -------
> 
> **Background** 
> The migration utility committed ealier has couple of short comings:
> - Relies on Export service.
>   - Needs _export-options.json_ to be specified.
>   - Exporting everything means meticuloulsy updating the options file. It is 
> likely some specification is missed and hence will lead to less data being 
> migrated. 
> - Suffers from performance problems for large data sets.
> 
> **Approach**
> The new approach uses _Titan's_ _GraphSON_ writer. This is configured to 
> export all data in _EXTENDED_ format.
> 
> The _EXTENDED_ format separates _vertices_ and _edges_. This open other 
> interesting avenues for import.
> 
> **Implementation**
> - Modified _Exporter_ to use _AtlasTypeRegistry_ and _GraphSONWriter_.
> - Produced files: 
>    - _atlas-typedef.json_: Contains type definitions of all types.
>    - _atlas-migration-data.json_: Contains data from the database.
> 
> 
> Diffs
> -----
> 
>   distro/src/main/assemblies/migration-exporter.xml 8f751ff9 
>   tools/atlas-migration-exporter/pom.xml 5c6c61ee 
>   
> tools/atlas-migration-exporter/src/main/java/org/apache/atlas/migration/Exporter.java
>  a9873df0 
>   tools/atlas-migration-exporter/src/main/resources/README 2f2bf3e1 
>   tools/atlas-migration-exporter/src/main/resources/atlas-log4j.xml 
> PRE-CREATION 
>   tools/atlas-migration-exporter/src/main/resources/atlas_migration.py 
> 199cde28 
>   
> tools/atlas-migration-exporter/src/main/resources/migration-export-request.json
>  64002aff 
> 
> 
> Diff: https://reviews.apache.org/r/66184/diff/3/
> 
> 
> Testing
> -------
> 
> **Functional tests**
> Export from repositories with:
> - Custom types.
> - Complex lineages.
> - Created hive entities via beeline.
> - Imported data.
> 
> **Gremlin Shell**
> - Used _Gremlin_ shell to perform export operation.
> 
> 
> Thanks,
> 
> Ashutosh Mestry
> 
>

Reply via email to