----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/56959/ -----------------------------------------------------------
(Updated May 23, 2017, 10:11 p.m.) Review request for atlas and Madhan Neethiraj. Changes ------- Added comments to patch. Bugs: ATLAS-1825 https://issues.apache.org/jira/browse/ATLAS-1825 Repository: atlas Description ------- Please refer to [ATLAS-1825](https://issues.apache.org/jira/browse/ATLAS-1825) for scenarios and solution approach. Impact ====== * New class _AtlasImportTransform_ is now part of _AtlasImportRequest_. This holds a map of entity-type to attribute and the tranforms to be applied. * New class _AtlasImportTransformBuiler_ reads the requests specified in the query string of the import API and creates a _AtlasImportTransfor_ structure. * _AtlasImportTransformBuiler_ has new classes that support transformations. They are: * _replace_ Implements find and replace on attribute values. * _lowercase_ Converts attribute value to lower case. * Additional classes that unit test these classes have been added. Implementation Approach ======================= * _ZipSource_ now accepts _AtlasImportTransform_ object. It applies the transform every entity that is fetched from the source. CURL ==== ``` curl -g -X POST -u admin:admin -H "Content-Type: application/octet-stream" -H "Cache-Control: no-cache" --data-binary @../docs/DefaultDB-Stocks-2.zip "http://localhost:21000/api/atlas/admin/import?tr={'transforms':{'hive_db':{'qualifiedName':['lowercase','replace:@cl1:@cl2']}}}" ``` ``` curl -g -X POST -u admin:admin -H "Content-Type: application/octet-stream" -H "Cache-Control: no-cache" --data-binary @../docs/DefaultDB-Stocks-2.zip "http://localhost:21000/api/atlas/admin/import?tr={'transforms':{'hive_table':{'qualifiedName':['replace:@cl1:@cl2']}}}" ``` Note: _-g_ needs to be added to CURL calls to prevent gobble check. Documentation ============= Need to update TWIKI documents. Diffs (updated) ----- intg/pom.xml 7f3ab12b intg/src/main/java/org/apache/atlas/model/impexp/AtlasImportRequest.java 7530196d intg/src/main/java/org/apache/atlas/model/impexp/AtlasImportTransform.java PRE-CREATION intg/src/main/java/org/apache/atlas/model/impexp/AtlasImportTransformBuilder.java PRE-CREATION intg/src/main/java/org/apache/atlas/model/impexp/ImportTransform.java PRE-CREATION intg/src/test/java/org/apache/atlas/model/impexp/AtlasImportTransformBuilderTest.java PRE-CREATION intg/src/test/java/org/apache/atlas/model/impexp/ContainsReplaceTransformTest.java PRE-CREATION intg/src/test/java/org/apache/atlas/model/impexp/TransformEvaluatorTest.java PRE-CREATION repository/src/main/java/org/apache/atlas/repository/impexp/ImportService.java 9ec15e00 repository/src/main/java/org/apache/atlas/repository/impexp/ZipSource.java 87c9f0eb repository/src/test/java/org/apache/atlas/repository/impexp/ImportServiceTest.java 91966f8f Diff: https://reviews.apache.org/r/56959/diff/3/ Changes: https://reviews.apache.org/r/56959/diff/2-3/ Testing ------- **Unit Test** * Added tests for *AtlasImportTransformBuilder* and contained classes. * Updated *ZipSourceTest* * Added tests to *ImportServiceTest* to exercise new option. **Functional Test** * Small size data export & import. * Medium size data export & import. **Performance Tests** Not exhaustive. Simple stop watch tests to compare against baseline. Thanks, Ashutosh Mestry