----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/56959/ -----------------------------------------------------------
(Updated May 25, 2017, 5:16 a.m.) Review request for atlas and Madhan Neethiraj. Changes ------- Fixed problem with enum used as factory. Bugs: ATLAS-1825 https://issues.apache.org/jira/browse/ATLAS-1825 Repository: atlas Description ------- Please refer to [ATLAS-1825](https://issues.apache.org/jira/browse/ATLAS-1825) for scenarios and solution approach. Impact ====== * New class _AtlasImportTransform_ is now part of _AtlasImportRequest_. This holds a map of entity-type to attribute and the tranforms to be applied. * New class _AtlasImportTransformBuiler_ reads the requests specified in the query string of the import API and creates a _AtlasImportTransfor_ structure. * _AtlasImportTransformBuiler_ has new classes that support transformations. They are: * _replace_ Implements find and replace on attribute values. * _lowercase_ Converts attribute value to lower case. * Additional classes that unit test these classes have been added. Implementation Approach ======================= * _ZipSource_ now accepts _AtlasImportTransform_ object. It applies the transform every entity that is fetched from the source. * _Import API_ now accepts multi-part input. This allows for elegantly specifying import parameters as well as the binary input from the file. * Please take note of _ImportTransformerEnum_. This is first case where Enum has state variables. CURL ==== *Setup* Create file with these contents call it _importTransform.json_: ```javascript { "options": {}, "importTransform": { "transforms": { "hive_table": { "qualifiedName": [ "lowercase", "replace:@cl1:@cl2" ] } } } } ``` ``` curl -g -X POST -u admin:admin -H "Content-Type: multipart/form-data" -H "Cache-Control: no-cache" -F request=@../docs/importTransform.json -F data=@../docs/Stocks-2.zip "http://localhost:21000/api/atlas/admin/import" ``` Note: _-g_ needs to be added to CURL calls to prevent gobble check. This is not required anymore. Notice the output from the API, it has the _AtlasRequest_ that was passed as input. Documentation ============= Need to update TWIKI documents. Diffs (updated) ----- intg/src/main/java/org/apache/atlas/model/impexp/AtlasImportRequest.java 7530196d intg/src/main/java/org/apache/atlas/model/impexp/AtlasImportTransforms.java PRE-CREATION intg/src/main/java/org/apache/atlas/model/impexp/ImportTransformerEnum.java PRE-CREATION intg/src/test/java/org/apache/atlas/model/impexp/ImportTransformerEnumTest.java PRE-CREATION pom.xml a93b8ad9 repository/src/main/java/org/apache/atlas/repository/impexp/ImportEntityTransformer.java PRE-CREATION repository/src/main/java/org/apache/atlas/repository/impexp/ImportService.java 9ec15e00 repository/src/main/java/org/apache/atlas/repository/impexp/ZipSource.java 87c9f0eb repository/src/test/java/org/apache/atlas/repository/impexp/ImportEntityTransformerTest.java PRE-CREATION repository/src/test/java/org/apache/atlas/repository/impexp/ImportServiceTest.java 08e9ee81 repository/src/test/java/org/apache/atlas/repository/impexp/TransformEvaluatorTest.java PRE-CREATION repository/src/test/java/org/apache/atlas/services/EntityDiscoveryServiceTest.java d4900838 webapp/pom.xml 4132912d webapp/src/main/java/org/apache/atlas/web/resources/AdminResource.java 01f9a860 Diff: https://reviews.apache.org/r/56959/diff/6/ Changes: https://reviews.apache.org/r/56959/diff/5-6/ Testing ------- **Unit Test** * Added tests for *AtlasImportTransformBuilder* and contained classes. * Updated *ZipSourceTest* * Added tests to *ImportServiceTest* to exercise new option. **Functional Test** * Small size data export & import. * Medium size data export & import. **Performance Tests** Not exhaustive. Simple stop watch tests to compare against baseline. Thanks, Ashutosh Mestry