----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/56959/#review176074 -----------------------------------------------------------
webapp/src/main/java/org/apache/atlas/web/resources/AdminResource.java Line 366 (original), 367 (patched) <https://reviews.apache.org/r/56959/#comment249405> Can you check if the data can directly be serialized the Java POJO here ? I did see a few similar usecases on StackOverflow and they were de-serializing into the POJO directly at the interface level. - Apoorv Naik On May 25, 2017, 5:16 a.m., Ashutosh Mestry wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/56959/ > ----------------------------------------------------------- > > (Updated May 25, 2017, 5:16 a.m.) > > > Review request for atlas and Madhan Neethiraj. > > > Bugs: ATLAS-1825 > https://issues.apache.org/jira/browse/ATLAS-1825 > > > Repository: atlas > > > Description > ------- > > Please refer to > [ATLAS-1825](https://issues.apache.org/jira/browse/ATLAS-1825) for scenarios > and solution approach. > > Impact > ====== > * New class _AtlasImportTransform_ is now part of _AtlasImportRequest_. This > holds a map of entity-type to attribute and the tranforms to be applied. > * New class _AtlasImportTransformBuiler_ reads the requests specified in the > query string of the import API and creates a _AtlasImportTransfor_ structure. > * _AtlasImportTransformBuiler_ has new classes that support transformations. > They are: > * _replace_ Implements find and replace on attribute values. > * _lowercase_ Converts attribute value to lower case. > * Additional classes that unit test these classes have been added. > > Implementation Approach > ======================= > * _ZipSource_ now accepts _AtlasImportTransform_ object. It applies the > transform every entity that is fetched from the source. > * _Import API_ now accepts multi-part input. This allows for elegantly > specifying import parameters as well as the binary input from the file. > * Please take note of _ImportTransformerEnum_. This is first case where Enum > has state variables. > > CURL > ==== > > *Setup* > Create file with these contents call it _importTransform.json_: > ```javascript > { > "options": {}, > "importTransform": { > "transforms": { > "hive_table": { > "qualifiedName": [ > "lowercase", > "replace:@cl1:@cl2" > ] > } > } > } > } > ``` > > ``` > curl -g -X POST -u admin:admin -H "Content-Type: multipart/form-data" -H > "Cache-Control: no-cache" -F request=@../docs/importTransform.json -F > data=@../docs/Stocks-2.zip "http://localhost:21000/api/atlas/admin/import"; > ``` > > Note: _-g_ needs to be added to CURL calls to prevent gobble check. This is > not required anymore. > > Notice the output from the API, it has the _AtlasRequest_ that was passed as > input. > > Documentation > ============= > Need to update TWIKI documents. > > > Diffs > ----- > > intg/src/main/java/org/apache/atlas/model/impexp/AtlasImportRequest.java > 7530196d > intg/src/main/java/org/apache/atlas/model/impexp/AtlasImportTransforms.java > PRE-CREATION > intg/src/main/java/org/apache/atlas/model/impexp/ImportTransformerEnum.java > PRE-CREATION > > intg/src/test/java/org/apache/atlas/model/impexp/ImportTransformerEnumTest.java > PRE-CREATION > pom.xml a93b8ad9 > > repository/src/main/java/org/apache/atlas/repository/impexp/ImportEntityTransformer.java > PRE-CREATION > > repository/src/main/java/org/apache/atlas/repository/impexp/ImportService.java > 9ec15e00 > repository/src/main/java/org/apache/atlas/repository/impexp/ZipSource.java > 87c9f0eb > > repository/src/test/java/org/apache/atlas/repository/impexp/ImportEntityTransformerTest.java > PRE-CREATION > > repository/src/test/java/org/apache/atlas/repository/impexp/ImportServiceTest.java > 08e9ee81 > > repository/src/test/java/org/apache/atlas/repository/impexp/TransformEvaluatorTest.java > PRE-CREATION > > repository/src/test/java/org/apache/atlas/services/EntityDiscoveryServiceTest.java > d4900838 > webapp/pom.xml 4132912d > webapp/src/main/java/org/apache/atlas/web/resources/AdminResource.java > 01f9a860 > > > Diff: https://reviews.apache.org/r/56959/diff/6/ > > > Testing > ------- > > **Unit Test** > * Added tests for *AtlasImportTransformBuilder* and contained classes. > * Updated *ZipSourceTest* > * Added tests to *ImportServiceTest* to exercise new option. > > **Functional Test** > * Small size data export & import. > * Medium size data export & import. > > **Performance Tests** > Not exhaustive. Simple stop watch tests to compare against baseline. > > > Thanks, > > Ashutosh Mestry > >
