-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71426/
-----------------------------------------------------------
(Updated Sept. 4, 2019, 7:58 p.m.)
Review request for atlas, Madhan Neethiraj, Nikhil Bonte, Nixon Rodrigues, and
Sarath Subramanian.
Changes
-------
Upates include:
- Addressed review comments.
- _backingDirectory_ is now specified at application level.
Bugs: ATLAS-3396
https://issues.apache.org/jira/browse/ATLAS-3396
Repository: atlas
Description
-------
**Background**
The approach adds another option to _ImportService_ to be able to optionally
use a directory on the server for storing data to be imported.
**Approach**
This is a backward compatible.
- - New: _ZipSourceWithBackingDirectory_: Uses a temporary directory for
storing contents of the zip file.
- Modified: _ImportService_ Now takes _InputStream_ as parameter instead of
_ZipSource_.
- Modified: _ImportRequest_ Additionl option takes _backingDirectory_ as
additional option. Specifying this will use the new
_ZipSourceWithBackingDirectory_ during import.
**CURL**
Use pre-configured temporary directory:
```
{
"options": {
"backingDirectory": "/grid/0/temp"
}
}
```
To use default temp directory:
```
{
"options": {
"backingDirectory": ""
}
}
``
_CURL_
```
curl -v -X POST -u admin:admin -H "Content-Type: multipart/form-data" -H
"Cache-Control: no-cache" -F request=@../docs/import-options.json -F
data=@../docs/smalldb.zip http://localhost:21000/api/atlas/admin/import
```
**Documentation**
Updated.
Diffs (updated)
-----
docs/src/site/twiki/Import-API-Options.twiki 4004e7013
intg/src/main/java/org/apache/atlas/model/impexp/AtlasImportRequest.java
6a9985642
repository/src/main/java/org/apache/atlas/repository/impexp/ImportService.java
b5d8b7c39
repository/src/main/java/org/apache/atlas/repository/impexp/ZipSourceWithBackingDirectory.java
PRE-CREATION
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasEntityStreamForImport.java
90ae15d1e
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/EntityImportStream.java
d4b6c5505
repository/src/test/java/org/apache/atlas/repository/impexp/ExportIncrementalTest.java
a355297a8
repository/src/test/java/org/apache/atlas/repository/impexp/ExportSkipLineageTest.java
eaf4602d5
repository/src/test/java/org/apache/atlas/repository/impexp/ImportServiceTest.java
693a163f8
repository/src/test/java/org/apache/atlas/repository/impexp/ReplicationEntityAttributeTest.java
868b732d3
repository/src/test/java/org/apache/atlas/repository/impexp/ZipFileResourceTestUtils.java
a2a5f58dc
webapp/src/main/java/org/apache/atlas/web/resources/AdminResource.java
8417e7eb0
Diff: https://reviews.apache.org/r/71426/diff/3/
Changes: https://reviews.apache.org/r/71426/diff/2-3/
Testing
-------
**Unit tests**
Updated to exercise the new implementation.
**Volume tests**
Imported 4GB ZIP file that took 26 hours. Memory and CPU stayed in normal range
during this operation.
**Functional tests**
Accuracy tests preformed on small size data.
**Pre-commit Build**
https://builds.apache.org/view/A/view/Atlas/job/PreCommit-ATLAS-Build-Test/1379/console
Thanks,
Ashutosh Mestry
