Review Request 68691: Export Service: Hdfs path entities are created before starting export
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68691/
---
Review request for atlas, Apoorv Naik, Madhan Neethiraj, Nixon Rodrigues, and
Sarath Subramanian.
Bugs: ATLAS-2869
https://issues.apache.org/jira/browse/ATLAS-2869
Repository: atlas
Description
---
**Approach**
- _HdfsPathCreator_ class creates hdfs_path entity based on parameters.
- _ExportService_ uses the new class.
**CURL**
curl -X POST -u admin:admin -H "Content-Type: application/json" -H
"Cache-Control: no-cache" -d @../docs/export-options.json
http://localhost:21000/api/atlas/admin/export > ../docs/hr.zip
_export-options.json_
```
{
"itemsToExport": [
{ "typeName": "hdfs_path", "uniqueAttributes": { "path":
"/warehouse/hr" }}
],
"options": {
"fetchType": "full",
"matchType": "startsWith",
"skipLineage": "true",
"replicatedTo": "clx",
"changeMarker": 0
}
}
```
Diffs
-
repository/src/main/java/org/apache/atlas/repository/impexp/ExportService.java
aded67caf
repository/src/main/java/org/apache/atlas/repository/impexp/HdfsPathEntityCreator.java
PRE-CREATION
repository/src/test/java/org/apache/atlas/repository/impexp/HdfsPathEntityCreatorTest.java
PRE-CREATION
Diff: https://reviews.apache.org/r/68691/diff/1/
Testing
---
**Unit tests**
New tests added.
**Functional tests**
Verified via UI.
Thanks,
Ashutosh Mestry
Re: Review Request 68691: Export Service: Hdfs path entities are created before starting export
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68691/#review208532
---
Looks good to me. Can you run it through the PreCommit pipeline ?
- Apoorv Naik
On Sept. 11, 2018, 10:32 p.m., Ashutosh Mestry wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/68691/
> ---
>
> (Updated Sept. 11, 2018, 10:32 p.m.)
>
>
> Review request for atlas, Apoorv Naik, Madhan Neethiraj, Nixon Rodrigues, and
> Sarath Subramanian.
>
>
> Bugs: ATLAS-2869
> https://issues.apache.org/jira/browse/ATLAS-2869
>
>
> Repository: atlas
>
>
> Description
> ---
>
> **Approach**
> - _HdfsPathCreator_ class creates hdfs_path entity based on parameters.
> - _ExportService_ uses the new class.
>
>
> **CURL**
>
> curl -X POST -u admin:admin -H "Content-Type: application/json" -H
> "Cache-Control: no-cache" -d @../docs/export-options.json
> http://localhost:21000/api/atlas/admin/export > ../docs/hr.zip
>
> _export-options.json_
> ```
> {
> "itemsToExport": [
> { "typeName": "hdfs_path", "uniqueAttributes": { "path":
> "/warehouse/hr" }}
> ],
> "options": {
> "fetchType": "full",
> "matchType": "startsWith",
> "skipLineage": "true",
> "replicatedTo": "clx",
> "changeMarker": 0
> }
> }
> ```
>
>
> Diffs
> -
>
>
> repository/src/main/java/org/apache/atlas/repository/impexp/ExportService.java
> aded67caf
>
> repository/src/main/java/org/apache/atlas/repository/impexp/HdfsPathEntityCreator.java
> PRE-CREATION
>
> repository/src/test/java/org/apache/atlas/repository/impexp/HdfsPathEntityCreatorTest.java
> PRE-CREATION
>
>
> Diff: https://reviews.apache.org/r/68691/diff/1/
>
>
> Testing
> ---
>
> **Unit tests**
>
> New tests added.
>
> **Functional tests**
>
> Verified via UI.
>
>
> Thanks,
>
> Ashutosh Mestry
>
>
Re: Review Request 68691: Export Service: Hdfs path entities are created before starting export
Ashutosh – review board has trouble in saving the comments. So here it is:
hdfs_path entity-type includes 'clusterName' attribute. Please update to
populate this attribute as well.
Thanks,
Madhan
From: Ashutosh Mestry on behalf of Ashutosh Mestry
Reply-To: Ashutosh Mestry
Date: Tuesday, September 11, 2018 at 3:32 PM
To: Apoorv Naik , Nixon Rodrigues
, Madhan Neethiraj ,
Sarath Subramanian
Cc: Ashutosh Mestry , atlas
Subject: Review Request 68691: Export Service: Hdfs path entities are created
before starting export
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68691/
Review request for atlas, Apoorv Naik, Madhan Neethiraj, Nixon Rodrigues, and
Sarath Subramanian.
By Ashutosh Mestry.
Bugs: ATLAS-2869<https://issues.apache.org/jira/browse/ATLAS-2869>
Repository: atlas
Description
Approach
- HdfsPathCreator class creates hdfs_path entity based on parameters.
- ExportService uses the new class.
CURL
curl -X POST -u admin:admin -H "Content-Type: application/json" -H
"Cache-Control: no-cache" -d @../docs/export-options.json
http://localhost:21000/api/atlas/admin/export > ../docs/hr.zip
export-options.json
{
"itemsToExport": [
{ "typeName": "hdfs_path", "uniqueAttributes": { "path":
"/warehouse/hr" }}
],
"options": {
"fetchType": "full",
"matchType": "startsWith",
"skipLineage": "true",
"replicatedTo": "clx",
"changeMarker": 0
}
}
Testing
Unit tests
New tests added.
Functional tests
Verified via UI.
Diffs
*
repository/src/main/java/org/apache/atlas/repository/impexp/ExportService.java
(aded67caf)
*
repository/src/main/java/org/apache/atlas/repository/impexp/HdfsPathEntityCreator.java
(PRE-CREATION)
*
repository/src/test/java/org/apache/atlas/repository/impexp/HdfsPathEntityCreatorTest.java
(PRE-CREATION)
View Diff<https://reviews.apache.org/r/68691/diff/1/>
Re: Review Request 68691: Export Service: Hdfs path entities are created before starting export
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68691/
---
(Updated Sept. 12, 2018, 4:32 a.m.)
Review request for atlas, Apoorv Naik, Madhan Neethiraj, Nixon Rodrigues, and
Sarath Subramanian.
Changes
---
Updates include: Addressed review comments.
Bugs: ATLAS-2869
https://issues.apache.org/jira/browse/ATLAS-2869
Repository: atlas
Description
---
**Approach**
- _HdfsPathCreator_ class creates hdfs_path entity based on parameters.
- _ExportService_ uses the new class.
**CURL**
curl -X POST -u admin:admin -H "Content-Type: application/json" -H
"Cache-Control: no-cache" -d @../docs/export-options.json
http://localhost:21000/api/atlas/admin/export > ../docs/hr.zip
_export-options.json_
```
{
"itemsToExport": [
{ "typeName": "hdfs_path", "uniqueAttributes": { "path":
"/warehouse/hr" }}
],
"options": {
"fetchType": "full",
"matchType": "startsWith",
"skipLineage": "true",
"replicatedTo": "clx",
"changeMarker": 0
}
}
```
Diffs (updated)
-
repository/src/main/java/org/apache/atlas/repository/impexp/ExportService.java
aded67caf
repository/src/main/java/org/apache/atlas/repository/impexp/HdfsPathEntityCreator.java
PRE-CREATION
repository/src/test/java/org/apache/atlas/repository/impexp/HdfsPathEntityCreatorTest.java
PRE-CREATION
Diff: https://reviews.apache.org/r/68691/diff/2/
Changes: https://reviews.apache.org/r/68691/diff/1-2/
Testing (updated)
---
**Unit tests**
New tests added.
**Functional tests**
Verified via UI.
**Pre-commit**
https://builds.apache.org/view/A/view/Atlas/job/PreCommit-ATLAS-Build-Test/659/
Thanks,
Ashutosh Mestry
Re: Review Request 68691: Export Service: Hdfs path entities are created before starting export
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68691/#review208541
---
Ship it!
- Madhan Neethiraj
On Sept. 12, 2018, 4:32 a.m., Ashutosh Mestry wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/68691/
> ---
>
> (Updated Sept. 12, 2018, 4:32 a.m.)
>
>
> Review request for atlas, Apoorv Naik, Madhan Neethiraj, Nixon Rodrigues, and
> Sarath Subramanian.
>
>
> Bugs: ATLAS-2869
> https://issues.apache.org/jira/browse/ATLAS-2869
>
>
> Repository: atlas
>
>
> Description
> ---
>
> **Approach**
> - _HdfsPathCreator_ class creates hdfs_path entity based on parameters.
> - _ExportService_ uses the new class.
>
>
> **CURL**
>
> curl -X POST -u admin:admin -H "Content-Type: application/json" -H
> "Cache-Control: no-cache" -d @../docs/export-options.json
> http://localhost:21000/api/atlas/admin/export > ../docs/hr.zip
>
> _export-options.json_
> ```
> {
> "itemsToExport": [
> { "typeName": "hdfs_path", "uniqueAttributes": { "path":
> "/warehouse/hr" }}
> ],
> "options": {
> "fetchType": "full",
> "matchType": "startsWith",
> "skipLineage": "true",
> "replicatedTo": "clx",
> "changeMarker": 0
> }
> }
> ```
>
>
> Diffs
> -
>
>
> repository/src/main/java/org/apache/atlas/repository/impexp/ExportService.java
> aded67caf
>
> repository/src/main/java/org/apache/atlas/repository/impexp/HdfsPathEntityCreator.java
> PRE-CREATION
>
> repository/src/test/java/org/apache/atlas/repository/impexp/HdfsPathEntityCreatorTest.java
> PRE-CREATION
>
>
> Diff: https://reviews.apache.org/r/68691/diff/2/
>
>
> Testing
> ---
>
> **Unit tests**
>
> New tests added.
>
> **Functional tests**
>
> Verified via UI.
>
> **Pre-commit**
> https://builds.apache.org/view/A/view/Atlas/job/PreCommit-ATLAS-Build-Test/659/
>
>
> Thanks,
>
> Ashutosh Mestry
>
>
