[jira] [Commented] (ATLAS-4318) Fix python client issues and code refactoring

[ASF subversion and git services (Jira)]

[ 
https://issues.apache.org/jira/browse/ATLAS-4318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17357084#comment-17357084
 ] 

ASF subversion and git services commented on ATLAS-4318:


Commit 6336faf3368b553c257c76e89552939b4fa5cb1a in atlas's branch 
refs/heads/branch-2.0 from Benedict Jin
[ https://gitbox.apache.org/repos/asf?p=atlas.git;h=6336faf ]

ATLAS-4318 : Remove unnecessary imports from python script (#134)

(cherry picked from commit f96a6530c09a83e4f0d6613e558c4e2e5d5f236a)


> Fix python client issues and code refactoring
> -
>
> Key: ATLAS-4318
> URL: https://issues.apache.org/jira/browse/ATLAS-4318
> Project: Atlas
>  Issue Type: Bug
>Reporter: Nixon Rodrigues
>Priority: Major
>  Time Spent: 50m
>  Remaining Estimate: 0h
>
> Merge PR reported on github.
> [https://github.com/apache/atlas/pull/130]
> https://github.com/apache/atlas/pull/135



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (ATLAS-4318) Fix python client issues and code refactoring

[ASF subversion and git services (Jira)]

[ 
https://issues.apache.org/jira/browse/ATLAS-4318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17357078#comment-17357078
 ] 

ASF subversion and git services commented on ATLAS-4318:


Commit f96a6530c09a83e4f0d6613e558c4e2e5d5f236a in atlas's branch 
refs/heads/master from Benedict Jin
[ https://gitbox.apache.org/repos/asf?p=atlas.git;h=f96a653 ]

ATLAS-4318 : Remove unnecessary imports from python script (#134)



> Fix python client issues and code refactoring
> -
>
> Key: ATLAS-4318
> URL: https://issues.apache.org/jira/browse/ATLAS-4318
> Project: Atlas
>  Issue Type: Bug
>Reporter: Nixon Rodrigues
>Priority: Major
>  Time Spent: 50m
>  Remaining Estimate: 0h
>
> Merge PR reported on github.
> [https://github.com/apache/atlas/pull/130]
> https://github.com/apache/atlas/pull/135



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[GitHub] [atlas] nixonrodrigues merged pull request #134: ATLAS-4318 : Remove unnecessary imports from python script

[GitBox]

nixonrodrigues merged pull request #134:
URL: https://github.com/apache/atlas/pull/134


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [atlas] nixonrodrigues commented on pull request #134: ATLAS-4318 : Remove unnecessary imports from python script

[GitBox]

nixonrodrigues commented on pull request #134:
URL: https://github.com/apache/atlas/pull/134#issuecomment-854374498


   +1 for the PR, thanks @asdf2014 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

Re: Review Request 73405: ATLAS-4302: Migrated Data: Process Entity Name not set to QualifiedName

[Nikhil Bonte]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/73405/#review223104
---


Ship it!




Ship It!

- Nikhil Bonte


On June 4, 2021, 10:18 a.m., Ashutosh Mestry wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/73405/
> ---
> 
> (Updated June 4, 2021, 10:18 a.m.)
> 
> 
> Review request for atlas, Nikhil Bonte, Nixon Rodrigues, and Sarath 
> Subramanian.
> 
> 
> Bugs: ATLAS-4302
> https://issues.apache.org/jira/browse/ATLAS-4302
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> **Approach**
> Java patch is optionally run when 
> ```atlas.process.name.update.patch
> 
> is set to true.
> 
> 
> Diffs
> -
> 
>   intg/src/main/java/org/apache/atlas/AtlasConfiguration.java e31afa89a 
>   
> repository/src/main/java/org/apache/atlas/repository/patches/AtlasPatchManager.java
>  fae28c440 
>   
> repository/src/main/java/org/apache/atlas/repository/patches/ProcessNamePatch.java
>  PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/73405/diff/2/
> 
> 
> Testing
> ---
> 
> **Functional test**
> - Validated with medium data set (5M entities).
> 
> **Volume test**
> - Validated improvement for a lineage entity with 1200 inputs and 1700 
> outputs. With this change, the _AtlasEntityHeader_ payload size reduces by 
> 90%.
> 
> 
> Thanks,
> 
> Ashutosh Mestry
> 
>

Re: Review Request 73404: ATLAS-4324: FS entity created for load data inpath is created as shell entity - Part 2

[Sarath Subramanian]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/73404/#review223103
---




addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/utils/ActiveEntityFilter.java
Lines 62 (patched)


If config property is set, user should be mindful of including all the 6 
path types as well, else shell entities will be created. The types specified 
from config should only append to the default types. Without defaultTypes all 
path lineage will be broken.



addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/utils/HiveDDLEntityFilter.java
Lines 182 (patched)


we may not need separate instanceof check for AtlasRelatedObjectId. Lines 
188-192 should suffice, since guid belongs to parent - AtlasObjectId. Consider 
removing 182-187



addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/utils/HiveDDLEntityFilter.java
Line 201 (original), 191 (patched)


This will also set all path entity types guid to null. If guid is null 
(non-negative guid is not specified). IdBasedResolver will be ignored and 
UniqueAttributeBasedResolver will not be able to fetch the entity and result in 
shell entity creation. The earlier check to ignore path entity types would 
avoid this. Please review.


- Sarath Subramanian


On June 3, 2021, 2:52 p.m., Ashutosh Mestry wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/73404/
> ---
> 
> (Updated June 3, 2021, 2:52 p.m.)
> 
> 
> Review request for atlas, Mandar Ambawane, Radhika Kundam, and Sarath 
> Subramanian.
> 
> 
> Bugs: ATLAS-4324
> https://issues.apache.org/jira/browse/ATLAS-4324
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> **Approach**
> Updates:
> - Simplified approach.
> - Maintained abstraction.
> - Updated unit tests.
> 
> 
> Diffs
> -
> 
>   
> addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/utils/ActiveEntityFilter.java
>  737c637bf 
>   
> addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/utils/HiveDDLEntityFilter.java
>  9163c4709 
>   
> addons/hive-bridge/src/test/java/org/apache/atlas/hive/hook/utils/ActiveEntityFilterTest.java
>  4dde1dc2f 
>   
> addons/hive-bridge/src/test/resources/json/hs2-create-db-with-no-pathentities-to-retain-v2.json
>  42553b5cf 
>   
> addons/hive-bridge/src/test/resources/json/hs2-create-db-with-no-pathentities-to-retain.json
>  a5b810f72 
>   addons/hive-bridge/src/test/resources/json/hs2-load-inpath-v2.json 
> dd31aa073 
>   
> addons/hive-bridge/src/test/resources/json/hs2-load-inpath-with-no-pathentities-to-retain-v2.json
>  7f90d1986 
>   
> addons/hive-bridge/src/test/resources/json/hs2-load-inpath-with-no-pathentities-to-retain.json
>  dd31aa073 
>   addons/hive-bridge/src/test/resources/json/hs2-load-inpath.json dd31aa073 
> 
> 
> Diff: https://reviews.apache.org/r/73404/diff/1/
> 
> 
> Testing
> ---
> 
> **Unit tests**
> - Additional unit test.
> 
> 
> Thanks,
> 
> Ashutosh Mestry
> 
>

Re: Review Request 73317: ATLAS-3812: Add schema for Apache Flink

[Nixon Rodrigues]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/73317/#review223102
---


Ship it!




Ship It!

- Nixon Rodrigues


On June 1, 2021, 6:59 p.m., Josh Yeh wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/73317/
> ---
> 
> (Updated June 1, 2021, 6:59 p.m.)
> 
> 
> Review request for atlas, Ashutosh Mestry, Pinal Shah, and Sarath Subramanian.
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> ATLAS-3812: Add schema for Apache Flink
> 
> 
> Diffs
> -
> 
>   addons/models/1000-Hadoop/1110-flink_model.json PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/73317/diff/4/
> 
> 
> Testing
> ---
> 
> Tested on my CDH cluster with flink example wordcount streaming job.
> 
> 
> File Attachments
> 
> 
> Screen Shot 2021-04-30 at 12.38.27 AM.png
>   
> https://reviews.apache.org/media/uploaded/files/2021/04/30/2ff59445-7237-4be1-a1a0-b6db70bc7827__Screen_Shot_2021-04-30_at_12.38.27_AM.png
> 
> 
> Thanks,
> 
> Josh Yeh
> 
>

Re: Review Request 73405: ATLAS-4302: Migrated Data: Process Entity Name not set to QualifiedName

[Ashutosh Mestry via Review Board]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/73405/
---

(Updated June 4, 2021, 4:48 a.m.)


Review request for atlas, Nikhil Bonte, Nixon Rodrigues, and Sarath Subramanian.


Changes
---

Updates include: Addressed review comments.


Bugs: ATLAS-4302
https://issues.apache.org/jira/browse/ATLAS-4302


Repository: atlas


Description
---

**Approach**
Java patch is optionally run when 
```atlas.process.name.update.patch

is set to true.


Diffs (updated)
-

  intg/src/main/java/org/apache/atlas/AtlasConfiguration.java e31afa89a 
  
repository/src/main/java/org/apache/atlas/repository/patches/AtlasPatchManager.java
 fae28c440 
  
repository/src/main/java/org/apache/atlas/repository/patches/ProcessNamePatch.java
 PRE-CREATION 


Diff: https://reviews.apache.org/r/73405/diff/2/

Changes: https://reviews.apache.org/r/73405/diff/1-2/


Testing
---

**Functional test**
- Validated with medium data set (5M entities).

**Volume test**
- Validated improvement for a lineage entity with 1200 inputs and 1700 outputs. 
With this change, the _AtlasEntityHeader_ payload size reduces by 90%.


Thanks,

Ashutosh Mestry

Re: Review Request 73405: ATLAS-4302: Migrated Data: Process Entity Name not set to QualifiedName

[Nixon Rodrigues]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/73405/#review223101
---


Ship it!




Ship It!

- Nixon Rodrigues


On June 3, 2021, 9:43 p.m., Ashutosh Mestry wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/73405/
> ---
> 
> (Updated June 3, 2021, 9:43 p.m.)
> 
> 
> Review request for atlas, Nikhil Bonte, Nixon Rodrigues, and Sarath 
> Subramanian.
> 
> 
> Bugs: ATLAS-4302
> https://issues.apache.org/jira/browse/ATLAS-4302
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> **Approach**
> Java patch is optionally run when 
> ```atlas.process.name.update.patch
> 
> is set to true.
> 
> 
> Diffs
> -
> 
>   intg/src/main/java/org/apache/atlas/AtlasConfiguration.java e31afa89a 
>   
> repository/src/main/java/org/apache/atlas/repository/patches/AtlasPatchManager.java
>  fae28c440 
>   
> repository/src/main/java/org/apache/atlas/repository/patches/ProcessNamePatch.java
>  PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/73405/diff/1/
> 
> 
> Testing
> ---
> 
> **Functional test**
> - Validated with medium data set (5M entities).
> 
> **Volume test**
> - Validated improvement for a lineage entity with 1200 inputs and 1700 
> outputs. With this change, the _AtlasEntityHeader_ payload size reduces by 
> 90%.
> 
> 
> Thanks,
> 
> Ashutosh Mestry
> 
>

Re: Review Request 73405: ATLAS-4302: Migrated Data: Process Entity Name not set to QualifiedName

[Nixon Rodrigues]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/73405/#review223100
---




repository/src/main/java/org/apache/atlas/repository/patches/ProcessNamePatch.java
Lines 120 (patched)


Add if debug is enabled case. 

if (LOG.isDebugEnabled()) {

same for line 129


- Nixon Rodrigues


On June 3, 2021, 9:43 p.m., Ashutosh Mestry wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/73405/
> ---
> 
> (Updated June 3, 2021, 9:43 p.m.)
> 
> 
> Review request for atlas, Nikhil Bonte, Nixon Rodrigues, and Sarath 
> Subramanian.
> 
> 
> Bugs: ATLAS-4302
> https://issues.apache.org/jira/browse/ATLAS-4302
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> **Approach**
> Java patch is optionally run when 
> ```atlas.process.name.update.patch
> 
> is set to true.
> 
> 
> Diffs
> -
> 
>   intg/src/main/java/org/apache/atlas/AtlasConfiguration.java e31afa89a 
>   
> repository/src/main/java/org/apache/atlas/repository/patches/AtlasPatchManager.java
>  fae28c440 
>   
> repository/src/main/java/org/apache/atlas/repository/patches/ProcessNamePatch.java
>  PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/73405/diff/1/
> 
> 
> Testing
> ---
> 
> **Functional test**
> - Validated with medium data set (5M entities).
> 
> **Volume test**
> - Validated improvement for a lineage entity with 1200 inputs and 1700 
> outputs. With this change, the _AtlasEntityHeader_ payload size reduces by 
> 90%.
> 
> 
> Thanks,
> 
> Ashutosh Mestry
> 
>

[jira] [Updated] (ATLAS-4302) Migrated Data: Process Entity Name not set to QualifiedName

[Ashutosh Mestry (Jira)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-4302?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ashutosh Mestry updated ATLAS-4302:
---
Attachment: ATLAS-4302-Migrated-Data-Process-Entity-Name-not-set.patch

> Migrated Data: Process Entity Name not set to QualifiedName
> ---
>
> Key: ATLAS-4302
> URL: https://issues.apache.org/jira/browse/ATLAS-4302
> Project: Atlas
>  Issue Type: Bug
>Reporter: Ashutosh Mestry
>Assignee: Ashutosh Mestry
>Priority: Major
> Attachments: 
> ATLAS-4302-Migrated-Data-Process-Entity-Name-not-set.patch
>
>
> *Background*
> In v0.8 of Atlas process names (_hive_process.name_) was using _queryText_ as 
> the name. This in v1.0 onwards where _name_ and _qualifiedName_ are same.
> *Solution*
> Add Java patch that updates the name property.
> *Impact of Not Doing this Update*
> The _queryText_ in _hive_process_ and _hive_columnlineage_ entities tends to 
> be large. The name field is part of _AtlasEntityHeader_. When fetching search 
> results, lineage display are some of the flows that have these entities.
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (ATLAS-4302) Migrated Data: Process Entity Name not set to QualifiedName

[Ashutosh Mestry (Jira)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-4302?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ashutosh Mestry updated ATLAS-4302:
---
Attachment: (was: 
ATLAS-4302-Update-hive-process-entities-name-to-qual.patch)

> Migrated Data: Process Entity Name not set to QualifiedName
> ---
>
> Key: ATLAS-4302
> URL: https://issues.apache.org/jira/browse/ATLAS-4302
> Project: Atlas
>  Issue Type: Bug
>Reporter: Ashutosh Mestry
>Assignee: Ashutosh Mestry
>Priority: Major
>
> *Background*
> In v0.8 of Atlas process names (_hive_process.name_) was using _queryText_ as 
> the name. This in v1.0 onwards where _name_ and _qualifiedName_ are same.
> *Solution*
> Add Java patch that updates the name property.
> *Impact of Not Doing this Update*
> The _queryText_ in _hive_process_ and _hive_columnlineage_ entities tends to 
> be large. The name field is part of _AtlasEntityHeader_. When fetching search 
> results, lineage display are some of the flows that have these entities.
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Review Request 73405: ATLAS-4302: Migrated Data: Process Entity Name not set to QualifiedName

[Ashutosh Mestry via Review Board]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/73405/
---

Review request for atlas, Nikhil Bonte, Nixon Rodrigues, and Sarath Subramanian.


Bugs: ATLAS-4302
https://issues.apache.org/jira/browse/ATLAS-4302


Repository: atlas


Description
---

**Approach**
Java patch is optionally run when 
```atlas.process.name.update.patch

is set to true.


Diffs
-

  intg/src/main/java/org/apache/atlas/AtlasConfiguration.java e31afa89a 
  
repository/src/main/java/org/apache/atlas/repository/patches/AtlasPatchManager.java
 fae28c440 
  
repository/src/main/java/org/apache/atlas/repository/patches/ProcessNamePatch.java
 PRE-CREATION 


Diff: https://reviews.apache.org/r/73405/diff/1/


Testing
---

**Functional test**
- Validated with medium data set (5M entities).

**Volume test**
- Validated improvement for a lineage entity with 1200 inputs and 1700 outputs. 
With this change, the _AtlasEntityHeader_ payload size reduces by 90%.


Thanks,

Ashutosh Mestry

Review Request 73404: ATLAS-4324: FS entity created for load data inpath is created as shell entity - Part 2

[Ashutosh Mestry via Review Board]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/73404/
---

Review request for atlas and Radhika Kundam.


Bugs: ATLAS-4324
https://issues.apache.org/jira/browse/ATLAS-4324


Repository: atlas


Description
---

**Approach**
Updates:
- Simplified approach.
- Maintained abstraction.
- Updated unit tests.


Diffs
-

  
addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/utils/ActiveEntityFilter.java
 737c637bf 
  
addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/utils/HiveDDLEntityFilter.java
 9163c4709 
  
addons/hive-bridge/src/test/java/org/apache/atlas/hive/hook/utils/ActiveEntityFilterTest.java
 4dde1dc2f 
  
addons/hive-bridge/src/test/resources/json/hs2-create-db-with-no-pathentities-to-retain-v2.json
 42553b5cf 
  
addons/hive-bridge/src/test/resources/json/hs2-create-db-with-no-pathentities-to-retain.json
 a5b810f72 
  addons/hive-bridge/src/test/resources/json/hs2-load-inpath-v2.json dd31aa073 
  
addons/hive-bridge/src/test/resources/json/hs2-load-inpath-with-no-pathentities-to-retain-v2.json
 7f90d1986 
  
addons/hive-bridge/src/test/resources/json/hs2-load-inpath-with-no-pathentities-to-retain.json
 dd31aa073 
  addons/hive-bridge/src/test/resources/json/hs2-load-inpath.json dd31aa073 


Diff: https://reviews.apache.org/r/73404/diff/1/


Testing
---

**Unit tests**
- Additional unit test.


Thanks,

Ashutosh Mestry

[jira] [Created] (ATLAS-4325) [Atlas: Glossary Term Bulk Import] [UI] Unable to perform bulk import glossary term via UI

[Dharshana M Krishnamoorthy (Jira)]

Dharshana M Krishnamoorthy created ATLAS-4325:
-

 Summary: [Atlas: Glossary Term Bulk Import] [UI] Unable to perform 
bulk import glossary term via UI
 Key: ATLAS-4325
 URL: https://issues.apache.org/jira/browse/ATLAS-4325
 Project: Atlas
  Issue Type: Bug
  Components: atlas-webui
Reporter: Dharshana M Krishnamoorthy
 Attachments: Screenshot 2021-06-03 at 5.41.57 PM.png

Bulk import is failing with "\{"msgDesc":"Missing header or invalid Header 
value for CSRF Vulnerability Protection"}" !Screenshot 2021-06-03 at 5.41.57 
PM.png|width=443,height=273!

NOTE: the same succeeds via api



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (ATLAS-4322) Hive HOOK fails with NPE while processing column-rename

[ASF subversion and git services (Jira)]

[ 
https://issues.apache.org/jira/browse/ATLAS-4322?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17356610#comment-17356610
 ] 

ASF subversion and git services commented on ATLAS-4322:


Commit a2a1e3b5d23ff5db45fbf7f5a28976ea465d9c46 in atlas's branch 
refs/heads/branch-2.0 from JP Li
[ https://gitbox.apache.org/repos/asf?p=atlas.git;h=a2a1e3b ]

ATLAS-4322: Hive HOOK NPE

Signed-off-by: sidmishra 
(cherry picked from commit ff0c65a49fb4617b85a5b8a674be53305f5cf64b)


> Hive HOOK fails with NPE while processing column-rename
> ---
>
> Key: ATLAS-4322
> URL: https://issues.apache.org/jira/browse/ATLAS-4322
> Project: Atlas
>  Issue Type: Bug
>  Components:  atlas-core
>Reporter: Radhika Kundam
>Assignee: Radhika Kundam
>Priority: Trivial
>
> While processing column-rename on temporary table, Atlas hook for Hive fails 
> with Null Pointer Exception. 
> {code:java}
> 2021-05-25 15:06:38,131 ERROR org.apache.atlas.hive.hook.HiveHook: 
> [HiveServer2-Background-Pool: Thread-529]: HiveHook.run(): failed to process 
> operation ALTERTABLE_RENAMECOL java.lang.NullPointerException: null at 
> java.util.ArrayList.(ArrayList.java:178) ~[?:1.8.0_292] at 
> org.apache.atlas.hive.hook.events.AlterTableRenameCol.getHiveMessages(AlterTableRenameCol.java:76)
>  ~[hive-bridge-2.1.0.7.2.9.1-2.jar:2.1.0.7.2.9.1-2] at 
> org.apache.atlas.hive.hook.events.AlterTableRenameCol.getNotificationMessages(AlterTableRenameCol.java:55)
>  ~[hive-bridge-2.1.0.7.2.9.1-2.jar:2.1.0.7.2.9.1-2] at 
> org.apache.atlas.hive.hook.HiveHook.run(HiveHook.java:252) 
> [hive-bridge-2.1.0.7.2.9.1-2.jar:2.1.0.7.2.9.1-2] at 
> org.apache.atlas.hive.hook.HiveHook.run(HiveHook.java:52) 
> [hive-bridge-shim-2.1.0.7.2.9.1-2.jar:2.1.0.7.2.9.1-2]{code}
>  
> In ATLAS-2757 a similar issue was fixed (in 2018) by adding a check for 
> {{baseMsgs}}
>  but in ATLAS-3148 it was re-introduced (in 2019).



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (ATLAS-4322) Hive HOOK fails with NPE while processing column-rename

[ASF subversion and git services (Jira)]

[ 
https://issues.apache.org/jira/browse/ATLAS-4322?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17356609#comment-17356609
 ] 

ASF subversion and git services commented on ATLAS-4322:


Commit ff0c65a49fb4617b85a5b8a674be53305f5cf64b in atlas's branch 
refs/heads/master from JP Li
[ https://gitbox.apache.org/repos/asf?p=atlas.git;h=ff0c65a ]

ATLAS-4322: Hive HOOK NPE

Signed-off-by: sidmishra 


> Hive HOOK fails with NPE while processing column-rename
> ---
>
> Key: ATLAS-4322
> URL: https://issues.apache.org/jira/browse/ATLAS-4322
> Project: Atlas
>  Issue Type: Bug
>  Components:  atlas-core
>Reporter: Radhika Kundam
>Assignee: Radhika Kundam
>Priority: Trivial
>
> While processing column-rename on temporary table, Atlas hook for Hive fails 
> with Null Pointer Exception. 
> {code:java}
> 2021-05-25 15:06:38,131 ERROR org.apache.atlas.hive.hook.HiveHook: 
> [HiveServer2-Background-Pool: Thread-529]: HiveHook.run(): failed to process 
> operation ALTERTABLE_RENAMECOL java.lang.NullPointerException: null at 
> java.util.ArrayList.(ArrayList.java:178) ~[?:1.8.0_292] at 
> org.apache.atlas.hive.hook.events.AlterTableRenameCol.getHiveMessages(AlterTableRenameCol.java:76)
>  ~[hive-bridge-2.1.0.7.2.9.1-2.jar:2.1.0.7.2.9.1-2] at 
> org.apache.atlas.hive.hook.events.AlterTableRenameCol.getNotificationMessages(AlterTableRenameCol.java:55)
>  ~[hive-bridge-2.1.0.7.2.9.1-2.jar:2.1.0.7.2.9.1-2] at 
> org.apache.atlas.hive.hook.HiveHook.run(HiveHook.java:252) 
> [hive-bridge-2.1.0.7.2.9.1-2.jar:2.1.0.7.2.9.1-2] at 
> org.apache.atlas.hive.hook.HiveHook.run(HiveHook.java:52) 
> [hive-bridge-shim-2.1.0.7.2.9.1-2.jar:2.1.0.7.2.9.1-2]{code}
>  
> In ATLAS-2757 a similar issue was fixed (in 2018) by adding a check for 
> {{baseMsgs}}
>  but in ATLAS-3148 it was re-introduced (in 2019).



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (ATLAS-4324) FS entity created for load data inpath is created as shell entity

[Radhika Kundam (Jira)]

[ 
https://issues.apache.org/jira/browse/ATLAS-4324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17356588#comment-17356588
 ] 

Radhika Kundam commented on ATLAS-4324:
---

With this patch we are including Path entities to be retained at HS2 
ActiveEntityfilter. And also provided configuration option at Hive Hook to add 
any entity types (comma separated)to be retained.

config property is : 
*atlas.hook.hive.hs2.filter.entity.types.to.retain=,*

 

> FS entity created for load data inpath is created as shell entity
> -
>
> Key: ATLAS-4324
> URL: https://issues.apache.org/jira/browse/ATLAS-4324
> Project: Atlas
>  Issue Type: Bug
>  Components: hive-integration
>Affects Versions: 2.1.0
>Reporter: Radhika Kundam
>Assignee: Radhika Kundam
>Priority: Major
>  Labels: hive-hooks
> Fix For: 3.0.0, 2.2.0
>
> Attachments: load_data_shell_entity.png
>
>
> beeline >
> create external table default.hive_table_cloud_load_data_in_path_123 
> (student_roll int, student_name string, student_dob date) ROW FORMAT 
> DELIMITED FIELDS TERMINATED BY ' ' STORED AS TEXTFILE location 
> 'hdfs://ns1/tmp/hive_table_cloud_load_data_in_path_123'
>  
> load data inpath 'hdfs://ns1/tmp/data123.txt' into table 
> hive_table_cloud_load_data_in_path_123;
>  
> Creates 'hdfs://ns1/tmp/data123.txt' as shell entity. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (ATLAS-4324) FS entity created for load data inpath is created as shell entity

[Radhika Kundam (Jira)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-4324?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Radhika Kundam updated ATLAS-4324:
--
Attachment: (was: image-2021-06-03-10-01-37-153.png)

> FS entity created for load data inpath is created as shell entity
> -
>
> Key: ATLAS-4324
> URL: https://issues.apache.org/jira/browse/ATLAS-4324
> Project: Atlas
>  Issue Type: Bug
>  Components: hive-integration
>Affects Versions: 2.1.0
>Reporter: Radhika Kundam
>Assignee: Radhika Kundam
>Priority: Major
>  Labels: hive-hooks
> Fix For: 3.0.0, 2.2.0
>
> Attachments: load_data_shell_entity.png
>
>
> beeline >
> create external table default.hive_table_cloud_load_data_in_path_123 
> (student_roll int, student_name string, student_dob date) ROW FORMAT 
> DELIMITED FIELDS TERMINATED BY ' ' STORED AS TEXTFILE location 
> 'hdfs://ns1/tmp/hive_table_cloud_load_data_in_path_123'
>  
> load data inpath 'hdfs://ns1/tmp/data123.txt' into table 
> hive_table_cloud_load_data_in_path_123;
>  
> Creates 'hdfs://ns1/tmp/data123.txt' as shell entity. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (ATLAS-4324) FS entity created for load data inpath is created as shell entity

[Radhika Kundam (Jira)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-4324?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Radhika Kundam updated ATLAS-4324:
--
Attachment: image-2021-06-03-10-01-37-153.png

> FS entity created for load data inpath is created as shell entity
> -
>
> Key: ATLAS-4324
> URL: https://issues.apache.org/jira/browse/ATLAS-4324
> Project: Atlas
>  Issue Type: Bug
>  Components: hive-integration
>Affects Versions: 2.1.0
>Reporter: Radhika Kundam
>Assignee: Radhika Kundam
>Priority: Major
>  Labels: hive-hooks
> Fix For: 3.0.0, 2.2.0
>
> Attachments: image-2021-06-03-10-01-37-153.png, 
> load_data_shell_entity.png
>
>
> beeline >
> create external table default.hive_table_cloud_load_data_in_path_123 
> (student_roll int, student_name string, student_dob date) ROW FORMAT 
> DELIMITED FIELDS TERMINATED BY ' ' STORED AS TEXTFILE location 
> 'hdfs://ns1/tmp/hive_table_cloud_load_data_in_path_123'
>  
> load data inpath 'hdfs://ns1/tmp/data123.txt' into table 
> hive_table_cloud_load_data_in_path_123;
>  
> Creates 'hdfs://ns1/tmp/data123.txt' as shell entity. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (ATLAS-4324) FS entity created for load data inpath is created as shell entity

[Sarath Subramanian (Jira)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-4324?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sarath Subramanian updated ATLAS-4324:
--
Labels: hive-hooks  (was: )

> FS entity created for load data inpath is created as shell entity
> -
>
> Key: ATLAS-4324
> URL: https://issues.apache.org/jira/browse/ATLAS-4324
> Project: Atlas
>  Issue Type: Bug
>  Components: hive-integration
>Affects Versions: 2.1.0
>Reporter: Radhika Kundam
>Assignee: Radhika Kundam
>Priority: Major
>  Labels: hive-hooks
> Fix For: 3.0.0, 2.2.0
>
> Attachments: load_data_shell_entity.png
>
>
> beeline >
> create external table default.hive_table_cloud_load_data_in_path_123 
> (student_roll int, student_name string, student_dob date) ROW FORMAT 
> DELIMITED FIELDS TERMINATED BY ' ' STORED AS TEXTFILE location 
> 'hdfs://ns1/tmp/hive_table_cloud_load_data_in_path_123'
>  
> load data inpath 'hdfs://ns1/tmp/data123.txt' into table 
> hive_table_cloud_load_data_in_path_123;
>  
> Creates 'hdfs://ns1/tmp/data123.txt' as shell entity. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Resolved] (ATLAS-4324) FS entity created for load data inpath is created as shell entity

[Sarath Subramanian (Jira)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-4324?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sarath Subramanian resolved ATLAS-4324.
---
Resolution: Fixed

> FS entity created for load data inpath is created as shell entity
> -
>
> Key: ATLAS-4324
> URL: https://issues.apache.org/jira/browse/ATLAS-4324
> Project: Atlas
>  Issue Type: Bug
>  Components: hive-integration
>Affects Versions: 2.1.0
>Reporter: Radhika Kundam
>Assignee: Radhika Kundam
>Priority: Major
>  Labels: hive-hooks
> Fix For: 3.0.0, 2.2.0
>
> Attachments: load_data_shell_entity.png
>
>
> beeline >
> create external table default.hive_table_cloud_load_data_in_path_123 
> (student_roll int, student_name string, student_dob date) ROW FORMAT 
> DELIMITED FIELDS TERMINATED BY ' ' STORED AS TEXTFILE location 
> 'hdfs://ns1/tmp/hive_table_cloud_load_data_in_path_123'
>  
> load data inpath 'hdfs://ns1/tmp/data123.txt' into table 
> hive_table_cloud_load_data_in_path_123;
>  
> Creates 'hdfs://ns1/tmp/data123.txt' as shell entity. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (ATLAS-4324) FS entity created for load data inpath is created as shell entity

[Sarath Subramanian (Jira)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-4324?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sarath Subramanian updated ATLAS-4324:
--
Component/s: hive-integration

> FS entity created for load data inpath is created as shell entity
> -
>
> Key: ATLAS-4324
> URL: https://issues.apache.org/jira/browse/ATLAS-4324
> Project: Atlas
>  Issue Type: Bug
>  Components: hive-integration
>Affects Versions: 2.1.0
>Reporter: Radhika Kundam
>Assignee: Radhika Kundam
>Priority: Major
> Fix For: 3.0.0, 2.2.0
>
> Attachments: load_data_shell_entity.png
>
>
> beeline >
> create external table default.hive_table_cloud_load_data_in_path_123 
> (student_roll int, student_name string, student_dob date) ROW FORMAT 
> DELIMITED FIELDS TERMINATED BY ' ' STORED AS TEXTFILE location 
> 'hdfs://ns1/tmp/hive_table_cloud_load_data_in_path_123'
>  
> load data inpath 'hdfs://ns1/tmp/data123.txt' into table 
> hive_table_cloud_load_data_in_path_123;
>  
> Creates 'hdfs://ns1/tmp/data123.txt' as shell entity. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (ATLAS-4324) FS entity created for load data inpath is created as shell entity

[Sarath Subramanian (Jira)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-4324?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sarath Subramanian updated ATLAS-4324:
--
Fix Version/s: 2.2.0
   3.0.0

> FS entity created for load data inpath is created as shell entity
> -
>
> Key: ATLAS-4324
> URL: https://issues.apache.org/jira/browse/ATLAS-4324
> Project: Atlas
>  Issue Type: Bug
>Affects Versions: 2.1.0
>Reporter: Radhika Kundam
>Assignee: Radhika Kundam
>Priority: Major
> Fix For: 3.0.0, 2.2.0
>
> Attachments: load_data_shell_entity.png
>
>
> beeline >
> create external table default.hive_table_cloud_load_data_in_path_123 
> (student_roll int, student_name string, student_dob date) ROW FORMAT 
> DELIMITED FIELDS TERMINATED BY ' ' STORED AS TEXTFILE location 
> 'hdfs://ns1/tmp/hive_table_cloud_load_data_in_path_123'
>  
> load data inpath 'hdfs://ns1/tmp/data123.txt' into table 
> hive_table_cloud_load_data_in_path_123;
>  
> Creates 'hdfs://ns1/tmp/data123.txt' as shell entity. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (ATLAS-4324) FS entity created for load data inpath is created as shell entity

[Sarath Subramanian (Jira)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-4324?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sarath Subramanian updated ATLAS-4324:
--
Affects Version/s: 2.1.0

> FS entity created for load data inpath is created as shell entity
> -
>
> Key: ATLAS-4324
> URL: https://issues.apache.org/jira/browse/ATLAS-4324
> Project: Atlas
>  Issue Type: Bug
>Affects Versions: 2.1.0
>Reporter: Radhika Kundam
>Assignee: Radhika Kundam
>Priority: Major
> Attachments: load_data_shell_entity.png
>
>
> beeline >
> create external table default.hive_table_cloud_load_data_in_path_123 
> (student_roll int, student_name string, student_dob date) ROW FORMAT 
> DELIMITED FIELDS TERMINATED BY ' ' STORED AS TEXTFILE location 
> 'hdfs://ns1/tmp/hive_table_cloud_load_data_in_path_123'
>  
> load data inpath 'hdfs://ns1/tmp/data123.txt' into table 
> hive_table_cloud_load_data_in_path_123;
>  
> Creates 'hdfs://ns1/tmp/data123.txt' as shell entity. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (ATLAS-4324) FS entity created for load data inpath is created as shell entity

[ASF subversion and git services (Jira)]

[ 
https://issues.apache.org/jira/browse/ATLAS-4324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17356563#comment-17356563
 ] 

ASF subversion and git services commented on ATLAS-4324:


Commit 3c607836173472dfa05e60ff102c50133e02b709 in atlas's branch 
refs/heads/branch-2.0 from Radhika Kundam
[ https://gitbox.apache.org/repos/asf?p=atlas.git;h=3c60783 ]

ATLAS-4324: FS entity created for load data inpath is created as shell entity

Signed-off-by: Sarath Subramanian 
(cherry picked from commit a9e528bca9277707a1847d110c33288188a8fff5)


> FS entity created for load data inpath is created as shell entity
> -
>
> Key: ATLAS-4324
> URL: https://issues.apache.org/jira/browse/ATLAS-4324
> Project: Atlas
>  Issue Type: Bug
>Reporter: Radhika Kundam
>Assignee: Radhika Kundam
>Priority: Major
> Attachments: load_data_shell_entity.png
>
>
> beeline >
> create external table default.hive_table_cloud_load_data_in_path_123 
> (student_roll int, student_name string, student_dob date) ROW FORMAT 
> DELIMITED FIELDS TERMINATED BY ' ' STORED AS TEXTFILE location 
> 'hdfs://ns1/tmp/hive_table_cloud_load_data_in_path_123'
>  
> load data inpath 'hdfs://ns1/tmp/data123.txt' into table 
> hive_table_cloud_load_data_in_path_123;
>  
> Creates 'hdfs://ns1/tmp/data123.txt' as shell entity. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (ATLAS-4324) FS entity created for load data inpath is created as shell entity

[ASF subversion and git services (Jira)]

[ 
https://issues.apache.org/jira/browse/ATLAS-4324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17356562#comment-17356562
 ] 

ASF subversion and git services commented on ATLAS-4324:


Commit a9e528bca9277707a1847d110c33288188a8fff5 in atlas's branch 
refs/heads/master from Radhika Kundam
[ https://gitbox.apache.org/repos/asf?p=atlas.git;h=a9e528b ]

ATLAS-4324: FS entity created for load data inpath is created as shell entity

Signed-off-by: Sarath Subramanian 


> FS entity created for load data inpath is created as shell entity
> -
>
> Key: ATLAS-4324
> URL: https://issues.apache.org/jira/browse/ATLAS-4324
> Project: Atlas
>  Issue Type: Bug
>Reporter: Radhika Kundam
>Assignee: Radhika Kundam
>Priority: Major
> Attachments: load_data_shell_entity.png
>
>
> beeline >
> create external table default.hive_table_cloud_load_data_in_path_123 
> (student_roll int, student_name string, student_dob date) ROW FORMAT 
> DELIMITED FIELDS TERMINATED BY ' ' STORED AS TEXTFILE location 
> 'hdfs://ns1/tmp/hive_table_cloud_load_data_in_path_123'
>  
> load data inpath 'hdfs://ns1/tmp/data123.txt' into table 
> hive_table_cloud_load_data_in_path_123;
>  
> Creates 'hdfs://ns1/tmp/data123.txt' as shell entity. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (ATLAS-4301) Handle Test Case Failure on Pre-commit environment

[Sarath Subramanian (Jira)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-4301?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sarath Subramanian updated ATLAS-4301:
--
Labels: hive-hooks integration-tests  (was: )

> Handle Test Case Failure on Pre-commit environment
> --
>
> Key: ATLAS-4301
> URL: https://issues.apache.org/jira/browse/ATLAS-4301
> Project: Atlas
>  Issue Type: Test
>  Components: hive-integration
>Affects Versions: 2.1.0
>Reporter: Mandar Ambawane
>Assignee: Mandar Ambawane
>Priority: Major
>  Labels: hive-hooks, integration-tests
> Fix For: 3.0.0, 2.2.0
>
> Attachments: ATLAS-4301.patch
>
>
> Getting following error on Pre-commit build due to change in the file path
> hive-bridge/target/logs/application.log
> {code:java}
> Wrong FS: 
> pfile:/home/jenkins/jenkins-agent/workspace/Atlas/PreCommit-ATLAS-Build-Test/addons/hive-bridge/target/partition-data-{dir},
>  expected: file:///
> {code}
>  
> Also,
> There is Enum "Type" in the "Entity"
> {code:java}
> Class: org.apache.hadoop.hive.ql.hooks.Entity
> Enum: Type{code}
> Enum "Type" has one constant "LOCAL_DIR"
> Due to recent changes, we need to provide support for this constant 
> "LOCAL_DIR" in Testing Environment.
> Without which following issues occuring on Testing Environment:
>  # While creating "hive_process" entity, The "outputs" attribute is not 
> getting set (Which is of type "hdfs_path").
>  # While setting the "qualifiedName" of "hive_process" entity, File path is 
> not getting appended.
> This causing Failure of some Test cases.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (ATLAS-4301) Handle Test Case Failure on Pre-commit environment

[Sarath Subramanian (Jira)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-4301?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sarath Subramanian updated ATLAS-4301:
--
Component/s: hive-integration

> Handle Test Case Failure on Pre-commit environment
> --
>
> Key: ATLAS-4301
> URL: https://issues.apache.org/jira/browse/ATLAS-4301
> Project: Atlas
>  Issue Type: Test
>  Components: hive-integration
>Affects Versions: 2.1.0
>Reporter: Mandar Ambawane
>Assignee: Mandar Ambawane
>Priority: Major
> Fix For: 3.0.0, 2.2.0
>
> Attachments: ATLAS-4301.patch
>
>
> Getting following error on Pre-commit build due to change in the file path
> hive-bridge/target/logs/application.log
> {code:java}
> Wrong FS: 
> pfile:/home/jenkins/jenkins-agent/workspace/Atlas/PreCommit-ATLAS-Build-Test/addons/hive-bridge/target/partition-data-{dir},
>  expected: file:///
> {code}
>  
> Also,
> There is Enum "Type" in the "Entity"
> {code:java}
> Class: org.apache.hadoop.hive.ql.hooks.Entity
> Enum: Type{code}
> Enum "Type" has one constant "LOCAL_DIR"
> Due to recent changes, we need to provide support for this constant 
> "LOCAL_DIR" in Testing Environment.
> Without which following issues occuring on Testing Environment:
>  # While creating "hive_process" entity, The "outputs" attribute is not 
> getting set (Which is of type "hdfs_path").
>  # While setting the "qualifiedName" of "hive_process" entity, File path is 
> not getting appended.
> This causing Failure of some Test cases.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (ATLAS-4301) Handle Test Case Failure on Pre-commit environment

[Sarath Subramanian (Jira)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-4301?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sarath Subramanian updated ATLAS-4301:
--
Issue Type: Test  (was: Improvement)

> Handle Test Case Failure on Pre-commit environment
> --
>
> Key: ATLAS-4301
> URL: https://issues.apache.org/jira/browse/ATLAS-4301
> Project: Atlas
>  Issue Type: Test
>Affects Versions: 2.1.0
>Reporter: Mandar Ambawane
>Assignee: Mandar Ambawane
>Priority: Major
> Attachments: ATLAS-4301.patch
>
>
> Getting following error on Pre-commit build due to change in the file path
> hive-bridge/target/logs/application.log
> {code:java}
> Wrong FS: 
> pfile:/home/jenkins/jenkins-agent/workspace/Atlas/PreCommit-ATLAS-Build-Test/addons/hive-bridge/target/partition-data-{dir},
>  expected: file:///
> {code}
>  
> Also,
> There is Enum "Type" in the "Entity"
> {code:java}
> Class: org.apache.hadoop.hive.ql.hooks.Entity
> Enum: Type{code}
> Enum "Type" has one constant "LOCAL_DIR"
> Due to recent changes, we need to provide support for this constant 
> "LOCAL_DIR" in Testing Environment.
> Without which following issues occuring on Testing Environment:
>  # While creating "hive_process" entity, The "outputs" attribute is not 
> getting set (Which is of type "hdfs_path").
>  # While setting the "qualifiedName" of "hive_process" entity, File path is 
> not getting appended.
> This causing Failure of some Test cases.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (ATLAS-4301) Handle Test Case Failure on Pre-commit environment

[Sarath Subramanian (Jira)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-4301?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sarath Subramanian updated ATLAS-4301:
--
Affects Version/s: 2.1.0

> Handle Test Case Failure on Pre-commit environment
> --
>
> Key: ATLAS-4301
> URL: https://issues.apache.org/jira/browse/ATLAS-4301
> Project: Atlas
>  Issue Type: Improvement
>Affects Versions: 2.1.0
>Reporter: Mandar Ambawane
>Assignee: Mandar Ambawane
>Priority: Major
> Attachments: ATLAS-4301.patch
>
>
> Getting following error on Pre-commit build due to change in the file path
> hive-bridge/target/logs/application.log
> {code:java}
> Wrong FS: 
> pfile:/home/jenkins/jenkins-agent/workspace/Atlas/PreCommit-ATLAS-Build-Test/addons/hive-bridge/target/partition-data-{dir},
>  expected: file:///
> {code}
>  
> Also,
> There is Enum "Type" in the "Entity"
> {code:java}
> Class: org.apache.hadoop.hive.ql.hooks.Entity
> Enum: Type{code}
> Enum "Type" has one constant "LOCAL_DIR"
> Due to recent changes, we need to provide support for this constant 
> "LOCAL_DIR" in Testing Environment.
> Without which following issues occuring on Testing Environment:
>  # While creating "hive_process" entity, The "outputs" attribute is not 
> getting set (Which is of type "hdfs_path").
>  # While setting the "qualifiedName" of "hive_process" entity, File path is 
> not getting appended.
> This causing Failure of some Test cases.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (ATLAS-4301) Handle Test Case Failure on Pre-commit environment

[Sarath Subramanian (Jira)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-4301?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sarath Subramanian updated ATLAS-4301:
--
Fix Version/s: 2.2.0
   3.0.0

> Handle Test Case Failure on Pre-commit environment
> --
>
> Key: ATLAS-4301
> URL: https://issues.apache.org/jira/browse/ATLAS-4301
> Project: Atlas
>  Issue Type: Test
>Affects Versions: 2.1.0
>Reporter: Mandar Ambawane
>Assignee: Mandar Ambawane
>Priority: Major
> Fix For: 3.0.0, 2.2.0
>
> Attachments: ATLAS-4301.patch
>
>
> Getting following error on Pre-commit build due to change in the file path
> hive-bridge/target/logs/application.log
> {code:java}
> Wrong FS: 
> pfile:/home/jenkins/jenkins-agent/workspace/Atlas/PreCommit-ATLAS-Build-Test/addons/hive-bridge/target/partition-data-{dir},
>  expected: file:///
> {code}
>  
> Also,
> There is Enum "Type" in the "Entity"
> {code:java}
> Class: org.apache.hadoop.hive.ql.hooks.Entity
> Enum: Type{code}
> Enum "Type" has one constant "LOCAL_DIR"
> Due to recent changes, we need to provide support for this constant 
> "LOCAL_DIR" in Testing Environment.
> Without which following issues occuring on Testing Environment:
>  # While creating "hive_process" entity, The "outputs" attribute is not 
> getting set (Which is of type "hdfs_path").
>  # While setting the "qualifiedName" of "hive_process" entity, File path is 
> not getting appended.
> This causing Failure of some Test cases.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (ATLAS-4301) Handle Test Case Failure on Pre-commit environment

[ASF subversion and git services (Jira)]

[ 
https://issues.apache.org/jira/browse/ATLAS-4301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17356546#comment-17356546
 ] 

ASF subversion and git services commented on ATLAS-4301:


Commit d24ce80914594831a03aa18e5e8cc7d129590d88 in atlas's branch 
refs/heads/branch-2.0 from Mandar Ambawane
[ https://gitbox.apache.org/repos/asf?p=atlas.git;h=d24ce80 ]

ATLAS-4301 Handle Integration Test Failures on HiveHook module

Signed-off-by: Sarath Subramanian 
(cherry picked from commit e993f85d7b2c5e11914a328f4b3aee4392c4e106)


> Handle Test Case Failure on Pre-commit environment
> --
>
> Key: ATLAS-4301
> URL: https://issues.apache.org/jira/browse/ATLAS-4301
> Project: Atlas
>  Issue Type: Improvement
>Reporter: Mandar Ambawane
>Assignee: Mandar Ambawane
>Priority: Major
> Attachments: ATLAS-4301.patch
>
>
> Getting following error on Pre-commit build due to change in the file path
> hive-bridge/target/logs/application.log
> {code:java}
> Wrong FS: 
> pfile:/home/jenkins/jenkins-agent/workspace/Atlas/PreCommit-ATLAS-Build-Test/addons/hive-bridge/target/partition-data-{dir},
>  expected: file:///
> {code}
>  
> Also,
> There is Enum "Type" in the "Entity"
> {code:java}
> Class: org.apache.hadoop.hive.ql.hooks.Entity
> Enum: Type{code}
> Enum "Type" has one constant "LOCAL_DIR"
> Due to recent changes, we need to provide support for this constant 
> "LOCAL_DIR" in Testing Environment.
> Without which following issues occuring on Testing Environment:
>  # While creating "hive_process" entity, The "outputs" attribute is not 
> getting set (Which is of type "hdfs_path").
>  # While setting the "qualifiedName" of "hive_process" entity, File path is 
> not getting appended.
> This causing Failure of some Test cases.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (ATLAS-4301) Handle Test Case Failure on Pre-commit environment

[ASF subversion and git services (Jira)]

[ 
https://issues.apache.org/jira/browse/ATLAS-4301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17356545#comment-17356545
 ] 

ASF subversion and git services commented on ATLAS-4301:


Commit e993f85d7b2c5e11914a328f4b3aee4392c4e106 in atlas's branch 
refs/heads/master from Mandar Ambawane
[ https://gitbox.apache.org/repos/asf?p=atlas.git;h=e993f85 ]

ATLAS-4301 Handle Integration Test Failures on HiveHook module

Signed-off-by: Sarath Subramanian 


> Handle Test Case Failure on Pre-commit environment
> --
>
> Key: ATLAS-4301
> URL: https://issues.apache.org/jira/browse/ATLAS-4301
> Project: Atlas
>  Issue Type: Improvement
>Reporter: Mandar Ambawane
>Assignee: Mandar Ambawane
>Priority: Major
> Attachments: ATLAS-4301.patch
>
>
> Getting following error on Pre-commit build due to change in the file path
> hive-bridge/target/logs/application.log
> {code:java}
> Wrong FS: 
> pfile:/home/jenkins/jenkins-agent/workspace/Atlas/PreCommit-ATLAS-Build-Test/addons/hive-bridge/target/partition-data-{dir},
>  expected: file:///
> {code}
>  
> Also,
> There is Enum "Type" in the "Entity"
> {code:java}
> Class: org.apache.hadoop.hive.ql.hooks.Entity
> Enum: Type{code}
> Enum "Type" has one constant "LOCAL_DIR"
> Due to recent changes, we need to provide support for this constant 
> "LOCAL_DIR" in Testing Environment.
> Without which following issues occuring on Testing Environment:
>  # While creating "hive_process" entity, The "outputs" attribute is not 
> getting set (Which is of type "hdfs_path").
>  # While setting the "qualifiedName" of "hive_process" entity, File path is 
> not getting appended.
> This causing Failure of some Test cases.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: Review Request 73402: ATLAS-4301 Handle Test Case Failure on Pre-commit environment

[Sarath Subramanian]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/73402/#review223096
---


Ship it!




good work! thanks for fixing the flaky tests :)

- Sarath Subramanian


On June 3, 2021, 7:23 a.m., Mandar Ambawane wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/73402/
> ---
> 
> (Updated June 3, 2021, 7:23 a.m.)
> 
> 
> Review request for atlas, Ashutosh Mestry, Jayendra Parab, Nixon Rodrigues, 
> and Sarath Subramanian.
> 
> 
> Bugs: ATLAS-4301
> https://issues.apache.org/jira/browse/ATLAS-4301
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> Getting following error on Pre-commit build due to change in the file path
> 
> hive-bridge/target/logs/application.log
> 
> Wrong FS: 
> pfile:/home/jenkins/jenkins-agent/workspace/Atlas/PreCommit-ATLAS-Build-Test/addons/hive-bridge/target/partition-data-{dir},
>  expected: file:///
>  
> 
> Also,
> 
> There is Enum "Type" in the "Entity"
> 
> Class: org.apache.hadoop.hive.ql.hooks.Entity
> Enum: Type
> Enum "Type" has one constant "LOCAL_DIR"
> 
> Due to recent changes, we need to provide support for this constant 
> "LOCAL_DIR" in Testing Environment.
> 
> Without which following issues occuring on Testing Environment:
> 
> 1. While creating "hive_process" entity, The "outputs" attribute is not 
> getting set (Which is of type "hdfs_path").
> 
> 2. While setting the "qualifiedName" of "hive_process" entity, File path is 
> not getting appended.
> 
> This causing Failure of some Test cases.
> 
> 
> Diffs
> -
> 
>   
> addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/events/BaseHiveEvent.java
>  21c0ae3 
>   addons/hive-bridge/src/test/java/org/apache/atlas/hive/HiveITBase.java 
> 7a11ec3 
>   addons/hive-bridge/src/test/java/org/apache/atlas/hive/hook/HiveHookIT.java 
> 52684e7 
> 
> 
> Diff: https://reviews.apache.org/r/73402/diff/1/
> 
> 
> Testing
> ---
> 
> PreCommit: 
> https://ci-builds.apache.org/job/Atlas/job/PreCommit-ATLAS-Build-Test/612/consoleFull
> 
> 
> Thanks,
> 
> Mandar Ambawane
> 
>

Re: Review Request 73402: ATLAS-4301 Handle Test Case Failure on Pre-commit environment

[Ashutosh Mestry via Review Board]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/73402/#review223095
---


Ship it!




Can you please add details of end-to-end testing done for HMS and HS2 hooks? 
Essentially we should exercise the 'load inpath' queries.

Good work!

- Ashutosh Mestry


On June 3, 2021, 2:23 p.m., Mandar Ambawane wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/73402/
> ---
> 
> (Updated June 3, 2021, 2:23 p.m.)
> 
> 
> Review request for atlas, Ashutosh Mestry, Jayendra Parab, Nixon Rodrigues, 
> and Sarath Subramanian.
> 
> 
> Bugs: ATLAS-4301
> https://issues.apache.org/jira/browse/ATLAS-4301
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> Getting following error on Pre-commit build due to change in the file path
> 
> hive-bridge/target/logs/application.log
> 
> Wrong FS: 
> pfile:/home/jenkins/jenkins-agent/workspace/Atlas/PreCommit-ATLAS-Build-Test/addons/hive-bridge/target/partition-data-{dir},
>  expected: file:///
>  
> 
> Also,
> 
> There is Enum "Type" in the "Entity"
> 
> Class: org.apache.hadoop.hive.ql.hooks.Entity
> Enum: Type
> Enum "Type" has one constant "LOCAL_DIR"
> 
> Due to recent changes, we need to provide support for this constant 
> "LOCAL_DIR" in Testing Environment.
> 
> Without which following issues occuring on Testing Environment:
> 
> 1. While creating "hive_process" entity, The "outputs" attribute is not 
> getting set (Which is of type "hdfs_path").
> 
> 2. While setting the "qualifiedName" of "hive_process" entity, File path is 
> not getting appended.
> 
> This causing Failure of some Test cases.
> 
> 
> Diffs
> -
> 
>   
> addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/events/BaseHiveEvent.java
>  21c0ae3 
>   addons/hive-bridge/src/test/java/org/apache/atlas/hive/HiveITBase.java 
> 7a11ec3 
>   addons/hive-bridge/src/test/java/org/apache/atlas/hive/hook/HiveHookIT.java 
> 52684e7 
> 
> 
> Diff: https://reviews.apache.org/r/73402/diff/1/
> 
> 
> Testing
> ---
> 
> PreCommit: 
> https://ci-builds.apache.org/job/Atlas/job/PreCommit-ATLAS-Build-Test/612/consoleFull
> 
> 
> Thanks,
> 
> Mandar Ambawane
> 
>

Re: Review Request 73317: ATLAS-3812: Add schema for Apache Flink

[Ashutosh Mestry via Review Board]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/73317/#review223094
---


Ship it!




Ship It!

- Ashutosh Mestry


On June 1, 2021, 6:59 p.m., Josh Yeh wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/73317/
> ---
> 
> (Updated June 1, 2021, 6:59 p.m.)
> 
> 
> Review request for atlas, Ashutosh Mestry, Pinal Shah, and Sarath Subramanian.
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> ATLAS-3812: Add schema for Apache Flink
> 
> 
> Diffs
> -
> 
>   addons/models/1000-Hadoop/1110-flink_model.json PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/73317/diff/4/
> 
> 
> Testing
> ---
> 
> Tested on my CDH cluster with flink example wordcount streaming job.
> 
> 
> File Attachments
> 
> 
> Screen Shot 2021-04-30 at 12.38.27 AM.png
>   
> https://reviews.apache.org/media/uploaded/files/2021/04/30/2ff59445-7237-4be1-a1a0-b6db70bc7827__Screen_Shot_2021-04-30_at_12.38.27_AM.png
> 
> 
> Thanks,
> 
> Josh Yeh
> 
>

Review Request 73402: ATLAS-4301 Handle Test Case Failure on Pre-commit environment

[Mandar Ambawane]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/73402/
---

Review request for atlas, Ashutosh Mestry, Jayendra Parab, Nixon Rodrigues, and 
Sarath Subramanian.


Bugs: ATLAS-4301
https://issues.apache.org/jira/browse/ATLAS-4301


Repository: atlas


Description
---

Getting following error on Pre-commit build due to change in the file path

hive-bridge/target/logs/application.log

Wrong FS: 
pfile:/home/jenkins/jenkins-agent/workspace/Atlas/PreCommit-ATLAS-Build-Test/addons/hive-bridge/target/partition-data-{dir},
 expected: file:///
 

Also,

There is Enum "Type" in the "Entity"

Class: org.apache.hadoop.hive.ql.hooks.Entity
Enum: Type
Enum "Type" has one constant "LOCAL_DIR"

Due to recent changes, we need to provide support for this constant "LOCAL_DIR" 
in Testing Environment.

Without which following issues occuring on Testing Environment:

1. While creating "hive_process" entity, The "outputs" attribute is not getting 
set (Which is of type "hdfs_path").

2. While setting the "qualifiedName" of "hive_process" entity, File path is not 
getting appended.

This causing Failure of some Test cases.


Diffs
-

  
addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/events/BaseHiveEvent.java
 21c0ae3 
  addons/hive-bridge/src/test/java/org/apache/atlas/hive/HiveITBase.java 
7a11ec3 
  addons/hive-bridge/src/test/java/org/apache/atlas/hive/hook/HiveHookIT.java 
52684e7 


Diff: https://reviews.apache.org/r/73402/diff/1/


Testing
---

PreCommit: 
https://ci-builds.apache.org/job/Atlas/job/PreCommit-ATLAS-Build-Test/612/consoleFull


Thanks,

Mandar Ambawane

[jira] [Updated] (ATLAS-4301) Handle Test Case Failure on Pre-commit environment

[Mandar Ambawane (Jira)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-4301?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mandar Ambawane updated ATLAS-4301:
---
Description: 
Getting following error on Pre-commit build due to change in the file path

hive-bridge/target/logs/application.log
{code:java}
Wrong FS: 
pfile:/home/jenkins/jenkins-agent/workspace/Atlas/PreCommit-ATLAS-Build-Test/addons/hive-bridge/target/partition-data-{dir},
 expected: file:///
{code}
 

Also,

There is Enum "Type" in the "Entity"
{code:java}
Class: org.apache.hadoop.hive.ql.hooks.Entity
Enum: Type{code}
Enum "Type" has one constant "LOCAL_DIR"

Due to recent changes, we need to provide support for this constant "LOCAL_DIR" 
in Testing Environment.

Without which following issues occuring on Testing Environment:
 # While creating "hive_process" entity, The "outputs" attribute is not getting 
set (Which is of type "hdfs_path").
 # While setting the "qualifiedName" of "hive_process" entity, File path is not 
getting appended.

This causing Failure of some Test cases.

  was:
Getting following error on Pre-commit build due to change in the file path

 hive-bridge/target/logs/application.log
{code:java}
Wrong FS: 
pfile:/home/jenkins/jenkins-agent/workspace/Atlas/PreCommit-ATLAS-Build-Test/addons/hive-bridge/target/partition-data-{dir},
 expected: file:///
{code}


> Handle Test Case Failure on Pre-commit environment
> --
>
> Key: ATLAS-4301
> URL: https://issues.apache.org/jira/browse/ATLAS-4301
> Project: Atlas
>  Issue Type: Improvement
>Reporter: Mandar Ambawane
>Assignee: Mandar Ambawane
>Priority: Major
>
> Getting following error on Pre-commit build due to change in the file path
> hive-bridge/target/logs/application.log
> {code:java}
> Wrong FS: 
> pfile:/home/jenkins/jenkins-agent/workspace/Atlas/PreCommit-ATLAS-Build-Test/addons/hive-bridge/target/partition-data-{dir},
>  expected: file:///
> {code}
>  
> Also,
> There is Enum "Type" in the "Entity"
> {code:java}
> Class: org.apache.hadoop.hive.ql.hooks.Entity
> Enum: Type{code}
> Enum "Type" has one constant "LOCAL_DIR"
> Due to recent changes, we need to provide support for this constant 
> "LOCAL_DIR" in Testing Environment.
> Without which following issues occuring on Testing Environment:
>  # While creating "hive_process" entity, The "outputs" attribute is not 
> getting set (Which is of type "hdfs_path").
>  # While setting the "qualifiedName" of "hive_process" entity, File path is 
> not getting appended.
> This causing Failure of some Test cases.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[GitHub] [atlas] asdf2014 commented on pull request #134: Remove unnecessary imports from python script

[GitBox]

asdf2014 commented on pull request #134:
URL: https://github.com/apache/atlas/pull/134#issuecomment-853814520


   Hi, @nixonrodrigues. Thanks for your comments. This patch is very minor, so 
I didn’t create a JIRA for it.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[jira] [Updated] (ATLAS-4311) [Business Metadata Bulk Import] Wrong Line Number returned in failedImportInfoList

[Radhika Kundam (Jira)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-4311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Radhika Kundam updated ATLAS-4311:
--
Fix Version/s: 3.0.0

> [Business Metadata Bulk Import] Wrong Line Number returned in 
> failedImportInfoList 
> ---
>
> Key: ATLAS-4311
> URL: https://issues.apache.org/jira/browse/ATLAS-4311
> Project: Atlas
>  Issue Type: Bug
>  Components:  atlas-core
>Reporter: Umesh Padashetty
>Assignee: Radhika Kundam
>Priority: Minor
> Fix For: 3.0.0, 2.2.0
>
>
> The api/atlas/v2/entity/businessmetadata/import returns the incorrect Line 
> Number
> For instance, following is the data in template file 
> {code:java}
> EntityType,EntityUniqueAttributeValue,BusinessAttributeName,BusinessAttributeValue,EntityUniqueAttributeName[optional]
>  
> hive_table,hive_table_kcsup@cm,BM3.BM3A1,stringval 
> hive_table,invalid_table@cm,BM3.BM3A1,stringval  {code}
> As seen above, on line #3, we have entity name as invalid_table@cm, this 
> entity is not existing in atlas.
> But the businessmetadata/import API returns 
> {code:java}
> { "failedImportInfoList": [ { "parentObjectName": "", "childObjectName": "", 
> "importStatus": "FAILED", "remarks": "Line #2: no hive_table entity found 
> with qualifiedName=invalid_table@cm" } ] }  {code}
> As seen above, Line number should have been returned as 3, but it is returned 
> as 2.
> This probably is happening because we intrinsically ignore the first line in 
> the template file



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (ATLAS-4311) [Business Metadata Bulk Import] Wrong Line Number returned in failedImportInfoList

[Radhika Kundam (Jira)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-4311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Radhika Kundam updated ATLAS-4311:
--
Fix Version/s: 2.2.0

> [Business Metadata Bulk Import] Wrong Line Number returned in 
> failedImportInfoList 
> ---
>
> Key: ATLAS-4311
> URL: https://issues.apache.org/jira/browse/ATLAS-4311
> Project: Atlas
>  Issue Type: Bug
>  Components:  atlas-core
>Reporter: Umesh Padashetty
>Assignee: Radhika Kundam
>Priority: Minor
> Fix For: 2.2.0
>
>
> The api/atlas/v2/entity/businessmetadata/import returns the incorrect Line 
> Number
> For instance, following is the data in template file 
> {code:java}
> EntityType,EntityUniqueAttributeValue,BusinessAttributeName,BusinessAttributeValue,EntityUniqueAttributeName[optional]
>  
> hive_table,hive_table_kcsup@cm,BM3.BM3A1,stringval 
> hive_table,invalid_table@cm,BM3.BM3A1,stringval  {code}
> As seen above, on line #3, we have entity name as invalid_table@cm, this 
> entity is not existing in atlas.
> But the businessmetadata/import API returns 
> {code:java}
> { "failedImportInfoList": [ { "parentObjectName": "", "childObjectName": "", 
> "importStatus": "FAILED", "remarks": "Line #2: no hive_table entity found 
> with qualifiedName=invalid_table@cm" } ] }  {code}
> As seen above, Line number should have been returned as 3, but it is returned 
> as 2.
> This probably is happening because we intrinsically ignore the first line in 
> the template file



--
This message was sent by Atlassian Jira
(v8.3.4#803005)