[jira] [Commented] (ATLAS-3226) Add QueryText for hive_table and hive_db for Impala integration

2019-05-30 Thread Na Li (JIRA) 


[ 
https://issues.apache.org/jira/browse/ATLAS-3226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16852660#comment-16852660
 ] 

Na Li commented on ATLAS-3226:
--

patch 3 pre-test succeeded at 
https://builds.apache.org/view/A/view/Atlas/job/PreCommit-ATLAS-Build-Test/1157/console
 

> Add QueryText for hive_table and hive_db for Impala integration
> ---
>
> Key: ATLAS-3226
> URL: https://issues.apache.org/jira/browse/ATLAS-3226
> Project: Atlas
>  Issue Type: New Feature
>  Components:  atlas-core
>Reporter: Na Li
>Assignee: Na Li
>Priority: Major
> Attachments: ATLAS-3226.001.patch, ATLAS-3226.002.patch, 
> ATLAS-3226.003.patch
>
>
> ATLAS-3197 introduced a new feature on Hive integration. It introduced new 
> model called Hive_table_ddl and hive_db_ddl which are used to track commands 
> that are executed either on a table entity or bd entity. DDL entities will be 
> added to a table/db entity through relationshipAttribute call HIVE_DB_TO_DDL 
> and hive_tb_to_ddl. Every time when a new command is executed, a new ddl 
> entity will be created, it will append to the existing ddlQueries field if a 
> table/db has already been created. Once a table/db is deleted, all these 
> relationship attributes will be deleted as well.
> We need to support this new feature on Impala integration.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: Review Request 70759: ATLAS-3226: Add QueryText for hive_table and hive_db for Impala integration

2019-05-30 Thread Na Li via Review Board 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/70759/#review215608
---




addons/models/1000-Hadoop/1030-hive_model.json
Line 727 (original), 693 (patched)


has to use common type for "hive_table_ddl" and "impala_table_ddl" in order 
for it to work for both hive and impala



addons/models/1000-Hadoop/1030-hive_model.json
Line 746 (original), 712 (patched)


the endpoint of the relationship has to be the common type of hive_db_ddl 
and impala_db_ddl in order for Atlas server to update the "ddlQueries" of a 
hive_db for both hive_db_ddl and impala_db_ddl


- Na Li


On May 31, 2019, 4:02 a.m., Na Li wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/70759/
> ---
> 
> (Updated May 31, 2019, 4:02 a.m.)
> 
> 
> Review request for atlas, madhan and Sarath Subramanian.
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> ATLAS-3197 introduced a new feature on Hive integration. It introduced new 
> model called Hive_table_ddl and hive_db_ddl which are used to track commands 
> that are executed either on a table entity or bd entity. DDL entities will be 
> added to a table/db entity through relationshipAttribute call HIVE_DB_TO_DDL 
> and hive_tb_to_ddl. Every time when a new command is executed, a new ddl 
> entity will be created, it will append to the existing ddlQueries field if a 
> table/db has already been created. Once a table/db is deleted, all these 
> relationship attributes will be deleted as well.
> 
> We need to support this new feature on Impala integration.
> 
> 
> Diffs
> -
> 
>   
> addons/impala-bridge/src/main/java/org/apache/atlas/impala/hook/events/BaseImpalaEvent.java
>  d241b6a 
>   
> addons/impala-bridge/src/main/java/org/apache/atlas/impala/hook/events/CreateImpalaProcess.java
>  b7506a4 
>   
> addons/impala-bridge/src/main/java/org/apache/atlas/impala/model/ImpalaDataType.java
>  10ce448 
>   
> addons/impala-bridge/src/test/java/org/apache/atlas/impala/ImpalaLineageITBase.java
>  b8cbf6b 
>   
> addons/impala-bridge/src/test/java/org/apache/atlas/impala/ImpalaLineageToolIT.java
>  7f9a534 
>   
> addons/impala-bridge/src/test/java/org/apache/atlas/impala/hook/ImpalaLineageHookIT.java
>  6156208 
>   addons/models/-Area0/0010-base_model.json d14b227 
>   addons/models/1000-Hadoop/1030-hive_model.json 417c4c5 
>   addons/models/1000-Hadoop/1090-impala_model.json 24eb5db 
> 
> 
> Diff: https://reviews.apache.org/r/70759/diff/3/
> 
> 
> Testing
> ---
> 
> add check for ddl entities in integration tests
> 
> 
> Thanks,
> 
> Na Li
> 
>

[jira] [Updated] (ATLAS-3226) Add QueryText for hive_table and hive_db for Impala integration

2019-05-30 Thread Na Li (JIRA) 


 [ 
https://issues.apache.org/jira/browse/ATLAS-3226?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Na Li updated ATLAS-3226:
-
Attachment: ATLAS-3226.003.patch

> Add QueryText for hive_table and hive_db for Impala integration
> ---
>
> Key: ATLAS-3226
> URL: https://issues.apache.org/jira/browse/ATLAS-3226
> Project: Atlas
>  Issue Type: New Feature
>  Components:  atlas-core
>Reporter: Na Li
>Assignee: Na Li
>Priority: Major
> Attachments: ATLAS-3226.001.patch, ATLAS-3226.002.patch, 
> ATLAS-3226.003.patch
>
>
> ATLAS-3197 introduced a new feature on Hive integration. It introduced new 
> model called Hive_table_ddl and hive_db_ddl which are used to track commands 
> that are executed either on a table entity or bd entity. DDL entities will be 
> added to a table/db entity through relationshipAttribute call HIVE_DB_TO_DDL 
> and hive_tb_to_ddl. Every time when a new command is executed, a new ddl 
> entity will be created, it will append to the existing ddlQueries field if a 
> table/db has already been created. Once a table/db is deleted, all these 
> relationship attributes will be deleted as well.
> We need to support this new feature on Impala integration.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (ATLAS-3226) Add QueryText for hive_table and hive_db for Impala integration

2019-05-30 Thread Na Li (JIRA) 


[ 
https://issues.apache.org/jira/browse/ATLAS-3226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16852634#comment-16852634
 ] 

Na Li commented on ATLAS-3226:
--

patch 3 pre-commit test at 
https://builds.apache.org/view/A/view/Atlas/job/PreCommit-ATLAS-Build-Test/1157/console

> Add QueryText for hive_table and hive_db for Impala integration
> ---
>
> Key: ATLAS-3226
> URL: https://issues.apache.org/jira/browse/ATLAS-3226
> Project: Atlas
>  Issue Type: New Feature
>  Components:  atlas-core
>Reporter: Na Li
>Assignee: Na Li
>Priority: Major
> Attachments: ATLAS-3226.001.patch, ATLAS-3226.002.patch, 
> ATLAS-3226.003.patch
>
>
> ATLAS-3197 introduced a new feature on Hive integration. It introduced new 
> model called Hive_table_ddl and hive_db_ddl which are used to track commands 
> that are executed either on a table entity or bd entity. DDL entities will be 
> added to a table/db entity through relationshipAttribute call HIVE_DB_TO_DDL 
> and hive_tb_to_ddl. Every time when a new command is executed, a new ddl 
> entity will be created, it will append to the existing ddlQueries field if a 
> table/db has already been created. Once a table/db is deleted, all these 
> relationship attributes will be deleted as well.
> We need to support this new feature on Impala integration.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: Review Request 70759: ATLAS-3226: Add QueryText for hive_table and hive_db for Impala integration

2019-05-30 Thread Na Li via Review Board 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/70759/
---

(Updated May 31, 2019, 4:02 a.m.)


Review request for atlas, madhan and Sarath Subramanian.


Repository: atlas


Description
---

ATLAS-3197 introduced a new feature on Hive integration. It introduced new 
model called Hive_table_ddl and hive_db_ddl which are used to track commands 
that are executed either on a table entity or bd entity. DDL entities will be 
added to a table/db entity through relationshipAttribute call HIVE_DB_TO_DDL 
and hive_tb_to_ddl. Every time when a new command is executed, a new ddl entity 
will be created, it will append to the existing ddlQueries field if a table/db 
has already been created. Once a table/db is deleted, all these relationship 
attributes will be deleted as well.

We need to support this new feature on Impala integration.


Diffs (updated)
-

  
addons/impala-bridge/src/main/java/org/apache/atlas/impala/hook/events/BaseImpalaEvent.java
 d241b6a 
  
addons/impala-bridge/src/main/java/org/apache/atlas/impala/hook/events/CreateImpalaProcess.java
 b7506a4 
  
addons/impala-bridge/src/main/java/org/apache/atlas/impala/model/ImpalaDataType.java
 10ce448 
  
addons/impala-bridge/src/test/java/org/apache/atlas/impala/ImpalaLineageITBase.java
 b8cbf6b 
  
addons/impala-bridge/src/test/java/org/apache/atlas/impala/ImpalaLineageToolIT.java
 7f9a534 
  
addons/impala-bridge/src/test/java/org/apache/atlas/impala/hook/ImpalaLineageHookIT.java
 6156208 
  addons/models/-Area0/0010-base_model.json d14b227 
  addons/models/1000-Hadoop/1030-hive_model.json 417c4c5 
  addons/models/1000-Hadoop/1090-impala_model.json 24eb5db 


Diff: https://reviews.apache.org/r/70759/diff/3/

Changes: https://reviews.apache.org/r/70759/diff/2-3/


Testing
---

add check for ddl entities in integration tests


Thanks,

Na Li

[jira] [Created] (ATLAS-3253) When I register Atlas Hook in Hbase-site.xml by adding the following:Restart HBase Error

2019-05-30 Thread He sheng (JIRA) 
He sheng created ATLAS-3253:
---

 Summary: When I register Atlas Hook in Hbase-site.xml by adding 
the following:Restart HBase Error
 Key: ATLAS-3253
 URL: https://issues.apache.org/jira/browse/ATLAS-3253
 Project: Atlas
  Issue Type: Bug
  Components:  atlas-core
Affects Versions: 1.1.0
 Environment: CDH5.7
atlas version：1.1.0
Reporter: He sheng


| |The coprocessor org.apache.atlas.hbase.hook.HBaseAtlasCoprocessor threw 
java.lang.ClassNotFoundException: 
org.apache.atlas.hbase.hook.HBaseAtlasCoprocessor
java.lang.ClassNotFoundException: 
org.apache.atlas.hbase.hook.HBaseAtlasCoprocessor
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at 
org.apache.hadoop.hbase.coprocessor.CoprocessorHost.loadSystemCoprocessors(CoprocessorHost.java:158)
at 
org.apache.hadoop.hbase.master.MasterCoprocessorHost.(MasterCoprocessorHost.java:87)
at 
org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:691)
at org.apache.hadoop.hbase.master.HMaster.access$600(HMaster.java:188)
at org.apache.hadoop.hbase.master.HMaster$1.run(HMaster.java:1777)
at java.lang.Thread.run(Thread.java:745)|
The
!moz-extension://8260c77a-a03d-49f1-8b6d-70c335e701e7/static/drowDown.svg!
!moz-extension://8260c77a-a03d-49f1-8b6d-70c335e701e7/static/sound.svg!
冠词那个
定义
冠词denoting one or more people or things already mentioned or assumed to be 
common knowledge.
 
 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (ATLAS-3252) When importing HBase metadata, the following error is reported

2019-05-30 Thread He sheng (JIRA) 


 [ 
https://issues.apache.org/jira/browse/ATLAS-3252?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

He sheng updated ATLAS-3252:

Description: 
When importing HBase metadata, the following error is reported

 

 

2019-05-31 09:49:14,771 ERROR - [main:] ~ ImportHBaseEntities failed 
(HBaseBridge:188)
 java.io.IOException: java.lang.reflect.InvocationTargetException
     at 
org.apache.hadoop.hbase.client.ConnectionFactory.createConnection(ConnectionFactory.java:240)
     at 
org.apache.hadoop.hbase.client.ConnectionFactory.createConnection(ConnectionFactory.java:218)
     at 
org.apache.hadoop.hbase.client.ConnectionFactory.createConnection(ConnectionFactory.java:119)
     at 
org.apache.hadoop.hbase.client.HBaseAdmin.checkHBaseAvailable(HBaseAdmin.java:3006)
     at org.apache.atlas.hbase.bridge.HBaseBridge.(HBaseBridge.java:202)
     at org.apache.atlas.hbase.bridge.HBaseBridge.main(HBaseBridge.java:148)
 Caused by: java.lang.reflect.InvocationTargetException
     at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
     at 
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
     at 
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
     at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
     at 
org.apache.hadoop.hbase.client.ConnectionFactory.createConnection(ConnectionFactory.java:238)
     ... 5 more
 Caused by: java.lang.NoSuchFieldError: HBASE_CLIENT_PREFETCH
     at 
org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.(ConnectionManager.java:713)
     at 
org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.(ConnectionManager.java:652)
     ... 10 more

  

  was:
When importing HBase metadata, the following error is reported

 

 

2019-05-31 09:49:14,771 ERROR - [main:] ~ ImportHBaseEntities failed 
(HBaseBridge:188)
java.io.IOException: java.lang.reflect.InvocationTargetException
    at 
org.apache.hadoop.hbase.client.ConnectionFactory.createConnection(ConnectionFactory.java:240)
    at 
org.apache.hadoop.hbase.client.ConnectionFactory.createConnection(ConnectionFactory.java:218)
    at 
org.apache.hadoop.hbase.client.ConnectionFactory.createConnection(ConnectionFactory.java:119)
    at 
org.apache.hadoop.hbase.client.HBaseAdmin.checkHBaseAvailable(HBaseAdmin.java:3006)
    at org.apache.atlas.hbase.bridge.HBaseBridge.(HBaseBridge.java:202)
    at org.apache.atlas.hbase.bridge.HBaseBridge.main(HBaseBridge.java:148)
Caused by: java.lang.reflect.InvocationTargetException
    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
    at 
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
    at 
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
    at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
    at 
org.apache.hadoop.hbase.client.ConnectionFactory.createConnection(ConnectionFactory.java:238)
    ... 5 more
Caused by: java.lang.NoSuchFieldError: HBASE_CLIENT_PREFETCH
    at 
org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.(ConnectionManager.java:713)
    at 
org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.(ConnectionManager.java:652)
    ... 10 more
When
!moz-extension://8260c77a-a03d-49f1-8b6d-70c335e701e7/static/drowDown.svg!
!moz-extension://8260c77a-a03d-49f1-8b6d-70c335e701e7/static/sound.svg!
感叹词几时?; 什么时间?
定义
副词at what time.
连词at or during the time that.
 


> When importing HBase metadata, the following error is reported
> --
>
> Key: ATLAS-3252
> URL: https://issues.apache.org/jira/browse/ATLAS-3252
> Project: Atlas
>  Issue Type: Bug
>  Components:  atlas-core
>Affects Versions: 1.1.0
> Environment: CDH 5.7;
> Atlas version：1.1.0
>Reporter: He sheng
>Priority: Major
>
> When importing HBase metadata, the following error is reported
>  
>  
> 2019-05-31 09:49:14,771 ERROR - [main:] ~ ImportHBaseEntities failed 
> (HBaseBridge:188)
>  java.io.IOException: java.lang.reflect.InvocationTargetException
>      at 
> org.apache.hadoop.hbase.client.ConnectionFactory.createConnection(ConnectionFactory.java:240)
>      at 
> org.apache.hadoop.hbase.client.ConnectionFactory.createConnection(ConnectionFactory.java:218)
>      at 
> org.apache.hadoop.hbase.client.ConnectionFactory.createConnection(ConnectionFactory.java:119)
>      at 
> org.apache.hadoop.hbase.client.HBaseAdmin.checkHBaseAvailable(HBaseAdmin.java:3006)
>      at org.apache.atlas.hbase.bridge.HBaseBridge.(HBaseBridge.java:202)
>      at org.apache.atlas.hbase.bridge.HBaseBridge.main(HBaseBridge.java:148)
>  Caused by: java.lang.reflect.InvocationTargetException
>      

[jira] [Created] (ATLAS-3252) When importing HBase metadata, the following error is reported

2019-05-30 Thread He sheng (JIRA) 
He sheng created ATLAS-3252:
---

 Summary: When importing HBase metadata, the following error is 
reported
 Key: ATLAS-3252
 URL: https://issues.apache.org/jira/browse/ATLAS-3252
 Project: Atlas
  Issue Type: Bug
  Components:  atlas-core
Affects Versions: 1.1.0
 Environment: CDH 5.7;
Atlas version：1.1.0
Reporter: He sheng


When importing HBase metadata, the following error is reported

 

 

2019-05-31 09:49:14,771 ERROR - [main:] ~ ImportHBaseEntities failed 
(HBaseBridge:188)
java.io.IOException: java.lang.reflect.InvocationTargetException
    at 
org.apache.hadoop.hbase.client.ConnectionFactory.createConnection(ConnectionFactory.java:240)
    at 
org.apache.hadoop.hbase.client.ConnectionFactory.createConnection(ConnectionFactory.java:218)
    at 
org.apache.hadoop.hbase.client.ConnectionFactory.createConnection(ConnectionFactory.java:119)
    at 
org.apache.hadoop.hbase.client.HBaseAdmin.checkHBaseAvailable(HBaseAdmin.java:3006)
    at org.apache.atlas.hbase.bridge.HBaseBridge.(HBaseBridge.java:202)
    at org.apache.atlas.hbase.bridge.HBaseBridge.main(HBaseBridge.java:148)
Caused by: java.lang.reflect.InvocationTargetException
    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
    at 
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
    at 
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
    at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
    at 
org.apache.hadoop.hbase.client.ConnectionFactory.createConnection(ConnectionFactory.java:238)
    ... 5 more
Caused by: java.lang.NoSuchFieldError: HBASE_CLIENT_PREFETCH
    at 
org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.(ConnectionManager.java:713)
    at 
org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.(ConnectionManager.java:652)
    ... 10 more
When
!moz-extension://8260c77a-a03d-49f1-8b6d-70c335e701e7/static/drowDown.svg!
!moz-extension://8260c77a-a03d-49f1-8b6d-70c335e701e7/static/sound.svg!
感叹词几时?; 什么时间?
定义
副词at what time.
连词at or during the time that.
 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: Review Request 70719: ATLAS-3227:Resolved exception in index cleaning when bulk deleting.

2019-05-30 Thread Diego Marino Monetti 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/70719/#review215606
---


Ship it!




Inviala!

- Diego Marino Monetti


On Mag. 24, 2019, 7:52 p.m., Diego Marino Monetti wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/70719/
> ---
> 
> (Updated Mag. 24, 2019, 7:52 p.m.)
> 
> 
> Review request for atlas and Madhan Neethiraj.
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> Bulk delete does not delete the indexes properly
> 
> 
> Diffs
> -
> 
>   intg/src/main/java/org/apache/atlas/type/AtlasTypeRegistry.java c0ac7ad15 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/AtlasTypeDefGraphStore.java
>  2dac776ec 
> 
> 
> Diff: https://reviews.apache.org/r/70719/diff/1/
> 
> 
> Testing
> ---
> 
> There are some files in attachment in JIra to verify the bug and to test the 
> patch.
> 
> 
> Thanks,
> 
> Diego Marino Monetti
> 
>

Re: Review Request 70606: ATLAS-2901: Change classification name rule - Support multiple languages (unicode) and do not allow dot mark

2019-05-30 Thread Sridhar K 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/70606/#review215605
---


Ship it!




Ship It!

- Sridhar K


On May 21, 2019, 12:06 a.m., Yu-Hsin Shih wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/70606/
> ---
> 
> (Updated May 21, 2019, 12:06 a.m.)
> 
> 
> Review request for atlas, Ashutosh Mestry, Aadarsh Jajodia, Sridhar K, Le Ma, 
> Madhan Neethiraj, and Sarath Subramanian.
> 
> 
> Bugs: ATLAS-2901
> https://issues.apache.org/jira/browse/ATLAS-2901
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> 1. Allow Unicode in classification name and follow the original regex rule as 
> possible as we can.
> 2. Update the error message to reflect that the rule allows space.
> 
> 
> Diffs
> -
> 
>   intg/src/main/java/org/apache/atlas/AtlasErrorCode.java c04f561c1 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasClassificationDefStoreV2.java
>  dd020d183 
>   
> repository/src/test/java/org/apache/atlas/repository/store/graph/v2/AtlasClassificationDefStoreV2Test.java
>  PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/70606/diff/2/
> 
> 
> Testing
> ---
> 
> Manual tested by using Chinese characters in both Classification and its 
> attribute.
> Add unit test cases.
> 
> 
> File Attachments
> 
> 
> Classification def
>   
> https://reviews.apache.org/media/uploaded/files/2019/05/07/f629f9db-e858-4f11-a018-5d91aba0a286__Screen_Shot_2019-05-07_at_1.50.01_PM.png
> Entity Association
>   
> https://reviews.apache.org/media/uploaded/files/2019/05/07/d36894df-5acf-4227-b591-11029bf3f009__Screen_Shot_2019-05-07_at_1.49.44_PM.png
> 
> 
> Thanks,
> 
> Yu-Hsin Shih
> 
>

[jira] [Commented] (ATLAS-3226) Add QueryText for hive_table and hive_db for Impala integration

2019-05-30 Thread Na Li (JIRA) 


[ 
https://issues.apache.org/jira/browse/ATLAS-3226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16852362#comment-16852362
 ] 

Na Li commented on ATLAS-3226:
--

pre-commit test in 1152 failed at hive hook test. restart the test at 
https://builds.apache.org/view/A/view/Atlas/job/PreCommit-ATLAS-Build-Test/1153/console

> Add QueryText for hive_table and hive_db for Impala integration
> ---
>
> Key: ATLAS-3226
> URL: https://issues.apache.org/jira/browse/ATLAS-3226
> Project: Atlas
>  Issue Type: New Feature
>  Components:  atlas-core
>Reporter: Na Li
>Assignee: Na Li
>Priority: Major
> Attachments: ATLAS-3226.001.patch, ATLAS-3226.002.patch
>
>
> ATLAS-3197 introduced a new feature on Hive integration. It introduced new 
> model called Hive_table_ddl and hive_db_ddl which are used to track commands 
> that are executed either on a table entity or bd entity. DDL entities will be 
> added to a table/db entity through relationshipAttribute call HIVE_DB_TO_DDL 
> and hive_tb_to_ddl. Every time when a new command is executed, a new ddl 
> entity will be created, it will append to the existing ddlQueries field if a 
> table/db has already been created. Once a table/db is deleted, all these 
> relationship attributes will be deleted as well.
> We need to support this new feature on Impala integration.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (ATLAS-3251) Implement Patch to populate classification text for legacy data.

2019-05-30 Thread Sridhar (JIRA) 
Sridhar created ATLAS-3251:
--

 Summary: Implement Patch to populate classification text for 
legacy data.
 Key: ATLAS-3251
 URL: https://issues.apache.org/jira/browse/ATLAS-3251
 Project: Atlas
  Issue Type: Improvement
Reporter: Sridhar
Assignee: Sridhar






--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (ATLAS-2974) Docker build: Failed to execute goal org.apache.maven.plugins:maven-enforcer-plugin...

2019-05-30 Thread Xinran Tinney (JIRA) 


[ 
https://issues.apache.org/jira/browse/ATLAS-2974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16852129#comment-16852129
 ] 

Xinran Tinney commented on ATLAS-2974:
--

Hi, [~jonesn], I have tried to run the image I built, but it has a solr 
exception '''SolrException: Cannot connect to cluster at localhost:2181: 
cluster not found/not ready''', I was wondering if you have seen this before? 
Thanks!

> Docker build: Failed to execute goal 
> org.apache.maven.plugins:maven-enforcer-plugin...
> --
>
> Key: ATLAS-2974
> URL: https://issues.apache.org/jira/browse/ATLAS-2974
> Project: Atlas
>  Issue Type: Bug
>Reporter: Dmitry Goryunov
>Assignee: Xinran Tinney
>Priority: Major
> Attachments: tmp
>
>
> Version: commit 945de013f5466ffcf71d16c6ec1772fdee83e2f6 (Also reproducable 
> with 1.1.0-rc2)
>  
> Runnign command:
> {code:java}
> docker build -t atlas_docker .
> {code}
> Breaks at some point with:
> {code:java}
> [ERROR] Failed to execute goal 
> org.apache.maven.plugins:maven-enforcer-plugin:3.0.0-M1:enforce 
> (enforce-versions) on project apache-atlas: Some Enforcer rules have failed. 
> Look above for specific messages explaining why the rule failed. -> [Help 1]
> [ERROR]
> [ERROR] To see the full stack trace of the errors, re-run Maven with the -e 
> switch.
> [ERROR] Re-run Maven using the -X switch to enable full debug logging.
> [ERROR]
> [ERROR] For more information about the errors and possible solutions, please 
> read the following articles:
> [ERROR] [Help 1] 
> http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
> [ERROR]
> [ERROR] After correcting the problems, you can resume the build with the 
> command
> [ERROR]   mvn  -rf :apache-atlas
> The command '/bin/sh -c mvn clean install -DskipTests 
> -Pdist,embedded-hbase-solr -f ./atlas/pom.xml' returned a non-zero code: 1
> {code}
> The complete logs are attached.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (ATLAS-3226) Add QueryText for hive_table and hive_db for Impala integration

2019-05-30 Thread Na Li (JIRA) 


[ 
https://issues.apache.org/jira/browse/ATLAS-3226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16852086#comment-16852086
 ] 

Na Li commented on ATLAS-3226:
--

patch 2 pre-commit test 
https://builds.apache.org/view/A/view/Atlas/job/PreCommit-ATLAS-Build-Test/1152/console

> Add QueryText for hive_table and hive_db for Impala integration
> ---
>
> Key: ATLAS-3226
> URL: https://issues.apache.org/jira/browse/ATLAS-3226
> Project: Atlas
>  Issue Type: New Feature
>  Components:  atlas-core
>Reporter: Na Li
>Assignee: Na Li
>Priority: Major
> Attachments: ATLAS-3226.001.patch, ATLAS-3226.002.patch
>
>
> ATLAS-3197 introduced a new feature on Hive integration. It introduced new 
> model called Hive_table_ddl and hive_db_ddl which are used to track commands 
> that are executed either on a table entity or bd entity. DDL entities will be 
> added to a table/db entity through relationshipAttribute call HIVE_DB_TO_DDL 
> and hive_tb_to_ddl. Every time when a new command is executed, a new ddl 
> entity will be created, it will append to the existing ddlQueries field if a 
> table/db has already been created. Once a table/db is deleted, all these 
> relationship attributes will be deleted as well.
> We need to support this new feature on Impala integration.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (ATLAS-3226) Add QueryText for hive_table and hive_db for Impala integration

2019-05-30 Thread Na Li (JIRA) 


 [ 
https://issues.apache.org/jira/browse/ATLAS-3226?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Na Li updated ATLAS-3226:
-
Attachment: ATLAS-3226.002.patch

> Add QueryText for hive_table and hive_db for Impala integration
> ---
>
> Key: ATLAS-3226
> URL: https://issues.apache.org/jira/browse/ATLAS-3226
> Project: Atlas
>  Issue Type: New Feature
>  Components:  atlas-core
>Reporter: Na Li
>Assignee: Na Li
>Priority: Major
> Attachments: ATLAS-3226.001.patch, ATLAS-3226.002.patch
>
>
> ATLAS-3197 introduced a new feature on Hive integration. It introduced new 
> model called Hive_table_ddl and hive_db_ddl which are used to track commands 
> that are executed either on a table entity or bd entity. DDL entities will be 
> added to a table/db entity through relationshipAttribute call HIVE_DB_TO_DDL 
> and hive_tb_to_ddl. Every time when a new command is executed, a new ddl 
> entity will be created, it will append to the existing ddlQueries field if a 
> table/db has already been created. Once a table/db is deleted, all these 
> relationship attributes will be deleted as well.
> We need to support this new feature on Impala integration.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: Review Request 70759: ATLAS-3226: Add QueryText for hive_table and hive_db for Impala integration

2019-05-30 Thread Na Li via Review Board 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/70759/
---

(Updated May 30, 2019, 5:06 p.m.)


Review request for atlas, madhan and Sarath Subramanian.


Repository: atlas


Description
---

ATLAS-3197 introduced a new feature on Hive integration. It introduced new 
model called Hive_table_ddl and hive_db_ddl which are used to track commands 
that are executed either on a table entity or bd entity. DDL entities will be 
added to a table/db entity through relationshipAttribute call HIVE_DB_TO_DDL 
and hive_tb_to_ddl. Every time when a new command is executed, a new ddl entity 
will be created, it will append to the existing ddlQueries field if a table/db 
has already been created. Once a table/db is deleted, all these relationship 
attributes will be deleted as well.

We need to support this new feature on Impala integration.


Diffs (updated)
-

  
addons/impala-bridge/src/main/java/org/apache/atlas/impala/hook/events/BaseImpalaEvent.java
 d241b6a 
  
addons/impala-bridge/src/main/java/org/apache/atlas/impala/hook/events/CreateImpalaProcess.java
 b7506a4 
  
addons/impala-bridge/src/main/java/org/apache/atlas/impala/model/ImpalaDataType.java
 10ce448 
  
addons/impala-bridge/src/test/java/org/apache/atlas/impala/ImpalaLineageITBase.java
 b8cbf6b 
  
addons/impala-bridge/src/test/java/org/apache/atlas/impala/ImpalaLineageToolIT.java
 7f9a534 
  
addons/impala-bridge/src/test/java/org/apache/atlas/impala/hook/ImpalaLineageHookIT.java
 6156208 
  addons/models/1000-Hadoop/1090-impala_model.json 24eb5db 


Diff: https://reviews.apache.org/r/70759/diff/2/

Changes: https://reviews.apache.org/r/70759/diff/1-2/


Testing
---

add check for ddl entities in integration tests


Thanks,

Na Li

[jira] [Updated] (ATLAS-3244) UI : Allows user to search the entities from any page (quick-search)

2019-05-30 Thread Keval Bhatt (JIRA) 


 [ 
https://issues.apache.org/jira/browse/ATLAS-3244?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Keval Bhatt updated ATLAS-3244:
---
Attachment: ATLAS-3244.1.patch

> UI : Allows user to search the entities from any page (quick-search)
> 
>
> Key: ATLAS-3244
> URL: https://issues.apache.org/jira/browse/ATLAS-3244
> Project: Atlas
>  Issue Type: Improvement
>Reporter: Keval Bhatt
>Assignee: Keval Bhatt
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: ATLAS-3244.1.patch, ATLAS-3244.patch, Quick-search.jpg
>
>
> Allows user to search the entities from any page.
> *The quick search bar is added inside the header*
> !Quick-search.jpg|width=600,height=300!



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[VOTE] Release Apache Atlas version 1.2.0 - rc2

2019-05-30 Thread Nixon Rodrigues 
Atlas team,

Apache Atlas 1.2.0 release candidate #2 is now available for a vote within
dev community.

Following commits went into branch-1.0 since the last release candidate
(rc1):

ATLAS-3250:- Update jackson version to 2.9.9 in pom.xml.

Links to the release artifacts are given below. Please review and vote.

The vote will be open for at least 72 hours or until necessary votes are
reached.
  [ ] +1 approve
  [ ] +0 no opinion
  [ ] -1 disapprove (and reason why)


Thanks,
Nixon Rodrigues


List of issues addressed in this release:
https://issues.apache.org/jira/browse/ATLAS-3201?jql=project%20%3D%20ATLAS%20AND%20resolution%20%3D%20Fixed%20AND%20fixVersion%20%3D%201.2.0%20ORDER%20BY%20key%20DESC

Git tag for the release:
https://github.com/apache/atlas/releases/tag/release-1.2.0-rc2

Sources for the release:
https://dist.apache.org/repos/dist/dev/atlas/1.2.0-rc2/apache-atlas-1.2.0-sources.tar.gz

Source release verification:
  PGP Signature:
https://dist.apache.org/repos/dist/dev/atlas/1.2.0-rc2/apache-atlas-1.2.0-sources.tar.gz.asc
  MD5 Hash:
https://dist.apache.org/repos/dist/dev/atlas/1.2.0-rc2/apache-atlas-1.2.0-sources.tar.gz.md5
  SHA512 Hash:
https://dist.apache.org/repos/dist/dev/atlas/1.2.0-rc2/apache-atlas-1.2.0-sources.tar.gz.sha512

Keys to verify the signature of the release artifacts are available at:
https://dist.apache.org/repos/dist/dev/atlas/KEYS

[jira] [Commented] (ATLAS-3250) Update jackson version to 2.9.9 in pom.xml

2019-05-30 Thread ASF subversion and git services (JIRA) 


[ 
https://issues.apache.org/jira/browse/ATLAS-3250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16851789#comment-16851789
 ] 

ASF subversion and git services commented on ATLAS-3250:


Commit eb46c0230dec81f9aa8a91cb02e7d37ac73ca1b0 in atlas's branch 
refs/heads/branch-1.0 from Nixon Rodrigues
[ https://gitbox.apache.org/repos/asf?p=atlas.git;h=eb46c02 ]

ATLAS-3250-Update jackson version to 2.9.9 in pom.xml


> Update jackson version to 2.9.9 in pom.xml
> --
>
> Key: ATLAS-3250
> URL: https://issues.apache.org/jira/browse/ATLAS-3250
> Project: Atlas
>  Issue Type: Bug
>Reporter: Nixon Rodrigues
>Assignee: Nixon Rodrigues
>Priority: Major
> Attachments: 
> 0001-ATLAS-3250-Update-jackson-version-to-2.9.9-in-pom.xm.patch
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (ATLAS-3250) Update jackson version to 2.9.9 in pom.xml

2019-05-30 Thread ASF subversion and git services (JIRA) 


[ 
https://issues.apache.org/jira/browse/ATLAS-3250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16851787#comment-16851787
 ] 

ASF subversion and git services commented on ATLAS-3250:


Commit aa4a04d96a813fd753a8a793be878a6d5d6893a4 in atlas's branch 
refs/heads/branch-2.0 from Nixon Rodrigues
[ https://gitbox.apache.org/repos/asf?p=atlas.git;h=aa4a04d ]

ATLAS-3250-Update jackson version to 2.9.9 in pom.xml


> Update jackson version to 2.9.9 in pom.xml
> --
>
> Key: ATLAS-3250
> URL: https://issues.apache.org/jira/browse/ATLAS-3250
> Project: Atlas
>  Issue Type: Bug
>Reporter: Nixon Rodrigues
>Assignee: Nixon Rodrigues
>Priority: Major
> Attachments: 
> 0001-ATLAS-3250-Update-jackson-version-to-2.9.9-in-pom.xm.patch
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (ATLAS-3250) Update jackson version to 2.9.9 in pom.xml

2019-05-30 Thread ASF subversion and git services (JIRA) 


[ 
https://issues.apache.org/jira/browse/ATLAS-3250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16851785#comment-16851785
 ] 

ASF subversion and git services commented on ATLAS-3250:


Commit 4b6380feb454b0cc2f0186820b6f10f42b47e119 in atlas's branch 
refs/heads/master from Nixon Rodrigues
[ https://gitbox.apache.org/repos/asf?p=atlas.git;h=4b6380f ]

ATLAS-3250-Update jackson version to 2.9.9 in pom.xml


> Update jackson version to 2.9.9 in pom.xml
> --
>
> Key: ATLAS-3250
> URL: https://issues.apache.org/jira/browse/ATLAS-3250
> Project: Atlas
>  Issue Type: Bug
>Reporter: Nixon Rodrigues
>Assignee: Nixon Rodrigues
>Priority: Major
> Attachments: 
> 0001-ATLAS-3250-Update-jackson-version-to-2.9.9-in-pom.xm.patch
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (ATLAS-3250) Update jackson version to 2.9.9 in pom.xml

2019-05-30 Thread Nixon Rodrigues (JIRA) 


[ 
https://issues.apache.org/jira/browse/ATLAS-3250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16851774#comment-16851774
 ] 

Nixon Rodrigues commented on ATLAS-3250:


Tested Atlas build and UI with this patch

Precommit passed successfully 

[https://builds.apache.org/job/PreCommit-ATLAS-Build-Test/1151/console]

 

> Update jackson version to 2.9.9 in pom.xml
> --
>
> Key: ATLAS-3250
> URL: https://issues.apache.org/jira/browse/ATLAS-3250
> Project: Atlas
>  Issue Type: Bug
>Reporter: Nixon Rodrigues
>Assignee: Nixon Rodrigues
>Priority: Major
> Attachments: 
> 0001-ATLAS-3250-Update-jackson-version-to-2.9.9-in-pom.xm.patch
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (ATLAS-3250) Update jackson version to 2.9.9 in pom.xml

2019-05-30 Thread Keval Bhatt (JIRA) 


[ 
https://issues.apache.org/jira/browse/ATLAS-3250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16851776#comment-16851776
 ] 

Keval Bhatt commented on ATLAS-3250:


+1 for the patch, Thanks [~nixonrodrigues]

> Update jackson version to 2.9.9 in pom.xml
> --
>
> Key: ATLAS-3250
> URL: https://issues.apache.org/jira/browse/ATLAS-3250
> Project: Atlas
>  Issue Type: Bug
>Reporter: Nixon Rodrigues
>Assignee: Nixon Rodrigues
>Priority: Major
> Attachments: 
> 0001-ATLAS-3250-Update-jackson-version-to-2.9.9-in-pom.xm.patch
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (ATLAS-3250) Update jackson version to 2.9.9 in pom.xml

2019-05-30 Thread Nixon Rodrigues (JIRA) 


 [ 
https://issues.apache.org/jira/browse/ATLAS-3250?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nixon Rodrigues updated ATLAS-3250:
---
Attachment: 0001-ATLAS-3250-Update-jackson-version-to-2.9.9-in-pom.xm.patch

> Update jackson version to 2.9.9 in pom.xml
> --
>
> Key: ATLAS-3250
> URL: https://issues.apache.org/jira/browse/ATLAS-3250
> Project: Atlas
>  Issue Type: Bug
>Reporter: Nixon Rodrigues
>Assignee: Nixon Rodrigues
>Priority: Major
> Attachments: 
> 0001-ATLAS-3250-Update-jackson-version-to-2.9.9-in-pom.xm.patch
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] [atlas] ZepHakase22 commented on issue #54: ATLAS-3219: New REST APIs for serviceType.

2019-05-30 Thread GitBox 
ZepHakase22 commented on issue #54: ATLAS-3219: New REST APIs for serviceType.
URL: https://github.com/apache/atlas/pull/54#issuecomment-497301805
 
 
   @mneethiraj - I looked more closely at the code and we could: 1) limit the 
rest api for the four types one for the post, one for the put, one for the 
update and one for the cache and then use the predicates.  2) limit to a single 
cache instead of having a cache for the guid one for the name and one for the 
service type.  If you agree I could start this task.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

Re: [VOTE] Release Apache Atlas version 1.2.0 - rc1

2019-05-30 Thread Binit Gutka 
+1 Release Apache Atlas 1.2.0 release candidate #1

- Verified PGP signature.
- Built successfully from the source tar file
- Ran quick start and validated basic functionality.
- Tested Atlas UI packaging in embedded-hbase mode.
- Performed few basic and advanced Search.
- Lineage Export functionality tested.
- Lineage Pan, Zoom, Filter, Search and settings functionality tested.

Thanks.

On Thu, May 30, 2019 at 3:40 AM Nixon Rodrigues <
nixon.rodrig...@freestoneinfotech.com> wrote:

> Atlas team,
>
> Apache Atlas 1.2.0 release candidate #1 is now available for a vote within
> dev community.
>
> Following commits went into branch-1.0 since the last release candidate
> (rc0):
>
> ATLAS-3240 :- Update 'http' urls to 'https' in maven settings.
> ATLAS-3213 :- UI - Png export not working in Safari Browser.
> ATLAS-3218 :- UI - Entity detail page columns sorting empty value check.
> ATLAS-3196 :- UI - Lineage support for IE.
> ATLAS-3190 :- UI - Lineage File changes & fixes.
>
> Links to the release artifacts are given below. Please review and vote.
>
> The vote will be open for at least 72 hours or until necessary votes are
> reached.
>   [ ] +1 approve
>   [ ] +0 no opinion
>   [ ] -1 disapprove (and reason why)
>
>
> Thanks,
> Nixon Rodrigues
>
>
> List of issues addressed in this release:
>
> https://issues.apache.org/jira/browse/ATLAS-3201?jql=project%20%3D%20ATLAS%20AND%20resolution%20%3D%20Fixed%20AND%20fixVersion%20%3D%201.2.0%20ORDER%20BY%20key%20DESC
>
> Git tag for the release:
> https://github.com/apache/atlas/releases/tag/release-1.2.0-rc1
>
> Sources for the release:
>
> https://dist.apache.org/repos/dist/dev/atlas/1.2.0-rc1/apache-atlas-1.2.0-sources.tar.gz
>
> Source release verification:
>   PGP Signature:
>
> https://dist.apache.org/repos/dist/dev/atlas/1.2.0-rc1/apache-atlas-1.2.0-sources.tar.gz.asc
>   MD5 Hash:
>
> https://dist.apache.org/repos/dist/dev/atlas/1.2.0-rc1/apache-atlas-1.2.0-sources.tar.gz.md5
>   SHA512 Hash:
>
> https://dist.apache.org/repos/dist/dev/atlas/1.2.0-rc1/apache-atlas-1.2.0-sources.tar.gz.sha512
>
> Keys to verify the signature of the release artifacts are available at:
> https://dist.apache.org/repos/dist/dev/atlas/KEYS
>


-- 
*Thanks,*
*Binit.*

[jira] [Created] (ATLAS-3250) Update jackson version to 2.9.9 in pom.xml

2019-05-30 Thread Nixon Rodrigues (JIRA) 
Nixon Rodrigues created ATLAS-3250:
--

 Summary: Update jackson version to 2.9.9 in pom.xml
 Key: ATLAS-3250
 URL: https://issues.apache.org/jira/browse/ATLAS-3250
 Project: Atlas
  Issue Type: Bug
Reporter: Nixon Rodrigues
Assignee: Nixon Rodrigues






--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Assigned] (ATLAS-3249) UI: "Show more"/"Show less" button for columns with multiple entries within a cell

2019-05-30 Thread Binit Gutka (JIRA) 


 [ 
https://issues.apache.org/jira/browse/ATLAS-3249?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Binit Gutka reassigned ATLAS-3249:
--

Assignee: Binit Gutka

> UI: "Show more"/"Show less" button for columns with multiple entries within a 
> cell
> --
>
> Key: ATLAS-3249
> URL: https://issues.apache.org/jira/browse/ATLAS-3249
> Project: Atlas
>  Issue Type: Improvement
>Reporter: Rahul Kurup
>Assignee: Binit Gutka
>Priority: Minor
> Attachments: showmoreshowless.png
>
>
> There should be a "Show more/Show less" button in a scenario where there are 
> a lot of entries within a cell.
> For example, in the attached screenshot below, if the user searches for a 
> hive_table and selects "Columns" as one of the displayed parameters, you can 
> see that one search result may have multiple entries within the "Columns" 
> column that increases the width of the search result and disrupts the UI 
> alignment.
> A Show More/Show Less button that expands or compresses the displayed entries 
> of the "Columns" table similar to the Classification would fix this alignment 
> issue. !showmoreshowless.png!



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Entity limits in Atlas

2019-05-30 Thread Rempter, A. (Adam) 
Hey Guys,

I was wondering if you have any best practices / limits for entity that is 
being created in atlas?
I am asking since we noticed some issues related to creating (or processing) 
entities.

For instance, we use “aws_s3_pseudo_dir” type from 3020-aws_s3_typedefs.json.
It has following property:

"name":"s3Objects",
"typeName":"array"

Now in AWS buckets you can have thousands of objects. This causes that 
s3Objects array grows quite quickly,
causing aws_s3_pseudo_dir entity Json to rich easly few MBs.

Then we start seeing problems like:

-  UI is dying on displaying entity properties or lineage

-  audit record too long: entityType=aws_s3_pseudo_dir, 
guid=24398271-6ba0-4db5-adfa-38e432dc55ce, size=1053931; maxSize=1048576. 
entity attribute values not stored in audit (EntityAuditListenerV2:234)

-  some errors with write to HBase (java.lang.IllegalArgumentException: 
KeyValue size too large, as workaround we set hbase.client.keyvalue.maxsize 
param to 0)

-  kafka consumer errors (we can of course set some parameters on 
consumer, but I think it is just workaround)

…

Exception in NotificationHookConsumer (NotificationHookConsumer:332)

org.apache.kafka.clients.consumer.CommitFailedException: Commit cannot be 
completed since the group has already rebalanced and assigned the partitions to 
another member. This means that the time between subsequen

t calls to poll() was longer than the configured max.poll.interval.ms, which 
typically implies that the poll loop is spending too much time message 
processing. You can address this either by increasing the sessio

n timeout or by reducing the maximum size of batches returned in poll() with 
max.poll.records.
…


We can of course skip creating of s3objects, but on the other hand it is quite 
usefull to have such info in pseudodirectory…


Thanks in advance for suggestions how to handle this…


Adam Rempter

Big Data Squad
[https://ispintranet.pl.ing.net/sites/default/files/content/logo_stopka2.png]
ul. Konduktorska 35, 40-155 Katowice
Poland


ING Business Shared Services B.V. z siedzibą w Amsterdamie, Holandia, VAT PL 
526-319-58-54, działająca w Polsce w formie oddziału, pod firmą ING Business 
Shared Services B.V. spółka z ograniczoną odpowiedzialnością Oddział w Polsce z 
siedzibą w Katowicach, ul. Konduktorska 35, 40-155 Katowice, NIP: 2050005130, 
wpisana do rejestru przedsiębiorców Krajowego Rejestru Sądowego prowadzonego 
przez Sąd Rejonowy Katowice-Wschód w Katowicach, VIII Wydział Gospodarczy 
Krajowego Rejestru Sądowego pod numerem KRS 702305.