[jira] [Commented] (ATLAS-4295) UI: The short/long description of term & category doesnt get updated immediately when made blank

2021-06-08 Thread Rahul Kurup (Jira) 


[ 
https://issues.apache.org/jira/browse/ATLAS-4295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17359809#comment-17359809
 ] 

Rahul Kurup commented on ATLAS-4295:


+1

> UI: The short/long description of term & category doesnt get updated 
> immediately when made blank
> 
>
> Key: ATLAS-4295
> URL: https://issues.apache.org/jira/browse/ATLAS-4295
> Project: Atlas
>  Issue Type: Bug
>Reporter: Rahul Kurup
>Assignee: Prasad P. Pawar
>Priority: Minor
> Attachments: 
> 0001-ATLAS-4295-UI-UI-The-short-long-description-of-term-.patch
>
>
> When one empties the the short and long description fields of term & category 
> and saves,
> the UI doesn't get reflected immediately. But on refresh, it gets reflected.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (ATLAS-4326) Atlas UI: Multiple continuous space is accepted on Short description for Classifications, Gloassarys, Categorys & Terms .

2021-06-08 Thread Rahul Kurup (Jira) 


[ 
https://issues.apache.org/jira/browse/ATLAS-4326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17359811#comment-17359811
 ] 

Rahul Kurup commented on ATLAS-4326:


+1

> Atlas UI: Multiple continuous space is accepted on Short description for 
> Classifications, Gloassarys, Categorys & Terms .
> -
>
> Key: ATLAS-4326
> URL: https://issues.apache.org/jira/browse/ATLAS-4326
> Project: Atlas
>  Issue Type: Bug
>  Components: atlas-webui
>Reporter: Prasad P. Pawar
>Assignee: Prasad P. Pawar
>Priority: Minor
> Attachments: 
> 0001-ATLAS-4326-1-Atlas-UI-Multiple-continuous-space-is-a.patch, 
> 0001-ATLAS-4326-2-Atlas-UI-Multiple-continuous-space-is-a.patch
>
>
> Add multiple continuous spaces in the short description field,  this value is 
> accepted on UI by clicking the Save/Update button.
> After updating the values, the Detail page doesn't show the entered value of 
> Short description files. eg: "   This      is  Term1   "



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (ATLAS-4328) Add flink favicon

2021-06-08 Thread Ashutosh Mestry (Jira) 


[ 
https://issues.apache.org/jira/browse/ATLAS-4328?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17359766#comment-17359766
 ] 

Ashutosh Mestry commented on ATLAS-4328:


[~jjyeh] Thanks! +1 from me!

> Add flink favicon
> -
>
> Key: ATLAS-4328
> URL: https://issues.apache.org/jira/browse/ATLAS-4328
> Project: Atlas
>  Issue Type: Task
>  Components: atlas-intg
>Reporter: Josh Yeh
>Assignee: Prasad P. Pawar
>Priority: Trivial
> Attachments: flink.png, flink_process.png, 
> image-2021-06-07-08-41-37-559.png
>
>
> While testing ATLAS-3812, I noticed flink_application and flink_process icons 
> are missing. File this Jira to track.
> !image-2021-06-07-08-41-37-559.png!



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Comment Edited] (ATLAS-4328) Add flink favicon

2021-06-08 Thread Ashutosh Mestry (Jira) 


[ 
https://issues.apache.org/jira/browse/ATLAS-4328?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17359766#comment-17359766
 ] 

Ashutosh Mestry edited comment on ATLAS-4328 at 6/9/21, 5:34 AM:
-

[~jjyeh] Thanks! +1 from me!

[~prasadpp13] Kindly check if color scheme is consistent with rest of the icons.


was (Author: ashutoshm):
[~jjyeh] Thanks! +1 from me!

> Add flink favicon
> -
>
> Key: ATLAS-4328
> URL: https://issues.apache.org/jira/browse/ATLAS-4328
> Project: Atlas
>  Issue Type: Task
>  Components: atlas-intg
>Reporter: Josh Yeh
>Assignee: Prasad P. Pawar
>Priority: Trivial
> Attachments: flink.png, flink_process.png, 
> image-2021-06-07-08-41-37-559.png
>
>
> While testing ATLAS-3812, I noticed flink_application and flink_process icons 
> are missing. File this Jira to track.
> !image-2021-06-07-08-41-37-559.png!



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: Review Request 73404: ATLAS-4324: FS entity created for load data inpath is created as shell entity - Part 2

2021-06-08 Thread Ashutosh Mestry via Review Board 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/73404/
---

(Updated June 8, 2021, 11:07 p.m.)


Review request for atlas, Mandar Ambawane, Radhika Kundam, and Sarath 
Subramanian.


Changes
---

Updates include: Added screen shot from functional testing.


Bugs: ATLAS-4324
https://issues.apache.org/jira/browse/ATLAS-4324


Repository: atlas


Description
---

**Approach**
Updates:
- Simplified approach.
- Maintained abstraction.
- Updated unit tests.


Diffs
-

  addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/HiveHook.java 
94ef22577 
  
addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/utils/ActiveEntityFilter.java
 737c637bf 
  
addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/utils/HiveDDLEntityFilter.java
 9163c4709 
  
addons/hive-bridge/src/test/java/org/apache/atlas/hive/hook/utils/ActiveEntityFilterTest.java
 4dde1dc2f 
  addons/hive-bridge/src/test/resources/json/hs2-create-db-v2.json 881ee105e 
  
addons/hive-bridge/src/test/resources/json/hs2-create-db-with-no-pathentities-to-retain-v2.json
 42553b5cf 
  
addons/hive-bridge/src/test/resources/json/hs2-create-db-with-no-pathentities-to-retain.json
 a5b810f72 
  addons/hive-bridge/src/test/resources/json/hs2-create-process-v2.json 
8e55b7282 
  addons/hive-bridge/src/test/resources/json/hs2-create-table-v2.json 801918e04 
  addons/hive-bridge/src/test/resources/json/hs2-load-inpath-v2.json dd31aa073 
  
addons/hive-bridge/src/test/resources/json/hs2-load-inpath-with-no-pathentities-to-retain-v2.json
 7f90d1986 
  
addons/hive-bridge/src/test/resources/json/hs2-load-inpath-with-no-pathentities-to-retain.json
 dd31aa073 
  addons/hive-bridge/src/test/resources/json/hs2-load-inpath.json dd31aa073 
  addons/hive-bridge/src/test/resources/json/hs2-table-rename-v2.json f2b2bf7a8 


Diff: https://reviews.apache.org/r/73404/diff/2/


Testing
---

**Unit tests**
- Additional unit test.

**PC Build**
https://ci-builds.apache.org/job/Atlas/job/PreCommit-ATLAS-Build-Test/617/


File Attachments (updated)


Screen Shot 2021-06-08 at 4.05.26 PM.png
  
https://reviews.apache.org/media/uploaded/files/2021/06/08/b15b33fc-007d-4ee3-8862-64750af16432__Screen_Shot_2021-06-08_at_4.05.26_PM.png


Thanks,

Ashutosh Mestry

Re: Review Request 73409: ATLAS-4330: Add Kafka topics lag information on metrics and log

2021-06-08 Thread Ashutosh Mestry via Review Board 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/73409/#review223122
---



Can you please add log output?

- Ashutosh Mestry


On June 7, 2021, 11:12 p.m., Sarath Subramanian wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/73409/
> ---
> 
> (Updated June 7, 2021, 11:12 p.m.)
> 
> 
> Review request for atlas, Ashutosh Mestry, Deep Singh, Jayendra Parab, Madhan 
> Neethiraj, Nikhil Bonte, Nixon Rodrigues, Radhika Kundam, and Sidharth Mishra.
> 
> 
> Bugs: ATLAS-4330
> https://issues.apache.org/jira/browse/ATLAS-4330
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> In metrics page, in addition to other Notification details. Lag information 
> of kafka consumer topics will be useful to check the current backed up 
> messages in Atlas: 
> 
> The solution involves using the kafka admin client APIs to get log end offset 
> of a topic for the consumer ('atlas')
> 
> Config property:
> 
> atlas.notification.consumer.topic.report.frequency.seconds=60 (default value 
> 3600secs or 1 hour)
> 
> 
> Diffs
> -
> 
>   common/src/main/java/org/apache/atlas/utils/KafkaUtils.java 167442259 
>   
> dashboardv2/public/js/templates/site/Statistics_Topic_Offset_table_tmpl.html 
> b4c8d8677 
>   dashboardv2/public/js/views/site/Statistics.js 32a66691c 
>   
> dashboardv3/public/js/templates/site/Statistics_Topic_Offset_table_tmpl.html 
> b4c8d8677 
>   dashboardv3/public/js/utils/Enums.js ee80313c2 
>   dashboardv3/public/js/views/site/Statistics.js 9eb714d45 
>   notification/src/main/java/org/apache/atlas/kafka/AtlasKafkaConsumer.java 
> 96dc5856a 
>   notification/src/main/java/org/apache/atlas/kafka/AtlasKafkaMessage.java 
> af3727df4 
>   
> notification/src/main/java/org/apache/atlas/notification/NotificationConsumer.java
>  1fb9f9989 
>   
> notification/src/test/java/org/apache/atlas/notification/AbstractNotificationConsumerTest.java
>  1b486e528 
>   repository/src/main/java/org/apache/atlas/util/AtlasMetricsUtil.java 
> beb90e67b 
>   repository/src/test/java/org/apache/atlas/services/MetricsServiceTest.java 
> bea8eb8b5 
>   
> webapp/src/main/java/org/apache/atlas/notification/NotificationHookConsumer.java
>  5643af95c 
> 
> 
> Diff: https://reviews.apache.org/r/73409/diff/1/
> 
> 
> Testing
> ---
> 
> Precommit: 
> https://ci-builds.apache.org/job/Atlas/job/PreCommit-ATLAS-Build-Test/624/console
> 
> Manually verified lag information is updated correctly on metrics page and 
> periodically printed on log
> 
> 
> Thanks,
> 
> Sarath Subramanian
> 
>

Re: Review Request 73409: ATLAS-4330: Add Kafka topics lag information on metrics and log

2021-06-08 Thread Madhan Neethiraj 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/73409/#review223121
---




notification/src/main/java/org/apache/atlas/kafka/AtlasKafkaConsumer.java
Line 137 (original), 138 (patched)


Would this result in an additional call to Kafka broker for each message, 
to retrieve endOffset? If yes, consider tracking of endOffset to metrics module 
where the offset will be retrieved only on demand.

Rest of the chages look good! This will be a very useful addition.


- Madhan Neethiraj


On June 7, 2021, 11:12 p.m., Sarath Subramanian wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/73409/
> ---
> 
> (Updated June 7, 2021, 11:12 p.m.)
> 
> 
> Review request for atlas, Ashutosh Mestry, Deep Singh, Jayendra Parab, Madhan 
> Neethiraj, Nikhil Bonte, Nixon Rodrigues, Radhika Kundam, and Sidharth Mishra.
> 
> 
> Bugs: ATLAS-4330
> https://issues.apache.org/jira/browse/ATLAS-4330
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> In metrics page, in addition to other Notification details. Lag information 
> of kafka consumer topics will be useful to check the current backed up 
> messages in Atlas: 
> 
> The solution involves using the kafka admin client APIs to get log end offset 
> of a topic for the consumer ('atlas')
> 
> Config property:
> 
> atlas.notification.consumer.topic.report.frequency.seconds=60 (default value 
> 3600secs or 1 hour)
> 
> 
> Diffs
> -
> 
>   common/src/main/java/org/apache/atlas/utils/KafkaUtils.java 167442259 
>   
> dashboardv2/public/js/templates/site/Statistics_Topic_Offset_table_tmpl.html 
> b4c8d8677 
>   dashboardv2/public/js/views/site/Statistics.js 32a66691c 
>   
> dashboardv3/public/js/templates/site/Statistics_Topic_Offset_table_tmpl.html 
> b4c8d8677 
>   dashboardv3/public/js/utils/Enums.js ee80313c2 
>   dashboardv3/public/js/views/site/Statistics.js 9eb714d45 
>   notification/src/main/java/org/apache/atlas/kafka/AtlasKafkaConsumer.java 
> 96dc5856a 
>   notification/src/main/java/org/apache/atlas/kafka/AtlasKafkaMessage.java 
> af3727df4 
>   
> notification/src/main/java/org/apache/atlas/notification/NotificationConsumer.java
>  1fb9f9989 
>   
> notification/src/test/java/org/apache/atlas/notification/AbstractNotificationConsumerTest.java
>  1b486e528 
>   repository/src/main/java/org/apache/atlas/util/AtlasMetricsUtil.java 
> beb90e67b 
>   repository/src/test/java/org/apache/atlas/services/MetricsServiceTest.java 
> bea8eb8b5 
>   
> webapp/src/main/java/org/apache/atlas/notification/NotificationHookConsumer.java
>  5643af95c 
> 
> 
> Diff: https://reviews.apache.org/r/73409/diff/1/
> 
> 
> Testing
> ---
> 
> Precommit: 
> https://ci-builds.apache.org/job/Atlas/job/PreCommit-ATLAS-Build-Test/624/console
> 
> Manually verified lag information is updated correctly on metrics page and 
> periodically printed on log
> 
> 
> Thanks,
> 
> Sarath Subramanian
> 
>

[jira] [Updated] (ATLAS-4328) Add flink favicon

2021-06-08 Thread Josh Yeh (Jira) 


 [ 
https://issues.apache.org/jira/browse/ATLAS-4328?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Josh Yeh updated ATLAS-4328:

Attachment: flink.png
flink_process.png

> Add flink favicon
> -
>
> Key: ATLAS-4328
> URL: https://issues.apache.org/jira/browse/ATLAS-4328
> Project: Atlas
>  Issue Type: Task
>  Components: atlas-intg
>Reporter: Josh Yeh
>Assignee: Prasad P. Pawar
>Priority: Trivial
> Attachments: flink.png, flink_process.png, 
> image-2021-06-07-08-41-37-559.png
>
>
> While testing ATLAS-3812, I noticed flink_application and flink_process icons 
> are missing. File this Jira to track.
> !image-2021-06-07-08-41-37-559.png!



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (ATLAS-4328) Add flink favicon

2021-06-08 Thread Josh Yeh (Jira) 


[ 
https://issues.apache.org/jira/browse/ATLAS-4328?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17359440#comment-17359440
 ] 

Josh Yeh commented on ATLAS-4328:
-

[~prasadpp13], please feel free to use the attached favicons.

> Add flink favicon
> -
>
> Key: ATLAS-4328
> URL: https://issues.apache.org/jira/browse/ATLAS-4328
> Project: Atlas
>  Issue Type: Task
>  Components: atlas-intg
>Reporter: Josh Yeh
>Assignee: Prasad P. Pawar
>Priority: Trivial
> Attachments: flink.png, flink_process.png, 
> image-2021-06-07-08-41-37-559.png
>
>
> While testing ATLAS-3812, I noticed flink_application and flink_process icons 
> are missing. File this Jira to track.
> !image-2021-06-07-08-41-37-559.png!



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: Add Keys to Kafka Notifications - ATLAS-4323

2021-06-08 Thread Adam Bellemare 
Hi All

I have an initial PR for this issue.
https://github.com/apache/atlas/pull/139

The main points:
1) Only String key format is supported. It is not clear to me that we need
support for more than this, particularly if we're keying on GUID (which
seems to be the best candidate).
2) Previous compatibility remains unchanged, since null keys are randomly
distributed amongst partitions.
3) V1 notifications just use null.
4) V2 notifications use the GUID.

Please let me know if there are areas I need to address, make clearer,
change, etc. I am hesitant to support arbitrary key types that can't be
represented as a String without a strong use-case.





On Wed, Jun 2, 2021 at 3:45 PM Ashutosh Mestry 
wrote:

> Hi Adam
>
> It is OK to ignore V1 types. Majority of customer have moved to newer
> versions that use V2 messages.
>
> Best regards,
>
> ~ ashutosh
> Ashutosh Mestry
> ...
> No hurry, no pause. – Tim Ferriss, Life Hacker, Author
>
>
> From: Adam Bellemare 
> Date: Wednesday, June 2, 2021 at 11:04 AM
> To: dev@atlas.apache.org 
> Subject: Add Keys to Kafka Notifications - ATLAS-4323
> HI All
>
> The JIRA I filed: https://issues.apache.org/jira/browse/ATLAS-4323
>
> I want to add Keys to the Kafka notifications. My current plan is to use
> the GUID as the key for the kafka event. I noticed, however, that the V1
> notifications appear to use LONG Ids.
>
> I am evaluating a few options here:
>
> 1) Make it so that all V2 notifications have keys by default. This won't
> affect any existing behaviour since null keys give no guarantees of
> partition data locality. The downside here is that there would be a minor
> increase in the amount of data sent over the wire (the GUID string).
> 2) Leave V1 notifications entirely alone, and only enable GUID String keys
> on V2 notifications.
> 3) Enable notifications with LONG key V1, String GUID key V2.
>
> The main issue is around serializing the arbitrary key type to Bytes. If we
> decide to only go with String GUIDs then this is much easier (just for V2),
> but I am curious as to if V1 users would also want this data locality?
>
> The most robust solution is to just support keys of arbitrary types. If
> this is the desired outcome I can do this as well, I just wanted to get a
> feeling for the limits of scope of responsibility for V1 and V2
> notifications.
>
> Thanks
> Adam
>

[GitHub] [atlas] bellemare opened a new pull request #139: ATLAS-4323: Add keys to Kafka notifications

2021-06-08 Thread GitBox 


bellemare opened a new pull request #139:
URL: https://github.com/apache/atlas/pull/139


   In draft to seek feedback:
   
   Important points:
   1) Key is fixed to String, which is the format of GUID.
   2) Keys will be GUID based, as the GUID field does not change. This will 
ensure data locality within the Kafka topic's partitions.
   3) Currently, no support for arbitrary keys. It is not clear to me that 
there are any use-cases that would require us to provide the user with the 
ability to key against arbitrary fields / data structures. 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

Re: Review Request 73302: ATLAS-4255 : Atlas - Upgrade netty to 4.1.61

2021-06-08 Thread chaitali 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/73302/
---

(Updated June 8, 2021, 11:34 a.m.)


Review request for atlas, Ashutosh Mestry, Jayendra Parab, Madhan Neethiraj, 
Nixon Rodrigues, and Sarath Subramanian.


Bugs: ATLAS-4255
https://issues.apache.org/jira/browse/ATLAS-4255


Repository: atlas


Description (updated)
---

Currently  Atlas is pulling netty jar versions 4.1.47,4.1.51,4.1.23,4.1.48 

Upgrade it to latest 4.1.61-final
Introduced a dependency as netty-bom which can take care of all transitive 
dependencies for io.netty atlas is pulling

Revised patch where unwanted jars were pulled are excluded.


Diffs (updated)
-

  addons/falcon-bridge/pom.xml e0d2f3be7 
  addons/hbase-bridge/pom.xml 50fb9e838 
  addons/hbase-testing-util/pom.xml 982e9c85e 
  addons/hive-bridge/pom.xml ea2de4d3b 
  addons/impala-bridge/pom.xml a1bd5920b 
  addons/kafka-bridge/pom.xml 7fe97eb39 
  addons/sqoop-bridge/pom.xml 9d5ac52d9 
  addons/storm-bridge/pom.xml 71525438b 
  common/pom.xml 616f66c53 
  graphdb/janus-hbase2/pom.xml 92a7104b9 
  graphdb/janus/pom.xml 8f1ed543a 
  intg/pom.xml 05914 
  notification/pom.xml 28d13bc13 
  pom.xml 91c23ee04 
  repository/pom.xml bf34efee2 
  test-tools/pom.xml c861ea67f 
  webapp/pom.xml ccbcc3141 


Diff: https://reviews.apache.org/r/73302/diff/2/

Changes: https://reviews.apache.org/r/73302/diff/1-2/


Testing
---

Atlas server up and running
Created entities,glossaries and tags
quickstart.py successful
PC : 
https://ci-builds.apache.org/job/Atlas/job/PreCommit-ATLAS-Build-Test/522/console


Thanks,

chaitali

[jira] [Updated] (ATLAS-4326) Atlas UI: Multiple continuous space is accepted on Short description for Classifications, Gloassarys, Categorys & Terms .

2021-06-08 Thread Prasad P. Pawar (Jira) 


 [ 
https://issues.apache.org/jira/browse/ATLAS-4326?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Prasad P. Pawar updated ATLAS-4326:
---
Attachment: 0001-ATLAS-4326-2-Atlas-UI-Multiple-continuous-space-is-a.patch

> Atlas UI: Multiple continuous space is accepted on Short description for 
> Classifications, Gloassarys, Categorys & Terms .
> -
>
> Key: ATLAS-4326
> URL: https://issues.apache.org/jira/browse/ATLAS-4326
> Project: Atlas
>  Issue Type: Bug
>  Components: atlas-webui
>Reporter: Prasad P. Pawar
>Assignee: Prasad P. Pawar
>Priority: Minor
> Attachments: 
> 0001-ATLAS-4326-1-Atlas-UI-Multiple-continuous-space-is-a.patch, 
> 0001-ATLAS-4326-2-Atlas-UI-Multiple-continuous-space-is-a.patch
>
>
> Add multiple continuous spaces in the short description field,  this value is 
> accepted on UI by clicking the Save/Update button.
> After updating the values, the Detail page doesn't show the entered value of 
> Short description files. eg: "   This      is  Term1   "



--
This message was sent by Atlassian Jira
(v8.3.4#803005)