Re: Review Request 51723: ATLAS-1149 : Changes to UI to sort the hive table schema based on "position" attribute of hive_column

[keval bhatt]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51723/#review148661
---


Ship it!




Ship It!

- keval bhatt


On Sept. 12, 2016, 9:05 a.m., Kalyani Kashikar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51723/
> ---
> 
> (Updated Sept. 12, 2016, 9:05 a.m.)
> 
> 
> Review request for atlas, keval bhatt, Madhan Neethiraj, Sarath Kumar 
> Subramanian, Shwetha GS, and Suma Shivaprasad.
> 
> 
> Bugs: ATLAS-1149
> https://issues.apache.org/jira/browse/ATLAS-1149
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> In hive column needs to be sorted based on its original column order. New 
> attribute "position" has been added to hive_column type which maintains the 
> column order number. UI needs to use this information to sort the column 
> based on this attribute on schema table & entity details page.
> 
> This patch includes changes for sort column based on "position" attribute
> 
> 
> Diffs
> -
> 
>   dashboardv2/public/js/collection/BaseCollection.js b039e30 
>   dashboardv2/public/js/views/entity/EntityDetailTableLayoutView.js cc67d01 
>   dashboardv2/public/js/views/schema/SchemaLayoutView.js dc19619 
> 
> Diff: https://reviews.apache.org/r/51723/diff/
> 
> 
> Testing
> ---
> 
> Added hive-brigde jar in atlas-impl dir in hive hook
> Created new attribute “position” for hive_column type with curl command.
> Tested entity details page UI for hive_table entities by adding new tables in 
> hive beeline client.
> In schema table the db column are sorted according to position attribute in 
> api.
> 
> 
> Thanks,
> 
> Kalyani Kashikar
> 
>

Re: Review Request 51660: [ATLAS-1098] Added keyword checks while creating a new tag/trait

[Apoorv Naik]

> On Sept. 13, 2016, 4:40 a.m., Shwetha GS wrote:
> > repository/src/main/java/org/apache/atlas/services/DefaultMetadataService.java,
> >  line 309
> > 
> >
> > This is equals check - typename equals keyword. Does it work if 
> > typename contains keyword e.g., if typename is 'isa PII'?
> > 
> > Not sure if keywords in DSL is case sensitive. You may want to verify
> 
> Apoorv Naik wrote:
> Right now this will go through.

I've tried changing the case but seems like the dsl is not case sensitive for 
those keywords


- Apoorv


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51660/#review148651
---


On Sept. 6, 2016, 11:27 p.m., Apoorv Naik wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51660/
> ---
> 
> (Updated Sept. 6, 2016, 11:27 p.m.)
> 
> 
> Review request for atlas, Madhan Neethiraj, Shwetha GS, Suma Shivaprasad, and 
> Vimal Sharma.
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> [ATLAS-1098] Added keyword checks while creating a new tag/trait
> 
> 
> Diffs
> -
> 
>   
> repository/src/main/java/org/apache/atlas/services/DefaultMetadataService.java
>  4d05d49 
> 
> Diff: https://reviews.apache.org/r/51660/diff/
> 
> 
> Testing
> ---
> 
> Tested with all possible keywords specified in the QueryParser, the UI shows 
> a red exception message stating that the type being created is a keyword
> 
> 
> Thanks,
> 
> Apoorv Naik
> 
>

Re: Review Request 51660: [ATLAS-1098] Added keyword checks while creating a new tag/trait

[Apoorv Naik]

> On Sept. 12, 2016, 8:24 a.m., Shwetha GS wrote:
> > repository/src/main/java/org/apache/atlas/services/DefaultMetadataService.java,
> >  line 295
> > 
> >
> > TypeSystem.verifyTypes() has all the validations. Please move this 
> > logic there.

Initially I was adding this logic in verifyTypes but looks like there has to a 
dependency change which in turn introduces a circular dependency that's the 
reason I haven't added it in verifyTypes


> On Sept. 12, 2016, 8:24 a.m., Shwetha GS wrote:
> > repository/src/main/java/org/apache/atlas/services/DefaultMetadataService.java,
> >  lines 246-247
> > 
> >
> > Can you also add tests for this?

Working on adding the tests


- Apoorv


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51660/#review148465
---


On Sept. 6, 2016, 11:27 p.m., Apoorv Naik wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51660/
> ---
> 
> (Updated Sept. 6, 2016, 11:27 p.m.)
> 
> 
> Review request for atlas, Madhan Neethiraj, Shwetha GS, Suma Shivaprasad, and 
> Vimal Sharma.
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> [ATLAS-1098] Added keyword checks while creating a new tag/trait
> 
> 
> Diffs
> -
> 
>   
> repository/src/main/java/org/apache/atlas/services/DefaultMetadataService.java
>  4d05d49 
> 
> Diff: https://reviews.apache.org/r/51660/diff/
> 
> 
> Testing
> ---
> 
> Tested with all possible keywords specified in the QueryParser, the UI shows 
> a red exception message stating that the type being created is a keyword
> 
> 
> Thanks,
> 
> Apoorv Naik
> 
>

Re: Review Request 51660: [ATLAS-1098] Added keyword checks while creating a new tag/trait

[Apoorv Naik]

> On Sept. 13, 2016, 4:40 a.m., Shwetha GS wrote:
> > repository/src/main/java/org/apache/atlas/services/DefaultMetadataService.java,
> >  line 309
> > 
> >
> > This is equals check - typename equals keyword. Does it work if 
> > typename contains keyword e.g., if typename is 'isa PII'?
> > 
> > Not sure if keywords in DSL is case sensitive. You may want to verify

Right now this will go through.


- Apoorv


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51660/#review148651
---


On Sept. 6, 2016, 11:27 p.m., Apoorv Naik wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51660/
> ---
> 
> (Updated Sept. 6, 2016, 11:27 p.m.)
> 
> 
> Review request for atlas, Madhan Neethiraj, Shwetha GS, Suma Shivaprasad, and 
> Vimal Sharma.
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> [ATLAS-1098] Added keyword checks while creating a new tag/trait
> 
> 
> Diffs
> -
> 
>   
> repository/src/main/java/org/apache/atlas/services/DefaultMetadataService.java
>  4d05d49 
> 
> Diff: https://reviews.apache.org/r/51660/diff/
> 
> 
> Testing
> ---
> 
> Tested with all possible keywords specified in the QueryParser, the UI shows 
> a red exception message stating that the type being created is a keyword
> 
> 
> Thanks,
> 
> Apoorv Naik
> 
>

Build failed in Jenkins: apache-atlas-nightly #422

[Apache Jenkins Server]

See 

Changes:

[sumasai.shivaprasad] ATLAS-1162: shutdown hooks to register with 
ShutdownHookManager, instead

--
[...truncated 9131 lines...]
127.0.0.1 - - [13/Sep/2016:04:37:54 +] "GET 
/api/atlas/lineage/hive/table/default.tableh9djyjeq4r@primary/inputs/graph 
HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [13/Sep/2016:04:37:59 +] "GET 
/api/atlas/entities?type=hive_process=qualifiedName=QUERY:default.tablekw5dwjnmdd@primary:1473741477000-%3E:PATH_WRITE
 HTTP/1.1" 404 - "-" "Java/1.7.0_80"
127.0.0.1 - - [13/Sep/2016:04:38:04 +] "GET 
/api/atlas/entities?type=hive_process=qualifiedName=QUERY:default.tablekw5dwjnmdd@primary:1473741477000-%3E:PATH_WRITE
 HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [13/Sep/2016:04:38:04 +] "GET 
/api/atlas/entities?type=hive_process=qualifiedName=QUERY:default.tablekw5dwjnmdd@primary:1473741477000-%3E:PATH_WRITE
 HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [13/Sep/2016:04:38:04 +] "GET 
/api/atlas/entities/4d65d338-3d6a-426f-99a6-4da0390c0eea HTTP/1.1" 200 - "-" 
"Java/1.7.0_80"
127.0.0.1 - - [13/Sep/2016:04:38:04 +] "GET 
/api/atlas/entities/0b8ad53b-0286-4a64-8cfc-aef150d8 HTTP/1.1" 200 - "-" 
"Java/1.7.0_80"
127.0.0.1 - - [13/Sep/2016:04:38:04 +] "GET 
/api/atlas/entities/acc825c3-4f65-435d-a481-59fb9f7c1c7f HTTP/1.1" 200 - "-" 
"Java/1.7.0_80"
127.0.0.1 - - [13/Sep/2016:04:38:04 +] "GET 
/api/atlas/entities?type=hdfs_path=qualifiedName=p
 HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [13/Sep/2016:04:38:04 +] "GET 
/api/atlas/entities?type=hdfs_path=qualifiedName=p
 HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [13/Sep/2016:04:38:04 +] "GET 
/api/atlas/entities/acc825c3-4f65-435d-a481-59fb9f7c1c7f HTTP/1.1" 200 - "-" 
"Java/1.7.0_80"
127.0.0.1 - - [13/Sep/2016:04:38:04 +] "GET 
/api/atlas/entities?type=hive_table=qualifiedName=default.tablekw5dwjnmdd@primary
 HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [13/Sep/2016:04:38:04 +] "GET 
/api/atlas/entities?type=hive_table=qualifiedName=default.tablekw5dwjnmdd@primary
 HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [13/Sep/2016:04:38:04 +] "GET 
/api/atlas/entities/0b8ad53b-0286-4a64-8cfc-aef150d8 HTTP/1.1" 200 - "-" 
"Java/1.7.0_80"
127.0.0.1 - - [13/Sep/2016:04:38:07 +] "GET 
/api/atlas/entities?type=hive_table=qualifiedName=default.tablekw5dwjnmdd@primary
 HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [13/Sep/2016:04:38:07 +] "GET 
/api/atlas/entities?type=hive_table=qualifiedName=default.tablekw5dwjnmdd@primary
 HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [13/Sep/2016:04:38:07 +] "GET 
/api/atlas/entities?type=hive_process=qualifiedName=QUERY:default.tablekw5dwjnmdd@primary:1473741477000-%3E:PATH_WRITE
 HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [13/Sep/2016:04:38:07 +] "GET 
/api/atlas/entities?type=hive_process=qualifiedName=QUERY:default.tablekw5dwjnmdd@primary:1473741477000-%3E:PATH_WRITE
 HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [13/Sep/2016:04:38:07 +] "GET 
/api/atlas/entities/4d65d338-3d6a-426f-99a6-4da0390c0eea HTTP/1.1" 200 - "-" 
"Java/1.7.0_80"
127.0.0.1 - - [13/Sep/2016:04:38:07 +] "GET 
/api/atlas/entities/0b8ad53b-0286-4a64-8cfc-aef150d8 HTTP/1.1" 200 - "-" 
"Java/1.7.0_80"
127.0.0.1 - - [13/Sep/2016:04:38:07 +] "GET 
/api/atlas/entities/acc825c3-4f65-435d-a481-59fb9f7c1c7f HTTP/1.1" 200 - "-" 
"Java/1.7.0_80"
127.0.0.1 - - [13/Sep/2016:04:38:07 +] "GET 
/api/atlas/entities?type=hdfs_path=qualifiedName=p
 HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [13/Sep/2016:04:38:07 +] "GET 
/api/atlas/entities?type=hdfs_path=qualifiedName=p
 HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [13/Sep/2016:04:38:07 +] "GET 
/api/atlas/entities/acc825c3-4f65-435d-a481-59fb9f7c1c7f HTTP/1.1" 200 - "-" 
"Java/1.7.0_80"
127.0.0.1 - - [13/Sep/2016:04:38:09 +] "GET 
/api/atlas/entities?type=hive_table=qualifiedName=default.tablekw5dwjnmdd@primary
 HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [13/Sep/2016:04:38:09 +] "GET 
/api/atlas/entities?type=hive_table=qualifiedName=default.tablekw5dwjnmdd@primary
 HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [13/Sep/2016:04:38:09 +] "GET 
/api/atlas/entities?type=hive_process=qualifiedName=QUERY:default.tablekw5dwjnmdd@primary:1473741477000-%3E:PATH_WRITE
 HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [13/Sep/2016:04:38:09 +] "GET 

Re: Review Request 51660: [ATLAS-1098] Added keyword checks while creating a new tag/trait

[Shwetha GS]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51660/#review148651
---




repository/src/main/java/org/apache/atlas/services/DefaultMetadataService.java 
(line 309)


This is equals check - typename equals keyword. Does it work if typename 
contains keyword e.g., if typename is 'isa PII'?

Not sure if keywords in DSL is case sensitive. You may want to verify


- Shwetha GS


On Sept. 6, 2016, 11:27 p.m., Apoorv Naik wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51660/
> ---
> 
> (Updated Sept. 6, 2016, 11:27 p.m.)
> 
> 
> Review request for atlas, Madhan Neethiraj, Shwetha GS, Suma Shivaprasad, and 
> Vimal Sharma.
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> [ATLAS-1098] Added keyword checks while creating a new tag/trait
> 
> 
> Diffs
> -
> 
>   
> repository/src/main/java/org/apache/atlas/services/DefaultMetadataService.java
>  4d05d49 
> 
> Diff: https://reviews.apache.org/r/51660/diff/
> 
> 
> Testing
> ---
> 
> Tested with all possible keywords specified in the QueryParser, the UI shows 
> a red exception message stating that the type being created is a keyword
> 
> 
> Thanks,
> 
> Apoorv Naik
> 
>

Current use cases for POST /entities

[Apoorv Naik]

Is anyone relying on the definition object within the response of a create 
entity call ? If not, let’s remove it from the response as it can be inferred 
from the request body itself and the only useful piece seems to be the GUIDs.

[jira] [Commented] (ATLAS-1098) Atlas allows creation of tag with name "isa" which causes exceptions during search

[Apoorv Naik (JIRA)]

[ 
https://issues.apache.org/jira/browse/ATLAS-1098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15486193#comment-15486193
 ] 

Apoorv Naik commented on ATLAS-1098:


Yes I tried escaping the keywords and it still doesn't work.

> Atlas allows creation of tag with name "isa" which causes exceptions during 
> search
> --
>
> Key: ATLAS-1098
> URL: https://issues.apache.org/jira/browse/ATLAS-1098
> Project: Atlas
>  Issue Type: Bug
>Reporter: Sharmadha Sainath
>Assignee: Apoorv Naik
>
> Created a tag with name "isa". DSL query "Table where Table isa isa" throws 
> exception . Search in Tag tab also throws exception.
>  



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Review Request 51829: ATLAS-1162: shutdown hooks to register with ShutdownHookManager, instead of System.Runtime

[Suma Shivaprasad]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51829/#review148638
---


Ship it!




Ship It!

- Suma Shivaprasad


On Sept. 12, 2016, 11:51 p.m., Madhan Neethiraj wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51829/
> ---
> 
> (Updated Sept. 12, 2016, 11:51 p.m.)
> 
> 
> Review request for atlas.
> 
> 
> Bugs: ATLAS-1162
> https://issues.apache.org/jira/browse/ATLAS-1162
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> Updated Atlas hooks and server to use Hadoop's ShutdownHookManager to 
> register shutdown hooks, instead of directly with System Runtime
> 
> 
> Diffs
> -
> 
>   
> addons/falcon-bridge/src/main/java/org/apache/atlas/falcon/hook/FalconHook.java
>  d724f57 
>   addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/HiveHook.java 
> fdd0199 
>   common/src/main/java/org/apache/atlas/AtlasConstants.java cee85b4 
>   webapp/src/main/java/org/apache/atlas/Atlas.java dd43c6d 
> 
> Diff: https://reviews.apache.org/r/51829/diff/
> 
> 
> Testing
> ---
> 
> Validated the shutdown hooks work as before by restarting Atlas server and 
> components that run impacted Atlas hooks
> 
> 
> Thanks,
> 
> Madhan Neethiraj
> 
>

[jira] [Updated] (ATLAS-1162) Register shutdown hooks with Hadoop's ShutdownHookManager, instead of directly with Java Runtime

[Madhan Neethiraj (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-1162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Madhan Neethiraj updated ATLAS-1162:

Attachment: ATLAS-1162.patch

> Register shutdown hooks with Hadoop's ShutdownHookManager, instead of 
> directly with Java Runtime
> 
>
> Key: ATLAS-1162
> URL: https://issues.apache.org/jira/browse/ATLAS-1162
> Project: Atlas
>  Issue Type: Bug
>Affects Versions: 0.7-incubating
>Reporter: Madhan Neethiraj
>Assignee: Madhan Neethiraj
> Attachments: ATLAS-1162.patch
>
>
> Atlas hooks and server currently register a shutdown hook directly with java 
> Runtime object. Hadoop provides an utility class, ShutdownHookManager, which 
> helps to ensure that the hooks are called in a well defined order. Atlas hook 
> registration should be updated to register its shutdown hook with Hadoop's 
> ShutdownHookManager:
> https://github.com/apache/hadoop/blob/master/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/ShutdownHookManager.java.
> Ref: RANGER-891



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Review Request 51829: ATLAS-1162: shutdown hooks to register with ShutdownHookManager, instead of System.Runtime

[Madhan Neethiraj]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51829/
---

Review request for atlas.


Bugs: ATLAS-1162
https://issues.apache.org/jira/browse/ATLAS-1162


Repository: atlas


Description
---

Updated Atlas hooks and server to use Hadoop's ShutdownHookManager to register 
shutdown hooks, instead of directly with System Runtime


Diffs
-

  
addons/falcon-bridge/src/main/java/org/apache/atlas/falcon/hook/FalconHook.java 
d724f57 
  addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/HiveHook.java 
fdd0199 
  common/src/main/java/org/apache/atlas/AtlasConstants.java cee85b4 
  webapp/src/main/java/org/apache/atlas/Atlas.java dd43c6d 

Diff: https://reviews.apache.org/r/51829/diff/


Testing
---

Validated the shutdown hooks work as before by restarting Atlas server and 
components that run impacted Atlas hooks


Thanks,

Madhan Neethiraj

[jira] [Created] (ATLAS-1162) Register shutdown hooks with Hadoop's ShutdownHookManager, instead of directly with Java Runtime

[Madhan Neethiraj (JIRA)]

Madhan Neethiraj created ATLAS-1162:
---

 Summary: Register shutdown hooks with Hadoop's 
ShutdownHookManager, instead of directly with Java Runtime
 Key: ATLAS-1162
 URL: https://issues.apache.org/jira/browse/ATLAS-1162
 Project: Atlas
  Issue Type: Bug
Affects Versions: 0.7-incubating
Reporter: Madhan Neethiraj
Assignee: Madhan Neethiraj


Atlas hooks and server currently register a shutdown hook directly with java 
Runtime object. Hadoop provides an utility class, ShutdownHookManager, which 
helps to ensure that the hooks are called in a well defined order. Atlas hook 
registration should be updated to register its shutdown hook with Hadoop's 
ShutdownHookManager:
https://github.com/apache/hadoop/blob/master/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/ShutdownHookManager.java.

Ref: RANGER-891



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (ATLAS-1098) Atlas allows creation of tag with name "isa" which causes exceptions during search

[Suma Shivaprasad (JIRA)]

[ 
https://issues.apache.org/jira/browse/ATLAS-1098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15485324#comment-15485324
 ] 

Suma Shivaprasad commented on ATLAS-1098:
-

[~apoorvnaik] Does the parser break even with an escaped keyword. For eg:Table 
where Table isA `isA`

> Atlas allows creation of tag with name "isa" which causes exceptions during 
> search
> --
>
> Key: ATLAS-1098
> URL: https://issues.apache.org/jira/browse/ATLAS-1098
> Project: Atlas
>  Issue Type: Bug
>Reporter: Sharmadha Sainath
>Assignee: Apoorv Naik
>
> Created a tag with name "isa". DSL query "Table where Table isa isa" throws 
> exception . Search in Tag tab also throws exception.
>  



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (ATLAS-597) Handle Cluster renaming

[Suma Shivaprasad (JIRA)]

[ 
https://issues.apache.org/jira/browse/ATLAS-597?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15299530#comment-15299530
 ] 

Suma Shivaprasad edited comment on ATLAS-597 at 9/12/16 8:12 PM:
-

Cluster entity

The cluster entity will have the following properties

name
description
list aliases - This will hold older cluster names or previous aliases
 

Hook changes for Entity gets/updates/creates /deletes

This also means that the hooks will have to pass the qualifiedName in entity 
creates/updates/deletes/gets  as  "dbName.tableName@clusterId" instead of the 
"dbName.tableName@clusterName"




was (Author: suma.shivaprasad):
Cluster entity

The cluster entity will have the following properties

name
description
list aliases - This will hold older cluster names or previous aliases
 

Hook changes for Entity updates/creates /deletes

This also means that the hooks will have to pass the qualifiedName in entity 
creates/updates/deletes  as  "dbName.tableName@clusterId" instead of the 
"dbName.tableName@clusterName"

> Handle Cluster renaming
> ---
>
> Key: ATLAS-597
> URL: https://issues.apache.org/jira/browse/ATLAS-597
> Project: Atlas
>  Issue Type: Sub-task
>Affects Versions: 0.7-incubating
>Reporter: Suma Shivaprasad
>
> Cluster name could easily be changed by users which is used in qualified 
> names to dedup entities. For eg: hive_table has a qualifiedName 
>  . Hence all entities will be recreated if they 
> are renamed unless a migration on repository is done with new cluster name.
> Hence cluster  should be represented by an id which is constant and tied to 
> clusterName which could change



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (ATLAS-1144) After updating/creating a set of entities, response contains the first entity definition only

[Suma Shivaprasad (JIRA)]

[ 
https://issues.apache.org/jira/browse/ATLAS-1144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15484967#comment-15484967
 ] 

Suma Shivaprasad commented on ATLAS-1144:
-

Having both definition and definitionList in the response is very confusing. We 
should have an all or nothing approach which returns  "definitions" if 
responseType=verbose if specified else returns no definitions and this should 
be a new version /v1/.. of the API.

> After updating/creating a set of entities, response contains the first entity 
> definition only
> -
>
> Key: ATLAS-1144
> URL: https://issues.apache.org/jira/browse/ATLAS-1144
> Project: Atlas
>  Issue Type: Bug
>Affects Versions: 0.7-incubating
>Reporter: Apoorv Naik
>Assignee: Apoorv Naik
> Fix For: 0.7-incubating
>
> Attachments: ATLAS_1144.patch
>
>
> Version: Atlas 0.7
> Steps to recreate:
> Create two entities (POST /entities) and note guid1 and guid2 that get 
> generated.
> Update these entities in the same PUT /entities and response contains 
> (truncated for convenience):
> {
>   "requestId": "...",
>   "GUID": [
> "guid1",
> "guid2"
>   ],
>   "definition": "{..*\"id\":\"guid1\"*,}
> }
> {color:red}Issue: definition does not contain the data for second 
> entity.{color}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (ATLAS-1139) Parameter name of a DataSet entity should contain filesystem path

[Suma Shivaprasad (JIRA)]

[ 
https://issues.apache.org/jira/browse/ATLAS-1139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15484890#comment-15484890
 ] 

Suma Shivaprasad commented on ATLAS-1139:
-

Similiar changes are needed in StormHook

> Parameter name of a DataSet entity should contain filesystem path
> -
>
> Key: ATLAS-1139
> URL: https://issues.apache.org/jira/browse/ATLAS-1139
> Project: Atlas
>  Issue Type: Bug
>Reporter: Vimal Sharma
>Assignee: Vimal Sharma
> Attachments: ATLAS-1139.patch
>
>
> In lineage graph, input and output entity can have the same value in 
> attribute "name". This causes ambiguity for the user to correctly understand 
> the lineage graph.
> The proposal is to set normalized path as the value of "name" attribute.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (ATLAS-205) Handle Titan Transaction failures

[Suma Shivaprasad (JIRA)]

[ 
https://issues.apache.org/jira/browse/ATLAS-205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15484857#comment-15484857
 ] 

Suma Shivaprasad commented on ATLAS-205:


User Impact :

Index update failures could cause inconsistencies between storage and indexing 
backend since there is no two-phase commit across these system in Titan . Hence 
users using full text search may be missing some search results compared to the 
equality match searches/dsl searches performed on the storage backend.

> Handle Titan Transaction failures
> -
>
> Key: ATLAS-205
> URL: https://issues.apache.org/jira/browse/ATLAS-205
> Project: Atlas
>  Issue Type: Task
>Reporter: Suma Shivaprasad
>
> Refer 
> http://s3.thinkaurelius.com/docs/titan/0.5.4/failure-recovery.html#_transaction_failure
>  for details



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (ATLAS-1139) Parameter name of a DataSet entity should contain filesystem path

[Suma Shivaprasad (JIRA)]

[ 
https://issues.apache.org/jira/browse/ATLAS-1139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15484839#comment-15484839
 ] 

Suma Shivaprasad commented on ATLAS-1139:
-

Can we use 
https://hadoop.apache.org/docs/r2.7.2/api/org/apache/hadoop/fs/Path.html#getPathWithoutSchemeAndAuthority(org.apache.hadoop.fs.Path)
 to get the HDFS path instead of the current way of doing it?

> Parameter name of a DataSet entity should contain filesystem path
> -
>
> Key: ATLAS-1139
> URL: https://issues.apache.org/jira/browse/ATLAS-1139
> Project: Atlas
>  Issue Type: Bug
>Reporter: Vimal Sharma
>Assignee: Vimal Sharma
> Attachments: ATLAS-1139.patch
>
>
> In lineage graph, input and output entity can have the same value in 
> attribute "name". This causes ambiguity for the user to correctly understand 
> the lineage graph.
> The proposal is to set normalized path as the value of "name" attribute.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (ATLAS-712) Support getTrait() API

[Suma Shivaprasad (JIRA)]

[ 
https://issues.apache.org/jira/browse/ATLAS-712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15484825#comment-15484825
 ] 

Suma Shivaprasad commented on ATLAS-712:


[~svimal2106] Can you add the corrresponding changes in AtlasClient?

> Support getTrait() API
> --
>
> Key: ATLAS-712
> URL: https://issues.apache.org/jira/browse/ATLAS-712
> Project: Atlas
>  Issue Type: Improvement
>Reporter: Suma Shivaprasad
>Assignee: Vimal Sharma
> Attachments: ATLAS-712.patch
>
>
> Given entity id and trait name, support rest API that returns the trait 
> instance for the entity. Currently, the other way of getting it is 
> getEntity() which returns full entity with all trait instances 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (ATLAS-1149) Changes to UI to sort the hive table schema based on "position" attribute of hive_column

[Madhan Neethiraj (JIRA)]

[ 
https://issues.apache.org/jira/browse/ATLAS-1149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15484433#comment-15484433
 ] 

Madhan Neethiraj commented on ATLAS-1149:
-

If we go with updating the query to handle sorting, make sure to handle update 
of the configuration during upgrade (and revert during downgrade/rollback..). 
Until the mechanism to handle this is in place, I think it makes sense for the 
UI to sort the columns.

> Changes to UI to sort the hive table schema based on "position" attribute of 
> hive_column
> 
>
> Key: ATLAS-1149
> URL: https://issues.apache.org/jira/browse/ATLAS-1149
> Project: Atlas
>  Issue Type: Bug
>Affects Versions: 0.7-incubating
>Reporter: Sarath Subramanian
>Assignee: Kalyani Kashikar
>  Labels: ui
> Attachments: ATLAS-1149.1.patch, ATLAS-1149.patch, attach1.json, 
> attach2.json
>
>
> This is related to ATLAS-772, hive column needs to be sorted based on its 
> original column order. New attribute "position" has been added to hive_column 
> type which maintains the column order number. UI needs to use this 
> information to sort the column based on this attribute



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (ATLAS-1144) After updating/creating a set of entities, response contains the first entity definition only

[Apoorv Naik (JIRA)]

[ 
https://issues.apache.org/jira/browse/ATLAS-1144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15484316#comment-15484316
 ] 

Apoorv Naik edited comment on ATLAS-1144 at 9/12/16 2:47 PM:
-

Here's how the API response with query param responseType=full would look like 
with the new field definitionList that contains the definitions of all the 
entities successfully modified

{code:javascript}
{
  "requestId": "qtp1297978429-21 - 14bdc7b5-3d44-4ae0-8e75-e315ca4e3d37",
  "entities": {
"updated": [
  "ac85e13e-926f-4618-97f9-779cbcf23f49",
  "23eeba58-2cf0-42bc-9bd0-54d0f9f2e0c8"
]
  },
  "definition": {
"jsonClass": 
"org.apache.atlas.typesystem.json.InstanceSerialization$_Reference",
"id": {
  "jsonClass": "org.apache.atlas.typesystem.json.InstanceSerialization$_Id",
  "id": "ac85e13e-926f-4618-97f9-779cbcf23f49",
  "version": 0,
  "typeName": "DataSet",
  "state": "ACTIVE"
},
"typeName": "DataSet",
"values": {
  "qualifiedName": "default@cluster12",
  "owner": "hbase_admin",
  "description": "Default HBase namespace",
  "name": "default"
},
"traitNames": [],
"traits": {}
  },
"definitionList": [
{
"jsonClass": 
"org.apache.atlas.typesystem.json.InstanceSerialization$_Reference",
"id": {
  "jsonClass": "org.apache.atlas.typesystem.json.InstanceSerialization$_Id",
  "id": "ac85e13e-926f-4618-97f9-779cbcf23f49",
  "version": 0,
  "typeName": "DataSet",
  "state": "ACTIVE"
},
"typeName": "DataSet",
"values": {
  "qualifiedName": "default@cluster12",
  "owner": "hbase_admin",
  "description": "Default HBase namespace",
  "name": "default"
},
"traitNames": [],
"traits": {}
  },
{
  "jsonClass": 
"org.apache.atlas.typesystem.json.InstanceSerialization$_Reference",
  "id": {
"jsonClass": 
"org.apache.atlas.typesystem.json.InstanceSerialization$_Id",
"id": "23eeba58-2cf0-42bc-9bd0-54d0f9f2e0c8",
"version": 0,
"typeName": "DataSet",
"state": "ACTIVE"
  },
  "typeName": "DataSet",
  "values": {
"qualifiedName": "default@cluster11",
"owner": "hbase_admin",
"description": "Default HBase namespace",
"name": "default"
  },
  "traitNames": [],
  "traits": {}
}
  ]
}
{code}


was (Author: apoorvnaik):
Here's how the API response with query param responseType=full would look like 
with the new field definitionList that contains the definitions of all the 
entities successfully modified
{
  "requestId": "qtp1297978429-21 - 14bdc7b5-3d44-4ae0-8e75-e315ca4e3d37",
  "entities": {
"updated": [
  "ac85e13e-926f-4618-97f9-779cbcf23f49",
  "23eeba58-2cf0-42bc-9bd0-54d0f9f2e0c8"
]
  },
  "definition": {
"jsonClass": 
"org.apache.atlas.typesystem.json.InstanceSerialization$_Reference",
"id": {
  "jsonClass": "org.apache.atlas.typesystem.json.InstanceSerialization$_Id",
  "id": "ac85e13e-926f-4618-97f9-779cbcf23f49",
  "version": 0,
  "typeName": "DataSet",
  "state": "ACTIVE"
},
"typeName": "DataSet",
"values": {
  "qualifiedName": "default@cluster12",
  "owner": "hbase_admin",
  "description": "Default HBase namespace",
  "name": "default"
},
"traitNames": [],
"traits": {}
  },
{color:red} "definitionList"{color:red}: [
{
"jsonClass": 
"org.apache.atlas.typesystem.json.InstanceSerialization$_Reference",
"id": {
  "jsonClass": "org.apache.atlas.typesystem.json.InstanceSerialization$_Id",
  "id": "ac85e13e-926f-4618-97f9-779cbcf23f49",
  "version": 0,
  "typeName": "DataSet",
  "state": "ACTIVE"
},
"typeName": "DataSet",
"values": {
  "qualifiedName": "default@cluster12",
  "owner": "hbase_admin",
  "description": "Default HBase namespace",
  "name": "default"
},
"traitNames": [],
"traits": {}
  },
{
  "jsonClass": 
"org.apache.atlas.typesystem.json.InstanceSerialization$_Reference",
  "id": {
"jsonClass": 
"org.apache.atlas.typesystem.json.InstanceSerialization$_Id",
"id": "23eeba58-2cf0-42bc-9bd0-54d0f9f2e0c8",
"version": 0,
"typeName": "DataSet",
"state": "ACTIVE"
  },
  "typeName": "DataSet",
  "values": {
"qualifiedName": "default@cluster11",
"owner": "hbase_admin",
"description": "Default HBase namespace",
"name": "default"
  },
  "traitNames": [],
  "traits": {}
}
  ]
}

> After updating/creating a set of entities, response contains the first entity 
> definition only
> -
>
> Key: ATLAS-1144
> URL: https://issues.apache.org/jira/browse/ATLAS-1144
> Project: Atlas
>  

[jira] [Comment Edited] (ATLAS-1144) After updating/creating a set of entities, response contains the first entity definition only

[Apoorv Naik (JIRA)]

[ 
https://issues.apache.org/jira/browse/ATLAS-1144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15484316#comment-15484316
 ] 

Apoorv Naik edited comment on ATLAS-1144 at 9/12/16 2:45 PM:
-

Here's how the API response with query param responseType=full would look like 
with the new field definitionList that contains the definitions of all the 
entities successfully modified
{
  "requestId": "qtp1297978429-21 - 14bdc7b5-3d44-4ae0-8e75-e315ca4e3d37",
  "entities": {
"updated": [
  "ac85e13e-926f-4618-97f9-779cbcf23f49",
  "23eeba58-2cf0-42bc-9bd0-54d0f9f2e0c8"
]
  },
  "definition": {
"jsonClass": 
"org.apache.atlas.typesystem.json.InstanceSerialization$_Reference",
"id": {
  "jsonClass": "org.apache.atlas.typesystem.json.InstanceSerialization$_Id",
  "id": "ac85e13e-926f-4618-97f9-779cbcf23f49",
  "version": 0,
  "typeName": "DataSet",
  "state": "ACTIVE"
},
"typeName": "DataSet",
"values": {
  "qualifiedName": "default@cluster12",
  "owner": "hbase_admin",
  "description": "Default HBase namespace",
  "name": "default"
},
"traitNames": [],
"traits": {}
  },
{color:red} "definitionList"{color:red}: [
{
"jsonClass": 
"org.apache.atlas.typesystem.json.InstanceSerialization$_Reference",
"id": {
  "jsonClass": "org.apache.atlas.typesystem.json.InstanceSerialization$_Id",
  "id": "ac85e13e-926f-4618-97f9-779cbcf23f49",
  "version": 0,
  "typeName": "DataSet",
  "state": "ACTIVE"
},
"typeName": "DataSet",
"values": {
  "qualifiedName": "default@cluster12",
  "owner": "hbase_admin",
  "description": "Default HBase namespace",
  "name": "default"
},
"traitNames": [],
"traits": {}
  },
{
  "jsonClass": 
"org.apache.atlas.typesystem.json.InstanceSerialization$_Reference",
  "id": {
"jsonClass": 
"org.apache.atlas.typesystem.json.InstanceSerialization$_Id",
"id": "23eeba58-2cf0-42bc-9bd0-54d0f9f2e0c8",
"version": 0,
"typeName": "DataSet",
"state": "ACTIVE"
  },
  "typeName": "DataSet",
  "values": {
"qualifiedName": "default@cluster11",
"owner": "hbase_admin",
"description": "Default HBase namespace",
"name": "default"
  },
  "traitNames": [],
  "traits": {}
}
  ]
}


was (Author: apoorvnaik):
Here's how the API response would look like

{
  "requestId": "qtp1297978429-21 - 14bdc7b5-3d44-4ae0-8e75-e315ca4e3d37",
  "entities": {
"updated": [
  "ac85e13e-926f-4618-97f9-779cbcf23f49",
  "23eeba58-2cf0-42bc-9bd0-54d0f9f2e0c8"
]
  },
  "definition": {
"jsonClass": 
"org.apache.atlas.typesystem.json.InstanceSerialization$_Reference",
"id": {
  "jsonClass": "org.apache.atlas.typesystem.json.InstanceSerialization$_Id",
  "id": "ac85e13e-926f-4618-97f9-779cbcf23f49",
  "version": 0,
  "typeName": "DataSet",
  "state": "ACTIVE"
},
"typeName": "DataSet",
"values": {
  "qualifiedName": "default@cluster12",
  "owner": "hbase_admin",
  "description": "Default HBase namespace",
  "name": "default"
},
"traitNames": [],
"traits": {}
  },
{color:red} "definitionList"{color:red}: [
{
"jsonClass": 
"org.apache.atlas.typesystem.json.InstanceSerialization$_Reference",
"id": {
  "jsonClass": "org.apache.atlas.typesystem.json.InstanceSerialization$_Id",
  "id": "ac85e13e-926f-4618-97f9-779cbcf23f49",
  "version": 0,
  "typeName": "DataSet",
  "state": "ACTIVE"
},
"typeName": "DataSet",
"values": {
  "qualifiedName": "default@cluster12",
  "owner": "hbase_admin",
  "description": "Default HBase namespace",
  "name": "default"
},
"traitNames": [],
"traits": {}
  },
{
  "jsonClass": 
"org.apache.atlas.typesystem.json.InstanceSerialization$_Reference",
  "id": {
"jsonClass": 
"org.apache.atlas.typesystem.json.InstanceSerialization$_Id",
"id": "23eeba58-2cf0-42bc-9bd0-54d0f9f2e0c8",
"version": 0,
"typeName": "DataSet",
"state": "ACTIVE"
  },
  "typeName": "DataSet",
  "values": {
"qualifiedName": "default@cluster11",
"owner": "hbase_admin",
"description": "Default HBase namespace",
"name": "default"
  },
  "traitNames": [],
  "traits": {}
}
  ]
}

> After updating/creating a set of entities, response contains the first entity 
> definition only
> -
>
> Key: ATLAS-1144
> URL: https://issues.apache.org/jira/browse/ATLAS-1144
> Project: Atlas
>  Issue Type: Bug
>Affects Versions: 0.7-incubating
>Reporter: Apoorv Naik
>Assignee: Apoorv Naik
> 

[jira] [Commented] (ATLAS-1144) After updating/creating a set of entities, response contains the first entity definition only

[Apoorv Naik (JIRA)]

[ 
https://issues.apache.org/jira/browse/ATLAS-1144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15484316#comment-15484316
 ] 

Apoorv Naik commented on ATLAS-1144:


Here's how the API response would look like

{
  "requestId": "qtp1297978429-21 - 14bdc7b5-3d44-4ae0-8e75-e315ca4e3d37",
  "entities": {
"updated": [
  "ac85e13e-926f-4618-97f9-779cbcf23f49",
  "23eeba58-2cf0-42bc-9bd0-54d0f9f2e0c8"
]
  },
  "definition": {
"jsonClass": 
"org.apache.atlas.typesystem.json.InstanceSerialization$_Reference",
"id": {
  "jsonClass": "org.apache.atlas.typesystem.json.InstanceSerialization$_Id",
  "id": "ac85e13e-926f-4618-97f9-779cbcf23f49",
  "version": 0,
  "typeName": "DataSet",
  "state": "ACTIVE"
},
"typeName": "DataSet",
"values": {
  "qualifiedName": "default@cluster12",
  "owner": "hbase_admin",
  "description": "Default HBase namespace",
  "name": "default"
},
"traitNames": [],
"traits": {}
  },
{color:red} "definitionList"{color:red}: [
{
"jsonClass": 
"org.apache.atlas.typesystem.json.InstanceSerialization$_Reference",
"id": {
  "jsonClass": "org.apache.atlas.typesystem.json.InstanceSerialization$_Id",
  "id": "ac85e13e-926f-4618-97f9-779cbcf23f49",
  "version": 0,
  "typeName": "DataSet",
  "state": "ACTIVE"
},
"typeName": "DataSet",
"values": {
  "qualifiedName": "default@cluster12",
  "owner": "hbase_admin",
  "description": "Default HBase namespace",
  "name": "default"
},
"traitNames": [],
"traits": {}
  },
{
  "jsonClass": 
"org.apache.atlas.typesystem.json.InstanceSerialization$_Reference",
  "id": {
"jsonClass": 
"org.apache.atlas.typesystem.json.InstanceSerialization$_Id",
"id": "23eeba58-2cf0-42bc-9bd0-54d0f9f2e0c8",
"version": 0,
"typeName": "DataSet",
"state": "ACTIVE"
  },
  "typeName": "DataSet",
  "values": {
"qualifiedName": "default@cluster11",
"owner": "hbase_admin",
"description": "Default HBase namespace",
"name": "default"
  },
  "traitNames": [],
  "traits": {}
}
  ]
}

> After updating/creating a set of entities, response contains the first entity 
> definition only
> -
>
> Key: ATLAS-1144
> URL: https://issues.apache.org/jira/browse/ATLAS-1144
> Project: Atlas
>  Issue Type: Bug
>Affects Versions: 0.7-incubating
>Reporter: Apoorv Naik
>Assignee: Apoorv Naik
> Fix For: 0.7-incubating
>
> Attachments: ATLAS_1144.patch
>
>
> Version: Atlas 0.7
> Steps to recreate:
> Create two entities (POST /entities) and note guid1 and guid2 that get 
> generated.
> Update these entities in the same PUT /entities and response contains 
> (truncated for convenience):
> {
>   "requestId": "...",
>   "GUID": [
> "guid1",
> "guid2"
>   ],
>   "definition": "{..*\"id\":\"guid1\"*,}
> }
> {color:red}Issue: definition does not contain the data for second 
> entity.{color}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (ATLAS-1161) Tags should be bound to an object's name and remain bound to all incarnations of that name

[Dan Markwat (JIRA)]

[ 
https://issues.apache.org/jira/browse/ATLAS-1161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15484294#comment-15484294
 ] 

Dan Markwat commented on ATLAS-1161:


[~davidrad], my thinking is - and it sounds something like as you describe - 
there would be a separate page in the app that takes a tag-sourced approach to 
managing or visualizing information (similar to the business taxonomy).  From 
this page, a user would have a view of all tags applied in the system with 
visual cues provided for which ones are tagged to live objects and which are 
not.  Administrators could then make any updates as they see fit.  
Additionally, this would facilitate easy navigation from a tag to any 
associated objects throughout history.

As for handling renames, I can't say...I had a very similar project I was on 
that dealt with this exact problem but the solution I used makes little sense 
here.  Would it be possible to support this as part of a "rename" operation?  
(Last I checked, it didn't look like Atlas had a [micro]-service style 
architecture, so not sure how easily that fits in?)  Since each hook would know 
what the physical operation in its system was, it stands to reason that the 
hook knows the Atlas-equivalent updates and operations needed to represent it.  
This might involve reverse lookups on the tags and then relinking to the 
newly-named object?  I'm just spit-balling here..  I suppose the tag could 
still bind to a GUID, but then an interesting query involving the name would 
still need to be done to accomplish the same thing.  A concept of "last object 
named X" is required to make this work, as far as I can tell.

> Tags should be bound to an object's name and remain bound to all incarnations 
> of that name
> --
>
> Key: ATLAS-1161
> URL: https://issues.apache.org/jira/browse/ATLAS-1161
> Project: Atlas
>  Issue Type: Improvement
>Affects Versions: trunk, 0.7-incubating
>Reporter: Dan Markwat
>
> As a user I would like tags I ascribe to an object in Atlas carry to the next 
> incarnation of that object.  In effect, tags would be ascribed to a 
> fully-qualified object name and all incarnations of that name would have the 
> tags apply to it.  (Not unlike Ranger and the way it applies policies to 
> objects).
> Example: I create a Hive table, TableA.  I tag TableA with tags, Tag1 and 
> Tag2.  I drop TableA.
> In the current Atlas world, if I create TableA again, Tag1 and Tag2 need to 
> be re-applied to TableA.  In the ideal governance/security world, if I 
> re-create TableA I should not have to re-tag it with Tag1 and Tag2; instead, 
> I should be required to *untag* TableA if I desire TableA to be clean and 
> untagged.  This effectively functions like a light switch: user turns on 
> light, just because the bulb is swapped out doesn't mean the switch turned 
> off - the user must explicitly turn the switch off, just as they did to turn 
> it on.  Think also about Ranger: just because I deleted an object doesn't 
> mean that policy goes away.
> By effectively deleting the binding of Tag1 and Tag2 to the name TableA 
> whenever TableA is deleted, Atlas ceases to be a book of record for tags 
> associated with TableA, as those tags would need to be applied again.  This 
> is bad in a world where creating/dropping objects and tagging objects are 
> part of 2 independent and asynchronous processes - one carried out by an 
> engineer, the other carried out by a governance/security administrator.  The 
> issue is compounded by the fact that tags can have security policies 
> associated with them in Ranger; and any object missing its tag at re-creation 
> of that object now is missing security policies previously attached to it.
> This is an especially annoying issue for organizations that have large 
> ingestion pipelines where tables are sometimes deleted or modified in ways 
> not easily accomplished through updating table metadata.  Not to mention, 
> (probably a new feature: ) easily-accessible records of what was tagged with 
> what - even if the object has been dropped or deleted - is especially 
> important for organizations that require auditing or have security controls 
> based on tag-based policies.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Review Request 51800: Support getTrait() API

[David Radley]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51800/#review148487
---



I am was thinking that a more intuitive API would be to follow the pattern used 
in the REST API for entities. So for entities with have 
.../entities/{entityguid}. So for consistancy we could have
.../entities/{entityguid}/traitnames which would return all the traitnames for 
the entity.   
.../entities/{entityguid}/traitnames/{traitname} which would return the 
specified traitname for the entity.

- David Radley


On Sept. 12, 2016, 11:09 a.m., Vimal Sharma wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51800/
> ---
> 
> (Updated Sept. 12, 2016, 11:09 a.m.)
> 
> 
> Review request for atlas.
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> Given entity id and trait name, support rest API that returns the trait 
> instance for the entity. Currently, the other way of getting it is 
> getEntity() which returns full entity with all trait instances
> 
> 
> Diffs
> -
> 
>   webapp/src/main/java/org/apache/atlas/web/resources/EntityResource.java 
> 82016d0 
>   
> webapp/src/test/java/org/apache/atlas/web/resources/EntityJerseyResourceIT.java
>  a1988ef 
> 
> Diff: https://reviews.apache.org/r/51800/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Vimal Sharma
> 
>

[jira] [Commented] (ATLAS-1161) Tags should be bound to an object's name and remain bound to all incarnations of that name

[David Radley (JIRA)]

[ 
https://issues.apache.org/jira/browse/ATLAS-1161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15483992#comment-15483992
 ] 

David Radley commented on ATLAS-1161:
-

Another thought. If we use the fully qualified name for this - how do we handle 
renames. Often we like to use a guid as the unique identifier to facilitate 
renames and moves.

> Tags should be bound to an object's name and remain bound to all incarnations 
> of that name
> --
>
> Key: ATLAS-1161
> URL: https://issues.apache.org/jira/browse/ATLAS-1161
> Project: Atlas
>  Issue Type: Improvement
>Affects Versions: trunk, 0.7-incubating
>Reporter: Dan Markwat
>
> As a user I would like tags I ascribe to an object in Atlas carry to the next 
> incarnation of that object.  In effect, tags would be ascribed to a 
> fully-qualified object name and all incarnations of that name would have the 
> tags apply to it.  (Not unlike Ranger and the way it applies policies to 
> objects).
> Example: I create a Hive table, TableA.  I tag TableA with tags, Tag1 and 
> Tag2.  I drop TableA.
> In the current Atlas world, if I create TableA again, Tag1 and Tag2 need to 
> be re-applied to TableA.  In the ideal governance/security world, if I 
> re-create TableA I should not have to re-tag it with Tag1 and Tag2; instead, 
> I should be required to *untag* TableA if I desire TableA to be clean and 
> untagged.  This effectively functions like a light switch: user turns on 
> light, just because the bulb is swapped out doesn't mean the switch turned 
> off - the user must explicitly turn the switch off, just as they did to turn 
> it on.  Think also about Ranger: just because I deleted an object doesn't 
> mean that policy goes away.
> By effectively deleting the binding of Tag1 and Tag2 to the name TableA 
> whenever TableA is deleted, Atlas ceases to be a book of record for tags 
> associated with TableA, as those tags would need to be applied again.  This 
> is bad in a world where creating/dropping objects and tagging objects are 
> part of 2 independent and asynchronous processes - one carried out by an 
> engineer, the other carried out by a governance/security administrator.  The 
> issue is compounded by the fact that tags can have security policies 
> associated with them in Ranger; and any object missing its tag at re-creation 
> of that object now is missing security policies previously attached to it.
> This is an especially annoying issue for organizations that have large 
> ingestion pipelines where tables are sometimes deleted or modified in ways 
> not easily accomplished through updating table metadata.  Not to mention, 
> (probably a new feature: ) easily-accessible records of what was tagged with 
> what - even if the object has been dropped or deleted - is especially 
> important for organizations that require auditing or have security controls 
> based on tag-based policies.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Review Request 51598: ATLAS-1133 : Jetty Server start doesn't throw exception when user-credential.properties file is not found

[Vimal Sharma]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51598/#review148479
---


Ship it!




Ship It!

- Vimal Sharma


On Sept. 2, 2016, 12:49 p.m., Nixon Rodrigues wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51598/
> ---
> 
> (Updated Sept. 2, 2016, 12:49 p.m.)
> 
> 
> Review request for atlas, Madhan Neethiraj, Shwetha GS, and Vimal Sharma.
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> This patch contains original fix ie to throw RuntimeException when 
> usercredentails file and polciystore file are not in filepath.
> Secondly this patch has the fix for path in UnitTest case and jsp compilation 
> issue by adding jetty-jsp as dependency in jetty-maven-plugin in pom.xml.
> 
> 
> Diffs
> -
> 
>   
> authorization/src/main/java/org/apache/atlas/authorize/simple/SimpleAtlasAuthorizer.java
>  30e2067 
>   webapp/pom.xml 708a216 
>   webapp/src/main/java/org/apache/atlas/web/dao/UserDao.java 6b83c21 
>   
> webapp/src/test/java/org/apache/atlas/web/security/NegativeSSLAndKerberosTest.java
>  4ad01cf 
> 
> Diff: https://reviews.apache.org/r/51598/diff/
> 
> 
> Testing
> ---
> 
> Executed *quick_start.py* script and tested the Atlas UI from distro module.
> Ran *mvn --projects webapp jetty:run* and tested atlas UI, created tags, 
> searched tags.
> Executed Unit testcase *mvn clean install*
> 
> 
> Thanks,
> 
> Nixon Rodrigues
> 
>

Review Request 51800: Support getTrait() API

[Vimal Sharma]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51800/
---

Review request for atlas.


Repository: atlas


Description
---

Given entity id and trait name, support rest API that returns the trait 
instance for the entity. Currently, the other way of getting it is getEntity() 
which returns full entity with all trait instances


Diffs
-

  webapp/src/main/java/org/apache/atlas/web/resources/EntityResource.java 
82016d0 
  
webapp/src/test/java/org/apache/atlas/web/resources/EntityJerseyResourceIT.java 
a1988ef 

Diff: https://reviews.apache.org/r/51800/diff/


Testing
---


Thanks,

Vimal Sharma

[jira] [Commented] (ATLAS-1149) Changes to UI to sort the hive table schema based on "position" attribute of hive_column

[Kalyani Kashikar (JIRA)]

[ 
https://issues.apache.org/jira/browse/ATLAS-1149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15483740#comment-15483740
 ] 

Kalyani Kashikar commented on ATLAS-1149:
-

 [~kevalbhatt] In entity details page column data sorting issue is fixed and 
patch is updated.

> Changes to UI to sort the hive table schema based on "position" attribute of 
> hive_column
> 
>
> Key: ATLAS-1149
> URL: https://issues.apache.org/jira/browse/ATLAS-1149
> Project: Atlas
>  Issue Type: Bug
>Affects Versions: 0.7-incubating
>Reporter: Sarath Subramanian
>Assignee: Kalyani Kashikar
>  Labels: ui
> Attachments: ATLAS-1149.1.patch, ATLAS-1149.patch, attach1.json, 
> attach2.json
>
>
> This is related to ATLAS-772, hive column needs to be sorted based on its 
> original column order. New attribute "position" has been added to hive_column 
> type which maintains the column order number. UI needs to use this 
> information to sort the column based on this attribute



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (ATLAS-1161) Tags should be bound to an object's name and remain bound to all incarnations of that name

[David Radley (JIRA)]

[ 
https://issues.apache.org/jira/browse/ATLAS-1161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15483640#comment-15483640
 ] 

David Radley commented on ATLAS-1161:
-

I can see that the relationship between an asset and a tag needs to be under 
control of govenrance lifecycles and not tied to the asset lifetime. 

The advantage of the current approach is that if a table is dropped then its 
associated tag relationships go as well. So we are not left with ghost tag to 
assets mappings to a non-existance asset. Also if TableA is recreated with a 
different shape we have not got out of date tag to asset mappings. 

If we make a change like this for the reasons you mention, we need to ensure 
that ghost mappings and out of sync mappings are handled in the fix design. If 
this use case is important, I suggest introducing asset type or asset to tag 
type, which has a tag lifetime rather than asset lifetime.  
  

   

> Tags should be bound to an object's name and remain bound to all incarnations 
> of that name
> --
>
> Key: ATLAS-1161
> URL: https://issues.apache.org/jira/browse/ATLAS-1161
> Project: Atlas
>  Issue Type: Improvement
>Affects Versions: trunk, 0.7-incubating
>Reporter: Dan Markwat
>
> As a user I would like tags I ascribe to an object in Atlas carry to the next 
> incarnation of that object.  In effect, tags would be ascribed to a 
> fully-qualified object name and all incarnations of that name would have the 
> tags apply to it.  (Not unlike Ranger and the way it applies policies to 
> objects).
> Example: I create a Hive table, TableA.  I tag TableA with tags, Tag1 and 
> Tag2.  I drop TableA.
> In the current Atlas world, if I create TableA again, Tag1 and Tag2 need to 
> be re-applied to TableA.  In the ideal governance/security world, if I 
> re-create TableA I should not have to re-tag it with Tag1 and Tag2; instead, 
> I should be required to *untag* TableA if I desire TableA to be clean and 
> untagged.  This effectively functions like a light switch: user turns on 
> light, just because the bulb is swapped out doesn't mean the switch turned 
> off - the user must explicitly turn the switch off, just as they did to turn 
> it on.  Think also about Ranger: just because I deleted an object doesn't 
> mean that policy goes away.
> By effectively deleting the binding of Tag1 and Tag2 to the name TableA 
> whenever TableA is deleted, Atlas ceases to be a book of record for tags 
> associated with TableA, as those tags would need to be applied again.  This 
> is bad in a world where creating/dropping objects and tagging objects are 
> part of 2 independent and asynchronous processes - one carried out by an 
> engineer, the other carried out by a governance/security administrator.  The 
> issue is compounded by the fact that tags can have security policies 
> associated with them in Ranger; and any object missing its tag at re-creation 
> of that object now is missing security policies previously attached to it.
> This is an especially annoying issue for organizations that have large 
> ingestion pipelines where tables are sometimes deleted or modified in ways 
> not easily accomplished through updating table metadata.  Not to mention, 
> (probably a new feature: ) easily-accessible records of what was tagged with 
> what - even if the object has been dropped or deleted - is especially 
> important for organizations that require auditing or have security controls 
> based on tag-based policies.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (ATLAS-1098) Atlas allows creation of tag with name "isa" which causes exceptions during search

[David Radley (JIRA)]

[ 
https://issues.apache.org/jira/browse/ATLAS-1098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15483587#comment-15483587
 ] 

David Radley commented on ATLAS-1098:
-

Hello,
I have just seen the commit going through for this. It looks like a tag of isa, 
clashed with a keyword in Gremlin and the fix was to police out these keywords. 
My feeling is that there should be no restrictions to tag names. If we want 
Atlas to be a widely used metadata server, I do not think it should have 
restrictions on the use of any metadata values (which I guess we would need to 
document) , based on restrictions passed through from the implementation. 
Without any restrictions metadata can be modelled intuitively without needing 
to be aware  and needing to compensate for the "quirks" of Atlas.  
  all the best, David. 

> Atlas allows creation of tag with name "isa" which causes exceptions during 
> search
> --
>
> Key: ATLAS-1098
> URL: https://issues.apache.org/jira/browse/ATLAS-1098
> Project: Atlas
>  Issue Type: Bug
>Reporter: Sharmadha Sainath
>Assignee: Apoorv Naik
>
> Created a tag with name "isa". DSL query "Table where Table isa isa" throws 
> exception . Search in Tag tab also throws exception.
>  



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

concern about ATLAS-1098

[David Radley]

Hello,
I have just seen the commit going through for ATLAS-1098. It looks like a 
tag of isa, clashed with a keyword in Gremlin and the fix was to police 
out these keywords. My feeling is that there should be no restrictions to 
tag names. If we want Atlas to be a widely used metadata server, I do not 
think it should have restrictions on the use of any metadata values (which 
I guess we would need to document) , based on restrictions passed through 
from the implementation. Without any restrictions metadata can be modelled 
intuitively without needing to be aware  and needing to compensate for the 
"quirks" of Atlas. 
  all the best, David. 

Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number 
741598. 
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

Re: Review Request 51723: ATLAS-1149 : Changes to UI to sort the hive table schema based on "position" attribute of hive_column

[Kalyani Kashikar]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51723/
---

(Updated Sept. 12, 2016, 9:05 a.m.)


Review request for atlas, keval bhatt, Madhan Neethiraj, Sarath Kumar 
Subramanian, Shwetha GS, and Suma Shivaprasad.


Bugs: ATLAS-1149
https://issues.apache.org/jira/browse/ATLAS-1149


Repository: atlas


Description
---

In hive column needs to be sorted based on its original column order. New 
attribute "position" has been added to hive_column type which maintains the 
column order number. UI needs to use this information to sort the column based 
on this attribute on schema table & entity details page.

This patch includes changes for sort column based on "position" attribute


Diffs (updated)
-

  dashboardv2/public/js/collection/BaseCollection.js b039e30 
  dashboardv2/public/js/views/entity/EntityDetailTableLayoutView.js cc67d01 
  dashboardv2/public/js/views/schema/SchemaLayoutView.js dc19619 

Diff: https://reviews.apache.org/r/51723/diff/


Testing
---

Added hive-brigde jar in atlas-impl dir in hive hook
Created new attribute “position” for hive_column type with curl command.
Tested entity details page UI for hive_table entities by adding new tables in 
hive beeline client.
In schema table the db column are sorted according to position attribute in api.


Thanks,

Kalyani Kashikar

[jira] [Commented] (ATLAS-1149) Changes to UI to sort the hive table schema based on "position" attribute of hive_column

[Shwetha G S (JIRA)]

[ 
https://issues.apache.org/jira/browse/ATLAS-1149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15483537#comment-15483537
 ] 

Shwetha G S commented on ATLAS-1149:


The UI shouldn't do the sort. Instead we should modify the query used for this 
(atlas.lineage.schema.query.hive_table=hive_table where __guid='%s'\, columns) 
to sort by position, so that all clients have the same info. Also, the query 
should filter deleted columns 

> Changes to UI to sort the hive table schema based on "position" attribute of 
> hive_column
> 
>
> Key: ATLAS-1149
> URL: https://issues.apache.org/jira/browse/ATLAS-1149
> Project: Atlas
>  Issue Type: Bug
>Affects Versions: 0.7-incubating
>Reporter: Sarath Subramanian
>Assignee: Kalyani Kashikar
>  Labels: ui
> Attachments: ATLAS-1149.patch, attach1.json, attach2.json
>
>
> This is related to ATLAS-772, hive column needs to be sorted based on its 
> original column order. New attribute "position" has been added to hive_column 
> type which maintains the column order number. UI needs to use this 
> information to sort the column based on this attribute



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (ATLAS-1098) Atlas allows creation of tag with name "isa" which causes exceptions during search

[Shwetha G S (JIRA)]

[ 
https://issues.apache.org/jira/browse/ATLAS-1098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15483468#comment-15483468
 ] 

Shwetha G S commented on ATLAS-1098:


Sorry for the late review as I was away last few days. I have added comments on 
reviewboard, can you address them?

> Atlas allows creation of tag with name "isa" which causes exceptions during 
> search
> --
>
> Key: ATLAS-1098
> URL: https://issues.apache.org/jira/browse/ATLAS-1098
> Project: Atlas
>  Issue Type: Bug
>Reporter: Sharmadha Sainath
>Assignee: Apoorv Naik
>
> Created a tag with name "isa". DSL query "Table where Table isa isa" throws 
> exception . Search in Tag tab also throws exception.
>  



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Review Request 51660: [ATLAS-1098] Added keyword checks while creating a new tag/trait

[Shwetha GS]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51660/#review148465
---




repository/src/main/java/org/apache/atlas/services/DefaultMetadataService.java 
(lines 246 - 247)


Can you also add tests for this?



repository/src/main/java/org/apache/atlas/services/DefaultMetadataService.java 
(line 295)


TypeSystem.verifyTypes() has all the validations. Please move this logic 
there.


- Shwetha GS


On Sept. 6, 2016, 11:27 p.m., Apoorv Naik wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51660/
> ---
> 
> (Updated Sept. 6, 2016, 11:27 p.m.)
> 
> 
> Review request for atlas, Madhan Neethiraj, Shwetha GS, Suma Shivaprasad, and 
> Vimal Sharma.
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> [ATLAS-1098] Added keyword checks while creating a new tag/trait
> 
> 
> Diffs
> -
> 
>   
> repository/src/main/java/org/apache/atlas/services/DefaultMetadataService.java
>  4d05d49 
> 
> Diff: https://reviews.apache.org/r/51660/diff/
> 
> 
> Testing
> ---
> 
> Tested with all possible keywords specified in the QueryParser, the UI shows 
> a red exception message stating that the type being created is a keyword
> 
> 
> Thanks,
> 
> Apoorv Naik
> 
>

[jira] [Comment Edited] (ATLAS-1098) Atlas allows creation of tag with name "isa" which causes exceptions during search

[Shwetha G S (JIRA)]

[ 
https://issues.apache.org/jira/browse/ATLAS-1098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15483078#comment-15483078
 ] 

Shwetha G S edited comment on ATLAS-1098 at 9/12/16 8:23 AM:
-

[~apoorvnaik], please link the review board request to the jira and add the 
patch in the jira for your future patches. Thanks


was (Author: shwethags):
[~apoorvnaik], please link the review board request to the jira for your future 
patches. Thanks

> Atlas allows creation of tag with name "isa" which causes exceptions during 
> search
> --
>
> Key: ATLAS-1098
> URL: https://issues.apache.org/jira/browse/ATLAS-1098
> Project: Atlas
>  Issue Type: Bug
>Reporter: Sharmadha Sainath
>Assignee: Apoorv Naik
>
> Created a tag with name "isa". DSL query "Table where Table isa isa" throws 
> exception . Search in Tag tab also throws exception.
>  



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)