[jira] [Comment Edited] (ATLAS-1277) Add feather use 'order by ' in the DSL search

[Qiang Zhang (JIRA)]

[ 
https://issues.apache.org/jira/browse/ATLAS-1277?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15741307#comment-15741307
 ] 

Qiang Zhang edited comment on ATLAS-1277 at 12/20/16 8:23 AM:
--

[~guptaneeru] Hi,Thanks for you review,I have removed The tab space and 
ATLAS-1277-2.patch is the new patch  I have make new patch as you say .Please 
review again!


was (Author: zhangqiang2):
[~guptaneeru] Hi,Thanks for you review,I have make new patch as you say .Please 
review again!

> Add  feather use 'order by ' in the DSL search 
> ---
>
> Key: ATLAS-1277
> URL: https://issues.apache.org/jira/browse/ATLAS-1277
> Project: Atlas
>  Issue Type: New Feature
>Affects Versions: 0.8-incubating
>Reporter: Qiang Zhang
>Assignee: Qiang Zhang
> Fix For: 0.8-incubating
>
> Attachments: ATLAS-1277-2.patch
>
>
> in the Search using DSL,sort with 'orderby' ,but with :'order by ' is  a more 
> customary way ,so consider add the 'order by '  is more desirable.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Assigned] (ATLAS-1344) Allow types to be created that support parent child hierarchies.

[David Radley (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-1344?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

David Radley reassigned ATLAS-1344:
---

Assignee: David Radley

> Allow types to be created that support parent child hierarchies.
> 
>
> Key: ATLAS-1344
> URL: https://issues.apache.org/jira/browse/ATLAS-1344
> Project: Atlas
>  Issue Type: Improvement
>Reporter: David Radley
>Assignee: David Radley
>
> enhance the Type system to allow entities to be created in parent child 
> hierarchies. 
> I notice that AttributeDefinition currently only support  COLLECTION, SET , 
> OPTIONAL and REQUIRED. There is no collection with a lower bound of 0- which 
> would be required for a parent child type relationship. 
> Also it seems to me that it would be ideal to create an attribute with the 
> guid of the parent / child and then derive the other one from the edge. I may 
> have misunderstood but reverseAttributeName seems to use the name of the 
> attribute not the guid.
> I suggest that iscomposite true and false is supported. 
> When we have this in place, we could use this mechanism for Glossary 
> Categories  
> 
>   



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Request for Atlas wiki or blog we can put Eclipse setup instructions on?

[David Radley]

Hi Hemanth,
Thank you for your considered reply. 

It seems to me that there are a spectrum of types of documentation, from 
formal bound manuals of old to tweets. My preference is to strive to 
embrace social and open up the documentation to be able to be updated by 
as many people as possible. It would be great if there was a vibrant blog, 
wiki culture around Atlas in addition to the dev list. I am thinking of 
Wikipedia as a great example of a self policing community - very much in 
line with Apache culture. 

I notice that there is https://blogs.apache.org/ we could use to blog 
about developer setups and the like. I guess this approach means there is 
less control of the documentation but it is more inclusive. I notice 
Apache Httpd has a wiki where users can contribute 
https://httpd.apache.org/docs/.

You raise a good question around workflow; the question I see is, how much 
control do we need around it. 

I can see API references like javadoc or Swagger generated from the code 
as living in Git and shipping with the product. Guides , designs, dev 
environment setup, performance tips, integration scenarios, new 
connectors, FVT testing suites would seem to me to benefit from being in 
blogs / wikis that many people could create,  edit, keep up to date. 

In the meantime I will create a Twiki file for the Eclipse information, to 
get it out, 
 all the best, David. 




From:   Hemanth Yamijala 
To: David Radley/UK/IBM@IBMGB
Cc: "dev@atlas.incubator.apache.org" 
Date:   20/12/2016 04:33
Subject:Re: Request for Atlas wiki or blog we can put Eclipse 
setup instructions on?



David,


I am including back dev@atlas. I think you are proposing a change to 
making the edits simpler from the current process, hence would be nice if 
other people can chime in with their opinion.​


I guess we get some advantages in doing it the current way:


* We can follow all current processes for patch review & commit for docs 
as for source. Indeed, it would be nice to update documentation whenever 
sources are updated in the same patch. I know some Apache projects did 
insist on this practice a while back.

* Versioning comes for free and in well defined manners (although, I 
believe all Wiki systems provide versioning as well - so may not be a big 
issue)

* Docs can be distributed with the source package of Atlas and built and 
deployed independently. (again may not be a big advantage)


In your proposal, I am not still clear on the workflow for a doc update - 
in particular, how the review / commit would work. So would be nice to 
understand that a bit more in detail.


I am not sure if the choice of doing documentation this way was a 
deliberate one given the above or similar advantages. Maybe someone more 
in the know-how can elaborate.


Thanks

hemanth



From: David Radley 
Sent: Monday, December 19, 2016 11:11 PM
To: Hemanth Yamijala
Subject: Re: Request for Atlas wiki or blog we can put Eclipse setup 
instructions on?

Hi Hemanth,
Ok thanks.

I have registered in twiki.org , made my changes in Microsoft Word, pasted 
them into twiki sandbox and then switched views to cut and paste the raw 
format out. This seems to work, though I need to manually change the size 
of the titles to fit what you have already done.

If I read this correctly, I assume the way we should be running this, is 
to the download and install Twiki on the web server running the Atlas site 
and then we could expose the rich text Wiki editor on the web site.  Maybe 
only allow committers create and update access for the wiki. The wiki 
would be the master then not Git.

all the best, David.







From:Hemanth Yamijala 
To:David Radley/UK/IBM@IBMGB
Cc:"dev@atlas.incubator.apache.org" 

Date:19/12/2016 15:59
Subject:Re: Request for Atlas wiki or blog we can put Eclipse 
setup instructions on?




David,

I have indeed edited files directly.. There are only a handful of syntax 
elements we need to know and the help for these is available in many 
online sources (e.g. 
http://twiki.org/cgi-bin/view/TWiki/TextFormattingRules)<
http://twiki.org/cgi-bin/view/TWiki/TextFormattingRules>.

Atlas docs can be built by running mvn docs (or something similar) and the 
files can be checked online. I don't know if there are twiki editors that 
make this easier to manage. Maybe others know?

Thanks
hemanth



From: David Radley 
Sent: Monday, December 19, 2016 9:11 PM
To: Hemanth Yamijala
Cc: dev@atlas.incubator.apache.org
Subject: Fw: Request for Atlas wiki or blog we can put Eclipse setup 
instructions on?

Hi Hemanth,
I have not worked with Twiki files before - I assume people do not work 
directly with the raw format I get from Git (Eclipse shows me the wiki in 
a preview panel but does not give me a wiki rich text editor). I wondered 
what development software / process you and community use to 

[jira] [Created] (ATLAS-1406) kafka NotificationException

[roxia (JIRA)]

roxia created ATLAS-1406:


 Summary: kafka NotificationException
 Key: ATLAS-1406
 URL: https://issues.apache.org/jira/browse/ATLAS-1406
 Project: Atlas
  Issue Type: Bug
Affects Versions: 0.8-incubating
Reporter: roxia
 Fix For: 0.8-incubating


when i create a entities ,the application.log is :
org.apache.atlas.notification.NotificationException: 
java.util.concurrent.ExecutionException: 
org.apache.kafka.common.errors.TimeoutException: Failed to update metadata 
after 6 ms

my atlas_application.properties is :
atlas.notification.embedded=false
atlas.kafka.data=${sys:atlas.home}/data/kafka
atlas.kafka.zookeeper.connect=hadoop-s1:2181,hadoop-s2:2181,hadoop-s3:2181
atlas.kafka.bootstrap.servers=10.224.2.233:9093
atlas.kafka.zookeeper.session.timeout.ms=600
atlas.kafka.zookeeper.connection.timeout.ms=300
atlas.kafka.zookeeper.sync.time.ms=20
atlas.kafka.auto.commit.interval.ms=1000
atlas.kafka.auto.offset.reset=smallest
atlas.kafka.hook.group.id=atlas
atlas.kafka.auto.commit.enable=false
atlas.notification.create.topics=true
atlas.notification.replicas=1
atlas.notification.topics=ATLAS_HOOK,ATLAS_ENTITIES
atlas.notification.log.failed.messages=true
atlas.notification.consumer.retry.interval=500
atlas.notification.hook.retry.interval=1000



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Improvement suggestion: change terms to be implemented as entities

[David Radley]

Hi Madhan,
Thanks for joining the discussion. 

As you are probably aware ATLAS-1186 is currently in review and introduces 
a parent child containment concept; this Jira is a pragmatic minimal 
change to get the functionality out. Longer term, I think we need support 
for optional multiple composition relationships in types. This would allow 
us to model parent child relationships in the type system. I have raised 
this in Jira  ATLAS- 1344. Once this is in place, the glossary objects 
could use that implementation.

For  ATLAS-1186 we manage the graph edges separately and use projections - 
similar to taxonomies. If you look in Jira 1186,  you will see a series of 
follow on Jiras listed  that are on in this area- all looking to make the 
Atlas glossary more mature,

My thinking on term to term relationships are in Jira 1254, where we are 
looking to add: 
1) has a 
2) antonym
3) homonym
4) replaces / replaced by. So the glossary can contain replacement 
knowledge explicitly rather than in rename history. 
5) relationship to 0 or more other glossary categories
6) a verb relationship . for example to be able to hold the information 
behind a statement like ' The customer C owns account A.'  

We could implement all of this in the glossary category style in 1186. Or 
we could enhance the type system as per 1144 to add in support for 
optional multiple composite relationships between entities. 

many thanks, David. 



From:   Madhan Neethiraj 
To: "dev@atlas.incubator.apache.org" , 
Shwetha Shivalingamurthy 
Date:   20/12/2016 02:49
Subject:Re: Improvement suggestion: change terms to be implemented 
as entities
Sent by:Madhan Neethiraj 



David,

Thanks for starting this thread with a lot of good details. I agree on 
modelling terms as entities instead of traits. This approach provides many 
flexibilities – like rename, move, delete.

In addition to support for has-a relationship between entities, I think we 
might need support for “parent” or “container” relationship – to capture 
the hierarchical organization of the terms in taxonomies.

Please give me couple of days to go through all the details and respond 
back.

Thanks,
Madhan

On 12/13/16, 6:23 AM, "David Radley"  wrote:

Hi Shweta,
Thank you, I had not considered the search side of things. I suggest 
that 
for a minium viable change for the first check in, we do not change 
this 
behavior; this may mean we need to special case the code to ensure 
that 
isa still performs as it did.
 
I do think that search needs to deal with taxonomies , glossary 
categories 
and terms as first level objects. I have raised Jira ATLAS-1372 to 
track 
this requirement; I think we should do this after ATLAS-1186 has gone 
into 
the code so it can consider glossary categories.
 
 In line with the way  I proposed in ATLAS-1186 and ATLAS-1327, I 
think we 
should also change terms to have a name (which is a simple name) and a 

fully qualified name (which we derive at runtime for the term 
inheritance 
hierarchy); we can do this when there is a unique id that identifies 
the 
term not the name. I think this term name change would be best done in 
a 
follow-on Jira (I have raise ATLAS-1373) as this would change the term 
API 
and may require an API version change.  I think we would need the term 

name change to occur before we could think about allowing terms to 
change 
taxonomies. 
 
   all the best,  David. 
 
 
 
 
 
 
From:   Shwetha Shivalingamurthy 
To: "dev@atlas.incubator.apache.org" 

Date:   13/12/2016 04:43
Subject:Re: Improvement suggestion: change terms to be 
implemented 
as entities
 
 
 
Modeling terms as traits also enabled search work out of the box. For
example, queries like search for assets with term will map to ŒAsset 
isa
¹ (though this worked only for leaf terms)
 
Modeling terms as entities will simplify some of the functionalities 
like
term renames, move term from one hierarchy to the other etc. Are you
planning to expose different way of searching or use existing search 
like
ŒAsset where terms = ¹?
 
Regards,
Shwetha
 
 
 
 
 
 
On 13/12/16, 7:50 AM, "Hemanth Yamijala"  
wrote:
 
>David,
>
>I hope folks who are more plugged into Atlas on a day-to-day basis 
will
>provide relevant feedback. I have a very few comments below.
>
>Regarding point 10: AFAIK, the most significant constraint of
>implementing terms as entities was that entity to entity 
relationships
>needed to be predefined, while tags / traits could be associated to 
any
>entity without this prior definition.
>
>Regarding point 7: Tags and traits are indeed interchangeable. In the
>Atlas UI specifically, we always refer to trait types as tags (which 
is
>confusing IMO, but well, that's where we are)
>
>Thanks
>hemanth
>___

[jira] [Updated] (ATLAS-1401) Document in detail how to set up Eclipse for Atlas dev environment.

[David Radley (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-1401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

David Radley updated ATLAS-1401:

Attachment: rb54899.patch

> Document in detail how to set up Eclipse for Atlas dev environment.
> ---
>
> Key: ATLAS-1401
> URL: https://issues.apache.org/jira/browse/ATLAS-1401
> Project: Atlas
>  Issue Type: Improvement
>Reporter: David Radley
>Assignee: David Radley
> Attachments: rb54899.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (ATLAS-1344) Allow types to be created that support parent child hierarchies.

[Madhan Neethiraj (JIRA)]

[ 
https://issues.apache.org/jira/browse/ATLAS-1344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15764607#comment-15764607
 ] 

Madhan Neethiraj commented on ATLAS-1344:
-

[~davidrad] with enhancements in ATLAS-1171, 
org.apache.atlas.typesystem.types.AttributeDefinition.java is replaced by 
org.apache.atlas.model.typedef.AtlasStructDef.AtlasAttributeDef. 
AtlasAttributeDef supports the notion of "constraints" - which can used to 
model parent-child relationships. Currently, "constraints" is used to replace 
existing fields "isComposite" and "reverseAttributeName" with "foreignKey" and 
"mappedFromRef" constraints. addons/models/0030-hive_model.json is a good 
reference for such usage.

Please review and let me know if you have any comments/questions.

> Allow types to be created that support parent child hierarchies.
> 
>
> Key: ATLAS-1344
> URL: https://issues.apache.org/jira/browse/ATLAS-1344
> Project: Atlas
>  Issue Type: Improvement
>Reporter: David Radley
>Assignee: David Radley
>
> enhance the Type system to allow entities to be created in parent child 
> hierarchies. 
> I notice that AttributeDefinition currently only support  COLLECTION, SET , 
> OPTIONAL and REQUIRED. There is no collection with a lower bound of 0- which 
> would be required for a parent child type relationship. 
> Also it seems to me that it would be ideal to create an attribute with the 
> guid of the parent / child and then derive the other one from the edge. I may 
> have misunderstood but reverseAttributeName seems to use the name of the 
> attribute not the guid.
> I suggest that iscomposite true and false is supported. 
> When we have this in place, we could use this mechanism for Glossary 
> Categories  
> 
>   



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (ATLAS-1407) Improve LOG statement performance

[Apoorv Naik (JIRA)]

Apoorv Naik created ATLAS-1407:
--

 Summary: Improve LOG statement performance
 Key: ATLAS-1407
 URL: https://issues.apache.org/jira/browse/ATLAS-1407
 Project: Atlas
  Issue Type: Improvement
Affects Versions: 0.8-incubating
Reporter: Apoorv Naik
Assignee: Apoorv Naik
 Fix For: 0.8-incubating


Currently lot of LOG statements use string concatenation which increases the 
overhead of the string computation regardless of the LOG level being enabled or 
disabled. 

This change converts all messages into a parameterized message which allows the 
logging system to only construct the message if the logging level is enabled.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (ATLAS-1344) Allow types to be created that support parent child hierarchies.

[Madhan Neethiraj (JIRA)]

[ 
https://issues.apache.org/jira/browse/ATLAS-1344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15764607#comment-15764607
 ] 

Madhan Neethiraj edited comment on ATLAS-1344 at 12/20/16 4:37 PM:
---

[~davidrad] - with enhancements in ATLAS-1171, 
org.apache.atlas.typesystem.types.AttributeDefinition.java is replaced by 
org.apache.atlas.model.typedef.AtlasStructDef.AtlasAttributeDef. 
AtlasAttributeDef supports the notion of "constraints" - which can used to 
model parent-child relationships. Currently, "constraints" is used to replace 
existing fields "isComposite" and "reverseAttributeName" with "foreignKey" and 
"mappedFromRef" constraints. addons/models/0030-hive_model.json is a good 
reference for such usage.

Please review and let me know if you have any comments/questions.


was (Author: madhan.neethiraj):
[~davidrad] with enhancements in ATLAS-1171, 
org.apache.atlas.typesystem.types.AttributeDefinition.java is replaced by 
org.apache.atlas.model.typedef.AtlasStructDef.AtlasAttributeDef. 
AtlasAttributeDef supports the notion of "constraints" - which can used to 
model parent-child relationships. Currently, "constraints" is used to replace 
existing fields "isComposite" and "reverseAttributeName" with "foreignKey" and 
"mappedFromRef" constraints. addons/models/0030-hive_model.json is a good 
reference for such usage.

Please review and let me know if you have any comments/questions.

> Allow types to be created that support parent child hierarchies.
> 
>
> Key: ATLAS-1344
> URL: https://issues.apache.org/jira/browse/ATLAS-1344
> Project: Atlas
>  Issue Type: Improvement
>Reporter: David Radley
>Assignee: David Radley
>
> enhance the Type system to allow entities to be created in parent child 
> hierarchies. 
> I notice that AttributeDefinition currently only support  COLLECTION, SET , 
> OPTIONAL and REQUIRED. There is no collection with a lower bound of 0- which 
> would be required for a parent child type relationship. 
> Also it seems to me that it would be ideal to create an attribute with the 
> guid of the parent / child and then derive the other one from the edge. I may 
> have misunderstood but reverseAttributeName seems to use the name of the 
> attribute not the guid.
> I suggest that iscomposite true and false is supported. 
> When we have this in place, we could use this mechanism for Glossary 
> Categories  
> 
>   



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (ATLAS-1406) kafka NotificationException

[Apoorv Naik (JIRA)]

[ 
https://issues.apache.org/jira/browse/ATLAS-1406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15764694#comment-15764694
 ] 

Apoorv Naik commented on ATLAS-1406:


How are you starting your server ? Also what's the command that you're using to 
build a war ?

> kafka NotificationException
> ---
>
> Key: ATLAS-1406
> URL: https://issues.apache.org/jira/browse/ATLAS-1406
> Project: Atlas
>  Issue Type: Bug
>Affects Versions: 0.8-incubating
>Reporter: roxia
> Fix For: 0.8-incubating
>
>
> when i create a entities ,the application.log is :
> org.apache.atlas.notification.NotificationException: 
> java.util.concurrent.ExecutionException: 
> org.apache.kafka.common.errors.TimeoutException: Failed to update metadata 
> after 6 ms
> my atlas_application.properties is :
> atlas.notification.embedded=false
> atlas.kafka.data=${sys:atlas.home}/data/kafka
> atlas.kafka.zookeeper.connect=hadoop-s1:2181,hadoop-s2:2181,hadoop-s3:2181
> atlas.kafka.bootstrap.servers=10.224.2.233:9093
> atlas.kafka.zookeeper.session.timeout.ms=600
> atlas.kafka.zookeeper.connection.timeout.ms=300
> atlas.kafka.zookeeper.sync.time.ms=20
> atlas.kafka.auto.commit.interval.ms=1000
> atlas.kafka.auto.offset.reset=smallest
> atlas.kafka.hook.group.id=atlas
> atlas.kafka.auto.commit.enable=false
> atlas.notification.create.topics=true
> atlas.notification.replicas=1
> atlas.notification.topics=ATLAS_HOOK,ATLAS_ENTITIES
> atlas.notification.log.failed.messages=true
> atlas.notification.consumer.retry.interval=500
> atlas.notification.hook.retry.interval=1000



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Review Request 54906: ATLAS-1407: Improve LOG statement performance

[Apoorv Naik]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54906/
---

Review request for atlas, Madhan Neethiraj, Sarath Subramanian, Suma 
Shivaprasad, and Vimal Sharma.


Bugs: ATLAS-1407
https://issues.apache.org/jira/browse/ATLAS-1407


Repository: atlas


Description
---

ATLAS-1407: Improve LOG statement performance


Diffs
-

  
addons/falcon-bridge-shim/src/main/java/org/apache/atlas/falcon/service/AtlasService.java
 6b4486a23ea945d24ef02143cd26d3aa47b11cb3 
  
addons/falcon-bridge/src/main/java/org/apache/atlas/falcon/hook/FalconHook.java 
b6312d73aa116a61e2898f60c511e4fe73ab9505 
  
addons/falcon-bridge/src/test/java/org/apache/atlas/falcon/hook/FalconHookIT.java
 2acc5755ba62c5368d19525709a4abbbff52c4b0 
  
addons/hive-bridge-shim/src/main/java/org/apache/atlas/hive/hook/HiveHook.java 
f4ca94fd8e7af9e6a787266020f2b90d3c60462b 
  
addons/hive-bridge/src/main/java/org/apache/atlas/hive/bridge/HiveMetaStoreBridge.java
 0f8afd55138f9af0ec66ba30967157fa0b1a7140 
  addons/hive-bridge/src/test/java/org/apache/atlas/hive/HiveITBase.java 
4b736a875135fc8b59d3fe86fcbf394207cf4bc7 
  
addons/sqoop-bridge-shim/src/main/java/org/apache/atlas/sqoop/hook/SqoopHook.java
 1170cb64c9fef0b72ccc024c5662ed50a5d26457 
  
addons/storm-bridge-shim/src/main/java/org/apache/atlas/storm/hook/StormAtlasHook.java
 2d304c5cdc5be4a848b226632a117ab0b58b746e 
  
authorization/src/main/java/org/apache/atlas/authorize/AtlasAuthorizerFactory.java
 5aad72d6e1989a0ab49a8d5d4bd273aa37b51adf 
  
authorization/src/main/java/org/apache/atlas/authorize/simple/AtlasAuthorizationUtils.java
 2ef4ea28c8d6786b6fe30067f27e9e5174104721 
  
authorization/src/main/java/org/apache/atlas/authorize/simple/PolicyParser.java 
5740a1cf500b5a8a90f2e9d4371bb10b18e01a38 
  authorization/src/main/java/org/apache/atlas/authorize/simple/PolicyUtil.java 
8a3850fae0d2c737e3a15991402eed48c177c667 
  
authorization/src/main/java/org/apache/atlas/authorize/simple/SimpleAtlasAuthorizer.java
 5956f5c9a4c580a96245b71c8182b5ddb486bc49 
  catalog/src/main/java/org/apache/atlas/catalog/query/QueryFactory.java 
a301912e2e1a0fbf3f5c5bfacc03d65f2390bbd7 
  common/src/main/java/org/apache/atlas/security/InMemoryJAASConfiguration.java 
ed42aa65e7b99058d44f50f02235d72c6b89b12d 
  common/src/main/java/org/apache/atlas/utils/AtlasPerfTracer.java 
05ec7f16276aa65559be483cf1523e83a14856b6 
  intg/src/test/java/org/apache/atlas/model/ModelTestUtil.java 
ee78350cd8501bd54d08851b0350fc55471e87b7 
  notification/src/main/java/org/apache/atlas/hook/AtlasHook.java 
5bdd5d3446fd409c0c13474433f2890efee2b851 
  notification/src/main/java/org/apache/atlas/kafka/KafkaNotification.java 
0ebfd47897ff2fce28320be0153125ac8c7b4dde 
  
plugin-classloader/src/main/java/org/apache/atlas/plugin/classloader/AtlasPluginClassLoader.java
 0dbf3528788bd3f63d97eb6286c52f661f6d2cd1 
  
plugin-classloader/src/main/java/org/apache/atlas/plugin/classloader/AtlasPluginClassLoaderUtil.java
 69b61d61aca8093cd25dda2821e13fcbbc9ac4bc 
  repository/src/main/java/org/apache/atlas/GraphTransactionInterceptor.java 
a7124bf8690f5f6a33e1d7a04aaaec976167ec6f 
  
repository/src/main/java/org/apache/atlas/repository/graph/GraphBackedMetadataRepository.java
 f0647a47221cf8bd13d8b68f12f25f4e721249f0 
  repository/src/main/java/org/apache/atlas/repository/graph/GraphHelper.java 
6eeddaea9d1868f01090dbe37a23ccda5fccf0f9 
  
repository/src/main/java/org/apache/atlas/repository/graph/TypedInstanceToGraphMapper.java
 78e276e5972f6bba68198b3ae5a3d88676cf7585 
  
repository/src/main/java/org/apache/atlas/repository/store/bootstrap/AtlasTypeDefStoreInitializer.java
 b3531b06967b99743a3c6a54b5fe85dff208b8eb 
  
repository/src/main/java/org/apache/atlas/services/DefaultMetadataService.java 
c2f8b3f9785db0d0302ddf181e1e12427562e6ca 
  server-api/src/main/java/org/apache/atlas/aspect/AtlasAspect.java 
28017500db3ddaf2d9636d11d3b6c8597dd52394 
  server-api/src/main/java/org/apache/atlas/ha/AtlasServerIdSelector.java 
f3d36a7140725ef2452d8a4a3b34e9edfe065858 
  
webapp/src/main/java/org/apache/atlas/notification/NotificationHookConsumer.java
 a4fd1c2fc77e05f69e26f8ac5bb4b1feabab3606 
  webapp/src/main/java/org/apache/atlas/web/dao/UserDao.java 
8f6613af7a357ea90f34534309c8c5d67f2b6582 
  
webapp/src/main/java/org/apache/atlas/web/filters/AtlasAuthenticationFilter.java
 b7528104c54c01fc125078e81592b2359241d02f 
  
webapp/src/main/java/org/apache/atlas/web/filters/AtlasAuthorizationFilter.java 
26e6b1e2ffc2512ed7039b2e0d309dfdb45a1353 
  
webapp/src/main/java/org/apache/atlas/web/filters/AtlasKnoxSSOAuthenticationFilter.java
 75a884dca7f0698a00aaed027b5b9ad21cae6aa3 
  webapp/src/main/java/org/apache/atlas/web/listeners/GuiceServletConfig.java 
3bab65005f6c02410ccecf07e611784c3aec67bf 
  webapp/src/main/java/org/apache/atlas/web/resources/EntityResource.java 
8a663c28dba0fbb9ab1549e

Review Request 54907: ATLAS-1391 Add exclusion mechanism for Atlas audit mechanism

[Neeru Gupta]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54907/
---

Review request for atlas.


Bugs: ATLAS-1391
https://issues.apache.org/jira/browse/ATLAS-1391


Repository: atlas


Description
---

Merge branch 'master' into OMS_852


Diffs
-

  
repository/src/main/java/org/apache/atlas/util/AtlasRepositoryConfiguration.java
 4b6f88f45dc6bf5ec9a149c06b38059c0c43559a 
  webapp/src/main/java/org/apache/atlas/web/filters/AuditFilter.java 
7499cde4c5dc6a2c5253cd7a354a6add7d8e02e4 
  webapp/src/test/java/org/apache/atlas/web/filters/AuditFilterTest.java 
PRE-CREATION 

Diff: https://reviews.apache.org/r/54907/diff/


Testing
---

Added new test class 'AuditFilterTest'


Thanks,

Neeru Gupta

[jira] [Updated] (ATLAS-1391) Add exclusion mechanism for Atlas audit mechanism

[Neeru Gupta (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-1391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Neeru Gupta updated ATLAS-1391:
---
Attachment: rb54907.patch

> Add exclusion mechanism for Atlas audit mechanism
> -
>
> Key: ATLAS-1391
> URL: https://issues.apache.org/jira/browse/ATLAS-1391
> Project: Atlas
>  Issue Type: Improvement
>Reporter: Neeru Gupta
>Assignee: Neeru Gupta
> Attachments: rb54907.patch
>
>
> As a consumer of Atlas, I want to be able to exclude certain operations, such 
> as ping, from the audit log so that it does not get filled up with noise.
> This should be a general mechanism. It should be possible to configure the 
> endpoints being excluded in the Atlas configuration file. The only 
> requirement at this time is to be able to filter based on the high level 
> operation being invoked (ie ping, version, etc). There is no need to be able 
> to filter based on the arguments to the operation.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (ATLAS-1391) Add exclusion mechanism for Atlas audit mechanism

[Neeru Gupta (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-1391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Neeru Gupta updated ATLAS-1391:
---
External issue URL: https://reviews.apache.org/r/54907/
   Description: 

As a consumer of Atlas, I want to be able to exclude certain operations, such 
as ping, from the audit log so that it does not get filled up with noise.

This should be a general mechanism. It should be possible to configure the 
endpoints being excluded in the Atlas configuration file. The only requirement 
at this time is to be able to filter based on the high level operation being 
invoked (ie ping, version, etc). There is no need to be able to filter based on 
the arguments to the operation.


  was:


As a consumer of Atlas, I want to be able to exclude certain operations, such 
as ping, from the audit log so that it does not get filled up with noise.

This should be a general mechanism. It should be possible to configure the 
endpoints being excluded in the Atlas configuration file. The only requirement 
at this time is to be able to filter based on the high level operation being 
invoked (ie ping, version, etc). There is no need to be able to filter based on 
the arguments to the operation.



> Add exclusion mechanism for Atlas audit mechanism
> -
>
> Key: ATLAS-1391
> URL: https://issues.apache.org/jira/browse/ATLAS-1391
> Project: Atlas
>  Issue Type: Improvement
>Reporter: Neeru Gupta
>Assignee: Neeru Gupta
> Attachments: rb54907.patch
>
>
> As a consumer of Atlas, I want to be able to exclude certain operations, such 
> as ping, from the audit log so that it does not get filled up with noise.
> This should be a general mechanism. It should be possible to configure the 
> endpoints being excluded in the Atlas configuration file. The only 
> requirement at this time is to be able to filter based on the high level 
> operation being invoked (ie ping, version, etc). There is no need to be able 
> to filter based on the arguments to the operation.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (ATLAS-1344) Allow types to be created that support parent child hierarchies.

[David Radley (JIRA)]

[ 
https://issues.apache.org/jira/browse/ATLAS-1344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15764844#comment-15764844
 ] 

David Radley commented on ATLAS-1344:
-

Hi Madhan,
thank you so much for drawing this to my attention. Some comments/questions:
1) So there are 2 pieces of code that deal with entity attributes, 
AttributeDefinition and 
org.apache.atlas.model.typedef.AtlasStructDef.AtlasAttributeDef. 
AtlasAttributeDef 
Is the idea to change the implementation of AttributeDefinition to call 
org.apache.atlas.model.typedef.AtlasStructDef.AtlasAttributeDef. 
AtlasAttributeDef? Or to change the callers of AttributeDefinitions to use the 
new class.   

I am not sure how the apache-atlas typesystem and the atlas-intg project 
content sit together as they seem to duplicate functionality. 

2) On the renames from "isComposite" and "reverseAttributeName" with 
"foreignKey" and "mappedFromRef".  

It seems to me that iscomposite would be a property of the attribute on the 
parent - referring to the child. If this could not be null I can see why you 
called it foreign key as they cannot be null  - so we cannot model optional 
multiple children. 

Also currently the child object must be created along with the parent object. 
But for parent child, containment hierarchies we need the child to be optional 
and not created when the parent is. So we need to police updates, adds and 
deletes. I suspect the current code creates and deletes the iscomposite 
children with the parent, which is fine for hive tables and columns, but not 
for our parent child hierarchies. Does this make sense?   
3) I have manually implemented the parent child hierarchy in the 1186 - it is 
not obvious that I can re-use this to model my parent child hierarchy. Have I 
understood correctly? I have not investigated the code in depth - so could have 
missed something.  

 all the best, David. 

> Allow types to be created that support parent child hierarchies.
> 
>
> Key: ATLAS-1344
> URL: https://issues.apache.org/jira/browse/ATLAS-1344
> Project: Atlas
>  Issue Type: Improvement
>Reporter: David Radley
>Assignee: David Radley
>
> enhance the Type system to allow entities to be created in parent child 
> hierarchies. 
> I notice that AttributeDefinition currently only support  COLLECTION, SET , 
> OPTIONAL and REQUIRED. There is no collection with a lower bound of 0- which 
> would be required for a parent child type relationship. 
> Also it seems to me that it would be ideal to create an attribute with the 
> guid of the parent / child and then derive the other one from the edge. I may 
> have misunderstood but reverseAttributeName seems to use the name of the 
> attribute not the guid.
> I suggest that iscomposite true and false is supported. 
> When we have this in place, we could use this mechanism for Glossary 
> Categories  
> 
>   



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Review Request 54906: ATLAS-1407: Improve LOG statement performance

[Apoorv Naik]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54906/
---

(Updated Dec. 20, 2016, 6:13 p.m.)


Review request for atlas, Madhan Neethiraj, Sarath Subramanian, Suma 
Shivaprasad, and Vimal Sharma.


Changes
---

Adding SLF4J link


Bugs: ATLAS-1407
https://issues.apache.org/jira/browse/ATLAS-1407


Repository: atlas


Description (updated)
---

ATLAS-1407: Improve LOG statement performance

Link to SLF4J docs -> http://slf4j.org/faq.html#logging_performance


Diffs
-

  
addons/falcon-bridge-shim/src/main/java/org/apache/atlas/falcon/service/AtlasService.java
 6b4486a23ea945d24ef02143cd26d3aa47b11cb3 
  
addons/falcon-bridge/src/main/java/org/apache/atlas/falcon/hook/FalconHook.java 
b6312d73aa116a61e2898f60c511e4fe73ab9505 
  
addons/falcon-bridge/src/test/java/org/apache/atlas/falcon/hook/FalconHookIT.java
 2acc5755ba62c5368d19525709a4abbbff52c4b0 
  
addons/hive-bridge-shim/src/main/java/org/apache/atlas/hive/hook/HiveHook.java 
f4ca94fd8e7af9e6a787266020f2b90d3c60462b 
  
addons/hive-bridge/src/main/java/org/apache/atlas/hive/bridge/HiveMetaStoreBridge.java
 0f8afd55138f9af0ec66ba30967157fa0b1a7140 
  addons/hive-bridge/src/test/java/org/apache/atlas/hive/HiveITBase.java 
4b736a875135fc8b59d3fe86fcbf394207cf4bc7 
  
addons/sqoop-bridge-shim/src/main/java/org/apache/atlas/sqoop/hook/SqoopHook.java
 1170cb64c9fef0b72ccc024c5662ed50a5d26457 
  
addons/storm-bridge-shim/src/main/java/org/apache/atlas/storm/hook/StormAtlasHook.java
 2d304c5cdc5be4a848b226632a117ab0b58b746e 
  
authorization/src/main/java/org/apache/atlas/authorize/AtlasAuthorizerFactory.java
 5aad72d6e1989a0ab49a8d5d4bd273aa37b51adf 
  
authorization/src/main/java/org/apache/atlas/authorize/simple/AtlasAuthorizationUtils.java
 2ef4ea28c8d6786b6fe30067f27e9e5174104721 
  
authorization/src/main/java/org/apache/atlas/authorize/simple/PolicyParser.java 
5740a1cf500b5a8a90f2e9d4371bb10b18e01a38 
  authorization/src/main/java/org/apache/atlas/authorize/simple/PolicyUtil.java 
8a3850fae0d2c737e3a15991402eed48c177c667 
  
authorization/src/main/java/org/apache/atlas/authorize/simple/SimpleAtlasAuthorizer.java
 5956f5c9a4c580a96245b71c8182b5ddb486bc49 
  catalog/src/main/java/org/apache/atlas/catalog/query/QueryFactory.java 
a301912e2e1a0fbf3f5c5bfacc03d65f2390bbd7 
  common/src/main/java/org/apache/atlas/security/InMemoryJAASConfiguration.java 
ed42aa65e7b99058d44f50f02235d72c6b89b12d 
  common/src/main/java/org/apache/atlas/utils/AtlasPerfTracer.java 
05ec7f16276aa65559be483cf1523e83a14856b6 
  intg/src/test/java/org/apache/atlas/model/ModelTestUtil.java 
ee78350cd8501bd54d08851b0350fc55471e87b7 
  notification/src/main/java/org/apache/atlas/hook/AtlasHook.java 
5bdd5d3446fd409c0c13474433f2890efee2b851 
  notification/src/main/java/org/apache/atlas/kafka/KafkaNotification.java 
0ebfd47897ff2fce28320be0153125ac8c7b4dde 
  
plugin-classloader/src/main/java/org/apache/atlas/plugin/classloader/AtlasPluginClassLoader.java
 0dbf3528788bd3f63d97eb6286c52f661f6d2cd1 
  
plugin-classloader/src/main/java/org/apache/atlas/plugin/classloader/AtlasPluginClassLoaderUtil.java
 69b61d61aca8093cd25dda2821e13fcbbc9ac4bc 
  repository/src/main/java/org/apache/atlas/GraphTransactionInterceptor.java 
a7124bf8690f5f6a33e1d7a04aaaec976167ec6f 
  
repository/src/main/java/org/apache/atlas/repository/graph/GraphBackedMetadataRepository.java
 f0647a47221cf8bd13d8b68f12f25f4e721249f0 
  repository/src/main/java/org/apache/atlas/repository/graph/GraphHelper.java 
6eeddaea9d1868f01090dbe37a23ccda5fccf0f9 
  
repository/src/main/java/org/apache/atlas/repository/graph/TypedInstanceToGraphMapper.java
 78e276e5972f6bba68198b3ae5a3d88676cf7585 
  
repository/src/main/java/org/apache/atlas/repository/store/bootstrap/AtlasTypeDefStoreInitializer.java
 b3531b06967b99743a3c6a54b5fe85dff208b8eb 
  
repository/src/main/java/org/apache/atlas/services/DefaultMetadataService.java 
c2f8b3f9785db0d0302ddf181e1e12427562e6ca 
  server-api/src/main/java/org/apache/atlas/aspect/AtlasAspect.java 
28017500db3ddaf2d9636d11d3b6c8597dd52394 
  server-api/src/main/java/org/apache/atlas/ha/AtlasServerIdSelector.java 
f3d36a7140725ef2452d8a4a3b34e9edfe065858 
  
webapp/src/main/java/org/apache/atlas/notification/NotificationHookConsumer.java
 a4fd1c2fc77e05f69e26f8ac5bb4b1feabab3606 
  webapp/src/main/java/org/apache/atlas/web/dao/UserDao.java 
8f6613af7a357ea90f34534309c8c5d67f2b6582 
  
webapp/src/main/java/org/apache/atlas/web/filters/AtlasAuthenticationFilter.java
 b7528104c54c01fc125078e81592b2359241d02f 
  
webapp/src/main/java/org/apache/atlas/web/filters/AtlasAuthorizationFilter.java 
26e6b1e2ffc2512ed7039b2e0d309dfdb45a1353 
  
webapp/src/main/java/org/apache/atlas/web/filters/AtlasKnoxSSOAuthenticationFilter.java
 75a884dca7f0698a00aaed027b5b9ad21cae6aa3 
  webapp/src/main/java/org/apache/atlas/web/listeners/GuiceServlet

[jira] [Comment Edited] (ATLAS-1344) Allow types to be created that support parent child hierarchies.

[David Radley (JIRA)]

[ 
https://issues.apache.org/jira/browse/ATLAS-1344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15764844#comment-15764844
 ] 

David Radley edited comment on ATLAS-1344 at 12/20/16 7:19 PM:
---

Hi Madhan,
Thank you very much for drawing this to my attention. It seems to me that
- iscomposite or foreign key could not be used for a containment hierarchy as 
the child needs to be optional and iscomposite is not.
- I will look into whether using the new class it might be possible to model 
non composite parent child hierarchies.  
all the best, David.


was (Author: davidrad):
Hi Madhan,
thank you so much for drawing this to my attention. Some comments/questions:
1) So there are 2 pieces of code that deal with entity attributes, 
AttributeDefinition and 
org.apache.atlas.model.typedef.AtlasStructDef.AtlasAttributeDef. 
AtlasAttributeDef 
Is the idea to change the implementation of AttributeDefinition to call 
org.apache.atlas.model.typedef.AtlasStructDef.AtlasAttributeDef. 
AtlasAttributeDef? Or to change the callers of AttributeDefinitions to use the 
new class.   

I am not sure how the apache-atlas typesystem and the atlas-intg project 
content sit together as they seem to duplicate functionality. 

2) On the renames from "isComposite" and "reverseAttributeName" with 
"foreignKey" and "mappedFromRef".  

It seems to me that iscomposite would be a property of the attribute on the 
parent - referring to the child. If this could not be null I can see why you 
called it foreign key as they cannot be null  - so we cannot model optional 
multiple children. 

Also currently the child object must be created along with the parent object. 
But for parent child, containment hierarchies we need the child to be optional 
and not created when the parent is. So we need to police updates, adds and 
deletes. I suspect the current code creates and deletes the iscomposite 
children with the parent, which is fine for hive tables and columns, but not 
for our parent child hierarchies. Does this make sense?   
3) I have manually implemented the parent child hierarchy in the 1186 - it is 
not obvious that I can re-use this to model my parent child hierarchy. Have I 
understood correctly? I have not investigated the code in depth - so could have 
missed something.  

 all the best, David. 

> Allow types to be created that support parent child hierarchies.
> 
>
> Key: ATLAS-1344
> URL: https://issues.apache.org/jira/browse/ATLAS-1344
> Project: Atlas
>  Issue Type: Improvement
>Reporter: David Radley
>Assignee: David Radley
>
> enhance the Type system to allow entities to be created in parent child 
> hierarchies. 
> I notice that AttributeDefinition currently only support  COLLECTION, SET , 
> OPTIONAL and REQUIRED. There is no collection with a lower bound of 0- which 
> would be required for a parent child type relationship. 
> Also it seems to me that it would be ideal to create an attribute with the 
> guid of the parent / child and then derive the other one from the edge. I may 
> have misunderstood but reverseAttributeName seems to use the name of the 
> attribute not the guid.
> I suggest that iscomposite true and false is supported. 
> When we have this in place, we could use this mechanism for Glossary 
> Categories  
> 
>   



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Review Request 54906: ATLAS-1407: Improve LOG statement performance

[Madhan Neethiraj]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54906/#review159749
---




addons/falcon-bridge/src/main/java/org/apache/atlas/falcon/hook/FalconHook.java 
(line 145)


Can you confirm if this prints the exception details - like the earlier 
usage? It it treats "t" as another variable-length argument, exception details 
might be lost in the log.



intg/src/test/java/org/apache/atlas/model/ModelTestUtil.java (line 312)


Can you confirm that "excp" is printed in the log, as in earlier usage? It 
shouldn't be treated as another variable-length argument (and ignored, as there 
are no references in the format-string).



intg/src/test/java/org/apache/atlas/model/ModelTestUtil.java (line 332)


Can you confirm that "excp" is printed in the log, as in earlier usage? It 
shouldn't be treated as another variable-length argument (and ignored, as there 
are no references in the format-string).



intg/src/test/java/org/apache/atlas/model/ModelTestUtil.java (line 353)


Can you confirm that "excp" is printed in the log, as in earlier usage? It 
shouldn't be treated as another variable-length argument (and ignored, as there 
are no references in the format-string).



notification/src/main/java/org/apache/atlas/hook/AtlasHook.java (line 193)


there are two "{}" - but with only one argument. please review.



notification/src/main/java/org/apache/atlas/hook/AtlasHook.java (line 200)


there are two "{}" but with only one argument. please review.



plugin-classloader/src/main/java/org/apache/atlas/plugin/classloader/AtlasPluginClassLoader.java
 (line 285)


Can you confirm that "t" is printed in the log, as in earlier usage? It 
shouldn't be treated as another variable-length argument (and ignored, as there 
are no references in the format-string).



plugin-classloader/src/main/java/org/apache/atlas/plugin/classloader/AtlasPluginClassLoaderUtil.java
 (line 81)


Can you confirm that "excp" is printed in the log, as in earlier usage? It 
shouldn't be treated as another variable-length argument (and ignored, as there 
are no references in the format-string).



plugin-classloader/src/main/java/org/apache/atlas/plugin/classloader/AtlasPluginClassLoaderUtil.java
 (line 87)


Can you confirm that "excp" is printed in the log, as in earlier usage? It 
shouldn't be treated as another variable-length argument (and ignored, as there 
are no references in the format-string).



repository/src/main/java/org/apache/atlas/repository/store/bootstrap/AtlasTypeDefStoreInitializer.java
 (line 133)


Can you confirm that "t" is printed in the log, as in earlier usage? It 
shouldn't be treated as another variable-length argument (and ignored, as there 
are no references in the format-string).



webapp/src/main/java/org/apache/atlas/web/filters/AtlasAuthenticationFilter.java
 (line 416)


Can you confirm that "ex" is printed in the log, as in earlier usage? It 
shouldn't be treated as another variable-length argument (and ignored, as there 
are no references in the format-string).



webapp/src/main/java/org/apache/atlas/web/resources/EntityResource.java (line 
281)


Can you confirm that "e" is printed in the log, as in earlier usage? It 
shouldn't be treated as another variable-length argument (and ignored, as there 
are no references in the format-string).



webapp/src/main/java/org/apache/atlas/web/resources/EntityResource.java (line 
284)


Can you confirm that "e" is printed in the log, as in earlier usage? It 
shouldn't be treated as another variable-length argument (and ignored, as there 
are no references in the format-string).


- Madhan Neethiraj


On Dec. 20, 2016, 6:13 p.m., Apoorv Naik wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/54906/
> ---
> 
> (Updated Dec. 20, 2016, 6:13 p.m.)
> 
> 
> Review request for atlas, Madhan Neethiraj, Sarath Subramanian, Suma 
> Shivaprasad, and Vimal Sharma.
> 
> 
> Bugs: ATLAS-1407
> https://issues.apache.org/jira/browse/ATLAS-1407
> 
> 
> Repository: a

Re: Review Request 54907: ATLAS-1391 Add exclusion mechanism for Atlas audit mechanism

[Madhan Neethiraj]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54907/#review159754
---




webapp/src/main/java/org/apache/atlas/web/filters/AuditFilter.java (line 137)


Consider moving lines #137 and #138 inside the 'if' block at line #146 - to 
avoid any overhead when there are no audit-exclusions.



webapp/src/main/java/org/apache/atlas/web/filters/AuditFilter.java (line 147)


Instead of iterating through the list, consider using a Set:

  Set auditExcludedOperations; // corresponding changes needed in 
AtlasRepositoryConfiguration

  skipped = 
auditExcludedOperations.contains(requestHttpMethod.toLowerCase() + ":" + 
requestOperation.toLowerCase());
  

Better yet, consider replacing this method with the following in 
AtlasRepositoryConfiguration, to isolate the details inside 
AtlasRepositoryConfiguration:

public class AtlasRepositoryConfiguration {
  public static boolean isExcludedFromAudit(String httpMethod, String 
httpUrl);
}


- Madhan Neethiraj


On Dec. 20, 2016, 5:49 p.m., Neeru Gupta wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/54907/
> ---
> 
> (Updated Dec. 20, 2016, 5:49 p.m.)
> 
> 
> Review request for atlas.
> 
> 
> Bugs: ATLAS-1391
> https://issues.apache.org/jira/browse/ATLAS-1391
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> Merge branch 'master' into OMS_852
> 
> 
> Diffs
> -
> 
>   
> repository/src/main/java/org/apache/atlas/util/AtlasRepositoryConfiguration.java
>  4b6f88f45dc6bf5ec9a149c06b38059c0c43559a 
>   webapp/src/main/java/org/apache/atlas/web/filters/AuditFilter.java 
> 7499cde4c5dc6a2c5253cd7a354a6add7d8e02e4 
>   webapp/src/test/java/org/apache/atlas/web/filters/AuditFilterTest.java 
> PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/54907/diff/
> 
> 
> Testing
> ---
> 
> Added new test class 'AuditFilterTest'
> 
> 
> Thanks,
> 
> Neeru Gupta
> 
>

Re: Review Request 54906: ATLAS-1407: Improve LOG statement performance

[Apoorv Naik]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54906/
---

(Updated Dec. 20, 2016, 10:16 p.m.)


Review request for atlas, Madhan Neethiraj, Sarath Subramanian, Suma 
Shivaprasad, and Vimal Sharma.


Bugs: ATLAS-1407
https://issues.apache.org/jira/browse/ATLAS-1407


Repository: atlas


Description
---

ATLAS-1407: Improve LOG statement performance

Link to SLF4J docs -> http://slf4j.org/faq.html#logging_performance


Diffs
-

  
addons/falcon-bridge-shim/src/main/java/org/apache/atlas/falcon/service/AtlasService.java
 6b4486a23ea945d24ef02143cd26d3aa47b11cb3 
  
addons/falcon-bridge/src/main/java/org/apache/atlas/falcon/hook/FalconHook.java 
b6312d73aa116a61e2898f60c511e4fe73ab9505 
  
addons/falcon-bridge/src/test/java/org/apache/atlas/falcon/hook/FalconHookIT.java
 2acc5755ba62c5368d19525709a4abbbff52c4b0 
  
addons/hive-bridge-shim/src/main/java/org/apache/atlas/hive/hook/HiveHook.java 
f4ca94fd8e7af9e6a787266020f2b90d3c60462b 
  
addons/hive-bridge/src/main/java/org/apache/atlas/hive/bridge/HiveMetaStoreBridge.java
 0f8afd55138f9af0ec66ba30967157fa0b1a7140 
  addons/hive-bridge/src/test/java/org/apache/atlas/hive/HiveITBase.java 
4b736a875135fc8b59d3fe86fcbf394207cf4bc7 
  
addons/sqoop-bridge-shim/src/main/java/org/apache/atlas/sqoop/hook/SqoopHook.java
 1170cb64c9fef0b72ccc024c5662ed50a5d26457 
  
addons/storm-bridge-shim/src/main/java/org/apache/atlas/storm/hook/StormAtlasHook.java
 2d304c5cdc5be4a848b226632a117ab0b58b746e 
  
authorization/src/main/java/org/apache/atlas/authorize/AtlasAuthorizerFactory.java
 5aad72d6e1989a0ab49a8d5d4bd273aa37b51adf 
  
authorization/src/main/java/org/apache/atlas/authorize/simple/AtlasAuthorizationUtils.java
 2ef4ea28c8d6786b6fe30067f27e9e5174104721 
  
authorization/src/main/java/org/apache/atlas/authorize/simple/PolicyParser.java 
5740a1cf500b5a8a90f2e9d4371bb10b18e01a38 
  authorization/src/main/java/org/apache/atlas/authorize/simple/PolicyUtil.java 
8a3850fae0d2c737e3a15991402eed48c177c667 
  
authorization/src/main/java/org/apache/atlas/authorize/simple/SimpleAtlasAuthorizer.java
 5956f5c9a4c580a96245b71c8182b5ddb486bc49 
  catalog/src/main/java/org/apache/atlas/catalog/query/QueryFactory.java 
a301912e2e1a0fbf3f5c5bfacc03d65f2390bbd7 
  common/src/main/java/org/apache/atlas/security/InMemoryJAASConfiguration.java 
ed42aa65e7b99058d44f50f02235d72c6b89b12d 
  common/src/main/java/org/apache/atlas/utils/AtlasPerfTracer.java 
05ec7f16276aa65559be483cf1523e83a14856b6 
  intg/src/test/java/org/apache/atlas/model/ModelTestUtil.java 
ee78350cd8501bd54d08851b0350fc55471e87b7 
  notification/src/main/java/org/apache/atlas/hook/AtlasHook.java 
5bdd5d3446fd409c0c13474433f2890efee2b851 
  notification/src/main/java/org/apache/atlas/kafka/KafkaNotification.java 
0ebfd47897ff2fce28320be0153125ac8c7b4dde 
  
plugin-classloader/src/main/java/org/apache/atlas/plugin/classloader/AtlasPluginClassLoader.java
 0dbf3528788bd3f63d97eb6286c52f661f6d2cd1 
  
plugin-classloader/src/main/java/org/apache/atlas/plugin/classloader/AtlasPluginClassLoaderUtil.java
 69b61d61aca8093cd25dda2821e13fcbbc9ac4bc 
  repository/src/main/java/org/apache/atlas/GraphTransactionInterceptor.java 
a7124bf8690f5f6a33e1d7a04aaaec976167ec6f 
  
repository/src/main/java/org/apache/atlas/repository/graph/GraphBackedMetadataRepository.java
 f0647a47221cf8bd13d8b68f12f25f4e721249f0 
  repository/src/main/java/org/apache/atlas/repository/graph/GraphHelper.java 
6eeddaea9d1868f01090dbe37a23ccda5fccf0f9 
  
repository/src/main/java/org/apache/atlas/repository/graph/TypedInstanceToGraphMapper.java
 78e276e5972f6bba68198b3ae5a3d88676cf7585 
  
repository/src/main/java/org/apache/atlas/repository/store/bootstrap/AtlasTypeDefStoreInitializer.java
 b3531b06967b99743a3c6a54b5fe85dff208b8eb 
  
repository/src/main/java/org/apache/atlas/services/DefaultMetadataService.java 
c2f8b3f9785db0d0302ddf181e1e12427562e6ca 
  server-api/src/main/java/org/apache/atlas/aspect/AtlasAspect.java 
28017500db3ddaf2d9636d11d3b6c8597dd52394 
  server-api/src/main/java/org/apache/atlas/ha/AtlasServerIdSelector.java 
f3d36a7140725ef2452d8a4a3b34e9edfe065858 
  
webapp/src/main/java/org/apache/atlas/notification/NotificationHookConsumer.java
 a4fd1c2fc77e05f69e26f8ac5bb4b1feabab3606 
  webapp/src/main/java/org/apache/atlas/web/dao/UserDao.java 
8f6613af7a357ea90f34534309c8c5d67f2b6582 
  
webapp/src/main/java/org/apache/atlas/web/filters/AtlasAuthenticationFilter.java
 b7528104c54c01fc125078e81592b2359241d02f 
  
webapp/src/main/java/org/apache/atlas/web/filters/AtlasAuthorizationFilter.java 
26e6b1e2ffc2512ed7039b2e0d309dfdb45a1353 
  
webapp/src/main/java/org/apache/atlas/web/filters/AtlasKnoxSSOAuthenticationFilter.java
 75a884dca7f0698a00aaed027b5b9ad21cae6aa3 
  webapp/src/main/java/org/apache/atlas/web/listeners/GuiceServletConfig.java 
3bab65005f6c02410ccecf07e611784c3

Re: Review Request 54906: ATLAS-1407: Improve LOG statement performance

[Apoorv Naik]

> On Dec. 20, 2016, 9:33 p.m., Madhan Neethiraj wrote:
> > addons/falcon-bridge/src/main/java/org/apache/atlas/falcon/hook/FalconHook.java,
> >  line 145
> > 
> >
> > Can you confirm if this prints the exception details - like the earlier 
> > usage? It it treats "t" as another variable-length argument, exception 
> > details might be lost in the log.

http://slf4j.org/faq.html#paramException


> On Dec. 20, 2016, 9:33 p.m., Madhan Neethiraj wrote:
> > webapp/src/main/java/org/apache/atlas/web/resources/EntityResource.java, 
> > line 284
> > 
> >
> > Can you confirm that "e" is printed in the log, as in earlier usage? It 
> > shouldn't be treated as another variable-length argument (and ignored, as 
> > there are no references in the format-string).

This needs to be fixed as the exception should be the last argument.


> On Dec. 20, 2016, 9:33 p.m., Madhan Neethiraj wrote:
> > webapp/src/main/java/org/apache/atlas/web/resources/EntityResource.java, 
> > line 281
> > 
> >
> > Can you confirm that "e" is printed in the log, as in earlier usage? It 
> > shouldn't be treated as another variable-length argument (and ignored, as 
> > there are no references in the format-string).

This needs to be fixed as the exception should be the last argument.


- Apoorv


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54906/#review159749
---


On Dec. 20, 2016, 10:16 p.m., Apoorv Naik wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/54906/
> ---
> 
> (Updated Dec. 20, 2016, 10:16 p.m.)
> 
> 
> Review request for atlas, Madhan Neethiraj, Sarath Subramanian, Suma 
> Shivaprasad, and Vimal Sharma.
> 
> 
> Bugs: ATLAS-1407
> https://issues.apache.org/jira/browse/ATLAS-1407
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> ATLAS-1407: Improve LOG statement performance
> 
> Link to SLF4J docs -> http://slf4j.org/faq.html#logging_performance
> 
> 
> Diffs
> -
> 
>   
> addons/falcon-bridge-shim/src/main/java/org/apache/atlas/falcon/service/AtlasService.java
>  6b4486a23ea945d24ef02143cd26d3aa47b11cb3 
>   
> addons/falcon-bridge/src/main/java/org/apache/atlas/falcon/hook/FalconHook.java
>  b6312d73aa116a61e2898f60c511e4fe73ab9505 
>   
> addons/falcon-bridge/src/test/java/org/apache/atlas/falcon/hook/FalconHookIT.java
>  2acc5755ba62c5368d19525709a4abbbff52c4b0 
>   
> addons/hive-bridge-shim/src/main/java/org/apache/atlas/hive/hook/HiveHook.java
>  f4ca94fd8e7af9e6a787266020f2b90d3c60462b 
>   
> addons/hive-bridge/src/main/java/org/apache/atlas/hive/bridge/HiveMetaStoreBridge.java
>  0f8afd55138f9af0ec66ba30967157fa0b1a7140 
>   addons/hive-bridge/src/test/java/org/apache/atlas/hive/HiveITBase.java 
> 4b736a875135fc8b59d3fe86fcbf394207cf4bc7 
>   
> addons/sqoop-bridge-shim/src/main/java/org/apache/atlas/sqoop/hook/SqoopHook.java
>  1170cb64c9fef0b72ccc024c5662ed50a5d26457 
>   
> addons/storm-bridge-shim/src/main/java/org/apache/atlas/storm/hook/StormAtlasHook.java
>  2d304c5cdc5be4a848b226632a117ab0b58b746e 
>   
> authorization/src/main/java/org/apache/atlas/authorize/AtlasAuthorizerFactory.java
>  5aad72d6e1989a0ab49a8d5d4bd273aa37b51adf 
>   
> authorization/src/main/java/org/apache/atlas/authorize/simple/AtlasAuthorizationUtils.java
>  2ef4ea28c8d6786b6fe30067f27e9e5174104721 
>   
> authorization/src/main/java/org/apache/atlas/authorize/simple/PolicyParser.java
>  5740a1cf500b5a8a90f2e9d4371bb10b18e01a38 
>   
> authorization/src/main/java/org/apache/atlas/authorize/simple/PolicyUtil.java 
> 8a3850fae0d2c737e3a15991402eed48c177c667 
>   
> authorization/src/main/java/org/apache/atlas/authorize/simple/SimpleAtlasAuthorizer.java
>  5956f5c9a4c580a96245b71c8182b5ddb486bc49 
>   catalog/src/main/java/org/apache/atlas/catalog/query/QueryFactory.java 
> a301912e2e1a0fbf3f5c5bfacc03d65f2390bbd7 
>   
> common/src/main/java/org/apache/atlas/security/InMemoryJAASConfiguration.java 
> ed42aa65e7b99058d44f50f02235d72c6b89b12d 
>   common/src/main/java/org/apache/atlas/utils/AtlasPerfTracer.java 
> 05ec7f16276aa65559be483cf1523e83a14856b6 
>   intg/src/test/java/org/apache/atlas/model/ModelTestUtil.java 
> ee78350cd8501bd54d08851b0350fc55471e87b7 
>   notification/src/main/java/org/apache/atlas/hook/AtlasHook.java 
> 5bdd5d3446fd409c0c13474433f2890efee2b851 
>   notification/src/main/java/org/apache/atlas/kafka/KafkaNotification.java 
> 0ebfd47897ff2fce28320be0153125ac8c7b4dde 
>   
> plugin-classloader/src/main/java/org/apache/atlas/plugin/classloader

Re: Review Request 54906: ATLAS-1407: Improve LOG statement performance

[Apoorv Naik]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54906/
---

(Updated Dec. 20, 2016, 10:30 p.m.)


Review request for atlas, Madhan Neethiraj, Sarath Subramanian, Suma 
Shivaprasad, and Vimal Sharma.


Changes
---

Addressed couple of issues.


Bugs: ATLAS-1407
https://issues.apache.org/jira/browse/ATLAS-1407


Repository: atlas


Description
---

ATLAS-1407: Improve LOG statement performance

Link to SLF4J docs -> http://slf4j.org/faq.html#logging_performance


Diffs (updated)
-

  
addons/falcon-bridge-shim/src/main/java/org/apache/atlas/falcon/service/AtlasService.java
 6b4486a2 
  
addons/falcon-bridge/src/main/java/org/apache/atlas/falcon/hook/FalconHook.java 
b6312d73 
  
addons/falcon-bridge/src/test/java/org/apache/atlas/falcon/hook/FalconHookIT.java
 2acc5755 
  
addons/hive-bridge-shim/src/main/java/org/apache/atlas/hive/hook/HiveHook.java 
f4ca94fd 
  
addons/hive-bridge/src/main/java/org/apache/atlas/hive/bridge/HiveMetaStoreBridge.java
 0f8afd55 
  addons/hive-bridge/src/test/java/org/apache/atlas/hive/HiveITBase.java 
4b736a87 
  
addons/sqoop-bridge-shim/src/main/java/org/apache/atlas/sqoop/hook/SqoopHook.java
 1170cb64 
  
addons/storm-bridge-shim/src/main/java/org/apache/atlas/storm/hook/StormAtlasHook.java
 2d304c5c 
  
authorization/src/main/java/org/apache/atlas/authorize/AtlasAuthorizerFactory.java
 5aad72d6 
  
authorization/src/main/java/org/apache/atlas/authorize/simple/AtlasAuthorizationUtils.java
 2ef4ea28 
  
authorization/src/main/java/org/apache/atlas/authorize/simple/PolicyParser.java 
5740a1cf 
  authorization/src/main/java/org/apache/atlas/authorize/simple/PolicyUtil.java 
8a3850fa 
  
authorization/src/main/java/org/apache/atlas/authorize/simple/SimpleAtlasAuthorizer.java
 5956f5c9 
  catalog/src/main/java/org/apache/atlas/catalog/query/QueryFactory.java 
a301912e 
  common/src/main/java/org/apache/atlas/security/InMemoryJAASConfiguration.java 
ed42aa65 
  common/src/main/java/org/apache/atlas/utils/AtlasPerfTracer.java 05ec7f16 
  intg/src/test/java/org/apache/atlas/model/ModelTestUtil.java ee78350c 
  notification/src/main/java/org/apache/atlas/hook/AtlasHook.java 5bdd5d34 
  notification/src/main/java/org/apache/atlas/kafka/KafkaNotification.java 
0ebfd478 
  
plugin-classloader/src/main/java/org/apache/atlas/plugin/classloader/AtlasPluginClassLoader.java
 0dbf3528 
  
plugin-classloader/src/main/java/org/apache/atlas/plugin/classloader/AtlasPluginClassLoaderUtil.java
 69b61d61 
  repository/src/main/java/org/apache/atlas/GraphTransactionInterceptor.java 
a7124bf8 
  
repository/src/main/java/org/apache/atlas/repository/graph/GraphBackedMetadataRepository.java
 f0647a47 
  repository/src/main/java/org/apache/atlas/repository/graph/GraphHelper.java 
6eeddaea 
  
repository/src/main/java/org/apache/atlas/repository/graph/TypedInstanceToGraphMapper.java
 78e276e5 
  
repository/src/main/java/org/apache/atlas/repository/store/bootstrap/AtlasTypeDefStoreInitializer.java
 b3531b06 
  
repository/src/main/java/org/apache/atlas/services/DefaultMetadataService.java 
c2f8b3f9 
  server-api/src/main/java/org/apache/atlas/aspect/AtlasAspect.java 28017500 
  server-api/src/main/java/org/apache/atlas/ha/AtlasServerIdSelector.java 
f3d36a71 
  
webapp/src/main/java/org/apache/atlas/notification/NotificationHookConsumer.java
 a4fd1c2f 
  webapp/src/main/java/org/apache/atlas/web/dao/UserDao.java 8f6613af 
  
webapp/src/main/java/org/apache/atlas/web/filters/AtlasAuthenticationFilter.java
 b7528104 
  
webapp/src/main/java/org/apache/atlas/web/filters/AtlasAuthorizationFilter.java 
26e6b1e2 
  
webapp/src/main/java/org/apache/atlas/web/filters/AtlasKnoxSSOAuthenticationFilter.java
 75a884dc 
  webapp/src/main/java/org/apache/atlas/web/listeners/GuiceServletConfig.java 
3bab6500 
  webapp/src/main/java/org/apache/atlas/web/resources/EntityResource.java 
8a663c28 
  webapp/src/main/java/org/apache/atlas/web/setup/AtlasSetup.java 259285e3 
  webapp/src/main/java/org/apache/atlas/web/setup/SetupSteps.java bc532e91 
  webapp/src/test/java/org/apache/atlas/notification/EntityNotificationIT.java 
ec62112f 
  
webapp/src/test/java/org/apache/atlas/web/resources/EntityJerseyResourceIT.java 
22bcc027 
  
webapp/src/test/java/org/apache/atlas/web/security/FileAuthenticationTest.java 
44b7cac5 

Diff: https://reviews.apache.org/r/54906/diff/


Testing
---

Not needed as it's only logging change.

Verified (few obvious ones) that the exceptions are getting printed in the 
logs, existing TypesResource does the same thing and the exceptions are getting 
printed after parameterization of the message.

Will try to capture a before and after server log for a clean install run.


Thanks,

Apoorv Naik

Re: [jira] [Updated] (ATLAS-1298) UI never finishes loading

[Russell Anderson]

Hi,

I made the code changes identified in the work around from ATLAS-1298.patch

However, I started with atlas7 rc2 as the code base.

My build was run with 'mvn clean package -Pdist,berkeley-elasticsearch
-DskipTests -DskipCheck=true'

All completed successfully - however - the same behavior results - infinite
circle - never ends

The import-hive.sh works fine, and the REST API returns lineage and works
as expected.

Regards,

Russ.




From:   "Keval Bhatt (JIRA)" 
To: dev@atlas.incubator.apache.org
Date:   12/20/2016 12:38 AM
Subject:[jira] [Updated] (ATLAS-1298) UI never finishes loading




 [
https://issues.apache.org/jira/browse/ATLAS-1298?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Keval Bhatt updated ATLAS-1298:
---
Fix Version/s: 0.7.1-incubating

> UI never finishes loading
> -
>
> Key: ATLAS-1298
> URL: https://issues.apache.org/jira/browse/ATLAS-1298
> Project: Atlas
>  Issue Type: Bug
>Affects Versions: 0.7-incubating
> Environment: Atlas 0.7 - Profile Berkeley/ElasticSearch
>Reporter: PJ Van Aeken
>Assignee: Keval Bhatt
> Fix For: 0.7.1-incubating
>
> Attachments: ATLAS-1298.patch, graycol.gif
>
>
> The REST API  works fine, and I can see the data from the quickstart
script but when I go to the UI, I only get the login screen, followed by
the two-tone background and a "loading circle" which never completes
loading. There are no errors in application.log.
> Note: I am running inside a docker container but port 21000 is exposed to
the host. I am querying the API from my browser on the host machine, so not
from inside the docker container. It works with both localhost and the
internal docker ip.
> Any help debugging/fixing this would be greatly appreciated.
> *EDIT:* Running the master branch outside of docker, "mylaptopname:21000"
works just fine but "localhost:21000" suffers from the same issue.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Review Request 54058: ATLAS-1311: Integration tests for the V2 Entity APIs

[Madhan Neethiraj]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54058/#review159764
---


Fix it, then Ship it!





client/src/main/java/org/apache/atlas/AtlasBaseClient.java (line 280)


Consider wrapping this inside: LOG.isDebugEnabled().



client/src/main/java/org/apache/atlas/AtlasBaseClient.java (line 299)


is ".toString()" necessary here?



typesystem/src/main/java/org/apache/atlas/typesystem/types/TypeSystemProvider.java
 (line 19)


This file has white-space changes only. Please review and revert.


- Madhan Neethiraj


On Dec. 19, 2016, 8:21 p.m., Apoorv Naik wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/54058/
> ---
> 
> (Updated Dec. 19, 2016, 8:21 p.m.)
> 
> 
> Review request for atlas, Madhan Neethiraj, Sarath Subramanian, Shwetha GS, 
> and Suma Shivaprasad.
> 
> 
> Bugs: ATLAS-1311
> https://issues.apache.org/jira/browse/ATLAS-1311
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> ATLAS-1311: Integration tests for the V2 Entity APIs
> 
> 
> Diffs
> -
> 
>   addons/hive-bridge/src/test/java/org/apache/atlas/hive/hook/HiveHookIT.java 
> e5bd65a8 
>   client/src/main/java/org/apache/atlas/AtlasBaseClient.java a95bf32d 
>   client/src/main/java/org/apache/atlas/AtlasClient.java 47231a87 
>   client/src/main/java/org/apache/atlas/AtlasEntitiesClientV2.java 16556c8c 
>   client/src/main/java/org/apache/atlas/AtlasServiceException.java 4719e7cf 
>   client/src/main/java/org/apache/atlas/AtlasTypedefClientV2.java a1935247 
>   intg/src/main/java/org/apache/atlas/model/SearchFilter.java 4d8b258e 
>   
> intg/src/main/java/org/apache/atlas/model/instance/EntityMutationResponse.java
>  35f2f144 
>   
> repository/src/main/java/org/apache/atlas/repository/graph/GraphToTypedInstanceMapper.java
>  1a3c31c9 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasEntityStoreV1.java
>  6b2b2166 
>   
> typesystem/src/main/java/org/apache/atlas/typesystem/types/TypeSystemProvider.java
>  4e1cd36b 
>   webapp/src/main/java/org/apache/atlas/util/RestUtils.java 7597dcbe 
>   
> webapp/src/main/java/org/apache/atlas/web/adapters/AtlasInstanceRestAdapters.java
>  5756adb5 
>   webapp/src/main/java/org/apache/atlas/web/resources/BaseService.java 
> dfd29b14 
>   webapp/src/main/java/org/apache/atlas/web/resources/EntityResource.java 
> 8a663c28 
>   webapp/src/main/java/org/apache/atlas/web/resources/EntityService.java 
> ac4f9f1a 
>   webapp/src/main/java/org/apache/atlas/web/resources/TaxonomyService.java 
> cc98207c 
>   webapp/src/main/java/org/apache/atlas/web/rest/EntitiesREST.java 768ef12c 
>   webapp/src/main/java/org/apache/atlas/web/rest/EntityREST.java 2529f0dd 
>   webapp/src/main/resources/spring-security.xml 4ba30255 
>   webapp/src/main/webapp/WEB-INF/applicationContext.xml 6129605d 
>   webapp/src/test/java/org/apache/atlas/examples/QuickStartIT.java 01e4d483 
>   
> webapp/src/test/java/org/apache/atlas/notification/EntityNotificationIT.java 
> ec62112f 
>   
> webapp/src/test/java/org/apache/atlas/notification/NotificationHookConsumerIT.java
>  de0a459b 
>   
> webapp/src/test/java/org/apache/atlas/notification/NotificationHookConsumerKafkaTest.java
>  e37839a2 
>   
> webapp/src/test/java/org/apache/atlas/web/resources/AdminJerseyResourceIT.java
>  2a4babae 
>   webapp/src/test/java/org/apache/atlas/web/resources/BaseResourceIT.java 
> 325c25b4 
>   
> webapp/src/test/java/org/apache/atlas/web/resources/DataSetLineageJerseyResourceIT.java
>  7e4db13f 
>   
> webapp/src/test/java/org/apache/atlas/web/resources/EntityJerseyResourceIT.java
>  22bcc027 
>   
> webapp/src/test/java/org/apache/atlas/web/resources/EntityLineageJerseyResourceIT.java
>  c681a0f9 
>   
> webapp/src/test/java/org/apache/atlas/web/resources/EntityV2JerseyResourceIT.java
>  PRE-CREATION 
>   
> webapp/src/test/java/org/apache/atlas/web/resources/MetadataDiscoveryJerseyResourceIT.java
>  ed37c3a1 
>   
> webapp/src/test/java/org/apache/atlas/web/resources/TypesJerseyResourceIT.java
>  87bf9a89 
> 
> Diff: https://reviews.apache.org/r/54058/diff/
> 
> 
> Testing
> ---
> 
> UTs and ITs are running fine. mvn clean install -Pdist was successful
> 
> 
> Thanks,
> 
> Apoorv Naik
> 
>

[jira] [Updated] (ATLAS-1404) Improve lineage performance with query filters using only GUID

[Suma Shivaprasad (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-1404?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Suma Shivaprasad updated ATLAS-1404:

Attachment: ATLAS-1404.patch

> Improve lineage performance with query filters using only GUID
> --
>
> Key: ATLAS-1404
> URL: https://issues.apache.org/jira/browse/ATLAS-1404
> Project: Atlas
>  Issue Type: Improvement
>Affects Versions: 0.8-incubating
>Reporter: Suma Shivaprasad
>Assignee: Suma Shivaprasad
> Fix For: 0.8-incubating
>
> Attachments: ATLAS-1404.patch
>
>
> Linage currently queries by superTypeName + guid combinations for which a 
> composite index does not exist and index exists for typeName or guid alone. 
> Subsequent filters are getting applied on typename query results which take a 
> long time to finish. Hence rewriting these queries to use the GUID alone for 
> querying and then apply the supertype checks once the entity definition is 
> retrieved from the repository using its type



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (ATLAS-1408) Creating an attribute definition of array type, where the elements have a dot in the typename causes issues

[Hemanth Yamijala (JIRA)]

Hemanth Yamijala created ATLAS-1408:
---

 Summary: Creating an attribute definition of array type, where the 
elements have a dot in the typename causes issues
 Key: ATLAS-1408
 URL: https://issues.apache.org/jira/browse/ATLAS-1408
 Project: Atlas
  Issue Type: Bug
Affects Versions: 0.8-incubating
Reporter: Hemanth Yamijala
Priority: Critical


I created a type File.v1 and then another type FileSet.v1 which contains an 
array of File.v1 type instances. The creation was using the 
{{AtlasClient.createType(TypeDef)}} API. This returned back successfully. 
However, post this Atlas server went into a funny state where simple DSL 
queries like {{hive_table}} started failing. Looking the application logs, 
noticed this error post the registration of the FileSet.v1 type:

{code}
2016-12-20 13:46:53,960 INFO  - [pool-2-thread-4 - 
8eaf688d-9cd4-46dc-92da-d84991512e9b:] ~ TypeSystem reset invoked by 
TypeRegistry changes (DefaultMetadataService:749)
 Many types restored and then 
2016-12-20 13:46:54,266 ERROR - [pool-2-thread-4 - 
8eaf688d-9cd4-46dc-92da-d84991512e9b:] ~ Failed to restore type-system after 
TypeRegistry changes (DefaultMetadataService:759)
org.apache.atlas.typesystem.exception.TypeNotFoundException: Unknown datatype: 
array
at org.apache.atlas.typesystem.types.TypeSystem.getDataType(TypeSystem.java:192)
at 
org.apache.atlas.typesystem.types.TypeSystem$TransientTypeSystem.getDataType(TypeSystem.java:669)
at org.apache.atlas.typesystem.types.AttributeInfo.(AttributeInfo.java:46)
at 
org.apache.atlas.typesystem.types.TypeSystem$TransientTypeSystem.constructAttributeInfo(TypeSystem.java:496)
at 
org.apache.atlas.typesystem.types.TypeSystem$TransientTypeSystem.constructHierarchicalType(TypeSystem.java:540)
at 
org.apache.atlas.typesystem.types.TypeSystem$TransientTypeSystem.orderAndConstructTypes(TypeSystem.java:583)
at 
org.apache.atlas.typesystem.types.TypeSystem$TransientTypeSystem.verifyTypes(TypeSystem.java:709)
at 
org.apache.atlas.typesystem.types.TypeSystem.createTransientTypeSystem(TypeSystem.java:346)
at 
org.apache.atlas.services.DefaultMetadataService.onChange(DefaultMetadataService.java:753)
at 
org.apache.atlas.repository.store.graph.AtlasTypeDefGraphStore$TypeRegistryUpdateHook.notifyListeners(AtlasTypeDefGraphStore.java:940)
at 
org.apache.atlas.repository.store.graph.AtlasTypeDefGraphStore$TypeRegistryUpdateHook.onComplete(AtlasTypeDefGraphStore.java:924)
{code}

DSL failures post this error had this trace:

{code}
2016-12-20 13:49:55,398 ERROR - [pool-2-thread-8 - 
3d671edf-9dce-4656-a5e3-04e3d3bd3ade:] ~ Unable to get entity list for dslQuery 
hive_table (MetadataDiscoveryResource:141)
org.apache.atlas.discovery.DiscoveryException: Invalid expression : hive_table
at 
org.apache.atlas.discovery.graph.GraphBackedDiscoveryService.evaluate(GraphBackedDiscoveryService.java:139)
at 
org.apache.atlas.discovery.graph.GraphBackedDiscoveryService.searchByDSL(GraphBackedDiscoveryService.java:124)
at 
org.apache.atlas.GraphTransactionInterceptor.invoke(GraphTransactionInterceptor.java:51)
at 
org.apache.atlas.web.resources.MetadataDiscoveryResource.searchUsingQueryDSL(MetadataDiscoveryResource.java:135)
at sun.reflect.GeneratedMethodAccessor41.invoke(Unknown Source)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
{code}

Restarting the server caused more issues because the types restoration failed 
as in the error in the first stack trace, which caused failures in 
initializations and so, effectively, the server did not come up. The workaround 
was the delete the HBase titan table and start over again.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (ATLAS-1311) Integration tests for Entity V2 REST APIs

[Apoorv Naik (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-1311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Apoorv Naik updated ATLAS-1311:
---
Attachment: ATLAS-1311.v7.patch

> Integration tests for Entity V2 REST APIs
> -
>
> Key: ATLAS-1311
> URL: https://issues.apache.org/jira/browse/ATLAS-1311
> Project: Atlas
>  Issue Type: Improvement
>Affects Versions: 0.8-incubating
>Reporter: Apoorv Naik
>Assignee: Apoorv Naik
> Fix For: 0.8-incubating
>
> Attachments: ATLAS-1311-v2.patch, ATLAS-1311-v3.patch, 
> ATLAS-1311.patch, ATLAS-1311.v4.patch, ATLAS-1311.v5.patch, 
> ATLAS-1311.v6.patch, ATLAS-1311.v7.patch
>
>
> Currently there's no IT for the new REST API.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Review Request 54058: ATLAS-1311: Integration tests for the V2 Entity APIs

[Apoorv Naik]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54058/
---

(Updated Dec. 21, 2016, 2:25 a.m.)


Review request for atlas, Madhan Neethiraj, Sarath Subramanian, Shwetha GS, and 
Suma Shivaprasad.


Changes
---

Addressed review comments.


Bugs: ATLAS-1311
https://issues.apache.org/jira/browse/ATLAS-1311


Repository: atlas


Description
---

ATLAS-1311: Integration tests for the V2 Entity APIs


Diffs (updated)
-

  .gitignore d2fbdc01 
  addons/hive-bridge/src/test/java/org/apache/atlas/hive/hook/HiveHookIT.java 
e5bd65a8 
  client/src/main/java/org/apache/atlas/AtlasBaseClient.java a95bf32d 
  client/src/main/java/org/apache/atlas/AtlasClient.java 47231a87 
  client/src/main/java/org/apache/atlas/AtlasEntitiesClientV2.java 16556c8c 
  client/src/main/java/org/apache/atlas/AtlasServiceException.java 4719e7cf 
  client/src/main/java/org/apache/atlas/AtlasTypedefClientV2.java a1935247 
  intg/src/main/java/org/apache/atlas/model/SearchFilter.java 4d8b258e 
  
intg/src/main/java/org/apache/atlas/model/instance/EntityMutationResponse.java 
35f2f144 
  
repository/src/main/java/org/apache/atlas/repository/graph/GraphToTypedInstanceMapper.java
 1a3c31c9 
  
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasEntityStoreV1.java
 6b2b2166 
  webapp/src/main/java/org/apache/atlas/util/RestUtils.java 7597dcbe 
  
webapp/src/main/java/org/apache/atlas/web/adapters/AtlasInstanceRestAdapters.java
 5756adb5 
  webapp/src/main/java/org/apache/atlas/web/resources/BaseService.java dfd29b14 
  webapp/src/main/java/org/apache/atlas/web/resources/EntityResource.java 
8a663c28 
  webapp/src/main/java/org/apache/atlas/web/resources/EntityService.java 
ac4f9f1a 
  webapp/src/main/java/org/apache/atlas/web/resources/TaxonomyService.java 
cc98207c 
  webapp/src/main/java/org/apache/atlas/web/rest/EntitiesREST.java 768ef12c 
  webapp/src/main/java/org/apache/atlas/web/rest/EntityREST.java 2529f0dd 
  webapp/src/main/resources/spring-security.xml 4ba30255 
  webapp/src/main/webapp/WEB-INF/applicationContext.xml 6129605d 
  webapp/src/test/java/org/apache/atlas/examples/QuickStartIT.java 01e4d483 
  webapp/src/test/java/org/apache/atlas/notification/EntityNotificationIT.java 
ec62112f 
  
webapp/src/test/java/org/apache/atlas/notification/NotificationHookConsumerIT.java
 de0a459b 
  
webapp/src/test/java/org/apache/atlas/notification/NotificationHookConsumerKafkaTest.java
 e37839a2 
  
webapp/src/test/java/org/apache/atlas/web/resources/AdminJerseyResourceIT.java 
2a4babae 
  webapp/src/test/java/org/apache/atlas/web/resources/BaseResourceIT.java 
325c25b4 
  
webapp/src/test/java/org/apache/atlas/web/resources/DataSetLineageJerseyResourceIT.java
 7e4db13f 
  
webapp/src/test/java/org/apache/atlas/web/resources/EntityJerseyResourceIT.java 
22bcc027 
  
webapp/src/test/java/org/apache/atlas/web/resources/EntityLineageJerseyResourceIT.java
 c681a0f9 
  
webapp/src/test/java/org/apache/atlas/web/resources/EntityV2JerseyResourceIT.java
 PRE-CREATION 
  
webapp/src/test/java/org/apache/atlas/web/resources/MetadataDiscoveryJerseyResourceIT.java
 ed37c3a1 
  
webapp/src/test/java/org/apache/atlas/web/resources/TypesJerseyResourceIT.java 
87bf9a89 

Diff: https://reviews.apache.org/r/54058/diff/


Testing
---

UTs and ITs are running fine. mvn clean install -Pdist was successful


Thanks,

Apoorv Naik

[jira] [Commented] (ATLAS-1408) Creating an attribute definition of array type, where the elements have a dot in the typename causes issues

[Hemanth Yamijala (JIRA)]

[ 
https://issues.apache.org/jira/browse/ATLAS-1408?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15765891#comment-15765891
 ] 

Hemanth Yamijala commented on ATLAS-1408:
-

[~madhan.neethiraj] pointed me to the '.' being the problem. Irrespective of 
the solutions to this specific problem, it would be nice to not get the server 
into an unrecoverable state.

> Creating an attribute definition of array type, where the elements have a dot 
> in the typename causes issues
> ---
>
> Key: ATLAS-1408
> URL: https://issues.apache.org/jira/browse/ATLAS-1408
> Project: Atlas
>  Issue Type: Bug
>Affects Versions: 0.8-incubating
>Reporter: Hemanth Yamijala
>Priority: Critical
>
> I created a type File.v1 and then another type FileSet.v1 which contains an 
> array of File.v1 type instances. The creation was using the 
> {{AtlasClient.createType(TypeDef)}} API. This returned back successfully. 
> However, post this Atlas server went into a funny state where simple DSL 
> queries like {{hive_table}} started failing. Looking the application logs, 
> noticed this error post the registration of the FileSet.v1 type:
> {code}
> 2016-12-20 13:46:53,960 INFO  - [pool-2-thread-4 - 
> 8eaf688d-9cd4-46dc-92da-d84991512e9b:] ~ TypeSystem reset invoked by 
> TypeRegistry changes (DefaultMetadataService:749)
>  Many types restored and then 
> 2016-12-20 13:46:54,266 ERROR - [pool-2-thread-4 - 
> 8eaf688d-9cd4-46dc-92da-d84991512e9b:] ~ Failed to restore type-system after 
> TypeRegistry changes (DefaultMetadataService:759)
> org.apache.atlas.typesystem.exception.TypeNotFoundException: Unknown 
> datatype: array
> at 
> org.apache.atlas.typesystem.types.TypeSystem.getDataType(TypeSystem.java:192)
> at 
> org.apache.atlas.typesystem.types.TypeSystem$TransientTypeSystem.getDataType(TypeSystem.java:669)
> at 
> org.apache.atlas.typesystem.types.AttributeInfo.(AttributeInfo.java:46)
> at 
> org.apache.atlas.typesystem.types.TypeSystem$TransientTypeSystem.constructAttributeInfo(TypeSystem.java:496)
> at 
> org.apache.atlas.typesystem.types.TypeSystem$TransientTypeSystem.constructHierarchicalType(TypeSystem.java:540)
> at 
> org.apache.atlas.typesystem.types.TypeSystem$TransientTypeSystem.orderAndConstructTypes(TypeSystem.java:583)
> at 
> org.apache.atlas.typesystem.types.TypeSystem$TransientTypeSystem.verifyTypes(TypeSystem.java:709)
> at 
> org.apache.atlas.typesystem.types.TypeSystem.createTransientTypeSystem(TypeSystem.java:346)
> at 
> org.apache.atlas.services.DefaultMetadataService.onChange(DefaultMetadataService.java:753)
> at 
> org.apache.atlas.repository.store.graph.AtlasTypeDefGraphStore$TypeRegistryUpdateHook.notifyListeners(AtlasTypeDefGraphStore.java:940)
> at 
> org.apache.atlas.repository.store.graph.AtlasTypeDefGraphStore$TypeRegistryUpdateHook.onComplete(AtlasTypeDefGraphStore.java:924)
> {code}
> DSL failures post this error had this trace:
> {code}
> 2016-12-20 13:49:55,398 ERROR - [pool-2-thread-8 - 
> 3d671edf-9dce-4656-a5e3-04e3d3bd3ade:] ~ Unable to get entity list for 
> dslQuery hive_table (MetadataDiscoveryResource:141)
> org.apache.atlas.discovery.DiscoveryException: Invalid expression : hive_table
> at 
> org.apache.atlas.discovery.graph.GraphBackedDiscoveryService.evaluate(GraphBackedDiscoveryService.java:139)
> at 
> org.apache.atlas.discovery.graph.GraphBackedDiscoveryService.searchByDSL(GraphBackedDiscoveryService.java:124)
> at 
> org.apache.atlas.GraphTransactionInterceptor.invoke(GraphTransactionInterceptor.java:51)
> at 
> org.apache.atlas.web.resources.MetadataDiscoveryResource.searchUsingQueryDSL(MetadataDiscoveryResource.java:135)
> at sun.reflect.GeneratedMethodAccessor41.invoke(Unknown Source)
> at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> {code}
> Restarting the server caused more issues because the types restoration failed 
> as in the error in the first stack trace, which caused failures in 
> initializations and so, effectively, the server did not come up. The 
> workaround was the delete the HBase titan table and start over again.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: [jira] [Updated] (ATLAS-1298) UI never finishes loading

[Hemanth Yamijala]

FWIW, I have found clearing the browser caches helps in this case -
assuming you have not tried this already.

On Wed, Dec 21, 2016 at 4:16 AM, Russell Anderson  wrote:

> Hi,
>
> I made the code changes identified in the work around from ATLAS-1298.patch
>
> However, I started with atlas7 rc2 as the code base.
>
> My build was run with 'mvn clean package -Pdist,berkeley-elasticsearch
> -DskipTests -DskipCheck=true'
>
> All completed successfully - however - the same behavior results -
> infinite circle - never ends
>
> The import-hive.sh works fine, and the REST API returns lineage and works
> as expected.
>
> Regards,
>
> Russ.
>
>
> [image: Inactive hide details for "Keval Bhatt (JIRA)" ---12/20/2016
> 12:38:40 AM--- [ https://issues.apache.org/jira/browse/ATLAS-1]"Keval
> Bhatt (JIRA)" ---12/20/2016 12:38:40 AM--- [ https://issues.apache.org/
> jira/browse/ATLAS-1298?page=com.atlassian.jira.plugin.system.issuet
>
> From: "Keval Bhatt (JIRA)" 
> To: dev@atlas.incubator.apache.org
> Date: 12/20/2016 12:38 AM
> Subject: [jira] [Updated] (ATLAS-1298) UI never finishes loading
> --
>
>
>
>
> [ https://issues.apache.org/jira/browse/ATLAS-1298?page=
> com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
>
> Keval Bhatt updated ATLAS-1298:
> ---
>Fix Version/s: 0.7.1-incubating
>
> > UI never finishes loading
> > -
> >
> > Key: ATLAS-1298
> > URL: https://issues.apache.org/jira/browse/ATLAS-1298
> > Project: Atlas
> >  Issue Type: Bug
> >Affects Versions: 0.7-incubating
> > Environment: Atlas 0.7 - Profile Berkeley/ElasticSearch
> >Reporter: PJ Van Aeken
> >Assignee: Keval Bhatt
> > Fix For: 0.7.1-incubating
> >
> > Attachments: ATLAS-1298.patch, graycol.gif
> >
> >
> > The REST API  works fine, and I can see the data from the quickstart
> script but when I go to the UI, I only get the login screen, followed by
> the two-tone background and a "loading circle" which never completes
> loading. There are no errors in application.log.
> > Note: I am running inside a docker container but port 21000 is exposed
> to the host. I am querying the API from my browser on the host machine, so
> not from inside the docker container. It works with both localhost and the
> internal docker ip.
> > Any help debugging/fixing this would be greatly appreciated.
> > *EDIT:* Running the master branch outside of docker,
> "mylaptopname:21000" works just fine but "localhost:21000" suffers from the
> same issue.
>
>
>
> --
> This message was sent by Atlassian JIRA
> (v6.3.4#6332)
>
>
>
>
>

[jira] [Resolved] (ATLAS-1298) UI never finishes loading

[Keval Bhatt (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-1298?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Keval Bhatt resolved ATLAS-1298.

Resolution: Fixed

Committed to 0.7-incubating 
(https://github.com/apache/incubator-atlas/commit/19f491d95c805b568aa5b1e2fa9af21f4e0062e9)

> UI never finishes loading
> -
>
> Key: ATLAS-1298
> URL: https://issues.apache.org/jira/browse/ATLAS-1298
> Project: Atlas
>  Issue Type: Bug
>Affects Versions: 0.7-incubating
> Environment: Atlas 0.7 - Profile Berkeley/ElasticSearch
>Reporter: PJ Van Aeken
>Assignee: Keval Bhatt
> Fix For: 0.7.1-incubating
>
> Attachments: ATLAS-1298.patch, graycol.gif
>
>
> The REST API  works fine, and I can see the data from the quickstart script 
> but when I go to the UI, I only get the login screen, followed by the 
> two-tone background and a "loading circle" which never completes loading. 
> There are no errors in application.log.
> Note: I am running inside a docker container but port 21000 is exposed to the 
> host. I am querying the API from my browser on the host machine, so not from 
> inside the docker container. It works with both localhost and the internal 
> docker ip.
> Any help debugging/fixing this would be greatly appreciated.
> *EDIT:* Running the master branch outside of docker, "mylaptopname:21000" 
> works just fine but "localhost:21000" suffers from the same issue.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)