Re: Seeking advice on Atlas XSS vulnerabilities

[Keval Bhatt]

Thanks, Melinda, for the detailed explanation. I am able to reproduce the
issue. wIll provide the patch for this.

On Tue, Jun 9, 2020 at 9:34 AM Madhan Neethiraj  wrote:

> Melinda,
>
> Thank you for adding details. I think the frontend must take steps to
> prevent JavaScript execution while handling values in REST API responses.
>
> Server side escaping all HTML characters in all responses wouldn’t be the
> right approach, as browser is not the only consumer of the responses.
> Consider applications that parse response from Atlas REST APIs; all such
> applications must be updated to deal with escaped HTML characters in
> responses. This will likely break existing Atlas integrations.
>
> Hope this helps.
>
> Regards,
> Madhan
>
> On 6/8/20, 6:31 PM, "Melinda Crane"  wrote:
>
> Apologies, looks like the second half of the email I just sent got
> formatted weirdly (I was copying from a failed earlier email attempt
> that
> got rejected for attaching images). Just to make sure it doesn’t get
> buried:
>
> Of course this specific case can be patched on the frontend, but doing
> some
> kind of HTML sanitization escaping into the
> whatever-is-handling-the-ajax
> (EntityREST or AtlasEntityStoreV2, maybe?) to blanket-catch HTML
> characters
> sounds safer.
>
> Hope this helps express my concerns.
> From, Melinda Crane
>
>
> On Mon, Jun 8, 2020 at 9:13 PM Melinda Crane  wrote:
>
> > Dear Madhan,
> >
> > Thank you kindly for your response and for offering to look into
> potential
> > solutions. I've been trying to step through how the ajax calls are
> handled,
> > I don't really see the AtlasJsonProvider being used much at all
> actually
> > for that. My concern is that the data from the ajax calls aren't
> being
> > sanitized on the backend. I see there is a ton of sanitization
> happening on
> > the frontend (lots of calls to _.escape()), but they're not all
> captured.
> > For example.
> >
> > In CommonViewFunction, in the propertyTable
> > <
> https://github.sc-corp.net/Snapchat/keystone/blob/master/dashboardv2/public/js/utils/CommonViewFunction.js#L267>
> function,
> > URLs are not escaped. So I can create an entity that has a key value
> of
> >
> > https://www.google.com/search?\;> width='\''1'\''
> > height='\''1'\'' onload='\''window.alert(\"boo\");'\''>
> >
> > and my script will successfully get successfully injected into my
> browser
> > when I try to view it. Looks like I can't attach images here but
> here's the
> > elements:
> >
> > 
> > 
> >  > class="hide-empty-value">
> > 
> > newSchema
> > 
> > 
> > N/A
> > 
> > 
> > 
> > 
> > classField
> > 
> > 
> >  class="blue-link"
> > href="https://www.google.com/search?;>
> >  > height="1" onload="window.alert(boo);">
> > "
> https://www.google.com/search
> > ?"
> >  > height="1" onload="window.alert(boo);">
> > ...
> >
> > ^You can see the iframe-script combo here.
> >
> > Of course this specific case can be patched on the frontend, but
> doing
> > some kind of HTML sanitization escaping on the
> > whatever-is-handling-the-ajax (EntityREST or AtlasEntityStoreV2,
> maybe?) to
> > blanket-catch HTML characters sounds safer.
> >
> > Hope this helps express my concerns.
> > From, Melinda Crane
> >
> >
> > On Sat, Jun 6, 2020 at 2:48 PM Madhan Neethiraj 
> wrote:
> >
> >> Melinda,
> >>
> >> Thank you for reaching out to Apache Atlas community.
> >>
> >> As you noted, AtlasJsonProvider is used to deserialize/serialize
> REST API
> >> requests and responses. In addition, methods in AtlasJson are used
> in to
> >> convert to/from Json. It will help if you can add few examples of
> potential
> >> issues with the current implementation.
> >>
> >> Keval is looking into the frontend for potential issues and should
> get
> >> back early next week.
> >>
> >> Thanks,
> >> Madhan
> >>
> >>
> >>
> >>
> >> On 5/29/20, 6:50 PM, "Melinda Crane" 
> wrote:
> >>
> >> Dear Apache Atlas devs,
> >>
> >> I’m from Snapchat, we’re working on building a catalog on top of
> >> Apache
> >> Atlas. I noticed looking at the frontend for 

atlas-release-2.1.0-rc1 start up error

[lk_hadoop]

hi,all: 
2020-06-09 11:29:41,005 INFO  - [main:] ~ Loading atlas-application.properties 
from 
file:/home/devuser/atlas/apache-atlas-2.1.0/conf/atlas-application.properties 
(ApplicationProperties:134)
2020-06-09 11:29:41,012 INFO  - [main:] ~ Using graphdb backend 'janus' 
(ApplicationProperties:314)
2020-06-09 11:29:41,012 INFO  - [main:] ~ Using storage backend 'hbase2' 
(ApplicationProperties:325)
2020-06-09 11:29:41,012 INFO  - [main:] ~ Using index backend 'solr' 
(ApplicationProperties:336)
2020-06-09 11:29:41,014 INFO  - [main:] ~ Atlas is running in MODE: PROD. 
(ApplicationProperties:340)
2020-06-09 11:29:41,014 INFO  - [main:] ~ Setting solr-wait-searcher property 
'true' (ApplicationProperties:346)
2020-06-09 11:29:41,014 INFO  - [main:] ~ Setting index.search.map-name 
property 'false' (ApplicationProperties:350)
2020-06-09 11:29:41,015 INFO  - [main:] ~ Setting 
atlas.graph.index.search.max-result-set-size = 150 (ApplicationProperties:360)
2020-06-09 11:29:41,015 INFO  - [main:] ~ Property (set to default) 
atlas.graph.cache.db-cache = true (ApplicationProperties:372)
2020-06-09 11:29:41,015 INFO  - [main:] ~ Property (set to default) 
atlas.graph.cache.db-cache-clean-wait = 20 (ApplicationProperties:372)
2020-06-09 11:29:41,015 INFO  - [main:] ~ Property (set to default) 
atlas.graph.cache.db-cache-size = 0.5 (ApplicationProperties:372)
2020-06-09 11:29:41,015 INFO  - [main:] ~ Property (set to default) 
atlas.graph.cache.tx-cache-size = 15000 (ApplicationProperties:372)
2020-06-09 11:29:41,015 INFO  - [main:] ~ Property (set to default) 
atlas.graph.cache.tx-dirty-size = 120 (ApplicationProperties:372)
2020-06-09 11:29:41,023 INFO  - [main:] ~ 

   Atlas Server (STARTUP)


project.name:apache-atlas
project.description:Metadata Management and Data Governance Platform over Hadoop
build.user:devuser
build.epoch:1591663542701
project.version:2.1.0
build.version:2.1.0
vc.revision:release
vc.source.url:scm:git:git://git.apache.org/atlas.git/atlas-webapp

 (Atlas:215)
2020-06-09 11:29:41,023 INFO  - [main:] ~ > 
(Atlas:216)
2020-06-09 11:29:41,023 INFO  - [main:] ~ Server starting with TLS ? false on 
port 21000 (Atlas:217)
2020-06-09 11:29:41,024 INFO  - [main:] ~ < 
(Atlas:218)
2020-06-09 11:29:41,811 INFO  - [main:] ~ Logged in user devuser (auth:SIMPLE) 
(LoginProcessor:77)
2020-06-09 11:29:42,174 INFO  - [main:] ~ Not running setup per configuration 
atlas.server.run.setup.on.start. (SetupSteps$SetupRequired:189)
2020-06-09 11:29:44,478 WARN  - [main:] ~ Local setting 
index.search.solr.mode=http (Type: GLOBAL_OFFLINE) is overridden by globally 
managed value (cloud).  Use the ManagementSystem interfa
ce instead of the local configuration to control this setting. 
(ReadConfigurationBuilder:268)
2020-06-09 11:29:44,480 WARN  - [main:] ~ Local setting 
index.search.backend=solr (Type: GLOBAL_OFFLINE) is overridden by globally 
managed value (elasticsearch).  Use the ManagementSystem i
nterface instead of the local configuration to control this setting. 
(ReadConfigurationBuilder:268)
2020-06-09 11:29:44,488 WARN  - [main:] ~ Local setting 
index.search.map-name=false (Type: GLOBAL) is overridden by globally managed 
value (true).  Use the ManagementSystem interface instea
d of the local configuration to control this setting. 
(ReadConfigurationBuilder:268)
2020-06-09 11:29:44,758 INFO  - [main:] ~ Failed to obtain graph instance, 
retrying 3 times, error: java.lang.IllegalArgumentException: Could not 
instantiate implementation: org.janusgraph.
diskstorage.es.ElasticSearchIndex (AtlasGraphProvider:100)


why  index.search.backend is overridden by globally managed value 
(elasticsearch) ?

Re: Seeking advice on Atlas XSS vulnerabilities

[Madhan Neethiraj]

Melinda,

Thank you for adding details. I think the frontend must take steps to prevent 
JavaScript execution while handling values in REST API responses.

Server side escaping all HTML characters in all responses wouldn’t be the right 
approach, as browser is not the only consumer of the responses. Consider 
applications that parse response from Atlas REST APIs; all such applications 
must be updated to deal with escaped HTML characters in responses. This will 
likely break existing Atlas integrations.

Hope this helps.

Regards,
Madhan

On 6/8/20, 6:31 PM, "Melinda Crane"  wrote:

Apologies, looks like the second half of the email I just sent got
formatted weirdly (I was copying from a failed earlier email attempt that
got rejected for attaching images). Just to make sure it doesn’t get buried:

Of course this specific case can be patched on the frontend, but doing some
kind of HTML sanitization escaping into the whatever-is-handling-the-ajax
(EntityREST or AtlasEntityStoreV2, maybe?) to blanket-catch HTML characters
sounds safer.

Hope this helps express my concerns.
From, Melinda Crane


On Mon, Jun 8, 2020 at 9:13 PM Melinda Crane  wrote:

> Dear Madhan,
>
> Thank you kindly for your response and for offering to look into potential
> solutions. I've been trying to step through how the ajax calls are 
handled,
> I don't really see the AtlasJsonProvider being used much at all actually
> for that. My concern is that the data from the ajax calls aren't being
> sanitized on the backend. I see there is a ton of sanitization happening 
on
> the frontend (lots of calls to _.escape()), but they're not all captured.
> For example.
>
> In CommonViewFunction, in the propertyTable
> 

 function,
> URLs are not escaped. So I can create an entity that has a key value of
>
> https://www.google.com/search?\;> height='\''1'\'' onload='\''window.alert(\"boo\");'\''>
>
> and my script will successfully get successfully injected into my browser
> when I try to view it. Looks like I can't attach images here but here's 
the
> elements:
>
> 
> 
>  class="hide-empty-value">
> 
> newSchema
> 
> 
> N/A
> 
> 
> 
> 
> classField
> 
> 
>  href="https://www.google.com/search?;>
>  height="1" onload="window.alert(boo);">
> "https://www.google.com/search
> ?"
>  height="1" onload="window.alert(boo);">
> ...
>
> ^You can see the iframe-script combo here.
>
> Of course this specific case can be patched on the frontend, but doing
> some kind of HTML sanitization escaping on the
> whatever-is-handling-the-ajax (EntityREST or AtlasEntityStoreV2, maybe?) 
to
> blanket-catch HTML characters sounds safer.
>
> Hope this helps express my concerns.
> From, Melinda Crane
>
>
> On Sat, Jun 6, 2020 at 2:48 PM Madhan Neethiraj  wrote:
>
>> Melinda,
>>
>> Thank you for reaching out to Apache Atlas community.
>>
>> As you noted, AtlasJsonProvider is used to deserialize/serialize REST API
>> requests and responses. In addition, methods in AtlasJson are used in to
>> convert to/from Json. It will help if you can add few examples of 
potential
>> issues with the current implementation.
>>
>> Keval is looking into the frontend for potential issues and should get
>> back early next week.
>>
>> Thanks,
>> Madhan
>>
>>
>>
>>
>> On 5/29/20, 6:50 PM, "Melinda Crane"  wrote:
>>
>> Dear Apache Atlas devs,
>>
>> I’m from Snapchat, we’re working on building a catalog on top of
>> Apache
>> Atlas. I noticed looking at the frontend for Atlas that there seem to
>> be
>> several places vulnerable to XSS, so I’ve got some questions for the
>> community (especially frontend folks) or other partner companies who
>> may
>> have dealt with similar problems:
>>
>>
>>1.
>>
>>the CSP allows unsafe-inline and unsafe-eval
>>2.
>>
>>the backend JSON content provider, AtlasJsonProvider, doesn’t
>> appear to
>>do any forced 

Re: Seeking advice on Atlas XSS vulnerabilities

[Melinda Crane]

Apologies, looks like the second half of the email I just sent got
formatted weirdly (I was copying from a failed earlier email attempt that
got rejected for attaching images). Just to make sure it doesn’t get buried:

Of course this specific case can be patched on the frontend, but doing some
kind of HTML sanitization escaping into the whatever-is-handling-the-ajax
(EntityREST or AtlasEntityStoreV2, maybe?) to blanket-catch HTML characters
sounds safer.

Hope this helps express my concerns.
From, Melinda Crane


On Mon, Jun 8, 2020 at 9:13 PM Melinda Crane  wrote:

> Dear Madhan,
>
> Thank you kindly for your response and for offering to look into potential
> solutions. I've been trying to step through how the ajax calls are handled,
> I don't really see the AtlasJsonProvider being used much at all actually
> for that. My concern is that the data from the ajax calls aren't being
> sanitized on the backend. I see there is a ton of sanitization happening on
> the frontend (lots of calls to _.escape()), but they're not all captured.
> For example.
>
> In CommonViewFunction, in the propertyTable
> 
>  function,
> URLs are not escaped. So I can create an entity that has a key value of
>
> https://www.google.com/search?\;> height='\''1'\'' onload='\''window.alert(\"boo\");'\''>
>
> and my script will successfully get successfully injected into my browser
> when I try to view it. Looks like I can't attach images here but here's the
> elements:
>
> 
> 
>  class="hide-empty-value">
> 
> newSchema
> 
> 
> N/A
> 
> 
> 
> 
> classField
> 
> 
>  href="https://www.google.com/search?;>
>  height="1" onload="window.alert(boo);">
> "https://www.google.com/search
> ?"
>  height="1" onload="window.alert(boo);">
> ...
>
> ^You can see the iframe-script combo here.
>
> Of course this specific case can be patched on the frontend, but doing
> some kind of HTML sanitization escaping on the
> whatever-is-handling-the-ajax (EntityREST or AtlasEntityStoreV2, maybe?) to
> blanket-catch HTML characters sounds safer.
>
> Hope this helps express my concerns.
> From, Melinda Crane
>
>
> On Sat, Jun 6, 2020 at 2:48 PM Madhan Neethiraj  wrote:
>
>> Melinda,
>>
>> Thank you for reaching out to Apache Atlas community.
>>
>> As you noted, AtlasJsonProvider is used to deserialize/serialize REST API
>> requests and responses. In addition, methods in AtlasJson are used in to
>> convert to/from Json. It will help if you can add few examples of potential
>> issues with the current implementation.
>>
>> Keval is looking into the frontend for potential issues and should get
>> back early next week.
>>
>> Thanks,
>> Madhan
>>
>>
>>
>>
>> On 5/29/20, 6:50 PM, "Melinda Crane"  wrote:
>>
>> Dear Apache Atlas devs,
>>
>> I’m from Snapchat, we’re working on building a catalog on top of
>> Apache
>> Atlas. I noticed looking at the frontend for Atlas that there seem to
>> be
>> several places vulnerable to XSS, so I’ve got some questions for the
>> community (especially frontend folks) or other partner companies who
>> may
>> have dealt with similar problems:
>>
>>
>>1.
>>
>>the CSP allows unsafe-inline and unsafe-eval
>>2.
>>
>>the backend JSON content provider, AtlasJsonProvider, doesn’t
>> appear to
>>do any forced escaping.
>>
>>
>> Regarding the unsafe-inline and unsafe-eval, I saw some inline
>> scripting
>> going on with the index html file in dashboardv2/v3, but that doesn’t
>> look
>> bad to move out. However, I noticed a lot of the third party node
>> modules
>> use inline scripting (such as backgrid-filter and
>> bootstrap-daterangepicker) and/or eval() (such as
>> javascript-stringify and
>> even requirejs). I’d like to get rid of unsafe-inline and unsafe-eval
>> in my
>> CSP; are there any recommendations on how to deal with the third party
>> plugins to achieve this?
>>
>> For the forced escaping, are there any other major places on backend
>> besides the JSON provider that should definitely be escaped? Also,
>> since
>> the AtlasJsonProvider seems to be *the* default object mapper, are
>> there
>> any insidious or otherwise consequences of force escaping HTML
>> sensitive
>> characters in it?
>>
>> Thank you kindly for any advice.
>>
>> From, 

Re: Seeking advice on Atlas XSS vulnerabilities

[Melinda Crane]

Dear Madhan,

Thank you kindly for your response and for offering to look into potential
solutions. I've been trying to step through how the ajax calls are handled,
I don't really see the AtlasJsonProvider being used much at all actually
for that. My concern is that the data from the ajax calls aren't being
sanitized on the backend. I see there is a ton of sanitization happening on
the frontend (lots of calls to _.escape()), but they're not all captured.
For example.

In CommonViewFunction, in the propertyTable

function,
URLs are not escaped. So I can create an entity that has a key value of

https://www.google.com/search?\;>

and my script will successfully get successfully injected into my browser
when I try to view it. Looks like I can't attach images here but here's the
elements:





newSchema


N/A




classField


https://www.google.com/search?;>

"https://www.google.com/search
?"

...

^You can see the iframe-script combo here.

Of course this specific case can be patched on the frontend, but doing some
kind of HTML sanitization escaping on the whatever-is-handling-the-ajax
(EntityREST or AtlasEntityStoreV2, maybe?) to blanket-catch HTML characters
sounds safer.

Hope this helps express my concerns.
From, Melinda Crane


On Sat, Jun 6, 2020 at 2:48 PM Madhan Neethiraj  wrote:

> Melinda,
>
> Thank you for reaching out to Apache Atlas community.
>
> As you noted, AtlasJsonProvider is used to deserialize/serialize REST API
> requests and responses. In addition, methods in AtlasJson are used in to
> convert to/from Json. It will help if you can add few examples of potential
> issues with the current implementation.
>
> Keval is looking into the frontend for potential issues and should get
> back early next week.
>
> Thanks,
> Madhan
>
>
>
>
> On 5/29/20, 6:50 PM, "Melinda Crane"  wrote:
>
> Dear Apache Atlas devs,
>
> I’m from Snapchat, we’re working on building a catalog on top of Apache
> Atlas. I noticed looking at the frontend for Atlas that there seem to
> be
> several places vulnerable to XSS, so I’ve got some questions for the
> community (especially frontend folks) or other partner companies who
> may
> have dealt with similar problems:
>
>
>1.
>
>the CSP allows unsafe-inline and unsafe-eval
>2.
>
>the backend JSON content provider, AtlasJsonProvider, doesn’t
> appear to
>do any forced escaping.
>
>
> Regarding the unsafe-inline and unsafe-eval, I saw some inline
> scripting
> going on with the index html file in dashboardv2/v3, but that doesn’t
> look
> bad to move out. However, I noticed a lot of the third party node
> modules
> use inline scripting (such as backgrid-filter and
> bootstrap-daterangepicker) and/or eval() (such as javascript-stringify
> and
> even requirejs). I’d like to get rid of unsafe-inline and unsafe-eval
> in my
> CSP; are there any recommendations on how to deal with the third party
> plugins to achieve this?
>
> For the forced escaping, are there any other major places on backend
> besides the JSON provider that should definitely be escaped? Also,
> since
> the AtlasJsonProvider seems to be *the* default object mapper, are
> there
> any insidious or otherwise consequences of force escaping HTML
> sensitive
> characters in it?
>
> Thank you kindly for any advice.
>
> From, Melinda Crane
>
>
>

[jira] [Commented] (ATLAS-3826) Basic search not using index query for string tokenized attributes

[ASF subversion and git services (Jira)]

[ 
https://issues.apache.org/jira/browse/ATLAS-3826?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17128741#comment-17128741
 ] 

ASF subversion and git services commented on ATLAS-3826:


Commit 7604c6d09927146a70964a429e588419fc06d640 in atlas's branch 
refs/heads/branch-2.0 from Sarath Subramanian
[ https://gitbox.apache.org/repos/asf?p=atlas.git;h=7604c6d ]

ATLAS-3826: Basic search not using index query for string tokenized attributes

(cherry picked from commit 931d6efdc97f742370df334761673af70f240837)


> Basic search not using index query for string tokenized attributes
> --
>
> Key: ATLAS-3826
> URL: https://issues.apache.org/jira/browse/ATLAS-3826
> Project: Atlas
>  Issue Type: Bug
>  Components:  atlas-core
>Affects Versions: 2.0.0
>Reporter: Sarath Subramanian
>Assignee: Sarath Subramanian
>Priority: Major
>  Labels: BasicSearch
> Fix For: 2.1.0
>
>
> Basic search not using index query for string tokenized attributes. 
> The following string tokenized attributes:
> {code:java}
> Asset.__s_name
>  Asset.__s_owner
>  Asset.__s_displayName
>  Asset.__s_userDescription
>  _s__createdBy
>  _s__modifiedBy{code}
>  
> uses graph query and this might be expensive when there are large number of 
> entities in the graph. String tokenized attributes should use index query for 
> faster performance.
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (ATLAS-3826) Basic search not using index query for string tokenized attributes

[ASF subversion and git services (Jira)]

[ 
https://issues.apache.org/jira/browse/ATLAS-3826?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17128740#comment-17128740
 ] 

ASF subversion and git services commented on ATLAS-3826:


Commit 931d6efdc97f742370df334761673af70f240837 in atlas's branch 
refs/heads/master from Sarath Subramanian
[ https://gitbox.apache.org/repos/asf?p=atlas.git;h=931d6ef ]

ATLAS-3826: Basic search not using index query for string tokenized attributes


> Basic search not using index query for string tokenized attributes
> --
>
> Key: ATLAS-3826
> URL: https://issues.apache.org/jira/browse/ATLAS-3826
> Project: Atlas
>  Issue Type: Bug
>  Components:  atlas-core
>Affects Versions: 2.0.0
>Reporter: Sarath Subramanian
>Assignee: Sarath Subramanian
>Priority: Major
>  Labels: BasicSearch
> Fix For: 2.1.0
>
>
> Basic search not using index query for string tokenized attributes. 
> The following string tokenized attributes:
> {code:java}
> Asset.__s_name
>  Asset.__s_owner
>  Asset.__s_displayName
>  Asset.__s_userDescription
>  _s__createdBy
>  _s__modifiedBy{code}
>  
> uses graph query and this might be expensive when there are large number of 
> entities in the graph. String tokenized attributes should use index query for 
> faster performance.
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: [VOTE] Release Apache Atlas version 2.1.0 - rc1

[sidharth mishra]

+1 for Apache Atlas 2.1 RC1

Validated the following items.

   1. Verified the md5/sha512 checksum of source bundle
   2. Packaging with -Pdist,embedded-hbase-solr works fine
   3. Starting Atlas server works fine
   4. quick_start.py works fine and all the entities and tags get created
successfully at UI
   5. Tested basic and quick search flow
   6. Tested entity and tag delete
   7. Tested entities purge
   8. Checked admin audit for entity purge

Regards,
Sid

On Mon, Jun 8, 2020 at 11:41 AM Madhan Neethiraj  wrote:

> Bolke,
>
> Can you send specific security concerns that need to addressed in 2.1.0
> release? If this is about questions from Melinda Crane, I asked for Melinda
> details to better understand the questions. I suggest we know the details
> before treating them as release blockers.
>
> Thanks,
> Madhan
>
> On 6/8/20, 9:56 AM, "Bolke de Bruin"  wrote:
>
> This is my -1 (non-binding) which I will revert to +1 when the
> security concerns have been addressed/clarified.
>
> Given that Atlas has pretty slow releases I think it smart to address
> them now.
>
> Cheers
> Bolke
>
> Sent from my iPhone
>
> > On 8 Jun 2020, at 18:18, Sharmadha Sainath
>  wrote:
> >
> > +1 for Apache Atlas 2.1 RC1
> >
> >
> >   - Validated MD5 and SHA512 checksum of source.
> >   - Brought up Atlas using embedded HBase and Solr.
> >   - Verified DSL , Basic Search , Quick search and Suggestions
> >   - Verified Basic search using system attributes filters (multiple
> >   classification search , labels , custom attributes , business
> metadata
> >   search)
> >   - Added labels , user defined properties (custom attributes) ,
> business
> >   metadata to entities and verified the entity properties and audits.
> >   - Purged Soft Deleted entity and verified purge audits.
> >   - Saved the Favorite searches and executed them.
> >
> > Thanks,
> > Sharmadha S.
> >
> >
> >
> >> On Mon, Jun 8, 2020 at 6:23 PM Nixon Rodrigues <
> >> nixon.rodrig...@freestoneinfotech.com> wrote:
> >>
> >> Thanks Madhan for putting 2.1 RC1 together for the vote.
> >>
> >> +1 for Apache Atlas 2.1 RC1
> >>
> >> Validated the following items.
> >>
> >>   1. verified the md5/sha checksum of source bundle.
> >>   2. packaging with -Pdist,embedded-hbase-solr is good.
> >>   3. Unit testcase passes.
> >>   4. Run quickstart and all the types and entities created are
> loading
> >>   correctly on the UI.
> >>   5. Create tags, glossary, term and its apply correctly on
> entities.
> >>   6. Created business metadata with attributes and applied it to
> entities.
> >>   7. tested basic and quick search flow.
> >>   8. Verified statistics page details
> >>
> >>
> >> Thanks & Regards
> >> Nixon
> >>
> >>
> >>
> >> On Mon, Jun 8, 2020 at 11:17 AM Sarath Subramanian <
> sar...@apache.org>
> >> wrote:
> >>
> >>> +1 for Apache Atlas 2.1 RC1
> >>>
> >>> Validated the following:
> >>>
> >>>   - No binaries in source
> >>>   - Verified source checksum - md5 and sha512
> >>>   - Verified signature
> >>>   - Build was successful on the source (mvn clean -DskipTests
> package
> >>>   -Pdist,embedded-hbase-solr)
> >>>   - Ran quickStart and validated types and entities were created
> >>>   successfully.
> >>>   - Validated tag propagation works
> >>>   - Added/removed labels to an entity and verified audit is getting
> >>>   updated correctly
> >>>   - Added user-defined properties and business metadata attributes.
> >>>   - Deleted few entities and purged those entities - validated if
> purge
> >>>   audit recorded correctly.
> >>>   - Basic sanity tests - basic search (with system attributes),
> advanced
> >>>   search, quick search.
> >>>
> >>>
> >>>
> >>> Thanks,
> >>> Sarath
> >>>
> >>>
> >>>
> >>> On Sat, Jun 6, 2020 at 10:22 AM Madhan Neethiraj <
> mad...@apache.org>
> >>> wrote:
> >>>
>  (resending with plain-text format)
> 
>  Atlas team,
> 
>  Apache Atlas 2.1.0 rc1, with following fixes since rc0, is now
> >> available
>  for vote within dev community.
> 
> ATLAS-3770: UI(Classic): Active and Deleted hyperlinks for
> certain
>  entities throwing error on click
> ATLAS-3766: Stats modal not close issue #2
> ATLAS-3674: ZipFileMigationImporter: Migration status display
> fix.
>  Part 2
> 
>  Links to the release artifacts are given below. Please review and
> vote.
> 
>  The vote will be open for at least 72 hours or until necessary
> votes
> >> are
>  reached.
>   [ ] +1 Approve
>   [ ] +0 No opinion
>   [ ] -1 Disapprove (and reason why)
> 

Re: Review Request 72576: ATLAS-3826: Basic search not using index query for string tokenized attributes

[Madhan Neethiraj]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72576/#review220975
---


Ship it!




Ship It!

- Madhan Neethiraj


On June 8, 2020, 10:48 p.m., Sarath Subramanian wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/72576/
> ---
> 
> (Updated June 8, 2020, 10:48 p.m.)
> 
> 
> Review request for atlas, Ashutosh Mestry, chaitali, Jayendra Parab, Madhan 
> Neethiraj, Mandar Ambawane, mayank, mayank jain, Nixon Rodrigues, Pinal Shah, 
> and Sidharth Mishra.
> 
> 
> Bugs: ATLAS-3826
> https://issues.apache.org/jira/browse/ATLAS-3826
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> Basic search not using index query for string tokenized attributes. 
> 
> The following string tokenized attributes:
> 
> Asset.__s_name
>  Asset.__s_owner
>  Asset.__s_displayName
>  Asset.__s_userDescription
>  _s__createdBy
>  _s__modifiedBy
>  
> 
> uses graph query and this might be expensive when there are large number of 
> entities in the graph. String tokenized attributes should use index query for 
> faster performance.
> 
> 
> Diffs
> -
> 
>   intg/src/main/java/org/apache/atlas/type/AtlasEntityType.java 3962c3c42 
>   intg/src/main/java/org/apache/atlas/type/AtlasStructType.java 4b0027ea5 
>   repository/src/main/java/org/apache/atlas/discovery/SearchProcessor.java 
> 2c5f925c7 
>   repository/src/main/java/org/apache/atlas/query/RegistryBasedLookup.java 
> d53690081 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasGraphUtilsV2.java
>  fc5d48d22 
> 
> 
> Diff: https://reviews.apache.org/r/72576/diff/2/
> 
> 
> Testing
> ---
> 
> manually validated, string tokenized attributes uses index query.
> 
> Precommit - 
> https://builds.apache.org/view/A/view/Atlas/job/PreCommit-ATLAS-Build-Test/1957/console
> 
> 
> Thanks,
> 
> Sarath Subramanian
> 
>

Re: Review Request 72576: ATLAS-3826: Basic search not using index query for string tokenized attributes

[Sarath Subramanian]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72576/
---

(Updated June 8, 2020, 3:48 p.m.)


Review request for atlas, Ashutosh Mestry, chaitali, Jayendra Parab, Madhan 
Neethiraj, Mandar Ambawane, mayank, mayank jain, Nixon Rodrigues, Pinal Shah, 
and Sidharth Mishra.


Changes
---

method renamed.


Bugs: ATLAS-3826
https://issues.apache.org/jira/browse/ATLAS-3826


Repository: atlas


Description
---

Basic search not using index query for string tokenized attributes. 

The following string tokenized attributes:

Asset.__s_name
 Asset.__s_owner
 Asset.__s_displayName
 Asset.__s_userDescription
 _s__createdBy
 _s__modifiedBy
 

uses graph query and this might be expensive when there are large number of 
entities in the graph. String tokenized attributes should use index query for 
faster performance.


Diffs (updated)
-

  intg/src/main/java/org/apache/atlas/type/AtlasEntityType.java 3962c3c42 
  intg/src/main/java/org/apache/atlas/type/AtlasStructType.java 4b0027ea5 
  repository/src/main/java/org/apache/atlas/discovery/SearchProcessor.java 
2c5f925c7 
  repository/src/main/java/org/apache/atlas/query/RegistryBasedLookup.java 
d53690081 
  
repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasGraphUtilsV2.java
 fc5d48d22 


Diff: https://reviews.apache.org/r/72576/diff/2/

Changes: https://reviews.apache.org/r/72576/diff/1-2/


Testing
---

manually validated, string tokenized attributes uses index query.

Precommit - 
https://builds.apache.org/view/A/view/Atlas/job/PreCommit-ATLAS-Build-Test/1957/console


Thanks,

Sarath Subramanian

Re: Review Request 72576: ATLAS-3826: Basic search not using index query for string tokenized attributes

[Madhan Neethiraj]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72576/#review220974
---


Fix it, then Ship it!





intg/src/main/java/org/apache/atlas/type/AtlasEntityType.java
Line 594 (original), 594 (patched)


Consider renaming getQualifiedAttributeName() => getVertexPropertyName(); 
this will help avoid potential confusion.


- Madhan Neethiraj


On June 8, 2020, 9:36 p.m., Sarath Subramanian wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/72576/
> ---
> 
> (Updated June 8, 2020, 9:36 p.m.)
> 
> 
> Review request for atlas, Ashutosh Mestry, chaitali, Jayendra Parab, Madhan 
> Neethiraj, Mandar Ambawane, mayank, mayank jain, Nixon Rodrigues, Pinal Shah, 
> and Sidharth Mishra.
> 
> 
> Bugs: ATLAS-3826
> https://issues.apache.org/jira/browse/ATLAS-3826
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> Basic search not using index query for string tokenized attributes. 
> 
> The following string tokenized attributes:
> 
> Asset.__s_name
>  Asset.__s_owner
>  Asset.__s_displayName
>  Asset.__s_userDescription
>  _s__createdBy
>  _s__modifiedBy
>  
> 
> uses graph query and this might be expensive when there are large number of 
> entities in the graph. String tokenized attributes should use index query for 
> faster performance.
> 
> 
> Diffs
> -
> 
>   intg/src/main/java/org/apache/atlas/type/AtlasEntityType.java 3962c3c42 
>   intg/src/main/java/org/apache/atlas/type/AtlasStructType.java 4b0027ea5 
> 
> 
> Diff: https://reviews.apache.org/r/72576/diff/1/
> 
> 
> Testing
> ---
> 
> manually validated, string tokenized attributes uses index query.
> 
> Precommit - 
> https://builds.apache.org/view/A/view/Atlas/job/PreCommit-ATLAS-Build-Test/1957/console
> 
> 
> Thanks,
> 
> Sarath Subramanian
> 
>

Review Request 72576: ATLAS-3826: Basic search not using index query for string tokenized attributes

[Sarath Subramanian]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72576/
---

Review request for atlas, Ashutosh Mestry, chaitali, Jayendra Parab, Madhan 
Neethiraj, Mandar Ambawane, mayank, mayank jain, Nixon Rodrigues, Pinal Shah, 
and Sidharth Mishra.


Bugs: ATLAS-3826
https://issues.apache.org/jira/browse/ATLAS-3826


Repository: atlas


Description
---

Basic search not using index query for string tokenized attributes. 

The following string tokenized attributes:

Asset.__s_name
 Asset.__s_owner
 Asset.__s_displayName
 Asset.__s_userDescription
 _s__createdBy
 _s__modifiedBy
 

uses graph query and this might be expensive when there are large number of 
entities in the graph. String tokenized attributes should use index query for 
faster performance.


Diffs
-

  intg/src/main/java/org/apache/atlas/type/AtlasEntityType.java 3962c3c42 
  intg/src/main/java/org/apache/atlas/type/AtlasStructType.java 4b0027ea5 


Diff: https://reviews.apache.org/r/72576/diff/1/


Testing
---

manually validated, string tokenized attributes uses index query.

Precommit - 
https://builds.apache.org/view/A/view/Atlas/job/PreCommit-ATLAS-Build-Test/1957/console


Thanks,

Sarath Subramanian

[jira] [Updated] (ATLAS-3826) Basic search not using index query for string tokenized attributes

[Sarath Subramanian (Jira)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-3826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sarath Subramanian updated ATLAS-3826:
--
Labels: BasicSearch  (was: )

> Basic search not using index query for string tokenized attributes
> --
>
> Key: ATLAS-3826
> URL: https://issues.apache.org/jira/browse/ATLAS-3826
> Project: Atlas
>  Issue Type: Bug
>  Components:  atlas-core
>Affects Versions: 2.0.0
>Reporter: Sarath Subramanian
>Assignee: Sarath Subramanian
>Priority: Major
>  Labels: BasicSearch
> Fix For: 2.1.0
>
>
> Basic search not using index query for string tokenized attributes. 
> The following string tokenized attributes:
> {code:java}
> Asset.__s_name
>  Asset.__s_owner
>  Asset.__s_displayName
>  Asset.__s_userDescription
>  _s__createdBy
>  _s__modifiedBy{code}
>  
> uses graph query and this might be expensive when there are large number of 
> entities in the graph. String tokenized attributes should use index query for 
> faster performance.
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (ATLAS-3826) Basic search not using index query for string tokenized attributes

[Sarath Subramanian (Jira)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-3826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sarath Subramanian updated ATLAS-3826:
--
Description: 
Basic search not using index query for string tokenized attributes. 

The following string tokenized attributes:
{code:java}
Asset.__s_name
 Asset.__s_owner
 Asset.__s_displayName
 Asset.__s_userDescription
 _s__createdBy
 _s__modifiedBy{code}
 

uses graph query and this might be expensive when there are large number of 
entities in the graph. String tokenized attributes should use index query for 
faster performance.

 

  was:
Basic search not using index query for string tokenized attributes. 

The following string tokenized attributes:
Asset.__s_name
Asset.__s_owner
Asset.__s_displayName
Asset.__s_userDescription
__s___createdBy
__s___modifiedBy
 

uses graph query and this might be expensive when there are large number of 
entities in the graph. String tokenized attributes should use index query for 
faster performance.

 


> Basic search not using index query for string tokenized attributes
> --
>
> Key: ATLAS-3826
> URL: https://issues.apache.org/jira/browse/ATLAS-3826
> Project: Atlas
>  Issue Type: Bug
>  Components:  atlas-core
>Affects Versions: 2.0.0
>Reporter: Sarath Subramanian
>Assignee: Sarath Subramanian
>Priority: Major
> Fix For: 2.1.0
>
>
> Basic search not using index query for string tokenized attributes. 
> The following string tokenized attributes:
> {code:java}
> Asset.__s_name
>  Asset.__s_owner
>  Asset.__s_displayName
>  Asset.__s_userDescription
>  _s__createdBy
>  _s__modifiedBy{code}
>  
> uses graph query and this might be expensive when there are large number of 
> entities in the graph. String tokenized attributes should use index query for 
> faster performance.
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (ATLAS-3826) Basic search not using index query for string tokenized attributes

[Sarath Subramanian (Jira)]

Sarath Subramanian created ATLAS-3826:
-

 Summary: Basic search not using index query for string tokenized 
attributes
 Key: ATLAS-3826
 URL: https://issues.apache.org/jira/browse/ATLAS-3826
 Project: Atlas
  Issue Type: Bug
  Components:  atlas-core
Affects Versions: 2.0.0
Reporter: Sarath Subramanian
Assignee: Sarath Subramanian
 Fix For: 2.1.0


Basic search not using index query for string tokenized attributes. 

The following string tokenized attributes:
Asset.__s_name
Asset.__s_owner
Asset.__s_displayName
Asset.__s_userDescription
__s___createdBy
__s___modifiedBy
 

uses graph query and this might be expensive when there are large number of 
entities in the graph. String tokenized attributes should use index query for 
faster performance.

 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: [VOTE] Release Apache Atlas version 2.1.0 - rc1

[Madhan Neethiraj]

Bolke,

Can you send specific security concerns that need to addressed in 2.1.0 
release? If this is about questions from Melinda Crane, I asked for Melinda 
details to better understand the questions. I suggest we know the details 
before treating them as release blockers.

Thanks,
Madhan

On 6/8/20, 9:56 AM, "Bolke de Bruin"  wrote:

This is my -1 (non-binding) which I will revert to +1 when the security 
concerns have been addressed/clarified. 

Given that Atlas has pretty slow releases I think it smart to address them 
now. 

Cheers
Bolke

Sent from my iPhone

> On 8 Jun 2020, at 18:18, Sharmadha Sainath 
 wrote:
> 
> +1 for Apache Atlas 2.1 RC1
> 
> 
>   - Validated MD5 and SHA512 checksum of source.
>   - Brought up Atlas using embedded HBase and Solr.
>   - Verified DSL , Basic Search , Quick search and Suggestions
>   - Verified Basic search using system attributes filters (multiple
>   classification search , labels , custom attributes , business metadata
>   search)
>   - Added labels , user defined properties (custom attributes) , business
>   metadata to entities and verified the entity properties and audits.
>   - Purged Soft Deleted entity and verified purge audits.
>   - Saved the Favorite searches and executed them.
> 
> Thanks,
> Sharmadha S.
> 
> 
> 
>> On Mon, Jun 8, 2020 at 6:23 PM Nixon Rodrigues <
>> nixon.rodrig...@freestoneinfotech.com> wrote:
>> 
>> Thanks Madhan for putting 2.1 RC1 together for the vote.
>> 
>> +1 for Apache Atlas 2.1 RC1
>> 
>> Validated the following items.
>> 
>>   1. verified the md5/sha checksum of source bundle.
>>   2. packaging with -Pdist,embedded-hbase-solr is good.
>>   3. Unit testcase passes.
>>   4. Run quickstart and all the types and entities created are loading
>>   correctly on the UI.
>>   5. Create tags, glossary, term and its apply correctly on entities.
>>   6. Created business metadata with attributes and applied it to 
entities.
>>   7. tested basic and quick search flow.
>>   8. Verified statistics page details
>> 
>> 
>> Thanks & Regards
>> Nixon
>> 
>> 
>> 
>> On Mon, Jun 8, 2020 at 11:17 AM Sarath Subramanian 
>> wrote:
>> 
>>> +1 for Apache Atlas 2.1 RC1
>>> 
>>> Validated the following:
>>> 
>>>   - No binaries in source
>>>   - Verified source checksum - md5 and sha512
>>>   - Verified signature
>>>   - Build was successful on the source (mvn clean -DskipTests package
>>>   -Pdist,embedded-hbase-solr)
>>>   - Ran quickStart and validated types and entities were created
>>>   successfully.
>>>   - Validated tag propagation works
>>>   - Added/removed labels to an entity and verified audit is getting
>>>   updated correctly
>>>   - Added user-defined properties and business metadata attributes.
>>>   - Deleted few entities and purged those entities - validated if purge
>>>   audit recorded correctly.
>>>   - Basic sanity tests - basic search (with system attributes), advanced
>>>   search, quick search.
>>> 
>>> 
>>> 
>>> Thanks,
>>> Sarath
>>> 
>>> 
>>> 
>>> On Sat, Jun 6, 2020 at 10:22 AM Madhan Neethiraj 
>>> wrote:
>>> 
 (resending with plain-text format)
 
 Atlas team,
 
 Apache Atlas 2.1.0 rc1, with following fixes since rc0, is now
>> available
 for vote within dev community.
 
ATLAS-3770: UI(Classic): Active and Deleted hyperlinks for certain
 entities throwing error on click
ATLAS-3766: Stats modal not close issue #2
ATLAS-3674: ZipFileMigationImporter: Migration status display fix.
 Part 2
 
 Links to the release artifacts are given below. Please review and vote.
 
 The vote will be open for at least 72 hours or until necessary votes
>> are
 reached.
  [ ] +1 Approve
  [ ] +0 No opinion
  [ ] -1 Disapprove (and reason why)
 
 Thanks,
 Madhan
 
 
 List of all issues addressed in this release:
 https://issues.apache.org/jira/issues/?jql=project=ATLAS AND
 status=Resolved AND fixVersion=2.1.0 ORDER BY key DESC
 
 Git tag for the release:
 https://github.com/apache/atlas/tree/release-2.1.0-rc1
 Sources for the release:
 
>>> 
>> 
https://dist.apache.org/repos/dist/dev/atlas/2.1.0-rc1/apache-atlas-2.1.0-sources.tar.gz
 
 Source release verification:
  PGP Signature:
 
>>> 
>> 
https://dist.apache.org/repos/dist/dev/atlas/2.1.0-rc1/apache-atlas-2.1.0-sources.tar.gz.asc
  SHA512 Hash:
 
>>> 
>> 

Re: [VOTE] Release Apache Atlas version 2.1.0 - rc1

[Bolke de Bruin]

This is my -1 (non-binding) which I will revert to +1 when the security 
concerns have been addressed/clarified. 

Given that Atlas has pretty slow releases I think it smart to address them now. 

Cheers
Bolke

Sent from my iPhone

> On 8 Jun 2020, at 18:18, Sharmadha Sainath  
> wrote:
> 
> +1 for Apache Atlas 2.1 RC1
> 
> 
>   - Validated MD5 and SHA512 checksum of source.
>   - Brought up Atlas using embedded HBase and Solr.
>   - Verified DSL , Basic Search , Quick search and Suggestions
>   - Verified Basic search using system attributes filters (multiple
>   classification search , labels , custom attributes , business metadata
>   search)
>   - Added labels , user defined properties (custom attributes) , business
>   metadata to entities and verified the entity properties and audits.
>   - Purged Soft Deleted entity and verified purge audits.
>   - Saved the Favorite searches and executed them.
> 
> Thanks,
> Sharmadha S.
> 
> 
> 
>> On Mon, Jun 8, 2020 at 6:23 PM Nixon Rodrigues <
>> nixon.rodrig...@freestoneinfotech.com> wrote:
>> 
>> Thanks Madhan for putting 2.1 RC1 together for the vote.
>> 
>> +1 for Apache Atlas 2.1 RC1
>> 
>> Validated the following items.
>> 
>>   1. verified the md5/sha checksum of source bundle.
>>   2. packaging with -Pdist,embedded-hbase-solr is good.
>>   3. Unit testcase passes.
>>   4. Run quickstart and all the types and entities created are loading
>>   correctly on the UI.
>>   5. Create tags, glossary, term and its apply correctly on entities.
>>   6. Created business metadata with attributes and applied it to entities.
>>   7. tested basic and quick search flow.
>>   8. Verified statistics page details
>> 
>> 
>> Thanks & Regards
>> Nixon
>> 
>> 
>> 
>> On Mon, Jun 8, 2020 at 11:17 AM Sarath Subramanian 
>> wrote:
>> 
>>> +1 for Apache Atlas 2.1 RC1
>>> 
>>> Validated the following:
>>> 
>>>   - No binaries in source
>>>   - Verified source checksum - md5 and sha512
>>>   - Verified signature
>>>   - Build was successful on the source (mvn clean -DskipTests package
>>>   -Pdist,embedded-hbase-solr)
>>>   - Ran quickStart and validated types and entities were created
>>>   successfully.
>>>   - Validated tag propagation works
>>>   - Added/removed labels to an entity and verified audit is getting
>>>   updated correctly
>>>   - Added user-defined properties and business metadata attributes.
>>>   - Deleted few entities and purged those entities - validated if purge
>>>   audit recorded correctly.
>>>   - Basic sanity tests - basic search (with system attributes), advanced
>>>   search, quick search.
>>> 
>>> 
>>> 
>>> Thanks,
>>> Sarath
>>> 
>>> 
>>> 
>>> On Sat, Jun 6, 2020 at 10:22 AM Madhan Neethiraj 
>>> wrote:
>>> 
 (resending with plain-text format)
 
 Atlas team,
 
 Apache Atlas 2.1.0 rc1, with following fixes since rc0, is now
>> available
 for vote within dev community.
 
ATLAS-3770: UI(Classic): Active and Deleted hyperlinks for certain
 entities throwing error on click
ATLAS-3766: Stats modal not close issue #2
ATLAS-3674: ZipFileMigationImporter: Migration status display fix.
 Part 2
 
 Links to the release artifacts are given below. Please review and vote.
 
 The vote will be open for at least 72 hours or until necessary votes
>> are
 reached.
  [ ] +1 Approve
  [ ] +0 No opinion
  [ ] -1 Disapprove (and reason why)
 
 Thanks,
 Madhan
 
 
 List of all issues addressed in this release:
 https://issues.apache.org/jira/issues/?jql=project=ATLAS AND
 status=Resolved AND fixVersion=2.1.0 ORDER BY key DESC
 
 Git tag for the release:
 https://github.com/apache/atlas/tree/release-2.1.0-rc1
 Sources for the release:
 
>>> 
>> https://dist.apache.org/repos/dist/dev/atlas/2.1.0-rc1/apache-atlas-2.1.0-sources.tar.gz
 
 Source release verification:
  PGP Signature:
 
>>> 
>> https://dist.apache.org/repos/dist/dev/atlas/2.1.0-rc1/apache-atlas-2.1.0-sources.tar.gz.asc
  SHA512 Hash:
 
>>> 
>> https://dist.apache.org/repos/dist/dev/atlas/2.1.0-rc1/apache-atlas-2.1.0-sources.tar.gz.sha512
  MD5 Hash:
 
>>> 
>> https://dist.apache.org/repos/dist/dev/atlas/2.1.0-rc1/apache-atlas-2.1.0-sources.tar.gz.md5
 
 Keys to verify the signature of the release artifacts are available at:
 https://dist.apache.org/repos/dist/release/atlas/KEYS
 
 New features/enhancements:
  - Quick Search: provides a simpler search experience with type-ahead
 suggestions
  - Business Metadata: enables augmenting entity-types with additional
 attributes, search entities using these attributes
  - Labels: ability to add/remove labels on entities, and search
>> entities
 using labels
  - Custom Attributes: ability to add entity instance specific custom
 attributes i.e. attributes not defined in entity-def or business
>> metadata
  - Entity Purge: added REST APIs to 

Re: [VOTE] Release Apache Atlas version 2.1.0 - rc1

[Sharmadha Sainath]

+1 for Apache Atlas 2.1 RC1


   - Validated MD5 and SHA512 checksum of source.
   - Brought up Atlas using embedded HBase and Solr.
   - Verified DSL , Basic Search , Quick search and Suggestions
   - Verified Basic search using system attributes filters (multiple
   classification search , labels , custom attributes , business metadata
   search)
   - Added labels , user defined properties (custom attributes) , business
   metadata to entities and verified the entity properties and audits.
   - Purged Soft Deleted entity and verified purge audits.
   - Saved the Favorite searches and executed them.

Thanks,
Sharmadha S.



On Mon, Jun 8, 2020 at 6:23 PM Nixon Rodrigues <
nixon.rodrig...@freestoneinfotech.com> wrote:

> Thanks Madhan for putting 2.1 RC1 together for the vote.
>
> +1 for Apache Atlas 2.1 RC1
>
> Validated the following items.
>
>1. verified the md5/sha checksum of source bundle.
>2. packaging with -Pdist,embedded-hbase-solr is good.
>3. Unit testcase passes.
>4. Run quickstart and all the types and entities created are loading
>correctly on the UI.
>5. Create tags, glossary, term and its apply correctly on entities.
>6. Created business metadata with attributes and applied it to entities.
>7. tested basic and quick search flow.
>8. Verified statistics page details
>
>
> Thanks & Regards
> Nixon
>
>
>
> On Mon, Jun 8, 2020 at 11:17 AM Sarath Subramanian 
> wrote:
>
> > +1 for Apache Atlas 2.1 RC1
> >
> > Validated the following:
> >
> >- No binaries in source
> >- Verified source checksum - md5 and sha512
> >- Verified signature
> >- Build was successful on the source (mvn clean -DskipTests package
> >-Pdist,embedded-hbase-solr)
> >- Ran quickStart and validated types and entities were created
> >successfully.
> >- Validated tag propagation works
> >- Added/removed labels to an entity and verified audit is getting
> >updated correctly
> >- Added user-defined properties and business metadata attributes.
> >- Deleted few entities and purged those entities - validated if purge
> >audit recorded correctly.
> >- Basic sanity tests - basic search (with system attributes), advanced
> >search, quick search.
> >
> >
> >
> > Thanks,
> > Sarath
> >
> >
> >
> > On Sat, Jun 6, 2020 at 10:22 AM Madhan Neethiraj 
> > wrote:
> >
> > > (resending with plain-text format)
> > >
> > > Atlas team,
> > >
> > > Apache Atlas 2.1.0 rc1, with following fixes since rc0, is now
> available
> > > for vote within dev community.
> > >
> > > ATLAS-3770: UI(Classic): Active and Deleted hyperlinks for certain
> > > entities throwing error on click
> > > ATLAS-3766: Stats modal not close issue #2
> > > ATLAS-3674: ZipFileMigationImporter: Migration status display fix.
> > > Part 2
> > >
> > > Links to the release artifacts are given below. Please review and vote.
> > >
> > > The vote will be open for at least 72 hours or until necessary votes
> are
> > > reached.
> > >   [ ] +1 Approve
> > >   [ ] +0 No opinion
> > >   [ ] -1 Disapprove (and reason why)
> > >
> > > Thanks,
> > > Madhan
> > >
> > >
> > > List of all issues addressed in this release:
> > > https://issues.apache.org/jira/issues/?jql=project=ATLAS AND
> > > status=Resolved AND fixVersion=2.1.0 ORDER BY key DESC
> > >
> > > Git tag for the release:
> > > https://github.com/apache/atlas/tree/release-2.1.0-rc1
> > > Sources for the release:
> > >
> >
> https://dist.apache.org/repos/dist/dev/atlas/2.1.0-rc1/apache-atlas-2.1.0-sources.tar.gz
> > >
> > > Source release verification:
> > >   PGP Signature:
> > >
> >
> https://dist.apache.org/repos/dist/dev/atlas/2.1.0-rc1/apache-atlas-2.1.0-sources.tar.gz.asc
> > >   SHA512 Hash:
> > >
> >
> https://dist.apache.org/repos/dist/dev/atlas/2.1.0-rc1/apache-atlas-2.1.0-sources.tar.gz.sha512
> > >   MD5 Hash:
> > >
> >
> https://dist.apache.org/repos/dist/dev/atlas/2.1.0-rc1/apache-atlas-2.1.0-sources.tar.gz.md5
> > >
> > > Keys to verify the signature of the release artifacts are available at:
> > > https://dist.apache.org/repos/dist/release/atlas/KEYS
> > >
> > > New features/enhancements:
> > >   - Quick Search: provides a simpler search experience with type-ahead
> > > suggestions
> > >   - Business Metadata: enables augmenting entity-types with additional
> > > attributes, search entities using these attributes
> > >   - Labels: ability to add/remove labels on entities, and search
> entities
> > > using labels
> > >   - Custom Attributes: ability to add entity instance specific custom
> > > attributes i.e. attributes not defined in entity-def or business
> metadata
> > >   - Entity Purge: added REST APIs to purge deleted entities
> > >
> > > Enhancements:
> > >   - Search: ability to find entities by more than one classification
> > >   - Performance: improvements in lineage retrieval and
> > > classification-propagation
> > >   - Notification: ability to process notifications from multiple Kafka
> > > topics
> > 

Re: Running Apache Atlas on prefix path instead of /

[Nixon Rodrigues]

Bolke,

Can you help to review if this is an issue with keycloak auth.

Regards,
Nixon


On Mon, Jun 8, 2020 at 12:57 PM Chitender Kumar 
wrote:

>
> @Keval Thank you.
>
> @Dev-group: looking forward for positive response.
>
> > On 07-Jun-2020, at 8:36 PM, Keval Bhatt  wrote:
> >
> > Hi Chitender,
> >
> > I am not sure about the solution I think anyone from server side could
> help you.
> >
> > I am adding atlas dev email in this thread.
> >
> >
> > Thanks
> >
> >
> >
> > On Fri, Jun 5, 2020, 17:54 Chitender Kumar  chiten...@atlan.com>> wrote:
> > Hello Keval,
> >
> > Hope you are doing good.
> >
> > Got your from GitHub.
> >
> > We are working on Apache Atlas in production setup. Where we are
> connecting it to Ranger and Keycloak for authentication and authorisation.
> The issue which we are facing is we are not able to figure out how to run
> Apache Atlas on prefix path like /atlas instead of /.
> >
> > If we configure /atlas in our K8s Nginx controller and when we hit
> DNS/atlas its getting redirected to /.
> >
> > Can you please help us out here or if you can redirect us to some
> reference who answer these query.
> >
> > Looking forward for your response.
> >
> > Thanks
> > Chitender
>
>

Re: build with profile embedded-cassandra-solr not work

[Madhan Neethiraj]

Starting with 0.5.0, JanusGraph doesn't support embedded Cassandra (see below 
from Atlas logs). Apache Atlas 2.1.0 use JanusGraph version 0.5.1. Please try 
with external Cassandra deployment.

2020-06-08 08:36:50,984 WARN  - [main:] ~ Cassandra embedded solution is 
deprecated and will be removed with JanusGraph 0.5.0. For testing, please use 
JanusGraph Docker images: https://github.com/JanusGraph/janusgraph-docker. For 
production, please use a side-by-side deployment of Cassandra or a pre-packaged 
version of Cassandra with a CQL backend in JanusGraph 0.5.0. For more details, 
see https://docs.janusgraph.org/basics/server/. 
(CassandraEmbeddedStoreManager:81)

Pierre - will you be able to help with Atlas 2.1.0 in Cassandra configuration?

Madhan



On 6/8/20, 6:21 AM, "lk_hadoop"  wrote:

hi,all:




 Forwarding messages 
From: "lk_hadoop" 
Date: 2020-06-08 21:15:24
To: u...@atlas.apache.org
Subject: build with profile embedded-cassandra-solr not work

hi,all:
I'm using atlas-release-2.1.0-rc0  build with : mvn clean -DskipTests 
package -Pdist,embedded-cassandra-solr
when I try atlas_start.py , I got error:
Context initialization failed (ContextLoader:350) 
org.springframework.beans.factory.BeanCreationException: Error creating bean 
with name 'org.springframework.context.event.internalEventListenerProcessor': 
BeanPostProcessor before instantia tion of bean failed; nested exception is 
org.springframework.beans.factory.UnsatisfiedDependencyException: Error 
creating bean with name 'graphTransactionAdvisor' defined in URL [jar:file:/ 
home/devuser/atlas/apache-atlas-2.1.0/server/webapp/atlas/WEB-INF/lib/atlas-repository-2.1.0.jar!/org/apache/atlas/GraphTransactionAdvisor.class]:
 Unsatisfied dependency expressed through c onstructor parameter 0; nested 
exception is org.springframework.beans.factory.UnsatisfiedDependencyException: 
Error creating bean with name 'graphTransactionInterceptor' defined in URL [jar 
:file:/home/devuser/atlas/apache-atlas-2.1.0/server/webapp/atlas/WEB-INF/lib/atlas-repository-2.1.0.jar!/org/apache/atlas/GraphTransactionInterceptor.class]:
 Unsatisfied dependency expresse d through constructor parameter 0; nested 
exception is org.springframework.beans.factory.BeanCreationException: Error 
creating bean with name 'get' defined in org.apache.atlas.repository.gr 
aph.AtlasGraphProvider: Bean instantiation via factory method failed; nested 
exception is org.springframework.beans.BeanInstantiationException: Failed to 
instantiate [org.apache.atlas.repos itory.graphdb.AtlasGraph]: Factory method 
'get' threw exception; nested exception is java.lang.NoClassDefFoundError: 
org/apache/cassandra/dht/RandomPartitioner$BigIntegerToken at 
org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:477)
 at 
org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:312)
 at 
org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:230)
 at 
org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:308)
 at 
org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:197)
 at 
org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:761)
 at 
org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:867)
 at 
org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:543)
 at 
org.springframework.web.context.ContextLoader.configureAndRefreshWebApplicationContext(ContextLoader.java:443)
 at 
org.springframework.web.context.ContextLoader.initWebApplicationContext(ContextLoader.java:325)
 at 
org.springframework.web.context.ContextLoaderListener.contextInitialized(ContextLoaderListener.java:107)
 at 
org.apache.atlas.web.setup.KerberosAwareListener.contextInitialized(KerberosAwareListener.java:31)
 at 
org.eclipse.jetty.server.handler.ContextHandler.callContextInitialized(ContextHandler.java:843)
 at 
org.eclipse.jetty.servlet.ServletContextHandler.callContextInitialized(ServletContextHandler.java:533)
 at 
org.eclipse.jetty.server.handler.ContextHandler.startContext(ContextHandler.java:816)
 at 
org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:345)
 at org.eclipse.jetty.webapp.WebAppContext.startWebapp(WebAppContext.java:1404) 
at org.eclipse.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1366) 
at 
org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:778)
 at 
org.eclipse.jetty.servlet.ServletContextHandler.doStart(ServletContextHandler.java:262)
 at org.eclipse.jetty.webapp.WebAppContext.doStart(WebAppContext.java:520) at 

Re: Review Request 72486: ATLAS-3780 : Change 'Status' from Classification System Attributes to EntityStatus

[Madhan Neethiraj]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72486/#review220971
---


Ship it!




Ship It!

- Madhan Neethiraj


On June 8, 2020, 3:14 p.m., Pinal Shah wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/72486/
> ---
> 
> (Updated June 8, 2020, 3:14 p.m.)
> 
> 
> Review request for atlas, Madhan Neethiraj, Nixon Rodrigues, and Sarath 
> Subramanian.
> 
> 
> Bugs: ATLAS-3780
> https://issues.apache.org/jira/browse/ATLAS-3780
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> We are having 'State' Attribute for Status i.e ACTIVE and DELETED.
> Whenever Entity is Deleted,'state' is always ACTIVE for Classification 
> vertices but 'entityState' gets reflected as 'DELETED'.
> 
> For this UI, needs to handle '_entityStatus' as enum and whenever DELETED is 
> selected, showHistoricalEntities needs to be true.
> 
> 
> Diffs
> -
> 
>   intg/src/main/java/org/apache/atlas/type/AtlasClassificationType.java 
> 4d0179c44 
>   intg/src/main/java/org/apache/atlas/type/Constants.java 3fc13056e 
> 
> 
> Diff: https://reviews.apache.org/r/72486/diff/3/
> 
> 
> Testing
> ---
> 
> Manually tested
> 
> 
> Thanks,
> 
> Pinal Shah
> 
>

Re: Review Request 72486: ATLAS-3780 : Change 'Status' from Classification System Attributes to EntityStatus

[Pinal Shah]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72486/
---

(Updated June 8, 2020, 3:14 p.m.)


Review request for atlas, Madhan Neethiraj, Nixon Rodrigues, and Sarath 
Subramanian.


Changes
---

Renamed status variable


Bugs: ATLAS-3780
https://issues.apache.org/jira/browse/ATLAS-3780


Repository: atlas


Description
---

We are having 'State' Attribute for Status i.e ACTIVE and DELETED.
Whenever Entity is Deleted,'state' is always ACTIVE for Classification vertices 
but 'entityState' gets reflected as 'DELETED'.

For this UI, needs to handle '_entityStatus' as enum and whenever DELETED is 
selected, showHistoricalEntities needs to be true.


Diffs (updated)
-

  intg/src/main/java/org/apache/atlas/type/AtlasClassificationType.java 
4d0179c44 
  intg/src/main/java/org/apache/atlas/type/Constants.java 3fc13056e 


Diff: https://reviews.apache.org/r/72486/diff/3/

Changes: https://reviews.apache.org/r/72486/diff/2-3/


Testing
---

Manually tested


Thanks,

Pinal Shah

Re: Review Request 72486: ATLAS-3780 : Change 'Status' from Classification System Attributes to EntityStatus

[Madhan Neethiraj]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72486/#review220969
---


Fix it, then Ship it!





intg/src/main/java/org/apache/atlas/type/Constants.java
Lines 52 (patched)


Consider following renaming to avoid potential confusion:
  ENTITY_STATUS_PROPERTY_KEY => CLASSIFICATION_ENTITY_STATUS_PROPERTY_KEY


- Madhan Neethiraj


On June 8, 2020, 10:48 a.m., Pinal Shah wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/72486/
> ---
> 
> (Updated June 8, 2020, 10:48 a.m.)
> 
> 
> Review request for atlas, Madhan Neethiraj, Nixon Rodrigues, and Sarath 
> Subramanian.
> 
> 
> Bugs: ATLAS-3780
> https://issues.apache.org/jira/browse/ATLAS-3780
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> We are having 'State' Attribute for Status i.e ACTIVE and DELETED.
> Whenever Entity is Deleted,'state' is always ACTIVE for Classification 
> vertices but 'entityState' gets reflected as 'DELETED'.
> 
> For this UI, needs to handle '_entityStatus' as enum and whenever DELETED is 
> selected, showHistoricalEntities needs to be true.
> 
> 
> Diffs
> -
> 
>   intg/src/main/java/org/apache/atlas/type/AtlasClassificationType.java 
> 4d0179c44 
>   intg/src/main/java/org/apache/atlas/type/Constants.java 3fc13056e 
> 
> 
> Diff: https://reviews.apache.org/r/72486/diff/2/
> 
> 
> Testing
> ---
> 
> Manually tested
> 
> 
> Thanks,
> 
> Pinal Shah
> 
>

Re: Review Request 72566: ATLAS-1798 : Fix Findbugs problems in repository module

[Madhan Neethiraj]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72566/#review220966
---




repository/src/main/java/org/apache/atlas/discovery/EntityDiscoveryService.java
Line 147 (original), 147 (patched)


Consider replacing #147, #148 with:
  for (Object entryObj : map.entrySet()) {
Map.Entry entry = (Map.Entry) entryObj;
Objectkey   = entry.getKey();
Objectvalue = entry.getValue();

Also, apply the same for other such changes as well.



repository/src/main/java/org/apache/atlas/discovery/EntitySearchProcessor.java
Line 265 (original)


Why is this code block removed? If necessary, please rebase.



repository/src/main/java/org/apache/atlas/discovery/SearchAggregatorImpl.java
Line 96 (original), 96 (patched)


aggregationMetricName => entry



repository/src/main/java/org/apache/atlas/discovery/SearchAggregatorImpl.java
Line 98 (original), 98 (patched)


Consider replacing "aggregationMetricName.getKey()" with:
  String aggregationMetricName = entry.getKey();

and use aggregationMetricName in #98, #99, 105.



repository/src/main/java/org/apache/atlas/glossary/GlossaryService.java
Line 905 (original), 905 (patched)


This update doesn't look right. Why is 'ret' initialized in #905, and have 
it reassinged in #915. What this the issue flagged for the existing code?



repository/src/main/java/org/apache/atlas/query/antlr4/AtlasDSLLexer.java
Line 86 (original), 88 (patched)


Use of Arrays.copyOf() could cause excessive memory allocation. Why is this 
necessary - in AtlasDSLLexer.java, AtlasDSLParser.java?



repository/src/main/java/org/apache/atlas/repository/audit/EntityAuditListener.java
Line 211 (original), 211 (patched)


Remove #211, instead of commenting.



repository/src/main/java/org/apache/atlas/repository/audit/HBaseBasedAuditRepository.java
Line 346 (original), 346 (patched)


Instead of commenting unused code, please remove.



repository/src/main/java/org/apache/atlas/repository/impexp/ExportService.java
Line 357 (original), 357 (patched)


If isSkipConnectedFetch is unused, please remove all references to this, 
instead of commenting it out.


- Madhan Neethiraj


On June 4, 2020, 2:33 a.m., mayank jain wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/72566/
> ---
> 
> (Updated June 4, 2020, 2:33 a.m.)
> 
> 
> Review request for atlas, Jayendra Parab, Madhan Neethiraj, Nixon Rodrigues, 
> and Sarath Subramanian.
> 
> 
> Bugs: ATLAS-1798
> https://issues.apache.org/jira/browse/ATLAS-1798
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> Currently Findbugs complaints about some problems (see attachment) in the 
> repository module. They should be fixed to get the code more reliable.
> 
> 
> Diffs
> -
> 
>   repository/src/main/java/org/apache/atlas/GraphTransactionInterceptor.java 
> 57e454a 
>   
> repository/src/main/java/org/apache/atlas/discovery/EntityDiscoveryService.java
>  dd4d1b4 
>   
> repository/src/main/java/org/apache/atlas/discovery/EntitySearchProcessor.java
>  56956e6 
>   
> repository/src/main/java/org/apache/atlas/discovery/SearchAggregatorImpl.java 
> e8f7dbc 
>   repository/src/main/java/org/apache/atlas/discovery/SearchProcessor.java 
> 804c694 
>   repository/src/main/java/org/apache/atlas/glossary/GlossaryService.java 
> d630f66 
>   repository/src/main/java/org/apache/atlas/glossary/GlossaryTermUtils.java 
> 2c84ec7 
>   repository/src/main/java/org/apache/atlas/glossary/GlossaryUtils.java 
> 2a2cebb 
>   repository/src/main/java/org/apache/atlas/query/AtlasDSL.java b8a744b 
>   repository/src/main/java/org/apache/atlas/query/GremlinQueryComposer.java 
> 801e898 
>   repository/src/main/java/org/apache/atlas/query/antlr4/AtlasDSLLexer.java 
> 142b9ca 
>   repository/src/main/java/org/apache/atlas/query/antlr4/AtlasDSLParser.java 
> 6b33edb 
>   
> repository/src/main/java/org/apache/atlas/repository/audit/AbstractStorageBasedAuditRepository.java
>  1aac375 
>   
> repository/src/main/java/org/apache/atlas/repository/audit/EntityAuditListener.java
>  69d373d 
>   
> repository/src/main/java/org/apache/atlas/repository/audit/EntityAuditListenerV2.java
>  79527ac 
>  

Fw:atlas 2 start error with hbase 1.2

[lk_hadoop]

hi,all : 
 Forwarding messages 
From: "lk_hadoop" 
Date: 2020-06-08 15:53:01
To: u...@atlas.apache.org
Subject: atlas 2 start error with hbase 1.2

hi,all:
I'm using atlas-release-2.1.0-rc0 ,with hbase 1.2.0 , when start atlas I 
got error :
Caused by: 
org.apache.hadoop.hbase.regionserver.NoSuchColumnFamilyException: 
org.apache.hadoop.hbase.regionserver.NoSuchColumnFamilyException: Column family 
table does not exist in region hbase:meta,,1.1588230740 in table 'hbase:meta', 
{TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => 
'|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}, 
{NAME => 'info', BLOOMFILTER => 'NONE', VERSIONS => '10', IN_MEMORY => 'true', 
KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', 
COMPRESSION => 'NONE', CACHE_DATA_IN_L1 => 'true', MIN_VERSIONS => '0', 
BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}


   It's that mean I can only use atlas 2 with hbase2.0+ ?




 

Fw:build with profile embedded-cassandra-solr not work

[lk_hadoop]

hi,all:




 Forwarding messages 
From: "lk_hadoop" 
Date: 2020-06-08 21:15:24
To: u...@atlas.apache.org
Subject: build with profile embedded-cassandra-solr not work

hi,all:
I'm using atlas-release-2.1.0-rc0  build with : mvn clean -DskipTests 
package -Pdist,embedded-cassandra-solr
when I try atlas_start.py , I got error:
Context initialization failed (ContextLoader:350) 
org.springframework.beans.factory.BeanCreationException: Error creating bean 
with name 'org.springframework.context.event.internalEventListenerProcessor': 
BeanPostProcessor before instantia tion of bean failed; nested exception is 
org.springframework.beans.factory.UnsatisfiedDependencyException: Error 
creating bean with name 'graphTransactionAdvisor' defined in URL [jar:file:/ 
home/devuser/atlas/apache-atlas-2.1.0/server/webapp/atlas/WEB-INF/lib/atlas-repository-2.1.0.jar!/org/apache/atlas/GraphTransactionAdvisor.class]:
 Unsatisfied dependency expressed through c onstructor parameter 0; nested 
exception is org.springframework.beans.factory.UnsatisfiedDependencyException: 
Error creating bean with name 'graphTransactionInterceptor' defined in URL [jar 
:file:/home/devuser/atlas/apache-atlas-2.1.0/server/webapp/atlas/WEB-INF/lib/atlas-repository-2.1.0.jar!/org/apache/atlas/GraphTransactionInterceptor.class]:
 Unsatisfied dependency expresse d through constructor parameter 0; nested 
exception is org.springframework.beans.factory.BeanCreationException: Error 
creating bean with name 'get' defined in org.apache.atlas.repository.gr 
aph.AtlasGraphProvider: Bean instantiation via factory method failed; nested 
exception is org.springframework.beans.BeanInstantiationException: Failed to 
instantiate [org.apache.atlas.repos itory.graphdb.AtlasGraph]: Factory method 
'get' threw exception; nested exception is java.lang.NoClassDefFoundError: 
org/apache/cassandra/dht/RandomPartitioner$BigIntegerToken at 
org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:477)
 at 
org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:312)
 at 
org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:230)
 at 
org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:308)
 at 
org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:197)
 at 
org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:761)
 at 
org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:867)
 at 
org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:543)
 at 
org.springframework.web.context.ContextLoader.configureAndRefreshWebApplicationContext(ContextLoader.java:443)
 at 
org.springframework.web.context.ContextLoader.initWebApplicationContext(ContextLoader.java:325)
 at 
org.springframework.web.context.ContextLoaderListener.contextInitialized(ContextLoaderListener.java:107)
 at 
org.apache.atlas.web.setup.KerberosAwareListener.contextInitialized(KerberosAwareListener.java:31)
 at 
org.eclipse.jetty.server.handler.ContextHandler.callContextInitialized(ContextHandler.java:843)
 at 
org.eclipse.jetty.servlet.ServletContextHandler.callContextInitialized(ServletContextHandler.java:533)
 at 
org.eclipse.jetty.server.handler.ContextHandler.startContext(ContextHandler.java:816)
 at 
org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:345)
 at org.eclipse.jetty.webapp.WebAppContext.startWebapp(WebAppContext.java:1404) 
at org.eclipse.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1366) 
at 
org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:778)
 at 
org.eclipse.jetty.servlet.ServletContextHandler.doStart(ServletContextHandler.java:262)
 at org.eclipse.jetty.webapp.WebAppContext.doStart(WebAppContext.java:520) at 
org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
 at 
org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:131)
 at org.eclipse.jetty.server.Server.start(Server.java:422) at 
org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:105)
 at 
org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:61)
 at org.eclipse.jetty.server.Server.doStart(Server.java:389) at 
org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
 at org.apache.atlas.web.service.EmbeddedServer.start(EmbeddedServer.java:98) 
at org.apache.atlas.Atlas.main(Atlas.java:133)
 




 

Re: [VOTE] Release Apache Atlas version 2.1.0 - rc1

[Nixon Rodrigues]

Thanks Madhan for putting 2.1 RC1 together for the vote.

+1 for Apache Atlas 2.1 RC1

Validated the following items.

   1. verified the md5/sha checksum of source bundle.
   2. packaging with -Pdist,embedded-hbase-solr is good.
   3. Unit testcase passes.
   4. Run quickstart and all the types and entities created are loading
   correctly on the UI.
   5. Create tags, glossary, term and its apply correctly on entities.
   6. Created business metadata with attributes and applied it to entities.
   7. tested basic and quick search flow.
   8. Verified statistics page details


Thanks & Regards
Nixon



On Mon, Jun 8, 2020 at 11:17 AM Sarath Subramanian 
wrote:

> +1 for Apache Atlas 2.1 RC1
>
> Validated the following:
>
>- No binaries in source
>- Verified source checksum - md5 and sha512
>- Verified signature
>- Build was successful on the source (mvn clean -DskipTests package
>-Pdist,embedded-hbase-solr)
>- Ran quickStart and validated types and entities were created
>successfully.
>- Validated tag propagation works
>- Added/removed labels to an entity and verified audit is getting
>updated correctly
>- Added user-defined properties and business metadata attributes.
>- Deleted few entities and purged those entities - validated if purge
>audit recorded correctly.
>- Basic sanity tests - basic search (with system attributes), advanced
>search, quick search.
>
>
>
> Thanks,
> Sarath
>
>
>
> On Sat, Jun 6, 2020 at 10:22 AM Madhan Neethiraj 
> wrote:
>
> > (resending with plain-text format)
> >
> > Atlas team,
> >
> > Apache Atlas 2.1.0 rc1, with following fixes since rc0, is now available
> > for vote within dev community.
> >
> > ATLAS-3770: UI(Classic): Active and Deleted hyperlinks for certain
> > entities throwing error on click
> > ATLAS-3766: Stats modal not close issue #2
> > ATLAS-3674: ZipFileMigationImporter: Migration status display fix.
> > Part 2
> >
> > Links to the release artifacts are given below. Please review and vote.
> >
> > The vote will be open for at least 72 hours or until necessary votes are
> > reached.
> >   [ ] +1 Approve
> >   [ ] +0 No opinion
> >   [ ] -1 Disapprove (and reason why)
> >
> > Thanks,
> > Madhan
> >
> >
> > List of all issues addressed in this release:
> > https://issues.apache.org/jira/issues/?jql=project=ATLAS AND
> > status=Resolved AND fixVersion=2.1.0 ORDER BY key DESC
> >
> > Git tag for the release:
> > https://github.com/apache/atlas/tree/release-2.1.0-rc1
> > Sources for the release:
> >
> https://dist.apache.org/repos/dist/dev/atlas/2.1.0-rc1/apache-atlas-2.1.0-sources.tar.gz
> >
> > Source release verification:
> >   PGP Signature:
> >
> https://dist.apache.org/repos/dist/dev/atlas/2.1.0-rc1/apache-atlas-2.1.0-sources.tar.gz.asc
> >   SHA512 Hash:
> >
> https://dist.apache.org/repos/dist/dev/atlas/2.1.0-rc1/apache-atlas-2.1.0-sources.tar.gz.sha512
> >   MD5 Hash:
> >
> https://dist.apache.org/repos/dist/dev/atlas/2.1.0-rc1/apache-atlas-2.1.0-sources.tar.gz.md5
> >
> > Keys to verify the signature of the release artifacts are available at:
> > https://dist.apache.org/repos/dist/release/atlas/KEYS
> >
> > New features/enhancements:
> >   - Quick Search: provides a simpler search experience with type-ahead
> > suggestions
> >   - Business Metadata: enables augmenting entity-types with additional
> > attributes, search entities using these attributes
> >   - Labels: ability to add/remove labels on entities, and search entities
> > using labels
> >   - Custom Attributes: ability to add entity instance specific custom
> > attributes i.e. attributes not defined in entity-def or business metadata
> >   - Entity Purge: added REST APIs to purge deleted entities
> >
> > Enhancements:
> >   - Search: ability to find entities by more than one classification
> >   - Performance: improvements in lineage retrieval and
> > classification-propagation
> >   - Notification: ability to process notifications from multiple Kafka
> > topics
> >   - Hive Hook: tracks process-executions via hive_process_execution
> > entities
> >   - Hive Hook: captures DDL operations via hive_db_ddl and hive_table_ddl
> > entities
> >   - Notification: introduced shell entities to record references to
> > non-existing entities in notifications
> >   - Spark: added model to capture Spark entities, processes and
> > relationships
> >   - AWS S3: introduced updated model to capture AWS S3 entities and
> > relationships
> >   - ADLS-Gen2: introduced model to capture Azure Data Lake Storage Gen2
> > entities and relationships
> >   - Dependencies: JanusGraph 0.5.1, Tinkerpop 3.4.6, Spring Framework
> > 4.3.20
> >   - Authorization: updated to cover new features, like: business
> metadata,
> > labels, purge
> >   - UI: multiple UI improvements, including a beta UI
> >
> >
> >
> >
> >
> >
> >
>

Re: Review Request 72566: ATLAS-1798 : Fix Findbugs problems in repository module

[Jayendra Parab]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72566/#review220965
---




repository/src/main/java/org/apache/atlas/discovery/SearchProcessor.java
Line 309 (original), 312 (patched)


Remove commented code, its there in multiple places, remove from there as 
well



repository/src/main/java/org/apache/atlas/glossary/GlossaryTermUtils.java
Line 537 (original), 537 (patched)


Remove this declaration and move it to line 572


- Jayendra Parab


On June 4, 2020, 2:33 a.m., mayank jain wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/72566/
> ---
> 
> (Updated June 4, 2020, 2:33 a.m.)
> 
> 
> Review request for atlas, Jayendra Parab, Madhan Neethiraj, Nixon Rodrigues, 
> and Sarath Subramanian.
> 
> 
> Bugs: ATLAS-1798
> https://issues.apache.org/jira/browse/ATLAS-1798
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> Currently Findbugs complaints about some problems (see attachment) in the 
> repository module. They should be fixed to get the code more reliable.
> 
> 
> Diffs
> -
> 
>   repository/src/main/java/org/apache/atlas/GraphTransactionInterceptor.java 
> 57e454a 
>   
> repository/src/main/java/org/apache/atlas/discovery/EntityDiscoveryService.java
>  dd4d1b4 
>   
> repository/src/main/java/org/apache/atlas/discovery/EntitySearchProcessor.java
>  56956e6 
>   
> repository/src/main/java/org/apache/atlas/discovery/SearchAggregatorImpl.java 
> e8f7dbc 
>   repository/src/main/java/org/apache/atlas/discovery/SearchProcessor.java 
> 804c694 
>   repository/src/main/java/org/apache/atlas/glossary/GlossaryService.java 
> d630f66 
>   repository/src/main/java/org/apache/atlas/glossary/GlossaryTermUtils.java 
> 2c84ec7 
>   repository/src/main/java/org/apache/atlas/glossary/GlossaryUtils.java 
> 2a2cebb 
>   repository/src/main/java/org/apache/atlas/query/AtlasDSL.java b8a744b 
>   repository/src/main/java/org/apache/atlas/query/GremlinQueryComposer.java 
> 801e898 
>   repository/src/main/java/org/apache/atlas/query/antlr4/AtlasDSLLexer.java 
> 142b9ca 
>   repository/src/main/java/org/apache/atlas/query/antlr4/AtlasDSLParser.java 
> 6b33edb 
>   
> repository/src/main/java/org/apache/atlas/repository/audit/AbstractStorageBasedAuditRepository.java
>  1aac375 
>   
> repository/src/main/java/org/apache/atlas/repository/audit/EntityAuditListener.java
>  69d373d 
>   
> repository/src/main/java/org/apache/atlas/repository/audit/EntityAuditListenerV2.java
>  79527ac 
>   
> repository/src/main/java/org/apache/atlas/repository/audit/HBaseBasedAuditRepository.java
>  9fca744 
>   
> repository/src/main/java/org/apache/atlas/repository/converters/AtlasArrayFormatConverter.java
>  c335f0a 
>   
> repository/src/main/java/org/apache/atlas/repository/converters/AtlasInstanceConverter.java
>  6fc0c65 
>   
> repository/src/main/java/org/apache/atlas/repository/converters/AtlasMapFormatConverter.java
>  0eacd8e 
>   
> repository/src/main/java/org/apache/atlas/repository/converters/AtlasStructFormatConverter.java
>  ae92b8b 
>   
> repository/src/main/java/org/apache/atlas/repository/graph/FullTextMapperV2.java
>  497a877 
>   
> repository/src/main/java/org/apache/atlas/repository/graph/GraphBackedSearchIndexer.java
>  4a09b08 
>   repository/src/main/java/org/apache/atlas/repository/graph/GraphHelper.java 
> 7b7ec65 
>   
> repository/src/main/java/org/apache/atlas/repository/impexp/AtlasServerService.java
>  542106f 
>   
> repository/src/main/java/org/apache/atlas/repository/impexp/ExportService.java
>  0491a85 
>   
> repository/src/main/java/org/apache/atlas/repository/impexp/ImportService.java
>  1d29bf8 
>   
> repository/src/main/java/org/apache/atlas/repository/impexp/ImportTransforms.java
>  a2f592c 
>   repository/src/main/java/org/apache/atlas/repository/impexp/ZipSink.java 
> 6375454 
>   repository/src/main/java/org/apache/atlas/repository/impexp/ZipSource.java 
> 812add9 
>   
> repository/src/main/java/org/apache/atlas/repository/impexp/ZipSourceDirect.java
>  04342fa 
>   
> repository/src/main/java/org/apache/atlas/repository/impexp/ZipSourceWithBackingDirectory.java
>  7963800 
>   
> repository/src/main/java/org/apache/atlas/repository/migration/DataMigrationService.java
>  0a2257e 
>   
> repository/src/main/java/org/apache/atlas/repository/migration/ZipFileMigrationImporter.java
>  d56261f 
>   
> repository/src/main/java/org/apache/atlas/repository/ogm/AtlasServerDTO.java 
> 2f7ca11 
>   
> repository/src/main/java/org/apache/atlas/repository/store/bootstrap/AtlasTypeDefStoreInitializer.java
>  8e7c1b3 
>   
> 

Re: Review Request 72486: ATLAS-3780 : Change 'Status' from Classification System Attributes to EntityStatus

[Pinal Shah]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72486/
---

(Updated June 8, 2020, 10:48 a.m.)


Review request for atlas, Madhan Neethiraj, Nixon Rodrigues, and Sarath 
Subramanian.


Changes
---

Moved ClassificationSearchProcessor changes, fixed and added test cases in 
https://reviews.apache.org/r/72567/


Bugs: ATLAS-3780
https://issues.apache.org/jira/browse/ATLAS-3780


Repository: atlas


Description
---

We are having 'State' Attribute for Status i.e ACTIVE and DELETED.
Whenever Entity is Deleted,'state' is always ACTIVE for Classification vertices 
but 'entityState' gets reflected as 'DELETED'.

For this UI, needs to handle '_entityStatus' as enum and whenever DELETED is 
selected, showHistoricalEntities needs to be true.


Diffs (updated)
-

  intg/src/main/java/org/apache/atlas/type/AtlasClassificationType.java 
4d0179c44 
  intg/src/main/java/org/apache/atlas/type/Constants.java 3fc13056e 


Diff: https://reviews.apache.org/r/72486/diff/2/

Changes: https://reviews.apache.org/r/72486/diff/1-2/


Testing
---

Manually tested


Thanks,

Pinal Shah

Re: Running Apache Atlas on prefix path instead of /

[Chitender Kumar]

@Keval Thank you.

@Dev-group: looking forward for positive response.

> On 07-Jun-2020, at 8:36 PM, Keval Bhatt  wrote:
> 
> Hi Chitender,
> 
> I am not sure about the solution I think anyone from server side could help 
> you.
> 
> I am adding atlas dev email in this thread.
> 
> 
> Thanks
> 
> 
> 
> On Fri, Jun 5, 2020, 17:54 Chitender Kumar  > wrote:
> Hello Keval,
> 
> Hope you are doing good.
> 
> Got your from GitHub.
> 
> We are working on Apache Atlas in production setup. Where we are connecting 
> it to Ranger and Keycloak for authentication and authorisation. The issue 
> which we are facing is we are not able to figure out how to run Apache Atlas 
> on prefix path like /atlas instead of /. 
> 
> If we configure /atlas in our K8s Nginx controller and when we hit DNS/atlas 
> its getting redirected to /.
> 
> Can you please help us out here or if you can redirect us to some reference 
> who answer these query.
> 
> Looking forward for your response.
> 
> Thanks
> Chitender