Re: [VOTE] Release Apache Atlas 0.7.1 (incubating) - release candidate 3 (dev group vote)

[Shwetha Shivalingamurthy]

+1

Regards,
Shwetha 

> On Jan 21, 2017, at 3:37 PM, Hemanth Yamijala  
> wrote:
> 
> Repeated these tests from last RC check:
> 
> * SHA / MD5 checksums fine
> * Tag exists and looks OK
> * No binaries in source
> * Built fine (minus tests)
> * Single node instance with HBase and Solr came up
> * Quickstart ran fine and UI looks good.
> 
> +1 (binding).
> 
> Thanks
> Hemanth
> 
> From: Madhan Neethiraj 
> Sent: Thursday, January 19, 2017 10:15 PM
> To: dev@atlas.incubator.apache.org
> Subject: [VOTE] Release Apache Atlas 0.7.1 (incubating) - release candidate 3 
> (dev group vote)
> 
> Atlas team,
> 
> 
> 
> Apache Atlas 0.7.1 (incubating) release candidate #3 is now available for a 
> vote within dev community. Links to the release artifacts are given below. 
> Can you please review and vote?
> 
> 
> 
> I apologize for yet another release-candidate. Only change in “release 
> candidate 3” is the update to build instructions in README.txt. There are no 
> other changes.
> 
> 
> 
> We currently have 7 binding votes and 7 non-binding votes for the earlier 
> release candidates. Thank you everyone for validating the release candidates, 
> your feedback and vote.
> 
>  +1 (binding):  7 votes
> 
> - Shwetha Shivalingamurthy
> 
> - Venkat Ranganathan
> 
> - Keval Bhatt
> 
> - Vimal Sharma
> 
> - Suma Shivaprasad
> 
> - Hemanth Yamijala
> 
> - Madhan Neethiraj
> 
> 
> 
>  +1 (non-binding): 7 votes
> 
> - David Radley
> 
> - Sarath Kumar Subramanian
> 
> - Ismaël Mejía
> 
> - Jean-Baptiste Onofré
> 
> - Ayub Khan Pathan
> 
> - Nixon Rodrigues
> 
> - Apoorv Naik
> 
> 
> 
> 
> 
> Changes since last release-candidate:
> 
>  - updated the build instructions in README.txt (ATLAS-1000)
> 
> 
> 
> The vote will be open for at least 72 hours or until necessary votes are 
> reached.
> 
> [ ] +1  approve
> 
> [ ] +0  no opinion
> 
> [ ] -1  disapprove (and reason why)
> 
> 
> 
> Thanks,
> 
> Madhan
> 
> 
> 
> 
> 
> List of issues addressed in this release: 
> https://issues.apache.org/jira/issues/?jql=project%20%3D%20Atlas%20AND%20fixVersion%20%3D%200.7.1-incubating%20ORDER%20BY%20key%20DESC
> 
> 
> 
> Git tag for the release: 
> https://github.com/apache/incubator-atlas/tree/release-0.7.1-rc3
> 
> 
> 
> Sources for the release: 
> https://dist.apache.org/repos/dist/dev/incubator/atlas/0.7.1-incubating-rc3/apache-atlas-0.7.1-incubating-sources.tar.gz
> 
> 
> 
> Source release verification:
> 
>  PGP Signature: 
> https://dist.apache.org/repos/dist/dev/incubator/atlas/0.7.1-incubating-rc3/apache-atlas-0.7.1-incubating-sources.tar.gz.asc
> 
>  MD5 Hash: 
> https://dist.apache.org/repos/dist/dev/incubator/atlas/0.7.1-incubating-rc3/apache-atlas-0.7.1-incubating-sources.tar.gz.mds
> 
>  Keys to verify the signature of the release artifacts are available at: 
> https://dist.apache.org/repos/dist/dev/incubator/atlas/KEYS
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 

Re: [VOTE] Release Apache Atlas 0.7.1 (incubating) - release candidate 2 (dev group vote)

[Shwetha Shivalingamurthy]

+1

Regards,
Shwetha






On 19/01/17, 9:45 AM, "Venkat Ranganathan" 
wrote:

>+1 (binding)
>
>
>Venkat
>
>On 1/18/17, 12:21 AM, "Madhan Neethiraj"  wrote:
>
>Atlas team,
>
>Apache Atlas 0.7.1 (incubating) release candidate #2 is now available
>for a vote within dev community. This update addresses the issues
>reported by Hemanth and Shwetha (thanks!).
>
>Changes since last release-candidate:
>- updated README.txt with build instructions (ATLAS-1000)
>
>Links to the release artifacts are given below. Can you please review
>and vote?
>
>The vote will be open for at least 72 hours or until necessary votes
>are reached.
>[ ] +1  approve
>[ ] +0  no opinion
>[ ] -1  disapprove (and reason why)
>
>Here is my +1
>
>Thanks,
>Madhan
>
>
>List of issues addressed in this release:
>https://issues.apache.org/jira/issues/?jql=project%20%3D%20Atlas%20AND%20f
>ixVersion%20%3D%200.7.1-incubating%20ORDER%20BY%20key%20DESC
>
>Git tag for the release:
>https://github.com/apache/incubator-atlas/tree/release-0.7.1-rc2
>
>Sources for the release:
>https://dist.apache.org/repos/dist/dev/incubator/atlas/0.7.1-incubating-rc
>2/apache-atlas-0.7.1-incubating-sources.tar.gz
>
>Source release verification:
>  PGP Signature:
>https://dist.apache.org/repos/dist/dev/incubator/atlas/0.7.1-incubating-rc
>2/apache-atlas-0.7.1-incubating-sources.tar.gz.asc
>  MD5 Hash:
>https://dist.apache.org/repos/dist/dev/incubator/atlas/0.7.1-incubating-rc
>2/apache-atlas-0.7.1-incubating-sources.tar.gz.mds
> 
>  Keys to verify the signature of the release artifacts are
>available at: https://dist.apache.org/repos/dist/dev/incubator/atlas/KEYS
>
>
>
>
>
>
>

Re: [VOTE] Release Apache Atlas 0.7.1 (incubating) - release candidate 1 (dev group vote)

[Shwetha Shivalingamurthy]

Git tag release-0.7.1-rc1 -
https://github.com/apache/incubator-atlas/tree/release-0.7.1-rc1 doesn¹t
exist


Otherwise,
Checksum and signature looks good
Build succeeds except for the test failures
Verified basic functionality with quick start with embedded hbase and solr


Regards,
Shwetha



On 18/01/17, 11:51 AM, "Hemanth Yamijala" 
wrote:

>Madhan,
>
>I verified the following:
>
>* SHA/MD5 checksums
>* incubating in release name
>* no binaries in release
>* 3rd party licenses for the JS / CSS files I could identify
>* build from source - a few minor points which I have pointed below.
>Build compile and package passed fine.
>* single node Atlas instance with HBase and Solr - came up fine.
>* Ran quickstart, browsed the UI - looks fine.
>
>Couple of minor issues:
>
>* The following tests failed:
>EntityJerseyResourceIT.testDeleteExistentTraitNonExistentForEntity,
>HiveHookIT.testCreateExternalTable, HiveHookIT.testInsertIntoTempTable. I
>know the community is working on some timing oriented test failures, and
>this seems a continuous effort. Given the basic functionality is fine, I
>think these are red herrings and will ignore them. Rest of the build was
>fine (including tests)
>
>* For the 0.7.0 release, there was feedback from IPMC to make the build
>instructions more explicit in the README itself, as opposed to having
>them only the website. We had filed ATLAS-1000 for this. Although this
>JIRA is marked for 0.8 release, since we are doing this anyway - maybe
>better to address it now. (Apologies for not pointing it out earlier -
>slipped my mind).
>
>Please let me know if you think ATLAS-1000 is worth covering now.
>
>Thanks
>Hemanth
>
>From: Madhan Neethiraj 
>Sent: Tuesday, January 17, 2017 11:57 PM
>To: dev@atlas.incubator.apache.org
>Subject: [VOTE] Release Apache Atlas 0.7.1 (incubating) - release
>candidate 1 (dev group vote)
>
>Atlas team,
>
>
>
>Apache Atlas 0.7.1 (incubating) release candidate #1 is now available for
>a vote within dev community.
>
>
>
>Changes since last release-candidate:
>
>-  files/directories that were inadvertently included in the last
>release candidate were removed. Thanks Hemanth for finding this issue
>
>
>
>Links to the release artifacts are given below. Can you please review and
>vote?
>
>
>
>The vote will be open for at least 72 hours or until necessary votes are
>reached.
>
> [ ] +1  approve
>
> [ ] +0  no opinion
>
> [ ] -1  disapprove (and reason why)
>
>
>
>Here is my +1
>
>
>
>Thanks,
>
>Madhan
>
>
>
>
>
>List of issues addressed in this release:
>https://issues.apache.org/jira/issues/?jql=project%20%3D%20Atlas%20AND%20f
>ixVersion%20%3D%200.7.1-incubating%20ORDER%20BY%20key%20DESC
>
>
>
>Git tag for the release:
>https://github.com/apache/incubator-atlas/tree/release-0.7.1-rc1
>
>
>
>Sources for the release:
>https://dist.apache.org/repos/dist/dev/incubator/atlas/0.7.1-incubating-rc
>1/apache-atlas-0.7.1-incubating-sources.tar.gz
>
>
>
>Source release verification:
>
>  PGP Signature:
>https://dist.apache.org/repos/dist/dev/incubator/atlas/0.7.1-incubating-rc
>1/apache-atlas-0.7.1-incubating-sources.tar.gz.asc
>
>  MD5 Hash: 
>https://dist.apache.org/repos/dist/dev/incubator/atlas/0.7.1-incubating-rc
>1/apache-atlas-0.7.1-incubating-sources.tar.gz.mds
>
>
>
>  Keys to verify the signature of the release artifacts are available
>at: https://dist.apache.org/repos/dist/dev/incubator/atlas/KEYS
>
>
>
>

Re: Unable to create Atlas JIRA: missing entries for components fields

[Shwetha Shivalingamurthy]

Done

Regards,
Shwetha


From: Madhan Neethiraj 
mailto:mneethi...@hortonworks.com>>
Reply-To: 
"priv...@atlas.incubator.apache.org" 
mailto:priv...@atlas.incubator.apache.org>>
Date: Thursday, 29 December 2016 at 3:05 PM
To: "dev@atlas.incubator.apache.org" 
mailto:dev@atlas.incubator.apache.org>>, 
"priv...@atlas.incubator.apache.org" 
mailto:priv...@atlas.incubator.apache.org>>
Subject: Unable to create Atlas JIRA: missing entries for components fields

Attempts to create an Atlas JIRA fails with the following error:
"Component/s" field is required and the project "Atlas" does not have any 
components.

Can one of the project admins please create few components asap? Here is a list 
of components to consider?

-  atlas-core

-  atlas-webui

-  atlas-intg

Thanks,
Madhan

Re: Improvement suggestion: change terms to be implemented as entities

[Shwetha Shivalingamurthy]

Modeling terms as traits also enabled search work out of the box. For
example, queries like search for assets with term will map to ŒAsset isa
¹ (though this worked only for leaf terms)

Modeling terms as entities will simplify some of the functionalities like
term renames, move term from one hierarchy to the other etc. Are you
planning to expose different way of searching or use existing search like
ŒAsset where terms = ¹?

Regards,
Shwetha






On 13/12/16, 7:50 AM, "Hemanth Yamijala"  wrote:

>David,
>
>I hope folks who are more plugged into Atlas on a day-to-day basis will
>provide relevant feedback. I have a very few comments below.
>
>Regarding point 10: AFAIK, the most significant constraint of
>implementing terms as entities was that entity to entity relationships
>needed to be predefined, while tags / traits could be associated to any
>entity without this prior definition.
>
>Regarding point 7: Tags and traits are indeed interchangeable. In the
>Atlas UI specifically, we always refer to trait types as tags (which is
>confusing IMO, but well, that's where we are)
>
>Thanks
>hemanth
>
>From: David Radley 
>Sent: Monday, December 12, 2016 11:27 PM
>To: dev@atlas.incubator.apache.org
>Subject: Improvement suggestion: change terms to be implemented as
>entities
>
>Hi,
>I have raised Atlas Jiras 1254 an 1245. I would like your feedback on
>changing the implementation of business/glossary terms to be entities,
>rather than trait types and trait instances. This would mean:
>
>1) A Term would have a guid for ATLAS-1245
>2) TermResourceDefinition could be changed to add relationship
>projections, to support ATLAS-1254. I suggest we have "has a" , homonymns
>and antonyms as the relationships.
>- has-a relationships would allow us to associate a Hive table
>with one term and its columns with other column related terms. So we could
>then work with the  the business glossary terms and it would be aware of
>the conceptual has-a relationship; rather than needing to interrogate the
>asset. Of course glossary terms could be associated using has-a
>relationships without being mapped to entities.
>- homonyms and antonyms are commonly used with business glossaries
>
>3) We would not have a new trait type that would be created for every term
>- that cannot be deleted. Instead we would have 1 system type for term
>that all terms entities would be associated with.
>4) We would need to ensure we could still support for available_as_tag for
>terms - this means we expose the term by name as a tag
>5) I suggest we tolerate gets on the term using the the guid in the URI as
>well as the fully qualified name. Creation of new terms should create
>hrefs with the guid.
>6) Term to term relationships would be simple in the code as we would use
>an entity to entity relationship.
>7) I notice in the the Atlas technical user guide (page 60), talks of
>traits and tags terminology as being interchangable. In the code (apart
>from in the supplied trait types),  it seems that traits are only used to
>implement terms, I guess because terms are often known by their name. Tags
>are somewhat different as they are used to interact with Ranger for tag
>based policies.
>8) The Atlas technical user guide talks of 2 ways of categorizing entities
>, the business taxonomy and tags / traits. This change would be in line
>with the separation.
>9) Having a guid for terms would allow us to rename the term without
>changing its identifier. I assume we should allow multiple terms of the
>same name in different taxonomies.
>10) I think the reason that terms were implemented as trait instances as
>traits are identified by name so do not need guids and if a trait was an
>entity, a user could define a relationship to a term entity, which would
>be confusing. My suggestion is that if the user chooses to create a type
>with a relationship to a term, then we reject the creation of the type .
>At the moment they presumably could create a relationship to a taxonomy
>which we should also reject.
>11) As part of these changes, I suggest that entities also contain a
>response field of terms. So it is more obvious to a REST client what the
>associated terms are with an entity.
>
>Please let me know if I have missed/misunderstood/misrepresented anything.
>I appreciate your feedback, as I hope to address these Jiras soon,
>
>many thanks , David.
>Unless stated otherwise above:
>IBM United Kingdom Limited - Registered in England and Wales with number
>741598.
>Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
>

Re: request: JIRA contributor permission

[Shwetha Shivalingamurthy]

Done

Regards,
Shwetha






On 30/11/16, 3:51 AM, "Cassio Dos Santos"  wrote:

>Hello,
>
>Could a PMC member with admin access please add me as a contributor in
>JIRA so that items may be assigned to me? My username is cassiodossantos.
>
>Thanks!
>Cassio
>
>
>

Re: request: JIRA contributor permission

[Shwetha Shivalingamurthy]

Done

Regards,
Shwetha






On 30/11/16, 2:15 AM, "Stephanie Hazlewood"  wrote:

>Hello,
>
>Could a PMC member with admin access please add me as a contributor in
>JIRA so that items may be assigned to me? My username is stephanie.
>
>Thanks!
>Stephanie
>

Issue in IDE while running tests

[Shwetha Shivalingamurthy]

Hi All,

FYI if you see error Œunable to access method
com.google.common.base.Stopwatch¹ in Intellij IDEA:

I recently committed ATLAS-1246 which adds shading of guava dependencies
for hbase and titan. After this change, you might see the error 'unable to
access method com.google.common.base.Stopwatch¹ while running tests in
Intellij IDEA. The issue is because Intellij IDEA doesn¹t handle mvn
shaded plugin. To resolve the issue, you need to re-import the modules.
For the modules titan0, shaded/hbase-client-shaded and
shaded/hbase-server-shaded, right click on the module -> Maven -> Ignore
Projects. Then do a "Maven" -> "Reimport" on the top-level pom.xml.
https://youtrack.jetbrains.com/issue/IDEA-126596 has details of the issue.

Mvn compile from the command line works fine.

If you face any issues regarding this, let me know. Thanks

Regards,
Shwetha

Re: Connecting Entities

[Shwetha Shivalingamurthy]

Hi,

I don’t think the lineage will work correctly if you model the query to
extend both dataset and process. You can define a process type to contain
the query dag instead and use table->process->table.

Can you file a jira so that we don’t lose this use case.

Regards,
Shwetha






On 20/10/16, 2:59 PM, "Ismaël Mejía"  wrote:

>​Hello,
>
>Thanks for your answer Shwetha, yes I forgot to be precise and say that I
>was talking about the case of lineage representation.
>
>I understand that the model is full graph based, and that I can relate any
>entity to another, that’s the reason why I was a little bit surprised to
>discover that I can’t connect two entities directly and see this in the
>lineage. Are there any plans to support an option like this?
>
>With the current model, would it be possible to create for example an
>Entity that inherits from both DataSet and Process to achieve this or it
>won't pass the type validation ?
>
>Thanks,
>Ismaël​
>
>
>On Thu, Oct 20, 2016 at 7:48 AM, Jean-Baptiste Onofré 
>wrote:
>
>> As far as I know, you want to sketch the lineage there, so you have to
>>use
>> an intermediate dataset (it could be a "fake" one when you push your
>>model)
>> to represent the changes (from a lineage perspective) between the
>>processes.
>>
>> Regards
>> JB
>>
>>
>> On 10/19/2016 06:18 PM, Ismaël Mejía wrote:
>>
>>> ​Hello,
>>>
>>> I would like to know if there is a way to connect two Datasets directly
>>> without an intermediary Process? Or is there a way to connect two
>>> Processes?
>>>
>>> I am thinking about representing something like this:
>>>
>>> Table -> Query1 -> Query2 -> Table
>>>
>>> but I cannot easily see how this can be modelled with the existing
>>> DataSet-> Process-> DataSet abstraction.
>>>
>>> Any ideas?
>>>
>>> Thanks,
>>> Ismaël​
>>>
>>>
>> --
>> Jean-Baptiste Onofré
>> jbono...@apache.org
>> http://blog.nanthrax.net
>> Talend - http://www.talend.com
>>

Re: Congratulations to the new committers

[Shwetha Shivalingamurthy]

Congratulations to the new committers!

Regards,
Shwetha 

> On Oct 21, 2016, at 9:06 AM, Hemanth Yamijala  
> wrote:
> 
> Jeff, Madhan & Vimal,
> 
> 
> Congratulations on the committership and best wishes to all of you in the 
> role!
> 
> 
> Thanks
> 
> Hemanth

Re: Connecting Entities

[Shwetha Shivalingamurthy]

Hi,

The model of Dataset -> Process -> Dataset is used for lineage. If you are
not interested in lineage, but just need references, you can define your
own model with references, like table has reference to database, columns.
But if you are interested in lineage, the dataset-process model is
required to capture generic lineage across different types of datasets.
You can model multiple queries as single process and the process can have
reference to the list of queries or dag of queries, like storm_topology
which is a process and it contains reference to the topology dag.

Regards,
Shwetha






On 19/10/16, 9:48 PM, "Ismaël Mejía"  wrote:

>​Hello,
>
>I would like to know if there is a way to connect two Datasets directly
>without an intermediary Process? Or is there a way to connect two
>Processes?
>
>I am thinking about representing something like this:
>
>Table -> Query1 -> Query2 -> Table
>
>but I cannot easily see how this can be modelled with the existing
>DataSet-> Process-> DataSet abstraction.
>
>Any ideas?
>
>Thanks,
>Ismaël​

Re: Web UI problem with Atlas 0.7

[Shwetha Shivalingamurthy]

Sure, go ahead

Regards,
Shwetha






On 19/10/16, 4:36 PM, "Jean-Baptiste Onofré"  wrote:

>If you don't mind, I would be more than happy to drive it ;)
>
>Let me know !
>
>Thanks !
>Regards
>JB
>
>On 10/19/2016 12:07 PM, Shwetha Shivalingamurthy wrote:
>> Sounds good.
>>
>> Jean, do you mean you will drive the release?
>>
>> Regards,
>> Shwetha
>>
>>
>>
>>
>>
>>
>> On 17/10/16, 5:40 PM, "Jean-Baptiste Onofré"  wrote:
>>
>>> Hi guys,
>>>
>>> As 0.7.0 is not super useful with this fix, I would propose to release
>>> 0.7.1 at least to fix that.
>>>
>>> I have couple of other Jira in my bucket I would like to include in a
>>> 0.7.1 release as well.
>>>
>>> Thoughts ?
>>>
>>> Regards
>>> JB
>>>
>>> On 10/17/2016 10:25 AM, Keval Bhatt wrote:
>>>> Hi Ismaël
>>>>
>>>> Recently Atlas UI was not loading after fresh build due to
>>>> jquery-asBreadcrumbs plugin changes and this is fixed on master.
>>>>
>>>> ATLAS-1199  <https://issues.apache.org/jira/browse/ATLAS-1199>
>>>>
>>>> Please checkout the latest code from master
>>>> <https://github.com/apache/incubator-atlas>
>>>>
>>>>
>>>> Thanks,
>>>> Keval Bhatt
>>>>
>>>> On Mon, Oct 17, 2016 at 1:15 PM, Ismaël Mejía 
>>>>wrote:
>>>>
>>>>> Hello,
>>>>>
>>>>> I used Atlas 0.5 and some of the earlier version of Atlas 0.7 with no
>>>>> issues.
>>>>> However I am trying to use the released version of Atlas 0.7 and I am
>>>>> having
>>>>> some problems. I built the binary distribution with embedded
>>>>>hbase/solr
>>>>> following the instructions from the website:
>>>>>
>>>>> mvn clean package -Pdist,embedded-hbase-solr -DskipTests
>>>>>
>>>>> Then I start Atlas like this:
>>>>>
>>>>> export MANAGE_LOCAL_SOLR=true
>>>>> export MANAGE_LOCAL_HBASE=true
>>>>> bin/atlas_start.py
>>>>>
>>>>> If I go to the initial webpage http://localhost:21000/
>>>>>
>>>>> I see the login/password page and once I log in with the admin user I
>>>>> get a
>>>>> blank page.
>>>>>
>>>>> I considered that maybe I was missing some basic data so I ran the
>>>>> quickstart:
>>>>>
>>>>> bin/quick_start.py
>>>>>
>>>>> Then I log in again but still I can't see any data. Am I missing
>>>>> something
>>>>> ?
>>>>>
>>>>> The weird thing is that I don't have any exception for the web app,
>>>>>the
>>>>> only
>>>>> exception in the logs is on atlas_start:
>>>>>
>>>>> 2016-10-17 09:06:48,174 INFO  - [main:] ~ Guice modules loaded
>>>>> (GuiceServletConfig:120)
>>>>> 2016-10-17 09:06:48,177 INFO  - [main:] ~ Starting services
>>>>> (GuiceServletConfig:140)
>>>>> 2016-10-17 09:06:48,224 WARN  - [main-SendThread(localhost:9026):] ~
>>>>> Session 0x0 for server null, unexpected error, closing socket
>>>>> connection
>>>>> and attempting reconnect (ClientCnxn$SendThread:1102)
>>>>> java.net.ConnectException: Connection refused
>>>>> at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
>>>>> at
>>>>> 
>>>>>sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
>>>>> at
>>>>> org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(
>>>>> ClientCnxnSocketNIO.java:361)
>>>>> at org.apache.zookeeper.ClientCnxn$SendThread.run(
>>>>> ClientCnxn.java:1081)
>>>>> 2016-10-17 09:06:48,255 INFO  - [main:] ~ HA is disabled. Hence
>>>>> creating
>>>>> table on startup. (HBaseBasedAuditRepository:287)
>>>>> 2016-10-17 09:06:48,256 INFO  - [main:] ~ Checking if table
>>>>> apache_atlas_entity_audit exists (HBaseBasedAuditRepository:249)
>>>>> 2016-10-17 09:06:48,263 INFO  - [main:] ~ Creating table
>>>>> apache_atlas_entity_audit (HBaseBasedAuditRepository:251)
&g

Re: Web UI problem with Atlas 0.7

[Shwetha Shivalingamurthy]

Sounds good. 

Jean, do you mean you will drive the release?

Regards,
Shwetha






On 17/10/16, 5:40 PM, "Jean-Baptiste Onofré"  wrote:

>Hi guys,
>
>As 0.7.0 is not super useful with this fix, I would propose to release
>0.7.1 at least to fix that.
>
>I have couple of other Jira in my bucket I would like to include in a
>0.7.1 release as well.
>
>Thoughts ?
>
>Regards
>JB
>
>On 10/17/2016 10:25 AM, Keval Bhatt wrote:
>> Hi Ismaël
>>
>> Recently Atlas UI was not loading after fresh build due to
>> jquery-asBreadcrumbs plugin changes and this is fixed on master.
>>
>> ATLAS-1199  
>>
>> Please checkout the latest code from master
>> 
>>
>>
>> Thanks,
>> Keval Bhatt
>>
>> On Mon, Oct 17, 2016 at 1:15 PM, Ismaël Mejía  wrote:
>>
>>> Hello,
>>>
>>> I used Atlas 0.5 and some of the earlier version of Atlas 0.7 with no
>>> issues.
>>> However I am trying to use the released version of Atlas 0.7 and I am
>>> having
>>> some problems. I built the binary distribution with embedded hbase/solr
>>> following the instructions from the website:
>>>
>>> mvn clean package -Pdist,embedded-hbase-solr -DskipTests
>>>
>>> Then I start Atlas like this:
>>>
>>> export MANAGE_LOCAL_SOLR=true
>>> export MANAGE_LOCAL_HBASE=true
>>> bin/atlas_start.py
>>>
>>> If I go to the initial webpage http://localhost:21000/
>>>
>>> I see the login/password page and once I log in with the admin user I
>>>get a
>>> blank page.
>>>
>>> I considered that maybe I was missing some basic data so I ran the
>>> quickstart:
>>>
>>> bin/quick_start.py
>>>
>>> Then I log in again but still I can't see any data. Am I missing
>>>something
>>> ?
>>>
>>> The weird thing is that I don't have any exception for the web app, the
>>> only
>>> exception in the logs is on atlas_start:
>>>
>>> 2016-10-17 09:06:48,174 INFO  - [main:] ~ Guice modules loaded
>>> (GuiceServletConfig:120)
>>> 2016-10-17 09:06:48,177 INFO  - [main:] ~ Starting services
>>> (GuiceServletConfig:140)
>>> 2016-10-17 09:06:48,224 WARN  - [main-SendThread(localhost:9026):] ~
>>> Session 0x0 for server null, unexpected error, closing socket
>>>connection
>>> and attempting reconnect (ClientCnxn$SendThread:1102)
>>> java.net.ConnectException: Connection refused
>>> at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
>>> at
>>> sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
>>> at
>>> org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(
>>> ClientCnxnSocketNIO.java:361)
>>> at org.apache.zookeeper.ClientCnxn$SendThread.run(
>>> ClientCnxn.java:1081)
>>> 2016-10-17 09:06:48,255 INFO  - [main:] ~ HA is disabled. Hence
>>>creating
>>> table on startup. (HBaseBasedAuditRepository:287)
>>> 2016-10-17 09:06:48,256 INFO  - [main:] ~ Checking if table
>>> apache_atlas_entity_audit exists (HBaseBasedAuditRepository:249)
>>> 2016-10-17 09:06:48,263 INFO  - [main:] ~ Creating table
>>> apache_atlas_entity_audit (HBaseBasedAuditRepository:251)
>>> 2016-10-17 09:06:49,326 WARN  - [main-SendThread(localhost:9026):] ~
>>> Session 0x0 for server null, unexpected error, closing socket
>>>connection
>>> and attempting reconnect (ClientCnxn$SendThread:1102)
>>> java.net.ConnectException: Connection refused
>>> at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
>>> at
>>> sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
>>> at
>>> org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(
>>> ClientCnxnSocketNIO.java:361)
>>> at org.apache.zookeeper.ClientCnxn$SendThread.run(
>>> ClientCnxn.java:1081)
>>>
>>> and also I found this exception when the quick_start script creates the
>>> first entity:
>>>
>>> 2016-10-17 09:21:40,566 WARN  - [qtp161960012-16 -
>>> 888beb21-202d-4e96-9750-25d5ebe3bcac:] ~ The configuration
>>> auto.commit.enable = false was supplied but isn't a known config.
>>> (AbstractConfig:186)
>>> 2016-10-17 09:21:40,700 WARN  - [kafka-producer-network-thread |
>>> producer-1:] ~ Error while fetching metadata with correlation id 0 :
>>> {ATLAS_ENTITIES=LEADER_NOT_AVAILABLE}
>>> (NetworkClient$DefaultMetadataUpdater:600)
>>> 2016-10-17 09:21:40,732 WARN  -
>>> [org.apache.atlas.kafka.KafkaNotification:Controller-
>>> 1-to-broker-1-send-thread:]
>>> ~
>>> [org.apache.atlas.kafka.KafkaNotification:Controller-
>>> 1-to-broker-1-send-thread],
>>> Controller 1 epoch 1 fails to send request
>>> {controller_id=1,controller_epoch=1,partition_states=[{
>>> topic=ATLAS_ENTITIES,partition=0,controller_epoch=
>>> 1,leader=1,leader_epoch=0,isr=[1],zk_version=0,replicas=[1]}
>>> ],live_leaders=[{id=1,host=localhost,port=9027}]}
>>> to broker localhost:9027 (id: 1 rack: null). Reconnecting to broker.
>>> (Logging$class:89)
>>> java.io.IOException: Connection to 1 was disconnected before the
>>>response
>>> was read
>>> at
>>> kafka.utils.NetworkClientBlockingOps$$anonfun$blockingSendAndReceive$
>>> extension$1

Re: Web UI problem with Atlas 0.7

[Shwetha Shivalingamurthy]

Distro built from master works for me. Embedded kafka data is in
/data/kafka. Try clearing /data and restart. Else,
try with external kafka

Regards,
Shwetha






On 19/10/16, 1:34 PM, "Ismaël Mejía"  wrote:

>Hello again,
>
>I enabled the ALL level in the log4j configuration and I can't see any
>additional exception (apart of the one I already reported),
>
>This exception seems to be related to Kafka not starting at the good
>moment, however I don't think this is related or critical for the
>dashboard
>initialization, is it ?
>
>Can anybody please help me confirm that the dashboard of the distro works
>out of the box on master (I assume that it is broke on 0.7 because of the
>breadcrumps issue).
>
>Thanks,
>Ismaël
>
>
>On Tue, Oct 18, 2016 at 6:18 AM, Shwetha Shivalingamurthy <
>sshivalingamur...@hortonworks.com> wrote:
>
>> Can you enable debug logs in atlas-log4j.xml and check if there is any
>> issue with service startup?
>>
>> Regards,
>> Shwetha
>>
>>
>>
>>
>>
>>
>> On 17/10/16, 9:52 PM, "Ismaël Mejía"  wrote:
>>
>> >Hello again,
>> >
>> >I tried with the git master version and I still have the same issue of
>>not
>> >seeing anything after login in the WebUI.
>> >
>> >What other data/table does the GUI depend on ? (apart of the given
>> >Types/Entities), because I think there is something else that maybe I
>>am
>> >missing.
>> >
>> >Regards,
>> >Ismaël
>> >
>> >On Mon, Oct 17, 2016 at 2:10 PM, Jean-Baptiste Onofré 
>> >wrote:
>> >
>> >> Hi guys,
>> >>
>> >> As 0.7.0 is not super useful with this fix, I would propose to
>>release
>> >> 0.7.1 at least to fix that.
>> >>
>> >> I have couple of other Jira in my bucket I would like to include in a
>> >> 0.7.1 release as well.
>> >>
>> >> Thoughts ?
>> >>
>> >> Regards
>> >> JB
>> >>
>> >>
>> >> On 10/17/2016 10:25 AM, Keval Bhatt wrote:
>> >>
>> >>> Hi Ismaël
>> >>>
>> >>> Recently Atlas UI was not loading after fresh build due to
>> >>> jquery-asBreadcrumbs plugin changes and this is fixed on master.
>> >>>
>> >>> ATLAS-1199  <https://issues.apache.org/jira/browse/ATLAS-1199>
>> >>>
>> >>> Please checkout the latest code from master
>> >>> <https://github.com/apache/incubator-atlas>
>> >>>
>> >>>
>> >>> Thanks,
>> >>> Keval Bhatt
>> >>>
>> >>> On Mon, Oct 17, 2016 at 1:15 PM, Ismaël Mejía 
>> >>>wrote:
>> >>>
>> >>> Hello,
>> >>>>
>> >>>> I used Atlas 0.5 and some of the earlier version of Atlas 0.7 with
>>no
>> >>>> issues.
>> >>>> However I am trying to use the released version of Atlas 0.7 and I
>>am
>> >>>> having
>> >>>> some problems. I built the binary distribution with embedded
>> >>>>hbase/solr
>> >>>> following the instructions from the website:
>> >>>>
>> >>>> mvn clean package -Pdist,embedded-hbase-solr -DskipTests
>> >>>>
>> >>>> Then I start Atlas like this:
>> >>>>
>> >>>> export MANAGE_LOCAL_SOLR=true
>> >>>> export MANAGE_LOCAL_HBASE=true
>> >>>> bin/atlas_start.py
>> >>>>
>> >>>> If I go to the initial webpage http://localhost:21000/
>> >>>>
>> >>>> I see the login/password page and once I log in with the admin
>>user I
>> >>>> get a
>> >>>> blank page.
>> >>>>
>> >>>> I considered that maybe I was missing some basic data so I ran the
>> >>>> quickstart:
>> >>>>
>> >>>> bin/quick_start.py
>> >>>>
>> >>>> Then I log in again but still I can't see any data. Am I missing
>> >>>> something
>> >>>> ?
>> >>>>
>> >>>> The weird thing is that I don't have any exception for the web app,
>> >>>>the
>> >>>> only
>> >>>> exception in the logs is on atlas_start:
>> &g

Re: Web UI problem with Atlas 0.7

[Shwetha Shivalingamurthy]

Can you enable debug logs in atlas-log4j.xml and check if there is any
issue with service startup?

Regards,
Shwetha






On 17/10/16, 9:52 PM, "Ismaël Mejía"  wrote:

>Hello again,
>
>I tried with the git master version and I still have the same issue of not
>seeing anything after login in the WebUI.
>
>What other data/table does the GUI depend on ? (apart of the given
>Types/Entities), because I think there is something else that maybe I am
>missing.
>
>Regards,
>Ismaël
>
>On Mon, Oct 17, 2016 at 2:10 PM, Jean-Baptiste Onofré 
>wrote:
>
>> Hi guys,
>>
>> As 0.7.0 is not super useful with this fix, I would propose to release
>> 0.7.1 at least to fix that.
>>
>> I have couple of other Jira in my bucket I would like to include in a
>> 0.7.1 release as well.
>>
>> Thoughts ?
>>
>> Regards
>> JB
>>
>>
>> On 10/17/2016 10:25 AM, Keval Bhatt wrote:
>>
>>> Hi Ismaël
>>>
>>> Recently Atlas UI was not loading after fresh build due to
>>> jquery-asBreadcrumbs plugin changes and this is fixed on master.
>>>
>>> ATLAS-1199  
>>>
>>> Please checkout the latest code from master
>>> 
>>>
>>>
>>> Thanks,
>>> Keval Bhatt
>>>
>>> On Mon, Oct 17, 2016 at 1:15 PM, Ismaël Mejía 
>>>wrote:
>>>
>>> Hello,

 I used Atlas 0.5 and some of the earlier version of Atlas 0.7 with no
 issues.
 However I am trying to use the released version of Atlas 0.7 and I am
 having
 some problems. I built the binary distribution with embedded
hbase/solr
 following the instructions from the website:

 mvn clean package -Pdist,embedded-hbase-solr -DskipTests

 Then I start Atlas like this:

 export MANAGE_LOCAL_SOLR=true
 export MANAGE_LOCAL_HBASE=true
 bin/atlas_start.py

 If I go to the initial webpage http://localhost:21000/

 I see the login/password page and once I log in with the admin user I
 get a
 blank page.

 I considered that maybe I was missing some basic data so I ran the
 quickstart:

 bin/quick_start.py

 Then I log in again but still I can't see any data. Am I missing
 something
 ?

 The weird thing is that I don't have any exception for the web app,
the
 only
 exception in the logs is on atlas_start:

 2016-10-17 09:06:48,174 INFO  - [main:] ~ Guice modules loaded
 (GuiceServletConfig:120)
 2016-10-17 09:06:48,177 INFO  - [main:] ~ Starting services
 (GuiceServletConfig:140)
 2016-10-17 09:06:48,224 WARN  - [main-SendThread(localhost:9026):] ~
 Session 0x0 for server null, unexpected error, closing socket
connection
 and attempting reconnect (ClientCnxn$SendThread:1102)
 java.net.ConnectException: Connection refused
 at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
 at
 sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
 at
 org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(
 ClientCnxnSocketNIO.java:361)
 at org.apache.zookeeper.ClientCnxn$SendThread.run(
 ClientCnxn.java:1081)
 2016-10-17 09:06:48,255 INFO  - [main:] ~ HA is disabled. Hence
creating
 table on startup. (HBaseBasedAuditRepository:287)
 2016-10-17 09:06:48,256 INFO  - [main:] ~ Checking if table
 apache_atlas_entity_audit exists (HBaseBasedAuditRepository:249)
 2016-10-17 09:06:48,263 INFO  - [main:] ~ Creating table
 apache_atlas_entity_audit (HBaseBasedAuditRepository:251)
 2016-10-17 09:06:49,326 WARN  - [main-SendThread(localhost:9026):] ~
 Session 0x0 for server null, unexpected error, closing socket
connection
 and attempting reconnect (ClientCnxn$SendThread:1102)
 java.net.ConnectException: Connection refused
 at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
 at
 sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
 at
 org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(
 ClientCnxnSocketNIO.java:361)
 at org.apache.zookeeper.ClientCnxn$SendThread.run(
 ClientCnxn.java:1081)

 and also I found this exception when the quick_start script creates
the
 first entity:

 2016-10-17 09:21:40,566 WARN  - [qtp161960012-16 -
 888beb21-202d-4e96-9750-25d5ebe3bcac:] ~ The configuration
 auto.commit.enable = false was supplied but isn't a known config.
 (AbstractConfig:186)
 2016-10-17 09:21:40,700 WARN  - [kafka-producer-network-thread |
 producer-1:] ~ Error while fetching metadata with correlation id 0 :
 {ATLAS_ENTITIES=LEADER_NOT_AVAILABLE}
 (NetworkClient$DefaultMetadataUpdater:600)
 2016-10-17 09:21:40,732 WARN  -
 [org.apache.atlas.kafka.KafkaNotification:Controller-
 1-to-broker-1-send-thread:]
 ~
 [org.apache.atlas.kafka.KafkaNotification:Controller-
 1-to-broker-1-send-thread],
 Control

Re: Atlas UI not loading after fresh build due to jquery-asBreadcrumbs plugin upgrade.

[Shwetha Shivalingamurthy]

We should release 0.7.1 with this fix

Regards,
Shwetha


From: Keval Bhatt mailto:kbh...@hortonworks.com>>
Date: Monday, 17 October 2016 at 1:42 PM
To: "dev@atlas.incubator.apache.org" 
mailto:dev@atlas.incubator.apache.org>>
Cc: hwx-atlas-dev 
mailto:hwx-atlas-...@hortonworks.com>>
Subject: Re: Atlas UI not loading after fresh build due to jquery-asBreadcrumbs 
plugin upgrade.


​Hi Team,


ATLAS-1199​ this issue is 
fixed on master. do we need to fix this issue in incubator-0.7 because if user 
using incubator-0.7 and if they build then they will face same issue.


Thanks

Keval Bhatt.


From: Keval Bhatt
Sent: Thursday, September 29, 2016 11:00 AM
To: dev@atlas.incubator.apache.org
Cc: hwx-atlas-dev
Subject: Atlas UI not loading after fresh build due to jquery-asBreadcrumbs 
plugin upgrade.


Hi Atlas team,


Atlas UI is not loading due to plugin upgrade after fresh build from master.


I have given fix for this issue 
ATLAS-1199


let me know if you find any issue.


Thanks,

Keval Bhatt

Re: whats different "tags" between atlas and falcon

[Shwetha Shivalingamurthy]

Atlas was not not conceptualised when tags were added in Falcon. Now that
Atlas maintains the tags, it makes sense to store the falcon entity’s tags
in Atlas and use Atlas as the source of truth, and falcon can use Atlas
APIs to fetch the tags.

Atlas now has falcon integration, Atlas maintains minimal info of falcon
entities. But falcon tags are still not integrated in Atlas

Regards,
Shwetha






On 02/10/16, 1:03 PM, "zhao.hong...@zte.com.cn" 
wrote:

>hi,
> recently when i integration atlas and falcon ,
> i find both have "tags",  as we know atlas have its tags and  it very
>userful for us to classifying metadata ,
> however falcon also have its "tags" and "colo",
>when i integration atlas and falcon
>i find falcon's tags can not transmission to atlas
>can anyone explain  what's "tags" different and connection between atlas
>and falcon 
> 
>
>
>
>
>
>赵洪涛 
>
>
>
>TEL:+86-15062208803
>
>
>
>
>
>
>
>

Re: Thoughts on the first step to understand the Atlas data model internals

[Shwetha Shivalingamurthy]

David,

EntityResourceDefinition was added as part of business catalog, and this
might change. 

org.apache.atlas.web.resources.EntityResource and
org.apache.atlas.web.resources.TypesResource are the initial APIs added,
these are used by AtlasClient and Atlas UI and is also documented in
Hemanth¹s doc

Regards,
Shwetha






On 15/09/16, 8:27 PM, "David Radley"  wrote:

>Hi Hermanth,
>Thanks for your response. I agree we should evolve the existing
>documentation. I was aware of the guide you pointed me to. This appears
>to 
>be useful for a user of the REST API. This sort of document is sometimes
>known as an application programming guide. The motivation of this persona
>is to effectively use the Atlas REST APIs to manage their metadata.
>
>I am interested in documenting the internal data model and motivation for
>the separations of concerns in the source code. This is for different
>personas, namely to document design consensus around the data model by
>and 
>for the Atlas contributers and committers.
> all the best David.
>
>
>
>From:   Hemanth Yamijala 
>To: "dev@atlas.incubator.apache.org" 
>Date:   15/09/2016 14:28
>Subject:Re: Thoughts on the first step to understand the Atlas
>data model internals
>
>
>
>David,
>
>This might not directly address all the points you mention below, but
>regarding learning about Atlas from a core understanding perspective, you
>may want to go through the document here:
>http://atlas.incubator.apache.org/AtlasTechnicalUserGuide.pdf. This is
>something we created just after the 0.7 release timeframe and feel it
>represents core concepts to a reasonable extent.
>
>There are several improvements that it needs:
>Likely, it is not complete and missing pieces.
>Further, being in PDF form makes it hard to edit, so moving it to a more
>editable format and specifically adding it to source code itself should
>be 
>a goal.
>
>Etc...
>
>But if it forms a starting point, then it would be great to base further
>improvements on top of it.
>
>Thanks
>Hemanth
>
>From: David Radley 
>Sent: Thursday, September 15, 2016 6:26 PM
>To: atlas
>Subject: Thoughts on the first step to understand the Atlas data model
>internals
>
>Hi,
>I am looking to understand the important pieces in the Atlas architecture
>and the order that is useful to think about them. I wanted to check my
>understanding and questions around the top level data model concept as
>someone relatively new to the project. then update the docs. I am
>interested in feedback / thoughts / things I may have misunderstood. I
>think the next areas for a person looking to understand the internals to
>understand is the type system and providers and maybe then tracking how
>the code flows from the web to the graph.
>
>It seems that the data model of Atlas is the where to start; the
>fundamental interface is interface ResourceDefinition and the base object
>is BaseResourceDefinition. As the top level data object, I think this
>needs to be simple and intuitive and have a defined purpose.
>I would expect top level metadata objects should have the ability to have
>relationships and attributes, which it seems to have and a way to identify
>them (names and guid - which I do not see in this object)
>
>I do not think the validate*** request call methods should live in this
>interface. I would separate out request logic from the core data model
>object; as they are different concerns.
>
>The baseResourceDefinition has :
> protected static final TypeSystem typeSystem = TypeSystem.getInstance
>();
>
>protected final Set instanceProperties = new HashSet<>();
>protected final Set collectionProperties = new HashSet<>();
>protected Map propertyDefs = new
>HashMap<>();
>protected Map properties = new HashMap<>();
>
>protected final Map projections = new HashMap<>();
>protected final Map relations = new HashMap<>();
>
>protected final PropertyMapper propertyMapper;
>protected final Map
>propertyValueFormatters = new HashMap<>();
>
>
>There is an implied concept of  Property here in the naming of these
>fields. If this is important then I suggest we have a Property class /
>Interface defining what we mean by it.
>I see there are projections and relationships. do we need both of these
>concepts in the top level object as the only implementation of a
>Projection is a RelationshipProjection.
>I see AttributeDefintion in the list, this class does not subclass
>ResourceDefintion - I am left thinking that in some way these are both
>Definitions which I would expect to be a super class / interface.   Also
>given that this is in a systemtypes package it should be a system type.
>I see AttributeInfo and a not sure how this related to
>AttributeDefintions. It would be great to be able to add in javadoc to
>help here.
>EntityResourceDefinition implements ResourceDefinition but says typename
>is not meaningful for it. This is confusing. We should have an interface
>without a gettypena

Re: Current use cases for POST /entities

[Shwetha Shivalingamurthy]

We can remove the definition, but I think we should return the guids.

The location in the response header contains the guid. But we should find
a way of returning all the guids created - probably list of locations?

When you change the create API, make sure to change even the update and
delete APIs.

Regards,
Shwetha




On 14/09/16, 3:46 AM, "Suma Shivaprasad" 
wrote:

>+1
>
>On Mon, Sep 12, 2016 at 9:22 PM, Apoorv Naik 
>wrote:
>
>> Is anyone relying on the definition object within the response of a
>>create
>> entity call ? If not, let¹s remove it from the response as it can be
>> inferred from the request body itself and the only useful piece seems
>>to be
>> the GUIDs.
>>

Re: Request to be a contributor in JIRA

[Shwetha Shivalingamurthy]

Hi David,

I have added you as contributor. Jiras can be assigned to you now

Regards,
Shwetha






On 09/09/16, 4:50 PM, "David Radley"  wrote:

>Thanks Vimal,
>I will wait for a kindly PMC member with admin access to make me a
>contributer, many thanks, David
>
>
>
>
>From:   Vimal Sharma 
>To: David Radley/UK/IBM@IBMGB
>Cc: "dev@atlas.incubator.apache.org" 
>Date:   09/09/2016 11:20
>Subject:FW: Request to be a contributor in JIRA
>
>
>
>Hi David,
>You can assign JIRAs to yourself only after some PMC member adds you as
>contributor.
>Sorry for the confusion.
>
>-Vimal
>
>
>
>
>On 9/9/16, 3:46 PM, "Ayub Khan Pathan"  wrote:
>
>>Hey Vimal,
>>
>>I guess some PMC member who has admin access has to add the person as
>³contributor². It doesn¹t happen automatically..
>>
>>Thanks
>>Ayub Khan
>>
>>
>>On 9/9/16, 3:40 PM, "Vimal Sharma"  wrote:
>>
>>Hi David,
>>Please create an account at
>>https://issues.apache.org/jira/login.jsp.
>Once you have an account, you can assign items to yourself.
>> 
>>Thanks
>>Vimal
>> 
>> 
>> 
>> 
>>On 9/9/16, 3:28 PM, "David Radley"  wrote:
>> 
>>>Hello,
>>>Please could someone add me as a contributor in JIRA so that items
>can be 
>>>assigned to me. My username is davidrad.
>>>  many thanks , David Radley.
>>>
>>>
>>>Unless stated otherwise above:
>>>IBM United Kingdom Limited - Registered in England and Wales with
>number 
>>>741598. 
>>>Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire
>PO6 3AU
>> 
>>
>>
>
>
>
>Unless stated otherwise above:
>IBM United Kingdom Limited - Registered in England and Wales with number
>741598. 
>Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
>

[ANNOUNCE] New Committer: Darshan Kumar

[Shwetha Shivalingamurthy]

Hi All,

The Apache Atlas PPMC has asked Darshan Kumar to become a committer and we
are pleased to announce that he has accepted.

Darshan contributed to the initial version of Atlas UI. Congratulations
Darshan and welcome onboard!


Regards,
Apache Atlas PPMC Team

[ANNOUNCE] New Committer: Keval Bhatt

[Shwetha Shivalingamurthy]

Hi All,

The Apache Atlas PPMC has asked Keval Bhatt to become a committer and we
are pleased to announce that he has accepted.

Keval has been a consistent contributor to Atlas and has contributed to
the latest version of Atlas UI. Congratulations Keval and welcome onboard!


Regards,
Apache Atlas PPMC Team

[ANNOUNCE] New Committer: Tom Beerbower

[Shwetha Shivalingamurthy]

Hi All,

The Apache Atlas PPMC has asked Tom Beerbower to become a committer and we
are pleased to announce that he has accepted.

Tom has been a consistent contributor to Atlas and has contributed entity
notifications, hbase and solr integrations and other fixes.
Congratulations Tom and welcome onboard!


Regards,
Apache Atlas PPMC Team

[ANNOUNCE] New Committer: Dave Kantor

[Shwetha Shivalingamurthy]

Hi All,


The Apache Atlas PPMC has asked Dave Kantor to become a committer and we
are pleased to announce that he has accepted.


Dave Kantor has been a consistent contributor to Atlas and has worked on
core Typesystem and repository. Congratulations Dave and welcome onboard!



Regards,
Apache Atlas PPMC

Re: Filter out deleted entities

[Shwetha Shivalingamurthy]

Hi Herman,

1. If you don¹t need to track deleted entities, you can disable
soft-deletes with config -
atlas.DeleteHandler.impl=org.apache.atlas.repository.graph.HardDeleteHandle
r. This will not store any deleted entities in db and hence deleted
entities don¹t show up on API/UI
2. If you want soft delete, but want to filter deleted entities, you can
use state filter in DSL query - hive_table where name = Œxx¹ and __state =
ŒACTIVE¹
3. On trunk, orderby query should be - hive_table orderby name. On 0.7,
orderby field should be qualified, so, hive_table orderby ŒAsset.name¹. We
need to update the documentation at apache website.
https://github.com/apache/incubator-atlas/blob/master/docs/src/site/twiki/S
earch.twiki contains the latest docs
4. We have a draft of the docs, need to update


Regards,
Shwetha






On 28/07/16, 7:31 PM, "Herman Yu"  wrote:

>Hi 
>
>ATLAS-725 highlights deleted entities shown in UI, Is there a way to
>filter out ³soft deleted² entities through DSL query? we are searching
>entities using qualifiedName and was expecting single record return.
>
>http://atlas.incubator.apache.org/Search.html
> does provides search
>syntax, but I found the syntax is hard to understand (may be just me),
>for example, ³hive_table orderby Œname¹² just doesn¹t work.  anybody has
>a clue on how to read/understand the search syntax?
>
>I found we spent a lot of time trying to figure out  how to call a
>RESTAPI or to run a DSL search, with Atlas is getting matured and more
>clients start using it, I think a good documentation (at least on REST
>API/Search syntax) will be greatly helpful. What do you think?
>
>Thanks
>Herman.
>

[ANNOUNCE] Apache Atlas 0.7-incubating released

[Shwetha Shivalingamurthy]

Hi All,

The Apache Atlas team is happy to announce the release of Apache Atlas -
version 0.7.0-incubating.

Atlas is a scalable and extensible set of core foundational governance
services ­ enabling enterprises to effectively and efficiently meet their
compliance requirements within Hadoop and allows integration with the whole
enterprise data ecosystem.

The release artifacts are available at:
http://www.apache.org/dyn/closer.cgi/incubator/atlas/0.7.0-incubating/

Release notes available at:
https://git-wip-us.apache.org/repos/asf?p=incubator-atlas.git;a=blob;f=rele
ase-log.txt;hb=refs/heads/0.7-incubating


To use these artifacts, please use the following documentation:
http://atlas.incubator.apache.org/0.7.0-incubating/index.html

More details on Apache Atlas can be found at:
http://atlas.incubator.apache.org/

We thank everyone who made this release possible.

Thanks,
The Apache Atlas team


DISCLAIMER

Apache Atlas is an effort undergoing incubation at the Apache
Software Foundation (ASF), sponsored by the Apache Incubator PMC.

Incubation is required of all newly accepted projects until a further
review indicates that the infrastructure, communications, and decision
making process have stabilized in a manner consistent with other
successful ASF projects.

While incubation status is not necessarily a reflection of the
completeness or stability of the code, it does indicate that the
project has yet to be fully endorsed by the ASF.

Re: Sample DSL queries?

[Shwetha Shivalingamurthy]

 http://atlas.incubator.apache.org/Search.html


DSL is sql like query syntax

Regards,
Shwetha






On 06/07/16, 6:30 PM, "Vimal Sharma"  wrote:

>You can use the sample DSL queries from QuickStart.java. Below is the link
>
>https://github.com/apache/incubator-atlas/blob/master/webapp/src/main/java
>/org/apache/atlas/examples/QuickStart.java
>
>
>You will however need to run quick_start.py script located at
>$ATLAS_HOME/bin/quick_start.py before trying out the sample queries.
>
>Thanks
>Vimal
>
>
>
>On 7/6/16, 6:22 PM, "Ernie Ostic"  wrote:
>
>>
>>Hi all...
>>
>>Is there a link to some sample DSL queries, as would be used in the
>>search
>>property for .7  ?Assuming at the moment that it is a JSON based
>>query
>>syntax.
>>
>>Thank you.
>>
>>Ernie
>>
>>
>>
>>Ernie Ostic
>>
>>WW Product Specialist, Information Server
>>IBM Analytics
>>Cell: (617) 331 8238
>>---
>>Open IGC is here!
>>
>>Extend the Catalog with custom objects and lineage definitions!
>>https://dsrealtime.wordpress.com/2015/07/29/open-igc-is-here/

[VOTE] Release Apache Atlas version 0.7-incubating RC2

[Shwetha Shivalingamurthy]

Hi All,

I have created a build for Apache Atlas 0.7-incubating, release candidate 2. 
Thanks to everyone who have contributed to this release and also tested this.

The source tarball (*.tar.gz), signature (*.asc), checksum (*.md5, *.sha512):
https://dist.apache.org/repos/dist/dev/incubator/atlas/0.7.0-incubating-rc2/

The SHA512 checksum of the archive is
8B923CEA 2CDD32C8 7CE9A066 38002BFA 14F666DC C56F95AF 2805D5C1 08B58F9E 436E3E3C
 61B40908 474C67E7 782E720E 1BFA3678 B8099EBC 282F64C2 C16A48C4

The commit id (15748e7bc5ed019a63326f201258e3a55d512d96) to be voted upon:
https://git-wip-us.apache.org/repos/asf?p=incubator-atlas.git;a=commit;h=15748e7bc5ed019a63326f201258e3a55d512d96

The tag to be voted upon:
https://git-wip-us.apache.org/repos/asf?p=incubator-atlas.git;a=log;h=refs/tags/release-0.7-rc2

The list of fixed issues:
https://git-wip-us.apache.org/repos/asf?p=incubator-atlas.git;a=blob;f=release-log.txt;hb=refs/heads/0.7-incubating

Keys to verify the signature of the release artifact are available at:
http://www.apache.org/dist/incubator/atlas/KEYS
PGP release keys:
http://pgp.mit.edu/pks/lookup?op=vindex&search=0xA0E6F9F5D96BF0FD

Note that this is a source only release and we are voting on the source 
release-0.7-incubating-rc2.

Please download, test, and try it out.

Vote will be open for at least 72 hours  till 5th July, 2016 10 PM PDT.

[ ] +1 approve
[ ] +0 no opinion
[ ] -1 disapprove (and reason why)

+1 from my side for the release.

For folks not familiar with vetting a release, please refer to
http://incubator.apache.org/guides/releasemanagement.html#check-list

Regards,
Shwetha

Re: Update/Delete Lineage

[Shwetha Shivalingamurthy]

Adding to Vimal¹s answer, the process instance is also an entity and you
can use entity update APIs to modify inputs/outputs

Regards,
Shwetha






On 01/07/16, 11:59 AM, "Vimal Sharma"  wrote:

>Hi Herman,
>Lineage update/delete is internal to Atlas backend and is not exposed via
>REST API. 
>If you update attributes of entities corresponding to input/output of a
>lineage instance, the changes should reflect in lineage process as well.
>
>Thanks
>Vimal
>
>
>
>On 6/30/16, 8:28 PM, "Herman Yu"  wrote:
>
>>Hi everyone,
>>
>>Is the capability update/delete a lineage (a classType inherits from
>>³Process²) exist with v0.7? REST API document only lists the following :
>>
>>/lineage/{guid}/inputs/graph
>>>__lineage_-guid-_inputs_graph.html>
>>/lineage/{guid}/outputs/graph
>>>__lineage_-guid-_outputs_graph.html>
>>/lineage/{guid}/schema
>>>__lineage_-guid-_schema.html> <>I have a lineage instance created and
>>then need to modify the inputs/outputs or other attributes, is this
>>supported with v0.7?
>>
>>thanks
>>Herman.
>>
>>

Re: [VOTE] Release Apache Atlas version 0.7-incubating RC1

[Shwetha Shivalingamurthy]

Hortonworks logo is a problem. Cancelling the vote, will fix it and start
another vote. Thanks for bringing this up

Regarding the test failure, is it the tests in repository module? The
tests work for us, so its probably an environment issue. Can you check the
test logs?

Regards,
Shwetha




On 30/06/16, 3:38 PM, "Ismaël Mejía"  wrote:

>+1 (non binding)
>
>verified signature + compilation ok, but some tests fail:
>Tests run: 203, Failures: 5, Errors: 0, Skipped: 26
>
>The hortonworks logo on dashboardv2/public/img/logo-green.png probably
>should
>not be included.
>
>
>
>On Thu, Jun 30, 2016 at 10:47 AM, Shwetha Shivalingamurthy <
>sshivalingamur...@hortonworks.com> wrote:
>
>> Hi All,
>>
>> I have created a build for Apache Atlas 0.7-incubating, release
>>candidate
>> 1. Thanks to everyone who have contributed to this release and also
>>tested
>> this.
>>
>> The source tarball (*.tar.gz), signature (*.asc), checksum (*.md5,
>> *.sha512):
>>
>> 
>>https://dist.apache.org/repos/dist/dev/incubator/atlas/0.7.0-incubating-r
>>c1/
>>
>>
>> The SHA512 checksum of the archive is
>> D581ED1A 405AF34C C6F9CB47 7BE6CC66 B1BE3BE6 00FE1A08 1CFCC140 92DD384D
>> A8B03488
>>  767C57E8 150AA94E 9A39299B 33928FA1 E70E995F B52C7344 0E0CED83
>>
>>
>> The commit id (91af61c49014bb0c24c53db0cd42f0bf9bda4bff) to be voted
>>upon:
>>
>> <
>> 
>>https://git-wip-us.apache.org/repos/asf?p=incubator-atlas.git;a=commit;h=
>>91af61c49014bb0c24c53db0cd42f0bf9bda4bff
>> >
>> 
>>https://git-wip-us.apache.org/repos/asf?p=incubator-atlas.git;a=commit;h=
>>91af61c49014bb0c24c53db0cd42f0bf9bda4bff
>>
>>
>> The tag to be voted upon:
>>
>>
>> 
>>https://git-wip-us.apache.org/repos/asf?p=incubator-atlas.git;a=tag;h=ref
>>s/tags/release-0.7-rc1
>>
>>
>> The list of fixed issues:
>>
>>
>> 
>>https://git-wip-us.apache.org/repos/asf?p=incubator-atlas.git;a=blob;f=re
>>lease-log.txt;h=a051afcadcedf94f7739f6b4e6402caa87eca0c8;hb=refs/heads/0.
>>7-incubating
>>
>>
>>
>> Keys to verify the signature of the release artifact are available at:
>> http://www.apache.org/dist/incubator/atlas/KEYS
>>
>> PGP release keys:
>> http://pgp.mit.edu/pks/lookup?op=vindex&search=0xA0E6F9F5D96BF0FD
>>
>> Note that this is a source only release and we are voting on the source
>> release-0.7-incubating-rc1.
>>
>> Please download, test, and try it out.
>>
>> Vote will be open for at least 72 hours till 3rd July, 2016 8 AM PDT.
>>
>> [ ] +1 approve
>> [ ] +0 no opinion
>> [ ] -1 disapprove (and reason why)
>>
>> +1 from my side for the release.
>>
>> For folks not familiar with vetting a release, please refer to
>> http://incubator.apache.org/guides/releasemanagement.html#check-list
>>
>>
>> Regards,
>> Shwetha
>>
>>

[VOTE] Release Apache Atlas version 0.7-incubating RC1

[Shwetha Shivalingamurthy]

Hi All,

I have created a build for Apache Atlas 0.7-incubating, release candidate 1. 
Thanks to everyone who have contributed to this release and also tested this.

The source tarball (*.tar.gz), signature (*.asc), checksum (*.md5, *.sha512):
https://dist.apache.org/repos/dist/dev/incubator/atlas/0.7.0-incubating-rc1/


The SHA512 checksum of the archive is
D581ED1A 405AF34C C6F9CB47 7BE6CC66 B1BE3BE6 00FE1A08 1CFCC140 92DD384D A8B03488
 767C57E8 150AA94E 9A39299B 33928FA1 E70E995F B52C7344 0E0CED83


The commit id (91af61c49014bb0c24c53db0cd42f0bf9bda4bff) to be voted upon:

https://git-wip-us.apache.org/repos/asf?p=incubator-atlas.git;a=commit;h=91af61c49014bb0c24c53db0cd42f0bf9bda4bff


The tag to be voted upon:

https://git-wip-us.apache.org/repos/asf?p=incubator-atlas.git;a=tag;h=refs/tags/release-0.7-rc1


The list of fixed issues:

https://git-wip-us.apache.org/repos/asf?p=incubator-atlas.git;a=blob;f=release-log.txt;h=a051afcadcedf94f7739f6b4e6402caa87eca0c8;hb=refs/heads/0.7-incubating



Keys to verify the signature of the release artifact are available at:
http://www.apache.org/dist/incubator/atlas/KEYS

PGP release keys:
http://pgp.mit.edu/pks/lookup?op=vindex&search=0xA0E6F9F5D96BF0FD

Note that this is a source only release and we are voting on the source 
release-0.7-incubating-rc1.

Please download, test, and try it out.

Vote will be open for at least 72 hours till 3rd July, 2016 8 AM PDT.

[ ] +1 approve
[ ] +0 no opinion
[ ] -1 disapprove (and reason why)

+1 from my side for the release.

For folks not familiar with vetting a release, please refer to
http://incubator.apache.org/guides/releasemanagement.html#check-list


Regards,
Shwetha

Re: [VOTE] Release Apache Atlas version 0.7-incubating

[Shwetha Shivalingamurthy]

Thanks Justin for verifying

We have used sha512 already. We need to remove glyphicons font. We will
also fix the LICENSE and NOTICE and get back. Thanks!

Regards,
Shwetha






On 28/06/16, 10:40 AM, "Justin Mclean"  wrote:

>Hi,
>
>-1 (binding) until font license clarified.
>
>The glypicons font is normally a commercially licensed font which is not
>compatible with the Apache license [9], under certain conditions (i.e.
>shipped with bootstrap) it¹s MIT (bottom of same page) but I don¹t see
>that here.
>
>Also please fix for the next release:
>- the incorrect year in NOTICE
>- missing information from NOTICE
>- missing licenses in LICENSE
>
>I checked:
>- file names including incubating
>- signatures and hashes correct (although you may consider using sha512
>as well as md5)
>- DISCLAIMER exists
>- NOTICE contains incorrect year, and is also missing information from
>this NOTICE file I think [1][2]
>- LICENSE is missing several things (see below)
>- all Apache licensed source files have headers
>- no unexpected binary files in the source release
>- can compile from source
>
>These items need to be listed in LICENSE I suggest using the short form
>and a pointer to the license. [10]
>- handbars.js (MIT licensed) [3]
>- animate (MIT licensed) [4]
>- json2 (public domain) [5]
>- font awesome (MIT licensed) [6]
>- font awesome (SIL license) [7]
>- bbs (MIT license) [8]
>
>Also as mentioned in last review of the incubating release - if these are
>not bundled please remove their licenses from the LICENSE file.
>"The license mentions Berkeley DB Java Edition and ASM but it doesn¹t
>look like these are bundled in the source release. If this is the case
>they should be removed from the source LICENSE file."
>
>Thanks,
>Justin
>
>1. https://github.com/thinkaurelius/titan/blob/titan10/NOTICE.txt
>2. http://www.apache.org/dev/licensing-howto.html#alv2-dep
>3. ./dashboardv2/public/js/require-handlebars-plugin/js/handlebars.js
>4. ./dashboardv2/public/css/animate.min.css
>5. ./dashboardv2/public/js/require-handlebars-plugin/js/json2.js
>6 ./dashboardv2/public/css/font-awesome.min.css
>7. apache-atlas-sources-0.7-incubating/dashboardv2/public/css/fonts/*
>8. dashboardv2/public/js/require-handlebars-plugin/js/hbs.js
>9.  http://glyphicons.com/license/
>10. http://www.apache.org/dev/licensing-howto.html#permissive-deps
>
>
>-
>To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
>For additional commands, e-mail: general-h...@incubator.apache.org
>
>

Re: quick_start.py : no server (connection refused)

[Shwetha Shivalingamurthy]

I use embedded-hbase-solr for local testing , you have to run
conf/atlas-env.sh before starting atlas. This works for me on mac laptop

Because of licensing issues, BerkelyDB jar is not packaged anymore in any
of the profiles. So, even if you build with BerkelyDB profile, you have to
copy the BerkelyDB jar manually and its documented in
InstallationSteps.twiki. BerkelyDB profile works as well after copying the
jar


Regards,
Shwetha



On 28/06/16, 8:58 PM, "Nigel Jones"  wrote:

>
>> Just to share experience: I generally use the default profile -Pdist to
>>build the atlas binary. This points to an external instance of Hbase and
>>Solr.
>>
>> This ³external² instance is just another process that runs on the same
>>machine (my Mac OS X laptop). I use Hbase 1.1.2 and Solr 5.5.1 (until
>>recently - I was using Solr 5.2.1). These two require Zookeeper to be
>>running. I generally just use the Zookeeper that comes with Kafka 0.10.
>>I use Kafka 0.10 if I need to test with external Kafka, but mostly just
>>default to the embedded one.
>
>> Possibly Linux is behaving differently with Solr etc. Any information /
>>fixes you can provide about the issue you are facing with Solr / Hbase
>>will help set this up correctly for other platforms as well.
>
>Thanks - not tried external yet .. (in part as this particular system [a
>CentOS cloud vm] doesn't [yet] have standalone hbase/solr]).
>
>
>The BerkelyDB version did start up ok & quick start runs successfully
>
>I'll go back to the others later to raise any required defects/changes
>
>Nigel.
>
>

[VOTE] Release Apache Atlas version 0.7-incubating

[Shwetha Shivalingamurthy]

Hi All,

This is a call for a vote on the Apache Atlas 0.7 incubating release.

A vote was held on developer mailing list and it passed with 10 +1's.

Vote thread: https://s.apache.org/r1o6
Results thread: https://s.apache.org/L3fE

The source tarball (*.tar.gz), signature (*.asc), checksum (*.md5, *.sha): 
https://dist.apache.org/repos/dist/dev/incubator/atlas/0.7.0-incubating-rc0/

The SHA512 checksum of the archive is
1B1F16A6 5F7A4EAE 057F410B DC4753B1 F1C0D49B 588B5C91 C8641634 77F0CDF8 15F46252
863D6CEA C98863B2 3853F156 B18D212E 30159C47 ACB1E7BA C6F8DB91

The commit id (ca9fea712d9a920e5949f16c32a759acd9222f19) to be voted upon:
https://git-wip-us.apache.org/repos/asf?p=incubator-atlas.git;a=commit;h=ca9fea712d9a920e5949f16c32a759acd9222f19

The tag to be voted upon:
https://git-wip-us.apache.org/repos/asf?p=incubator-atlas.git;a=log;h=refs/tags/release-0.7-rc0

The list of fixed issues:
https://git-wip-us.apache.org/repos/asf?p=incubator-atlas.git;a=blob;f=release-log.txt;h=5d72620f1db75c3e032745c3f7c6cefa85b9505c;hb=refs/heads/0.7-incubating

Keys to verify the signature of the release artifact are available at:
http://www.apache.org/dist/incubator/atlas/KEYS
PGP release keys:
http://pgp.mit.edu/pks/lookup?op=vindex&search=0xA0E6F9F5D96BF0FD

Note that this is a source only release and we are voting on the source 
release-0.7-incubating-rc0.

Please download, test, and try it out.

Vote will be open for at least 72 hours till 29th June, 2016 11 PM PDT.

[ ] +1 approve
[ ] +0 no opinion
[ ] -1 disapprove (and reason why)

Thanks!

Regards,
Shwetha

[RESULT][VOTE] Release Apache Atlas version 0.7-incubating

[Shwetha Shivalingamurthy]

Thanks for taking time to review and vote on this release.  The vote
passes with 10 +1's and no 0/-1’s.


Binding votes:
Hemanth Yamijala (PPMC)
Jean-Baptiste Onofré (IPMC)
Shwetha GS (PPMC)
Suma Shivaprasad (PPMC)
Venkat Ranganathan (PPMC)
Venkatesh Seetharam (IPMC, PPMC)


Non-Binding Votes:
Ismaël Mejía
Madhan Neethiraj
Selvamohan Neethiraj
Vimal Sharma



I'll start voting on Incubator general.



Regards,
Shwetha





On 24/06/16, 1:18 PM, "Ismaël Mejía"  wrote:

>+1 (non-binding),
>
>It compiles, some tests fail (ubuntu 16.04 java: oracle-jdk-8):
>Tests run: 203, Failures: 5, Errors: 0, Skipped: 26
>
>Please don't forget to publish the artifacts on Maven Central (as it
>happened with 0.6), those are essential for people writing integrations or
>other products based on Atlas.
>
>​Ismaël​
>
>
>On Thu, Jun 23, 2016 at 11:53 PM, Selvamohan Neethiraj
>
>wrote:
>
>> +1
>>
>> Downloaded source ;  Able to build successfully ; Validated the
>>signature
>> and checksum ….
>>
>> Thanks,
>> Selva-
>>
>> From:  Shwetha Shivalingamurthy 
>> Reply-To:  "dev@atlas.incubator.apache.org" <
>> dev@atlas.incubator.apache.org>
>> Date:  Wednesday, June 22, 2016 at 6:33 AM
>> To:  "dev@atlas.incubator.apache.org" 
>> Subject:  [VOTE] Release Apache Atlas version 0.7-incubating
>>
>> Hi All,
>>
>> I have created a build for Apache Atlas 0.7-incubating, release
>>candidate
>> 0. Thanks to everyone who have contributed to this release and also
>>tested
>> this.
>>
>> The source tarball (*.tar.gz), signature (*.asc), checksum (*.md5,
>>*.sha):
>> <
>> 
>>https://dist.apache.org/repos/dist/dev/incubator/atlas/0.7.0-incubating-r
>>c0/
>> >
>> 
>>https://dist.apache.org/repos/dist/dev/incubator/atlas/0.7.0-incubating-r
>>c0/
>>
>> The SHA512 checksum of the archive is
>> 1B1F16A6 5F7A4EAE 057F410B DC4753B1 F1C0D49B 588B5C91 C8641634 77F0CDF8
>> 15F46252
>>  863D6CEA C98863B2 3853F156 B18D212E 30159C47 ACB1E7BA C6F8DB91
>>
>> The commit id (ca9fea712d9a920e5949f16c32a759acd9222f19) to be voted
>>upon:
>>
>> 
>>https://git-wip-us.apache.org/repos/asf?p=incubator-atlas.git;a=commit;h=
>>ca9fea712d9a920e5949f16c32a759acd9222f19
>>
>> The tag to be voted upon:
>>
>> 
>>https://git-wip-us.apache.org/repos/asf?p=incubator-atlas.git;a=log;h=ref
>>s/tags/release-0.7-rc0
>>
>> The list of fixed issues:
>>
>> 
>>https://git-wip-us.apache.org/repos/asf?p=incubator-atlas.git;a=blob;f=re
>>lease-log.txt;h=5d72620f1db75c3e032745c3f7c6cefa85b9505c;hb=refs/heads/0.
>>7-incubating
>>
>> Keys to verify the signature of the release artifact are available at:
>> http://www.apache.org/dist/incubator/atlas/KEYS
>> PGP release keys:
>> http://pgp.mit.edu/pks/lookup?op=vindex&search=0xA0E6F9F5D96BF0FD
>>
>> Note that this is a source only release and we are voting on the source
>> release-0.7-incubating-rc0.
>>
>> Please download, test, and try it out.
>>
>> Vote will be open for at least 72 hours (until the required number of
>>IPMC
>> votes are obtained) till 26th June, 2016 10 AM PDT.
>>
>> [ ] +1 approve
>> [ ] +0 no opinion
>> [ ] -1 disapprove (and reason why)
>>
>> +1 from my side for the release.
>>
>> For folks not familiar with vetting a release, please refer to
>> http://incubator.apache.org/guides/releasemanagement.html#check-list
>>
>> Regards,
>> Shwetha
>>
>>
>>

[VOTE] Release Apache Atlas version 0.7-incubating

[Shwetha Shivalingamurthy]

Hi All,

I have created a build for Apache Atlas 0.7-incubating, release candidate 0. 
Thanks to everyone who have contributed to this release and also tested this.

The source tarball (*.tar.gz), signature (*.asc), checksum (*.md5, *.sha):
https://dist.apache.org/repos/dist/dev/incubator/atlas/0.7.0-incubating-rc0/

The SHA512 checksum of the archive is
1B1F16A6 5F7A4EAE 057F410B DC4753B1 F1C0D49B 588B5C91 C8641634 77F0CDF8 15F46252
 863D6CEA C98863B2 3853F156 B18D212E 30159C47 ACB1E7BA C6F8DB91

The commit id (ca9fea712d9a920e5949f16c32a759acd9222f19) to be voted upon:
https://git-wip-us.apache.org/repos/asf?p=incubator-atlas.git;a=commit;h=ca9fea712d9a920e5949f16c32a759acd9222f19

The tag to be voted upon:
https://git-wip-us.apache.org/repos/asf?p=incubator-atlas.git;a=log;h=refs/tags/release-0.7-rc0

The list of fixed issues:
https://git-wip-us.apache.org/repos/asf?p=incubator-atlas.git;a=blob;f=release-log.txt;h=5d72620f1db75c3e032745c3f7c6cefa85b9505c;hb=refs/heads/0.7-incubating

Keys to verify the signature of the release artifact are available at:
http://www.apache.org/dist/incubator/atlas/KEYS
PGP release keys:
http://pgp.mit.edu/pks/lookup?op=vindex&search=0xA0E6F9F5D96BF0FD

Note that this is a source only release and we are voting on the source 
release-0.7-incubating-rc0.

Please download, test, and try it out.

Vote will be open for at least 72 hours (until the required number of IPMC 
votes are obtained) till 26th June, 2016 10 AM PDT.

[ ] +1 approve
[ ] +0 no opinion
[ ] -1 disapprove (and reason why)

+1 from my side for the release.

For folks not familiar with vetting a release, please refer to
http://incubator.apache.org/guides/releasemanagement.html#check-list

Regards,
Shwetha

Re: Atlas and sqoop hook

[Shwetha Shivalingamurthy]

Make sure you have latest sqoop, built from sqoop trunk.

Use ‹verbose in sqoop command to get debug logs. From the logs, you can
verify if sqoop atlas hook is invoked. Also make sure that the sqoop hook
picks the right atlas conf

Regards,
Shwetha






On 20/06/16, 11:58 PM, "Margus Roo"  wrote:

>Hi
>
>Downloaded and compiled atlas 0.7.
>Hive hook is working - create table [tablename] as select * from [src
>tablename] is working and data lineage is generated in atlas.
>Next I tried sqoop hook and followed
>http://atlas.incubator.apache.org/Bridge-Sqoop.html
>
>Command:
>sqoop-import --connect jdbc:mysql://mysqlhost/test --table sqoop_test
>--split-by id --hive-import -hive-table sqoop_test19 --username margusja
>--P
>creates a new table in Hive and new table is in atlas also but no data
>lineage
>
>I see from 
>http://hortonworks.com/hadoop-tutorial/cross-component-lineage-apache-atla
>s/
>There I can see that extra config parameters are loaded (in picture
>https://raw.githubusercontent.com/hortonworks/tutorials/atlas-ranger-tp/as
>sets/cross-component-lineage-with-atlas/8-sqoop-import-finish.png)
>and kafka producer creating ouutput but in my command:
>sqoop-import --connect jdbc:mysql://mysqlhost/test --table sqoop_test
>--split-by id --hive-import -hive-table sqoop_test19 --username margusja
>--P
>there is no extra output only:
>
>Warning: /usr/hdp/2.4.0.0-169/accumulo does not exist! Accumulo imports
>will fail.
>Please set $ACCUMULO_HOME to the root of your Accumulo installation.
>16/06/20 21:25:47 INFO sqoop.Sqoop: Running Sqoop version:
>1.4.6.2.4.0.0-169
>16/06/20 21:25:47 WARN tool.BaseSqoopTool: Setting your password on the
>command-line is insecure. Consider using -P instead.
>16/06/20 21:25:47 INFO tool.BaseSqoopTool: Using Hive-specific
>delimiters for output. You can override
>16/06/20 21:25:47 INFO tool.BaseSqoopTool: delimiters with
>--fields-terminated-by, etc.
>16/06/20 21:25:47 INFO manager.MySQLManager: Preparing to use a MySQL
>streaming resultset.
>16/06/20 21:25:47 INFO tool.CodeGenTool: Beginning code generation
>16/06/20 21:25:47 INFO manager.SqlManager: Executing SQL statement:
>SELECT t.* FROM `sqoop_test` AS t LIMIT 1
>16/06/20 21:25:47 INFO manager.SqlManager: Executing SQL statement:
>SELECT t.* FROM `sqoop_test` AS t LIMIT 1
>16/06/20 21:25:47 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is
>/usr/hdp/2.4.0.0-169/hadoop-mapreduce
>Note: 
>/tmp/sqoop-root/compile/49b525e14ebd68542d86b68dc399bd84/sqoop_test.java
>uses or overrides a deprecated API.
>Note: Recompile with -Xlint:deprecation for details.
>16/06/20 21:25:48 INFO orm.CompilationManager: Writing jar file:
>/tmp/sqoop-root/compile/49b525e14ebd68542d86b68dc399bd84/sqoop_test.jar
>16/06/20 21:25:48 WARN manager.MySQLManager: It looks like you are
>importing from mysql.
>16/06/20 21:25:48 WARN manager.MySQLManager: This transfer can be
>faster! Use the --direct
>16/06/20 21:25:48 WARN manager.MySQLManager: option to exercise a
>MySQL-specific fast path.
>16/06/20 21:25:48 INFO manager.MySQLManager: Setting zero DATETIME
>behavior to convertToNull (mysql)
>16/06/20 21:25:48 INFO mapreduce.ImportJobBase: Beginning import of
>sqoop_test
>SLF4J: Class path contains multiple SLF4J bindings.
>SLF4J: Found binding in
>[jar:file:/usr/hdp/2.4.0.0-169/hadoop/lib/slf4j-log4j12-1.7.10.jar!/org/sl
>f4j/impl/StaticLoggerBinder.class]
>SLF4J: Found binding in
>[jar:file:/usr/hdp/2.4.0.0-169/zookeeper/lib/slf4j-log4j12-1.6.1.jar!/org/
>slf4j/impl/StaticLoggerBinder.class]
>SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
>explanation.
>SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
>16/06/20 21:25:50 INFO impl.TimelineClientImpl: Timeline service
>address: http://bigdata21.webmedia.int:8188/ws/v1/timeline/
>16/06/20 21:25:50 INFO client.RMProxy: Connecting to ResourceManager at
>bigdata21.webmedia.int/192.168.81.110:8050
>16/06/20 21:25:52 INFO db.DBInputFormat: Using read commited transaction
>isolation
>16/06/20 21:25:52 INFO db.DataDrivenDBInputFormat: BoundingValsQuery:
>SELECT MIN(`id`), MAX(`id`) FROM `sqoop_test`
>16/06/20 21:25:52 INFO mapreduce.JobSubmitter: number of splits:2
>16/06/20 21:25:52 INFO mapreduce.JobSubmitter: Submitting tokens for
>job: job_1460979043517_0118
>16/06/20 21:25:53 INFO impl.YarnClientImpl: Submitted application
>application_1460979043517_0118
>16/06/20 21:25:53 INFO mapreduce.Job: The url to track the job:
>http://bigdata21.webmedia.int:8088/proxy/application_1460979043517_0118/
>16/06/20 21:25:53 INFO mapreduce.Job: Running job: job_1460979043517_0118
>16/06/20 21:25:58 INFO mapreduce.Job: Job job_1460979043517_0118 running
>in uber mode : false
>16/06/20 21:25:58 INFO mapreduce.Job:  map 0% reduce 0%
>16/06/20 21:26:02 INFO mapreduce.Job:  map 50% reduce 0%
>16/06/20 21:26:03 INFO mapreduce.Job:  map 100% reduce 0%
>16/06/20 21:26:03 INFO mapreduce.Job: Job job_1460979043517_0118
>completed successfully
>16/06/20 21:26:03 INFO mapreduce.Job: Counters: 

[ANNOUNCE] New Committer: Hemanth Yamijala

[Shwetha Shivalingamurthy]

Hi All,

The Apache Atlas PPMC has asked Hemanth Yamijala to become a committer and we 
are pleased to announce that he has accepted.

Regards,
Shwetha

Re: Review Request 45029: ATLAS-494 : Authentication

[Shwetha Shivalingamurthy]

Lets keep it as admin=admin, consistent with ambari default?

Regards,
Shwetha


From: Hemanth Yamijala 
mailto:nore...@reviews.apache.org>> on behalf of 
Hemanth Yamijala mailto:yhema...@gmail.com>>
Reply-To: Hemanth Yamijala mailto:yhema...@gmail.com>>
Date: Wednesday, 6 April 2016 at 10:11 AM
To: Erik Bergenholtz 
mailto:ebergenho...@hortonworks.com>>, Hemanth 
Yamijala mailto:yhema...@gmail.com>>, Shwetha 
Shivalingamurthy 
mailto:sshivalingamur...@hortonworks.com>>
Cc: atlas 
mailto:dev@atlas.incubator.apache.org>>, Nixon 
Rodrigues 
mailto:nixon.rodrig...@freestoneinfotech.com>>
Subject: Re: Review Request 45029: ATLAS-494 : Authentication

This is an automatically generated e-mail. To reply, visit: 
https://reviews.apache.org/r/45029/

distro/src/conf/users-credentials.properties<https://reviews.apache.org/r/45029/diff/2/?file=1324472#file1324472line3>(Diff
 revision 2)

3

username=password


Lets just set this to 2 users: admin=admin123, and user=user123 (for 2 
different roles).


- Hemanth Yamijala


On April 5th, 2016, 6:25 a.m. UTC, Nixon Rodrigues wrote:

Review request for atlas, Erik Bergenholtz, Shwetha GS, Suma Shivaprasad, and 
Hemanth Yamijala.
By Nixon Rodrigues.

Updated April 5, 2016, 6:25 a.m.

Bugs: ATLAS-494<https://issues.apache.org/jira/browse/ATLAS-494>
Repository: atlas
Description

Problem Statement

Atlas needs an authentication mechanism besides kerberos.
Requirements
Provide ability to authenticate against local file
Provide Authentication that supports LDAP
For consistency, configuration should use exactly the same configuration 
(properties) as Ranger & Knox


Testing

Done.

Some existing test case were failing for master.

Failed tests:
  
HiveLineageJerseyResourceIT.setUp:52->setupInstances:169->table:234->BaseResourceIT.createInstance:129
 » AtlasService

Tests run: 72, Failures: 1, Errors: 0, Skipped: 5

Tests run: 6, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 28.161 sec <<< 
FAILURE! - in 
org.apache.atlas.repository.graph.GraphBackedMetadataRepositoryDeleteEntitiesTest
testDisconnectUnidirectionalArrayReferenceFromClassType(org.apache.atlas.repository.graph.GraphBackedMetadataRepositoryDeleteEntitiesTest)
  Time elapsed: 0.756 sec  <<< FAILURE!
org.apache.atlas.repository.RepositoryException: 
org.apache.atlas.typesystem.types.ValueConversionException$NullConversionException:
 For field 'columns'
at org.apache.atlas.typesystem.types.ClassType.convert(ClassType.java:157)
at org.apache.atlas.typesystem.types.ClassType.convert(ClassType.java:45)
at 
org.apache.atlas.typesystem.types.DataTypes$ArrayType.convert(DataTypes.java:490)
at 
org.apache.atlas.typesystem.types.DataTypes$ArrayType.convert(DataTypes.java:459)
at 
org.apache.atlas.typesystem.persistence.StructInstance.set(StructInstance.java:122)
at 
org.apache.atlas.repository.graph.GraphToTypedInstanceMapper.mapVertexToArrayInstance(GraphToTypedInstanceMapper.java:196)
at 
org.apache.atlas.repository.graph.GraphToTypedInstanceMapper.mapVertexToAttribute(GraphToTypedInstanceMapper.java:122)
at 
org.apache.atlas.repository.graph.GraphToTypedInstanceMapper.mapVertexToInstance(GraphToTypedInstanceMapper.java:95)
at 
org.apache.atlas.repository.graph.GraphToTypedInstanceMapper.mapGraphToTypedInstance(GraphToTypedInstanceMapper.java:74)
at 
org.apache.atlas.repository.graph.GraphBackedMetadataRepository.getEntityDefinition(GraphBackedMetadataRepository.java:144)
at 
org.apache.atlas.GraphTransactionInterceptor.invoke(GraphTransactionInterceptor.java:42)
at 
org.apache.atlas.repository.graph.GraphBackedMetadataRepositoryDeleteEntitiesTest.testDisconnectUnidirectionalArrayReferenceFromClassType(GraphBackedMetadataRepositoryDeleteEntitiesTest.java:324)


Diffs

  *   dashboard/public/css/login.css (PRE-CREATION)
  *   dashboard/public/modules/home/views/header.html (896cfb4)
  *   distro/src/conf/atlas-application.properties (29df5b3)
  *   distro/src/conf/users-credentials.properties (PRE-CREATION)
  *   pom.xml (226529d)
  *   webapp/pom.xml (85c9471)
  *   webapp/src/main/java/org/apache/atlas/util/PropertiesUtil.java 
(PRE-CREATION)
  *   webapp/src/main/java/org/apache/atlas/util/XMLPropertiesUtil.java 
(PRE-CREATION)
  *   webapp/src/main/java/org/apache/atlas/web/dao/UserDao.java (PRE-CREATION)
  *   
webapp/src/main/java/org/apache/atlas/web/filters/AtlasAuthenticationEntryPoint.java
 (PRE-CREATION)
  *   webapp/src/main/java/org/apache/atlas/web/model/User.java (PRE-CREATION)
  *   
webapp/src/main/java/org/apache/atlas/web/security/AtlasADAuthenticationProvider.java
 (PRE-CREATION)
  *   
webapp/src/main/java/org/apache/atlas/web/security/AtlasAbstractAuthenticationProvider.java
 (PRE-CREATION)
  *   
webapp/src/main/java/org/apache/atlas/web/security/AtlasAuthenticationException.java
 (PRE-CREATION)
  *   
webapp/src/main/java/org/apache/atlas

Re: Atlas build error - npm install --color=false failed

[Shwetha Shivalingamurthy]

We should document this in InstallationSteps.twiki. Can you update?

Regards,
Shwetha






On 31/03/16, 3:49 PM, "Venkata R Madugundu" 
wrote:

>
>FYI
>
>With latest sources of Atlas, I am seeing the following build error (on
>Windows7) ...
>
>---
>  
>  
> 
> [ERROR] Failed to execute goal
>com.github.eirslett:frontend-maven-plugin:0.0.23:npm (npm install) on
>project atlas-dashboard: Failed to run task:
> 'npm install --color=false' failed. (error code 50) -> [Help 1]
>---
>  
>  
>  
> 
> After a bit of searching around on web, someone hinted to run cmd as
>'Run as Administrator' to resolve 'npm' related maven build errors.
> 
> And that helped incase anyone comes across a similar error.
>  
> 
>  
>  
> 
>

Re: Incompatible changes tag in JIRA

[Shwetha Shivalingamurthy]

Release notes is picked up from release-log.txt which already has a
section for incompatible changes. But its useful to have a label in jira
as well

Regards,
Shwetha






On 11/03/16, 11:22 AM, "Hemanth Yamijala" 
wrote:

>Hi all,
>
>Some other Hadoop ecosystem projects I know have an incompatible change
>tag in JIRA that becomes very useful to let users in the community know
>of incompatible changes. It also becomes useful to create release notes
>and other operational activities. Can we do this for ATLAS as well?
>Thoughts?
>
>Thanks
>hemanth

Re: Filter lineages being displayed

[Shwetha Shivalingamurthy]

Currently, atlas maintains just the latest lineage. Point in time lineage
requires graph versioning.

We can probably add filter for lineage. Can you file a jira for that?

Regards,
Shwetha






On 26/02/16, 8:19 PM, "Herman Yu"  wrote:

>Hi Shwetha,
>
>Use case 1: display point-in-time lineage information only
>Use case 2: display certain types of lineage only. e.g. only technical
>lineage, or only business semantic lineage, or only lineage at
>table/column level...
>Use case 3: with built-in hive model, filter out those ³temporary² hive
>tables and lineages associated with themŠ
>
>Thanks
>Herman.
>
>
>> On Feb 25, 2016, at 3:10 AM, Shwetha Shivalingamurthy
>> wrote:
>> 
>> Herman,
>> 
>> Whats your use case for this filtering?
>> 
>> Regards,
>> Shwetha
>> 
>> 
>> 
>> 
>> 
>> 
>> On 24/02/16, 8:10 PM, "Herman Yu"  wrote:
>> 
>>> Is there a way to filter the lineages being displayed? for example,
>>>for a
>>> given hive_table entity, there are two inputs and two outputs, is
>>>there a
>>> way only display one input and one output in the data lineage diagram?
>>> 
>>> Thanks
>>> Herman.
>>> 
>>> 
>> 
>
>

Re: Entity Types Need Overly Verbose JSON

[Shwetha Shivalingamurthy]

Hi Joseph,

Yes, it makes sense to not have empty attributes in json. Can you file a
bug?

Regards,
Shwetha


On 27/02/16, 10:28 PM, "Joseph Niemiec"  wrote:

>Hi Atlas Dev,
>
>I have recently been playing around with making my own custom types. After
>much work I came to realize that even if an element would be empty it has
>to be included in the message itself, examples of this would be:
>traitNames, traits, and values which can often be completely blank.
>
>To reduce overhead and datasize would it not be better to treat it so that
>if the values (when part of traits, or just blank), traitNames and/or
>traits is missing that is it blank and not completely fail during JSON
>de-serialization?
>
>
>{
>"jsonClass":
>"org.apache.atlas.typesystem.json.InstanceSerialization$_Reference",
>"id": {
>"jsonClass": "org.apache.atlas.typesystem.json.InstanceSerialization$_Id",
>"id": "-19029388374",
>"version": 0,
>"typeName": "CUSTOM"
>},
>"typeName": "CUSTOM",
>"values": {
>"name": "A",
>"description": "Non Prod",
>},
>* "traitNames": [],*
>* "traits": {}*
>}
>
>-- 
>Joseph

Re: Filter lineages being displayed

[Shwetha Shivalingamurthy]

Herman,

Whats your use case for this filtering?

Regards,
Shwetha






On 24/02/16, 8:10 PM, "Herman Yu"  wrote:

>Is there a way to filter the lineages being displayed? for example, for a
>given hive_table entity, there are two inputs and two outputs, is there a
>way only display one input and one output in the data lineage diagram?
>
>Thanks
>Herman.
>
>

Re: Request to add me as a contributor to Apache ATLAS project ....

[Shwetha Shivalingamurthy]

Hi Selva,

I have added you as contributor. Thank you

There is no atlas specific criteria for making someone committer. Just
like in other apache projects, if someone is actively involved and has
enough knowledge of atlas, there will be proposal and voting in the atlas
PMC. 

Regards,
Shwetha




On 04/02/16, 10:37 AM, "Selvamohan Neethiraj" 
wrote:

>Hi,
>
>I am a newbie to Apache ATLAS dev group and have been working as a core
>committer on Apache Ranger.
>I would like to contribute to Apache ATLAS on few areas where I am very
>familiar with and learn new areas from Apache Atlas dev group.
>
>Can you make me as a contributor to the Apache ATLAS projector? This
>would allow me to pick-up task/bugs and provide patches as desired and
>needed.
>
>Also, Is there any specific criteria for moving from Contributor to
>Committer in the Apache ATLAS project?
>
>Thanks,
>Selva-
>

Re: Build failed in Jenkins: apache-atlas-nightly #174

[Shwetha Shivalingamurthy]

The build succeeds on my local machine. So, probably need to increase the
wait timeout

Regards,
Shwetha






On 01/02/16, 12:18 PM, "Shwetha Shivalingamurthy"
 wrote:

>Created https://issues.apache.org/jira/browse/ATLAS-462
>
>
>Regards,
>Shwetha
>
>
>
>
>
>
>On 29/01/16, 1:34 PM, "Apache Jenkins Server" 
>wrote:
>
>>See <https://builds.apache.org/job/apache-atlas-nightly/174/changes>
>>
>>Changes:
>>
>>[sshivalingamurthy] ATLAS-415 Hive import fails when importing a table
>>that is already
>>
>>[sshivalingamurthy] ATLAS-199 webapp build fails (grunt + tests) (sanjayp
>>via shwethags)
>>
>>--
>>[...truncated 78967 lines...]
>>127.0.0.1 - - [29/Jan/2016:07:56:45 +] "PUT /api/atlas/entities
>>HTTP/1.1" 200 - "-" "Java/1.7.0_80"
>>127.0.0.1 - - [29/Jan/2016:07:56:45 +] "GET
>>/api/atlas/discovery/search?query=hive_table+as+t+where+tableName+%3D+'ta
>>b
>>le7qzx0jevrk',+db+where+name+%3D+'dbhwktsw4tpv'+and+clusterName+%3D+'test
>>'
>>+select+t HTTP/1.1" 200 - "-" "Java/1.7.0_80"
>>127.0.0.1 - - [29/Jan/2016:07:56:51 +] "OPTIONS
>>/api/atlas/discovery/search?query=hive_table+as+t+where+tableName+%3D+'ta
>>b
>>le7qzx0jevrk',+db+where+name+%3D+'dbhwktsw4tpv'+and+clusterName+%3D+'test
>>'
>>+select+t&user.name=jenkins HTTP/1.1" 200 - "-" "Java/1.7.0_80"
>>127.0.0.1 - - [29/Jan/2016:07:56:51 +] "GET
>>/api/atlas/discovery/search?query=hive_table+as+t+where+tableName+%3D+'ta
>>b
>>le7qzx0jevrk',+db+where+name+%3D+'dbhwktsw4tpv'+and+clusterName+%3D+'test
>>'
>>+select+t HTTP/1.1" 200 - "-" "Java/1.7.0_80"
>>127.0.0.1 - - [29/Jan/2016:07:56:55 +] "OPTIONS
>>/api/atlas/discovery/search?query=hive_column+where+name+%3D+'colulpk1fga
>>z
>>9'&user.name=jenkins HTTP/1.1" 200 - "-" "Java/1.7.0_80"
>>127.0.0.1 - - [29/Jan/2016:07:56:55 +] "GET
>>/api/atlas/discovery/search?query=hive_column+where+name+%3D+'colulpk1fga
>>z
>>9' HTTP/1.1" 200 - "-" "Java/1.7.0_80"
>>127.0.0.1 - - [29/Jan/2016:07:57:00 +] "OPTIONS
>>/api/atlas/discovery/search?query=hive_column+where+name+%3D+'colulpk1fga
>>z
>>9'&user.name=jenkins HTTP/1.1" 200 - "-" "Java/1.7.0_80"
>>127.0.0.1 - - [29/Jan/2016:07:57:00 +] "GET
>>/api/atlas/discovery/search?query=hive_column+where+name+%3D+'colulpk1fga
>>z
>>9' HTTP/1.1" 200 - "-" "Java/1.7.0_80"
>>127.0.0.1 - - [29/Jan/2016:07:57:05 +] "OPTIONS
>>/api/atlas/entities/0738845b-079a-46d8-aa48-5e16584c362e?user.name=jenkin
>>s
>> HTTP/1.1" 200 - "-" "Java/1.7.0_80"
>>127.0.0.1 - - [29/Jan/2016:07:57:05 +] "GET
>>/api/atlas/entities/0738845b-079a-46d8-aa48-5e16584c362e HTTP/1.1" 200 -
>>"-" "Java/1.7.0_80"
>>127.0.0.1 - - [29/Jan/2016:07:57:05 +] "OPTIONS
>>/api/atlas/discovery/search?query=hive_table+as+t+where+tableName+%3D+'ta
>>b
>>leabjmoea4aa',+db+where+name+%3D+'default'+and+clusterName+%3D+'test'+sel
>>e
>>ct+t&user.name=jenkins HTTP/1.1" 200 - "-" "Java/1.7.0_80"
>>127.0.0.1 - - [29/Jan/2016:07:57:05 +] "OPTIONS
>>/api/atlas/entities?user.name=jenkins HTTP/1.1" 200 - "-" "Java/1.7.0_80"
>>127.0.0.1 - - [29/Jan/2016:07:57:05 +] "PUT /api/atlas/entities
>>HTTP/1.1" 200 - "-" "Java/1.7.0_80"
>>127.0.0.1 - - [29/Jan/2016:07:57:05 +] "GET
>>/api/atlas/discovery/search?query=hive_table+as+t+where+tableName+%3D+'ta
>>b
>>leabjmoea4aa',+db+where+name+%3D+'default'+and+clusterName+%3D+'test'+sel
>>e
>>ct+t HTTP/1.1" 200 - "-" "Java/1.7.0_80"
>>127.0.0.1 - - [29/Jan/2016:07:57:11 +] "OPTIONS
>>/api/atlas/discovery/search?query=hive_table+as+t+where+tableName+%3D+'ta
>>b
>>leabjmoea4aa',+db+where+name+%3D+'default'+and+clusterName+%3D+'test'+sel
>>e
>>ct+t&user.name=jenkins HTTP/1.1" 200 - "-" "Java/1.7.0_80"
>>127.0.0.1 - - [29/Jan/2016:07:57:11 +] "GET
>>/api/atlas/discovery/search?query=hi

Re: Build failed in Jenkins: apache-atlas-nightly #174

[Shwetha Shivalingamurthy]

Created https://issues.apache.org/jira/browse/ATLAS-462


Regards,
Shwetha






On 29/01/16, 1:34 PM, "Apache Jenkins Server" 
wrote:

>See 
>
>Changes:
>
>[sshivalingamurthy] ATLAS-415 Hive import fails when importing a table
>that is already
>
>[sshivalingamurthy] ATLAS-199 webapp build fails (grunt + tests) (sanjayp
>via shwethags)
>
>--
>[...truncated 78967 lines...]
>127.0.0.1 - - [29/Jan/2016:07:56:45 +] "PUT /api/atlas/entities
>HTTP/1.1" 200 - "-" "Java/1.7.0_80"
>127.0.0.1 - - [29/Jan/2016:07:56:45 +] "GET
>/api/atlas/discovery/search?query=hive_table+as+t+where+tableName+%3D+'tab
>le7qzx0jevrk',+db+where+name+%3D+'dbhwktsw4tpv'+and+clusterName+%3D+'test'
>+select+t HTTP/1.1" 200 - "-" "Java/1.7.0_80"
>127.0.0.1 - - [29/Jan/2016:07:56:51 +] "OPTIONS
>/api/atlas/discovery/search?query=hive_table+as+t+where+tableName+%3D+'tab
>le7qzx0jevrk',+db+where+name+%3D+'dbhwktsw4tpv'+and+clusterName+%3D+'test'
>+select+t&user.name=jenkins HTTP/1.1" 200 - "-" "Java/1.7.0_80"
>127.0.0.1 - - [29/Jan/2016:07:56:51 +] "GET
>/api/atlas/discovery/search?query=hive_table+as+t+where+tableName+%3D+'tab
>le7qzx0jevrk',+db+where+name+%3D+'dbhwktsw4tpv'+and+clusterName+%3D+'test'
>+select+t HTTP/1.1" 200 - "-" "Java/1.7.0_80"
>127.0.0.1 - - [29/Jan/2016:07:56:55 +] "OPTIONS
>/api/atlas/discovery/search?query=hive_column+where+name+%3D+'colulpk1fgaz
>9'&user.name=jenkins HTTP/1.1" 200 - "-" "Java/1.7.0_80"
>127.0.0.1 - - [29/Jan/2016:07:56:55 +] "GET
>/api/atlas/discovery/search?query=hive_column+where+name+%3D+'colulpk1fgaz
>9' HTTP/1.1" 200 - "-" "Java/1.7.0_80"
>127.0.0.1 - - [29/Jan/2016:07:57:00 +] "OPTIONS
>/api/atlas/discovery/search?query=hive_column+where+name+%3D+'colulpk1fgaz
>9'&user.name=jenkins HTTP/1.1" 200 - "-" "Java/1.7.0_80"
>127.0.0.1 - - [29/Jan/2016:07:57:00 +] "GET
>/api/atlas/discovery/search?query=hive_column+where+name+%3D+'colulpk1fgaz
>9' HTTP/1.1" 200 - "-" "Java/1.7.0_80"
>127.0.0.1 - - [29/Jan/2016:07:57:05 +] "OPTIONS
>/api/atlas/entities/0738845b-079a-46d8-aa48-5e16584c362e?user.name=jenkins
> HTTP/1.1" 200 - "-" "Java/1.7.0_80"
>127.0.0.1 - - [29/Jan/2016:07:57:05 +] "GET
>/api/atlas/entities/0738845b-079a-46d8-aa48-5e16584c362e HTTP/1.1" 200 -
>"-" "Java/1.7.0_80"
>127.0.0.1 - - [29/Jan/2016:07:57:05 +] "OPTIONS
>/api/atlas/discovery/search?query=hive_table+as+t+where+tableName+%3D+'tab
>leabjmoea4aa',+db+where+name+%3D+'default'+and+clusterName+%3D+'test'+sele
>ct+t&user.name=jenkins HTTP/1.1" 200 - "-" "Java/1.7.0_80"
>127.0.0.1 - - [29/Jan/2016:07:57:05 +] "OPTIONS
>/api/atlas/entities?user.name=jenkins HTTP/1.1" 200 - "-" "Java/1.7.0_80"
>127.0.0.1 - - [29/Jan/2016:07:57:05 +] "PUT /api/atlas/entities
>HTTP/1.1" 200 - "-" "Java/1.7.0_80"
>127.0.0.1 - - [29/Jan/2016:07:57:05 +] "GET
>/api/atlas/discovery/search?query=hive_table+as+t+where+tableName+%3D+'tab
>leabjmoea4aa',+db+where+name+%3D+'default'+and+clusterName+%3D+'test'+sele
>ct+t HTTP/1.1" 200 - "-" "Java/1.7.0_80"
>127.0.0.1 - - [29/Jan/2016:07:57:11 +] "OPTIONS
>/api/atlas/discovery/search?query=hive_table+as+t+where+tableName+%3D+'tab
>leabjmoea4aa',+db+where+name+%3D+'default'+and+clusterName+%3D+'test'+sele
>ct+t&user.name=jenkins HTTP/1.1" 200 - "-" "Java/1.7.0_80"
>127.0.0.1 - - [29/Jan/2016:07:57:11 +] "GET
>/api/atlas/discovery/search?query=hive_table+as+t+where+tableName+%3D+'tab
>leabjmoea4aa',+db+where+name+%3D+'default'+and+clusterName+%3D+'test'+sele
>ct+t HTTP/1.1" 200 - "-" "Java/1.7.0_80"
>127.0.0.1 - - [29/Jan/2016:07:57:16 +] "OPTIONS
>/api/atlas/entities/377549c5-bab1-471f-9a2d-cb8281b67710?user.name=jenkins
> HTTP/1.1" 200 - "-" "Java/1.7.0_80"
>127.0.0.1 - - [29/Jan/2016:07:57:16 +] "GET
>/api/atlas/entities/377549c5-bab1-471f-9a2d-cb8281b67710 HTTP/1.1" 200 -
>"-" "Java/1.7.0_80"
>127.0.0.1 - - [29/Jan/2016:07:57:16 +] "OPTIONS
>/api/atlas/discovery/search?query=hive_db+where+name+%3D+'default'+and+clu
>sterName+%3D+'test'&user.name=jenkins HTTP/1.1" 200 - "-" "Java/1.7.0_80"
>127.0.0.1 - - [29/Jan/2016:07:57:16 +] "GET
>/api/atlas/discovery/search?query=hive_db+where+name+%3D+'default'+and+clu
>sterName+%3D+'test' HTTP/1.1" 200 - "-" "Java/1.7.0_80"
>127.0.0.1 - - [29/Jan/2016:07:57:21 +] "OPTIONS
>/api/atlas/discovery/search?query=hive_db+where+name+%3D+'default'+and+clu
>sterName+%3D+'test'&user.name=jenkins HTTP/1.1" 200 - "-" "Java/1.7.0_80"
>127.0.0.1 - - [29/Jan/2016:07:57:21 +] "GET
>/api/atlas/discovery/search?query=hive_db+where+name+%3D+'default'+and+clu
>sterName+%3D+'test' HTTP/1.1" 200 - "-" "Java/1.7.0_80"
>127.0.0.1 - - [29/Jan/2016:07:57:26 +] "OPTIONS
>/api/atlas/entities?user.name=jenkins HTTP/1.1" 200 - "-" "Java/1.7.0_80"
>127.0.0.1 - - [29/Jan/2016:07:57:27 +] "OPTIONS
>/api/atlas/discovery/search?query=g.V.has('__typeName',+'hive_process').ha
>s('hive_process.queryText',+%22create+view

Re: Update entity attribute which has an array type

[Shwetha Shivalingamurthy]

The API has a queryparam called Œproperty¹.
1. If the property is set, the request payload should contain the value of
the attribute to be updated.
2. If the property is not set, the request payload should contain the
entity json. The entity json can contain partial entity(with only the
attributes that need to be updated set). The attributes can be of type
primary/array/map/reference. For array, note that there is no way to pass
just the new element to be added to the array. It expects the complete
array. You can use AtlasClient.updateEntity(String guid, Referenceable
entity)

Yes, we still have to document the APIs in detail.

Regards,
Shwetha






On 27/01/16 1:11 am, "Herman Yu"  wrote:

>It seems that update to entity attribute with complex data types (e.g.
>array) is supported with v0.6 (ATLAS-47), where can I find the syntax of
>the REST api that does the update to an attribute with array type?
>
>PUT method on  /entities/{guid} still seems only take simple value.
>
>Thanks
>Herman.
>
>
>

Re: v0.6 on HDP 2.3.2 sandbox

[Shwetha Shivalingamurthy]

Hive-env.sh should contain
export 
HADOOP_CLASSPATH=/etc/atlas/conf:/usr/hdp/current/atlas-server/hook/hive:${
HADOOP_CLASSPATH}


So, /etc/atlas/conf/ should contain atlas’ application.properties


Regards,
Shwetha






On 21/01/16 7:46 pm, "Herman Yu"  wrote:

>Thanks Shwetha. that’s the reason, the hook is getting application and
>client configurations from jar files under hook/hive. I must be missing
>some configuration steps? is it supposed to get those two config files
>from Hive configuration folder?
>
>Thanks
>Herman.
>
>
>> On Jan 20, 2016, at 11:27 PM, Shwetha Shivalingamurthy
>> wrote:
>> 
>> Check if the atlas conf is picked up correctly. You can enable DEBUG
>>logs
>> for atlas by modifying the hive’s log4j.xml - add log level for package
>> org.apache.atlas. DEBUG log level will print the location of atlas’
>> application.properties that’s picked up
>> 
>> Regards,
>> Shwetha
>> 
>> 
>> 
>> 
>> 
>> 
>> On 21/01/16 12:41 am, "Herman Yu"  wrote:
>> 
>>> Hi Shewtha,
>>> 
>>> Thanks for the link.  I followed the link, but am still getting the
>>> following error in hiveserver2.log. It seems to me that atlasclient is
>>> refused when trying to connect to atlas server via http.  I already
>>>have
>>> atlas.http.authentication.enabled=false set to false in both
>>>application
>>> and client properties files.
>>> 
>>> 
>>> any other places I need to check?
>>> 
>>> thanks
>>> Herman.
>>> 
>>> 
>>> 2016-01-20 13:55:22,568 INFO  [Atlas Logger 3]: hook.HiveHook
>>> (HiveHook.java:fireAndForget(192)) - Entered Atlas hook for hook type
>>> POST_EXEC_HOOK operation CREATETABLE
>>> 2016-01-20 13:55:22,572 INFO  [HiveServer2-Background-Pool:
>>>Thread-191]:
>>> log.PerfLogger (PerfLogger.java:PerfLogEnd(148)) - >> method=releaseLocks start=1453316122568 end=1453316122572 duration=4
>>> from=org.apache.hadoop.hive.ql.Driver>
>>> 2016-01-20 13:55:22,573 INFO  [HiveServer2-Background-Pool:
>>>Thread-191]:
>>> log.PerfLogger (PerfLogger.java:PerfLogEnd(148)) - >> method=Driver.run start=1453316122342 end=1453316122573 duration=231
>>> from=org.apache.hadoop.hive.ql.Driver>
>>> 2016-01-20 13:55:22,634 INFO  [Atlas Logger 3]:
>>> security.SecureClientUtils
>>> (SecureClientUtils.java:getClientConnectionHandler(90)) - Real User:
>>>hive
>>> (auth:SIMPLE), is from ticket cache? false
>>> 2016-01-20 13:55:22,635 INFO  [Atlas Logger 3]:
>>> security.SecureClientUtils
>>> (SecureClientUtils.java:getClientConnectionHandler(93)) - doAsUser: hue
>>> 2016-01-20 13:55:22,635 INFO  [Atlas Logger 2]: hook.HiveHook
>>> (HiveHook.java:run(182)) - Atlas hook failed
>>> com.sun.jersey.api.client.ClientHandlerException: java.io.IOException:
>>> java.net.ConnectException: Connection refused
>>>   at 
>>> 
>>>com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(UR
>>>LC
>>> onnectionClientHandler.java:149)
>>>   at com.sun.jersey.api.client.Client.handle(Client.java:648)
>>>   at 
>>> com.sun.jersey.api.client.WebResource.handle(WebResource.java:670)
>>>   at 
>>> com.sun.jersey.api.client.WebResource.access$200(WebResource.java:74)
>>>   at 
>>> 
>>>com.sun.jersey.api.client.WebResource$Builder.method(WebResource.java:62
>>>3)
>>>   at 
>>> org.apache.atlas.AtlasClient.callAPIWithResource(AtlasClient.java:584)
>>>   at 
>>> org.apache.atlas.AtlasClient.callAPIWithResource(AtlasClient.java:579)
>>>   at org.apache.atlas.AtlasClient.getType(AtlasClient.java:257)
>>>   at 
>>> 
>>>org.apache.atlas.hive.bridge.HiveMetaStoreBridge.registerHiveDataModel(H
>>>iv
>>> eMetaStoreBridge.java:487)
>>>   at 
>>> org.apache.atlas.hive.hook.HiveHook.fireAndForget(HiveHook.java:197)
>>>   at 
>>> org.apache.atlas.hive.hook.HiveHook.access$200(HiveHook.java:66)
>>>   at org.apache.atlas.hive.hook.HiveHook$2.run(HiveHook.java:180)
>>>   at 
>>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
>>>   at java.util.concurrent.FutureTask.run(FutureTask.java:262)
>>>   at 
>>> 
>>>java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.jav
>>>a:
>&

Re: v0.6 on HDP 2.3.2 sandbox

[Shwetha Shivalingamurthy]

java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
>at java.net.Socket.connect(Socket.java:579)
>at sun.net.NetworkClient.doConnect(NetworkClient.java:175)
>at sun.net.www.http.HttpClient.openServer(HttpClient.java:432)
>at sun.net.www.http.HttpClient.openServer(HttpClient.java:527)
>at sun.net.www.http.HttpClient.(HttpClient.java:211)
>at sun.net.www.http.HttpClient.New(HttpClient.java:308)
>at sun.net.www.http.HttpClient.New(HttpClient.java:326)
>at 
>sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(HttpURLConnec
>tion.java:998)
>at 
>sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection
>.java:934)
>at 
>sun.net.www.protocol.http.HttpURLConnection.connect(HttpURLConnection.java
>:852)
>at 
>org.apache.hadoop.security.authentication.client.PseudoAuthenticator.authe
>nticate(PseudoAuthenticator.java:76)
>at 
>org.apache.hadoop.security.token.delegation.web.DelegationTokenAuthenticat
>or.authenticate(DelegationTokenAuthenticator.java:128)
>at 
>org.apache.hadoop.security.authentication.client.AuthenticatedURL.openConn
>ection(AuthenticatedURL.java:215)
>at 
>org.apache.hadoop.security.token.delegation.web.DelegationTokenAuthenticat
>edURL.openConnection(DelegationTokenAuthenticatedURL.java:322)
>at 
>org.apache.atlas.security.SecureClientUtils$1$1.run(SecureClientUtils.java
>:102)
>... 23 more
>
>
>> On Jan 20, 2016, at 2:35 AM, Shwetha Shivalingamurthy
>> wrote:
>> 
>> 
>>http://dev.hortonworks.com.s3.amazonaws.com/HDPDocuments/Ambari-2.2.0.0/b
>>k_
>> ambari_reference_guide/content/ch_integrating_atlas.html has the
>> instructions on setting up 0.6 Atlas with Ambari
>> 
>> 
>> Regards,
>> Shwetha
>> 
>> 
>> 
>> 
>> 
>> 
>> On 20/01/16 2:26 am, "Herman Yu"  wrote:
>> 
>>> Hi everyone,
>>> 
>>> I am having some problems with installing and configuring v0.6 on HDP
>>> 2.3.2 sandbox.
>>> 
>>> HDP 2.3.2 comes with v0.5 pre-configured with Ambari. After compiling
>>> v0.6 source code, and updated the link /usr/hdp/current/atlas-server to
>>> be associated with the new v0.6 folder. I also made necessary changes
>>>in
>>> amber¹s (e.g. those notification related parameters in
>>> application.properties). Atlas starts fine (even though with some
>>>errors
>>> in log file which are kafka related), however, Hive stopped working
>>>with
>>> the following errors with any operations.
>>> 
>>> I suspect this is the Hive hook related, after several tries, I figured
>>> out that if I update atlas.hook.hive.synchronous from true to false,
>>>Hive
>>> starts working fine however the hook doesn¹t capture any hive queries
>>>to
>>> atlas.
>>> 
>>> Did anyone experience the same problem? Also, how does hive know where
>>> the jar file of the hook is located? in HDP 2.3.2, I don¹t see
>>> HIVE_AUX_PATH is appended with /bridge/hive, I had to
>>> manually append this in Ambari.
>>> 
>>> 
>>> org.apache.hive.service.cli.HiveSQLException: Error while processing
>>> statement: FAILED: Hive Internal Error:
>>> com.sun.jersey.api.client.ClientHandlerException(java.io.IOException:
>>> java.net.ConnectException: Connection refused)
>>>   at 
>>> 
>>>org.apache.hive.service.cli.operation.Operation.toSQLException(Operation
>>>.j
>>> ava:315)
>>>   at 
>>> 
>>>org.apache.hive.service.cli.operation.SQLOperation.runQuery(SQLOperation
>>>.j
>>> ava:156)
>>>   at 
>>> 
>>>org.apache.hive.service.cli.operation.SQLOperation.runInternal(SQLOperat
>>>io
>>> n.java:183)
>>>   at 
>>> org.apache.hive.service.cli.operation.Operation.run(Operation.java:257)
>>>   at 
>>> 
>>>org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementInte
>>>rn
>>> al(HiveSessionImpl.java:388)
>>>   at 
>>> 
>>>org.apache.hive.service.cli.session.HiveSessionImpl.executeStatement(Hiv
>>>eS
>>> essionImpl.java:369)
>>>   at 
>>> 
>>>org.apache.hive.service.cli.CLIService.executeStatement(CLIService.java:
>>>26
>>> 1)
>>>   at 
>>> 
>>>org.apache.hive.service.cli.thrift.ThriftCLISer

Re: v0.6 on HDP 2.3.2 sandbox

[Shwetha Shivalingamurthy]

http://dev.hortonworks.com.s3.amazonaws.com/HDPDocuments/Ambari-2.2.0.0/bk_
ambari_reference_guide/content/ch_integrating_atlas.html has the
instructions on setting up 0.6 Atlas with Ambari


Regards,
Shwetha






On 20/01/16 2:26 am, "Herman Yu"  wrote:

>Hi everyone,
>
>I am having some problems with installing and configuring v0.6 on HDP
>2.3.2 sandbox.
>
>HDP 2.3.2 comes with v0.5 pre-configured with Ambari. After compiling
>v0.6 source code, and updated the link /usr/hdp/current/atlas-server to
>be associated with the new v0.6 folder. I also made necessary changes in
>amber¹s (e.g. those notification related parameters in
>application.properties). Atlas starts fine (even though with some errors
>in log file which are kafka related), however, Hive stopped working with
>the following errors with any operations.
>
>I suspect this is the Hive hook related, after several tries, I figured
>out that if I update atlas.hook.hive.synchronous from true to false, Hive
>starts working fine however the hook doesn¹t capture any hive queries to
>atlas.
>
>Did anyone experience the same problem? Also, how does hive know where
>the jar file of the hook is located? in HDP 2.3.2, I don¹t see
>HIVE_AUX_PATH is appended with /bridge/hive, I had to
>manually append this in Ambari.
>
>
>org.apache.hive.service.cli.HiveSQLException: Error while processing
>statement: FAILED: Hive Internal Error:
>com.sun.jersey.api.client.ClientHandlerException(java.io.IOException:
>java.net.ConnectException: Connection refused)
>at 
>org.apache.hive.service.cli.operation.Operation.toSQLException(Operation.j
>ava:315)
>at 
>org.apache.hive.service.cli.operation.SQLOperation.runQuery(SQLOperation.j
>ava:156)
>at 
>org.apache.hive.service.cli.operation.SQLOperation.runInternal(SQLOperatio
>n.java:183)
>at 
>org.apache.hive.service.cli.operation.Operation.run(Operation.java:257)
>at 
>org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementIntern
>al(HiveSessionImpl.java:388)
>at 
>org.apache.hive.service.cli.session.HiveSessionImpl.executeStatement(HiveS
>essionImpl.java:369)
>at 
>org.apache.hive.service.cli.CLIService.executeStatement(CLIService.java:26
>1)
>at 
>org.apache.hive.service.cli.thrift.ThriftCLIService.ExecuteStatement(Thrif
>tCLIService.java:486)
>at 
>org.apache.hive.service.cli.thrift.TCLIService$Processor$ExecuteStatement.
>getResult(TCLIService.java:1313)
>at 
>org.apache.hive.service.cli.thrift.TCLIService$Processor$ExecuteStatement.
>getResult(TCLIService.java:1298)
>at 
>org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
>at 
>org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
>at 
>org.apache.hive.service.auth.TSetIpAddressProcessor.process(TSetIpAddressP
>rocessor.java:56)
>at 
>org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolSe
>rver.java:285)
>at 
>java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:
>1145)
>at 
>java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java
>:615)
>at java.lang.Thread.run(Thread.java:745)
>Caused by: com.sun.jersey.api.client.ClientHandlerException:
>java.io.IOException: java.net.ConnectException: Connection refused
>at 
>com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLC
>onnectionClientHandler.java:149)
>at com.sun.jersey.api.client.Client.handle(Client.java:648)
>at 
>com.sun.jersey.api.client.WebResource.handle(WebResource.java:670)
>at 
>com.sun.jersey.api.client.WebResource.access$200(WebResource.java:74)
>at 
>com.sun.jersey.api.client.WebResource$Builder.method(WebResource.java:623)
>at 
>org.apache.atlas.AtlasClient.callAPIWithResource(AtlasClient.java:584)
>at 
>org.apache.atlas.AtlasClient.callAPIWithResource(AtlasClient.java:579)
>at org.apache.atlas.AtlasClient.getType(AtlasClient.java:257)
>at 
>org.apache.atlas.hive.bridge.HiveMetaStoreBridge.registerHiveDataModel(Hiv
>eMetaStoreBridge.java:487)
>at 
>org.apache.atlas.hive.hook.HiveHook.fireAndForget(HiveHook.java:197)
>at org.apache.atlas.hive.hook.HiveHook.run(HiveHook.java:174)
>at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1522)
>at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1195)
>at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1059)
>at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1054)
>at 
>org.apache.hive.service.cli.operation.SQLOperation.runQuery(SQLOperation.j
>ava:154)
>... 15 more
>
>
>Thanks
>Herman.
>

Re: Hive column level lineage

[Shwetha Shivalingamurthy]

https://issues.apache.org/jira/browse/ATLAS-247 - the one that you had
created


Regards,
Shwetha






On 18/01/16 6:57 pm, "Herman Yu"  wrote:

>Thanks Shwetha.  Is there a Jira item tracking this feature?  Herman.
>
>> On Jan 18, 2016, at 1:13 AM, Shwetha Shivalingamurthy
>> wrote:
>> 
>> Hi Herman,
>> 
>> We will need the query plan from hive to be able to implement hive
>>column
>> lineage. Once we have the query plan, we have to modify the hive model
>>to
>> implement column level lineage in Atlas. We are targetting this for the
>> next Atlas release.
>> 
>> Regards,
>> Shwetha
>> 
>> 
>> 
>> 
>> 
>> 
>> On 15/01/16 8:54 pm, "Herman Yu"  wrote:
>> 
>>> Hi Everyone,
>>> 
>>> I think the current lineage supporting with Hive is at Hive table
>>>level,
>>> is there a plan to enhance the hive bridge and extend the lineage
>>>support
>>> to Hive column level?
>>> 
>>> Hive column level lineage can be built with customized logic through
>>>API
>>> calls and with ³Process². However, the built-in hive_column entity is
>>>NOT
>>> inherited from Dataset, is there a specific reason?
>>> 
>>> Thanks
>>> Herman.
>>> 
>>> 
>>> 
>> 
>
>

Re: Hive column level lineage

[Shwetha Shivalingamurthy]

Hi Herman,

We will need the query plan from hive to be able to implement hive column
lineage. Once we have the query plan, we have to modify the hive model to
implement column level lineage in Atlas. We are targetting this for the
next Atlas release.

Regards,
Shwetha






On 15/01/16 8:54 pm, "Herman Yu"  wrote:

>Hi Everyone,
>
>I think the current lineage supporting with Hive is at Hive table level,
>is there a plan to enhance the hive bridge and extend the lineage support
>to Hive column level?
>
>Hive column level lineage can be built with customized logic through API
>calls and with ³Process². However, the built-in hive_column entity is NOT
>inherited from Dataset, is there a specific reason?
>
>Thanks
>Herman.
>
>
>

Re: [VOTE] Release Apache Atlas version 0.6-incubating

[Shwetha Shivalingamurthy]

+1

Regards,
Shwetha 

> On Dec 26, 2015, at 12:10 AM, Arun C Murthy  wrote:
> 
> +1 (binding)
> 
> Arun
> 
>> On Thursday, December 24, 2015, Suma Shivaprasad  wrote:
>> 
>> Hello folks,
>> 
>> This is a call for a vote on the Apache Atlas 0.6 incubating release.
>> 
>> A vote was held on developer mailing list and it passed with five +1's.
>> 
>> Vote thread: http://s.apache.org/nxl
>> Results thread: http://s.apache.org/HyG
>> 
>> The source tarball (*.tar.gz), signature (*.asc), checksum (*.md5, *.sha):
>> 
>>> *
>> https://dist.apache.org/repos/dist/dev/incubator/atlas/0.6.0-incubating-rc2/
>> <
>> https://dist.apache.org/repos/dist/dev/incubator/atlas/0.6.0-incubating-rc2/
>>> *
>> 
>> The SHA512 checksum of the archive is
>> 
>> CD7441A1 1790B2DA 52DA09D6 1ED4CF10 2B202D56 109DE7AE D86D1026 8EDE3A11
>> CF9AD7BF 9FAEDF55 E5175EB2 137A01B6 A311ECA1 EEA6EBFE 3D6CFA7F 23BE6CC8
>> The commit id (c8109a0177058d4a788d476a74d425a7593e7a9b) to be voted upon:
>> 
>> 
>> *
>> https://git-wip-us.apache.org/repos/asf?p=incubator-atlas.git;a=commit;h=c8109a0177058d4a788d476a74d425a7593e7a9b
>> <
>> https://git-wip-us.apache.org/repos/asf?p=incubator-atlas.git;a=commit;h=c8109a0177058d4a788d476a74d425a7593e7a9b
>>> *
>> The tag to be voted upon:
>> 
>>> *
>> https://git-wip-us.apache.org/repos/asf?p=incubator-atlas.git;a=tag;h=refs/tags/release-0.6-rc2
>> <
>> https://git-wip-us.apache.org/repos/asf?p=incubator-atlas.git;a=tag;h=refs/tags/release-0.6-rc2
>>> *
>> 
>> The list of fixed issues:
>> 
>>> *
>> https://git-wip-us.apache.org/repos/asf?p=incubator-atlas.git;a=blob;f=release-log.txt;h=c778ff0014b355b88ede8296a6c849a19f8202d3;hb=refs/heads/branch-0.6-incubating
>> <
>> https://git-wip-us.apache.org/repos/asf?p=incubator-atlas.git;a=blob;f=release-log.txt;h=c778ff0014b355b88ede8296a6c849a19f8202d3;hb=refs/heads/branch-0.6-incubating
>>> *
>> 
>> Keys to verify the signature of the release artifact are available at:
>> http://www.apache.org/dist/incubator/atlas/KEYS
>> PGP release keys:
>> http://pgp.mit.edu/pks/lookup?op=vindex&search=0x2DBD4D9B
>> 
>> Note that this is a source only release and we are voting on the source
>> (release-0.6-incubating-rc2 /
>> 
>> Commit:
>> c8109a0177058d4a788d476a74d425a7593e7a9b /
>> 
>> Checksums:
>> SHA512:
>> CD7441A1 1790B2DA 52DA09D6 1ED4CF10 2B202D56 109DE7AE D86D1026 8EDE3A11
>> CF9AD7BF 9FAEDF55 E5175EB2 137A01B6 A311ECA1 EEA6EBFE 3D6CFA7F 23BE6CC8
>> MD5 : 32 A1 13 D6 E8 7A 8E C9  34 1C D3 45 E8 4D 90 E1).
>> 
>> Vote will be open for atleast 72 hours ( atleast till 30th Dec 7:30 PM PST/
>> 10:30 PM EST /31st Dec 9 AM IST).
>> 
>> [ ] +1 approve
>> [ ] +0 no opinion
>> [ ] -1 disapprove (and reason why)
>> 
>> Thanks!
>> 
>> Regards,
>> Suma
>> 

Re: [VOTE] Release Apache Atlas version 0.6.0-incubating (Release Candidate 2)

[Shwetha Shivalingamurthy]

+1 - verified the build, signatures, hive hook, hive import and lineage in
UI

Regards,
Shwetha






On 21/12/15 4:25 pm, "Suma Shivaprasad"  wrote:

>Hi Folks,
>
>I have created a build for Apache Atlas 0.6-incubating, release candidate
>2.
>Thanks to everyone who have contributed to this release and also tested
>this.
>
>The source tarball (*.tar.gz), signature (*.asc), checksum (*.md5, *.sha):
>
>> 
>>*https://dist.apache.org/repos/dist/dev/incubator/atlas/0.6.0-incubating-
>>rc2/
>c2/>*
>
>The SHA512 checksum of the archive is
>
>CD7441A1 1790B2DA 52DA09D6 1ED4CF10 2B202D56 109DE7AE D86D1026 8EDE3A11
>CF9AD7BF 9FAEDF55 E5175EB2 137A01B6 A311ECA1 EEA6EBFE 3D6CFA7F 23BE6CC8
>The commit id (c8109a0177058d4a788d476a74d425a7593e7a9b) to be voted upon:
>
>>
>
>*https://git-wip-us.apache.org/repos/asf?p=incubator-atlas.git;a=commit;h=
>c8109a0177058d4a788d476a74d425a7593e7a9b
>c8109a0177058d4a788d476a74d425a7593e7a9b>*
>The tag to be voted upon:
>
>> 
>>*https://git-wip-us.apache.org/repos/asf?p=incubator-atlas.git;a=tag;h=re
>>fs/tags/release-0.6-rc2
>s/tags/release-0.6-rc2>*
>
>The list of fixed issues:
>
>> 
>>*https://git-wip-us.apache.org/repos/asf?p=incubator-atlas.git;a=blob;f=r
>>elease-log.txt;h=c778ff0014b355b88ede8296a6c849a19f8202d3;hb=refs/heads/b
>>ranch-0.6-incubating
>lease-log.txt;h=c778ff0014b355b88ede8296a6c849a19f8202d3;hb=refs/heads/bra
>nch-0.6-incubating>*
>
>Keys to verify the signature of the release artifact are available at:
>http://www.apache.org/dist/incubator/atlas/KEYS
>PGP release keys:
>http://pgp.mit.edu/pks/lookup?op=vindex&search=0x2DBD4D9B
>
>Note that this is a source only release and we are voting on the source
>(release-0.6-incubating-rc2 / Commit:
>c8109a0177058d4a788d476a74d425a7593e7a9b / SHA512:
>CD7441A1 1790B2DA 52DA09D6 1ED4CF10 2B202D56 109DE7AE D86D1026 8EDE3A11
>CF9AD7BF 9FAEDF55 E5175EB2 137A01B6 A311ECA1 EEA6EBFE 3D6CFA7F 23BE6CC8).
>
>Please download, test, and try it out.
>
>Vote will be open for at least 72 hours - (Dec 24 - 9 AM PST /  12 PM
>EST/ 10:30
>PM IST)
>
>[ ] +1 approve
>[ ] +0 no opinion
>[ ] -1 disapprove (and reason why)
>
>+1 from my side for the release.
>
>For folks not familiar with vetting a release, please refer to
>http://incubator.apache.org/guides/releasemanagement.html#check-list
>
>Regards,
>Suma

Re: [VOTE] Release Apache Atlas version 0.6.0-incubating (Release Candidate 1)

[Shwetha Shivalingamurthy]

Verified the signatures, build, end to end flow of hive hook, quick start
and UI

+1


Regards,
Shwetha






On 10/12/15 2:53 am, "Aaron.Dossett"  wrote:

>I am +1
>
>Downloaded distribution, built from source, started server, verified basic
>API calls with curl, ran quick_start and verified results.
>
>I could not run the hive bridge against a local, kerberized Hadoop
>cluster, but that may be a configuration issue on my side. It failed with
>kerberos authentication errors.  I don¹t think it¹s a blocker, but I am
>continuing to research.
>
>On 12/9/15, 5:35 AM, "Suma Shivaprasad" 
>wrote:
>
>>Hi Folks,
>>
>>I have created a build for Apache Atlas 0.6-incubating, release candidate
>>1. Thanks to everyone who have contributed to this release and also
>>tested
>>this.
>>
>>The source tarball (*.tar.gz), signature (*.asc), checksum (*.md5,
>>*.sha):
>>
>>> 
>>>*https://dist.apache.org/repos/dist/dev/incubator/atlas/0.6.0-incubating
>>>-
>>>rc1
>>>r
>>c1>*
>>
>>The SHA512 checksum of the archive is
>>
>>9CE1B3DD F9D18DCB FFA95815 A61CF8E3 8783DB40 28C3741D 7906B5FF A022A085
>>69552EAE 34B4299C 1160D43C DC196F6C B9EAE9E2 26AA7BF0 DA25EE79 D727A8DE
>>
>>The commit id (49167a349535b2d57d2e268ba5f1da52b6112d42) to be voted
>>upon:
>>
>>> 
>>>*https://git-wip-us.apache.org/repos/asf?p=incubator-atlas.git;a=commit;
>>>h
>>>=49167a349535b2d57d2e268ba5f1da52b6112d42
>>>=
>>49167a349535b2d57d2e268ba5f1da52b6112d42>*
>>
>>The tag to be voted upon:
>>
>>> 
>>>*https://git-wip-us.apache.org/repos/asf?p=incubator-atlas.git;a=tag;h=r
>>>e
>>>fs/tags/release-0.6-rc1
>>>f
>>s/tags/release-0.6-rc1>*
>>
>>The list of fixed issues:
>>
>>>
>>https://git-wip-us.apache.org/repos/asf?p=incubator-atlas.git;a=blob;f=re
>>l
>>ease-log.txt;h=9d2c7a29b9c69e67b8e4a1b4a2641520924b2692;hb=49167a349535b2
>>d
>>57d2e268ba5f1da52b6112d42
>>
>>Keys to verify the signature of the release artifact are available at:
>>http://www.apache.org/dist/incubator/atlas/KEYS
>>PGP release keys:
>>http://pgp.mit.edu/pks/lookup?op=vindex&search=0x2DBD4D9B
>>
>>Note that this is a source only release and we are voting on the source
>>(release-0.6-incubating-rc1 / Commit:
>>49167a349535b2d57d2e268ba5f1da52b6112d42 / SHA512:
>>9CE1B3DD F9D18DCB FFA95815 A61CF8E3 8783DB40 28C3741D 7906B5FF A022A085
>>69552EAE 34B4299C 1160D43C DC196F6C B9EAE9E2 26AA7BF0 DA25EE79 D727A8DE).
>>
>>Please download, test, and try it out.
>>
>>Vote will be open for at least 72 hours - (Dec 14 - 9 AM PST /  12 PM
>>EST/ 10:30
>>PM IST)  (until the required number of IPMC votes are obtained).
>>
>>[ ] +1 approve
>>[ ] +0 no opinion
>>[ ] -1 disapprove (and reason why)
>>
>>+1 from my side for the release.
>>
>>For folks not familiar with vetting a release, please refer to
>>http://incubator.apache.org/guides/releasemanagement.html#check-list
>>
>>Regards,
>>Suma
>
>

Atlas 0.6-incubating release

[Shwetha Shivalingamurthy]

Hi All,

We had 0.5-incubating release around June, 2015 and we have added the following 
since then:

  1.  Integration with kafka for reliable hook notifications(from external 
component bridge) and entity/trait notifications(from atlas)
  2.  HBase and Solr integration for titan
  3.  Fix search issues
  4.  UI features - option to tag entities, clickable lineage
  5.  And many other bug fixes

So, we are planning to start a 0.6-incubating release of Atlas early next week 
and this release will be from trunk. We will wait for the following jiras:
1. https://issues.apache.org/jira/browse/ATLAS-47
2. https://issues.apache.org/jira/browse/ATLAS-171

If you want any un-committed jiras to be part of this release, let us know

Regards,
Shwetha

Re: Build Failing

[Shwetha Shivalingamurthy]

Doesn¹t Œmvn clean¹ on the whole project help this time as well? Else, you
can try fresh checkout

Regards,
Shwetha






On 05/11/15 3:28 pm, "Rishabh Bhardwaj"  wrote:

>Hi again,
>
>The build was working properly for me but suddenly it starts throwing this
>error now,
>
>[ERROR] Failed to execute goal
>com.github.eirslett:frontend-maven-plugin:0.0.23:install-node-and-npm
>(install node and npm) on project atlas-dashboard: Could not extract the
>Node archive: Could not extract archive:
>'/Users/admin/Documents/office/atlas/myRepo/incubator-atlas/dashboard/targ
>et/node_tmp/node.tar.gz':
>EOFException -> [Help 1]
>
>Can someone help? I have tried building many times but every time getting
>the same error ?
>
>On Tue, Nov 3, 2015 at 9:10 AM, Rishabh Bhardwaj 
>wrote:
>
>> Thanks Shwetha.It works.
>> I was doing
>> 1. 'mvn -DskipTests=true clean install'  (Failing)
>> But,
>> 1. 'mvn clean'
>> 2. 'mvn -DskipTests=true Install'
>> This works.
>>
>> I guess this is some intermittent error.
>>
>> On Mon, Nov 2, 2015 at 7:40 PM, Shwetha Shivalingamurthy <
>> sshivalingamur...@hortonworks.com> wrote:
>>
>>> Doesn't 'mvn clean' help?
>>>
>>> Regards,
>>> Shwetha
>>>
>>>
>>>
>>>
>>>
>>>
>>> On 02/11/15 6:01 pm, "Rishabh Bhardwaj"  wrote:
>>>
>>> >Hi,
>>> >My build fails with this error,
>>> >[ERROR] Failed to execute goal
>>> >com.github.eirslett:frontend-maven-plugin:0.0.23:install-node-and-npm
>>> >(install node and npm) on project atlas-dashboard: Could not extract
>>>the
>>> >Node archive: Could not extract archive:
>>>
>>> 
>>>>'/Users/admin/Documents/office/atlas/myRepo/incubator-atlas/dashboard/t
>>>>arg
>>> >et/node_tmp/node.tar.gz':
>>> >EOFException -> [Help 1]
>>> >
>>> >Can anyone suggest workaroud for this ?
>>> >
>>> >Thanks,
>>> >Rishabh.
>>>
>>>
>>

Re: Build Failing

[Shwetha Shivalingamurthy]

Doesn't 'mvn clean' help?

Regards,
Shwetha






On 02/11/15 6:01 pm, "Rishabh Bhardwaj"  wrote:

>Hi,
>My build fails with this error,
>[ERROR] Failed to execute goal
>com.github.eirslett:frontend-maven-plugin:0.0.23:install-node-and-npm
>(install node and npm) on project atlas-dashboard: Could not extract the
>Node archive: Could not extract archive:
>'/Users/admin/Documents/office/atlas/myRepo/incubator-atlas/dashboard/targ
>et/node_tmp/node.tar.gz':
>EOFException -> [Help 1]
>
>Can anyone suggest workaroud for this ?
>
>Thanks,
>Rishabh.

Re: Search syntax

[Shwetha Shivalingamurthy]

The new hive hook doesn’t call any atlas APIs. Looks like you have old
atlas libraries in the hive hook, replace them with the new ones from
atlas package

Regards,
Shwetha




On 01/10/15 3:00 am, "Herman Yu"  wrote:

>I managed deployed v0.6, however, the hive hook doesn’t work. see error
>exceptions below when queries Hive. are there any hive hook related
>configuration changes needed with v0.6? I went through the document but
>didn’t find any.
>
>
>I also see many lines like this in application.log, not sure if they are
>related. 
>
>
>2015-09-30 17:26:01,276 INFO  - [qtp491264922-13 -
>2100abbb-10c8-461f-8fff-be0264c16a8f:] ~ Couldn't find JAX-B element for
>class javax.ws.rs.core.Response (WadlGeneratorJAXBGrammarGenerator:508)
>
>Thanks
>Herman.
>
>---
>
>
>hive> select * from sample_07;
>FAILED: Hive Internal Error:
>org.apache.atlas.AtlasServiceException(org.codehaus.jettison.json.JSONExce
>ption: JSONObject["results"] is not a JSONObject.)
>org.apache.atlas.AtlasServiceException:
>org.codehaus.jettison.json.JSONException: JSONObject["results"] is not a
>JSONObject.
>   at org.apache.atlas.AtlasClient.searchByDSL(AtlasClient.java:280)
>   at 
>org.apache.atlas.hive.bridge.HiveMetaStoreBridge.getEntityReferenceFromDSL
>(HiveMetaStoreBridge.java:188)
>   at 
>org.apache.atlas.hive.bridge.HiveMetaStoreBridge.getDatabaseReference(Hive
>MetaStoreBridge.java:169)
>   at 
>org.apache.atlas.hive.bridge.HiveMetaStoreBridge.registerDatabase(HiveMeta
>StoreBridge.java:103)
>   at 
>org.apache.atlas.hive.bridge.HiveMetaStoreBridge.registerTable(HiveMetaSto
>reBridge.java:270)
>   at 
> org.apache.atlas.hive.hook.HiveHook.registerProcess(HiveHook.java:300)
>   at org.apache.atlas.hive.hook.HiveHook.fireAndForget(HiveHook.java:193)
>   at org.apache.atlas.hive.hook.HiveHook.run(HiveHook.java:152)
>   at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1520)
>   at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1195)
>   at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1059)
>   at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1049)
>   at 
>org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:213)
>   at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:165)
>   at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:376)
>   at 
> org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:736)
>   at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:681)
>   at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:621)
>   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
>sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:
>57)
>   at 
>sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorIm
>pl.java:43)
>   at java.lang.reflect.Method.invoke(Method.java:606)
>   at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
>   at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
>Caused by: org.codehaus.jettison.json.JSONException:
>JSONObject["results"] is not a JSONObject.
>   at 
>org.codehaus.jettison.json.JSONObject.getJSONObject(JSONObject.java:458)
>   at org.apache.atlas.AtlasClient.searchByDSL(AtlasClient.java:278)
>   ... 23 more
>
>
>On Sep 30, 2015, at 12:04 PM, Herman Yu  wrote:
>
>> yes, that’s what I did since I don’t have Kerberos enabled on local
>>environment and one test failed. The built completed successfully, I am
>>in the process deploying v0.6.
>> 
>> Thanks for all the help.
>> Herman. 
>> 
>> 
>> On Sep 30, 2015, at 11:59 AM, Shwetha Shivalingamurthy
>> wrote:
>> 
>>> You can run with -DskipTests as well
>>> 
>>> Regards,
>>> Shwetha
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> On 30/09/15 11:15 am, "Shwetha Shivalingamurthy"
>>>  wrote:
>>> 
>>>> I took the trunk, applied ATLAS-128 and the build went through fine.
>>>> Delete atlas/target and try again. If the tests fail still, send the
>>>> application.log.
>>>> 
>>>> Meanwhile, I have modified the nightly build to build the distro.
>>>> https://builds.apache.org/job/apache-atlas-nightly/ #81 onwards should
>>>> have the tarball generated
>>>> 
>>>> Regards,
>>>> Shwetha
>>>> 
>>>> 
>>>> 
>>>> 
>>>> On 30/09/15 1:54 am, "Her

Re: Search syntax

[Shwetha Shivalingamurthy]

You can run with -DskipTests as well

Regards,
Shwetha






On 30/09/15 11:15 am, "Shwetha Shivalingamurthy"
 wrote:

>I took the trunk, applied ATLAS-128 and the build went through fine.
>Delete atlas/target and try again. If the tests fail still, send the
>application.log.
>
>Meanwhile, I have modified the nightly build to build the distro.
>https://builds.apache.org/job/apache-atlas-nightly/ #81 onwards should
>have the tarball generated
>
>Regards,
>Shwetha
>
>
>
>
>On 30/09/15 1:54 am, "Herman Yu"  wrote:
>
>>thanks Shwetha,
>>
>>I applied ATLAS-128 and the moved to the step building web app. After
>>figuring out the Node JS step, it failed at the test step, with following
>>errors.
>>
>>Thanks for all the help so far, are we seeing the lights at the end of
>>the tunnel?
>>
>>Thanks
>>Herman.
>>
>>127.0.0.1 - - [29/Sep/2015:20:19:52 +] "GET
>>/api/atlas/lineage/hive/table/schema HTTP/1.1" 404 - "-" "Java/1.7.0_85"
>>127.0.0.1 - - [29/Sep/2015:20:19:52 +] "GET
>>/api/atlas/lineage/hive/table/blah/schema HTTP/1.1" 404 - "-"
>>"Java/1.7.0_85"
>>Tests run: 5, Failures: 2, Errors: 0, Skipped: 0, Time elapsed: 9.985 sec
>><<< FAILURE! - in
>>org.apache.atlas.web.resources.HiveLineageJerseyResourceIT
>>testInputsGraph(org.apache.atlas.web.resources.HiveLineageJerseyResourceI
>>T
>>)  Time elapsed: 1.913 sec  <<< FAILURE!
>>java.lang.AssertionError: expected:<4> but was:<0>
>>  at org.testng.Assert.fail(Assert.java:89)
>>  at org.testng.Assert.failNotEquals(Assert.java:489)
>>  at org.testng.Assert.assertEquals(Assert.java:118)
>>  at org.testng.Assert.assertEquals(Assert.java:365)
>>  at org.testng.Assert.assertEquals(Assert.java:375)
>>  at 
>>org.apache.atlas.web.resources.HiveLineageJerseyResourceIT.testInputsGrap
>>h
>>(HiveLineageJerseyResourceIT.java:77)
>>
>>testOutputsGraph(org.apache.atlas.web.resources.HiveLineageJerseyResource
>>I
>>T)  Time elapsed: 1.757 sec  <<< FAILURE!
>>java.lang.AssertionError: expected:<3> but was:<0>
>>  at org.testng.Assert.fail(Assert.java:89)
>>  at org.testng.Assert.failNotEquals(Assert.java:489)
>>  at org.testng.Assert.assertEquals(Assert.java:118)
>>  at org.testng.Assert.assertEquals(Assert.java:365)
>>  at org.testng.Assert.assertEquals(Assert.java:375)
>>  at 
>>org.apache.atlas.web.resources.HiveLineageJerseyResourceIT.testOutputsGra
>>p
>>h(HiveLineageJerseyResourceIT.java:105)
>>
>>Running org.apache.atlas.web.resources.AdminJerseyResourceIT
>>127.0.0.1 - - [29/Sep/2015:20:19:54 +] "GET /api/atlas/admin/version
>>HTTP/1.1" 200 - "-" "Java/1.7.0_85"
>>Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.67 sec
>>- in org.apache.atlas.web.resources.AdminJerseyResourceIT
>>Running org.apache.atlas.web.resources.TypesJerseyResourceIT
>>127.0.0.1 - - [29/Sep/2015:20:19:56 +] "POST /api/atlas/types
>>HTTP/1.1" 409 - "-" "Java/1.7.0_85"
>>127.0.0.1 - - [29/Sep/2015:20:19:56 +] "POST /api/atlas/types
>>HTTP/1.1" 409 - "-" "Java/1.7.0_85"
>>127.0.0.1 - - [29/Sep/2015:20:19:57 +] "GET /api/atlas/types/blah
>>HTTP/1.1" 404 - "-" "Java/1.7.0_85"
>>Tests run: 5, Failures: 2, Errors: 0, Skipped: 2, Time elapsed: 1.985 sec
>><<< FAILURE! - in org.apache.atlas.web.resources.TypesJerseyResourceIT
>>testSubmit(org.apache.atlas.web.resources.TypesJerseyResourceIT)  Time
>>elapsed: 0.333 sec  <<< FAILURE!
>>java.lang.AssertionError: expected:<201> but was:<409>
>>  at org.testng.Assert.fail(Assert.java:89)
>>  at org.testng.Assert.failNotEquals(Assert.java:489)
>>  at org.testng.Assert.assertEquals(Assert.java:118)
>>  at org.testng.Assert.assertEquals(Assert.java:365)
>>  at org.testng.Assert.assertEquals(Assert.java:375)
>>  at 
>>org.apache.atlas.web.resources.TypesJerseyResourceIT.testSubmit(TypesJers
>>e
>>yResourceIT.java:77)
>>
>>testGetTraitNames(org.apache.atlas.web.resources.TypesJerseyResourceIT)
>>Time elapsed: 0.01 sec  <<< FAILURE!
>>java.lang.AssertionError: expected:<201> but was:<409>
>>  at org.testng.Assert.fail(Assert.java:89)
>>  at org.testng.Assert.failNotEquals(Assert.java:489)
>>  at org.testng.Assert.assertEqua

Re: Search syntax

[Shwetha Shivalingamurthy]

27;dbgWqD7gYjSc'
>HTTP/1.1" 200 - "-" "Java/1.7.0_85"
>127.0.0.1 - - [29/Sep/2015:20:20:13 +] "OPTIONS
>/api/atlas/entities?user.name=root HTTP/1.1" 200 964 "-" "Java/1.7.0_85"
>127.0.0.1 - - [29/Sep/2015:20:20:13 +] "OPTIONS
>/api/atlas/discovery/search/dsl?query=hive_table+where+name%3D'pctLrmHZoK'
>&user.name=root HTTP/1.1" 200 773 "-" "Java/1.7.0_85"
>127.0.0.1 - - [29/Sep/2015:20:20:13 +] "POST /api/atlas/entities
>HTTP/1.1" 201 - "-" "Java/1.7.0_85"
>127.0.0.1 - - [29/Sep/2015:20:20:13 +] "GET
>/api/atlas/discovery/search/dsl?query=hive_table+where+name%3D'pctLrmHZoK'
> HTTP/1.1" 200 - "-" "Java/1.7.0_85"
>127.0.0.1 - - [29/Sep/2015:20:20:13 +] "OPTIONS
>/api/atlas/discovery/search/dsl?query=hive_db+where+name%3D'dbgWqD7gYjSc'&
>user.name=root HTTP/1.1" 200 773 "-" "Java/1.7.0_85"
>127.0.0.1 - - [29/Sep/2015:20:20:13 +] "GET
>/api/atlas/discovery/search/dsl?query=hive_db+where+name%3D'dbgWqD7gYjSc'
>HTTP/1.1" 200 - "-" "Java/1.7.0_85"
>Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 4.43 sec
>- in org.apache.atlas.notification.NotificationHookConsumerIT
>
>Results :
>
>Failed tests: 
>  
>EntityJerseyResourceIT.testUTF8:492->BaseResourceIT.createType:87->BaseRes
>ourceIT.createType:96 expected:<201> but was:<409>
>  HiveLineageJerseyResourceIT.testInputsGraph:77 expected:<4> but was:<0>
>  HiveLineageJerseyResourceIT.testOutputsGraph:105 expected:<3> but
>was:<0>
>  
>TypesJerseyResourceIT.testGetTraitNames:155->addTraits:182->BaseResourceIT
>.createType:96 expected:<201> but was:<409>
>  TypesJerseyResourceIT.testSubmit:77 expected:<201> but was:<409>
>
>Tests run: 51, Failures: 5, Errors: 0, Skipped: 2
>
>[INFO] 
>[INFO] --- jetty-maven-plugin:9.2.12.v20150709:stop (stop-jetty) @
>atlas-webapp ---
>[INFO] 
>[INFO] --- maven-failsafe-plugin:2.18.1:verify (verify) @ atlas-webapp ---
>[INFO] Stopped ServerConnector@79f0e2c0{HTTP/1.1}{0.0.0.0:21000}
>[INFO] Failsafe report directory:
>/teeup/source_repo/atlas/webapp/target/failsafe-reports
>[INFO] 
>
>[INFO] Reactor Summary:
>[INFO] 
>[INFO] Apache Atlas Web Application ... FAILURE
>[03:18 min]
>[INFO] Apache Atlas Documentation . SKIPPED
>[INFO] Apache Atlas Hive Bridge ... SKIPPED
>[INFO] Apache Atlas Distribution .. SKIPPED
>[INFO] 
>
>[INFO] BUILD FAILURE
>[INFO] 
>
>[INFO] Total time: 03:19 min
>[INFO] Finished at: 2015-09-29T20:20:14+00:00
>[INFO] Final Memory: 105M/531M
>[INFO] 
>
>[ERROR] Failed to execute goal
>org.apache.maven.plugins:maven-failsafe-plugin:2.18.1:verify (verify) on
>project atlas-webapp: There are test failures.
>
>
>On Sep 29, 2015, at 6:28 AM, Shwetha Shivalingamurthy
> wrote:
>
>> ATLAS-180 is already committed. So, you can take the latest code and
>>apply
>> ATLAS-128.
>> 
>> Regards,
>> Shwetha
>> 
>> 
>> 
>> 
>> 
>> 
>> On 28/09/15 9:17 pm, "Herman Yu"  wrote:
>> 
>>> Before getting to detail error messages I got. I have a metadata model
>>> implemented and want to test all the search syntax/patterns, as well as
>>> REST API creating/update entities. Prefer to test with v-0.6. Is there
>>>a
>>> binary tar ball of v-0.6 available for download? I think there was a
>>>Jira
>>> asking for binary tall ball and the last update is to provide this.
>>> 
>>> 
>>> I downloaded and applied ATLAS-128.patch.  When tried to apply patch
>>> 180.v2, got the following error message:
>>> 
>>> atlas]# git apply --check ../ATLAS-180-v2.patch
>>> error: patch failed: docs/pom.xml:32
>>> error: docs/pom.xml: patch does not apply
>>> error: patch failed: docs/src/site/site.xml:77
>>> error: docs/src/site/site.xml: patch does not apply
>>> error: patch failed: docs/src/site/twiki/Architecture.twiki:14
>>> error: docs/src/site/twiki/Architecture.twiki: patch does not apply
>>> error: patch failed: docs/src/site/twiki/Bridg

Re: Search syntax

[Shwetha Shivalingamurthy]

ATLAS-180 is already committed. So, you can take the latest code and apply
ATLAS-128.

Regards,
Shwetha






On 28/09/15 9:17 pm, "Herman Yu"  wrote:

>Before getting to detail error messages I got. I have a metadata model
>implemented and want to test all the search syntax/patterns, as well as
>REST API creating/update entities. Prefer to test with v-0.6. Is there a
>binary tar ball of v-0.6 available for download? I think there was a Jira
>asking for binary tall ball and the last update is to provide this.
>
>
>I downloaded and applied ATLAS-128.patch.  When tried to apply patch
>180.v2, got the following error message:
>
>atlas]# git apply --check ../ATLAS-180-v2.patch
>error: patch failed: docs/pom.xml:32
>error: docs/pom.xml: patch does not apply
>error: patch failed: docs/src/site/site.xml:77
>error: docs/src/site/site.xml: patch does not apply
>error: patch failed: docs/src/site/twiki/Architecture.twiki:14
>error: docs/src/site/twiki/Architecture.twiki: patch does not apply
>error: patch failed: docs/src/site/twiki/Bridge-Hive.twiki:22
>error: docs/src/site/twiki/Bridge-Hive.twiki: patch does not apply
>error: patch failed: docs/src/site/twiki/index.twiki:11
>error: docs/src/site/twiki/index.twiki: patch does not apply
>error: patch failed: pom.xml:24
>error: pom.xml: patch does not apply
>error: patch failed: webapp/pom.xml:54
>error: webapp/pom.xml: patch does not apply
>
>I then manually checked atlas/pom.xml, line 1332 and 1333 already have
>1.7 and 1.7, so I assume this patch had
>been applied since I retrieved from git repository this morning.
>
>Went ahead and kick off the built, this time, error messages are shown
>below. I am using JDK 1.8.0_40-b25.
>
>[INFO] --- scala-maven-plugin:3.2.0:compile (scala-compile-first) @
>atlas-typesystem ---
>[WARNING] Zinc server is not available at port 3030 - reverting to normal
>incremental compile
>[INFO] Using incremental compilation
>[INFO] Compiling 6 Scala sources and 45 Java sources to
>/teeup/source_repo/atlas/typesystem/target/classes...
>[ERROR] javac: invalid target release: 1.7
>[ERROR] Usage: javac  
>[ERROR] use -help for a list of possible options
>[INFO] 
>
>[INFO] Reactor Summary:
>[INFO] 
>[INFO] apache-atlas ... SUCCESS
>[01:27 min]
>[INFO] Apache Atlas Typesystem  FAILURE
>[03:33 min]
>[INFO] Apache Atlas Common  SKIPPED
>[INFO] Apache Atlas Client  SKIPPED
>[INFO] Apache Atlas Notification .. SKIPPED
>[INFO] Apache Atlas Repository  SKIPPED
>[INFO] Apache Atlas Web Application ... SKIPPED
>[INFO] Apache Atlas Documentation . SKIPPED
>[INFO] Apache Atlas Hive Bridge ... SKIPPED
>[INFO] Apache Atlas Distribution .. SKIPPED
>[INFO] 
>
>[INFO] BUILD FAILURE
>[INFO] 
>
>[INFO] Total time: 05:16 min
>[INFO] Finished at: 2015-09-28T11:33:57-04:00
>[INFO] Final Memory: 43M/219M
>[INFO] 
>
>[ERROR] Failed to execute goal
>net.alchim31.maven:scala-maven-plugin:3.2.0:compile (scala-compile-first)
>on project atlas-typesystem: Execution scala-compile-first of goal
>net.alchim31.maven:scala-maven-plugin:3.2.0:compile failed. CompileFailed
>-> [Help 1]
>[ERROR] 
>[ERROR] To see the full stack trace of the errors, re-run Maven with the
>-e switch.
>[ERROR] Re-run Maven using the -X switch to enable full debug logging.
>[ERROR] 
>[ERROR] For more information about the errors and possible solutions,
>please read the following articles:
>[ERROR] [Help 1] 
>http://cwiki.apache.org/confluence/display/MAVEN/PluginExecutionException
>[ERROR] 
>[ERROR] After correcting the problems, you can resume the build with the
>command
>[ERROR]   mvn  -rf :atlas-typesystem
>
>
>Thanks
>Herman.
>
>On Sep 28, 2015, at 8:58 AM, Herman Yu  wrote:
>
>> Thanks Suma and Shwetha,  can I manually apply patches from 180, and
>>then 128, and try to build the package now?
>> 
>> Thanks
>> Herman.
>> 
>> 
>> 
>> On Sep 28, 2015, at 1:33 AM, Suma Shivaprasad
>> wrote:
>> 
>>> Also pls watch for https://issues.apache.org/jira/browse/ATLAS-128 to
>>>be
>>> committed before you try this.
>>> 
>>> Thanks
>>> Suma
>

Re: Search syntax

[Shwetha Shivalingamurthy]

Add following configuration to scala-maven-plugin. We will fix it as part
of https://issues.apache.org/jira/browse/ATLAS-180. Thanks

1.7
1.7



Regards,
Shwetha






On 24/09/15 10:22 pm, "Herman Yu"  wrote:

>I had open JDK 1.6 (java version “1.6.0_36”, OpenJDK Runtime Environment
>(IcedTea6 1.13.8) (rhel-1.13.8.1.el6_7-x86_64),OpenJDK 64-Bit Server VM
>(build 23.25-b01, mixed mode)))
>
>I then switched to JDK 1.8 ( java version “1.8.0_40”,Java(TM) SE Runtime
>Environment (build 1.8.0_40-b25) Java HotSpot(TM) 64-Bit Server VM (build
>25.40-b25, mixed mode)) and reran “mvm clean install” , but got the same
>error messages. 
>
>Thanks
>Herman.
>
>
>
>On Sep 24, 2015, at 12:10 PM, Shwetha Shivalingamurthy
> wrote:
>
>> Which java version are you using?
>> 
>> Regards,
>> Shwetha
>> 
>> 
>> 
>> 
>> 
>> 
>> On 24/09/15 9:36 pm, "Herman Yu"  wrote:
>> 
>>> Thanks Suma and Shwetha.
>>> 
>>> Followed instruction here:
>>> http://atlas.incubator.apache.org/InstallationSteps.html, I tried to
>>> built v0.6 version to test the search syntax.The built is failed. Error
>>> message attached.
>>> 
>>> Is there a binary version of v0.6 for download?
>>> 
>>> Is this mailing list the right place for this type of questions/help?
>>>Or
>>> should I create a Jira ?
>>> 
>>> Thanks
>>> Herman.
>>> 
>>> 
>>> [INFO] --- scala-maven-plugin:3.2.0:compile (scala-compile-first) @
>>> atlas-typesystem ---
>>> [WARNING] Zinc server is not available at port 3030 - reverting to
>>>normal
>>> incremental compile
>>> [INFO] Using incremental compilation
>>> [INFO] Compiling 6 Scala sources and 45 Java sources to
>>> /teeup/source_repo/atlas/typesystem/target/classes...
>>> [ERROR] 
>>> 
>>>/teeup/source_repo/atlas/typesystem/src/main/java/org/apache/atlas/types
>>>ys
>>> tem/Struct.java:33: illegal start of type
>>> [ERROR] values = new HashMap<>();
>>> [ERROR]  ^
>>> [ERROR] 
>>> 
>>>/teeup/source_repo/atlas/typesystem/src/main/java/org/apache/atlas/types
>>>ys
>>> tem/persistence/DownCastStructInstance.java:62: illegal start of type
>>> [ERROR] Map m = new HashMap<>();
>>> [ERROR] ^
>>> [ERROR] 
>>> 
>>>/teeup/source_repo/atlas/typesystem/src/main/java/org/apache/atlas/types
>>>ys
>>> tem/persistence/StructInstance.java:241: illegal start of type
>>> [ERROR] Map m = new HashMap<>();
>>> [ERROR] ^
>>> [ERROR] 
>>> 
>>>/teeup/source_repo/atlas/typesystem/src/main/java/org/apache/atlas/types
>>>ys
>>> tem/types/HierarchicalType.java:153: illegal start of type
>>> [ERROR] Map attributeNameToType = new
>>>HashMap<>();
>>> [ERROR]   ^
>>> [ERROR] 
>>> 
>>>/teeup/source_repo/atlas/typesystem/src/main/java/org/apache/atlas/types
>>>ys
>>> tem/types/ObjectGraphWalker.java:58: illegal start of type
>>> [ERROR] queue = new LinkedList<>();
>>> [ERROR]^
>>> [ERROR] 
>>> 
>>>/teeup/source_repo/atlas/typesystem/src/main/java/org/apache/atlas/types
>>>ys
>>> tem/types/ObjectGraphWalker.java:59: illegal start of type
>>> [ERROR] processedIds = new HashSet<>();
>>> [ERROR]^
>>> [ERROR] 
>>> 
>>>/teeup/source_repo/atlas/typesystem/src/main/java/org/apache/atlas/types
>>>ys
>>> tem/types/TypeSystem.java:85: illegal start of type
>>> [ERROR] types = new ConcurrentHashMap<>();
>>> [ERROR]   ^
>>> [ERROR] 
>>> 
>>>/teeup/source_repo/atlas/typesystem/src/main/java/org/apache/atlas/types
>>>ys
>>> tem/types/TypeSystem.java:98: illegal start of type
>>> [ERROR] List typeNames = new
>>>ArrayList<>(types.keySet());
>>> [ERROR]^
>>> [ERROR] 
>>> 
>>>/teeup/source_repo/atlas/typesystem/src/main/java/org/apache/atlas/types
>>>ys
>>> tem/types/TypeSystem.java:306: illegal start of type
>>> [ERROR] 

Re: Search syntax

[Shwetha Shivalingamurthy]

tlas/typesystem/src/main/java/org/apache/atlas/typesys
>tem/types/TypeSystem.java:329: illegal start of type
>[ERROR] recursiveArrayTypes = new ArrayList<>();
>[ERROR] ^
>[ERROR] 
>/teeup/source_repo/atlas/typesystem/src/main/java/org/apache/atlas/typesys
>tem/types/TypeSystem.java:330: illegal start of type
>[ERROR] recursiveMapTypes = new ArrayList<>();
>[ERROR]   ^
>[ERROR] 
>/teeup/source_repo/atlas/typesystem/src/main/java/org/apache/atlas/typesys
>tem/types/TypeSystem.java:331: illegal start of type
>[ERROR] transientTypes = new LinkedHashSet<>();
>[ERROR]^
>[ERROR] 
>/teeup/source_repo/atlas/typesystem/src/main/java/org/apache/atlas/typesys
>tem/types/TypeSystem.java:394: illegal start of type
>[ERROR] Set s = new HashSet<>();
>[ERROR] ^
>[ERROR] 
>/teeup/source_repo/atlas/typesystem/src/main/java/org/apache/atlas/typesys
>tem/types/TypeSystem.java:505: illegal start of type
>[ERROR] List traitTypes = new ArrayList<>();
>[ERROR]^
>[ERROR] 
>/teeup/source_repo/atlas/typesystem/src/main/java/org/apache/atlas/typesys
>tem/types/TypeSystem.java:511: illegal start of type
>[ERROR] List classTypes = new ArrayList<>();
>[ERROR]^
>[ERROR] 
>/teeup/source_repo/atlas/typesystem/src/main/java/org/apache/atlas/typesys
>tem/types/TypeSystem.java:564: illegal start of type
>[ERROR] Map newTypes = new HashMap<>();
>[ERROR]   ^
>[ERROR] 
>/teeup/source_repo/atlas/typesystem/src/main/java/org/apache/atlas/typesys
>tem/types/utils/TypesUtil.java:66: illegal start of type
>[ERROR] return new HierarchicalTypeDefinition<>(TraitType.class,
>name, superTypes, attrDefs);
>[ERROR]   ^
>[ERROR] 
>/teeup/source_repo/atlas/typesystem/src/main/java/org/apache/atlas/typesys
>tem/types/utils/TypesUtil.java:75: illegal start of type
>[ERROR] return new HierarchicalTypeDefinition<>(ClassType.class,
>name, superTypes, attrDefs);
>[ERROR]   ^
>[ERROR] 24 errors
>[INFO] 
>
>[INFO] Reactor Summary:
>[INFO] 
>[INFO] apache-atlas ... SUCCESS
>[03:00 min]
>[INFO] Apache Atlas Typesystem  FAILURE
>[03:48 min]
>[INFO] Apache Atlas Common  SKIPPED
>[INFO] Apache Atlas Client  SKIPPED
>[INFO] Apache Atlas Notification .. SKIPPED
>[INFO] Apache Atlas Repository  SKIPPED
>[INFO] Apache Atlas Web Application ... SKIPPED
>[INFO] Apache Atlas Documentation . SKIPPED
>[INFO] Apache Atlas Hive Bridge ... SKIPPED
>[INFO] Apache Atlas Distribution .. SKIPPED
>[INFO] 
>
>[INFO] BUILD FAILURE
>[INFO] 
>
>[INFO] Total time: 07:38 min
>[INFO] Finished at: 2015-09-24T11:31:51-04:00
>[INFO] Final Memory: 42M/200M
>[INFO] 
>
>[ERROR] Failed to execute goal
>net.alchim31.maven:scala-maven-plugin:3.2.0:compile (scala-compile-first)
>on project atlas-typesystem: Execution scala-compile-first of goal
>net.alchim31.maven:scala-maven-plugin:3.2.0:compile failed. CompileFailed
>-> [Help 1]
>
>
>
>
>On Sep 23, 2015, at 12:16 PM, Suma Shivaprasad
> wrote:
>
>> Could be an issue with 0.5. I fixed an issue while querying for entities
>> with traits as part of https://reviews.apache.org/r/38179/. . Should be
>> committed soon.
>> 
>> Thanks
>> Suma
>> 
>> 
>> On Wed, Sep 23, 2015 at 8:40 PM, Shwetha Shivalingamurthy <
>> sshivalingamur...@hortonworks.com> wrote:
>> 
>>> 'hive_table is ¹ should work. Here is the doc on DSL syntax:
>>> http://atlas.incubator.apache.org/Search.html
>>> 
>>> Regards,
>>> Shwetha
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> On 23/09/15 8:30 pm, "Herman Yu"  wrote:
>>> 
>>&g

Re: Jenkins build is back to normal : apache-atlas-nightly #75

[Shwetha Shivalingamurthy]

Rat was failing because of dashboard/v2. Manually removed the files
yesterday

Regards,
Shwetha






On 24/09/15 10:29 am, "Seetharam Venkatesh" 
wrote:

>This is good news, we have a stable build.
>
>On Wed, Sep 23, 2015 at 6:32 PM Apache Jenkins Server <
>jenk...@builds.apache.org> wrote:
>
>> See 
>>
>>

Re: Search syntax

[Shwetha Shivalingamurthy]

'hive_table is ¹ should work. Here is the doc on DSL syntax:
http://atlas.incubator.apache.org/Search.html

Regards,
Shwetha






On 23/09/15 8:30 pm, "Herman Yu"  wrote:

>Hi Suma,
>Thanks for the help. I tried as you suggested, ³hive_table is
>collection², however "0 results matching your search query hive_table is
>collection were found².
>³collection² is a trait I defined, if I just search with ³collection² in
>the UI, I have two hive_table entities returned.
>I also tried with those types and traits defined with the quick_start.py,
>(e.g. "Table is Fact" didn¹t return anything.
>Is this a bug with v0.5? or I used wrong search syntax? Is there a
>document with all valid search syntax?
>
>thanks
>Herman.
>
>
>On Sep 21, 2015, at 10:54 PM, Herman Yu  wrote:
>
>Hi Herman,
>
>You can fire a DSL query in the UI as "hive_table  is "trait_name"
>
>Thanks
>Suma
>> Hi,
>> 
>> With version v0.5, is search based on combination of a Trait and a type
>>supported?  I have both hive_tables and hive_columns associated with a
>>Trait. I can search base on the Trait, or search based on the
>>type(hive_table or hive_colume). How to combine them together, for
>>example, search the trait but only want to return hive_table?
>> 
>> I tried  and ,  and
>>typeName= in both UI and REST API, it doesn¹t work.
>> 
>> This is my first time posting here, sorry if this is not the right
>>place asking questions like this. If so, what would be the correct
>>channel?
>> 
>> Thanks
>> Herman.
>> 
>> 
>
>

Re: [Atlas] A source code question

[Shwetha Shivalingamurthy]

Should be struct. Can you file a bug please? Thanks

Regards,
Shwetha






On 10/09/15 1:55 pm, "ltf"  wrote:

>Hello,
>When reading the source code of trunk, I think the following code may be
>a issue, or is intended ???
>
>
>file : org/apache/atlas/typesystem/types/TypeSystem.java
>for (StructTypeDefinition structDef : structDefs) {
>constructStructureType(structDef);
>typeCategoriesToTypeNamesMap.put(DataTypes.TypeCategory.CLASS,
>structDef.typeName);   //should be DataTypes.TypeCategory.STRUCT  ???
>}
>
>for (TraitType traitType : traitTypes) {
>constructHierarchicalType(TraitType.class,
>traitNameToDefMap.get(traitType.getName()));
>typeCategoriesToTypeNamesMap.put(DataTypes.TypeCategory.TRAIT,
>traitType.getName());
>}
>
>for (ClassType classType : classTypes) {
>constructHierarchicalType(ClassType.class,
>classNameToDefMap.get(classType.getName()));
>typeCategoriesToTypeNamesMap.put(DataTypes.TypeCategory.CLASS,
>classType.getName());
>}
>
>
>Best, liutongfeng

Re: Some code questions

[Shwetha Shivalingamurthy]

1, 2 and 4 look like valid issues. Can you file a bug and submit the
patches. Thanks

AttributeStores is used in some tests. Memory based repository is not
used. But can be fixed



Regards,
Shwetha






On 01/09/15 11:43 am, "ltf"  wrote:

>Hello,
>I'm very new to Atlas, and I still in the status of learning.
>When I browsed the source code of Atlas, I have the following 4
>questions, maybe or maybe not issues, thanks.
>
>
>1.
>file : 
>incubator-atlas\typesystem\src\main\java\org\apache\atlas\typesystem\types
>\ObjectGraphTraversal.java
>line : 56
>void processValue(IDataType dT, Object val) throws AtlasException {
>if (val != null) {
>if (dT.getTypeCategory() == DataTypes.TypeCategory.ARRAY) {
>IDataType elemType = ((DataTypes.ArrayType)
>dT).getElemType();
>processCollection(elemType, val);
>} else if (dT.getTypeCategory() ==
>DataTypes.TypeCategory.MAP) {
>IDataType keyType = ((DataTypes.MapType) dT).getKeyType();
>IDataType valueType = ((DataTypes.MapType)
>dT).getKeyType();//   should be getValueType(); ???
>processMap(keyType, valueType, val);
>} else if (dT.getTypeCategory() ==
>DataTypes.TypeCategory.STRUCT
>|| dT.getTypeCategory() ==
>DataTypes.TypeCategory.TRAIT) {
>processStruct(val);
>} else if (dT.getTypeCategory() ==
>DataTypes.TypeCategory.CLASS) {
>processReferenceableInstance(val);
>}
>}
>}
>
>
>2. 
>file : 
>incubator-atlas\typesystem\src\main\java\org\apache\atlas\typesystem\types
>\ObjectGraphWalker.java
>line : 94
>void traverseValue(IDataType dT, Object val) throws AtlasException {
>if (val != null) {
>if (dT.getTypeCategory() == DataTypes.TypeCategory.ARRAY) {
>IDataType elemType = ((DataTypes.ArrayType)
>dT).getElemType();
>visitCollection(elemType, val);
>} else if (dT.getTypeCategory() ==
>DataTypes.TypeCategory.MAP) {
>IDataType keyType = ((DataTypes.MapType) dT).getKeyType();
>IDataType valueType = ((DataTypes.MapType)
>dT).getKeyType();//   should be getValueType(); ???
>visitMap(keyType, valueType, val);
>} else if (dT.getTypeCategory() ==
>DataTypes.TypeCategory.STRUCT
>|| dT.getTypeCategory() ==
>DataTypes.TypeCategory.TRAIT) {
>visitStruct(val);
>} else if (dT.getTypeCategory() ==
>DataTypes.TypeCategory.CLASS) {
>visitReferenceableInstance(val);
>}
>}
>}
>
>
>3.
>file : 
>incubator-atlas\repository\src\main\java\org\apache\atlas\repository\memor
>y\AttributeStores.java
>line : 58
>static IAttributeStore createStore(AttributeInfo i) throws
>RepositoryException {
>switch (i.dataType().getTypeCategory()) {
>case PRIMITIVE:
>if (i.dataType() == DataTypes.BOOLEAN_TYPE) {
>return new BooleanAttributeStore(i);
>} else if (i.dataType() == DataTypes.BYTE_TYPE) {
>return new ByteAttributeStore(i);
>} else if (i.dataType() == DataTypes.SHORT_TYPE) {
>new ShortAttributeStore(i);
>   //   should be return ???
>} else if (i.dataType() == DataTypes.INT_TYPE) {
>return new IntAttributeStore(i);
>
>
>4.
>file : incubator-atlas\src\test\python\scripts\TestMetadata.py
>line : 52
>self.assertTrue(java_mock.called)
>if IS_WINDOWS:
>  java_mock.assert_called_with(
>'org.apache.atlas.Main',
>['-app', 'metadata_home/server/webapp/atlas'],
>   //  when os is WINDOWS, '/' should be '\\', otherwise the test
>fails, build will stop!!!
>
>'metadata_home/conf:metadata_home/server/webapp/atlas/WEB-INF/classes:meta
>data_home/server/webapp/atlas/WEB-INF/lib\\*:metadata_home/libext\\*',
>['-Datlas.log.dir=metadata_home/logs',
>'-Datlas.log.file=application.log', '-Datlas.home=metadata_home',
>'-Datlas.conf=metadata_home/conf', '-Xmx1024m',
>'-Dlog4j.configuration=atlas-log4j.xml'], 'metadata_home/logs')
>else:
>
>
>Best, liutongfeng

Re: JDK version

[Shwetha Shivalingamurthy]

We use oracle java 7, but should work with openjdk.

Currently, Atlas doesn¹t build with java 8. There is a bug open for it

Regards,
Shwetha

On 29/07/15 8:04 pm, "Nigel Jones"  wrote:

>I'm getting a few "glitches" rebuilding - it turns out they are the same
>issues the Jenkins build hit... so I'll track that issue.
>
>However it seemed prudent to ask what compiler/JDK version most people
>are currently using on the project?
>
>oracle java 7? (update 80)
>
>I presume no 8, 9 yet? openjdk?
>
>Thanks
>Nigel.
>

Re: Build failed in Jenkins: apache-atlas-nightly #8

[Shwetha Shivalingamurthy]

We will check. Thanks Giri

On 01/07/15 1:29 pm, "Giridharan Kesavan"  wrote:

>Build went past the maven issue, now the builds are failing for UT.
>
>
>
>On 6/30/15, 4:05 PM, "Seetharam Venkatesh" 
>wrote:
>
>>Thanks Giri, sincerely appreciate it. I copied the script from falcon
>>builds assuming this is available on all hosts. Looks like its not - how
>>is that triggered?
>>
>>Thanks!
>>
>>--
>>Regards,
>>Venkatesh
>>
>>________
>>From: Giridharan Kesavan
>>Sent: Tuesday, June 30, 2015 3:50 PM
>>To: Seetharam Venkatesh; Shwetha Shivalingamurthy
>>Cc: dev@atlas.incubator.apache.org
>>Subject: Re: Build failed in Jenkins: apache-atlas-nightly #8
>>
>>Updated the maven home to point to the exact location and triggered a
>>build.
>>Build is in queue, if you guys still see an issue, let me know.
>>
>>Thanks
>>giri
>>
>>
>>
>>On 6/29/15, 9:36 AM, "Seetharam Venkatesh" 
>>wrote:
>>
>>>I think I need help from Giri.
>>>
>>>Giri, can you please help with atlas builds at apache? Thanks!
>>>
>>>--
>>>Regards,
>>>Venkatesh
>>>
>>>
>>>From: Shwetha Shivalingamurthy
>>>Sent: Monday, June 29, 2015 2:01 AM
>>>To: Seetharam Venkatesh
>>>Cc: dev@atlas.incubator.apache.org
>>>Subject: Re: Build failed in Jenkins: apache-atlas-nightly #8
>>>
>>>Venkatesh,
>>>
>>>Can you fix the config please?
>>>
>>>+ /bin/mvn clean verify -Drat.numUnapprovedLicenses=100
>>>/tmp/hudson5441687896016250199.sh: line 6: /bin/mvn: No such file or
>>>directory
>>>
>>>
>>>Thanks,
>>>Shwetha
>>>
>>>On 29/06/15 8:26 am, "Apache Jenkins Server" 
>>>wrote:
>>>
>>>>See <https://builds.apache.org/job/apache-atlas-nightly/8/>
>>>>
>>>>--
>>>>[...truncated 148 lines...]
>>>>retired.png was last modified before this build started. Ignoring it.
>>>>sandbox.png was last modified before this build started. Ignoring it.
>>>>remove.gif was last modified before this build started. Ignoring it.
>>>>rss.png was last modified before this build started. Ignoring it.
>>>>update.gif was last modified before this build started. Ignoring it.
>>>>window-new.png was last modified before this build started. Ignoring
>>>>it.
>>>>types-instance.png was last modified before this build started.
>>>>Ignoring
>>>>it.
>>>>site.xml was last modified before this build started. Ignoring it.
>>>>Architecture.twiki was last modified before this build started.
>>>>Ignoring
>>>>it.
>>>>Configuration.twiki was last modified before this build started.
>>>>Ignoring
>>>>it.
>>>>InstallationSteps.twiki was last modified before this build started.
>>>>Ignoring it.
>>>>LICENSE.txt was last modified before this build started. Ignoring it.
>>>>QuickStart.twiki was last modified before this build started. Ignoring
>>>>it.
>>>>Repository.twiki was last modified before this build started. Ignoring
>>>>it.
>>>>Search.twiki was last modified before this build started. Ignoring it.
>>>>TypeSystem.twiki was last modified before this build started. Ignoring
>>>>it.
>>>>index.twiki was last modified before this build started. Ignoring it.
>>>>security.twiki was last modified before this build started. Ignoring
>>>>it.
>>>>pom.xml was last modified before this build started. Ignoring it.
>>>>release-log.txt was last modified before this build started. Ignoring
>>>>it.
>>>>pom.xml was last modified before this build started. Ignoring it.
>>>>GraphTransaction.java was last modified before this build started.
>>>>Ignoring it.
>>>>GraphTransactionInterceptor.java was last modified before this build
>>>>started. Ignoring it.
>>>>RepositoryMetadataModule.java was last modified before this build
>>>>started. Ignoring it.
>>>>DiscoveryException.java was last modified before this build started.
>>>>Ignoring it.
>>>>DiscoveryService.java was last modified before this build started.
>>>>Ignoring it.
>>>>Hiv

Re: Build failed in Jenkins: apache-atlas-nightly #8

[Shwetha Shivalingamurthy]

Venkatesh,

Can you fix the config please?

+ /bin/mvn clean verify -Drat.numUnapprovedLicenses=100
/tmp/hudson5441687896016250199.sh: line 6: /bin/mvn: No such file or
directory


Thanks,
Shwetha

On 29/06/15 8:26 am, "Apache Jenkins Server" 
wrote:

>See 
>
>--
>[...truncated 148 lines...]
>retired.png was last modified before this build started. Ignoring it.
>sandbox.png was last modified before this build started. Ignoring it.
>remove.gif was last modified before this build started. Ignoring it.
>rss.png was last modified before this build started. Ignoring it.
>update.gif was last modified before this build started. Ignoring it.
>window-new.png was last modified before this build started. Ignoring it.
>types-instance.png was last modified before this build started. Ignoring
>it.
>site.xml was last modified before this build started. Ignoring it.
>Architecture.twiki was last modified before this build started. Ignoring
>it.
>Configuration.twiki was last modified before this build started. Ignoring
>it.
>InstallationSteps.twiki was last modified before this build started.
>Ignoring it.
>LICENSE.txt was last modified before this build started. Ignoring it.
>QuickStart.twiki was last modified before this build started. Ignoring it.
>Repository.twiki was last modified before this build started. Ignoring it.
>Search.twiki was last modified before this build started. Ignoring it.
>TypeSystem.twiki was last modified before this build started. Ignoring it.
>index.twiki was last modified before this build started. Ignoring it.
>security.twiki was last modified before this build started. Ignoring it.
>pom.xml was last modified before this build started. Ignoring it.
>release-log.txt was last modified before this build started. Ignoring it.
>pom.xml was last modified before this build started. Ignoring it.
>GraphTransaction.java was last modified before this build started.
>Ignoring it.
>GraphTransactionInterceptor.java was last modified before this build
>started. Ignoring it.
>RepositoryMetadataModule.java was last modified before this build
>started. Ignoring it.
>DiscoveryException.java was last modified before this build started.
>Ignoring it.
>DiscoveryService.java was last modified before this build started.
>Ignoring it.
>HiveLineageService.java was last modified before this build started.
>Ignoring it.
>LineageService.java was last modified before this build started. Ignoring
>it.
>SearchIndexer.java was last modified before this build started. Ignoring
>it.
>DefaultGraphPersistenceStrategy.java was last modified before this build
>started. Ignoring it.
>GraphBackedDiscoveryService.java was last modified before this build
>started. Ignoring it.
>EntityChangeListener.java was last modified before this build started.
>Ignoring it.
>TypesChangeListener.java was last modified before this build started.
>Ignoring it.
>Constants.java was last modified before this build started. Ignoring it.
>DiscoverInstances.java was last modified before this build started.
>Ignoring it.
>EntityNotFoundException.java was last modified before this build started.
>Ignoring it.
>IRepository.java was last modified before this build started. Ignoring it.
>IndexCreationException.java was last modified before this build started.
>Ignoring it.
>IndexException.java was last modified before this build started. Ignoring
>it.
>MetadataRepository.java was last modified before this build started.
>Ignoring it.
>RepositoryException.java was last modified before this build started.
>Ignoring it.
>GraphBackedMetadataRepository.java was last modified before this build
>started. Ignoring it.
>GraphBackedSearchIndexer.java was last modified before this build
>started. Ignoring it.
>GraphHelper.java was last modified before this build started. Ignoring it.
>GraphProvider.java was last modified before this build started. Ignoring
>it.
>TitanGraphProvider.java was last modified before this build started.
>Ignoring it.
>AttributeStores.java was last modified before this build started.
>Ignoring it.
>ClassStore.java was last modified before this build started. Ignoring it.
>HierarchicalTypeStore.java was last modified before this build started.
>Ignoring it.
>IAttributeStore.java was last modified before this build started.
>Ignoring it.
>MemRepository.java was last modified before this build started. Ignoring
>it.
>ReplaceIdWithInstance.java was last modified before this build started.
>Ignoring it.
>StructStore.java was last modified before this build started. Ignoring it.
>TraitStore.java was last modified before this build started. Ignoring it.
>GraphBackedTypeStore.java was last modified before this build started.
>Ignoring it.
>ITypeStore.java was last modified before this build started. Ignoring it.
>StorageException.java was last modified before this build started.
>Ignoring it.
>DefaultMetadataService.java was last modified before this build starte

Re: [VOTE] Release Apache Atlas version 0.5-incubating

[Shwetha Shivalingamurthy]

The build fails with empty local m2 repository 

Sent from my iPad

> On 19-Jun-2015, at 9:23 pm, Suma Shivaprasad  
> wrote:
> 
> There is a build failure due to respository.codehause.org getting
> deprecated on a clean maven repo. Thanks to Shwetha for pointing this out.
> 
> [ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-site-plugin:3.3:site (default) on project
> atlas-docs: SiteToolException: The site descriptor cannot be resolved from
> the repository: ArtifactResolutionException: Unable to locate site
> descriptor: Could not transfer artifact org.apache:apache:xml:site_en:13
> from/to Codehaus repository (http://repository.codehaus.org/): Failed to
> transfer file:
> http://repository.codehaus.org/org/apache/apache/13/apache-13-site_en.xml.
> Return code is: 410 , ReasonPhrase:Gone.
> [ERROR] org.apache:apache:xml:13
> 
> Have fixed it here - https://issues.apache.org/jira/browse/ATLAS-31. Should
> we include this into the 0.5 release?
> 
>> On Fri, Jun 19, 2015 at 8:45 PM, Harish Butani  wrote:
>> 
>> +1
>> did a build, ran tests
>> 
>> On Thu, Jun 18, 2015 at 11:29 PM, Venkat Ranganathan <
>> vranganat...@hortonworks.com> wrote:
>> 
>>> +1
>>> 
>>> Built and ran tests
>>> 
>>> Verified signature, and digests
>>> 
>>> Venkat
>>> 
>>> 
>>> On 18/06/15 1:52 am, "Seetharam Venkatesh" 
>>> wrote:
>>> 
 Hi Folks,
 
 I have created a build for Apache Atlas 0.5-incubating, release
>> candidate
 0. Thanks to everyone who have contributed to this release and also
>> tested
 this.
 
 The source tarball (*.tar.gz), signature (*.asc), checksum (*.md5,
>> *.sha):
>> https://dist.apache.org/repos/dist/dev/incubator/atlas/0.5.0-incubating-rc
 0
 
 The SHA1 checksum of the archive is
 ab21ce037e488a8e5b8986353adb853dc515abd4
 
 The commit id (318abdacd4c4d17a3d613c1cda04a58194042715) to be voted
>> upon:
>> https://git-wip-us.apache.org/repos/asf?p=incubator-atlas.git;a=commit;h=3
 18abdacd4c4d17a3d613c1cda04a58194042715
 
 The tag to be voted upon:
>> https://git-wip-us.apache.org/repos/asf?p=incubator-atlas.git;a=tag;h=refs
 /tags/release-0.5-rc0
 
 The list of fixed issues:
>> https://git-wip-us.apache.org/repos/asf?p=incubator-atlas.git;a=blob;f=rel
>>> 
>>> ease-log.txt;h=df92e95d408469b2bea5b988fb9be3de802b9f2b;hb=318abdacd4c4d17
 a3d613c1cda04a58194042715
 
 Keys to verify the signature of the release artifact are available at:
 http://www.apache.org/dist/incubator/atlas/KEYS
 PGP release keys:
 http://pgp.mit.edu/pks/lookup?op=vindex&search=0x1B16738C42C7A5EA
 
 Note that this is a source only release and we are voting on the source
 (release-0.5-incubating-rc0 / Commit:
 318abdacd4c4d17a3d613c1cda04a58194042715 / SHA:
 ab21ce037e488a8e5b8986353adb853dc515abd4).
 
 Please download, test, and try it out.
 
 Vote will be open for at least 72 hours (until the required number of
>> IPMC
 votes are obtained).
 
 [ ] +1 approve
 [ ] +0 no opinion
 [ ] -1 disapprove (and reason why)
 
 +1 from my side for the release.
 
 For folks not familiar with vetting a release, please refer to
 http://incubator.apache.org/guides/releasemanagement.html#check-list
 
 Regards,
 Venkatesh
>> 

Re: [VOTE] Release Apache Atlas version 0.5-incubating

[Shwetha Shivalingamurthy]

+1 - verified the build, deployment and UI

On 18/06/15 1:52 am, "Seetharam Venkatesh"  wrote:

>Hi Folks,
>
>I have created a build for Apache Atlas 0.5-incubating, release candidate
>0. Thanks to everyone who have contributed to this release and also tested
>this.
>
>The source tarball (*.tar.gz), signature (*.asc), checksum (*.md5, *.sha):
>https://dist.apache.org/repos/dist/dev/incubator/atlas/0.5.0-incubating-rc
>0
>
>The SHA1 checksum of the archive is
>ab21ce037e488a8e5b8986353adb853dc515abd4
>
>The commit id (318abdacd4c4d17a3d613c1cda04a58194042715) to be voted upon:
>https://git-wip-us.apache.org/repos/asf?p=incubator-atlas.git;a=commit;h=3
>18abdacd4c4d17a3d613c1cda04a58194042715
>
>The tag to be voted upon:
>https://git-wip-us.apache.org/repos/asf?p=incubator-atlas.git;a=tag;h=refs
>/tags/release-0.5-rc0
>
>The list of fixed issues:
>https://git-wip-us.apache.org/repos/asf?p=incubator-atlas.git;a=blob;f=rel
>ease-log.txt;h=df92e95d408469b2bea5b988fb9be3de802b9f2b;hb=318abdacd4c4d17
>a3d613c1cda04a58194042715
>
>Keys to verify the signature of the release artifact are available at:
>http://www.apache.org/dist/incubator/atlas/KEYS
>PGP release keys:
>http://pgp.mit.edu/pks/lookup?op=vindex&search=0x1B16738C42C7A5EA
>
>Note that this is a source only release and we are voting on the source
>(release-0.5-incubating-rc0 / Commit:
>318abdacd4c4d17a3d613c1cda04a58194042715 / SHA:
>ab21ce037e488a8e5b8986353adb853dc515abd4).
>
>Please download, test, and try it out.
>
>Vote will be open for at least 72 hours (until the required number of IPMC
>votes are obtained).
>
>[ ] +1 approve
>[ ] +0 no opinion
>[ ] -1 disapprove (and reason why)
>
>+1 from my side for the release.
>
>For folks not familiar with vetting a release, please refer to
>http://incubator.apache.org/guides/releasemanagement.html#check-list
>
>Regards,
>Venkatesh