[jira] [Updated] (ATLAS-1892) Implement logic to create relationship attributes in AtlasEntityType
[ https://issues.apache.org/jira/browse/ATLAS-1892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sarath Subramanian updated ATLAS-1892: -- Attachment: ATLAS-1856.4.patch > Implement logic to create relationship attributes in AtlasEntityType > > > Key: ATLAS-1892 > URL: https://issues.apache.org/jira/browse/ATLAS-1892 > Project: Atlas > Issue Type: New Feature > Components: atlas-core >Affects Versions: trunk, 0.9-incubating >Reporter: Sarath Subramanian >Assignee: Sarath Subramanian > Attachments: ATLAS-1856.4.patch > > > When a new relationshipDef is created, relation attributes needs to be > populated in AtlasEntityType during resolveReference stage. > This JIRA also adds UT and IT and addresses review comments in > https://reviews.apache.org/r/59769/ -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (ATLAS-1892) Implement logic to create relationship attributes in AtlasEntityType
[ https://issues.apache.org/jira/browse/ATLAS-1892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sarath Subramanian updated ATLAS-1892: -- Description: When a new relationshipDef is created, relation attributes needs to be populated in AtlasEntityType during resolveReference stage. This JIRA also adds UT and IT and addresses review comments in https://reviews.apache.org/r/59769/ > Implement logic to create relationship attributes in AtlasEntityType > > > Key: ATLAS-1892 > URL: https://issues.apache.org/jira/browse/ATLAS-1892 > Project: Atlas > Issue Type: New Feature > Components: atlas-core >Affects Versions: trunk, 0.9-incubating >Reporter: Sarath Subramanian >Assignee: Sarath Subramanian > > When a new relationshipDef is created, relation attributes needs to be > populated in AtlasEntityType during resolveReference stage. > This JIRA also adds UT and IT and addresses review comments in > https://reviews.apache.org/r/59769/ -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (ATLAS-1892) Implement logic to create relationship attributes in AtlasEntityType
Sarath Subramanian created ATLAS-1892: - Summary: Implement logic to create relationship attributes in AtlasEntityType Key: ATLAS-1892 URL: https://issues.apache.org/jira/browse/ATLAS-1892 Project: Atlas Issue Type: New Feature Components: atlas-core Affects Versions: trunk, 0.9-incubating Reporter: Sarath Subramanian Assignee: Sarath Subramanian -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (ATLAS-1891) Introduce prechecks for relationshipDef update
David Radley created ATLAS-1891: --- Summary: Introduce prechecks for relationshipDef update Key: ATLAS-1891 URL: https://issues.apache.org/jira/browse/ATLAS-1891 Project: Atlas Issue Type: Sub-task Reporter: David Radley Assignee: David Radley -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (ATLAS-1890) IntegrationTest cases failing in BaseResourceIT.waitForNotification method
Nixon Rodrigues created ATLAS-1890:
--
Summary: IntegrationTest cases failing in
BaseResourceIT.waitForNotification method
Key: ATLAS-1890
URL: https://issues.apache.org/jira/browse/ATLAS-1890
Project: Atlas
Issue Type: Bug
Reporter: Nixon Rodrigues
Problem:-
This test cases are failing in BaseResourceIT.waitForNotification method.
In waitForNotification method it waits for certain Time for Kafka notification
of the entity created/update/deleted etc to read from KafkaTopic.
Currently for Integration tests in Atlas, a metadata server is running on port
31000 along berkely-db store and kafka/zookeeper for notifitication.
This Atlas services is used by All Integration tests that are running from
maven test suite.
Since testcases run independent of eachother and if they access shared like
kafka notification via consumer, the consumer will results different type of
results and also with some IO and timing issues.
{code}
testEntityDeduping(org.apache.atlas.web.integration.EntityJerseyResourceIT)
Time elapsed: 61.463 sec <<< FAILURE!
java.lang.Exception: Waiting timed out after 6000 msec
at
org.apache.atlas.web.integration.EntityJerseyResourceIT.testEntityDeduping(EntityJerseyResourceIT.java:221)
EntityV2JerseyResourceIT.testEntityDeduping:170->BaseResourceIT.waitForNotification:633->BaseResourceIT.waitFor:625
»
EntityNotificationIT.testDeleteEntity:108->BaseResourceIT.waitForNotification:633->BaseResourceIT.waitFor:625
»
{code}
The purpose of waitForNotification method is to valid if notifications reach
the Kafka topic.
I propose to remove this dependeny to check the entity created notification on
kafka topic from the above IntegrationTest.
The validation of created/update/deleted notification of entity into kafka
topic can be done seperate UT/IT.
Let me know your thoughts.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (ATLAS-1889) Failure in simultaneous updates to an entity tag association
[
https://issues.apache.org/jira/browse/ATLAS-1889?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ashutosh Mestry updated ATLAS-1889:
---
Attachment: ATLAS-1889-Handling-multiple-entities.patch
> Failure in simultaneous updates to an entity tag association
>
>
> Key: ATLAS-1889
> URL: https://issues.apache.org/jira/browse/ATLAS-1889
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Affects Versions: 0.8-incubating
>Reporter: Ashutosh Mestry
>Assignee: Ashutosh Mestry
> Fix For: trunk, 0.8.1-incubating
>
> Attachments: ATLAS-1889-Handling-multiple-entities.patch
>
>
> *Background*
> When multiple requests simultaneously attempt to change the tags associated
> with an entity, the current implementation might result in APIs returning
> fewer tags for the entity than is actually associated.
> The problem thus causes UI not to behave correctly.
>
> *Solution*
> Implements a synchronization mechanism.
>
> Steps to duplicate the problem:
> * Create several tags. Eg. tag1, tag2, tag3, tag4.
> * Attach these tags to and entity say ‘testtable1’
> * Use the following CURL commands to delete the tags:
> {code}
> curl -X DELETE -u admin:admin
> 'http://localhost:21000/api/atlas/entities/a2d0b023-aafe-4ddd-bb90-65164af8796c/traits/tag1'
> -H 'Content-Type: application/json' &
> curl -X DELETE -u admin:admin
> 'http://localhost:21000/api/atlas/entities/a2d0b023-aafe-4ddd-bb90-65164af8796c/traits/tag2'
> -H 'Content-Type: application/json' &
> curl -X DELETE -u admin:admin
> 'http://localhost:21000/api/atlas/entities/a2d0b023-aafe-4ddd-bb90-65164af8796c/traits/tag3'
> -H 'Content-Type: application/json' &
> curl -X DELETE -u admin:admin
> 'http://localhost:21000/api/atlas/entities/a2d0b023-aafe-4ddd-bb90-65164af8796c/traits/tag4'
> -H 'Content-Type: application/json'
> {code}
> You will notice that entity _testtable1_ ends up with some tags not deleted.
> Additional querying with gremlin will show that the edges are deleted
> correctly.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (ATLAS-1889) Failure in simultaneous updates to an entity tag association
[
https://issues.apache.org/jira/browse/ATLAS-1889?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ashutosh Mestry updated ATLAS-1889:
---
Description:
*Background*
When multiple requests simultaneously attempt to change the tags associated
with an entity, the current implementation might result in APIs returning fewer
tags for the entity than is actually associated.
The problem thus causes UI not to behave correctly.
*Solution*
Implements a synchronization mechanism.
Steps to duplicate the problem:
* Create several tags. Eg. tag1, tag2, tag3, tag4.
* Attach these tags to and entity say ‘testtable1’
* Use the following CURL commands to delete the tags:
{code}
curl -X DELETE -u admin:admin
'http://localhost:21000/api/atlas/entities/a2d0b023-aafe-4ddd-bb90-65164af8796c/traits/tag1'
-H 'Content-Type: application/json' &
curl -X DELETE -u admin:admin
'http://localhost:21000/api/atlas/entities/a2d0b023-aafe-4ddd-bb90-65164af8796c/traits/tag2'
-H 'Content-Type: application/json' &
curl -X DELETE -u admin:admin
'http://localhost:21000/api/atlas/entities/a2d0b023-aafe-4ddd-bb90-65164af8796c/traits/tag3'
-H 'Content-Type: application/json' &
curl -X DELETE -u admin:admin
'http://localhost:21000/api/atlas/entities/a2d0b023-aafe-4ddd-bb90-65164af8796c/traits/tag4'
-H 'Content-Type: application/json'
{code}
You will notice that entity _testtable1_ ends up with some tags not deleted.
Additional querying with gremlin will show that the edges are deleted correctly.
was:
*Background*
When multiple requests simultaneously attempt to change the tags associated
with an entity, the current implementation might result in APIs returning fewer
tags for the entity than is actually associated.
The problem thus causes UI not to behave correctly.
*Solution*
Implements a synchronization mechanism.
Steps to duplicate the problem:
* Create several tags. Eg. tag1, tag2, tag3, tag4.
* Attach these tags to and entity say ‘testtable1’
* Use the following CURL commands to delete the tags:
_curl -X DELETE -u admin:admin
'http://localhost:21000/api/atlas/entities/a2d0b023-aafe-4ddd-bb90-65164af8796c/traits/tag1'
-H 'Content-Type: application/json' &
curl -X DELETE -u admin:admin
'http://localhost:21000/api/atlas/entities/a2d0b023-aafe-4ddd-bb90-65164af8796c/traits/tag2'
-H 'Content-Type: application/json' &
curl -X DELETE -u admin:admin
'http://localhost:21000/api/atlas/entities/a2d0b023-aafe-4ddd-bb90-65164af8796c/traits/tag3'
-H 'Content-Type: application/json' &
curl -X DELETE -u admin:admin
'http://localhost:21000/api/atlas/entities/a2d0b023-aafe-4ddd-bb90-65164af8796c/traits/tag4'
-H 'Content-Type: application/json'
_
You will notice that entity _testtable1_ ends up with some tags not deleted.
Additional querying with gremlin will show that the edges are deleted correctly.
> Failure in simultaneous updates to an entity tag association
>
>
> Key: ATLAS-1889
> URL: https://issues.apache.org/jira/browse/ATLAS-1889
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Affects Versions: 0.8-incubating
>Reporter: Ashutosh Mestry
>Assignee: Ashutosh Mestry
> Fix For: trunk, 0.8.1-incubating
>
>
> *Background*
> When multiple requests simultaneously attempt to change the tags associated
> with an entity, the current implementation might result in APIs returning
> fewer tags for the entity than is actually associated.
> The problem thus causes UI not to behave correctly.
>
> *Solution*
> Implements a synchronization mechanism.
>
> Steps to duplicate the problem:
> * Create several tags. Eg. tag1, tag2, tag3, tag4.
> * Attach these tags to and entity say ‘testtable1’
> * Use the following CURL commands to delete the tags:
> {code}
> curl -X DELETE -u admin:admin
> 'http://localhost:21000/api/atlas/entities/a2d0b023-aafe-4ddd-bb90-65164af8796c/traits/tag1'
> -H 'Content-Type: application/json' &
> curl -X DELETE -u admin:admin
> 'http://localhost:21000/api/atlas/entities/a2d0b023-aafe-4ddd-bb90-65164af8796c/traits/tag2'
> -H 'Content-Type: application/json' &
> curl -X DELETE -u admin:admin
> 'http://localhost:21000/api/atlas/entities/a2d0b023-aafe-4ddd-bb90-65164af8796c/traits/tag3'
> -H 'Content-Type: application/json' &
> curl -X DELETE -u admin:admin
> 'http://localhost:21000/api/atlas/entities/a2d0b023-aafe-4ddd-bb90-65164af8796c/traits/tag4'
> -H 'Content-Type: application/json'
> {code}
> You will notice that entity _testtable1_ ends up with some tags not deleted.
> Additional querying with gremlin will show that the edges are deleted
> correctly.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (ATLAS-1889) Failure in simultaneous updates to an entity tag association
Ashutosh Mestry created ATLAS-1889: -- Summary: Failure in simultaneous updates to an entity tag association Key: ATLAS-1889 URL: https://issues.apache.org/jira/browse/ATLAS-1889 Project: Atlas Issue Type: Bug Components: atlas-core Affects Versions: 0.8-incubating Reporter: Ashutosh Mestry Assignee: Ashutosh Mestry Fix For: trunk, 0.8.1-incubating *Background* When multiple requests simultaneously attempt to change the tags associated with an entity, the current implementation might result in APIs returning fewer tags for the entity than is actually associated. The problem thus causes UI not to behave correctly. *Solution* Implements a synchronization mechanism. Steps to duplicate the problem: * Create several tags. Eg. tag1, tag2, tag3, tag4. * Attach these tags to and entity say ‘testtable1’ * Use the following CURL commands to delete the tags: _curl -X DELETE -u admin:admin 'http://localhost:21000/api/atlas/entities/a2d0b023-aafe-4ddd-bb90-65164af8796c/traits/tag1' -H 'Content-Type: application/json' & curl -X DELETE -u admin:admin 'http://localhost:21000/api/atlas/entities/a2d0b023-aafe-4ddd-bb90-65164af8796c/traits/tag2' -H 'Content-Type: application/json' & curl -X DELETE -u admin:admin 'http://localhost:21000/api/atlas/entities/a2d0b023-aafe-4ddd-bb90-65164af8796c/traits/tag3' -H 'Content-Type: application/json' & curl -X DELETE -u admin:admin 'http://localhost:21000/api/atlas/entities/a2d0b023-aafe-4ddd-bb90-65164af8796c/traits/tag4' -H 'Content-Type: application/json' _ You will notice that entity _testtable1_ ends up with some tags not deleted. Additional querying with gremlin will show that the edges are deleted correctly. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (ATLAS-1095) Open connector framework
[ https://issues.apache.org/jira/browse/ATLAS-1095?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mandy Chessell updated ATLAS-1095: -- Attachment: Open Connector Framework - 20th June 2017.doc Improved descriptions and definitions of interfaces - Still needs work around interfaces related to metadata such as ConnectedAssetDescription, Endpoint and ConnectorType. In addition need a definition for AuditLog from GAF. > Open connector framework > > > Key: ATLAS-1095 > URL: https://issues.apache.org/jira/browse/ATLAS-1095 > Project: Atlas > Issue Type: New Feature >Affects Versions: 0.8-incubating >Reporter: Stephanie Hazlewood >Assignee: Mandy Chessell > Labels: VirtualDataConnector > Attachments: Open Connector Framework - 20th June 2017.doc, Open > Connector Framework - 9th May 2017.doc > > > Atlas provides a common approach to metadata management and governance across > all systems and data within an organization. Today Atlas provides access to > metadata. A connector provides access to a data source. As connectors are > the proxy of all data, they can also be explicit providers of metadata. > This JIRA proposes an open connector framework to manage connectors that > provide access to both data and the metadata Atlas provides together through > a single connector interface. > This will help data tools to to better the exchange of information between > platforms. It also offers new opportunities for the consistent enforcement of > the governance policies and rules (e.g., rules of visibility). Source > connector/connection metadata provides the nucleus around which all other > metadata describing the data builds. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (ATLAS-1879) Updating classification removes some properties
[
https://issues.apache.org/jira/browse/ATLAS-1879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16055867#comment-16055867
]
Madhan Neethiraj commented on ATLAS-1879:
-
[~bpgergo] - type renaming should not be allowed. Can you add details of REST
calls that resulted in above error?
> Updating classification removes some properties
> ---
>
> Key: ATLAS-1879
> URL: https://issues.apache.org/jira/browse/ATLAS-1879
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Affects Versions: 0.8-incubating
>Reporter: Laura Ngo
> Attachments: Atlas-1789.postman_collection.json
>
>
> * Created classification via POST.
> * Updated via PUT
> * Lost properties
> POST http://127.0.0.1:21000/api/atlas/v2/types/typedefs
> {code}
> {
> "classificationDefs": [{
> "name": "test_classification_11",
> "description": "",
> "createdBy" : "admin",
> "superTypes": [],
> "attributeDefs": [{
> "name" : "test_class_11",
> "typeName" : "string",
> "isOptional" : true,
> "isUnique" : true,
> "isIndexable" : true,
> "cardinality": "SINGLE",
> "valuesMinCount": 0,
> "valuesMaxCount": 1
> }]
> }],
> "entityDefs": [],
> "enumDefs": [],
> "structDefs": []
> }
> {code}
> GET
> http://127.0.0.1:21000/api/atlas/v2/types/classification/name/test_classification_11
> {code}
> {
> "category": "CLASSIFICATION",
> "guid": "83162fe1-4bb4-4a87-b2b8-364e751a1265",
> "createdBy": "admin",
> "createTime": 1497485890857,
> "updateTime": 1497485890857,
> "version": 1,
> "name": "test_classification_11",
> "description": "",
> "typeVersion": "1.0",
> "attributeDefs": [
> {
> "name": "test_class_11",
> "typeName": "string",
> "isOptional": true,
> "cardinality": "SINGLE",
> "valuesMinCount": 0,
> "valuesMaxCount": 1,
> "isUnique": true,
> "isIndexable": true
> }
> ],
> "superTypes": [],
> }
> {code}
> PUT http://127.0.0.1:21000/api/atlas/v2/types/typedefs
> Update attribute.
> GET
> http://127.0.0.1:21000/api/atlas/v2/types/classification/name/test_classification_11
> {code}
> {
> "category": "CLASSIFICATION",
> "createdBy": "admin",
> "name": "test_classification_11",
> "description": "",
> "attributeDefs": [
> {
> "name": "test_class_11",
> "typeName": "string",
> "isOptional": true,
> "cardinality": "SINGLE",
> "valuesMinCount": 0,
> "valuesMaxCount": 1,
> "isUnique": true,
> "isIndexable": false
> }
> ],
> "superTypes": [],
> }
> {code}
> Some properties are missing after PUT update of attribute "isIndexable"
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (ATLAS-1879) Updating classification removes some properties
[
https://issues.apache.org/jira/browse/ATLAS-1879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16055795#comment-16055795
]
Péter Gergő Barna commented on ATLAS-1879:
--
The problem is not only some properties are missing,
if you update a type by name, then you will no longer be able to retrieve it by
guid,
you will get HTTP 404
{noformat}
{
"errorCode": "ATLAS-404-00-002",
"errorMessage": "Given type guid a629a714-5150-4255-a4cd-54d7ddd396e7 was
invalid"
}
{noformat}
> Updating classification removes some properties
> ---
>
> Key: ATLAS-1879
> URL: https://issues.apache.org/jira/browse/ATLAS-1879
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Affects Versions: 0.8-incubating
>Reporter: Laura Ngo
> Attachments: Atlas-1789.postman_collection.json
>
>
> * Created classification via POST.
> * Updated via PUT
> * Lost properties
> POST http://127.0.0.1:21000/api/atlas/v2/types/typedefs
> {code}
> {
> "classificationDefs": [{
> "name": "test_classification_11",
> "description": "",
> "createdBy" : "admin",
> "superTypes": [],
> "attributeDefs": [{
> "name" : "test_class_11",
> "typeName" : "string",
> "isOptional" : true,
> "isUnique" : true,
> "isIndexable" : true,
> "cardinality": "SINGLE",
> "valuesMinCount": 0,
> "valuesMaxCount": 1
> }]
> }],
> "entityDefs": [],
> "enumDefs": [],
> "structDefs": []
> }
> {code}
> GET
> http://127.0.0.1:21000/api/atlas/v2/types/classification/name/test_classification_11
> {code}
> {
> "category": "CLASSIFICATION",
> "guid": "83162fe1-4bb4-4a87-b2b8-364e751a1265",
> "createdBy": "admin",
> "createTime": 1497485890857,
> "updateTime": 1497485890857,
> "version": 1,
> "name": "test_classification_11",
> "description": "",
> "typeVersion": "1.0",
> "attributeDefs": [
> {
> "name": "test_class_11",
> "typeName": "string",
> "isOptional": true,
> "cardinality": "SINGLE",
> "valuesMinCount": 0,
> "valuesMaxCount": 1,
> "isUnique": true,
> "isIndexable": true
> }
> ],
> "superTypes": [],
> }
> {code}
> PUT http://127.0.0.1:21000/api/atlas/v2/types/typedefs
> Update attribute.
> GET
> http://127.0.0.1:21000/api/atlas/v2/types/classification/name/test_classification_11
> {code}
> {
> "category": "CLASSIFICATION",
> "createdBy": "admin",
> "name": "test_classification_11",
> "description": "",
> "attributeDefs": [
> {
> "name": "test_class_11",
> "typeName": "string",
> "isOptional": true,
> "cardinality": "SINGLE",
> "valuesMinCount": 0,
> "valuesMaxCount": 1,
> "isUnique": true,
> "isIndexable": false
> }
> ],
> "superTypes": [],
> }
> {code}
> Some properties are missing after PUT update of attribute "isIndexable"
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (ATLAS-1888) UI - While Update Entity the data of type Long is not updating in entity from UI.
[ https://issues.apache.org/jira/browse/ATLAS-1888?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kalyani Kashikar updated ATLAS-1888: Attachment: ATLAS-1888.patch > UI - While Update Entity the data of type Long is not updating in entity from > UI. > - > > Key: ATLAS-1888 > URL: https://issues.apache.org/jira/browse/ATLAS-1888 > Project: Atlas > Issue Type: Bug >Reporter: Kalyani Kashikar >Assignee: Kalyani Kashikar > Attachments: ATLAS-1888.patch > > > Step to reproduce :- > * Search for the Table. > * Go to sales_fact_monthly_mv, and update CreateTime, LastAccessTime > * Click on update button > * CreateTime and LastAccessTime are not updating properly. > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Resolved] (ATLAS-1662) New governance action API to support Ranger tags from v2 glossary
[
https://issues.apache.org/jira/browse/ATLAS-1662?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nigel Jones resolved ATLAS-1662.
Resolution: Fixed
This and ATLAS-1696 are effectively duplicates
> New governance action API to support Ranger tags from v2 glossary
> -
>
> Key: ATLAS-1662
> URL: https://issues.apache.org/jira/browse/ATLAS-1662
> Project: Atlas
> Issue Type: New Feature
>Reporter: Nigel Jones
>Assignee: Nigel Jones
>
> Today Ranger obtains classification ("tags") from atlas via a tagsync process
> which
> - looks at interesting types
> - going through entities of those types
> - pulling traits/trait instances to get tags
> - formulating these into a map
> - persisting in ranger
> ATLAS-1410 proposes a new enterprise ready glossary which allows for a more
> sophticated approach to classification.
> To support this we will develop a new consumer-centric ("omas") API to
> support Ranger or other enforcement engines . The implementation behind the
> first function of this API will navigate the new glossary structure to
> present ranger with the simplified entity:classification map it needs rather
> than the more elaborate structure of the glossary.
> A new ranger tagsync process will then use this api to push tags into ranger
> without needing changes
> More generally this API will support anything an enforcement engine needs
> more detail to follow
> (Note to observers: could you assign this one to me or grant permission)
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Assigned] (ATLAS-1869) Atlas "plugin" for Ranger (metadata capture)
[ https://issues.apache.org/jira/browse/ATLAS-1869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nigel Jones reassigned ATLAS-1869: -- Assignee: Nigel Jones > Atlas "plugin" for Ranger (metadata capture) > > > Key: ATLAS-1869 > URL: https://issues.apache.org/jira/browse/ATLAS-1869 > Project: Atlas > Issue Type: Bug >Reporter: Nigel Jones >Assignee: Nigel Jones > > With a variety of data processing engines in Hadoop such as Hive, we have an > Atlas plugin & hook that captures new & updated metadata from those engines > and pushes it to Atlas. This can then be used to support governance including > lineage. > We already have a "ranger plugin" for Atlas which allows ranger to control > access to metadata in atlas - this is NOT the subject of this Jira, but > rather "the other way around" > Examples might include > * Capture information about the policies that are deployed in a ranger > server - the types of assets they refer to, the classifications that are > used. > * Capture information about the topology of ranger - by this I mean the > plugins that are deployed and active, the nodes they run on, and feed this > back into an operational model in Atlas > In each case the information could be published by Ranger, consumed by Atlas, > and stewardship activities around the atlas metadata could help in tying > things together > The benefit would be > - better end to end view (since we know the endpoints, identifiers in audit > logs) > - optimizing the interfaces (rest & kafka) by being able to better targer > useful information - ie if only a hive plugin is being used & configured for > tags, let's just worry the tags it needs. > I see this of use around our open metadata work & specifically VDC, though > not essential for an initial MVP > Placeholder for now... will elaborate further > At the same time the coupling would be loose, and shouldn't hinder any > existing integrations, or decisions as to what is done in Atlas vs Ranger -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (ATLAS-1888) UI - While Update Entity the data of type Long is not updating in entity from UI.
Kalyani Kashikar created ATLAS-1888: --- Summary: UI - While Update Entity the data of type Long is not updating in entity from UI. Key: ATLAS-1888 URL: https://issues.apache.org/jira/browse/ATLAS-1888 Project: Atlas Issue Type: Bug Reporter: Kalyani Kashikar Assignee: Kalyani Kashikar Step to reproduce :- * Search for the Table. * Go to sales_fact_monthly_mv, and update CreateTime, LastAccessTime * Click on update button * CreateTime and LastAccessTime are not updating properly. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (ATLAS-1824) Cannot create hive lineage when using query "insert into table X select A.Y, B.Z from A,B"
[ https://issues.apache.org/jira/browse/ATLAS-1824?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinzhi,Luo updated ATLAS-1824: -- Attachment: (was: 0001-fix-ATLAS-1824.patch) > Cannot create hive lineage when using query "insert into table X select A.Y, > B.Z from A,B" > -- > > Key: ATLAS-1824 > URL: https://issues.apache.org/jira/browse/ATLAS-1824 > Project: Atlas > Issue Type: Bug > Components: atlas-core >Affects Versions: trunk, 0.7-incubating, 0.8-incubating, 0.7.1-incubating, > 0.9-incubating, 0.8.1-incubating >Reporter: qinglin,xia >Assignee: Xinzhi,Luo > Labels: atlas > Fix For: 0.9-incubating > > Attachments: 0001-fix-ATLAS-1824.patch, > failed_lineage_input_output.png, failed_lineage.png > > > Problem: > I am using atlas to create hive lineage, I found that when using the hive > query "insert into table A select B.name ,C.name from B, C", the lineage > between A, B and C cannot be created. > Steps to reproduce: > 1. log in to the hive shell > 2. create hive table a, b,c and insert into data into a,b and c > 3. using query "insert into table datalake_demo.d9 select a.id as key, a.id, > a.aname ,b.bname,c.cname from datalake_demo.a,datalake_demo.b,datalake_demo.c > where a.id = b.id and b.id = c.id" to create hive table d9 > Expected Result: > hive_process is created , input of the process is a, b, c, output of the > process is datalake_demo.d9, and the process could be shown on atlas UI > Actual Result: > hive_process is created, input of the process is a, b, b, d9, output of the > process is null, and the process could not be shown on atlas UI, according to > the attached pic "failed_lineage.png" and "failed_lineage_input_output.png" > PS: We are working on fixing the issue! -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (ATLAS-1824) Cannot create hive lineage when using query "insert into table X select A.Y, B.Z from A,B"
[ https://issues.apache.org/jira/browse/ATLAS-1824?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinzhi,Luo updated ATLAS-1824: -- Attachment: 0001-fix-ATLAS-1824.patch > Cannot create hive lineage when using query "insert into table X select A.Y, > B.Z from A,B" > -- > > Key: ATLAS-1824 > URL: https://issues.apache.org/jira/browse/ATLAS-1824 > Project: Atlas > Issue Type: Bug > Components: atlas-core >Affects Versions: trunk, 0.7-incubating, 0.8-incubating, 0.7.1-incubating, > 0.9-incubating, 0.8.1-incubating >Reporter: qinglin,xia >Assignee: Xinzhi,Luo > Labels: atlas > Fix For: 0.9-incubating > > Attachments: 0001-fix-ATLAS-1824.patch, > failed_lineage_input_output.png, failed_lineage.png > > > Problem: > I am using atlas to create hive lineage, I found that when using the hive > query "insert into table A select B.name ,C.name from B, C", the lineage > between A, B and C cannot be created. > Steps to reproduce: > 1. log in to the hive shell > 2. create hive table a, b,c and insert into data into a,b and c > 3. using query "insert into table datalake_demo.d9 select a.id as key, a.id, > a.aname ,b.bname,c.cname from datalake_demo.a,datalake_demo.b,datalake_demo.c > where a.id = b.id and b.id = c.id" to create hive table d9 > Expected Result: > hive_process is created , input of the process is a, b, c, output of the > process is datalake_demo.d9, and the process could be shown on atlas UI > Actual Result: > hive_process is created, input of the process is a, b, b, d9, output of the > process is null, and the process could not be shown on atlas UI, according to > the attached pic "failed_lineage.png" and "failed_lineage_input_output.png" > PS: We are working on fixing the issue! -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (ATLAS-1887) Coverity Scan defects in AtlasRelationshipType
[
https://issues.apache.org/jira/browse/ATLAS-1887?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Radley updated ATLAS-1887:
Attachment: rb60213.patch
Amended the logic in AtlasRelationshipDef so that it does not address
relationship end content and then later check whether the relationship end was
null. I introduced a junit as well.
> Coverity Scan defects in AtlasRelationshipType
> --
>
> Key: ATLAS-1887
> URL: https://issues.apache.org/jira/browse/ATLAS-1887
> Project: Atlas
> Issue Type: Bug
>Reporter: David Radley
>Assignee: David Radley
> Attachments: rb60213.patch
>
>
> 3 new defect(s) introduced to Apache Atlas found with Coverity Scan.
> 3 defect(s), reported by Coverity Scan earlier, were marked fixed in the
> recent build analyzed by Coverity Scan.
>
> New defect(s) Reported-by: Coverity Scan
> Showing 3 of 3 defect(s)
>
>
> ** CID 164610:(REVERSE_INULL)
> /intg/src/main/java/org/apache/atlas/type/AtlasRelationshipType.java: 140
> in
> org.apache.atlas.type.AtlasRelationshipType.validateAtlasRelationshipDef(org.apache.atlas.model.typedef.AtlasRelationshipDef)()
> /intg/src/main/java/org/apache/atlas/type/AtlasRelationshipType.java: 147
> in
> org.apache.atlas.type.AtlasRelationshipType.validateAtlasRelationshipDef(org.apache.atlas.model.typedef.AtlasRelationshipDef)()
>
>
>
>
> *** CID 164610:(REVERSE_INULL)
> /intg/src/main/java/org/apache/atlas/type/AtlasRelationshipType.java: 140
> in
> org.apache.atlas.type.AtlasRelationshipType.validateAtlasRelationshipDef(org.apache.atlas.model.typedef.AtlasRelationshipDef)()
> 134 // composition containers should not be multiple
> cardinality
> 135 if (endDef1 != null &&
> 136 endDef1.getCardinality() ==
> AtlasAttributeDef.Cardinality.SET &&
> 137 endDef1.getIsContainer()) {
> 138 throw new
> AtlasBaseException(AtlasErrorCode.RELATIONSHIPDEF_COMPOSITION_SET_CONTAINER,
> name);
> 139 }
> >>> CID 164610:(REVERSE_INULL)
> >>> Null-checking "endDef2" suggests that it may be null, but it has
> already been dereferenced on all paths leading to the check.
> 140 if (endDef2 != null && endDef2 != null &&
> 141 endDef2.getCardinality() ==
> AtlasAttributeDef.Cardinality.SET &&
> 142 endDef2.getIsContainer()) {
> 143 throw new
> AtlasBaseException(AtlasErrorCode.RELATIONSHIPDEF_COMPOSITION_SET_CONTAINER,
> name);
> 144 }
> 145 }
> /intg/src/main/java/org/apache/atlas/type/AtlasRelationshipType.java: 147
> in
> org.apache.atlas.type.AtlasRelationshipType.validateAtlasRelationshipDef(org.apache.atlas.model.typedef.AtlasRelationshipDef)()
> 141 endDef2.getCardinality() ==
> AtlasAttributeDef.Cardinality.SET &&
> 142 endDef2.getIsContainer()) {
> 143 throw new
> AtlasBaseException(AtlasErrorCode.RELATIONSHIPDEF_COMPOSITION_SET_CONTAINER,
> name);
> 144 }
> 145 }
> 146 if ((endDef1 != null && endDef1.getCardinality() ==
> AtlasAttributeDef.Cardinality.LIST) ||
> >>> CID 164610:(REVERSE_INULL)
> >>> Null-checking "endDef2" suggests that it may be null, but it has
> already been dereferenced on all paths leading to the check.
> 147 (endDef2 != null && endDef2.getCardinality() ==
> AtlasAttributeDef.Cardinality.LIST)) {
> 148 throw new
> AtlasBaseException(AtlasErrorCode.RELATIONSHIPDEF_LIST_ON_END, name);
> 149 }
> 150 }
>
> ** CID 164609:(REVERSE_INULL)
> /intg/src/main/java/org/apache/atlas/type/AtlasRelationshipType.java: 135
> in
> org.apache.atlas.type.AtlasRelationshipType.validateAtlasRelationshipDef(org.apache.atlas.model.typedef.AtlasRelationshipDef)()
> /intg/src/main/java/org/apache/atlas/type/AtlasRelationshipType.java: 146
> in
> org.apache.atlas.type.AtlasRelationshipType.validateAtlasRelationshipDef(org.
[jira] [Commented] (ATLAS-1887) Coverity Scan defects in AtlasRelationshipType
[
https://issues.apache.org/jira/browse/ATLAS-1887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16054897#comment-16054897
]
David Radley commented on ATLAS-1887:
-
https://reviews.apache.org/r/60213
> Coverity Scan defects in AtlasRelationshipType
> --
>
> Key: ATLAS-1887
> URL: https://issues.apache.org/jira/browse/ATLAS-1887
> Project: Atlas
> Issue Type: Bug
>Reporter: David Radley
>Assignee: David Radley
>
> 3 new defect(s) introduced to Apache Atlas found with Coverity Scan.
> 3 defect(s), reported by Coverity Scan earlier, were marked fixed in the
> recent build analyzed by Coverity Scan.
>
> New defect(s) Reported-by: Coverity Scan
> Showing 3 of 3 defect(s)
>
>
> ** CID 164610:(REVERSE_INULL)
> /intg/src/main/java/org/apache/atlas/type/AtlasRelationshipType.java: 140
> in
> org.apache.atlas.type.AtlasRelationshipType.validateAtlasRelationshipDef(org.apache.atlas.model.typedef.AtlasRelationshipDef)()
> /intg/src/main/java/org/apache/atlas/type/AtlasRelationshipType.java: 147
> in
> org.apache.atlas.type.AtlasRelationshipType.validateAtlasRelationshipDef(org.apache.atlas.model.typedef.AtlasRelationshipDef)()
>
>
>
>
> *** CID 164610:(REVERSE_INULL)
> /intg/src/main/java/org/apache/atlas/type/AtlasRelationshipType.java: 140
> in
> org.apache.atlas.type.AtlasRelationshipType.validateAtlasRelationshipDef(org.apache.atlas.model.typedef.AtlasRelationshipDef)()
> 134 // composition containers should not be multiple
> cardinality
> 135 if (endDef1 != null &&
> 136 endDef1.getCardinality() ==
> AtlasAttributeDef.Cardinality.SET &&
> 137 endDef1.getIsContainer()) {
> 138 throw new
> AtlasBaseException(AtlasErrorCode.RELATIONSHIPDEF_COMPOSITION_SET_CONTAINER,
> name);
> 139 }
> >>> CID 164610:(REVERSE_INULL)
> >>> Null-checking "endDef2" suggests that it may be null, but it has
> already been dereferenced on all paths leading to the check.
> 140 if (endDef2 != null && endDef2 != null &&
> 141 endDef2.getCardinality() ==
> AtlasAttributeDef.Cardinality.SET &&
> 142 endDef2.getIsContainer()) {
> 143 throw new
> AtlasBaseException(AtlasErrorCode.RELATIONSHIPDEF_COMPOSITION_SET_CONTAINER,
> name);
> 144 }
> 145 }
> /intg/src/main/java/org/apache/atlas/type/AtlasRelationshipType.java: 147
> in
> org.apache.atlas.type.AtlasRelationshipType.validateAtlasRelationshipDef(org.apache.atlas.model.typedef.AtlasRelationshipDef)()
> 141 endDef2.getCardinality() ==
> AtlasAttributeDef.Cardinality.SET &&
> 142 endDef2.getIsContainer()) {
> 143 throw new
> AtlasBaseException(AtlasErrorCode.RELATIONSHIPDEF_COMPOSITION_SET_CONTAINER,
> name);
> 144 }
> 145 }
> 146 if ((endDef1 != null && endDef1.getCardinality() ==
> AtlasAttributeDef.Cardinality.LIST) ||
> >>> CID 164610:(REVERSE_INULL)
> >>> Null-checking "endDef2" suggests that it may be null, but it has
> already been dereferenced on all paths leading to the check.
> 147 (endDef2 != null && endDef2.getCardinality() ==
> AtlasAttributeDef.Cardinality.LIST)) {
> 148 throw new
> AtlasBaseException(AtlasErrorCode.RELATIONSHIPDEF_LIST_ON_END, name);
> 149 }
> 150 }
>
> ** CID 164609:(REVERSE_INULL)
> /intg/src/main/java/org/apache/atlas/type/AtlasRelationshipType.java: 135
> in
> org.apache.atlas.type.AtlasRelationshipType.validateAtlasRelationshipDef(org.apache.atlas.model.typedef.AtlasRelationshipDef)()
> /intg/src/main/java/org/apache/atlas/type/AtlasRelationshipType.java: 146
> in
> org.apache.atlas.type.AtlasRelationshipType.validateAtlasRelationshipDef(org.apache.atlas.model.typedef.AtlasRelationshipDef)()
>
>
>
>
>
[jira] [Created] (ATLAS-1887) Coverity Scan defects in AtlasRelationshipType
David Radley created ATLAS-1887:
---
Summary: Coverity Scan defects in AtlasRelationshipType
Key: ATLAS-1887
URL: https://issues.apache.org/jira/browse/ATLAS-1887
Project: Atlas
Issue Type: Bug
Reporter: David Radley
Assignee: David Radley
3 new defect(s) introduced to Apache Atlas found with Coverity Scan.
3 defect(s), reported by Coverity Scan earlier, were marked fixed in the
recent build analyzed by Coverity Scan.
New defect(s) Reported-by: Coverity Scan
Showing 3 of 3 defect(s)
** CID 164610:(REVERSE_INULL)
/intg/src/main/java/org/apache/atlas/type/AtlasRelationshipType.java: 140
in
org.apache.atlas.type.AtlasRelationshipType.validateAtlasRelationshipDef(org.apache.atlas.model.typedef.AtlasRelationshipDef)()
/intg/src/main/java/org/apache/atlas/type/AtlasRelationshipType.java: 147
in
org.apache.atlas.type.AtlasRelationshipType.validateAtlasRelationshipDef(org.apache.atlas.model.typedef.AtlasRelationshipDef)()
*** CID 164610:(REVERSE_INULL)
/intg/src/main/java/org/apache/atlas/type/AtlasRelationshipType.java: 140
in
org.apache.atlas.type.AtlasRelationshipType.validateAtlasRelationshipDef(org.apache.atlas.model.typedef.AtlasRelationshipDef)()
134 // composition containers should not be multiple
cardinality
135 if (endDef1 != null &&
136 endDef1.getCardinality() ==
AtlasAttributeDef.Cardinality.SET &&
137 endDef1.getIsContainer()) {
138 throw new
AtlasBaseException(AtlasErrorCode.RELATIONSHIPDEF_COMPOSITION_SET_CONTAINER,
name);
139 }
>>> CID 164610:(REVERSE_INULL)
>>> Null-checking "endDef2" suggests that it may be null, but it has
already been dereferenced on all paths leading to the check.
140 if (endDef2 != null && endDef2 != null &&
141 endDef2.getCardinality() ==
AtlasAttributeDef.Cardinality.SET &&
142 endDef2.getIsContainer()) {
143 throw new
AtlasBaseException(AtlasErrorCode.RELATIONSHIPDEF_COMPOSITION_SET_CONTAINER,
name);
144 }
145 }
/intg/src/main/java/org/apache/atlas/type/AtlasRelationshipType.java: 147
in
org.apache.atlas.type.AtlasRelationshipType.validateAtlasRelationshipDef(org.apache.atlas.model.typedef.AtlasRelationshipDef)()
141 endDef2.getCardinality() ==
AtlasAttributeDef.Cardinality.SET &&
142 endDef2.getIsContainer()) {
143 throw new
AtlasBaseException(AtlasErrorCode.RELATIONSHIPDEF_COMPOSITION_SET_CONTAINER,
name);
144 }
145 }
146 if ((endDef1 != null && endDef1.getCardinality() ==
AtlasAttributeDef.Cardinality.LIST) ||
>>> CID 164610:(REVERSE_INULL)
>>> Null-checking "endDef2" suggests that it may be null, but it has
already been dereferenced on all paths leading to the check.
147 (endDef2 != null && endDef2.getCardinality() ==
AtlasAttributeDef.Cardinality.LIST)) {
148 throw new
AtlasBaseException(AtlasErrorCode.RELATIONSHIPDEF_LIST_ON_END, name);
149 }
150 }
** CID 164609:(REVERSE_INULL)
/intg/src/main/java/org/apache/atlas/type/AtlasRelationshipType.java: 135
in
org.apache.atlas.type.AtlasRelationshipType.validateAtlasRelationshipDef(org.apache.atlas.model.typedef.AtlasRelationshipDef)()
/intg/src/main/java/org/apache/atlas/type/AtlasRelationshipType.java: 146
in
org.apache.atlas.type.AtlasRelationshipType.validateAtlasRelationshipDef(org.apache.atlas.model.typedef.AtlasRelationshipDef)()
*** CID 164609:(REVERSE_INULL)
/intg/src/main/java/org/apache/atlas/type/AtlasRelationshipType.java: 135
in
org.apache.atlas.type.AtlasRelationshipType.validateAtlasRelationshipDef(org.apache.atlas.model.typedef.AtlasRelationshipDef)()
129 // AGGREGATION needs one end to be the container.
130 throw new
AtlasBaseException(AtlasErrorCode.RELATIONSHIPDEF_AGGREGATION_NO_CONTAINER,
name);
131 }
132 }
133 if (relationshipCategory ==
RelationshipCategory.COMPOSITION) {
134 // composition containers should not be multiple
cardinality
>>> CID 1646
[jira] [Updated] (ATLAS-1876) Import/Export : AtlasSchemaViolationException during Import of entity associated to a tag with tag attribute value of datatype float
[
https://issues.apache.org/jira/browse/ATLAS-1876?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ashutosh Mestry updated ATLAS-1876:
---
Attachment: ATLAS-1876-Type-range-check-added.patch
> Import/Export : AtlasSchemaViolationException during Import of entity
> associated to a tag with tag attribute value of datatype float
>
>
> Key: ATLAS-1876
> URL: https://issues.apache.org/jira/browse/ATLAS-1876
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Affects Versions: 0.9-incubating
>Reporter: Sharmadha Sainath
>Assignee: Ashutosh Mestry
> Attachments: ATLAS-1876-Type-range-check-added.patch,
> AtlasSchemaViolationException_ExportImport.txt
>
>
> 1.In cluster1, created a tag float_tag with attribute float_max of datatype
> float.
> 2. In the same cluster , created table table1 and associated the tag
> float_tag to table1 , with attribute value for float_max as 3.4028235e+38
> (maximum value for float).
> 3.Creation and tag association are successful.
> 4.Created table1.zip by using Export API
> 5. Using Import API , tried to import table1.zip to cluster2. Import failed
> with
> {code}
> {"errorCode":"ATLAS-500-00-001","errorMessage":"org.apache.atlas.exception.AtlasBaseException:
> org.apache.atlas.repository.graphdb.AtlasSchemaViolationException:
> com.thinkaurelius.titan.core.SchemaViolationException: Value [3.4028235E38]
> is not an instance of the expected data type for property key
> [float_tag.float_max] and cannot be converted. Expected: class
> java.lang.Float, found: class java.lang.Double"}
> {code}
> Attached the error stack trace found in cluster2 application logs.
> This exception is seen only when the value for float attribute is given as
> 3.4028235e+38 . Other values like 0 , 2.5 , 1.4e+10 etc., are accepted and
> import is done successfully.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (ATLAS-1879) Updating classification removes some properties
[
https://issues.apache.org/jira/browse/ATLAS-1879?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Péter Gergő Barna updated ATLAS-1879:
-
Attachment: Atlas-1789.postman_collection.json
Attached a postman session which can be used to reproduce the error
> Updating classification removes some properties
> ---
>
> Key: ATLAS-1879
> URL: https://issues.apache.org/jira/browse/ATLAS-1879
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Affects Versions: 0.8-incubating
>Reporter: Laura Ngo
> Attachments: Atlas-1789.postman_collection.json
>
>
> * Created classification via POST.
> * Updated via PUT
> * Lost properties
> POST http://127.0.0.1:21000/api/atlas/v2/types/typedefs
> {code}
> {
> "classificationDefs": [{
> "name": "test_classification_11",
> "description": "",
> "createdBy" : "admin",
> "superTypes": [],
> "attributeDefs": [{
> "name" : "test_class_11",
> "typeName" : "string",
> "isOptional" : true,
> "isUnique" : true,
> "isIndexable" : true,
> "cardinality": "SINGLE",
> "valuesMinCount": 0,
> "valuesMaxCount": 1
> }]
> }],
> "entityDefs": [],
> "enumDefs": [],
> "structDefs": []
> }
> {code}
> GET
> http://127.0.0.1:21000/api/atlas/v2/types/classification/name/test_classification_11
> {code}
> {
> "category": "CLASSIFICATION",
> "guid": "83162fe1-4bb4-4a87-b2b8-364e751a1265",
> "createdBy": "admin",
> "createTime": 1497485890857,
> "updateTime": 1497485890857,
> "version": 1,
> "name": "test_classification_11",
> "description": "",
> "typeVersion": "1.0",
> "attributeDefs": [
> {
> "name": "test_class_11",
> "typeName": "string",
> "isOptional": true,
> "cardinality": "SINGLE",
> "valuesMinCount": 0,
> "valuesMaxCount": 1,
> "isUnique": true,
> "isIndexable": true
> }
> ],
> "superTypes": [],
> }
> {code}
> PUT http://127.0.0.1:21000/api/atlas/v2/types/typedefs
> Update attribute.
> GET
> http://127.0.0.1:21000/api/atlas/v2/types/classification/name/test_classification_11
> {code}
> {
> "category": "CLASSIFICATION",
> "createdBy": "admin",
> "name": "test_classification_11",
> "description": "",
> "attributeDefs": [
> {
> "name": "test_class_11",
> "typeName": "string",
> "isOptional": true,
> "cardinality": "SINGLE",
> "valuesMinCount": 0,
> "valuesMaxCount": 1,
> "isUnique": true,
> "isIndexable": false
> }
> ],
> "superTypes": [],
> }
> {code}
> Some properties are missing after PUT update of attribute "isIndexable"
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (ATLAS-1781) Atlas UI cannot show lineage pic when inputs and outputs of lineage are the same entity
[
https://issues.apache.org/jira/browse/ATLAS-1781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16054074#comment-16054074
]
Xinzhi,Luo commented on ATLAS-1781:
---
[~kevalbhatt] Hi, I fixed the problem, now the arrow color should be red,
please take a look at the 0001-fix-not-working-with-more-than-2-nodes.patch and
the attached pics pic1.png and pic2.png.
> Atlas UI cannot show lineage pic when inputs and outputs of lineage are the
> same entity
> ---
>
> Key: ATLAS-1781
> URL: https://issues.apache.org/jira/browse/ATLAS-1781
> Project: Atlas
> Issue Type: Bug
> Components: atlas-webui
>Affects Versions: trunk, 0.7-incubating, 0.8-incubating, 0.7.1-incubating,
> 0.9-incubating
>Reporter: qinglin,xia
>Assignee: Xinzhi,Luo
> Labels: atlas
> Fix For: 0.9-incubating
>
> Attachments: 0001-fix-not-working-with-more-than-2-nodes.patch,
> after_test-wrong_op.png, browser_error.png, expected.png, fixed.png, Linage
> fix not working with more than 2 nodes with ATLAS-1781.png,
> Lineage_Create_Successfully_when_inputs_outputs_different.png,
> Lineage_keep_loading.png, pic1.png, pic2.png, Process_Type_Not_Blue.png,
> Starting_point_is_Impact_than_icon&arrow_colour_is_wrong(KEVAL_2_review).jpg
>
>
> I was working with the lineage for hbase, when I added a column family in a
> hbase table, I want to create a lineage for the table to show there is a
> change within the table schema, yet when I create a process using the atlas
> api, I found that the process entity is successfully created, but the lineage
> is not shown on the atlas web page.
> Steps to Reproduce this issue is:
> 1. build up an entity extends the DataSet Type
> 2. build up a lineage Process
> 3. set the process input and output to the same entity
> The code is like:
> Referenceable table1 =
> atlasClient.getEntity(HBaseDataTypes.HBASE_TABLE_TYPE_NAME,
> AtlasClient.REFERENCEABLE_ATTRIBUTE_NAME, input_table_name);
> Referenceable table2 =
> atlasClient.getEntity(HBaseDataTypes.HBASE_TABLE_TYPE_NAME,
> AtlasClient.REFERENCEABLE_ATTRIBUTE_NAME, output_table_name);
> HBaseProcess process = new HBaseProcess("process");
> process.setInput(table1.getId());
> process.setoutput(table2.getId());
> here I set the input_table_name and output_table_name the same string name,
> when I set them different values, the lineage pic can be shown for table1 and
> table2 successfully
> Screenshots are:
> 1) The lineage pic keep loading on the atlas UI
> 2) The error log reported by the browser console
> 3) The lineage pic showed successfully when set the inputs and outputs of the
> lineage to different entities.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (ATLAS-1781) Atlas UI cannot show lineage pic when inputs and outputs of lineage are the same entity
[
https://issues.apache.org/jira/browse/ATLAS-1781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xinzhi,Luo updated ATLAS-1781:
--
Attachment: pic1.png
pic2.png
> Atlas UI cannot show lineage pic when inputs and outputs of lineage are the
> same entity
> ---
>
> Key: ATLAS-1781
> URL: https://issues.apache.org/jira/browse/ATLAS-1781
> Project: Atlas
> Issue Type: Bug
> Components: atlas-webui
>Affects Versions: trunk, 0.7-incubating, 0.8-incubating, 0.7.1-incubating,
> 0.9-incubating
>Reporter: qinglin,xia
>Assignee: Xinzhi,Luo
> Labels: atlas
> Fix For: 0.9-incubating
>
> Attachments: 0001-fix-not-working-with-more-than-2-nodes.patch,
> after_test-wrong_op.png, browser_error.png, expected.png, fixed.png, Linage
> fix not working with more than 2 nodes with ATLAS-1781.png,
> Lineage_Create_Successfully_when_inputs_outputs_different.png,
> Lineage_keep_loading.png, pic1.png, pic2.png, Process_Type_Not_Blue.png,
> Starting_point_is_Impact_than_icon&arrow_colour_is_wrong(KEVAL_2_review).jpg
>
>
> I was working with the lineage for hbase, when I added a column family in a
> hbase table, I want to create a lineage for the table to show there is a
> change within the table schema, yet when I create a process using the atlas
> api, I found that the process entity is successfully created, but the lineage
> is not shown on the atlas web page.
> Steps to Reproduce this issue is:
> 1. build up an entity extends the DataSet Type
> 2. build up a lineage Process
> 3. set the process input and output to the same entity
> The code is like:
> Referenceable table1 =
> atlasClient.getEntity(HBaseDataTypes.HBASE_TABLE_TYPE_NAME,
> AtlasClient.REFERENCEABLE_ATTRIBUTE_NAME, input_table_name);
> Referenceable table2 =
> atlasClient.getEntity(HBaseDataTypes.HBASE_TABLE_TYPE_NAME,
> AtlasClient.REFERENCEABLE_ATTRIBUTE_NAME, output_table_name);
> HBaseProcess process = new HBaseProcess("process");
> process.setInput(table1.getId());
> process.setoutput(table2.getId());
> here I set the input_table_name and output_table_name the same string name,
> when I set them different values, the lineage pic can be shown for table1 and
> table2 successfully
> Screenshots are:
> 1) The lineage pic keep loading on the atlas UI
> 2) The error log reported by the browser console
> 3) The lineage pic showed successfully when set the inputs and outputs of the
> lineage to different entities.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (ATLAS-1781) Atlas UI cannot show lineage pic when inputs and outputs of lineage are the same entity
[
https://issues.apache.org/jira/browse/ATLAS-1781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xinzhi,Luo updated ATLAS-1781:
--
Attachment: 0001-fix-not-working-with-more-than-2-nodes.patch
> Atlas UI cannot show lineage pic when inputs and outputs of lineage are the
> same entity
> ---
>
> Key: ATLAS-1781
> URL: https://issues.apache.org/jira/browse/ATLAS-1781
> Project: Atlas
> Issue Type: Bug
> Components: atlas-webui
>Affects Versions: trunk, 0.7-incubating, 0.8-incubating, 0.7.1-incubating,
> 0.9-incubating
>Reporter: qinglin,xia
>Assignee: Xinzhi,Luo
> Labels: atlas
> Fix For: 0.9-incubating
>
> Attachments: 0001-fix-not-working-with-more-than-2-nodes.patch,
> after_test-wrong_op.png, browser_error.png, expected.png, fixed.png, Linage
> fix not working with more than 2 nodes with ATLAS-1781.png,
> Lineage_Create_Successfully_when_inputs_outputs_different.png,
> Lineage_keep_loading.png, Process_Type_Not_Blue.png,
> Starting_point_is_Impact_than_icon&arrow_colour_is_wrong(KEVAL_2_review).jpg
>
>
> I was working with the lineage for hbase, when I added a column family in a
> hbase table, I want to create a lineage for the table to show there is a
> change within the table schema, yet when I create a process using the atlas
> api, I found that the process entity is successfully created, but the lineage
> is not shown on the atlas web page.
> Steps to Reproduce this issue is:
> 1. build up an entity extends the DataSet Type
> 2. build up a lineage Process
> 3. set the process input and output to the same entity
> The code is like:
> Referenceable table1 =
> atlasClient.getEntity(HBaseDataTypes.HBASE_TABLE_TYPE_NAME,
> AtlasClient.REFERENCEABLE_ATTRIBUTE_NAME, input_table_name);
> Referenceable table2 =
> atlasClient.getEntity(HBaseDataTypes.HBASE_TABLE_TYPE_NAME,
> AtlasClient.REFERENCEABLE_ATTRIBUTE_NAME, output_table_name);
> HBaseProcess process = new HBaseProcess("process");
> process.setInput(table1.getId());
> process.setoutput(table2.getId());
> here I set the input_table_name and output_table_name the same string name,
> when I set them different values, the lineage pic can be shown for table1 and
> table2 successfully
> Screenshots are:
> 1) The lineage pic keep loading on the atlas UI
> 2) The error log reported by the browser console
> 3) The lineage pic showed successfully when set the inputs and outputs of the
> lineage to different entities.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (ATLAS-1781) Atlas UI cannot show lineage pic when inputs and outputs of lineage are the same entity
[
https://issues.apache.org/jira/browse/ATLAS-1781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xinzhi,Luo updated ATLAS-1781:
--
Attachment: (was: 0001-fix-ATLAS-1781.patch)
> Atlas UI cannot show lineage pic when inputs and outputs of lineage are the
> same entity
> ---
>
> Key: ATLAS-1781
> URL: https://issues.apache.org/jira/browse/ATLAS-1781
> Project: Atlas
> Issue Type: Bug
> Components: atlas-webui
>Affects Versions: trunk, 0.7-incubating, 0.8-incubating, 0.7.1-incubating,
> 0.9-incubating
>Reporter: qinglin,xia
>Assignee: Xinzhi,Luo
> Labels: atlas
> Fix For: 0.9-incubating
>
> Attachments: after_test-wrong_op.png, browser_error.png,
> expected.png, fixed.png, Linage fix not working with more than 2 nodes with
> ATLAS-1781.png,
> Lineage_Create_Successfully_when_inputs_outputs_different.png,
> Lineage_keep_loading.png, Process_Type_Not_Blue.png,
> Starting_point_is_Impact_than_icon&arrow_colour_is_wrong(KEVAL_2_review).jpg
>
>
> I was working with the lineage for hbase, when I added a column family in a
> hbase table, I want to create a lineage for the table to show there is a
> change within the table schema, yet when I create a process using the atlas
> api, I found that the process entity is successfully created, but the lineage
> is not shown on the atlas web page.
> Steps to Reproduce this issue is:
> 1. build up an entity extends the DataSet Type
> 2. build up a lineage Process
> 3. set the process input and output to the same entity
> The code is like:
> Referenceable table1 =
> atlasClient.getEntity(HBaseDataTypes.HBASE_TABLE_TYPE_NAME,
> AtlasClient.REFERENCEABLE_ATTRIBUTE_NAME, input_table_name);
> Referenceable table2 =
> atlasClient.getEntity(HBaseDataTypes.HBASE_TABLE_TYPE_NAME,
> AtlasClient.REFERENCEABLE_ATTRIBUTE_NAME, output_table_name);
> HBaseProcess process = new HBaseProcess("process");
> process.setInput(table1.getId());
> process.setoutput(table2.getId());
> here I set the input_table_name and output_table_name the same string name,
> when I set them different values, the lineage pic can be shown for table1 and
> table2 successfully
> Screenshots are:
> 1) The lineage pic keep loading on the atlas UI
> 2) The error log reported by the browser console
> 3) The lineage pic showed successfully when set the inputs and outputs of the
> lineage to different entities.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (ATLAS-1781) Atlas UI cannot show lineage pic when inputs and outputs of lineage are the same entity
[
https://issues.apache.org/jira/browse/ATLAS-1781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xinzhi,Luo updated ATLAS-1781:
--
Attachment: (was:
0001-fix-not-working-with-more-than-2-nodes-with-ATLAS-17.patch)
> Atlas UI cannot show lineage pic when inputs and outputs of lineage are the
> same entity
> ---
>
> Key: ATLAS-1781
> URL: https://issues.apache.org/jira/browse/ATLAS-1781
> Project: Atlas
> Issue Type: Bug
> Components: atlas-webui
>Affects Versions: trunk, 0.7-incubating, 0.8-incubating, 0.7.1-incubating,
> 0.9-incubating
>Reporter: qinglin,xia
>Assignee: Xinzhi,Luo
> Labels: atlas
> Fix For: 0.9-incubating
>
> Attachments: 0001-fix-ATLAS-1781.patch, after_test-wrong_op.png,
> browser_error.png, expected.png, fixed.png, Linage fix not working with more
> than 2 nodes with ATLAS-1781.png,
> Lineage_Create_Successfully_when_inputs_outputs_different.png,
> Lineage_keep_loading.png, Process_Type_Not_Blue.png,
> Starting_point_is_Impact_than_icon&arrow_colour_is_wrong(KEVAL_2_review).jpg
>
>
> I was working with the lineage for hbase, when I added a column family in a
> hbase table, I want to create a lineage for the table to show there is a
> change within the table schema, yet when I create a process using the atlas
> api, I found that the process entity is successfully created, but the lineage
> is not shown on the atlas web page.
> Steps to Reproduce this issue is:
> 1. build up an entity extends the DataSet Type
> 2. build up a lineage Process
> 3. set the process input and output to the same entity
> The code is like:
> Referenceable table1 =
> atlasClient.getEntity(HBaseDataTypes.HBASE_TABLE_TYPE_NAME,
> AtlasClient.REFERENCEABLE_ATTRIBUTE_NAME, input_table_name);
> Referenceable table2 =
> atlasClient.getEntity(HBaseDataTypes.HBASE_TABLE_TYPE_NAME,
> AtlasClient.REFERENCEABLE_ATTRIBUTE_NAME, output_table_name);
> HBaseProcess process = new HBaseProcess("process");
> process.setInput(table1.getId());
> process.setoutput(table2.getId());
> here I set the input_table_name and output_table_name the same string name,
> when I set them different values, the lineage pic can be shown for table1 and
> table2 successfully
> Screenshots are:
> 1) The lineage pic keep loading on the atlas UI
> 2) The error log reported by the browser console
> 3) The lineage pic showed successfully when set the inputs and outputs of the
> lineage to different entities.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Comment Edited] (ATLAS-1877) UI: In Lineage for Process entity process icon is not showing.
[
https://issues.apache.org/jira/browse/ATLAS-1877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16053823#comment-16053823
]
qinglin,xia edited comment on ATLAS-1877 at 6/19/17 11:33 AM:
--
Haved attached the revised patch.
The attached pic shows the result of creating a process entity of the nested
type grand_child_loadProcess
{code:java}
{
"enumTypes": [],
"structTypes": [],
"traitTypes": [],
"classTypes": [{
"superTypes": ["child_loadProcess"],
"hierarchicalMetaTypeName": "org.apache.atlas.typesystem.types.ClassType",
"typeName": "grand_child_loadProcess",
"typeDescription": null,
"attributeDefinitions": []
}]
}
{code}
and child_loadProcess is a child of loadProcess type.
was (Author: xiaqinglin):
Haved attached the revised patch.
The attached pic shows the result of creating a process entity of the nested
type grand_child_loadProcess
{
"enumTypes": [],
"structTypes": [],
"traitTypes": [],
"classTypes": [{
"superTypes": ["child_loadProcess"],
"hierarchicalMetaTypeName": "org.apache.atlas.typesystem.types.ClassType",
"typeName": "grand_child_loadProcess",
"typeDescription": null,
"attributeDefinitions": []
}]
}
and child_loadProcess is a child of loadProcess type.
> UI: In Lineage for Process entity process icon is not showing.
> --
>
> Key: ATLAS-1877
> URL: https://issues.apache.org/jira/browse/ATLAS-1877
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Affects Versions: 0.8-incubating
>Reporter: Keval Bhatt
>Assignee: qinglin,xia
> Labels: atlas
> Fix For: trunk, 0.9-incubating
>
> Attachments: 0001-Fix-ATLAS-1877-check-nested-supertype.patch,
> 0001-fix-ATLAS-1877.patch, grand_child_load_process.png,
> Process_Icon_Fixed.png, wrong_process_icon.png
>
>
> UI should check Process entity in all parent superType, right know its
> checking for only 1 level
> Step to reproduce :
> # Run quickstart
> # Create entity of type Process and while creating process select time_dim
> (Table entity) as a input and output.
> # Now got to sales_fact_monthly_mv entity and you will see new output (Please
> find attached screenshot )
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (ATLAS-1877) UI: In Lineage for Process entity process icon is not showing.
[
https://issues.apache.org/jira/browse/ATLAS-1877?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
qinglin,xia updated ATLAS-1877:
---
Attachment: 0001-Fix-ATLAS-1877-check-nested-supertype.patch
grand_child_load_process.png
Component/s: atlas-core
Haved attached the revised patch.
The attached pic shows the result of creating a process entity of the nested
type grand_child_loadProcess
{
"enumTypes": [],
"structTypes": [],
"traitTypes": [],
"classTypes": [{
"superTypes": ["child_loadProcess"],
"hierarchicalMetaTypeName": "org.apache.atlas.typesystem.types.ClassType",
"typeName": "grand_child_loadProcess",
"typeDescription": null,
"attributeDefinitions": []
}]
}
and child_loadProcess is a child of loadProcess type.
> UI: In Lineage for Process entity process icon is not showing.
> --
>
> Key: ATLAS-1877
> URL: https://issues.apache.org/jira/browse/ATLAS-1877
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Affects Versions: 0.8-incubating
>Reporter: Keval Bhatt
>Assignee: qinglin,xia
> Labels: atlas
> Fix For: trunk, 0.9-incubating
>
> Attachments: 0001-Fix-ATLAS-1877-check-nested-supertype.patch,
> 0001-fix-ATLAS-1877.patch, grand_child_load_process.png,
> Process_Icon_Fixed.png, wrong_process_icon.png
>
>
> UI should check Process entity in all parent superType, right know its
> checking for only 1 level
> Step to reproduce :
> # Run quickstart
> # Create entity of type Process and while creating process select time_dim
> (Table entity) as a input and output.
> # Now got to sales_fact_monthly_mv entity and you will see new output (Please
> find attached screenshot )
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (ATLAS-1880) Search using entity and trait attributes
[ https://issues.apache.org/jira/browse/ATLAS-1880?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16053656#comment-16053656 ] Graham Wallis commented on ATLAS-1880: -- Will this mean that we don't need to fix issue ATLAS-1868? > Search using entity and trait attributes > > > Key: ATLAS-1880 > URL: https://issues.apache.org/jira/browse/ATLAS-1880 > Project: Atlas > Issue Type: Improvement >Reporter: Apoorv Naik >Assignee: Apoorv Naik > Attachments: > 0001-ATLAS-1880-Search-using-entity-and-trait-attributes.patch > > > This improvement patch enhances the search experience provided by Atlas. > Currently the basic search only supports type, tag and lucene type query. > New search API will allow complex combinations of attribute search (ANDs / > ORs). The first step will be to use titan's index query on the vertex index > and then subsequently use gremlin to narrow the results if any non-indexed > attributes show up in the search request -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (ATLAS-1886) DSL queries with LIKE fail for composite searches
[ https://issues.apache.org/jira/browse/ATLAS-1886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Madhan Neethiraj updated ATLAS-1886: Attachment: ATLAS-1866.patch > DSL queries with LIKE fail for composite searches > - > > Key: ATLAS-1886 > URL: https://issues.apache.org/jira/browse/ATLAS-1886 > Project: Atlas > Issue Type: Improvement > Components: atlas-core >Affects Versions: 0.9-incubating >Reporter: Hemanth Yamijala >Assignee: Madhan Neethiraj > Fix For: 0.9-incubating > > Attachments: ATLAS-1866.patch > > > Search DSL was enhanced to support the LIKE query in ATLAS-1807. This was a > great enhancement from a usability point of view and works as expected for > String attributes of a type - like hive_table.name, hive_table.owner etc. > Another type of query that is useful is composite searches - for e.g. > hive_tables in a DB (given a DB name). This query currently works - > hive_table where db.name='xyz'. > However, when we try a 'like' operator here, it fails - i.e. hive_table where > db.name LIKE '*xyz*' - this returns an error: Discovery query failed > `hive_table` db.name LIKE '*financ*' > Can this be looked into if its an easy fix? -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Assigned] (ATLAS-1886) DSL queries with LIKE fail for composite searches
[ https://issues.apache.org/jira/browse/ATLAS-1886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Madhan Neethiraj reassigned ATLAS-1886: --- Assignee: Madhan Neethiraj > DSL queries with LIKE fail for composite searches > - > > Key: ATLAS-1886 > URL: https://issues.apache.org/jira/browse/ATLAS-1886 > Project: Atlas > Issue Type: Improvement > Components: atlas-core >Affects Versions: 0.9-incubating >Reporter: Hemanth Yamijala >Assignee: Madhan Neethiraj > > Search DSL was enhanced to support the LIKE query in ATLAS-1807. This was a > great enhancement from a usability point of view and works as expected for > String attributes of a type - like hive_table.name, hive_table.owner etc. > Another type of query that is useful is composite searches - for e.g. > hive_tables in a DB (given a DB name). This query currently works - > hive_table where db.name='xyz'. > However, when we try a 'like' operator here, it fails - i.e. hive_table where > db.name LIKE '*xyz*' - this returns an error: Discovery query failed > `hive_table` db.name LIKE '*financ*' > Can this be looked into if its an easy fix? -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Resolved] (ATLAS-1885) DELETE classification causes error message
[
https://issues.apache.org/jira/browse/ATLAS-1885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Madhan Neethiraj resolved ATLAS-1885.
-
Resolution: Duplicate
This issue was fixed in ATLAS-1673; the fix will be available in next release
from master and 0.8-incubating branches.
> DELETE classification causes error message
> --
>
> Key: ATLAS-1885
> URL: https://issues.apache.org/jira/browse/ATLAS-1885
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Affects Versions: 0.8-incubating
>Reporter: Laura Ngo
>
> * Created classification via POST (/v2/types/typedefs)
> * Tried to delete it via DELETE (/v2/types/typedefs)
> * Get 204 no content response (should have been removed)
> * Go to UI tag is still there
> * Get error in UI
> Discovery query failed g.V().has('__traitNames', T.in, traitNames)
> [startIdx.. {code}
> {
> "classificationDefs": [{
> "name": "test_classification_10",
> "superTypes": [],
> "attributeDefs": []
> }],
> "entityDefs": [],
> "enumDefs": [],
> "structDefs": []
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (ATLAS-1878) NPE when a request without any query path lands on atlas
[
https://issues.apache.org/jira/browse/ATLAS-1878?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16053223#comment-16053223
]
Nixon Rodrigues commented on ATLAS-1878:
Thanks [~sarath.ku...@gmail.com] and [~apoorvnaik] for the review.
Committed patch on following brach
* master ->
http://git-wip-us.apache.org/repos/asf/incubator-atlas/commit/906f3651
* 0.8-incubating ->
http://git-wip-us.apache.org/repos/asf/incubator-atlas/commit/fca6855e
> NPE when a request without any query path lands on atlas
>
>
> Key: ATLAS-1878
> URL: https://issues.apache.org/jira/browse/ATLAS-1878
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Affects Versions: 0.9-incubating
>Reporter: Ayub Pathan
>Assignee: Nixon Rodrigues
>Priority: Critical
> Fix For: 0.9-incubating, 0.8.1-incubating
>
> Attachments: ATLAS-1878.patch
>
>
> {noformat}
> 2017-06-16 07:02:13,515 WARN - [pool-2-thread-10:] ~ got exception trying to
> get groups for user admin: id: admin: no such user
> id: admin: no such user
> (ShellBasedUnixGroupsMapping:87)
> 2017-06-16 07:02:13,515 ERROR - [pool-2-thread-10:] ~ Exception while
> fetching groups (AtlasAbstractAuthenticationProvider:120)
> java.io.IOException: No groups found for user admin
> at org.apache.hadoop.security.Groups.noGroupsForUser(Groups.java:190)
> at org.apache.hadoop.security.Groups.access$400(Groups.java:69)
> at org.apache.hadoop.security.Groups$GroupCacheLoader.load(Groups.java:307)
> at org.apache.hadoop.security.Groups$GroupCacheLoader.load(Groups.java:257)
> at
> com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3542)
> at com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2323)
> at
> com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2286)
> at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2201)
> at com.google.common.cache.LocalCache.get(LocalCache.java:3953)
> at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3957)
> at
> com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4875)
> at org.apache.hadoop.security.Groups.getGroups(Groups.java:215)
> at
> org.apache.atlas.web.security.AtlasAbstractAuthenticationProvider.getAuthoritiesFromUGI(AtlasAbstractAuthenticationProvider.java:113)
> at
> org.apache.atlas.web.filters.AtlasKnoxSSOAuthenticationFilter.doFilter(AtlasKnoxSSOAuthenticationFilter.java:171)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
> at
> org.apache.atlas.web.filters.StaleTransactionCleanupFilter.doFilter(StaleTransactionCleanupFilter.java:55)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
> at
> org.apache.atlas.web.filters.ActiveServerFilter.doFilter(ActiveServerFilter.java:85)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
> at
> org.springframework.security.web.authentication.www.BasicAuthenticationFilter.doFilterInternal(BasicAuthenticationFilter.java:158)
> at
> org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
> at
> org.springframework.security.web.authentication.AbstractAuthenticationProcessingFilter.doFilter(AbstractAuthenticationProcessingFilter.java:200)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
> at
> org.springframework.security.web.authentication.logout.LogoutFilter.doFilter(LogoutFilter.java:116)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
> at
> org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:105)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
> at
> org.springframework.security.web.context.request.async.WebAsyncManagerIntegrationFilter.doFilterInternal(WebAsyncManagerIntegrationFilter.java:56)
> at
> org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
> at
> org.springframework.security.web.FilterChainProxy.
[jira] [Updated] (ATLAS-1878) NPE when a request without any query path lands on atlas
[
https://issues.apache.org/jira/browse/ATLAS-1878?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nixon Rodrigues updated ATLAS-1878:
---
Fix Version/s: 0.8.1-incubating
> NPE when a request without any query path lands on atlas
>
>
> Key: ATLAS-1878
> URL: https://issues.apache.org/jira/browse/ATLAS-1878
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Affects Versions: 0.9-incubating
>Reporter: Ayub Pathan
>Assignee: Nixon Rodrigues
>Priority: Critical
> Fix For: 0.9-incubating, 0.8.1-incubating
>
> Attachments: ATLAS-1878.patch
>
>
> {noformat}
> 2017-06-16 07:02:13,515 WARN - [pool-2-thread-10:] ~ got exception trying to
> get groups for user admin: id: admin: no such user
> id: admin: no such user
> (ShellBasedUnixGroupsMapping:87)
> 2017-06-16 07:02:13,515 ERROR - [pool-2-thread-10:] ~ Exception while
> fetching groups (AtlasAbstractAuthenticationProvider:120)
> java.io.IOException: No groups found for user admin
> at org.apache.hadoop.security.Groups.noGroupsForUser(Groups.java:190)
> at org.apache.hadoop.security.Groups.access$400(Groups.java:69)
> at org.apache.hadoop.security.Groups$GroupCacheLoader.load(Groups.java:307)
> at org.apache.hadoop.security.Groups$GroupCacheLoader.load(Groups.java:257)
> at
> com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3542)
> at com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2323)
> at
> com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2286)
> at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2201)
> at com.google.common.cache.LocalCache.get(LocalCache.java:3953)
> at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3957)
> at
> com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4875)
> at org.apache.hadoop.security.Groups.getGroups(Groups.java:215)
> at
> org.apache.atlas.web.security.AtlasAbstractAuthenticationProvider.getAuthoritiesFromUGI(AtlasAbstractAuthenticationProvider.java:113)
> at
> org.apache.atlas.web.filters.AtlasKnoxSSOAuthenticationFilter.doFilter(AtlasKnoxSSOAuthenticationFilter.java:171)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
> at
> org.apache.atlas.web.filters.StaleTransactionCleanupFilter.doFilter(StaleTransactionCleanupFilter.java:55)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
> at
> org.apache.atlas.web.filters.ActiveServerFilter.doFilter(ActiveServerFilter.java:85)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
> at
> org.springframework.security.web.authentication.www.BasicAuthenticationFilter.doFilterInternal(BasicAuthenticationFilter.java:158)
> at
> org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
> at
> org.springframework.security.web.authentication.AbstractAuthenticationProcessingFilter.doFilter(AbstractAuthenticationProcessingFilter.java:200)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
> at
> org.springframework.security.web.authentication.logout.LogoutFilter.doFilter(LogoutFilter.java:116)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
> at
> org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:105)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
> at
> org.springframework.security.web.context.request.async.WebAsyncManagerIntegrationFilter.doFilterInternal(WebAsyncManagerIntegrationFilter.java:56)
> at
> org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
> at
> org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:214)
> at
> org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:177)
> at
> org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:346)
> at
> org.springframew
[jira] [Created] (ATLAS-1886) DSL queries with LIKE fail for composite searches
Hemanth Yamijala created ATLAS-1886: --- Summary: DSL queries with LIKE fail for composite searches Key: ATLAS-1886 URL: https://issues.apache.org/jira/browse/ATLAS-1886 Project: Atlas Issue Type: Improvement Components: atlas-core Affects Versions: 0.9-incubating Reporter: Hemanth Yamijala Search DSL was enhanced to support the LIKE query in ATLAS-1807. This was a great enhancement from a usability point of view and works as expected for String attributes of a type - like hive_table.name, hive_table.owner etc. Another type of query that is useful is composite searches - for e.g. hive_tables in a DB (given a DB name). This query currently works - hive_table where db.name='xyz'. However, when we try a 'like' operator here, it fails - i.e. hive_table where db.name LIKE '*xyz*' - this returns an error: Discovery query failed `hive_table` db.name LIKE '*financ*' Can this be looked into if its an easy fix? -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (ATLAS-1885) DELETE classification causes error message
[
https://issues.apache.org/jira/browse/ATLAS-1885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16053141#comment-16053141
]
Laura Ngo commented on ATLAS-1885:
--
I was doing my tests on the HDP 2.6 sandbox (at work 2.6 is live so when our
system engineers get Atlas to work we'll also use that version, plus some
patches.)
I'll try now to follow the instructions
[here|http://atlas.incubator.apache.org/EclipseSetup.html] to get the branches
you suggest. (I'm a business user with limited technical experience, so this is
all new to me!)
> DELETE classification causes error message
> --
>
> Key: ATLAS-1885
> URL: https://issues.apache.org/jira/browse/ATLAS-1885
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Affects Versions: 0.8-incubating
>Reporter: Laura Ngo
>
> * Created classification via POST (/v2/types/typedefs)
> * Tried to delete it via DELETE (/v2/types/typedefs)
> * Get 204 no content response (should have been removed)
> * Go to UI tag is still there
> * Get error in UI
> Discovery query failed g.V().has('__traitNames', T.in, traitNames)
> [startIdx.. {code}
> {
> "classificationDefs": [{
> "name": "test_classification_10",
> "superTypes": [],
> "attributeDefs": []
> }],
> "entityDefs": [],
> "enumDefs": [],
> "structDefs": []
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (ATLAS-1885) DELETE classification causes error message
[
https://issues.apache.org/jira/browse/ATLAS-1885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16053076#comment-16053076
]
Madhan Neethiraj commented on ATLAS-1885:
-
[~LauraNgo] - this issue looks to be a duplicate of ATLAS-1673, which has been
fixed in master and 0.8-incubting branches. But the fix is not present in
0.8-incubating release. Can you try on the latest in 0.8-incubating branch or
master branch?
> DELETE classification causes error message
> --
>
> Key: ATLAS-1885
> URL: https://issues.apache.org/jira/browse/ATLAS-1885
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Affects Versions: 0.8-incubating
>Reporter: Laura Ngo
>
> * Created classification via POST (/v2/types/typedefs)
> * Tried to delete it via DELETE (/v2/types/typedefs)
> * Get 204 no content response (should have been removed)
> * Go to UI tag is still there
> * Get error in UI
> Discovery query failed g.V().has('__traitNames', T.in, traitNames)
> [startIdx.. {code}
> {
> "classificationDefs": [{
> "name": "test_classification_10",
> "superTypes": [],
> "attributeDefs": []
> }],
> "entityDefs": [],
> "enumDefs": [],
> "structDefs": []
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (ATLAS-1879) Updating classification removes some properties
[
https://issues.apache.org/jira/browse/ATLAS-1879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16053029#comment-16053029
]
Madhan Neethiraj commented on ATLAS-1879:
-
[~LauraNgo] - thanks for the details. I see the issue now - fields that are not
included in the PUT call (like guid, createTime, updateTime, version in this
case) are not returned in subsequent GET calls. The value of these fields are
in fact present in the store, but they are not present in the cache (from which
GET call reads). Will fix this issue shortly.
> Updating classification removes some properties
> ---
>
> Key: ATLAS-1879
> URL: https://issues.apache.org/jira/browse/ATLAS-1879
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Affects Versions: 0.8-incubating
>Reporter: Laura Ngo
>
> * Created classification via POST.
> * Updated via PUT
> * Lost properties
> POST http://127.0.0.1:21000/api/atlas/v2/types/typedefs
> {code}
> {
> "classificationDefs": [{
> "name": "test_classification_11",
> "description": "",
> "createdBy" : "admin",
> "superTypes": [],
> "attributeDefs": [{
> "name" : "test_class_11",
> "typeName" : "string",
> "isOptional" : true,
> "isUnique" : true,
> "isIndexable" : true,
> "cardinality": "SINGLE",
> "valuesMinCount": 0,
> "valuesMaxCount": 1
> }]
> }],
> "entityDefs": [],
> "enumDefs": [],
> "structDefs": []
> }
> {code}
> GET
> http://127.0.0.1:21000/api/atlas/v2/types/classification/name/test_classification_11
> {code}
> {
> "category": "CLASSIFICATION",
> "guid": "83162fe1-4bb4-4a87-b2b8-364e751a1265",
> "createdBy": "admin",
> "createTime": 1497485890857,
> "updateTime": 1497485890857,
> "version": 1,
> "name": "test_classification_11",
> "description": "",
> "typeVersion": "1.0",
> "attributeDefs": [
> {
> "name": "test_class_11",
> "typeName": "string",
> "isOptional": true,
> "cardinality": "SINGLE",
> "valuesMinCount": 0,
> "valuesMaxCount": 1,
> "isUnique": true,
> "isIndexable": true
> }
> ],
> "superTypes": [],
> }
> {code}
> PUT http://127.0.0.1:21000/api/atlas/v2/types/typedefs
> Update attribute.
> GET
> http://127.0.0.1:21000/api/atlas/v2/types/classification/name/test_classification_11
> {code}
> {
> "category": "CLASSIFICATION",
> "createdBy": "admin",
> "name": "test_classification_11",
> "description": "",
> "attributeDefs": [
> {
> "name": "test_class_11",
> "typeName": "string",
> "isOptional": true,
> "cardinality": "SINGLE",
> "valuesMinCount": 0,
> "valuesMaxCount": 1,
> "isUnique": true,
> "isIndexable": false
> }
> ],
> "superTypes": [],
> }
> {code}
> Some properties are missing after PUT update of attribute "isIndexable"
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Comment Edited] (ATLAS-1885) DELETE classification causes error message
[
https://issues.apache.org/jira/browse/ATLAS-1885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16053011#comment-16053011
]
Laura Ngo edited comment on ATLAS-1885 at 6/17/17 10:24 PM:
The refresh button seems not to work. I think the presence of the error stops
the refresh.
Refreshing the webpage, hard refresh of the web page and a different browser
make no difference. But after stopping and restarting the Atlas service the
error disappears, the classification is deleted and the classification is
removed from the UI.
The classification is also still there when I use GET
http://127.0.0.1:21000/api/atlas/v2/types/typedef/name/{name}
It is only removed when I restart the Atlas service.
The error is 'Discovery query failed g.V().has('__traitNames', T.in,
traitNames) [startIdx..http://127.0.0.1:21000/api/atlas/v2/types/typedef/name/{name}. It is only
removed when I restart the Atlas service.
The error is 'Discovery query failed g.V().has('__traitNames', T.in,
traitNames) [startIdx.. DELETE classification causes error message
> --
>
> Key: ATLAS-1885
> URL: https://issues.apache.org/jira/browse/ATLAS-1885
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Affects Versions: 0.8-incubating
>Reporter: Laura Ngo
>
> * Created classification via POST (/v2/types/typedefs)
> * Tried to delete it via DELETE (/v2/types/typedefs)
> * Get 204 no content response (should have been removed)
> * Go to UI tag is still there
> * Get error in UI
> Discovery query failed g.V().has('__traitNames', T.in, traitNames)
> [startIdx.. {code}
> {
> "classificationDefs": [{
> "name": "test_classification_10",
> "superTypes": [],
> "attributeDefs": []
> }],
> "entityDefs": [],
> "enumDefs": [],
> "structDefs": []
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Comment Edited] (ATLAS-1885) DELETE classification causes error message
[
https://issues.apache.org/jira/browse/ATLAS-1885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16053011#comment-16053011
]
Laura Ngo edited comment on ATLAS-1885 at 6/17/17 10:24 PM:
The refresh button seems not to work. I think the presence of the error stops
the refresh.
Refreshing the webpage, hard refresh of the web page and a different browser
make no difference. But after stopping and restarting the Atlas service the
error disappears, the classification is deleted and the classification is
removed from the UI.
The classification is also still there when I use GET
http://127.0.0.1:21000/api/atlas/v2/types/typedef/name/{name}. It is only
removed when I restart the Atlas service.
The error is 'Discovery query failed g.V().has('__traitNames', T.in,
traitNames) [startIdx.. DELETE classification causes error message
> --
>
> Key: ATLAS-1885
> URL: https://issues.apache.org/jira/browse/ATLAS-1885
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Affects Versions: 0.8-incubating
>Reporter: Laura Ngo
>
> * Created classification via POST (/v2/types/typedefs)
> * Tried to delete it via DELETE (/v2/types/typedefs)
> * Get 204 no content response (should have been removed)
> * Go to UI tag is still there
> * Get error in UI
> Discovery query failed g.V().has('__traitNames', T.in, traitNames)
> [startIdx.. {code}
> {
> "classificationDefs": [{
> "name": "test_classification_10",
> "superTypes": [],
> "attributeDefs": []
> }],
> "entityDefs": [],
> "enumDefs": [],
> "structDefs": []
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (ATLAS-1885) DELETE classification causes error message
[
https://issues.apache.org/jira/browse/ATLAS-1885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16053011#comment-16053011
]
Laura Ngo commented on ATLAS-1885:
--
The refresh button seems not to work. I think the presence of the error stops
the refresh.
Refreshing the webpage, hard refresh of the web page and a different browser
make no difference. But after stopping and restarting the Atlas service the
error disappears, the classification is deleted and the classification is
removed from the UI.
The error is 'Discovery query failed g.V().has('__traitNames', T.in,
traitNames) [startIdx.. DELETE classification causes error message
> --
>
> Key: ATLAS-1885
> URL: https://issues.apache.org/jira/browse/ATLAS-1885
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Affects Versions: 0.8-incubating
>Reporter: Laura Ngo
>
> * Created classification via POST (/v2/types/typedefs)
> * Tried to delete it via DELETE (/v2/types/typedefs)
> * Get 204 no content response (should have been removed)
> * Go to UI tag is still there
> * Get error in UI
> Discovery query failed g.V().has('__traitNames', T.in, traitNames)
> [startIdx.. {code}
> {
> "classificationDefs": [{
> "name": "test_classification_10",
> "superTypes": [],
> "attributeDefs": []
> }],
> "entityDefs": [],
> "enumDefs": [],
> "structDefs": []
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (ATLAS-1879) Updating classification removes some properties
[
https://issues.apache.org/jira/browse/ATLAS-1879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16053010#comment-16053010
]
Laura Ngo commented on ATLAS-1879:
--
This was the PUT.
{code}
{
"classificationDefs": [{
"name": "test_classification_11",
"description": "",
"createdBy" : "admin",
"updatedtedBy" : "admin",
"superTypes": [],
"attributeDefs": [{
"name" : "test_class_11",
"typeName" : "string",
"isOptional" : true,
"isUnique" : true,
"isIndexable" : false,
"cardinality": "SINGLE",
"valuesMinCount": 0,
"valuesMaxCount": 1
}]
}],
"entityDefs": [],
"enumDefs": [],
"structDefs": []
}
{code}
I changed isIndexable to false.
> Updating classification removes some properties
> ---
>
> Key: ATLAS-1879
> URL: https://issues.apache.org/jira/browse/ATLAS-1879
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Affects Versions: 0.8-incubating
>Reporter: Laura Ngo
>
> * Created classification via POST.
> * Updated via PUT
> * Lost properties
> POST http://127.0.0.1:21000/api/atlas/v2/types/typedefs
> {code}
> {
> "classificationDefs": [{
> "name": "test_classification_11",
> "description": "",
> "createdBy" : "admin",
> "superTypes": [],
> "attributeDefs": [{
> "name" : "test_class_11",
> "typeName" : "string",
> "isOptional" : true,
> "isUnique" : true,
> "isIndexable" : true,
> "cardinality": "SINGLE",
> "valuesMinCount": 0,
> "valuesMaxCount": 1
> }]
> }],
> "entityDefs": [],
> "enumDefs": [],
> "structDefs": []
> }
> {code}
> GET
> http://127.0.0.1:21000/api/atlas/v2/types/classification/name/test_classification_11
> {code}
> {
> "category": "CLASSIFICATION",
> "guid": "83162fe1-4bb4-4a87-b2b8-364e751a1265",
> "createdBy": "admin",
> "createTime": 1497485890857,
> "updateTime": 1497485890857,
> "version": 1,
> "name": "test_classification_11",
> "description": "",
> "typeVersion": "1.0",
> "attributeDefs": [
> {
> "name": "test_class_11",
> "typeName": "string",
> "isOptional": true,
> "cardinality": "SINGLE",
> "valuesMinCount": 0,
> "valuesMaxCount": 1,
> "isUnique": true,
> "isIndexable": true
> }
> ],
> "superTypes": [],
> }
> {code}
> PUT http://127.0.0.1:21000/api/atlas/v2/types/typedefs
> Update attribute.
> GET
> http://127.0.0.1:21000/api/atlas/v2/types/classification/name/test_classification_11
> {code}
> {
> "category": "CLASSIFICATION",
> "createdBy": "admin",
> "name": "test_classification_11",
> "description": "",
> "attributeDefs": [
> {
> "name": "test_class_11",
> "typeName": "string",
> "isOptional": true,
> "cardinality": "SINGLE",
> "valuesMinCount": 0,
> "valuesMaxCount": 1,
> "isUnique": true,
> "isIndexable": false
> }
> ],
> "superTypes": [],
> }
> {code}
> Some properties are missing after PUT update of attribute "isIndexable"
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (ATLAS-1879) Updating classification removes some properties
[
https://issues.apache.org/jira/browse/ATLAS-1879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16052949#comment-16052949
]
Madhan Neethiraj commented on ATLAS-1879:
-
bq. PUT http://127.0.0.1:21000/api/atlas/v2/types/typedefs
[~LauraNgo] - can you add contents of the PUT body? Did it include
'isIndexable: true' for attribute 'test_class_11'?
> Updating classification removes some properties
> ---
>
> Key: ATLAS-1879
> URL: https://issues.apache.org/jira/browse/ATLAS-1879
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Affects Versions: 0.8-incubating
>Reporter: Laura Ngo
>
> * Created classification via POST.
> * Updated via PUT
> * Lost properties
> POST http://127.0.0.1:21000/api/atlas/v2/types/typedefs
> {code}
> {
> "classificationDefs": [{
> "name": "test_classification_11",
> "description": "",
> "createdBy" : "admin",
> "superTypes": [],
> "attributeDefs": [{
> "name" : "test_class_11",
> "typeName" : "string",
> "isOptional" : true,
> "isUnique" : true,
> "isIndexable" : true,
> "cardinality": "SINGLE",
> "valuesMinCount": 0,
> "valuesMaxCount": 1
> }]
> }],
> "entityDefs": [],
> "enumDefs": [],
> "structDefs": []
> }
> {code}
> GET
> http://127.0.0.1:21000/api/atlas/v2/types/classification/name/test_classification_11
> {code}
> {
> "category": "CLASSIFICATION",
> "guid": "83162fe1-4bb4-4a87-b2b8-364e751a1265",
> "createdBy": "admin",
> "createTime": 1497485890857,
> "updateTime": 1497485890857,
> "version": 1,
> "name": "test_classification_11",
> "description": "",
> "typeVersion": "1.0",
> "attributeDefs": [
> {
> "name": "test_class_11",
> "typeName": "string",
> "isOptional": true,
> "cardinality": "SINGLE",
> "valuesMinCount": 0,
> "valuesMaxCount": 1,
> "isUnique": true,
> "isIndexable": true
> }
> ],
> "superTypes": [],
> }
> {code}
> PUT http://127.0.0.1:21000/api/atlas/v2/types/typedefs
> Update attribute.
> GET
> http://127.0.0.1:21000/api/atlas/v2/types/classification/name/test_classification_11
> {code}
> {
> "category": "CLASSIFICATION",
> "createdBy": "admin",
> "name": "test_classification_11",
> "description": "",
> "attributeDefs": [
> {
> "name": "test_class_11",
> "typeName": "string",
> "isOptional": true,
> "cardinality": "SINGLE",
> "valuesMinCount": 0,
> "valuesMaxCount": 1,
> "isUnique": true,
> "isIndexable": false
> }
> ],
> "superTypes": [],
> }
> {code}
> Some properties are missing after PUT update of attribute "isIndexable"
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (ATLAS-1885) DELETE classification causes error message
[
https://issues.apache.org/jira/browse/ATLAS-1885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16052944#comment-16052944
]
Madhan Neethiraj commented on ATLAS-1885:
-
{quote}
- Tried to delete it via DELETE (/v2/types/typedefs)
- Get 204 no content response (should have been removed)
- Go to UI tag is still there
{quote}
[~LauraNgo] - After classifications were added/deleted via REST API, UI needs
to be refreshed by clicking on 'refresh' icon. Can you please try?
{quote}
- Get error in UI
{quote}
Can you please add details of the error?
> DELETE classification causes error message
> --
>
> Key: ATLAS-1885
> URL: https://issues.apache.org/jira/browse/ATLAS-1885
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Affects Versions: 0.8-incubating
>Reporter: Laura Ngo
>
> * Created classification via POST (/v2/types/typedefs)
> * Tried to delete it via DELETE (/v2/types/typedefs)
> * Get 204 no content response (should have been removed)
> * Go to UI tag is still there
> * Get error in UI
> Discovery query failed g.V().has('__traitNames', T.in, traitNames)
> [startIdx.. {code}
> {
> "classificationDefs": [{
> "name": "test_classification_10",
> "superTypes": [],
> "attributeDefs": []
> }],
> "entityDefs": [],
> "enumDefs": [],
> "structDefs": []
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (ATLAS-1885) DELETE classification causes error message
Laura Ngo created ATLAS-1885:
Summary: DELETE classification causes error message
Key: ATLAS-1885
URL: https://issues.apache.org/jira/browse/ATLAS-1885
Project: Atlas
Issue Type: Bug
Components: atlas-core
Affects Versions: 0.8-incubating
Reporter: Laura Ngo
* Created classification via POST (/v2/types/typedefs)
* Tried to delete it via DELETE (/v2/types/typedefs)
* Get 204 no content response (should have been removed)
* Go to UI tag is still there
* Get error in UI
Discovery query failed g.V().has('__traitNames', T.in, traitNames)
[startIdx..
[jira] [Resolved] (ATLAS-1875) Gremlin id is no longer returned for vertices in gremlin query
[
https://issues.apache.org/jira/browse/ATLAS-1875?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Madhan Neethiraj resolved ATLAS-1875.
-
Resolution: Fixed
Fix Version/s: 0.8.1-incubating
0.9-incubating
Committed to the following branches:
- master:
http://git-wip-us.apache.org/repos/asf/incubator-atlas/commit/c6081ddc
- 0.8-incubating:
http://git-wip-us.apache.org/repos/asf/incubator-atlas/commit/e183fe83
Thanks [~christianmr] for the patch and [~grahamwallis] for your help.
> Gremlin id is no longer returned for vertices in gremlin query
> --
>
> Key: ATLAS-1875
> URL: https://issues.apache.org/jira/browse/ATLAS-1875
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Affects Versions: trunk, 0.8-incubating
>Reporter: Christian R
> Labels: dsl, gremlin
> Fix For: 0.9-incubating, 0.8.1-incubating
>
>
> Hi,
> while investigating a move from atlas 0.7 to 0.8 (HDP 2.5 to HDP2.6) our
> tests fail on gremlin queries. It turns out that the returned entities in a
> gremlin search in 0.8 does not include the 'id' attribute.
> I've built commit a0bd93945cd45457bbf34a8cb819d4fa4ba72964 (0.8-rc1) on linux
> using berkely and elasticearch to test with.
> The query
> :21000/api/atlas/discovery/search/gremlin?g.V.has('__type.name',
> 'Infrastructure').collect()
> on our 0.7 cluster gives
> {code:json}
> {
> __type.name: "Infrastructure",
> __type.category: "CLASS",
> __type: "typeSystem",
> id: "16640"
> }{code}
> while the same query on my 0.8-rc1 installation gives
> {code:javascript}
> {
> __type.name: "Infrastructure",
> __version: "1",
> __type.category: "CLASS",
> __type.version: "1.0",
> __modificationTimestamp: "1497448424134",
> __type: "typeSystem",
> __type.options: "null",
> __type.description: "Infrastructure",
> __guid: "77d07283-7622-4305-9c0c-09ac5aee86c8",
> __timestamp: "1497448424134"
> }
> {code}
> Certainly more information, but id is missing.
> The very poor DSL performance (see ATLAS-1868) and a need for advanced
> queries led us to base our queries on gremlin. This has worked very well so
> far. We include both edges and nodes in the result set and use the inVertex,
> outVertex and label info on the edges to rebuild our tree on the client side.
> I also see that gremlin has disappeared from version two of the API. Since
> addE and addV lets you insert into the graph I can see how exposing a full
> gremlin endpoint might not be wanted.
> As an example of the queries we run that I haven't been able to express in
> the DSL is
> {noformat}
> query = g.V.has('__guid','').copySplit(
> _().out('track'),
> _().as('x')
> .out('functions', 'component')
> .loop('x'){true}{true}
> .copySplit(
> _(),
> _().in('part_of'),
> _().outE('functions',
> 'component'),
> _().inE('part_of')
> )
> .exhaustMerge.dedup
> )
> .exhaustMerge
> .collect()
> {noformat}
> this might not be a normal usecase.
> edit to add:
> I looked at GraphBackedDiscoveryService.java and notice that:
> 0.7:
> {code:java}
> else if (r instanceof TitanVertex) {
> Iterable ps = ((TitanVertex)
> r).getProperties();
> for (TitanProperty tP : ps) {
> String pName = tP.getPropertyKey().getName();
> Object pValue = ((TitanVertex) r).getProperty(pName);
> if (pValue != null) {
> oRow.put(pName, pValue.toString());
> }
> }
> {code}
> Vs 0.8 code:
> {code:java}
> else if (value instanceof AtlasVertex) {
> AtlasVertex vertex = (AtlasVertex)value;
> for (String key : vertex.getPropertyKeys()) {
> Object pro
[jira] [Commented] (ATLAS-1876) Import/Export : AtlasSchemaViolationException during Import of entity associated to a tag with tag attribute value of datatype float
[
https://issues.apache.org/jira/browse/ATLAS-1876?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16052454#comment-16052454
]
Ashutosh Mestry commented on ATLAS-1876:
I was able to duplicate this without using Import-Export.
Here are the steps:
- Create a tag with 1 float field. Say tag is 'fl1' and float attribute is 'fv'.
- Try to associate fl1 to an entity. Provide 4028235e+38 for 'fv' during
association.
High chance that the association will fail. The server log indicate error in
serialization.
Some research yielded similar problems faced by other applications. Yet to
think of a fix.
> Import/Export : AtlasSchemaViolationException during Import of entity
> associated to a tag with tag attribute value of datatype float
>
>
> Key: ATLAS-1876
> URL: https://issues.apache.org/jira/browse/ATLAS-1876
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Affects Versions: 0.9-incubating
>Reporter: Sharmadha Sainath
>Assignee: Ashutosh Mestry
> Attachments: AtlasSchemaViolationException_ExportImport.txt
>
>
> 1.In cluster1, created a tag float_tag with attribute float_max of datatype
> float.
> 2. In the same cluster , created table table1 and associated the tag
> float_tag to table1 , with attribute value for float_max as 3.4028235e+38
> (maximum value for float).
> 3.Creation and tag association are successful.
> 4.Created table1.zip by using Export API
> 5. Using Import API , tried to import table1.zip to cluster2. Import failed
> with
> {code}
> {"errorCode":"ATLAS-500-00-001","errorMessage":"org.apache.atlas.exception.AtlasBaseException:
> org.apache.atlas.repository.graphdb.AtlasSchemaViolationException:
> com.thinkaurelius.titan.core.SchemaViolationException: Value [3.4028235E38]
> is not an instance of the expected data type for property key
> [float_tag.float_max] and cannot be converted. Expected: class
> java.lang.Float, found: class java.lang.Double"}
> {code}
> Attached the error stack trace found in cluster2 application logs.
> This exception is seen only when the value for float attribute is given as
> 3.4028235e+38 . Other values like 0 , 2.5 , 1.4e+10 etc., are accepted and
> import is done successfully.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (ATLAS-1875) Gremlin id is no longer returned for vertices in gremlin query
[
https://issues.apache.org/jira/browse/ATLAS-1875?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16052276#comment-16052276
]
Christian R commented on ATLAS-1875:
https://reviews.apache.org/r/60163/
On Fri, Jun 16, 2017 at 8:59 PM, Christian Rieck
> Gremlin id is no longer returned for vertices in gremlin query
> --
>
> Key: ATLAS-1875
> URL: https://issues.apache.org/jira/browse/ATLAS-1875
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Affects Versions: trunk, 0.8-incubating
>Reporter: Christian R
> Labels: dsl, gremlin
>
> Hi,
> while investigating a move from atlas 0.7 to 0.8 (HDP 2.5 to HDP2.6) our
> tests fail on gremlin queries. It turns out that the returned entities in a
> gremlin search in 0.8 does not include the 'id' attribute.
> I've built commit a0bd93945cd45457bbf34a8cb819d4fa4ba72964 (0.8-rc1) on linux
> using berkely and elasticearch to test with.
> The query
> :21000/api/atlas/discovery/search/gremlin?g.V.has('__type.name',
> 'Infrastructure').collect()
> on our 0.7 cluster gives
> {code:json}
> {
> __type.name: "Infrastructure",
> __type.category: "CLASS",
> __type: "typeSystem",
> id: "16640"
> }{code}
> while the same query on my 0.8-rc1 installation gives
> {code:javascript}
> {
> __type.name: "Infrastructure",
> __version: "1",
> __type.category: "CLASS",
> __type.version: "1.0",
> __modificationTimestamp: "1497448424134",
> __type: "typeSystem",
> __type.options: "null",
> __type.description: "Infrastructure",
> __guid: "77d07283-7622-4305-9c0c-09ac5aee86c8",
> __timestamp: "1497448424134"
> }
> {code}
> Certainly more information, but id is missing.
> The very poor DSL performance (see ATLAS-1868) and a need for advanced
> queries led us to base our queries on gremlin. This has worked very well so
> far. We include both edges and nodes in the result set and use the inVertex,
> outVertex and label info on the edges to rebuild our tree on the client side.
> I also see that gremlin has disappeared from version two of the API. Since
> addE and addV lets you insert into the graph I can see how exposing a full
> gremlin endpoint might not be wanted.
> As an example of the queries we run that I haven't been able to express in
> the DSL is
> {noformat}
> query = g.V.has('__guid','').copySplit(
> _().out('track'),
> _().as('x')
> .out('functions', 'component')
> .loop('x'){true}{true}
> .copySplit(
> _(),
> _().in('part_of'),
> _().outE('functions',
> 'component'),
> _().inE('part_of')
> )
> .exhaustMerge.dedup
> )
> .exhaustMerge
> .collect()
> {noformat}
> this might not be a normal usecase.
> edit to add:
> I looked at GraphBackedDiscoveryService.java and notice that:
> 0.7:
> {code:java}
> else if (r instanceof TitanVertex) {
> Iterable ps = ((TitanVertex)
> r).getProperties();
> for (TitanProperty tP : ps) {
> String pName = tP.getPropertyKey().getName();
> Object pValue = ((TitanVertex) r).getProperty(pName);
> if (pValue != null) {
> oRow.put(pName, pValue.toString());
> }
> }
> {code}
> Vs 0.8 code:
> {code:java}
> else if (value instanceof AtlasVertex) {
> AtlasVertex vertex = (AtlasVertex)value;
> for (String key : vertex.getPropertyKeys()) {
> Object propertyValue =
> GraphHelper.getProperty(vertex, key);
> if (propertyValue != null) {
> oRow.put(key, propertyValue.toString());
> }
> }
> {code}
> look very similar.
> However, 0.8 handles id in
[jira] [Commented] (ATLAS-1875) Gremlin id is no longer returned for vertices in gremlin query
[
https://issues.apache.org/jira/browse/ATLAS-1875?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16052262#comment-16052262
]
Christian R commented on ATLAS-1875:
Guidance needed: I cannot publish without a summary but the form will not
tell me how to create one. Neither the faq nor the markdown guide mentions
"summary".
On Fri, Jun 16, 2017 at 8:36 PM, Sarath Subramanian (JIRA)
> Gremlin id is no longer returned for vertices in gremlin query
> --
>
> Key: ATLAS-1875
> URL: https://issues.apache.org/jira/browse/ATLAS-1875
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Affects Versions: trunk, 0.8-incubating
>Reporter: Christian R
> Labels: dsl, gremlin
>
> Hi,
> while investigating a move from atlas 0.7 to 0.8 (HDP 2.5 to HDP2.6) our
> tests fail on gremlin queries. It turns out that the returned entities in a
> gremlin search in 0.8 does not include the 'id' attribute.
> I've built commit a0bd93945cd45457bbf34a8cb819d4fa4ba72964 (0.8-rc1) on linux
> using berkely and elasticearch to test with.
> The query
> :21000/api/atlas/discovery/search/gremlin?g.V.has('__type.name',
> 'Infrastructure').collect()
> on our 0.7 cluster gives
> {code:json}
> {
> __type.name: "Infrastructure",
> __type.category: "CLASS",
> __type: "typeSystem",
> id: "16640"
> }{code}
> while the same query on my 0.8-rc1 installation gives
> {code:javascript}
> {
> __type.name: "Infrastructure",
> __version: "1",
> __type.category: "CLASS",
> __type.version: "1.0",
> __modificationTimestamp: "1497448424134",
> __type: "typeSystem",
> __type.options: "null",
> __type.description: "Infrastructure",
> __guid: "77d07283-7622-4305-9c0c-09ac5aee86c8",
> __timestamp: "1497448424134"
> }
> {code}
> Certainly more information, but id is missing.
> The very poor DSL performance (see ATLAS-1868) and a need for advanced
> queries led us to base our queries on gremlin. This has worked very well so
> far. We include both edges and nodes in the result set and use the inVertex,
> outVertex and label info on the edges to rebuild our tree on the client side.
> I also see that gremlin has disappeared from version two of the API. Since
> addE and addV lets you insert into the graph I can see how exposing a full
> gremlin endpoint might not be wanted.
> As an example of the queries we run that I haven't been able to express in
> the DSL is
> {noformat}
> query = g.V.has('__guid','').copySplit(
> _().out('track'),
> _().as('x')
> .out('functions', 'component')
> .loop('x'){true}{true}
> .copySplit(
> _(),
> _().in('part_of'),
> _().outE('functions',
> 'component'),
> _().inE('part_of')
> )
> .exhaustMerge.dedup
> )
> .exhaustMerge
> .collect()
> {noformat}
> this might not be a normal usecase.
> edit to add:
> I looked at GraphBackedDiscoveryService.java and notice that:
> 0.7:
> {code:java}
> else if (r instanceof TitanVertex) {
> Iterable ps = ((TitanVertex)
> r).getProperties();
> for (TitanProperty tP : ps) {
> String pName = tP.getPropertyKey().getName();
> Object pValue = ((TitanVertex) r).getProperty(pName);
> if (pValue != null) {
> oRow.put(pName, pValue.toString());
> }
> }
> {code}
> Vs 0.8 code:
> {code:java}
> else if (value instanceof AtlasVertex) {
> AtlasVertex vertex = (AtlasVertex)value;
> for (String key : vertex.getPropertyKeys()) {
> Object propertyValue =
> GraphHelper.getProperty(vertex, key);
> if (propertyValue != null) {
> oRow.put(key, property
[jira] [Commented] (ATLAS-1875) Gremlin id is no longer returned for vertices in gremlin query
[
https://issues.apache.org/jira/browse/ATLAS-1875?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16052240#comment-16052240
]
Sarath Subramanian commented on ATLAS-1875:
---
yes, that URL is right. You can upload your patch.
> Gremlin id is no longer returned for vertices in gremlin query
> --
>
> Key: ATLAS-1875
> URL: https://issues.apache.org/jira/browse/ATLAS-1875
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Affects Versions: trunk, 0.8-incubating
>Reporter: Christian R
> Labels: dsl, gremlin
>
> Hi,
> while investigating a move from atlas 0.7 to 0.8 (HDP 2.5 to HDP2.6) our
> tests fail on gremlin queries. It turns out that the returned entities in a
> gremlin search in 0.8 does not include the 'id' attribute.
> I've built commit a0bd93945cd45457bbf34a8cb819d4fa4ba72964 (0.8-rc1) on linux
> using berkely and elasticearch to test with.
> The query
> :21000/api/atlas/discovery/search/gremlin?g.V.has('__type.name',
> 'Infrastructure').collect()
> on our 0.7 cluster gives
> {code:json}
> {
> __type.name: "Infrastructure",
> __type.category: "CLASS",
> __type: "typeSystem",
> id: "16640"
> }{code}
> while the same query on my 0.8-rc1 installation gives
> {code:javascript}
> {
> __type.name: "Infrastructure",
> __version: "1",
> __type.category: "CLASS",
> __type.version: "1.0",
> __modificationTimestamp: "1497448424134",
> __type: "typeSystem",
> __type.options: "null",
> __type.description: "Infrastructure",
> __guid: "77d07283-7622-4305-9c0c-09ac5aee86c8",
> __timestamp: "1497448424134"
> }
> {code}
> Certainly more information, but id is missing.
> The very poor DSL performance (see ATLAS-1868) and a need for advanced
> queries led us to base our queries on gremlin. This has worked very well so
> far. We include both edges and nodes in the result set and use the inVertex,
> outVertex and label info on the edges to rebuild our tree on the client side.
> I also see that gremlin has disappeared from version two of the API. Since
> addE and addV lets you insert into the graph I can see how exposing a full
> gremlin endpoint might not be wanted.
> As an example of the queries we run that I haven't been able to express in
> the DSL is
> {noformat}
> query = g.V.has('__guid','').copySplit(
> _().out('track'),
> _().as('x')
> .out('functions', 'component')
> .loop('x'){true}{true}
> .copySplit(
> _(),
> _().in('part_of'),
> _().outE('functions',
> 'component'),
> _().inE('part_of')
> )
> .exhaustMerge.dedup
> )
> .exhaustMerge
> .collect()
> {noformat}
> this might not be a normal usecase.
> edit to add:
> I looked at GraphBackedDiscoveryService.java and notice that:
> 0.7:
> {code:java}
> else if (r instanceof TitanVertex) {
> Iterable ps = ((TitanVertex)
> r).getProperties();
> for (TitanProperty tP : ps) {
> String pName = tP.getPropertyKey().getName();
> Object pValue = ((TitanVertex) r).getProperty(pName);
> if (pValue != null) {
> oRow.put(pName, pValue.toString());
> }
> }
> {code}
> Vs 0.8 code:
> {code:java}
> else if (value instanceof AtlasVertex) {
> AtlasVertex vertex = (AtlasVertex)value;
> for (String key : vertex.getPropertyKeys()) {
> Object propertyValue =
> GraphHelper.getProperty(vertex, key);
> if (propertyValue != null) {
> oRow.put(key, propertyValue.toString());
> }
> }
> {code}
> look very similar.
> However, 0.8 handles id in
[jira] [Updated] (ATLAS-1880) Search using entity and trait attributes
[ https://issues.apache.org/jira/browse/ATLAS-1880?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apoorv Naik updated ATLAS-1880: --- Attachment: 0001-ATLAS-1880-Search-using-entity-and-trait-attributes.patch Sub-task 1 & 2 > Search using entity and trait attributes > > > Key: ATLAS-1880 > URL: https://issues.apache.org/jira/browse/ATLAS-1880 > Project: Atlas > Issue Type: Improvement >Reporter: Apoorv Naik >Assignee: Apoorv Naik > Attachments: > 0001-ATLAS-1880-Search-using-entity-and-trait-attributes.patch > > > This improvement patch enhances the search experience provided by Atlas. > Currently the basic search only supports type, tag and lucene type query. > New search API will allow complex combinations of attribute search (ANDs / > ORs). The first step will be to use titan's index query on the vertex index > and then subsequently use gremlin to narrow the results if any non-indexed > attributes show up in the search request -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (ATLAS-1875) Gremlin id is no longer returned for vertices in gremlin query
[
https://issues.apache.org/jira/browse/ATLAS-1875?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16052195#comment-16052195
]
Christian R commented on ATLAS-1875:
I am sorry, this is new to me.
Where can I find this link? Shall I register at
https://reviews.apache.org/r/ and create something there?
Christian.
On Fri, Jun 16, 2017 at 7:45 PM, Sarath Subramanian (JIRA)
> Gremlin id is no longer returned for vertices in gremlin query
> --
>
> Key: ATLAS-1875
> URL: https://issues.apache.org/jira/browse/ATLAS-1875
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Affects Versions: trunk, 0.8-incubating
>Reporter: Christian R
> Labels: dsl, gremlin
>
> Hi,
> while investigating a move from atlas 0.7 to 0.8 (HDP 2.5 to HDP2.6) our
> tests fail on gremlin queries. It turns out that the returned entities in a
> gremlin search in 0.8 does not include the 'id' attribute.
> I've built commit a0bd93945cd45457bbf34a8cb819d4fa4ba72964 (0.8-rc1) on linux
> using berkely and elasticearch to test with.
> The query
> :21000/api/atlas/discovery/search/gremlin?g.V.has('__type.name',
> 'Infrastructure').collect()
> on our 0.7 cluster gives
> {code:json}
> {
> __type.name: "Infrastructure",
> __type.category: "CLASS",
> __type: "typeSystem",
> id: "16640"
> }{code}
> while the same query on my 0.8-rc1 installation gives
> {code:javascript}
> {
> __type.name: "Infrastructure",
> __version: "1",
> __type.category: "CLASS",
> __type.version: "1.0",
> __modificationTimestamp: "1497448424134",
> __type: "typeSystem",
> __type.options: "null",
> __type.description: "Infrastructure",
> __guid: "77d07283-7622-4305-9c0c-09ac5aee86c8",
> __timestamp: "1497448424134"
> }
> {code}
> Certainly more information, but id is missing.
> The very poor DSL performance (see ATLAS-1868) and a need for advanced
> queries led us to base our queries on gremlin. This has worked very well so
> far. We include both edges and nodes in the result set and use the inVertex,
> outVertex and label info on the edges to rebuild our tree on the client side.
> I also see that gremlin has disappeared from version two of the API. Since
> addE and addV lets you insert into the graph I can see how exposing a full
> gremlin endpoint might not be wanted.
> As an example of the queries we run that I haven't been able to express in
> the DSL is
> {noformat}
> query = g.V.has('__guid','').copySplit(
> _().out('track'),
> _().as('x')
> .out('functions', 'component')
> .loop('x'){true}{true}
> .copySplit(
> _(),
> _().in('part_of'),
> _().outE('functions',
> 'component'),
> _().inE('part_of')
> )
> .exhaustMerge.dedup
> )
> .exhaustMerge
> .collect()
> {noformat}
> this might not be a normal usecase.
> edit to add:
> I looked at GraphBackedDiscoveryService.java and notice that:
> 0.7:
> {code:java}
> else if (r instanceof TitanVertex) {
> Iterable ps = ((TitanVertex)
> r).getProperties();
> for (TitanProperty tP : ps) {
> String pName = tP.getPropertyKey().getName();
> Object pValue = ((TitanVertex) r).getProperty(pName);
> if (pValue != null) {
> oRow.put(pName, pValue.toString());
> }
> }
> {code}
> Vs 0.8 code:
> {code:java}
> else if (value instanceof AtlasVertex) {
> AtlasVertex vertex = (AtlasVertex)value;
> for (String key : vertex.getPropertyKeys()) {
> Object propertyValue =
> GraphHelper.getProperty(vertex, key);
> if (propertyValue != null) {
> oRow.put(key, propertyValue.toString());
&g
[jira] [Commented] (ATLAS-1878) NPE when a request without any query path lands on atlas
[
https://issues.apache.org/jira/browse/ATLAS-1878?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16052183#comment-16052183
]
Sarath Subramanian commented on ATLAS-1878:
---
+1 for the patch.
> NPE when a request without any query path lands on atlas
>
>
> Key: ATLAS-1878
> URL: https://issues.apache.org/jira/browse/ATLAS-1878
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Affects Versions: 0.9-incubating
>Reporter: Ayub Pathan
>Assignee: Nixon Rodrigues
>Priority: Critical
> Fix For: 0.9-incubating
>
> Attachments: ATLAS-1878.patch
>
>
> {noformat}
> 2017-06-16 07:02:13,515 WARN - [pool-2-thread-10:] ~ got exception trying to
> get groups for user admin: id: admin: no such user
> id: admin: no such user
> (ShellBasedUnixGroupsMapping:87)
> 2017-06-16 07:02:13,515 ERROR - [pool-2-thread-10:] ~ Exception while
> fetching groups (AtlasAbstractAuthenticationProvider:120)
> java.io.IOException: No groups found for user admin
> at org.apache.hadoop.security.Groups.noGroupsForUser(Groups.java:190)
> at org.apache.hadoop.security.Groups.access$400(Groups.java:69)
> at org.apache.hadoop.security.Groups$GroupCacheLoader.load(Groups.java:307)
> at org.apache.hadoop.security.Groups$GroupCacheLoader.load(Groups.java:257)
> at
> com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3542)
> at com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2323)
> at
> com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2286)
> at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2201)
> at com.google.common.cache.LocalCache.get(LocalCache.java:3953)
> at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3957)
> at
> com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4875)
> at org.apache.hadoop.security.Groups.getGroups(Groups.java:215)
> at
> org.apache.atlas.web.security.AtlasAbstractAuthenticationProvider.getAuthoritiesFromUGI(AtlasAbstractAuthenticationProvider.java:113)
> at
> org.apache.atlas.web.filters.AtlasKnoxSSOAuthenticationFilter.doFilter(AtlasKnoxSSOAuthenticationFilter.java:171)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
> at
> org.apache.atlas.web.filters.StaleTransactionCleanupFilter.doFilter(StaleTransactionCleanupFilter.java:55)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
> at
> org.apache.atlas.web.filters.ActiveServerFilter.doFilter(ActiveServerFilter.java:85)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
> at
> org.springframework.security.web.authentication.www.BasicAuthenticationFilter.doFilterInternal(BasicAuthenticationFilter.java:158)
> at
> org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
> at
> org.springframework.security.web.authentication.AbstractAuthenticationProcessingFilter.doFilter(AbstractAuthenticationProcessingFilter.java:200)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
> at
> org.springframework.security.web.authentication.logout.LogoutFilter.doFilter(LogoutFilter.java:116)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
> at
> org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:105)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
> at
> org.springframework.security.web.context.request.async.WebAsyncManagerIntegrationFilter.doFilterInternal(WebAsyncManagerIntegrationFilter.java:56)
> at
> org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
> at
> org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:214)
> at
> org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:177)
> at
> org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:346)
> at
> org.sp
[jira] [Created] (ATLAS-1883) Gremlin search implementation
Apoorv Naik created ATLAS-1883: -- Summary: Gremlin search implementation Key: ATLAS-1883 URL: https://issues.apache.org/jira/browse/ATLAS-1883 Project: Atlas Issue Type: Sub-task Reporter: Apoorv Naik Assignee: Apoorv Naik -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (ATLAS-1884) Mixed-mode and intelligent querying
Apoorv Naik created ATLAS-1884: -- Summary: Mixed-mode and intelligent querying Key: ATLAS-1884 URL: https://issues.apache.org/jira/browse/ATLAS-1884 Project: Atlas Issue Type: Sub-task Reporter: Apoorv Naik Assignee: Apoorv Naik -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (ATLAS-1882) Solr (indexed) search implementation
Apoorv Naik created ATLAS-1882: -- Summary: Solr (indexed) search implementation Key: ATLAS-1882 URL: https://issues.apache.org/jira/browse/ATLAS-1882 Project: Atlas Issue Type: Sub-task Reporter: Apoorv Naik Assignee: Apoorv Naik -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (ATLAS-1881) Request structure design
Apoorv Naik created ATLAS-1881: -- Summary: Request structure design Key: ATLAS-1881 URL: https://issues.apache.org/jira/browse/ATLAS-1881 Project: Atlas Issue Type: Sub-task Reporter: Apoorv Naik Assignee: Apoorv Naik -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (ATLAS-1880) Search using entity and trait attributes
Apoorv Naik created ATLAS-1880: -- Summary: Search using entity and trait attributes Key: ATLAS-1880 URL: https://issues.apache.org/jira/browse/ATLAS-1880 Project: Atlas Issue Type: Improvement Reporter: Apoorv Naik Assignee: Apoorv Naik This improvement patch enhances the search experience provided by Atlas. Currently the basic search only supports type, tag and lucene type query. New search API will allow complex combinations of attribute search (ANDs / ORs). The first step will be to use titan's index query on the vertex index and then subsequently use gremlin to narrow the results if any non-indexed attributes show up in the search request -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (ATLAS-1875) Gremlin id is no longer returned for vertices in gremlin query
[
https://issues.apache.org/jira/browse/ATLAS-1875?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16052181#comment-16052181
]
Sarath Subramanian commented on ATLAS-1875:
---
[~christianmr], We use Review Board to review/comment on patches for Atlas.
Could you update this JIRA with the Review Board link.
> Gremlin id is no longer returned for vertices in gremlin query
> --
>
> Key: ATLAS-1875
> URL: https://issues.apache.org/jira/browse/ATLAS-1875
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Affects Versions: trunk, 0.8-incubating
>Reporter: Christian R
> Labels: dsl, gremlin
>
> Hi,
> while investigating a move from atlas 0.7 to 0.8 (HDP 2.5 to HDP2.6) our
> tests fail on gremlin queries. It turns out that the returned entities in a
> gremlin search in 0.8 does not include the 'id' attribute.
> I've built commit a0bd93945cd45457bbf34a8cb819d4fa4ba72964 (0.8-rc1) on linux
> using berkely and elasticearch to test with.
> The query
> :21000/api/atlas/discovery/search/gremlin?g.V.has('__type.name',
> 'Infrastructure').collect()
> on our 0.7 cluster gives
> {code:json}
> {
> __type.name: "Infrastructure",
> __type.category: "CLASS",
> __type: "typeSystem",
> id: "16640"
> }{code}
> while the same query on my 0.8-rc1 installation gives
> {code:javascript}
> {
> __type.name: "Infrastructure",
> __version: "1",
> __type.category: "CLASS",
> __type.version: "1.0",
> __modificationTimestamp: "1497448424134",
> __type: "typeSystem",
> __type.options: "null",
> __type.description: "Infrastructure",
> __guid: "77d07283-7622-4305-9c0c-09ac5aee86c8",
> __timestamp: "1497448424134"
> }
> {code}
> Certainly more information, but id is missing.
> The very poor DSL performance (see ATLAS-1868) and a need for advanced
> queries led us to base our queries on gremlin. This has worked very well so
> far. We include both edges and nodes in the result set and use the inVertex,
> outVertex and label info on the edges to rebuild our tree on the client side.
> I also see that gremlin has disappeared from version two of the API. Since
> addE and addV lets you insert into the graph I can see how exposing a full
> gremlin endpoint might not be wanted.
> As an example of the queries we run that I haven't been able to express in
> the DSL is
> {noformat}
> query = g.V.has('__guid','').copySplit(
> _().out('track'),
> _().as('x')
> .out('functions', 'component')
> .loop('x'){true}{true}
> .copySplit(
> _(),
> _().in('part_of'),
> _().outE('functions',
> 'component'),
> _().inE('part_of')
> )
> .exhaustMerge.dedup
> )
> .exhaustMerge
> .collect()
> {noformat}
> this might not be a normal usecase.
> edit to add:
> I looked at GraphBackedDiscoveryService.java and notice that:
> 0.7:
> {code:java}
> else if (r instanceof TitanVertex) {
> Iterable ps = ((TitanVertex)
> r).getProperties();
> for (TitanProperty tP : ps) {
> String pName = tP.getPropertyKey().getName();
> Object pValue = ((TitanVertex) r).getProperty(pName);
> if (pValue != null) {
> oRow.put(pName, pValue.toString());
> }
> }
> {code}
> Vs 0.8 code:
> {code:java}
> else if (value instanceof AtlasVertex) {
> AtlasVertex vertex = (AtlasVertex)value;
> for (String key : vertex.getPropertyKeys()) {
> Object propertyValue =
> GraphHelper.getProperty(vertex, key);
> if (propertyValue != null) {
> oRow.put(key, propertyValue.toString());
> }
>
[jira] [Commented] (ATLAS-1878) NPE when a request without any query path lands on atlas
[
https://issues.apache.org/jira/browse/ATLAS-1878?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16052060#comment-16052060
]
Apoorv Naik commented on ATLAS-1878:
looks good. Will commit shortly
> NPE when a request without any query path lands on atlas
>
>
> Key: ATLAS-1878
> URL: https://issues.apache.org/jira/browse/ATLAS-1878
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Affects Versions: 0.9-incubating
>Reporter: Ayub Pathan
>Assignee: Nixon Rodrigues
>Priority: Critical
> Fix For: 0.9-incubating
>
> Attachments: ATLAS-1878.patch
>
>
> {noformat}
> 2017-06-16 07:02:13,515 WARN - [pool-2-thread-10:] ~ got exception trying to
> get groups for user admin: id: admin: no such user
> id: admin: no such user
> (ShellBasedUnixGroupsMapping:87)
> 2017-06-16 07:02:13,515 ERROR - [pool-2-thread-10:] ~ Exception while
> fetching groups (AtlasAbstractAuthenticationProvider:120)
> java.io.IOException: No groups found for user admin
> at org.apache.hadoop.security.Groups.noGroupsForUser(Groups.java:190)
> at org.apache.hadoop.security.Groups.access$400(Groups.java:69)
> at org.apache.hadoop.security.Groups$GroupCacheLoader.load(Groups.java:307)
> at org.apache.hadoop.security.Groups$GroupCacheLoader.load(Groups.java:257)
> at
> com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3542)
> at com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2323)
> at
> com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2286)
> at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2201)
> at com.google.common.cache.LocalCache.get(LocalCache.java:3953)
> at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3957)
> at
> com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4875)
> at org.apache.hadoop.security.Groups.getGroups(Groups.java:215)
> at
> org.apache.atlas.web.security.AtlasAbstractAuthenticationProvider.getAuthoritiesFromUGI(AtlasAbstractAuthenticationProvider.java:113)
> at
> org.apache.atlas.web.filters.AtlasKnoxSSOAuthenticationFilter.doFilter(AtlasKnoxSSOAuthenticationFilter.java:171)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
> at
> org.apache.atlas.web.filters.StaleTransactionCleanupFilter.doFilter(StaleTransactionCleanupFilter.java:55)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
> at
> org.apache.atlas.web.filters.ActiveServerFilter.doFilter(ActiveServerFilter.java:85)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
> at
> org.springframework.security.web.authentication.www.BasicAuthenticationFilter.doFilterInternal(BasicAuthenticationFilter.java:158)
> at
> org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
> at
> org.springframework.security.web.authentication.AbstractAuthenticationProcessingFilter.doFilter(AbstractAuthenticationProcessingFilter.java:200)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
> at
> org.springframework.security.web.authentication.logout.LogoutFilter.doFilter(LogoutFilter.java:116)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
> at
> org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:105)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
> at
> org.springframework.security.web.context.request.async.WebAsyncManagerIntegrationFilter.doFilterInternal(WebAsyncManagerIntegrationFilter.java:56)
> at
> org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
> at
> org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:214)
> at
> org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:177)
> at
> org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:346)
> at
> org.sp
[jira] [Created] (ATLAS-1879) Updating classification removes some properties
Laura Ngo created ATLAS-1879:
Summary: Updating classification removes some properties
Key: ATLAS-1879
URL: https://issues.apache.org/jira/browse/ATLAS-1879
Project: Atlas
Issue Type: Bug
Components: atlas-core
Affects Versions: 0.8-incubating
Reporter: Laura Ngo
* Created classification via POST.
* Updated via PUT
* Lost properties
POST http://127.0.0.1:21000/api/atlas/v2/types/typedefs
{code}
{
"classificationDefs": [{
"name": "test_classification_11",
"description": "",
"createdBy" : "admin",
"superTypes": [],
"attributeDefs": [{
"name" : "test_class_11",
"typeName" : "string",
"isOptional" : true,
"isUnique" : true,
"isIndexable" : true,
"cardinality": "SINGLE",
"valuesMinCount": 0,
"valuesMaxCount": 1
}]
}],
"entityDefs": [],
"enumDefs": [],
"structDefs": []
}
{code}
GET
http://127.0.0.1:21000/api/atlas/v2/types/classification/name/test_classification_11
{code}
{
"category": "CLASSIFICATION",
"guid": "83162fe1-4bb4-4a87-b2b8-364e751a1265",
"createdBy": "admin",
"createTime": 1497485890857,
"updateTime": 1497485890857,
"version": 1,
"name": "test_classification_11",
"description": "",
"typeVersion": "1.0",
"attributeDefs": [
{
"name": "test_class_11",
"typeName": "string",
"isOptional": true,
"cardinality": "SINGLE",
"valuesMinCount": 0,
"valuesMaxCount": 1,
"isUnique": true,
"isIndexable": true
}
],
"superTypes": [],
}
{code}
PUT http://127.0.0.1:21000/api/atlas/v2/types/typedefs
Update attribute.
GET
http://127.0.0.1:21000/api/atlas/v2/types/classification/name/test_classification_11
{code}
{
"category": "CLASSIFICATION",
"createdBy": "admin",
"name": "test_classification_11",
"description": "",
"attributeDefs": [
{
"name": "test_class_11",
"typeName": "string",
"isOptional": true,
"cardinality": "SINGLE",
"valuesMinCount": 0,
"valuesMaxCount": 1,
"isUnique": true,
"isIndexable": false
}
],
"superTypes": [],
}
{code}
Some properties are missing after PUT update of attribute "isIndexable"
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (ATLAS-1878) NPE when a request without any query path lands on atlas
[
https://issues.apache.org/jira/browse/ATLAS-1878?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nixon Rodrigues updated ATLAS-1878:
---
Attachment: ATLAS-1878.patch
This patch handles NPE in getAPI method.
> NPE when a request without any query path lands on atlas
>
>
> Key: ATLAS-1878
> URL: https://issues.apache.org/jira/browse/ATLAS-1878
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Affects Versions: 0.9-incubating
>Reporter: Ayub Pathan
>Assignee: Nixon Rodrigues
>Priority: Critical
> Fix For: 0.9-incubating
>
> Attachments: ATLAS-1878.patch
>
>
> {noformat}
> 2017-06-16 07:02:13,515 WARN - [pool-2-thread-10:] ~ got exception trying to
> get groups for user admin: id: admin: no such user
> id: admin: no such user
> (ShellBasedUnixGroupsMapping:87)
> 2017-06-16 07:02:13,515 ERROR - [pool-2-thread-10:] ~ Exception while
> fetching groups (AtlasAbstractAuthenticationProvider:120)
> java.io.IOException: No groups found for user admin
> at org.apache.hadoop.security.Groups.noGroupsForUser(Groups.java:190)
> at org.apache.hadoop.security.Groups.access$400(Groups.java:69)
> at org.apache.hadoop.security.Groups$GroupCacheLoader.load(Groups.java:307)
> at org.apache.hadoop.security.Groups$GroupCacheLoader.load(Groups.java:257)
> at
> com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3542)
> at com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2323)
> at
> com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2286)
> at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2201)
> at com.google.common.cache.LocalCache.get(LocalCache.java:3953)
> at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3957)
> at
> com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4875)
> at org.apache.hadoop.security.Groups.getGroups(Groups.java:215)
> at
> org.apache.atlas.web.security.AtlasAbstractAuthenticationProvider.getAuthoritiesFromUGI(AtlasAbstractAuthenticationProvider.java:113)
> at
> org.apache.atlas.web.filters.AtlasKnoxSSOAuthenticationFilter.doFilter(AtlasKnoxSSOAuthenticationFilter.java:171)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
> at
> org.apache.atlas.web.filters.StaleTransactionCleanupFilter.doFilter(StaleTransactionCleanupFilter.java:55)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
> at
> org.apache.atlas.web.filters.ActiveServerFilter.doFilter(ActiveServerFilter.java:85)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
> at
> org.springframework.security.web.authentication.www.BasicAuthenticationFilter.doFilterInternal(BasicAuthenticationFilter.java:158)
> at
> org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
> at
> org.springframework.security.web.authentication.AbstractAuthenticationProcessingFilter.doFilter(AbstractAuthenticationProcessingFilter.java:200)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
> at
> org.springframework.security.web.authentication.logout.LogoutFilter.doFilter(LogoutFilter.java:116)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
> at
> org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:105)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
> at
> org.springframework.security.web.context.request.async.WebAsyncManagerIntegrationFilter.doFilterInternal(WebAsyncManagerIntegrationFilter.java:56)
> at
> org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
> at
> org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:214)
> at
> org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:177)
> at
> org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:346)
> at
> org.springframew
[jira] [Assigned] (ATLAS-1878) NPE when a request without any query path lands on atlas
[
https://issues.apache.org/jira/browse/ATLAS-1878?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nixon Rodrigues reassigned ATLAS-1878:
--
Assignee: Nixon Rodrigues
> NPE when a request without any query path lands on atlas
>
>
> Key: ATLAS-1878
> URL: https://issues.apache.org/jira/browse/ATLAS-1878
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Affects Versions: 0.9-incubating
>Reporter: Ayub Pathan
>Assignee: Nixon Rodrigues
>Priority: Critical
> Fix For: 0.9-incubating
>
>
> {noformat}
> 2017-06-16 07:02:13,515 WARN - [pool-2-thread-10:] ~ got exception trying to
> get groups for user admin: id: admin: no such user
> id: admin: no such user
> (ShellBasedUnixGroupsMapping:87)
> 2017-06-16 07:02:13,515 ERROR - [pool-2-thread-10:] ~ Exception while
> fetching groups (AtlasAbstractAuthenticationProvider:120)
> java.io.IOException: No groups found for user admin
> at org.apache.hadoop.security.Groups.noGroupsForUser(Groups.java:190)
> at org.apache.hadoop.security.Groups.access$400(Groups.java:69)
> at org.apache.hadoop.security.Groups$GroupCacheLoader.load(Groups.java:307)
> at org.apache.hadoop.security.Groups$GroupCacheLoader.load(Groups.java:257)
> at
> com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3542)
> at com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2323)
> at
> com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2286)
> at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2201)
> at com.google.common.cache.LocalCache.get(LocalCache.java:3953)
> at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3957)
> at
> com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4875)
> at org.apache.hadoop.security.Groups.getGroups(Groups.java:215)
> at
> org.apache.atlas.web.security.AtlasAbstractAuthenticationProvider.getAuthoritiesFromUGI(AtlasAbstractAuthenticationProvider.java:113)
> at
> org.apache.atlas.web.filters.AtlasKnoxSSOAuthenticationFilter.doFilter(AtlasKnoxSSOAuthenticationFilter.java:171)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
> at
> org.apache.atlas.web.filters.StaleTransactionCleanupFilter.doFilter(StaleTransactionCleanupFilter.java:55)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
> at
> org.apache.atlas.web.filters.ActiveServerFilter.doFilter(ActiveServerFilter.java:85)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
> at
> org.springframework.security.web.authentication.www.BasicAuthenticationFilter.doFilterInternal(BasicAuthenticationFilter.java:158)
> at
> org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
> at
> org.springframework.security.web.authentication.AbstractAuthenticationProcessingFilter.doFilter(AbstractAuthenticationProcessingFilter.java:200)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
> at
> org.springframework.security.web.authentication.logout.LogoutFilter.doFilter(LogoutFilter.java:116)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
> at
> org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:105)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
> at
> org.springframework.security.web.context.request.async.WebAsyncManagerIntegrationFilter.doFilterInternal(WebAsyncManagerIntegrationFilter.java:56)
> at
> org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
> at
> org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:214)
> at
> org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:177)
> at
> org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:346)
> at
> org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:262)
> at
[jira] [Comment Edited] (ATLAS-1877) UI: In Lineage for Process entity process icon is not showing.
[
https://issues.apache.org/jira/browse/ATLAS-1877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16051850#comment-16051850
]
Keval Bhatt edited comment on ATLAS-1877 at 6/16/17 1:00 PM:
-
[~xiaqinglin] If entity it self is a Process or entity super type is process
then there is no need to check further.
you will check if it is not at first level or its is not a Process it self.
The above JSON Is list of Types (i.e entityDef) so you need to create one type
which is derived from Load Process.
and the create a entity from newly created type.
Steps.
# Create a type which has superType as LoadProcess .
URL - http://localhost:/api/atlas/v2/types/typedefs
{code} {
"entityDefs": [{
"category": "ENTITY",
"version": 1,
"name": "child_loadProcess",
"description": "child of load process",
"typeVersion": "1.0",
"attributeDefs": [{
"name": "test_att",
"typeName": "string",
"isOptional": false,
"cardinality": "SINGLE",
"valuesMinCount": 1,
"valuesMaxCount": 1,
"isUnique": false,
"isIndexable": true
}
],
"superTypes": [
"LoadProcess"
]
}
]
}
{code}
# Now create a entity from this newly created type.
# Then use this entity for Lineage.
For more details please go through atlas [Type
system|http://atlas.incubator.apache.org/TypeSystem.html] and [Rest API
|http://atlas.incubator.apache.org/api/v2/index.html] documentation
was (Author: kevalbhatt18):
[~xiaqinglin] If entity it self is a Process or entity super type is process
then there is no need to check further.
you will check if it is not at first level or its is not a Process it self.
The above JSON Is list of Types (i.e entityDef) so you need to create one type
which is derived from Load Process.
and the create a entity from newly created type.
Steps.
# Create a type which has superType as LoadProcess .
URL - http://localhost:/api/atlas/v2/types/typedefs
{code} {
"entityDefs": [{
"category": "ENTITY",
"version": 1,
"name": "child_loadProcess",
"description": "child of load process",
"typeVersion": "1.0",
"attributeDefs": [{
"name": "test_att",
"typeName": "string",
"isOptional": false,
"cardinality": "SINGLE",
"valuesMinCount": 1,
"valuesMaxCount": 1,
"isUnique": false,
"isIndexable": true
}
],
"superTypes": [
"LoadProcess"
]
}
]
}
{code}
# Now create a entity from this newly created type.
# Then use this entity for Lineage.
For more details please go through atlas [Type system
documentation|http://atlas.incubator.apache.org/TypeSystem.html] and [Rest API
|http://atlas.incubator.apache.org/api/v2/index.html]
> UI: In Lineage for Process entity process icon is not showing.
> --
>
> Key: ATLAS-1877
> URL: https://issues.apache.org/jira/browse/ATLAS-1877
> Project: Atlas
> Issue Type: Bug
>Affects Versions: 0.8-incubating
>Reporter: Keval Bhatt
>Assignee: qinglin,xia
> Labels: atlas
> Fix For: trunk, 0.9-incubating
>
> Attachments: 0001-fix-ATLAS-1877.patch, Process_Icon_Fixed.png,
> wrong_process_icon.png
>
>
> UI should check Process entity in all parent superType, right know its
> checking for only 1 level
> Step to reproduce :
> # Run quickstart
> # Create entity of type Process and while creating process select time_dim
> (Table entity) as a input and output.
> # Now got to sales_fact_monthly_mv entity and you will see new output (Please
> find attached screenshot )
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Comment Edited] (ATLAS-1877) UI: In Lineage for Process entity process icon is not showing.
[
https://issues.apache.org/jira/browse/ATLAS-1877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16051850#comment-16051850
]
Keval Bhatt edited comment on ATLAS-1877 at 6/16/17 12:59 PM:
--
[~xiaqinglin] If entity it self is a Process or entity super type is process
then there is no need to check further.
you will check if it is not at first level or its is not a Process it self.
The above JSON Is list of Types (i.e entityDef) so you need to create one type
which is derived from Load Process.
and the create a entity from newly created type.
Steps.
# Create a type which has superType as LoadProcess .
URL - http://localhost:/api/atlas/v2/types/typedefs
{code} {
"entityDefs": [{
"category": "ENTITY",
"version": 1,
"name": "child_loadProcess",
"description": "child of load process",
"typeVersion": "1.0",
"attributeDefs": [{
"name": "test_att",
"typeName": "string",
"isOptional": false,
"cardinality": "SINGLE",
"valuesMinCount": 1,
"valuesMaxCount": 1,
"isUnique": false,
"isIndexable": true
}
],
"superTypes": [
"LoadProcess"
]
}
]
}
{code}
# Now create a entity from this newly created type.
# Then use this entity for Lineage.
For more details please go through atlas [Type system
documentation|http://atlas.incubator.apache.org/TypeSystem.html] and [Rest API
|http://atlas.incubator.apache.org/api/v2/index.html]
was (Author: kevalbhatt18):
[~xiaqinglin] If entity it self is a Process or entity super type is process
then there is no need to check further.
you will check if it is not at first level or its is not a Process it self.
The above JSON Is list of Types (i.e entityDef) so you need to create one type
which is derived from Load Process.
and the create a entity from newly created type.
Steps.
# Create a type which has superType as LoadProcess .
URL - http://localhost:/api/atlas/v2/types/typedefs
{code} {
"entityDefs": [{
"category": "ENTITY",
"version": 1,
"name": "child_loadProcess",
"description": "child of load process",
"typeVersion": "1.0",
"attributeDefs": [{
"name": "test_att",
"typeName": "string",
"isOptional": false,
"cardinality": "SINGLE",
"valuesMinCount": 1,
"valuesMaxCount": 1,
"isUnique": false,
"isIndexable": true
}
],
"superTypes": [
"LoadProcess"
]
}
]
}
{code}
# Now create a entity from this newly created type.
# Then use this entity for Lineage.
> UI: In Lineage for Process entity process icon is not showing.
> --
>
> Key: ATLAS-1877
> URL: https://issues.apache.org/jira/browse/ATLAS-1877
> Project: Atlas
> Issue Type: Bug
>Affects Versions: 0.8-incubating
>Reporter: Keval Bhatt
>Assignee: qinglin,xia
> Labels: atlas
> Fix For: trunk, 0.9-incubating
>
> Attachments: 0001-fix-ATLAS-1877.patch, Process_Icon_Fixed.png,
> wrong_process_icon.png
>
>
> UI should check Process entity in all parent superType, right know its
> checking for only 1 level
> Step to reproduce :
> # Run quickstart
> # Create entity of type Process and while creating process select time_dim
> (Table entity) as a input and output.
> # Now got to sales_fact_monthly_mv entity and you will see new output (Please
> find attached screenshot )
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (ATLAS-1877) UI: In Lineage for Process entity process icon is not showing.
[
https://issues.apache.org/jira/browse/ATLAS-1877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16051850#comment-16051850
]
Keval Bhatt commented on ATLAS-1877:
[~xiaqinglin] If entity it self is a Process or entity super type is process
then there is no need to check further.
you will check if it is not at first level or its is not a Process it self.
The above JSON Is list of Types (i.e entityDef) so you need to create one type
which is derived from Load Process.
and the create a entity from newly created type.
Steps.
# Create a type which has superType as LoadProcess .
URL - http://localhost:/api/atlas/v2/types/typedefs
{code} {
"entityDefs": [{
"category": "ENTITY",
"version": 1,
"name": "child_loadProcess",
"description": "child of load process",
"typeVersion": "1.0",
"attributeDefs": [{
"name": "test_att",
"typeName": "string",
"isOptional": false,
"cardinality": "SINGLE",
"valuesMinCount": 1,
"valuesMaxCount": 1,
"isUnique": false,
"isIndexable": true
}
],
"superTypes": [
"LoadProcess"
]
}
]
}
{code}
# Now create a entity from this newly created type.
# Then use this entity for Lineage.
> UI: In Lineage for Process entity process icon is not showing.
> --
>
> Key: ATLAS-1877
> URL: https://issues.apache.org/jira/browse/ATLAS-1877
> Project: Atlas
> Issue Type: Bug
>Affects Versions: 0.8-incubating
>Reporter: Keval Bhatt
>Assignee: qinglin,xia
> Labels: atlas
> Fix For: trunk, 0.9-incubating
>
> Attachments: 0001-fix-ATLAS-1877.patch, Process_Icon_Fixed.png,
> wrong_process_icon.png
>
>
> UI should check Process entity in all parent superType, right know its
> checking for only 1 level
> Step to reproduce :
> # Run quickstart
> # Create entity of type Process and while creating process select time_dim
> (Table entity) as a input and output.
> # Now got to sales_fact_monthly_mv entity and you will see new output (Please
> find attached screenshot )
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (ATLAS-1877) UI: In Lineage for Process entity process icon is not showing.
[ https://issues.apache.org/jira/browse/ATLAS-1877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16051743#comment-16051743 ] qinglin,xia commented on ATLAS-1877: [~kevalbhatt] I just have a question of this, coz Process is already the superType, how can it be at other level except the first level? To reproduce the example you gave here, is it that I need to create four nested types: Process, e3,e2 and e1, and then create entities of the four types? > UI: In Lineage for Process entity process icon is not showing. > -- > > Key: ATLAS-1877 > URL: https://issues.apache.org/jira/browse/ATLAS-1877 > Project: Atlas > Issue Type: Bug >Affects Versions: 0.8-incubating >Reporter: Keval Bhatt >Assignee: qinglin,xia > Labels: atlas > Fix For: trunk, 0.9-incubating > > Attachments: 0001-fix-ATLAS-1877.patch, Process_Icon_Fixed.png, > wrong_process_icon.png > > > UI should check Process entity in all parent superType, right know its > checking for only 1 level > Step to reproduce : > # Run quickstart > # Create entity of type Process and while creating process select time_dim > (Table entity) as a input and output. > # Now got to sales_fact_monthly_mv entity and you will see new output (Please > find attached screenshot ) -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Comment Edited] (ATLAS-1877) UI: In Lineage for Process entity process icon is not showing.
[
https://issues.apache.org/jira/browse/ATLAS-1877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16051636#comment-16051636
]
Keval Bhatt edited comment on ATLAS-1877 at 6/16/17 9:22 AM:
-
[~xiaqinglin] In master we checking only one level of [superType |
https://github.com/apache/incubator-atlas/blob/master/dashboardv2/public/js/views/graph/LineageLayoutView.js#L128].
so if any entity which is derived from more then 1 level and Process entity is
not in first level the it will not work. To fix this issue you should check for
current entity is process or not and if it is not then search for all the
nested superType to check whether it has process entity in it.
Example:
{code}
[{
category:"ENTITY",
superTypes:['e2'],
name:"e1",
},
{
category:"ENTITY",
superTypes:['e3'],
name:"e2",
..
},{
category:"ENTITY",
superTypes:["Process"],
name:"e3",
..
},{
category:"ENTITY",
superTypes:[],
name:"Process",
..
}]
{code}
So now to check *e1* entity is process or not, you need to check in e2 -> e3
and in e3 you have process entity so e1 is process.
*Note:* To get all the nested type please use this
[Utility|https://github.com/apache/incubator-atlas/blob/master/dashboardv2/public/js/views/detail_page/DetailPageLayoutView.js#L245]
function.
was (Author: kevalbhatt18):
[~xiaqinglin] In master we checking only one level of [superType |
https://github.com/apache/incubator-atlas/blob/master/dashboardv2/public/js/views/graph/LineageLayoutView.js#L128].
so if any entity which is derived from more then 1 level and Process entity is
not in first level the it will not work. To fix this issue you should check for
current entity is process or not and if it is not then search for all the
nested superType to check whether it has process entity in it.
Example:
{code}
[{
category:"ENTITY",
superTypes:['e2'],
name:"e1",
},
{
category:"ENTITY",
superTypes:['e3'],
name:"e2",
..
},{
category:"ENTITY",
superTypes:["'Process"],
name:"e3",
..
},{
category:"ENTITY",
superTypes:[],
name:"Process",
..
}]
{code}
So now to check *e1* entity is process or not, you need to check in e2 -> e3
and in e3 you have process entity so e1 is process.
*Note:* To get all the nested type please use this
[Utility|https://github.com/apache/incubator-atlas/blob/master/dashboardv2/public/js/views/detail_page/DetailPageLayoutView.js#L245]
function.
> UI: In Lineage for Process entity process icon is not showing.
> --
>
> Key: ATLAS-1877
> URL: https://issues.apache.org/jira/browse/ATLAS-1877
> Project: Atlas
> Issue Type: Bug
>Affects Versions: 0.8-incubating
>Reporter: Keval Bhatt
>Assignee: qinglin,xia
> Labels: atlas
> Fix For: trunk, 0.9-incubating
>
> Attachments: 0001-fix-ATLAS-1877.patch, Process_Icon_Fixed.png,
> wrong_process_icon.png
>
>
> UI should check Process entity in all parent superType, right know its
> checking for only 1 level
> Step to reproduce :
> # Run quickstart
> # Create entity of type Process and while creating process select time_dim
> (Table entity) as a input and output.
> # Now got to sales_fact_monthly_mv entity and you will see new output (Please
> find attached screenshot )
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Comment Edited] (ATLAS-1877) UI: In Lineage for Process entity process icon is not showing.
[
https://issues.apache.org/jira/browse/ATLAS-1877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16051636#comment-16051636
]
Keval Bhatt edited comment on ATLAS-1877 at 6/16/17 9:22 AM:
-
[~xiaqinglin] In master we checking only one level of [superType |
https://github.com/apache/incubator-atlas/blob/master/dashboardv2/public/js/views/graph/LineageLayoutView.js#L128].
so if any entity which is derived from more then 1 level and Process entity is
not in first level the it will not work. To fix this issue you should check for
current entity is process or not and if it is not then search for all the
nested superType to check whether it has process entity in it.
Example:
{code}
[{
category:"ENTITY",
superTypes:['e2'],
name:"e1",
},
{
category:"ENTITY",
superTypes:['e3'],
name:"e2",
..
},{
category:"ENTITY",
superTypes:["'Process"],
name:"e3",
..
},{
category:"ENTITY",
superTypes:[],
name:"Process",
..
}]
{code}
So now to check *e1* entity is process or not, you need to check in e2 -> e3
and in e3 you have process entity so e1 is process.
*Note:* To get all the nested type please use this
[Utility|https://github.com/apache/incubator-atlas/blob/master/dashboardv2/public/js/views/detail_page/DetailPageLayoutView.js#L245]
function.
was (Author: kevalbhatt18):
[~xiaqinglin] In master we checking only one level of [superType |
https://github.com/apache/incubator-atlas/blob/master/dashboardv2/public/js/views/graph/LineageLayoutView.js#L128].
so if any entity which is derived from more then 1 level and Process entity is
not in first level the it will not work. To fix this issue you should check for
current entity is process or not and if it is not then search for all the
nested superType to check whether it has process entity in it.
Example:
{code}
[{
category:"ENTITY",
superTypes:['e2'],
name:"e1",
},
{
category:"ENTITY",
superTypes:['e3'],
name:"e2",
..
},{
category:"ENTITY",
superTypes:[''Process"],
name:"e3",
..
},,{
category:"ENTITY",
superTypes:[],
name:"Process",
..
}]
{code}
So now to check *e1* entity is process or not, you need to check in e2 -> e3
and in e3 you have process entity so e1 is process.
*Note:* To get all the nested type please use this
[Utility|https://github.com/apache/incubator-atlas/blob/master/dashboardv2/public/js/views/detail_page/DetailPageLayoutView.js#L245]
function.
> UI: In Lineage for Process entity process icon is not showing.
> --
>
> Key: ATLAS-1877
> URL: https://issues.apache.org/jira/browse/ATLAS-1877
> Project: Atlas
> Issue Type: Bug
>Affects Versions: 0.8-incubating
>Reporter: Keval Bhatt
>Assignee: qinglin,xia
> Labels: atlas
> Fix For: trunk, 0.9-incubating
>
> Attachments: 0001-fix-ATLAS-1877.patch, Process_Icon_Fixed.png,
> wrong_process_icon.png
>
>
> UI should check Process entity in all parent superType, right know its
> checking for only 1 level
> Step to reproduce :
> # Run quickstart
> # Create entity of type Process and while creating process select time_dim
> (Table entity) as a input and output.
> # Now got to sales_fact_monthly_mv entity and you will see new output (Please
> find attached screenshot )
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Comment Edited] (ATLAS-1877) UI: In Lineage for Process entity process icon is not showing.
[
https://issues.apache.org/jira/browse/ATLAS-1877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16051636#comment-16051636
]
Keval Bhatt edited comment on ATLAS-1877 at 6/16/17 9:12 AM:
-
[~xiaqinglin] In master we checking only one level of [superType |
https://github.com/apache/incubator-atlas/blob/master/dashboardv2/public/js/views/graph/LineageLayoutView.js#L128].
so if any entity which is derived from more then 1 level and Process entity is
not in first level the it will not work. To fix this issue you should check for
current entity is process or not and if it is not then search for all the
nested superType to check whether it has process entity in it.
Example:
{code}
[{
category:"ENTITY",
superTypes:['e2'],
name:"e1",
},
{
category:"ENTITY",
superTypes:['e3'],
name:"e2",
..
},{
category:"ENTITY",
superTypes:[''Process"],
name:"e3",
..
},,{
category:"ENTITY",
superTypes:[],
name:"Process",
..
}]
{code}
So now to check *e1* entity is process or not, you need to check in e2 -> e3
and in e3 you have process entity so e1 is process.
*Note:* To get all the nested type please use this
[Utility|https://github.com/apache/incubator-atlas/blob/master/dashboardv2/public/js/views/detail_page/DetailPageLayoutView.js#L245]
function.
was (Author: kevalbhatt18):
[~xiaqinglin] In master we checking only one level of [superType |
https://github.com/apache/incubator-atlas/blob/master/dashboardv2/public/js/views/graph/LineageLayoutView.js#L128].
so if any entity which is derived from more then 1 level and Process entity is
not in first level the it will not work. To fix this issue you should check for
current entity is process or not and if it is not then search for all the
nested superType to check whether it has process entity in it.
Example:
{code}
[{
category:"ENTITY",
superTypes:['e2'],
name:"e1",
},
{
category:"ENTITY",
superTypes:['e3'],
name:"e2",
..
},{
category:"ENTITY",
superTypes:[''Process"],
name:"e3",
..
},,{
category:"ENTITY",
superTypes:[],
name:"Process",
..
}]
{code}
So now to check *e1* entity is process or not, you need to check in e2 -> e3
and in e3 you have process entity so e1 is process.
To get all the nested type please follow this
[Utility|https://github.com/apache/incubator-atlas/blob/master/dashboardv2/public/js/views/detail_page/DetailPageLayoutView.js#L245]
> UI: In Lineage for Process entity process icon is not showing.
> --
>
> Key: ATLAS-1877
> URL: https://issues.apache.org/jira/browse/ATLAS-1877
> Project: Atlas
> Issue Type: Bug
>Affects Versions: 0.8-incubating
>Reporter: Keval Bhatt
>Assignee: qinglin,xia
> Labels: atlas
> Fix For: trunk, 0.9-incubating
>
> Attachments: 0001-fix-ATLAS-1877.patch, Process_Icon_Fixed.png,
> wrong_process_icon.png
>
>
> UI should check Process entity in all parent superType, right know its
> checking for only 1 level
> Step to reproduce :
> # Run quickstart
> # Create entity of type Process and while creating process select time_dim
> (Table entity) as a input and output.
> # Now got to sales_fact_monthly_mv entity and you will see new output (Please
> find attached screenshot )
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Comment Edited] (ATLAS-1877) UI: In Lineage for Process entity process icon is not showing.
[
https://issues.apache.org/jira/browse/ATLAS-1877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16051636#comment-16051636
]
Keval Bhatt edited comment on ATLAS-1877 at 6/16/17 9:10 AM:
-
[~xiaqinglin] In master we checking only one level of [superType |
https://github.com/apache/incubator-atlas/blob/master/dashboardv2/public/js/views/graph/LineageLayoutView.js#L128].
so if any entity which is derived from more then 1 level and Process entity is
not in first level the it will not work. To fix this issue you should check for
current entity is process or not and if it is not then search for all the
nested superType to check whether it has process entity in it.
Example:
{code}
[{
category:"ENTITY",
superTypes:['e2'],
name:"e1",
},
{
category:"ENTITY",
superTypes:['e3'],
name:"e2",
..
},{
category:"ENTITY",
superTypes:[''Process"],
name:"e3",
..
},,{
category:"ENTITY",
superTypes:[],
name:"Process",
..
}]
{code}
So now to check *e1* entity is process or not, you need to check in e2 -> e3
and in e3 you have process entity so e1 is process.
To get all the nested type please follow this
[Utility|https://github.com/apache/incubator-atlas/blob/master/dashboardv2/public/js/views/detail_page/DetailPageLayoutView.js#L245]
was (Author: kevalbhatt18):
[~xiaqinglin] In master we checking only one level of [superType |
https://github.com/apache/incubator-atlas/blob/master/dashboardv2/public/js/views/graph/LineageLayoutView.js#L128].
so if any entity which is derived from more then 1 level and Process entity is
not in first level the it will not work. To fix this issue you should check for
current entity is process or not and if it is not then search for all the
nested superType to check whether it has process entity in it.
Example:
{code}
[{
category:"ENTITY",
superTypes:['e2'],
name:"e1",
},
{
category:"ENTITY",
superTypes:['e3'],
name:"e2",
..
},{
category:"ENTITY",
superTypes:[''Process"],
name:"e3",
..
},,{
category:"ENTITY",
superTypes:[],
name:"Process",
..
}]
{code}
So now to check *e1* entity is process or not, you need to check in e2 -> e3
and in e3 you have process entity so e1 is process.
> UI: In Lineage for Process entity process icon is not showing.
> --
>
> Key: ATLAS-1877
> URL: https://issues.apache.org/jira/browse/ATLAS-1877
> Project: Atlas
> Issue Type: Bug
>Affects Versions: 0.8-incubating
>Reporter: Keval Bhatt
>Assignee: qinglin,xia
> Labels: atlas
> Fix For: trunk, 0.9-incubating
>
> Attachments: 0001-fix-ATLAS-1877.patch, Process_Icon_Fixed.png,
> wrong_process_icon.png
>
>
> UI should check Process entity in all parent superType, right know its
> checking for only 1 level
> Step to reproduce :
> # Run quickstart
> # Create entity of type Process and while creating process select time_dim
> (Table entity) as a input and output.
> # Now got to sales_fact_monthly_mv entity and you will see new output (Please
> find attached screenshot )
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (ATLAS-1877) UI: In Lineage for Process entity process icon is not showing.
[
https://issues.apache.org/jira/browse/ATLAS-1877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16051636#comment-16051636
]
Keval Bhatt commented on ATLAS-1877:
[~xiaqinglin] In master we checking only one level of [superType |
https://github.com/apache/incubator-atlas/blob/master/dashboardv2/public/js/views/graph/LineageLayoutView.js#L128].
so if any entity which is derived from more then 1 level and Process entity is
not in first level the it will not work. To fix this issue you should check for
current entity is process or not and if it is not then search for all the
nested superType to check whether it has process entity in it.
Example:
{code}
[{
category:"ENTITY",
superTypes:['e2'],
name:"e1",
},
{
category:"ENTITY",
superTypes:['e3'],
name:"e2",
..
},{
category:"ENTITY",
superTypes:[''Process"],
name:"e3",
..
},,{
category:"ENTITY",
superTypes:[],
name:"Process",
..
}]
{code}
So now to check *e1* entity is process or not, you need to check in e2 -> e3
and in e3 you have process entity so e1 is process.
> UI: In Lineage for Process entity process icon is not showing.
> --
>
> Key: ATLAS-1877
> URL: https://issues.apache.org/jira/browse/ATLAS-1877
> Project: Atlas
> Issue Type: Bug
>Affects Versions: 0.8-incubating
>Reporter: Keval Bhatt
>Assignee: qinglin,xia
> Labels: atlas
> Fix For: trunk, 0.9-incubating
>
> Attachments: 0001-fix-ATLAS-1877.patch, Process_Icon_Fixed.png,
> wrong_process_icon.png
>
>
> UI should check Process entity in all parent superType, right know its
> checking for only 1 level
> Step to reproduce :
> # Run quickstart
> # Create entity of type Process and while creating process select time_dim
> (Table entity) as a input and output.
> # Now got to sales_fact_monthly_mv entity and you will see new output (Please
> find attached screenshot )
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (ATLAS-1875) Gremlin id is no longer returned for vertices in gremlin query
[
https://issues.apache.org/jira/browse/ATLAS-1875?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16051623#comment-16051623
]
ASF GitHub Bot commented on ATLAS-1875:
---
GitHub user cmar81 opened a pull request:
https://github.com/apache/incubator-atlas/pull/36
[ATLAS-1875] Adding the gremlin id for a vertex back into the search …
…result for a gremlin search. This functionality was present in Atlas 0.7
but disappeared when an abstraction layer was introduced
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/cmar81/incubator-atlas master
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/incubator-atlas/pull/36.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #36
commit 485775868dae4afb1a27fe9f0c2c5bb4074a0d93
Author: Christian Rieck
Date: 2017-06-16T08:45:22Z
[ATLAS-1875] Adding the gremlin id for a vertex back into the search result
for a gremlin search. This functionality was present in Atlas 0.7 but
disappeared when an abstraction layer was introduced
> Gremlin id is no longer returned for vertices in gremlin query
> --
>
> Key: ATLAS-1875
> URL: https://issues.apache.org/jira/browse/ATLAS-1875
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Affects Versions: trunk, 0.8-incubating
>Reporter: Christian R
> Labels: dsl, gremlin
>
> Hi,
> while investigating a move from atlas 0.7 to 0.8 (HDP 2.5 to HDP2.6) our
> tests fail on gremlin queries. It turns out that the returned entities in a
> gremlin search in 0.8 does not include the 'id' attribute.
> I've built commit a0bd93945cd45457bbf34a8cb819d4fa4ba72964 (0.8-rc1) on linux
> using berkely and elasticearch to test with.
> The query
> :21000/api/atlas/discovery/search/gremlin?g.V.has('__type.name',
> 'Infrastructure').collect()
> on our 0.7 cluster gives
> {code:json}
> {
> __type.name: "Infrastructure",
> __type.category: "CLASS",
> __type: "typeSystem",
> id: "16640"
> }{code}
> while the same query on my 0.8-rc1 installation gives
> {code:javascript}
> {
> __type.name: "Infrastructure",
> __version: "1",
> __type.category: "CLASS",
> __type.version: "1.0",
> __modificationTimestamp: "1497448424134",
> __type: "typeSystem",
> __type.options: "null",
> __type.description: "Infrastructure",
> __guid: "77d07283-7622-4305-9c0c-09ac5aee86c8",
> __timestamp: "1497448424134"
> }
> {code}
> Certainly more information, but id is missing.
> The very poor DSL performance (see ATLAS-1868) and a need for advanced
> queries led us to base our queries on gremlin. This has worked very well so
> far. We include both edges and nodes in the result set and use the inVertex,
> outVertex and label info on the edges to rebuild our tree on the client side.
> I also see that gremlin has disappeared from version two of the API. Since
> addE and addV lets you insert into the graph I can see how exposing a full
> gremlin endpoint might not be wanted.
> As an example of the queries we run that I haven't been able to express in
> the DSL is
> {noformat}
> query = g.V.has('__guid','').copySplit(
> _().out('track'),
> _().as('x')
> .out('functions', 'component')
> .loop('x'){true}{true}
> .copySplit(
> _(),
> _().in('part_of'),
> _().outE('functions',
> 'component'),
> _().inE('part_of')
> )
> .exhaustMerge.dedup
> )
> .exhaustMerge
> .collect()
> {noformat}
> this might not be a normal usecase.
> edit to add:
> I looked at GraphBackedDiscoveryService.java and notice that:
> 0.7:
> {code:java}
> else if (r instanceof TitanVertex) {
> Iterable ps = ((TitanVertex)
> r).get
[jira] [Commented] (ATLAS-1875) Gremlin id is no longer returned for vertices in gremlin query
[
https://issues.apache.org/jira/browse/ATLAS-1875?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16051592#comment-16051592
]
Christian R commented on ATLAS-1875:
Git does have its learning curve :)
This will give me some experience in contributing to larger projects as well.
> Gremlin id is no longer returned for vertices in gremlin query
> --
>
> Key: ATLAS-1875
> URL: https://issues.apache.org/jira/browse/ATLAS-1875
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Affects Versions: trunk, 0.8-incubating
>Reporter: Christian R
> Labels: dsl, gremlin
>
> Hi,
> while investigating a move from atlas 0.7 to 0.8 (HDP 2.5 to HDP2.6) our
> tests fail on gremlin queries. It turns out that the returned entities in a
> gremlin search in 0.8 does not include the 'id' attribute.
> I've built commit a0bd93945cd45457bbf34a8cb819d4fa4ba72964 (0.8-rc1) on linux
> using berkely and elasticearch to test with.
> The query
> :21000/api/atlas/discovery/search/gremlin?g.V.has('__type.name',
> 'Infrastructure').collect()
> on our 0.7 cluster gives
> {code:json}
> {
> __type.name: "Infrastructure",
> __type.category: "CLASS",
> __type: "typeSystem",
> id: "16640"
> }{code}
> while the same query on my 0.8-rc1 installation gives
> {code:javascript}
> {
> __type.name: "Infrastructure",
> __version: "1",
> __type.category: "CLASS",
> __type.version: "1.0",
> __modificationTimestamp: "1497448424134",
> __type: "typeSystem",
> __type.options: "null",
> __type.description: "Infrastructure",
> __guid: "77d07283-7622-4305-9c0c-09ac5aee86c8",
> __timestamp: "1497448424134"
> }
> {code}
> Certainly more information, but id is missing.
> The very poor DSL performance (see ATLAS-1868) and a need for advanced
> queries led us to base our queries on gremlin. This has worked very well so
> far. We include both edges and nodes in the result set and use the inVertex,
> outVertex and label info on the edges to rebuild our tree on the client side.
> I also see that gremlin has disappeared from version two of the API. Since
> addE and addV lets you insert into the graph I can see how exposing a full
> gremlin endpoint might not be wanted.
> As an example of the queries we run that I haven't been able to express in
> the DSL is
> {noformat}
> query = g.V.has('__guid','').copySplit(
> _().out('track'),
> _().as('x')
> .out('functions', 'component')
> .loop('x'){true}{true}
> .copySplit(
> _(),
> _().in('part_of'),
> _().outE('functions',
> 'component'),
> _().inE('part_of')
> )
> .exhaustMerge.dedup
> )
> .exhaustMerge
> .collect()
> {noformat}
> this might not be a normal usecase.
> edit to add:
> I looked at GraphBackedDiscoveryService.java and notice that:
> 0.7:
> {code:java}
> else if (r instanceof TitanVertex) {
> Iterable ps = ((TitanVertex)
> r).getProperties();
> for (TitanProperty tP : ps) {
> String pName = tP.getPropertyKey().getName();
> Object pValue = ((TitanVertex) r).getProperty(pName);
> if (pValue != null) {
> oRow.put(pName, pValue.toString());
> }
> }
> {code}
> Vs 0.8 code:
> {code:java}
> else if (value instanceof AtlasVertex) {
> AtlasVertex vertex = (AtlasVertex)value;
> for (String key : vertex.getPropertyKeys()) {
> Object propertyValue =
> GraphHelper.getProperty(vertex, key);
> if (propertyValue != null) {
> oRow.put(key, propertyValue.toString());
> }
> }
> {code}
>
[jira] [Commented] (ATLAS-1875) Gremlin id is no longer returned for vertices in gremlin query
[
https://issues.apache.org/jira/browse/ATLAS-1875?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16051578#comment-16051578
]
Graham Wallis commented on ATLAS-1875:
--
Hi Christian
If you're happy to do that it would be great. If not then I'm happy to do it
(although it would probably take me a little longer as I am not yet very
confident with git, especially on earlier releases - by the way thanks for your
tips on how to do that :)
Do you want to test for 'label' on edges as well? I'm not sure whether it's
important, but just note that it is included in the edge result, although
intriguingly 'label' was never included for vertices.
> Gremlin id is no longer returned for vertices in gremlin query
> --
>
> Key: ATLAS-1875
> URL: https://issues.apache.org/jira/browse/ATLAS-1875
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Affects Versions: trunk, 0.8-incubating
>Reporter: Christian R
> Labels: dsl, gremlin
>
> Hi,
> while investigating a move from atlas 0.7 to 0.8 (HDP 2.5 to HDP2.6) our
> tests fail on gremlin queries. It turns out that the returned entities in a
> gremlin search in 0.8 does not include the 'id' attribute.
> I've built commit a0bd93945cd45457bbf34a8cb819d4fa4ba72964 (0.8-rc1) on linux
> using berkely and elasticearch to test with.
> The query
> :21000/api/atlas/discovery/search/gremlin?g.V.has('__type.name',
> 'Infrastructure').collect()
> on our 0.7 cluster gives
> {code:json}
> {
> __type.name: "Infrastructure",
> __type.category: "CLASS",
> __type: "typeSystem",
> id: "16640"
> }{code}
> while the same query on my 0.8-rc1 installation gives
> {code:javascript}
> {
> __type.name: "Infrastructure",
> __version: "1",
> __type.category: "CLASS",
> __type.version: "1.0",
> __modificationTimestamp: "1497448424134",
> __type: "typeSystem",
> __type.options: "null",
> __type.description: "Infrastructure",
> __guid: "77d07283-7622-4305-9c0c-09ac5aee86c8",
> __timestamp: "1497448424134"
> }
> {code}
> Certainly more information, but id is missing.
> The very poor DSL performance (see ATLAS-1868) and a need for advanced
> queries led us to base our queries on gremlin. This has worked very well so
> far. We include both edges and nodes in the result set and use the inVertex,
> outVertex and label info on the edges to rebuild our tree on the client side.
> I also see that gremlin has disappeared from version two of the API. Since
> addE and addV lets you insert into the graph I can see how exposing a full
> gremlin endpoint might not be wanted.
> As an example of the queries we run that I haven't been able to express in
> the DSL is
> {noformat}
> query = g.V.has('__guid','').copySplit(
> _().out('track'),
> _().as('x')
> .out('functions', 'component')
> .loop('x'){true}{true}
> .copySplit(
> _(),
> _().in('part_of'),
> _().outE('functions',
> 'component'),
> _().inE('part_of')
> )
> .exhaustMerge.dedup
> )
> .exhaustMerge
> .collect()
> {noformat}
> this might not be a normal usecase.
> edit to add:
> I looked at GraphBackedDiscoveryService.java and notice that:
> 0.7:
> {code:java}
> else if (r instanceof TitanVertex) {
> Iterable ps = ((TitanVertex)
> r).getProperties();
> for (TitanProperty tP : ps) {
> String pName = tP.getPropertyKey().getName();
> Object pValue = ((TitanVertex) r).getProperty(pName);
> if (pValue != null) {
> oRow.put(pName, pValue.toString());
> }
> }
> {code}
> Vs 0.8 code:
> {code:java}
> else if (value instanceof AtlasVertex) {
> AtlasVertex
[jira] [Assigned] (ATLAS-1877) UI: In Lineage for Process entity process icon is not showing.
[ https://issues.apache.org/jira/browse/ATLAS-1877?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] qinglin,xia reassigned ATLAS-1877: -- Assignee: qinglin,xia Attachment: 0001-fix-ATLAS-1877.patch Process_Icon_Fixed.png > UI: In Lineage for Process entity process icon is not showing. > -- > > Key: ATLAS-1877 > URL: https://issues.apache.org/jira/browse/ATLAS-1877 > Project: Atlas > Issue Type: Bug >Reporter: Keval Bhatt >Assignee: qinglin,xia > Attachments: 0001-fix-ATLAS-1877.patch, Process_Icon_Fixed.png, > wrong_process_icon.png > > > UI should check Process entity in all parent superType, right know its > checking for only 1 level > Step to reproduce : > # Run quickstart > # Create entity of type Process and while creating process select time_dim > (Table entity) as a input and output. > # Now got to sales_fact_monthly_mv entity and you will see new output (Please > find attached screenshot ) -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Comment Edited] (ATLAS-1875) Gremlin id is no longer returned for vertices in gremlin query
[
https://issues.apache.org/jira/browse/ATLAS-1875?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16051537#comment-16051537
]
Christian R edited comment on ATLAS-1875 at 6/16/17 7:58 AM:
-
Well, this is great news!
How to best proceed from here? Shall I make a pull request against master and
another one against 0.8?
I am writing a simple test now to verify that vertexes do return id , and that
edges return invertex,outvertex.
was (Author: christianmr):
Well, this is great news!
How to best proceed from here? Shall I make a pull request against master and
another one against 0.8?
I am writing a simple test now to verify that vertexes do return id , and that
edges return inedge,outedge.
> Gremlin id is no longer returned for vertices in gremlin query
> --
>
> Key: ATLAS-1875
> URL: https://issues.apache.org/jira/browse/ATLAS-1875
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Affects Versions: trunk, 0.8-incubating
>Reporter: Christian R
> Labels: dsl, gremlin
>
> Hi,
> while investigating a move from atlas 0.7 to 0.8 (HDP 2.5 to HDP2.6) our
> tests fail on gremlin queries. It turns out that the returned entities in a
> gremlin search in 0.8 does not include the 'id' attribute.
> I've built commit a0bd93945cd45457bbf34a8cb819d4fa4ba72964 (0.8-rc1) on linux
> using berkely and elasticearch to test with.
> The query
> :21000/api/atlas/discovery/search/gremlin?g.V.has('__type.name',
> 'Infrastructure').collect()
> on our 0.7 cluster gives
> {code:json}
> {
> __type.name: "Infrastructure",
> __type.category: "CLASS",
> __type: "typeSystem",
> id: "16640"
> }{code}
> while the same query on my 0.8-rc1 installation gives
> {code:javascript}
> {
> __type.name: "Infrastructure",
> __version: "1",
> __type.category: "CLASS",
> __type.version: "1.0",
> __modificationTimestamp: "1497448424134",
> __type: "typeSystem",
> __type.options: "null",
> __type.description: "Infrastructure",
> __guid: "77d07283-7622-4305-9c0c-09ac5aee86c8",
> __timestamp: "1497448424134"
> }
> {code}
> Certainly more information, but id is missing.
> The very poor DSL performance (see ATLAS-1868) and a need for advanced
> queries led us to base our queries on gremlin. This has worked very well so
> far. We include both edges and nodes in the result set and use the inVertex,
> outVertex and label info on the edges to rebuild our tree on the client side.
> I also see that gremlin has disappeared from version two of the API. Since
> addE and addV lets you insert into the graph I can see how exposing a full
> gremlin endpoint might not be wanted.
> As an example of the queries we run that I haven't been able to express in
> the DSL is
> {noformat}
> query = g.V.has('__guid','').copySplit(
> _().out('track'),
> _().as('x')
> .out('functions', 'component')
> .loop('x'){true}{true}
> .copySplit(
> _(),
> _().in('part_of'),
> _().outE('functions',
> 'component'),
> _().inE('part_of')
> )
> .exhaustMerge.dedup
> )
> .exhaustMerge
> .collect()
> {noformat}
> this might not be a normal usecase.
> edit to add:
> I looked at GraphBackedDiscoveryService.java and notice that:
> 0.7:
> {code:java}
> else if (r instanceof TitanVertex) {
> Iterable ps = ((TitanVertex)
> r).getProperties();
> for (TitanProperty tP : ps) {
> String pName = tP.getPropertyKey().getName();
> Object pValue = ((TitanVertex) r).getProperty(pName);
> if (pValue != null) {
> oRow.put(pName, pValue.toString());
> }
> }
> {code}
> Vs 0.8 code:
> {code:java}
> else if (value instanceof AtlasVertex) {
>
[jira] [Comment Edited] (ATLAS-1875) Gremlin id is no longer returned for vertices in gremlin query
[
https://issues.apache.org/jira/browse/ATLAS-1875?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16051537#comment-16051537
]
Christian R edited comment on ATLAS-1875 at 6/16/17 7:51 AM:
-
Well, this is great news!
How to best proceed from here? Shall I make a pull request against master and
another one against 0.8?
I am writing a simple test now to verify that vertexes do return id , and that
edges return inedge,outedge.
was (Author: christianmr):
Well, that was great news!
How to best proceed from here? Shall I make a pull request against master and
another one against 0.8?
I am writing a simple test now to verify that vertexes do return id , and that
edges return inedge,outedge.
> Gremlin id is no longer returned for vertices in gremlin query
> --
>
> Key: ATLAS-1875
> URL: https://issues.apache.org/jira/browse/ATLAS-1875
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Affects Versions: trunk, 0.8-incubating
>Reporter: Christian R
> Labels: dsl, gremlin
>
> Hi,
> while investigating a move from atlas 0.7 to 0.8 (HDP 2.5 to HDP2.6) our
> tests fail on gremlin queries. It turns out that the returned entities in a
> gremlin search in 0.8 does not include the 'id' attribute.
> I've built commit a0bd93945cd45457bbf34a8cb819d4fa4ba72964 (0.8-rc1) on linux
> using berkely and elasticearch to test with.
> The query
> :21000/api/atlas/discovery/search/gremlin?g.V.has('__type.name',
> 'Infrastructure').collect()
> on our 0.7 cluster gives
> {code:json}
> {
> __type.name: "Infrastructure",
> __type.category: "CLASS",
> __type: "typeSystem",
> id: "16640"
> }{code}
> while the same query on my 0.8-rc1 installation gives
> {code:javascript}
> {
> __type.name: "Infrastructure",
> __version: "1",
> __type.category: "CLASS",
> __type.version: "1.0",
> __modificationTimestamp: "1497448424134",
> __type: "typeSystem",
> __type.options: "null",
> __type.description: "Infrastructure",
> __guid: "77d07283-7622-4305-9c0c-09ac5aee86c8",
> __timestamp: "1497448424134"
> }
> {code}
> Certainly more information, but id is missing.
> The very poor DSL performance (see ATLAS-1868) and a need for advanced
> queries led us to base our queries on gremlin. This has worked very well so
> far. We include both edges and nodes in the result set and use the inVertex,
> outVertex and label info on the edges to rebuild our tree on the client side.
> I also see that gremlin has disappeared from version two of the API. Since
> addE and addV lets you insert into the graph I can see how exposing a full
> gremlin endpoint might not be wanted.
> As an example of the queries we run that I haven't been able to express in
> the DSL is
> {noformat}
> query = g.V.has('__guid','').copySplit(
> _().out('track'),
> _().as('x')
> .out('functions', 'component')
> .loop('x'){true}{true}
> .copySplit(
> _(),
> _().in('part_of'),
> _().outE('functions',
> 'component'),
> _().inE('part_of')
> )
> .exhaustMerge.dedup
> )
> .exhaustMerge
> .collect()
> {noformat}
> this might not be a normal usecase.
> edit to add:
> I looked at GraphBackedDiscoveryService.java and notice that:
> 0.7:
> {code:java}
> else if (r instanceof TitanVertex) {
> Iterable ps = ((TitanVertex)
> r).getProperties();
> for (TitanProperty tP : ps) {
> String pName = tP.getPropertyKey().getName();
> Object pValue = ((TitanVertex) r).getProperty(pName);
> if (pValue != null) {
> oRow.put(pName, pValue.toString());
> }
> }
> {code}
> Vs 0.8 code:
> {code:java}
> else if (value instanceof AtlasVertex) {
>
[jira] [Commented] (ATLAS-1875) Gremlin id is no longer returned for vertices in gremlin query
[
https://issues.apache.org/jira/browse/ATLAS-1875?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16051537#comment-16051537
]
Christian R commented on ATLAS-1875:
Well, that was great news!
How to best proceed from here? Shall I make a pull request against master and
another one against 0.8?
I am writing a simple test now to verify that vertexes do return id , and that
edges return inedge,outedge.
> Gremlin id is no longer returned for vertices in gremlin query
> --
>
> Key: ATLAS-1875
> URL: https://issues.apache.org/jira/browse/ATLAS-1875
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Affects Versions: trunk, 0.8-incubating
>Reporter: Christian R
> Labels: dsl, gremlin
>
> Hi,
> while investigating a move from atlas 0.7 to 0.8 (HDP 2.5 to HDP2.6) our
> tests fail on gremlin queries. It turns out that the returned entities in a
> gremlin search in 0.8 does not include the 'id' attribute.
> I've built commit a0bd93945cd45457bbf34a8cb819d4fa4ba72964 (0.8-rc1) on linux
> using berkely and elasticearch to test with.
> The query
> :21000/api/atlas/discovery/search/gremlin?g.V.has('__type.name',
> 'Infrastructure').collect()
> on our 0.7 cluster gives
> {code:json}
> {
> __type.name: "Infrastructure",
> __type.category: "CLASS",
> __type: "typeSystem",
> id: "16640"
> }{code}
> while the same query on my 0.8-rc1 installation gives
> {code:javascript}
> {
> __type.name: "Infrastructure",
> __version: "1",
> __type.category: "CLASS",
> __type.version: "1.0",
> __modificationTimestamp: "1497448424134",
> __type: "typeSystem",
> __type.options: "null",
> __type.description: "Infrastructure",
> __guid: "77d07283-7622-4305-9c0c-09ac5aee86c8",
> __timestamp: "1497448424134"
> }
> {code}
> Certainly more information, but id is missing.
> The very poor DSL performance (see ATLAS-1868) and a need for advanced
> queries led us to base our queries on gremlin. This has worked very well so
> far. We include both edges and nodes in the result set and use the inVertex,
> outVertex and label info on the edges to rebuild our tree on the client side.
> I also see that gremlin has disappeared from version two of the API. Since
> addE and addV lets you insert into the graph I can see how exposing a full
> gremlin endpoint might not be wanted.
> As an example of the queries we run that I haven't been able to express in
> the DSL is
> {noformat}
> query = g.V.has('__guid','').copySplit(
> _().out('track'),
> _().as('x')
> .out('functions', 'component')
> .loop('x'){true}{true}
> .copySplit(
> _(),
> _().in('part_of'),
> _().outE('functions',
> 'component'),
> _().inE('part_of')
> )
> .exhaustMerge.dedup
> )
> .exhaustMerge
> .collect()
> {noformat}
> this might not be a normal usecase.
> edit to add:
> I looked at GraphBackedDiscoveryService.java and notice that:
> 0.7:
> {code:java}
> else if (r instanceof TitanVertex) {
> Iterable ps = ((TitanVertex)
> r).getProperties();
> for (TitanProperty tP : ps) {
> String pName = tP.getPropertyKey().getName();
> Object pValue = ((TitanVertex) r).getProperty(pName);
> if (pValue != null) {
> oRow.put(pName, pValue.toString());
> }
> }
> {code}
> Vs 0.8 code:
> {code:java}
> else if (value instanceof AtlasVertex) {
> AtlasVertex vertex = (AtlasVertex)value;
> for (String key : vertex.getPropertyKeys()) {
> Object propertyValue =
> GraphHelper.getProperty(vertex, key);
> if (propertyValue != null) {
> o
[jira] [Created] (ATLAS-1878) NPE when a request without any query path lands on atlas
Ayub Pathan created ATLAS-1878:
--
Summary: NPE when a request without any query path lands on atlas
Key: ATLAS-1878
URL: https://issues.apache.org/jira/browse/ATLAS-1878
Project: Atlas
Issue Type: Bug
Components: atlas-core
Affects Versions: 0.9-incubating
Reporter: Ayub Pathan
Priority: Critical
Fix For: 0.9-incubating
{noformat}
2017-06-16 07:02:13,515 WARN - [pool-2-thread-10:] ~ got exception trying to
get groups for user admin: id: admin: no such user
id: admin: no such user
(ShellBasedUnixGroupsMapping:87)
2017-06-16 07:02:13,515 ERROR - [pool-2-thread-10:] ~ Exception while fetching
groups (AtlasAbstractAuthenticationProvider:120)
java.io.IOException: No groups found for user admin
at org.apache.hadoop.security.Groups.noGroupsForUser(Groups.java:190)
at org.apache.hadoop.security.Groups.access$400(Groups.java:69)
at org.apache.hadoop.security.Groups$GroupCacheLoader.load(Groups.java:307)
at org.apache.hadoop.security.Groups$GroupCacheLoader.load(Groups.java:257)
at
com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3542)
at com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2323)
at
com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2286)
at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2201)
at com.google.common.cache.LocalCache.get(LocalCache.java:3953)
at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3957)
at
com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4875)
at org.apache.hadoop.security.Groups.getGroups(Groups.java:215)
at
org.apache.atlas.web.security.AtlasAbstractAuthenticationProvider.getAuthoritiesFromUGI(AtlasAbstractAuthenticationProvider.java:113)
at
org.apache.atlas.web.filters.AtlasKnoxSSOAuthenticationFilter.doFilter(AtlasKnoxSSOAuthenticationFilter.java:171)
at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
at
org.apache.atlas.web.filters.StaleTransactionCleanupFilter.doFilter(StaleTransactionCleanupFilter.java:55)
at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
at
org.apache.atlas.web.filters.ActiveServerFilter.doFilter(ActiveServerFilter.java:85)
at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
at
org.springframework.security.web.authentication.www.BasicAuthenticationFilter.doFilterInternal(BasicAuthenticationFilter.java:158)
at
org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
at
org.springframework.security.web.authentication.AbstractAuthenticationProcessingFilter.doFilter(AbstractAuthenticationProcessingFilter.java:200)
at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
at
org.springframework.security.web.authentication.logout.LogoutFilter.doFilter(LogoutFilter.java:116)
at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
at
org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:105)
at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
at
org.springframework.security.web.context.request.async.WebAsyncManagerIntegrationFilter.doFilterInternal(WebAsyncManagerIntegrationFilter.java:56)
at
org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
at
org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:214)
at
org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:177)
at
org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:346)
at
org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:262)
at
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:577)
at
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:223)
at
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)
at
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185
[jira] [Updated] (ATLAS-1856) Relationship instance API Java & REST implementation
[ https://issues.apache.org/jira/browse/ATLAS-1856?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sarath Subramanian updated ATLAS-1856: -- Attachment: curl_create_new_relationDef curl_create_new_relationInstance ATLAS-1856.2.patch > Relationship instance API Java & REST implementation > > > Key: ATLAS-1856 > URL: https://issues.apache.org/jira/browse/ATLAS-1856 > Project: Atlas > Issue Type: New Feature > Components: atlas-core >Affects Versions: trunk, 0.9-incubating >Reporter: Sarath Subramanian >Assignee: Sarath Subramanian > Attachments: ATLAS-1856.2.patch, curl_create_new_relationDef, > curl_create_new_relationInstance > > > Based on the relationDef implementation in ATLAS-1852. Implement instance API > and REST for entity relationships -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (ATLAS-1856) Relationship instance API Java & REST implementation
[ https://issues.apache.org/jira/browse/ATLAS-1856?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sarath Subramanian updated ATLAS-1856: -- Attachment: (was: curl_create_new_relationDef) > Relationship instance API Java & REST implementation > > > Key: ATLAS-1856 > URL: https://issues.apache.org/jira/browse/ATLAS-1856 > Project: Atlas > Issue Type: New Feature > Components: atlas-core >Affects Versions: trunk, 0.9-incubating >Reporter: Sarath Subramanian >Assignee: Sarath Subramanian > > Based on the relationDef implementation in ATLAS-1852. Implement instance API > and REST for entity relationships -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (ATLAS-1856) Relationship instance API Java & REST implementation
[ https://issues.apache.org/jira/browse/ATLAS-1856?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sarath Subramanian updated ATLAS-1856: -- Attachment: (was: curl_create_new_relationInstance) > Relationship instance API Java & REST implementation > > > Key: ATLAS-1856 > URL: https://issues.apache.org/jira/browse/ATLAS-1856 > Project: Atlas > Issue Type: New Feature > Components: atlas-core >Affects Versions: trunk, 0.9-incubating >Reporter: Sarath Subramanian >Assignee: Sarath Subramanian > > Based on the relationDef implementation in ATLAS-1852. Implement instance API > and REST for entity relationships -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (ATLAS-1856) Relationship instance API Java & REST implementation
[ https://issues.apache.org/jira/browse/ATLAS-1856?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sarath Subramanian updated ATLAS-1856: -- Attachment: (was: ATLAS-1856.1.patch) > Relationship instance API Java & REST implementation > > > Key: ATLAS-1856 > URL: https://issues.apache.org/jira/browse/ATLAS-1856 > Project: Atlas > Issue Type: New Feature > Components: atlas-core >Affects Versions: trunk, 0.9-incubating >Reporter: Sarath Subramanian >Assignee: Sarath Subramanian > > Based on the relationDef implementation in ATLAS-1852. Implement instance API > and REST for entity relationships -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (ATLAS-1875) Gremlin id is no longer returned for vertices in gremlin query
[
https://issues.apache.org/jira/browse/ATLAS-1875?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16050586#comment-16050586
]
Graham Wallis commented on ATLAS-1875:
--
Hi Christian,
The removal of the Vertex id from the result was accidental. So we should add
it back in, presumably we should add this to 0.8 and 0.9.
> Gremlin id is no longer returned for vertices in gremlin query
> --
>
> Key: ATLAS-1875
> URL: https://issues.apache.org/jira/browse/ATLAS-1875
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Affects Versions: trunk, 0.8-incubating
>Reporter: Christian R
> Labels: dsl, gremlin
>
> Hi,
> while investigating a move from atlas 0.7 to 0.8 (HDP 2.5 to HDP2.6) our
> tests fail on gremlin queries. It turns out that the returned entities in a
> gremlin search in 0.8 does not include the 'id' attribute.
> I've built commit a0bd93945cd45457bbf34a8cb819d4fa4ba72964 (0.8-rc1) on linux
> using berkely and elasticearch to test with.
> The query
> :21000/api/atlas/discovery/search/gremlin?g.V.has('__type.name',
> 'Infrastructure').collect()
> on our 0.7 cluster gives
> {code:json}
> {
> __type.name: "Infrastructure",
> __type.category: "CLASS",
> __type: "typeSystem",
> id: "16640"
> }{code}
> while the same query on my 0.8-rc1 installation gives
> {code:javascript}
> {
> __type.name: "Infrastructure",
> __version: "1",
> __type.category: "CLASS",
> __type.version: "1.0",
> __modificationTimestamp: "1497448424134",
> __type: "typeSystem",
> __type.options: "null",
> __type.description: "Infrastructure",
> __guid: "77d07283-7622-4305-9c0c-09ac5aee86c8",
> __timestamp: "1497448424134"
> }
> {code}
> Certainly more information, but id is missing.
> The very poor DSL performance (see ATLAS-1868) and a need for advanced
> queries led us to base our queries on gremlin. This has worked very well so
> far. We include both edges and nodes in the result set and use the inVertex,
> outVertex and label info on the edges to rebuild our tree on the client side.
> I also see that gremlin has disappeared from version two of the API. Since
> addE and addV lets you insert into the graph I can see how exposing a full
> gremlin endpoint might not be wanted.
> As an example of the queries we run that I haven't been able to express in
> the DSL is
> {noformat}
> query = g.V.has('__guid','').copySplit(
> _().out('track'),
> _().as('x')
> .out('functions', 'component')
> .loop('x'){true}{true}
> .copySplit(
> _(),
> _().in('part_of'),
> _().outE('functions',
> 'component'),
> _().inE('part_of')
> )
> .exhaustMerge.dedup
> )
> .exhaustMerge
> .collect()
> {noformat}
> this might not be a normal usecase.
> edit to add:
> I looked at GraphBackedDiscoveryService.java and notice that:
> 0.7:
> {code:java}
> else if (r instanceof TitanVertex) {
> Iterable ps = ((TitanVertex)
> r).getProperties();
> for (TitanProperty tP : ps) {
> String pName = tP.getPropertyKey().getName();
> Object pValue = ((TitanVertex) r).getProperty(pName);
> if (pValue != null) {
> oRow.put(pName, pValue.toString());
> }
> }
> {code}
> Vs 0.8 code:
> {code:java}
> else if (value instanceof AtlasVertex) {
> AtlasVertex vertex = (AtlasVertex)value;
> for (String key : vertex.getPropertyKeys()) {
> Object propertyValue =
> GraphHelper.getProperty(vertex, key);
> if (propertyValue != null) {
> oRow.put(key, propertyValue.toString());
> }
>
[jira] [Updated] (ATLAS-1863) Set default value for primitive types attributes in entity based on attributeDef in Typedef
[
https://issues.apache.org/jira/browse/ATLAS-1863?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nixon Rodrigues updated ATLAS-1863:
---
Attachment: ATLAS-1863.3.patch
> Set default value for primitive types attributes in entity based on
> attributeDef in Typedef
> ---
>
> Key: ATLAS-1863
> URL: https://issues.apache.org/jira/browse/ATLAS-1863
> Project: Atlas
> Issue Type: Improvement
> Components: atlas-intg
>Reporter: Nixon Rodrigues
>Assignee: Ruchi Solani
>Priority: Critical
> Fix For: 0.9-incubating
>
> Attachments: ATLAS-1863.2.patch, ATLAS-1863.3.patch, ATLAS-1863.patch
>
>
> While creating entity if attribute value are not set explicitly for primitive
> type which are optional, then default value should be set from attributedef.
> eg of typedef attributeDef
> {code}
> "attributeDefs": [{
> "name": "sourceCode",
> "typeName": "string",
> "isOptional": true,
> "cardinality": "SINGLE",
> "valuesMinCount": 0,
> "valuesMaxCount": 1,
> "isUnique": false,
> "isIndexable": true,
> "defaultValue": "xyz"
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Assigned] (ATLAS-1876) Import/Export : AtlasSchemaViolationException during Import of entity associated to a tag with tag attribute value of datatype float
[
https://issues.apache.org/jira/browse/ATLAS-1876?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ashutosh Mestry reassigned ATLAS-1876:
--
Assignee: Ashutosh Mestry
> Import/Export : AtlasSchemaViolationException during Import of entity
> associated to a tag with tag attribute value of datatype float
>
>
> Key: ATLAS-1876
> URL: https://issues.apache.org/jira/browse/ATLAS-1876
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Affects Versions: 0.9-incubating
>Reporter: Sharmadha Sainath
>Assignee: Ashutosh Mestry
> Attachments: AtlasSchemaViolationException_ExportImport.txt
>
>
> 1.In cluster1, created a tag float_tag with attribute float_max of datatype
> float.
> 2. In the same cluster , created table table1 and associated the tag
> float_tag to table1 , with attribute value for float_max as 3.4028235e+38
> (maximum value for float).
> 3.Creation and tag association are successful.
> 4.Created table1.zip by using Export API
> 5. Using Import API , tried to import table1.zip to cluster2. Import failed
> with
> {code}
> {"errorCode":"ATLAS-500-00-001","errorMessage":"org.apache.atlas.exception.AtlasBaseException:
> org.apache.atlas.repository.graphdb.AtlasSchemaViolationException:
> com.thinkaurelius.titan.core.SchemaViolationException: Value [3.4028235E38]
> is not an instance of the expected data type for property key
> [float_tag.float_max] and cannot be converted. Expected: class
> java.lang.Float, found: class java.lang.Double"}
> {code}
> Attached the error stack trace found in cluster2 application logs.
> This exception is seen only when the value for float attribute is given as
> 3.4028235e+38 . Other values like 0 , 2.5 , 1.4e+10 etc., are accepted and
> import is done successfully.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (ATLAS-1875) Gremlin id is no longer returned for vertices in gremlin query
[
https://issues.apache.org/jira/browse/ATLAS-1875?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16050538#comment-16050538
]
Graham Wallis commented on ATLAS-1875:
--
Hi Christian
The id appears to have been removed when the move was made Titan classes to
Atlas classes (i.e. the abstraction layer).
I've contacted the committer to check whether deliberate or accidental. Will
post answer here.
> Gremlin id is no longer returned for vertices in gremlin query
> --
>
> Key: ATLAS-1875
> URL: https://issues.apache.org/jira/browse/ATLAS-1875
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Affects Versions: trunk, 0.8-incubating
>Reporter: Christian R
> Labels: dsl, gremlin
>
> Hi,
> while investigating a move from atlas 0.7 to 0.8 (HDP 2.5 to HDP2.6) our
> tests fail on gremlin queries. It turns out that the returned entities in a
> gremlin search in 0.8 does not include the 'id' attribute.
> I've built commit a0bd93945cd45457bbf34a8cb819d4fa4ba72964 (0.8-rc1) on linux
> using berkely and elasticearch to test with.
> The query
> :21000/api/atlas/discovery/search/gremlin?g.V.has('__type.name',
> 'Infrastructure').collect()
> on our 0.7 cluster gives
> {code:json}
> {
> __type.name: "Infrastructure",
> __type.category: "CLASS",
> __type: "typeSystem",
> id: "16640"
> }{code}
> while the same query on my 0.8-rc1 installation gives
> {code:javascript}
> {
> __type.name: "Infrastructure",
> __version: "1",
> __type.category: "CLASS",
> __type.version: "1.0",
> __modificationTimestamp: "1497448424134",
> __type: "typeSystem",
> __type.options: "null",
> __type.description: "Infrastructure",
> __guid: "77d07283-7622-4305-9c0c-09ac5aee86c8",
> __timestamp: "1497448424134"
> }
> {code}
> Certainly more information, but id is missing.
> The very poor DSL performance (see ATLAS-1868) and a need for advanced
> queries led us to base our queries on gremlin. This has worked very well so
> far. We include both edges and nodes in the result set and use the inVertex,
> outVertex and label info on the edges to rebuild our tree on the client side.
> I also see that gremlin has disappeared from version two of the API. Since
> addE and addV lets you insert into the graph I can see how exposing a full
> gremlin endpoint might not be wanted.
> As an example of the queries we run that I haven't been able to express in
> the DSL is
> {noformat}
> query = g.V.has('__guid','').copySplit(
> _().out('track'),
> _().as('x')
> .out('functions', 'component')
> .loop('x'){true}{true}
> .copySplit(
> _(),
> _().in('part_of'),
> _().outE('functions',
> 'component'),
> _().inE('part_of')
> )
> .exhaustMerge.dedup
> )
> .exhaustMerge
> .collect()
> {noformat}
> this might not be a normal usecase.
> edit to add:
> I looked at GraphBackedDiscoveryService.java and notice that:
> 0.7:
> {code:java}
> else if (r instanceof TitanVertex) {
> Iterable ps = ((TitanVertex)
> r).getProperties();
> for (TitanProperty tP : ps) {
> String pName = tP.getPropertyKey().getName();
> Object pValue = ((TitanVertex) r).getProperty(pName);
> if (pValue != null) {
> oRow.put(pName, pValue.toString());
> }
> }
> {code}
> Vs 0.8 code:
> {code:java}
> else if (value instanceof AtlasVertex) {
> AtlasVertex vertex = (AtlasVertex)value;
> for (String key : vertex.getPropertyKeys()) {
> Object propertyValue =
> GraphHelper.getProperty(vertex, key);
> if (propertyValue != null) {
>
[jira] [Assigned] (ATLAS-1877) UI: In Lineage for Process entity process icon is not showing.
[ https://issues.apache.org/jira/browse/ATLAS-1877?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Keval Bhatt reassigned ATLAS-1877: -- Assignee: (was: Keval Bhatt) > UI: In Lineage for Process entity process icon is not showing. > -- > > Key: ATLAS-1877 > URL: https://issues.apache.org/jira/browse/ATLAS-1877 > Project: Atlas > Issue Type: Bug >Reporter: Keval Bhatt > Attachments: wrong_process_icon.png > > > UI should check Process entity in all parent superType, right know its > checking for only 1 level > Step to reproduce : > # Run quickstart > # Create entity of type Process and while creating process select time_dim > (Table entity) as a input and output. > # Now got to sales_fact_monthly_mv entity and you will see new output (Please > find attached screenshot ) -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (ATLAS-1877) UI: In Lineage for Process entity process icon is not showing.
[ https://issues.apache.org/jira/browse/ATLAS-1877?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Keval Bhatt updated ATLAS-1877: --- Attachment: wrong_process_icon.png > UI: In Lineage for Process entity process icon is not showing. > -- > > Key: ATLAS-1877 > URL: https://issues.apache.org/jira/browse/ATLAS-1877 > Project: Atlas > Issue Type: Bug >Reporter: Keval Bhatt >Assignee: Keval Bhatt > Attachments: wrong_process_icon.png > > > UI should check Process entity in all parent superType, right know its > checking for only 1 level > Step to reproduce : > # Run quickstart > # Create entity of type Process and while creating process select time_dim > (Table entity) as a input and output. > # Now got to sales_fact_monthly_mv entity and you will see new output (Please > find attached screenshot ) -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (ATLAS-1877) UI: In Lineage for Process entity process icon is not showing.
[ https://issues.apache.org/jira/browse/ATLAS-1877?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Keval Bhatt updated ATLAS-1877: --- Description: UI should check Process entity in all parent superType, right know its checking for only 1 level Step to reproduce : # Run quickstart # Create entity of type Process and while creating process select time_dim (Table entity) as a input and output. # Now got to sales_fact_monthly_mv entity and you will see new output (Please find attached screenshot ) was: UI should check Process entity in all parent superType, right know its checking for only 1 level Step to reproduce : # Run quickstart # Create entity of type Process and while creating process select time_dim (Table entity) as a input and output. # Now got to time_dim entity and you will see new output (Please find attached screenshot ) > UI: In Lineage for Process entity process icon is not showing. > -- > > Key: ATLAS-1877 > URL: https://issues.apache.org/jira/browse/ATLAS-1877 > Project: Atlas > Issue Type: Bug >Reporter: Keval Bhatt >Assignee: Keval Bhatt > > UI should check Process entity in all parent superType, right know its > checking for only 1 level > Step to reproduce : > # Run quickstart > # Create entity of type Process and while creating process select time_dim > (Table entity) as a input and output. > # Now got to sales_fact_monthly_mv entity and you will see new output (Please > find attached screenshot ) -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (ATLAS-1877) UI: In Lineage for Process entity process icon is not showing.
[ https://issues.apache.org/jira/browse/ATLAS-1877?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Keval Bhatt updated ATLAS-1877: --- Description: UI should check Process entity in all parent superType, right know its checking for only 1 level Step to reproduce : # Run quickstart # Create entity of type Process and while creating process select time_dim (Table entity) as a input and output. # Now got to time_dim entity and you will see new output (Please find attached screenshot ) was:UI should check Process entity in all parent superType, right know its checking for only 1 level > UI: In Lineage for Process entity process icon is not showing. > -- > > Key: ATLAS-1877 > URL: https://issues.apache.org/jira/browse/ATLAS-1877 > Project: Atlas > Issue Type: Bug >Reporter: Keval Bhatt >Assignee: Keval Bhatt > > UI should check Process entity in all parent superType, right know its > checking for only 1 level > Step to reproduce : > # Run quickstart > # Create entity of type Process and while creating process select time_dim > (Table entity) as a input and output. > # Now got to time_dim entity and you will see new output (Please find > attached screenshot ) -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Comment Edited] (ATLAS-1781) Atlas UI cannot show lineage pic when inputs and outputs of lineage are the same entity
[
https://issues.apache.org/jira/browse/ATLAS-1781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16050434#comment-16050434
]
Keval Bhatt edited comment on ATLAS-1781 at 6/15/17 12:57 PM:
--
[~xiaqinglin] for Icon issue I created another jira ATLAS-1877
was (Author: kevalbhatt18):
[~xiaqinglin] For Icon issue I created another jira ATLAS-1877
> Atlas UI cannot show lineage pic when inputs and outputs of lineage are the
> same entity
> ---
>
> Key: ATLAS-1781
> URL: https://issues.apache.org/jira/browse/ATLAS-1781
> Project: Atlas
> Issue Type: Bug
> Components: atlas-webui
>Affects Versions: trunk, 0.7-incubating, 0.8-incubating, 0.7.1-incubating,
> 0.9-incubating
>Reporter: qinglin,xia
>Assignee: Xinzhi,Luo
> Labels: atlas
> Fix For: 0.9-incubating
>
> Attachments: 0001-fix-ATLAS-1781.patch,
> 0001-fix-not-working-with-more-than-2-nodes-with-ATLAS-17.patch,
> after_test-wrong_op.png, browser_error.png, expected.png, fixed.png, Linage
> fix not working with more than 2 nodes with ATLAS-1781.png,
> Lineage_Create_Successfully_when_inputs_outputs_different.png,
> Lineage_keep_loading.png, Process_Type_Not_Blue.png,
> Starting_point_is_Impact_than_icon&arrow_colour_is_wrong(KEVAL_2_review).jpg
>
>
> I was working with the lineage for hbase, when I added a column family in a
> hbase table, I want to create a lineage for the table to show there is a
> change within the table schema, yet when I create a process using the atlas
> api, I found that the process entity is successfully created, but the lineage
> is not shown on the atlas web page.
> Steps to Reproduce this issue is:
> 1. build up an entity extends the DataSet Type
> 2. build up a lineage Process
> 3. set the process input and output to the same entity
> The code is like:
> Referenceable table1 =
> atlasClient.getEntity(HBaseDataTypes.HBASE_TABLE_TYPE_NAME,
> AtlasClient.REFERENCEABLE_ATTRIBUTE_NAME, input_table_name);
> Referenceable table2 =
> atlasClient.getEntity(HBaseDataTypes.HBASE_TABLE_TYPE_NAME,
> AtlasClient.REFERENCEABLE_ATTRIBUTE_NAME, output_table_name);
> HBaseProcess process = new HBaseProcess("process");
> process.setInput(table1.getId());
> process.setoutput(table2.getId());
> here I set the input_table_name and output_table_name the same string name,
> when I set them different values, the lineage pic can be shown for table1 and
> table2 successfully
> Screenshots are:
> 1) The lineage pic keep loading on the atlas UI
> 2) The error log reported by the browser console
> 3) The lineage pic showed successfully when set the inputs and outputs of the
> lineage to different entities.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (ATLAS-1781) Atlas UI cannot show lineage pic when inputs and outputs of lineage are the same entity
[
https://issues.apache.org/jira/browse/ATLAS-1781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16050434#comment-16050434
]
Keval Bhatt commented on ATLAS-1781:
[~xiaqinglin] For Icon issue I created another jira ATLAS-1877
> Atlas UI cannot show lineage pic when inputs and outputs of lineage are the
> same entity
> ---
>
> Key: ATLAS-1781
> URL: https://issues.apache.org/jira/browse/ATLAS-1781
> Project: Atlas
> Issue Type: Bug
> Components: atlas-webui
>Affects Versions: trunk, 0.7-incubating, 0.8-incubating, 0.7.1-incubating,
> 0.9-incubating
>Reporter: qinglin,xia
>Assignee: Xinzhi,Luo
> Labels: atlas
> Fix For: 0.9-incubating
>
> Attachments: 0001-fix-ATLAS-1781.patch,
> 0001-fix-not-working-with-more-than-2-nodes-with-ATLAS-17.patch,
> after_test-wrong_op.png, browser_error.png, expected.png, fixed.png, Linage
> fix not working with more than 2 nodes with ATLAS-1781.png,
> Lineage_Create_Successfully_when_inputs_outputs_different.png,
> Lineage_keep_loading.png, Process_Type_Not_Blue.png,
> Starting_point_is_Impact_than_icon&arrow_colour_is_wrong(KEVAL_2_review).jpg
>
>
> I was working with the lineage for hbase, when I added a column family in a
> hbase table, I want to create a lineage for the table to show there is a
> change within the table schema, yet when I create a process using the atlas
> api, I found that the process entity is successfully created, but the lineage
> is not shown on the atlas web page.
> Steps to Reproduce this issue is:
> 1. build up an entity extends the DataSet Type
> 2. build up a lineage Process
> 3. set the process input and output to the same entity
> The code is like:
> Referenceable table1 =
> atlasClient.getEntity(HBaseDataTypes.HBASE_TABLE_TYPE_NAME,
> AtlasClient.REFERENCEABLE_ATTRIBUTE_NAME, input_table_name);
> Referenceable table2 =
> atlasClient.getEntity(HBaseDataTypes.HBASE_TABLE_TYPE_NAME,
> AtlasClient.REFERENCEABLE_ATTRIBUTE_NAME, output_table_name);
> HBaseProcess process = new HBaseProcess("process");
> process.setInput(table1.getId());
> process.setoutput(table2.getId());
> here I set the input_table_name and output_table_name the same string name,
> when I set them different values, the lineage pic can be shown for table1 and
> table2 successfully
> Screenshots are:
> 1) The lineage pic keep loading on the atlas UI
> 2) The error log reported by the browser console
> 3) The lineage pic showed successfully when set the inputs and outputs of the
> lineage to different entities.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (ATLAS-1877) UI: In Lineage for Process entity process icon is not showing.
Keval Bhatt created ATLAS-1877: -- Summary: UI: In Lineage for Process entity process icon is not showing. Key: ATLAS-1877 URL: https://issues.apache.org/jira/browse/ATLAS-1877 Project: Atlas Issue Type: Bug Reporter: Keval Bhatt Assignee: Keval Bhatt UI should check Process entity in all parent superType, right know its checking for only 1 level -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (ATLAS-1876) Import/Export : AtlasSchemaViolationException during Import of entity associated to a tag with tag attribute value of datatype float
[
https://issues.apache.org/jira/browse/ATLAS-1876?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sharmadha Sainath updated ATLAS-1876:
-
Description:
1.In cluster1, created a tag float_tag with attribute float_max of datatype
float.
2. In the same cluster , created table table1 and associated the tag float_tag
to table1 , with attribute value for float_max as 3.4028235e+38 (maximum value
for float).
3.Creation and tag association are successful.
4.Created table1.zip by using Export API
5. Using Import API , tried to import table1.zip to cluster2. Import failed
with
{code}
{"errorCode":"ATLAS-500-00-001","errorMessage":"org.apache.atlas.exception.AtlasBaseException:
org.apache.atlas.repository.graphdb.AtlasSchemaViolationException:
com.thinkaurelius.titan.core.SchemaViolationException: Value [3.4028235E38] is
not an instance of the expected data type for property key
[float_tag.float_max] and cannot be converted. Expected: class java.lang.Float,
found: class java.lang.Double"}
{code}
Attached the error stack trace found in cluster2 application logs.
This exception is seen only when the value for float attribute is given as
3.4028235e+38 . Other values like 0 , 2.5 , 1.4e+10 etc., are accepted and
import is done successfully.
was:
1.In cluster1, created a tag float_tag with attribute float_max of datatype
float.
2. In the same cluster , created table table1 and associated the tag float_tag
to table1 , with attribute value for float_max as 3.4028235e+38 (maximum value
for float).
3.Creation and tag association are successful.
4.Created table1.zip by using Export API
5. Using Import API , tried to import table1.zip to cluster2. Import failed
with
{code}
{"errorCode":"ATLAS-500-00-001","errorMessage":"org.apache.atlas.exception.AtlasBaseException:
org.apache.atlas.repository.graphdb.AtlasSchemaViolationException:
com.thinkaurelius.titan.core.SchemaViolationException: Value [3.4028235E38] is
not an instance of the expected data type for property key
[float_tag.float_max] and cannot be converted. Expected: class java.lang.Float,
found: class java.lang.Double"}
{code}
Attached the error stack trace found in cluster2 application logs.
> Import/Export : AtlasSchemaViolationException during Import of entity
> associated to a tag with tag attribute value of datatype float
> ----
>
> Key: ATLAS-1876
> URL: https://issues.apache.org/jira/browse/ATLAS-1876
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Affects Versions: 0.9-incubating
>Reporter: Sharmadha Sainath
> Attachments: AtlasSchemaViolationException_ExportImport.txt
>
>
> 1.In cluster1, created a tag float_tag with attribute float_max of datatype
> float.
> 2. In the same cluster , created table table1 and associated the tag
> float_tag to table1 , with attribute value for float_max as 3.4028235e+38
> (maximum value for float).
> 3.Creation and tag association are successful.
> 4.Created table1.zip by using Export API
> 5. Using Import API , tried to import table1.zip to cluster2. Import failed
> with
> {code}
> {"errorCode":"ATLAS-500-00-001","errorMessage":"org.apache.atlas.exception.AtlasBaseException:
> org.apache.atlas.repository.graphdb.AtlasSchemaViolationException:
> com.thinkaurelius.titan.core.SchemaViolationException: Value [3.4028235E38]
> is not an instance of the expected data type for property key
> [float_tag.float_max] and cannot be converted. Expected: class
> java.lang.Float, found: class java.lang.Double"}
> {code}
> Attached the error stack trace found in cluster2 application logs.
> This exception is seen only when the value for float attribute is given as
> 3.4028235e+38 . Other values like 0 , 2.5 , 1.4e+10 etc., are accepted and
> import is done successfully.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (ATLAS-1876) Import/Export : AtlasSchemaViolationException during Import of entity associated to a tag with tag attribute value of datatype float
Sharmadha Sainath created ATLAS-1876:
Summary: Import/Export : AtlasSchemaViolationException during
Import of entity associated to a tag with tag attribute value of datatype float
Key: ATLAS-1876
URL: https://issues.apache.org/jira/browse/ATLAS-1876
Project: Atlas
Issue Type: Bug
Components: atlas-core
Affects Versions: 0.9-incubating
Reporter: Sharmadha Sainath
Attachments: AtlasSchemaViolationException_ExportImport.txt
1.In cluster1, created a tag float_tag with attribute float_max of datatype
float.
2. In the same cluster , created table table1 and associated the tag float_tag
to table1 , with attribute value for float_max as 3.4028235e+38 (maximum value
for float).
3.Creation and tag association are successful.
4.Created table1.zip by using Export API
5. Using Import API , tried to import table1.zip to cluster2. Import failed
with
{code}
{"errorCode":"ATLAS-500-00-001","errorMessage":"org.apache.atlas.exception.AtlasBaseException:
org.apache.atlas.repository.graphdb.AtlasSchemaViolationException:
com.thinkaurelius.titan.core.SchemaViolationException: Value [3.4028235E38] is
not an instance of the expected data type for property key
[float_tag.float_max] and cannot be converted. Expected: class java.lang.Float,
found: class java.lang.Double"}
{code}
Attached the error stack trace found in cluster2 application logs.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Comment Edited] (ATLAS-1875) Gremlin id is no longer returned for vertices in gremlin query
[
https://issues.apache.org/jira/browse/ATLAS-1875?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16050355#comment-16050355
]
Christian R edited comment on ATLAS-1875 at 6/15/17 11:30 AM:
--
Hi Graham,
The big uncertainty for me is if the omission of id was deliberate or not from
the comitters' side.
You can checkout any revision by {{git checkout }} . You can get the
sha1's for the relevant commits from the [releases tab
|https://github.com/apache/incubator-atlas/releases] on github.
{{git checkout e48bd3558a81e0d4c104f315e623ed0f6200d169}} gives you 0.7-rc3.
{{git checkout a0bd93945cd45457bbf34a8cb819d4fa4ba72964}} gives you 0.8-rc1.
Then a "mvn clean package" will build it for you.
I have added
{code:java}
oRow.put("id", vertex.getId().toString());
{code}
to GraphBackedDiscoveryService line 227 on revision 0.8-rc1 and rebuilt with
tests executing. In my local build {{id}} is indeed returned. Most of our
integration tests against 0.7 then work against 0.8. (Some behaviour in POSTing
to entities/ changed, we had to move to PUT on entities)
was (Author: christianmr):
Hi Graham,
The big uncertainty for me is if the omission of id was deliberate or not from
the comitters' side.
You can checkout any revision by {{git checkout }} . You can get the
sha1's for the relevant commits from the [releases tab
|https://github.com/apache/incubator-atlas/releases] on github.
{{git checkout e48bd3558a81e0d4c104f315e623ed0f6200d169}} gives you 0.7-rc3.
{{git checkout a0bd93945cd45457bbf34a8cb819d4fa4ba72964}} gives you 0.8-rc1.
Then a "mvn clean package" will build it for you.
I have added
{code:java}
oRow.put("id", vertex.getId().toString());
{code}
to GraphBackedDiscoveryService line 227 on revision 0.8-rc1 and rebuilt with
tests executing. With my local build id is returned works fine. Most of our
integration tests against 0.7 then work against 0.8. (Some behaviour in POSTing
to entities/ changed, we had to move to PUT on entities)
> Gremlin id is no longer returned for vertices in gremlin query
> --
>
> Key: ATLAS-1875
> URL: https://issues.apache.org/jira/browse/ATLAS-1875
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Affects Versions: trunk, 0.8-incubating
>Reporter: Christian R
> Labels: dsl, gremlin
>
> Hi,
> while investigating a move from atlas 0.7 to 0.8 (HDP 2.5 to HDP2.6) our
> tests fail on gremlin queries. It turns out that the returned entities in a
> gremlin search in 0.8 does not include the 'id' attribute.
> I've built commit a0bd93945cd45457bbf34a8cb819d4fa4ba72964 (0.8-rc1) on linux
> using berkely and elasticearch to test with.
> The query
> :21000/api/atlas/discovery/search/gremlin?g.V.has('__type.name',
> 'Infrastructure').collect()
> on our 0.7 cluster gives
> {code:json}
> {
> __type.name: "Infrastructure",
> __type.category: "CLASS",
> __type: "typeSystem",
> id: "16640"
> }{code}
> while the same query on my 0.8-rc1 installation gives
> {code:javascript}
> {
> __type.name: "Infrastructure",
> __version: "1",
> __type.category: "CLASS",
> __type.version: "1.0",
> __modificationTimestamp: "1497448424134",
> __type: "typeSystem",
> __type.options: "null",
> __type.description: "Infrastructure",
> __guid: "77d07283-7622-4305-9c0c-09ac5aee86c8",
> __timestamp: "1497448424134"
> }
> {code}
> Certainly more information, but id is missing.
> The very poor DSL performance (see ATLAS-1868) and a need for advanced
> queries led us to base our queries on gremlin. This has worked very well so
> far. We include both edges and nodes in the result set and use the inVertex,
> outVertex and label info on the edges to rebuild our tree on the client side.
> I also see that gremlin has disappeared from version two of the API. Since
> addE and addV lets you insert into the graph I can see how exposing a full
> gremlin endpoint might not be wanted.
> As an example of the queries we run that I haven't been able to express in
> the DSL is
> {noformat}
> query = g.V.has('__guid','').copySplit(
> _().out('track'),
> _().as('x')
> .out('functions', 'component&
[jira] [Comment Edited] (ATLAS-1875) Gremlin id is no longer returned for vertices in gremlin query
[
https://issues.apache.org/jira/browse/ATLAS-1875?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16050355#comment-16050355
]
Christian R edited comment on ATLAS-1875 at 6/15/17 11:27 AM:
--
Hi Graham,
The big uncertainty for me is if the omission of id was deliberate or not from
the comitters' side.
You can checkout any revision by {{git checkout }} . You can get the
sha1's for the relevant commits from the [releases tab
|https://github.com/apache/incubator-atlas/releases] on github.
{{git checkout e48bd3558a81e0d4c104f315e623ed0f6200d169}} gives you 0.7-rc3.
{{git checkout a0bd93945cd45457bbf34a8cb819d4fa4ba72964}} gives you 0.8-rc1.
Then a "mvn clean package" will build it for you.
I have added
{code:java}
oRow.put("id", vertex.getId().toString());
{code}
to GraphBackedDiscoveryService line 227 on revision 0.8-rc1 and rebuilt with
tests executing. With my local build id is returned works fine. Most of our
integration tests against 0.7 then work against 0.8. (Some behaviour in POSTing
to entities/ changed, we had to move to PUT on entities)
was (Author: christianmr):
Hi Graham,
you can checkout any revision by {{git checkout }} . You can get the
sha1's for the relevant commits from the [releases tab
|https://github.com/apache/incubator-atlas/releases] on github.
{{git checkout e48bd3558a81e0d4c104f315e623ed0f6200d169}} gives you 0.7-rc3.
{{git checkout a0bd93945cd45457bbf34a8cb819d4fa4ba72964}} gives you 0.8-rc1.
Then a "mvn clean package" will build it for you.
I have added
{code:java}
oRow.put("id", vertex.getId().toString());
{code}
to GraphBackedDiscoveryService line 227 on revision 0.8-rc1 and rebuilt with
tests executing. With my local build id is returned works fine. Most of our
integration tests against 0.7 then work against 0.8. (Some behaviour in POSTing
to entities/ changed, we had to move to PUT on entities)
The big uncertainty for me is if the omission of id was deliberate or not from
the comitters' side.
> Gremlin id is no longer returned for vertices in gremlin query
> --
>
> Key: ATLAS-1875
> URL: https://issues.apache.org/jira/browse/ATLAS-1875
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Affects Versions: trunk, 0.8-incubating
>Reporter: Christian R
> Labels: dsl, gremlin
>
> Hi,
> while investigating a move from atlas 0.7 to 0.8 (HDP 2.5 to HDP2.6) our
> tests fail on gremlin queries. It turns out that the returned entities in a
> gremlin search in 0.8 does not include the 'id' attribute.
> I've built commit a0bd93945cd45457bbf34a8cb819d4fa4ba72964 (0.8-rc1) on linux
> using berkely and elasticearch to test with.
> The query
> :21000/api/atlas/discovery/search/gremlin?g.V.has('__type.name',
> 'Infrastructure').collect()
> on our 0.7 cluster gives
> {code:json}
> {
> __type.name: "Infrastructure",
> __type.category: "CLASS",
> __type: "typeSystem",
> id: "16640"
> }{code}
> while the same query on my 0.8-rc1 installation gives
> {code:javascript}
> {
> __type.name: "Infrastructure",
> __version: "1",
> __type.category: "CLASS",
> __type.version: "1.0",
> __modificationTimestamp: "1497448424134",
> __type: "typeSystem",
> __type.options: "null",
> __type.description: "Infrastructure",
> __guid: "77d07283-7622-4305-9c0c-09ac5aee86c8",
> __timestamp: "1497448424134"
> }
> {code}
> Certainly more information, but id is missing.
> The very poor DSL performance (see ATLAS-1868) and a need for advanced
> queries led us to base our queries on gremlin. This has worked very well so
> far. We include both edges and nodes in the result set and use the inVertex,
> outVertex and label info on the edges to rebuild our tree on the client side.
> I also see that gremlin has disappeared from version two of the API. Since
> addE and addV lets you insert into the graph I can see how exposing a full
> gremlin endpoint might not be wanted.
> As an example of the queries we run that I haven't been able to express in
> the DSL is
> {noformat}
> query = g.V.has('__guid','').copySplit(
> _().out('track'),
> _().as('x')
> .out('functions', 'component&
