[jira] [Created] (ATLAS-3818) Set 0.0.0.0 in atlas.server.bind.address property forces Atlas to run on localhost
Ricardo Martinelli de Oliveira created ATLAS-3818:
-
Summary: Set 0.0.0.0 in atlas.server.bind.address property forces
Atlas to run on localhost
Key: ATLAS-3818
URL: https://issues.apache.org/jira/browse/ATLAS-3818
Project: Atlas
Issue Type: Bug
Components: atlas-webui
Affects Versions: 2.0.0
Environment: kubernetes
Reporter: Ricardo Martinelli de Oliveira
I'm working on getting Atlas run on a k8s environment, and that requires the
application running in the pod to listen to 0.0.0.0 bind address as there is no
way to know the pod IP address in advance. However, setting
atlas.server.bind.address property to 0.0.0.0 will force Atlas to listen to
localhost.
The problem relies on this code in atlas_config.py[1]:
{code:java}
if (host == '0.0.0.0'):
host = DEFAULT_ATLAS_SERVER_HOST
print "starting atlas on host %s" % host
return host
{code}
So unless there's a good reason to return localhost when setting to 0.0.0.0
this code is wrong.
[1]
[https://github.com/apache/atlas/blob/master/distro/src/bin/atlas_config.py#L505-L506]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
Re: Review Request 72540: ATLAS-3806: Classifications information missing in notification events during entity create/update
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72540/#review220908
---
Ship it!
Ship It!
- Sarath Subramanian
On May 28, 2020, 4:01 p.m., Sidharth Mishra wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/72540/
> ---
>
> (Updated May 28, 2020, 4:01 p.m.)
>
>
> Review request for atlas, Ashutosh Mestry, Madhan Neethiraj, and Sarath
> Subramanian.
>
>
> Bugs: ATLAS-3806
> https://issues.apache.org/jira/browse/ATLAS-3806
>
>
> Repository: atlas
>
>
> Description
> ---
>
> Classifications information missing in notification events during entity
> create/update
>
>
> Diffs
> -
>
>
> intg/src/main/java/org/apache/atlas/model/instance/EntityMutationResponse.java
> d83f0e0f1
>
> repository/src/main/java/org/apache/atlas/repository/store/graph/v2/EntityGraphMapper.java
> 2ed524ff5
>
>
> Diff: https://reviews.apache.org/r/72540/diff/3/
>
>
> Testing
> ---
>
> For add new ctas notification looks like (Just shown for ctas entity):
> {
> "version": {
> "version": "1.0.0",
> "versionParts": [
> 1
> ]
> },
> "msgCompressionKind": "NONE",
> "msgSplitIdx": 1,
> "msgSplitCount": 1,
> "msgSourceIP": "172.27.168.128",
> "msgCreatedBy": "",
> "msgCreationTime": 1590176023734,
> "message": {
> "type": "ENTITY_NOTIFICATION_V2",
> "entity": {
> "typeName": "hive_table",
> "attributes": {
> "owner": "anonymous",
> "createTime": 1590176022000,
> "qualifiedName": "sid_hive_db.sid_hive_table_ctas7@bulk",
> "name": "sid_hive_table_ctas7"
> },
> "guid": "9a0f46e1-99ad-4b1d-8818-ee60778b5619",
> "status": "ACTIVE",
> "displayText": "sid_hive_table_ctas7",
> "classificationNames": [
> "ETL",
> "PII"
> ],
> "classifications": [
> {
> "typeName": "ETL",
> "entityGuid": "410cb0f3-6df7-4523-ba23-8cd65752a811",
> "entityStatus": "ACTIVE",
> "propagate": true,
> "validityPeriods": [],
> "removePropagationsOnEntityDelete": false
> },
> {
> "typeName": "PII",
> "entityGuid": "410cb0f3-6df7-4523-ba23-8cd65752a811",
> "entityStatus": "ACTIVE",
> "propagate": true,
> "validityPeriods": [],
> "removePropagationsOnEntityDelete": false
> }
> ]
> },
> "operationType": "CLASSIFICATION_ADD",
> "eventTime": 1590176022456
> }
> }
>
>
> Add new tag:
> ---
> {
> "version": {
> "version": "1.0.0",
> "versionParts": [
> 1
> ]
> },
> "msgCompressionKind": "NONE",
> "msgSplitIdx": 1,
> "msgSplitCount": 1,
> "msgSourceIP": "172.27.168.128",
> "msgCreatedBy": "",
> "msgCreationTime": 1590181680675,
> "message": {
> "type": "ENTITY_NOTIFICATION_V2",
> "entity": {
> "typeName": "hive_table",
> "attributes": {
> "owner": "anonymous",
> "createTime": 159004867,
> "qualifiedName": "sid_hive_db.sid_hive_table_1@bulk",
> "name": "sid_hive_table_1"
> },
> "guid": "410cb0f3-6df7-4523-ba23-8cd65752a811",
> "status": "ACTIVE",
> "displayText": "sid_hive_table_1",
> "classificationNames": [
> "ETL",
> "PII",
> "Fact"
> ],
> "classifications": [
> {
> "typeName": "ETL",
> "entityGuid": "410cb0f3-6df7-4523-ba23-8cd65752a811",
> "entityStatus": "ACTIVE",
> "propagate": true,
> "validityPeriods": [],
> "removePropagationsOnEntityDelete": false
> },
> {
> "typeName": "PII",
> "entityGuid": "410cb0f3-6df7-4523-ba23-8cd65752a811",
> "entityStatus": "ACTIVE",
> "propagate": true,
> "validityPeriods": [],
> "removePropagationsOnEntityDelete": false
> },
> {
> "typeName": "Fact",
> "entityGuid":
Re: Review Request 72540: ATLAS-3806: Classifications information missing in notification events during entity create/update
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72540/
---
(Updated May 28, 2020, 11:01 p.m.)
Review request for atlas, Ashutosh Mestry, Madhan Neethiraj, and Sarath
Subramanian.
Bugs: ATLAS-3806
https://issues.apache.org/jira/browse/ATLAS-3806
Repository: atlas
Description
---
Classifications information missing in notification events during entity
create/update
Diffs (updated)
-
intg/src/main/java/org/apache/atlas/model/instance/EntityMutationResponse.java
d83f0e0f1
repository/src/main/java/org/apache/atlas/repository/store/graph/v2/EntityGraphMapper.java
2ed524ff5
Diff: https://reviews.apache.org/r/72540/diff/3/
Changes: https://reviews.apache.org/r/72540/diff/2-3/
Testing
---
For add new ctas notification looks like (Just shown for ctas entity):
{
"version": {
"version": "1.0.0",
"versionParts": [
1
]
},
"msgCompressionKind": "NONE",
"msgSplitIdx": 1,
"msgSplitCount": 1,
"msgSourceIP": "172.27.168.128",
"msgCreatedBy": "",
"msgCreationTime": 1590176023734,
"message": {
"type": "ENTITY_NOTIFICATION_V2",
"entity": {
"typeName": "hive_table",
"attributes": {
"owner": "anonymous",
"createTime": 1590176022000,
"qualifiedName": "sid_hive_db.sid_hive_table_ctas7@bulk",
"name": "sid_hive_table_ctas7"
},
"guid": "9a0f46e1-99ad-4b1d-8818-ee60778b5619",
"status": "ACTIVE",
"displayText": "sid_hive_table_ctas7",
"classificationNames": [
"ETL",
"PII"
],
"classifications": [
{
"typeName": "ETL",
"entityGuid": "410cb0f3-6df7-4523-ba23-8cd65752a811",
"entityStatus": "ACTIVE",
"propagate": true,
"validityPeriods": [],
"removePropagationsOnEntityDelete": false
},
{
"typeName": "PII",
"entityGuid": "410cb0f3-6df7-4523-ba23-8cd65752a811",
"entityStatus": "ACTIVE",
"propagate": true,
"validityPeriods": [],
"removePropagationsOnEntityDelete": false
}
]
},
"operationType": "CLASSIFICATION_ADD",
"eventTime": 1590176022456
}
}
Add new tag:
---
{
"version": {
"version": "1.0.0",
"versionParts": [
1
]
},
"msgCompressionKind": "NONE",
"msgSplitIdx": 1,
"msgSplitCount": 1,
"msgSourceIP": "172.27.168.128",
"msgCreatedBy": "",
"msgCreationTime": 1590181680675,
"message": {
"type": "ENTITY_NOTIFICATION_V2",
"entity": {
"typeName": "hive_table",
"attributes": {
"owner": "anonymous",
"createTime": 159004867,
"qualifiedName": "sid_hive_db.sid_hive_table_1@bulk",
"name": "sid_hive_table_1"
},
"guid": "410cb0f3-6df7-4523-ba23-8cd65752a811",
"status": "ACTIVE",
"displayText": "sid_hive_table_1",
"classificationNames": [
"ETL",
"PII",
"Fact"
],
"classifications": [
{
"typeName": "ETL",
"entityGuid": "410cb0f3-6df7-4523-ba23-8cd65752a811",
"entityStatus": "ACTIVE",
"propagate": true,
"validityPeriods": [],
"removePropagationsOnEntityDelete": false
},
{
"typeName": "PII",
"entityGuid": "410cb0f3-6df7-4523-ba23-8cd65752a811",
"entityStatus": "ACTIVE",
"propagate": true,
"validityPeriods": [],
"removePropagationsOnEntityDelete": false
},
{
"typeName": "Fact",
"entityGuid": "410cb0f3-6df7-4523-ba23-8cd65752a811",
"entityStatus": "ACTIVE",
"propagate": true,
"validityPeriods": [],
"removePropagationsOnEntityDelete": false
}
]
},
"operationType": "CLASSIFICATION_ADD",
"eventTime": 1590181673717
}
}
{
"version": {
"version": "1.0.0",
"versionParts": [
1
]
},
"msgCompressionKind": "NONE",
"msgSplitIdx": 1,
"msgSplitCount": 1,
"msgSourceIP":
Re: Review Request 72540: ATLAS-3806: Classifications information missing in notification events during entity create/update
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72540/
---
(Updated May 28, 2020, 10:58 p.m.)
Review request for atlas, Ashutosh Mestry, Madhan Neethiraj, and Sarath
Subramanian.
Bugs: ATLAS-3806
https://issues.apache.org/jira/browse/ATLAS-3806
Repository: atlas
Description
---
Classifications information missing in notification events during entity
create/update
Diffs (updated)
-
intg/src/main/java/org/apache/atlas/model/instance/EntityMutationResponse.java
d83f0e0f1
repository/src/main/java/org/apache/atlas/repository/store/graph/v2/EntityGraphMapper.java
2ed524ff5
webapp/src/main/java/org/apache/atlas/notification/EntityNotificationListenerV2.java
a677b315c
Diff: https://reviews.apache.org/r/72540/diff/2/
Changes: https://reviews.apache.org/r/72540/diff/1-2/
Testing
---
For add new ctas notification looks like (Just shown for ctas entity):
{
"version": {
"version": "1.0.0",
"versionParts": [
1
]
},
"msgCompressionKind": "NONE",
"msgSplitIdx": 1,
"msgSplitCount": 1,
"msgSourceIP": "172.27.168.128",
"msgCreatedBy": "",
"msgCreationTime": 1590176023734,
"message": {
"type": "ENTITY_NOTIFICATION_V2",
"entity": {
"typeName": "hive_table",
"attributes": {
"owner": "anonymous",
"createTime": 1590176022000,
"qualifiedName": "sid_hive_db.sid_hive_table_ctas7@bulk",
"name": "sid_hive_table_ctas7"
},
"guid": "9a0f46e1-99ad-4b1d-8818-ee60778b5619",
"status": "ACTIVE",
"displayText": "sid_hive_table_ctas7",
"classificationNames": [
"ETL",
"PII"
],
"classifications": [
{
"typeName": "ETL",
"entityGuid": "410cb0f3-6df7-4523-ba23-8cd65752a811",
"entityStatus": "ACTIVE",
"propagate": true,
"validityPeriods": [],
"removePropagationsOnEntityDelete": false
},
{
"typeName": "PII",
"entityGuid": "410cb0f3-6df7-4523-ba23-8cd65752a811",
"entityStatus": "ACTIVE",
"propagate": true,
"validityPeriods": [],
"removePropagationsOnEntityDelete": false
}
]
},
"operationType": "CLASSIFICATION_ADD",
"eventTime": 1590176022456
}
}
Add new tag:
---
{
"version": {
"version": "1.0.0",
"versionParts": [
1
]
},
"msgCompressionKind": "NONE",
"msgSplitIdx": 1,
"msgSplitCount": 1,
"msgSourceIP": "172.27.168.128",
"msgCreatedBy": "",
"msgCreationTime": 1590181680675,
"message": {
"type": "ENTITY_NOTIFICATION_V2",
"entity": {
"typeName": "hive_table",
"attributes": {
"owner": "anonymous",
"createTime": 159004867,
"qualifiedName": "sid_hive_db.sid_hive_table_1@bulk",
"name": "sid_hive_table_1"
},
"guid": "410cb0f3-6df7-4523-ba23-8cd65752a811",
"status": "ACTIVE",
"displayText": "sid_hive_table_1",
"classificationNames": [
"ETL",
"PII",
"Fact"
],
"classifications": [
{
"typeName": "ETL",
"entityGuid": "410cb0f3-6df7-4523-ba23-8cd65752a811",
"entityStatus": "ACTIVE",
"propagate": true,
"validityPeriods": [],
"removePropagationsOnEntityDelete": false
},
{
"typeName": "PII",
"entityGuid": "410cb0f3-6df7-4523-ba23-8cd65752a811",
"entityStatus": "ACTIVE",
"propagate": true,
"validityPeriods": [],
"removePropagationsOnEntityDelete": false
},
{
"typeName": "Fact",
"entityGuid": "410cb0f3-6df7-4523-ba23-8cd65752a811",
"entityStatus": "ACTIVE",
"propagate": true,
"validityPeriods": [],
"removePropagationsOnEntityDelete": false
}
]
},
"operationType": "CLASSIFICATION_ADD",
"eventTime": 1590181673717
}
}
{
"version": {
"version": "1.0.0",
"versionParts": [
1
]
},
Re: Review Request 72540: ATLAS-3806: Classifications information missing in notification events during entity create/update
> On May 26, 2020, 10:45 p.m., Madhan Neethiraj wrote:
> > Sid - I suggest to consider the following alternate approach to address
> > this issue:
> >
> > public class EntityGraphMapper{
> > private AtlasEntityHeader constructHeader(AtlasEntity entity, final
> > AtlasEntityType type, AtlasVertex vertex) {
> > /*
> > * replace current implementation with a call to
> > entityRetriever.toAtlasEntityHeaderWithClassifications()
> > *
> > AtlasEntityHeader header = new
> > AtlasEntityHeader(entity.getTypeName());
> >
> > header.setGuid(getIdFromVertex(vertex));
> > header.setStatus(entity.getStatus());
> > header.setIsIncomplete(entity.getIsIncomplete());
> >
> > for (AtlasAttribute attribute : type.getUniqAttributes().values()) {
> > header.setAttribute(attribute.getName(),
> > entity.getAttribute(attribute.getName()));
> > }
> >
> > return header;
> > *
> > */
> >
> > return
> > entityRetriever.toAtlasEntityHeaderWithClassifications(vertex);
> > }
> > }
> >
> > public class EntityMutationResponse {
> > JsonIgnore
> > public void addEntity(EntityOperation op, AtlasEntityHeader header) {
> > // if an entity is already included in CREATE, update the header,
> > to capture propagated classifications
> > if (op == EntityOperation.UPDATE || op ==
> > EntityOperation.PARTIAL_UPDATE) {
> > if (entityHeaderExists(getCreatedEntities(), header.getGuid()))
> > {
> > // return;
> > op = EntityOperaton.CREATE;
> > }
> > }
> >
> > ...
> > }
> > }
As you suggested I have changed the code. The issue is, we cache the entity at
repository/src/main/java/org/apache/atlas/repository/store/graph/v2/EntityGraphMapper.java:285
and 313 which is not getting updated with classification. At
org.apache.atlas.repository.store.graph.v2.AtlasEntityChangeNotifier#notifyPropagatedEntities
when we call fullTextMapperV2.getAndCacheEntity(guid) we still get the old one
without classification. So along with this change I added code to
constructHeader so that it will set classification to entity from entityHeader.
Now the updated solution works.
- Sidharth
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72540/#review220873
---
On May 28, 2020, 10:58 p.m., Sidharth Mishra wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/72540/
> ---
>
> (Updated May 28, 2020, 10:58 p.m.)
>
>
> Review request for atlas, Ashutosh Mestry, Madhan Neethiraj, and Sarath
> Subramanian.
>
>
> Bugs: ATLAS-3806
> https://issues.apache.org/jira/browse/ATLAS-3806
>
>
> Repository: atlas
>
>
> Description
> ---
>
> Classifications information missing in notification events during entity
> create/update
>
>
> Diffs
> -
>
>
> intg/src/main/java/org/apache/atlas/model/instance/EntityMutationResponse.java
> d83f0e0f1
>
> repository/src/main/java/org/apache/atlas/repository/store/graph/v2/EntityGraphMapper.java
> 2ed524ff5
>
> webapp/src/main/java/org/apache/atlas/notification/EntityNotificationListenerV2.java
> a677b315c
>
>
> Diff: https://reviews.apache.org/r/72540/diff/2/
>
>
> Testing
> ---
>
> For add new ctas notification looks like (Just shown for ctas entity):
> {
> "version": {
> "version": "1.0.0",
> "versionParts": [
> 1
> ]
> },
> "msgCompressionKind": "NONE",
> "msgSplitIdx": 1,
> "msgSplitCount": 1,
> "msgSourceIP": "172.27.168.128",
> "msgCreatedBy": "",
> "msgCreationTime": 1590176023734,
> "message": {
> "type": "ENTITY_NOTIFICATION_V2",
> "entity": {
> "typeName": "hive_table",
> "attributes": {
> "owner": "anonymous",
> "createTime": 1590176022000,
> "qualifiedName": "sid_hive_db.sid_hive_table_ctas7@bulk",
> "name": "sid_hive_table_ctas7"
> },
> "guid": "9a0f46e1-99ad-4b1d-8818-ee60778b5619",
> "status": "ACTIVE",
> "displayText": "sid_hive_table_ctas7",
> "classificationNames": [
> "ETL",
> "PII"
> ],
> "classifications": [
> {
> "typeName": "ETL",
> "entityGuid": "410cb0f3-6df7-4523-ba23-8cd65752a811",
> "entityStatus": "ACTIVE",
> "propagate": true,
> "validityPeriods": [],
>
[jira] [Commented] (ATLAS-3722) ZipFileMigrationImporter: Allow AtlasPatchService to Run Before Migration Starts
[ https://issues.apache.org/jira/browse/ATLAS-3722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17119022#comment-17119022 ] ASF subversion and git services commented on ATLAS-3722: Commit 00bef883c5561596ccc2b7b1075e8379bf8ac6bb in atlas's branch refs/heads/master from Ashutosh Mestry [ https://gitbox.apache.org/repos/asf?p=atlas.git;h=00bef88 ] ATLAS-3722: ZipFileMigrationImporter: Allow AtlasPatchService to Run Before Migration Starts > ZipFileMigrationImporter: Allow AtlasPatchService to Run Before Migration > Starts > > > Key: ATLAS-3722 > URL: https://issues.apache.org/jira/browse/ATLAS-3722 > Project: Atlas > Issue Type: Improvement > Components: atlas-core >Reporter: Ashutosh Mestry >Assignee: Ashutosh Mestry >Priority: Major > Attachments: > ATLAS-3722-Run-AtlasPatchService-before-ZipFileMigra.patch > > > *Background* > Existing implementation if _ZipFileMigrationImporter_ starts migration before > Java patches are applied. This is causes long patch application process to > happen after migration is done. This is not necessary since migration uses > AtlasEntity format for import. > *Solution* > Start migration after the service is run, that way it will not run after > migration completes. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (ATLAS-3803) Update tag propagation to NONE for relationshipDefs in aws_s3_v2 and azure_adls_gen2 models
[ https://issues.apache.org/jira/browse/ATLAS-3803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17118934#comment-17118934 ] ASF subversion and git services commented on ATLAS-3803: Commit 3dc447e80daf2b2e5ee26f676024b354f184705a in atlas's branch refs/heads/branch-2.0 from Sarath Subramanian [ https://gitbox.apache.org/repos/asf?p=atlas.git;h=3dc447e ] ATLAS-3803: Update tag propagation to NONE for relationshipDefs in aws_s3_v2 and azure_adls_gen2 models (cherry picked from commit e58c98ec65008280182472c4e63d7206393c99a9) > Update tag propagation to NONE for relationshipDefs in aws_s3_v2 and > azure_adls_gen2 models > --- > > Key: ATLAS-3803 > URL: https://issues.apache.org/jira/browse/ATLAS-3803 > Project: Atlas > Issue Type: Improvement > Components: atlas-core >Affects Versions: 2.0.0 >Reporter: Sarath Subramanian >Assignee: Sarath Subramanian >Priority: Major > Fix For: 2.1.0 > > Attachments: ATLAS-3803.001.patch, ATLAS-3803.002.patch > > > The tag propagation value for AWS S3 model between "aws_s3_v2_container" and > "aws_s3_v2_contained" is ONE_TO_TWO. This change is to change the default > propagation direction to NONE. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (ATLAS-3803) Update tag propagation to NONE for relationshipDefs in aws_s3_v2 and azure_adls_gen2 models
[ https://issues.apache.org/jira/browse/ATLAS-3803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17118933#comment-17118933 ] ASF subversion and git services commented on ATLAS-3803: Commit e58c98ec65008280182472c4e63d7206393c99a9 in atlas's branch refs/heads/master from Sarath Subramanian [ https://gitbox.apache.org/repos/asf?p=atlas.git;h=e58c98e ] ATLAS-3803: Update tag propagation to NONE for relationshipDefs in aws_s3_v2 and azure_adls_gen2 models > Update tag propagation to NONE for relationshipDefs in aws_s3_v2 and > azure_adls_gen2 models > --- > > Key: ATLAS-3803 > URL: https://issues.apache.org/jira/browse/ATLAS-3803 > Project: Atlas > Issue Type: Improvement > Components: atlas-core >Affects Versions: 2.0.0 >Reporter: Sarath Subramanian >Assignee: Sarath Subramanian >Priority: Major > Fix For: 2.1.0 > > Attachments: ATLAS-3803.001.patch, ATLAS-3803.002.patch > > > The tag propagation value for AWS S3 model between "aws_s3_v2_container" and > "aws_s3_v2_contained" is ONE_TO_TWO. This change is to change the default > propagation direction to NONE. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (ATLAS-3816) Backport: Basic search issue with IndexSerializer
[
https://issues.apache.org/jira/browse/ATLAS-3816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nikhil Bonte updated ATLAS-3816:
Description:
*Background:*
Atlas with property
_"atlas.use.index.query.to.find.entity.by.unique.attributes=true"_
Direct index query which has pattern *"v."* throws exception, e.g.
v."Referenceable.qualifiedName":"atscale_agg_in{color:#ff}v.{color}as_agg_2cfb287d_dal_fixed_inco@edldev"
This will throw
{code:java}
java.lang.IllegalArgumentException: Found reference to non-existant property
key in query at position...{code}
*Solution:*
Backporting ATLAS-2371.
It is resolved with janusgraph, for titan implementation backporting required.
was:
*Background:*
Direct index query which has pattern *"v."* throws exception, e.g.
v."Referenceable.qualifiedName":"atscale_agg_in{color:#ff}v.{color}as_agg_2cfb287d_dal_fixed_inco@edldev"
This will throw
{code:java}
java.lang.IllegalArgumentException: Found reference to non-existant property
key in query at position...{code}
*Solution:*
Backporting ATLAS-2371.
It is resolved with janusgraph, for titan implementation backporting required.
> Backport: Basic search issue with IndexSerializer
> -
>
> Key: ATLAS-3816
> URL: https://issues.apache.org/jira/browse/ATLAS-3816
> Project: Atlas
> Issue Type: Bug
>Affects Versions: 0.8.4
>Reporter: Nikhil Bonte
>Assignee: Nikhil Bonte
>Priority: Major
>
> *Background:*
> Atlas with property
> _"atlas.use.index.query.to.find.entity.by.unique.attributes=true"_
> Direct index query which has pattern *"v."* throws exception, e.g.
>
> v."Referenceable.qualifiedName":"atscale_agg_in{color:#ff}v.{color}as_agg_2cfb287d_dal_fixed_inco@edldev"
>
> This will throw
> {code:java}
> java.lang.IllegalArgumentException: Found reference to non-existant property
> key in query at position...{code}
>
> *Solution:*
> Backporting ATLAS-2371.
> It is resolved with janusgraph, for titan implementation backporting required.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (ATLAS-3817) Could not instantiate implementation: org.janusgraph.diskstorage.es.ElasticSearchIndex
Gaurav Sehgal created ATLAS-3817:
Summary: Could not instantiate implementation:
org.janusgraph.diskstorage.es.ElasticSearchIndex
Key: ATLAS-3817
URL: https://issues.apache.org/jira/browse/ATLAS-3817
Project: Atlas
Issue Type: Bug
Components: atlas-core
Affects Versions: 3.0.0
Reporter: Gaurav Sehgal
Getting this error while starting the atlas server with Elasticsearch.
{code:java}
java.lang.IllegalArgumentException: Could not instantiate implementation:
org.janusgraph.diskstorage.es.ElasticSearchIndexjava.lang.IllegalArgumentException:
Could not instantiate implementation:
org.janusgraph.diskstorage.es.ElasticSearchIndex at
org.janusgraph.util.system.ConfigurationUtil.instantiate(ConfigurationUtil.java:64)
at org.janusgraph.diskstorage.Backend.getImplementationClass(Backend.java:440)
at org.janusgraph.diskstorage.Backend.getIndexes(Backend.java:427) at
org.janusgraph.diskstorage.Backend.(Backend.java:150) at
org.janusgraph.graphdb.configuration.GraphDatabaseConfiguration.getBackend(GraphDatabaseConfiguration.java:1359)
at
org.janusgraph.graphdb.database.StandardJanusGraph.(StandardJanusGraph.java:146)
at org.janusgraph.core.JanusGraphFactory.open(JanusGraphFactory.java:161) at
org.janusgraph.core.JanusGraphFactory.open(JanusGraphFactory.java:132) at
org.janusgraph.core.JanusGraphFactory.open(JanusGraphFactory.java:112) at
org.apache.atlas.repository.graphdb.janus.AtlasJanusGraphDatabase.initJanusGraph(AtlasJanusGraphDatabase.java:182)
at
org.apache.atlas.repository.graphdb.janus.AtlasJanusGraphDatabase.getGraphInstance(AtlasJanusGraphDatabase.java:169)
at
org.apache.atlas.repository.graphdb.janus.AtlasJanusGraphDatabase.getGraph(AtlasJanusGraphDatabase.java:276)
at
org.apache.atlas.repository.graph.AtlasGraphProvider.getGraphInstance(AtlasGraphProvider.java:52)
at
org.apache.atlas.repository.graph.AtlasGraphProvider.retry(AtlasGraphProvider.java:114)
at
org.apache.atlas.repository.graph.AtlasGraphProvider.get(AtlasGraphProvider.java:102)
at
org.apache.atlas.repository.graph.AtlasGraphProvider$$EnhancerBySpringCGLIB$$35082b34.CGLIB$get$1()
at
org.apache.atlas.repository.graph.AtlasGraphProvider$$EnhancerBySpringCGLIB$$35082b34$$FastClassBySpringCGLIB$$12375d15.invoke()
at
org.springframework.cglib.proxy.MethodProxy.invokeSuper(MethodProxy.java:228)
at
org.springframework.context.annotation.ConfigurationClassEnhancer$BeanMethodInterceptor.intercept(ConfigurationClassEnhancer.java:358)
at
org.apache.atlas.repository.graph.AtlasGraphProvider$$EnhancerBySpringCGLIB$$35082b34.get()
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498) at
org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:162)
at
org.springframework.beans.factory.support.ConstructorResolver.instantiateUsingFactoryMethod(ConstructorResolver.java:588)
at
org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.instantiateUsingFactoryMethod(AbstractAutowireCapableBeanFactory.java:1178)
at
org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:1072)
at
org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:511)
at
org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:481)
at
org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:312)
at
org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:230)
at
org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:308)
at
org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:202)
at
org.springframework.beans.factory.config.DependencyDescriptor.resolveCandidate(DependencyDescriptor.java:208)
at
org.springframework.beans.factory.support.DefaultListableBeanFactory.doResolveDependency(DefaultListableBeanFactory.java:1136)
at
org.springframework.beans.factory.support.DefaultListableBeanFactory.resolveDependency(DefaultListableBeanFactory.java:1064)
at
org.springframework.beans.factory.support.ConstructorResolver.resolveAutowiredArgument(ConstructorResolver.java:835)
at
org.springframework.beans.factory.support.ConstructorResolver.createArgumentArray(ConstructorResolver.java:741)
at
[jira] [Updated] (ATLAS-3816) Backport: Basic search issue with IndexSerializer
[
https://issues.apache.org/jira/browse/ATLAS-3816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nikhil Bonte updated ATLAS-3816:
Description:
*Background:*
Direct index query which has pattern *"v."* throws exception, e.g.
v."Referenceable.qualifiedName":"atscale_agg_in{color:#ff}v.{color}as_agg_2cfb287d_dal_fixed_inco@edldev"
This will throw
{code:java}
java.lang.IllegalArgumentException: Found reference to non-existant property
key in query at position...{code}
*Solution:*
Backporting ATLAS-2371.
It is resolved with janusgraph, for titan implementation backporting required.
was:
*Background:*
Direct index query which has pattern *"v."* throws exception, e.g.
v."Referenceable.qualifiedName":"atscale_agg_in{color:#ff}v.{color}as_agg_2cfb287d_dal_fixed_inco@edldev"
This will throw
{code:java}
java.lang.IllegalArgumentException: Found reference to non-existant property
key in query at position...{code}
*Solution:*
Backporting direct index query issue resolved in ATLAS-2371.
> Backport: Basic search issue with IndexSerializer
> -
>
> Key: ATLAS-3816
> URL: https://issues.apache.org/jira/browse/ATLAS-3816
> Project: Atlas
> Issue Type: Bug
>Affects Versions: 0.8.4
>Reporter: Nikhil Bonte
>Assignee: Nikhil Bonte
>Priority: Major
>
> *Background:*
> Direct index query which has pattern *"v."* throws exception, e.g.
>
> v."Referenceable.qualifiedName":"atscale_agg_in{color:#ff}v.{color}as_agg_2cfb287d_dal_fixed_inco@edldev"
>
> This will throw
> {code:java}
> java.lang.IllegalArgumentException: Found reference to non-existant property
> key in query at position...{code}
>
> *Solution:*
> Backporting ATLAS-2371.
> It is resolved with janusgraph, for titan implementation backporting required.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (ATLAS-3816) Backport: Basic search issue with IndexSerializer
[
https://issues.apache.org/jira/browse/ATLAS-3816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nikhil Bonte updated ATLAS-3816:
Description:
*Background:*
Direct index query which has pattern *"v."* throws exception, e.g.
v."Referenceable.qualifiedName":"atscale_agg_in{color:#ff}v.{color}as_agg_2cfb287d_dal_fixed_inco@edldev"
This will throw
{code:java}
java.lang.IllegalArgumentException: Found reference to non-existant property
key in query at position...{code}
*Solution:*
Backporting direct index query issue resolved in ATLAS-2371.
was:
*Background:*
Direct index query which has pattern *"v."* throws exception, e.g.
v."Referenceable.qualifiedName":"atscale_agg_in*{color:#FF}v.{color}*as_agg_2cfb287d_dal_fixed_inco@edldev"
This will throw
{code:java}
java.lang.IllegalArgumentException: Found reference to non-existant property
key in query at position...{code}
*Solution:*
Backporting direct index query issue resolved in ATLAS-2371.
> Backport: Basic search issue with IndexSerializer
> -
>
> Key: ATLAS-3816
> URL: https://issues.apache.org/jira/browse/ATLAS-3816
> Project: Atlas
> Issue Type: Bug
>Affects Versions: 0.8.4
>Reporter: Nikhil Bonte
>Assignee: Nikhil Bonte
>Priority: Major
>
> *Background:*
> Direct index query which has pattern *"v."* throws exception, e.g.
>
> v."Referenceable.qualifiedName":"atscale_agg_in{color:#ff}v.{color}as_agg_2cfb287d_dal_fixed_inco@edldev"
>
> This will throw
> {code:java}
> java.lang.IllegalArgumentException: Found reference to non-existant property
> key in query at position...{code}
>
> *Solution:*
> Backporting direct index query issue resolved in ATLAS-2371.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (ATLAS-3816) Backport: Basic search issue with IndexSerializer
[
https://issues.apache.org/jira/browse/ATLAS-3816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nikhil Bonte updated ATLAS-3816:
Description:
*Background:*
Direct index query which has pattern *"v."* throws exception, e.g.
v."Referenceable.qualifiedName":"atscale_agg_in*{color:#FF}v.{color}*as_agg_2cfb287d_dal_fixed_inco@edldev"
This will throw
{code:java}
java.lang.IllegalArgumentException: Found reference to non-existant property
key in query at position...{code}
*Solution:*
Backporting direct index query issue resolved in ATLAS-2371.
was:
*Background:*
Direct index query which has pattern *"v."* throws exception, e.g.
v."Referenceable.qualifiedName":"atscale_agg_in*v.*as_agg_2cfb287d_dal_fixed_inco@edldev"
This will throw
{code:java}
java.lang.IllegalArgumentException: Found reference to non-existant property
key in query at position...{code}
*Solution:*
Backporting direct index query issue resolved in ATLAS-2371.
> Backport: Basic search issue with IndexSerializer
> -
>
> Key: ATLAS-3816
> URL: https://issues.apache.org/jira/browse/ATLAS-3816
> Project: Atlas
> Issue Type: Bug
>Affects Versions: 0.8.4
>Reporter: Nikhil Bonte
>Assignee: Nikhil Bonte
>Priority: Major
>
> *Background:*
> Direct index query which has pattern *"v."* throws exception, e.g.
>
> v."Referenceable.qualifiedName":"atscale_agg_in*{color:#FF}v.{color}*as_agg_2cfb287d_dal_fixed_inco@edldev"
>
> This will throw
> {code:java}
> java.lang.IllegalArgumentException: Found reference to non-existant property
> key in query at position...{code}
>
> *Solution:*
> Backporting direct index query issue resolved in ATLAS-2371.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (ATLAS-3816) Backport: Basic search issue with IndexSerializer
Nikhil Bonte created ATLAS-3816:
---
Summary: Backport: Basic search issue with IndexSerializer
Key: ATLAS-3816
URL: https://issues.apache.org/jira/browse/ATLAS-3816
Project: Atlas
Issue Type: Bug
Affects Versions: 0.8.4
Reporter: Nikhil Bonte
Assignee: Nikhil Bonte
*Background:*
Direct index query which has pattern *"v."* throws exception, e.g.
v."Referenceable.qualifiedName":"atscale_agg_in*v.*as_agg_2cfb287d_dal_fixed_inco@edldev"
This will throw
{code:java}
java.lang.IllegalArgumentException: Found reference to non-existant property
key in query at position...{code}
*Solution:*
Backporting direct index query issue resolved in ATLAS-2371.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (ATLAS-3812) Add schema for Apache Flink
[
https://issues.apache.org/jira/browse/ATLAS-3812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17118521#comment-17118521
]
Márton Balassi commented on ATLAS-3812:
---
Thanks [~madhan] for the detailed information. Will consider the issue you have
mentioned with the supertype.
I am planning to post a bridge/hook implementation soon, in fact already have a
prototype implementation - however it is dependent on pending changes in Apache
Flink, and I would like to add the schema itself sooner if possible
> Add schema for Apache Flink
> ---
>
> Key: ATLAS-3812
> URL: https://issues.apache.org/jira/browse/ATLAS-3812
> Project: Atlas
> Issue Type: New Feature
> Components: atlas-intg
>Reporter: Márton Balassi
>Priority: Major
>
> Hi Apache Atlas team,
> I am an Apache Flink PMC member pursuing to add a Flink bridge to Atlas. As a
> first step of this I would like to contribute the following schema, ideally
> to the upcoming 2.1 release.
> {code}
> {
> "enumDefs": [],
> "structDefs": [],
> "classificationDefs": [],
> "entityDefs": [
> {
> "name": "flink_application",
> "superTypes": [
> "Process"
> ],
> "serviceType": "flink",
> "typeVersion": "1.0",
> "attributeDefs": [
> {
> "name": "id",
> "typeName": "string",
> "cardinality": "SINGLE",
> "isIndexable": true,
> "isOptional": false,
> "isUnique": true
> },
> {
> "name": "startTime",
> "typeName": "date",
> "cardinality": "SINGLE",
> "isIndexable": false,
> "isOptional": true,
> "isUnique": false
> },
> {
> "name": "endTime",
> "typeName": "date",
> "cardinality": "SINGLE",
> "isIndexable": false,
> "isOptional": true,
> "isUnique": false
> },
> {
> "name": "conf",
> "typeName": "map",
> "cardinality": "SINGLE",
> "isIndexable": false,
> "isOptional": true,
> "isUnique": false
> },
> {
> "name": "inputs",
> "typeName": "array",
> "cardinality": "LIST",
> "isIndexable": false,
> "isOptional": false,
> "isUnique": false
> },
> {
> "name": "outputs",
> "typeName": "array",
> "cardinality": "LIST",
> "isIndexable": false,
> "isOptional": false,
> "isUnique": false
> }
> ]
> }
> ],
> "relationshipDefs": []
> }
> {code}
> Also please add an icon based on the white outline from the Flink material
> page similarly to the existing Kafka icon.
> https://flink.apache.org/material.html
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
Failed to execute bulk Elasticsearch mutation
Hello everyone, I'm new to Atlas and need some help in debugging this issue. https://issues.apache.org/jira/browse/ATLAS-3815 Thanks Gaurav Sehgal
