[jira] [Commented] (ATLAS-3545) NullPointerException while trying to delete classification
[ https://issues.apache.org/jira/browse/ATLAS-3545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16987104#comment-16987104 ] Ayub Pathan commented on ATLAS-3545: [~bolke] Can you please add the repro steps here? > NullPointerException while trying to delete classification > -- > > Key: ATLAS-3545 > URL: https://issues.apache.org/jira/browse/ATLAS-3545 > Project: Atlas > Issue Type: Bug >Affects Versions: trunk >Reporter: Bolke de Bruin >Priority: Critical > Attachments: Screenshot 2019-11-30 at 22.16.44.png, Screenshot > 2019-11-30 at 22.28.00.png, Screenshot 2019-12-02 at 21.43.29.png > > > We see an issue where there is a NullPointerException while trying to delete > a classification that is propagating. > It seems (stack trace is as of yet unavailable) that it is caused due to a > typeName being NULL in AtlasTypeRegistry.getType. The UI looks fishy too see > screenshots. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (ATLAS-2639) UI: Associating a classification to entity from term page results in adding tag to term rather than entity
Ayub Pathan created ATLAS-2639: -- Summary: UI: Associating a classification to entity from term page results in adding tag to term rather than entity Key: ATLAS-2639 URL: https://issues.apache.org/jira/browse/ATLAS-2639 Project: Atlas Issue Type: Bug Components: atlas-core Affects Versions: 1.0.0 Reporter: Ayub Pathan Fix For: 1.0.0 * Create a term(term1) and classification(tag1) * Create an entity(E1) and associate term1 to E1. * In the term page, entity association can be seen, associate tag1 to E1. * This results in associating tag1 to term instead of E1. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (ATLAS-2636) Business catalog: Creating a glossary with incorrect GUID, fails first time. When the same request with the same payload returns "409 conflict" saying "Glossary category
Ayub Pathan created ATLAS-2636:
--
Summary: Business catalog: Creating a glossary with incorrect
GUID, fails first time. When the same request with the same payload returns
"409 conflict" saying "Glossary category with qualifiedName category1 already
exists"
Key: ATLAS-2636
URL: https://issues.apache.org/jira/browse/ATLAS-2636
Project: Atlas
Issue Type: Bug
Components: atlas-core
Affects Versions: 1.0.0
Reporter: Ayub Pathan
Fix For: 1.0.0
* Create a category with below payload, this will fail with 500 internal error.
{noformat}
{
"qualifiedName": "category1",
"displayName": "Bank accounts",
"anchor": {
"glossaryGuid": "-1"
}
}
{noformat}
* Repeat the same request again, now atlas throws below error
{noformat}
{
"errorCode": "ATLAS-409-00-00A",
"errorMessage": "Glossary category with qualifiedName category1 already
exists"
}
{noformat}
Does this mean, the first request succeeded to register the category?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (ATLAS-2616) NPE for GET call of /glossary//categories, when there are no categories associated to glossary
Ayub Pathan created ATLAS-2616:
--
Summary: NPE for GET call of /glossary//categories, when
there are no categories associated to glossary
Key: ATLAS-2616
URL: https://issues.apache.org/jira/browse/ATLAS-2616
Project: Atlas
Issue Type: Improvement
Components: atlas-core
Affects Versions: 1.0.0
Reporter: Ayub Pathan
Fix For: 1.0.0
* Two glossaries configured with one glossary having categories
{noformat}
apathan@Ayub-sys-101:~ > curl -k -u admin:admin
https://ctr-e138-1518143905142-256977-01-04.hwx.site:21443/api/atlas/v2/glossary/
| python -m json.tool
% Total% Received % Xferd Average Speed TimeTime Time Current
Dload Upload Total SpentLeft Speed
100 6330 6330 0386 0 --:--:-- 0:00:01 --:--:-- 386
[
{
"guid": "bf43f063-f499-472c-b526-0f81de685e02",
"displayName": "true",
"shortDescription": "asd",
"longDescription": "Glossary of bank - long description",
"qualifiedName": "HortoniaBank-11"
},
{
"guid": "c445e6c3-1113-4d55-acbd-d421c8c98db1",
"displayName": "Banking-11",
"shortDescription": "sdfds",
"longDescription": "sdfdsf",
"qualifiedName": "HortoniaBank-11",
"categories": [
{
"categoryGuid": "59295311-2ef8-400c-9188-ea2adfdca65b",
"relationGuid": "39113069-211b-44e1-b924-b0402c997a20",
"displayText": "Bank accounts"
},
{
"categoryGuid": "625e13a6-e7ef-49d2-bfe3-bb31e8b63ef0",
"relationGuid": "4366623b-2f7e-43df-83f2-2d438f3fd120",
"displayText": "sub"
}
]
}
]
{noformat}
* GET request on categories is fine when there are categories associated.
{noformat}
apathan@Ayub-sys-101:~ >
apathan@Ayub-sys-101:~ > curl -k -u admin:admin
https://ctr-e138-1518143905142-256977-01-04.hwx.site:21443/api/atlas/v2/glossary/c445e6c3-1113-4d55-acbd-d421c8c98db1/categories
| python -m json.tool
% Total% Received % Xferd Average Speed TimeTime Time Current
Dload Upload Total SpentLeft Speed
100 2710 2710 0 97 0 --:--:-- 0:00:02 --:--:--97
[
{
"categoryGuid": "59295311-2ef8-400c-9188-ea2adfdca65b",
"relationGuid": "39113069-211b-44e1-b924-b0402c997a20",
"displayText": "Bank accounts"
},
{
"categoryGuid": "625e13a6-e7ef-49d2-bfe3-bb31e8b63ef0",
"relationGuid": "4366623b-2f7e-43df-83f2-2d438f3fd120",
"displayText": "sub"
}
]
{noformat}
* GET request on categories returns 500 with NPE when there are no categories
associated to glossary.
{noformat}
apathan@Ayub-sys-101:~ > curl -k -u admin:admin
https://ctr-e138-1518143905142-256977-01-04.hwx.site:21443/api/atlas/v2/glossary/bf43f063-f499-472c-b526-0f81de685e02/categories
| pj
% Total% Received % Xferd Average Speed TimeTime Time Current
Dload Upload Total SpentLeft Speed
100850850 0 33 0 --:--:-- 0:00:02 --:--:--33
Expecting value: line 1 column 1 (char 0)
apathan@Ayub-sys-101:~ >
{noformat}
{noformat}
2018-04-26 11:59:39,528 ERROR - [pool-2-thread-4 -
29e30189-d093-4245-a623-1c50c5aad766:] ~ Error handling a request:
935035fa3fbef294 (ExceptionMapperUtil:32)
java.lang.NullPointerException
at java.util.ArrayList.(ArrayList.java:178)
at
org.apache.atlas.glossary.GlossaryService.getGlossaryCategories(GlossaryService.java:618)
at
org.apache.atlas.web.rest.GlossaryREST.getGlossaryCategories(GlossaryREST.java:645)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60)
at
com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$TypeOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:185)
at
com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75)
at
com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:302)
at
com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
at
com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:108)
at
[jira] [Created] (ATLAS-2615) Bulk glossary creation support
Ayub Pathan created ATLAS-2615: -- Summary: Bulk glossary creation support Key: ATLAS-2615 URL: https://issues.apache.org/jira/browse/ATLAS-2615 Project: Atlas Issue Type: Improvement Components: atlas-core Affects Versions: 1.0.0 Reporter: Ayub Pathan Fix For: 1.0.0 With current implementation, there is no support to create multiple glossaries with single REST call. This requires multiple REST call, with each call creating one glossary. >From user's point, creating bulk glossaries in one shot could be helpful. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (ATLAS-2614) Uniqueness of qualified names is not honored while updating a glossary
Ayub Pathan created ATLAS-2614:
--
Summary: Uniqueness of qualified names is not honored while
updating a glossary
Key: ATLAS-2614
URL: https://issues.apache.org/jira/browse/ATLAS-2614
Project: Atlas
Issue Type: Bug
Components: atlas-core
Affects Versions: 1.0.0
Reporter: Ayub Pathan
Fix For: 1.0.0
Uniqueness of qualified names is not honored while updating a glossary
* creating 2 glossaries with unique names
{noformat}
apathan@Ayub-sys-101:~ > curl -k -H "Content-Type: application/json" -X POST -u
admin:admin
https://ctr-e138-1518143905142-256977-01-04.hwx.site:21443/api/atlas/v2/glossary/
-d '{ "qualifiedName" : "HortoniaBank-11", "displayName": "Banking-11" }' |
python -m json.tool
% Total% Received % Xferd Average Speed TimeTime Time Current
Dload Upload Total SpentLeft Speed
100 1760 108 10068 23 14 0:00:04 0:00:04 --:--:--43
{
"guid": "c445e6c3-1113-4d55-acbd-d421c8c98db1",
"displayName": "Banking-11",
"qualifiedName": "HortoniaBank-11"
}
apathan@Ayub-sys-101:~ > curl -k -H "Content-Type: application/json" -X POST -u
admin:admin
https://ctr-e138-1518143905142-256977-01-04.hwx.site:21443/api/atlas/v2/glossary/
-d '{ "qualifiedName" : "HortoniaBank-12", "displayName": "Banking-12" }' |
python -m json.tool
% Total% Received % Xferd Average Speed TimeTime Time Current
Dload Upload Total SpentLeft Speed
100 1760 108 10068 32 20 0:00:03 0:00:03 --:--:--52
{
"guid": "bf43f063-f499-472c-b526-0f81de685e02",
"displayName": "Banking-12",
"qualifiedName": "HortoniaBank-12"
}
apathan@Ayub-sys-101:~ > curl -k -u admin:admin
https://ctr-e138-1518143905142-256977-01-04.hwx.site:21443/api/atlas/v2/glossary/
| python -m json.tool
% Total% Received % Xferd Average Speed TimeTime Time Current
Dload Upload Total SpentLeft Speed
100 2190 2190 0 64 0 --:--:-- 0:00:03 --:--:--64
[
{
"guid": "bf43f063-f499-472c-b526-0f81de685e02",
"displayName": "Banking-12",
"qualifiedName": "HortoniaBank-12"
},
{
"guid": "c445e6c3-1113-4d55-acbd-d421c8c98db1",
"displayName": "Banking-11",
"qualifiedName": "HortoniaBank-11"
}
]
{noformat}
* Updating glossary-2 with glossary-1's qualified name, which is allowed.
{noformat}
curl -k -H "Content-Type: application/json" -X PUT -u admin:admin
https://ctr-e138-1518143905142-256977-01-04.hwx.site:21443/api/atlas/v2/glossary/bf43f063-f499-472c-b526-0f81de685e02
-d '{ "qualifiedName" : "HortoniaBank-11", "displayName": "Banking-11" }' |
python -m json.tool
% Total% Received % Xferd Average Speed TimeTime Time Current
Dload Upload Total SpentLeft Speed
100 1760 108 10068 38 23 0:00:02 0:00:02 --:--:--62
{
"guid": "bf43f063-f499-472c-b526-0f81de685e02",
"displayName": "Banking-11",
"qualifiedName": "HortoniaBank-11"
}
{noformat}
* Now, you can see both the entities have same qualifedName
{noformat}
apathan@Ayub-sys-101:~ > curl -k -u admin:admin
https://ctr-e138-1518143905142-256977-01-04.hwx.site:21443/api/atlas/v2/glossary/
| python -m json.tool
% Total% Received % Xferd Average Speed TimeTime Time Current
Dload Upload Total SpentLeft Speed
100 2190 2190 0141 0 --:--:-- 0:00:01 --:--:-- 141
[
{
"guid": "bf43f063-f499-472c-b526-0f81de685e02",
"displayName": "Banking-11",
"qualifiedName": "HortoniaBank-11"
},
{
"guid": "c445e6c3-1113-4d55-acbd-d421c8c98db1",
"displayName": "Banking-11",
"qualifiedName": "HortoniaBank-11"
}
]
apathan@Ayub-sys-101:~ >
{noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (ATLAS-2613) Submitting a glossary with guid=-1, atlas returns 404 but actually creates the glossary
Ayub Pathan created ATLAS-2613:
--
Summary: Submitting a glossary with guid=-1, atlas returns 404 but
actually creates the glossary
Key: ATLAS-2613
URL: https://issues.apache.org/jira/browse/ATLAS-2613
Project: Atlas
Issue Type: Bug
Components: atlas-core
Affects Versions: 1.0.0
Reporter: Ayub Pathan
Fix For: 1.0.0
Submitting below glossary returns 404 with error "Given instance guid -1 is
invalid/not found" but actually glossary is created.
{noformat}
https://ctr-e138-1518143905142-256977-01-04.hwx.site:21443/api/atlas/v2/glossary
{
"guid" : "-1",
"qualifiedName" : "HortoniaBank-8",
"displayName": "Banking",
"shortDescription": "Glossary of bank",
"longDescription": "Glossary of bank - long description",
"language": "English",
"usage": "N/A"
}
{noformat}
Atlas log also show graph rollback
{noformat}
2018-04-25 10:38:05,551 ERROR - [pool-2-thread-10 -
8ef9844f-deac-4df0-969c-e077f4664ed3:] ~ graph rollback due to exception
AtlasBaseException:Given instance guid -1 is invalid/not found
(GraphTransactionInterceptor:73)
{noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (ATLAS-2612) Atlas returns term definition with "/glossary/" API. Is this accurate? The correct API should be /glossary/term/
Ayub Pathan created ATLAS-2612:
--
Summary: Atlas returns term definition with
"/glossary/" API. Is this accurate? The correct API should be
/glossary/term/
Key: ATLAS-2612
URL: https://issues.apache.org/jira/browse/ATLAS-2612
Project: Atlas
Issue Type: Bug
Components: atlas-core
Affects Versions: 1.0.0
Reporter: Ayub Pathan
Fix For: 1.0.0
* Below are the list of glossary and terms
{noformat}
curl -k -u admin:admin
'https://ctr-e138-1518143905142-256977-01-04.hwx.site:21443/api/atlas/v2/glossary/'|
python -m json.tool
% Total% Received % Xferd Average Speed TimeTime Time Current
Dload Upload Total SpentLeft Speed
100 5640 5640 0224 0 --:--:-- 0:00:02 --:--:-- 224
[
{
"guid": "7fd4da3a-e6df-402f-a801-15883567d762",
"displayName": "Org",
"shortDescription": "hortonworks",
"longDescription": "HWX",
"qualifiedName": "Org",
"terms": [
{
"termGuid": "eac56951-c61f-4b49-a578-8be5d37d2679",
"relationGuid": "fb87ff3e-50e1-461f-be56-e0ab3d701c15",
"displayText": "apache"
},
{
"termGuid": "2cf05982-2e39-41c5-a656-b3703dbc8b4c",
"relationGuid": "a870627e-febe-474e-994e-31dc37aac716",
"displayText": "abc"
}
]
},
{
"guid": "1627ad45-84a3-4189-b708-54939fa18afa",
"displayName": "apache",
"shortDescription": "apache",
"longDescription": "apache",
"qualifiedName": "apache"
}
]
{noformat}
* */glossary/ API returns term definition*
{noformat}
curl -k -u admin:admin
'https://ctr-e138-1518143905142-256977-01-04.hwx.site:21443/api/atlas/v2/glossary/2cf05982-2e39-41c5-a656-b3703dbc8b4c'
| python -m json.tool
% Total% Received % Xferd Average Speed TimeTime Time Current
Dload Upload Total SpentLeft Speed
100 1420 1420 0 57 0 --:--:-- 0:00:02 --:--:--57
{
"guid": "2cf05982-2e39-41c5-a656-b3703dbc8b4c",
"displayName": "abc",
"shortDescription": "abc",
"longDescription": "abc",
"qualifiedName": "abc@Org"
}
{noformat}
atlas version
{noformat}
{
"Description": "Metadata Management and Data Governance Platform over
Hadoop",
"Version": "1.0.0.3.0.0.0-1242-r9b7265e3c2e483a3942f42f111c25b38885259fb",
"Name": "apache-atlas"
}
{noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (ATLAS-2429) ValuesMinCount and ValuesMaxCount in attributeDef always defaults to 1 for any value.
Ayub Pathan created ATLAS-2429:
--
Summary: ValuesMinCount and ValuesMaxCount in attributeDef always
defaults to 1 for any value.
Key: ATLAS-2429
URL: https://issues.apache.org/jira/browse/ATLAS-2429
Project: Atlas
Issue Type: Bug
Components: atlas-core
Affects Versions: 1.0.0
Reporter: Ayub Pathan
Fix For: 1.0.0
{noformat}
curl -u 'admin:admin' -X POST -d
'{"structDefs":[{"category":"STRUCT","name":"struct_sample4","attributeDefs":[{"name":"string","typeName":"string","isOptional":false,"cardinality":"SINGLE","valuesMinCount":-2147483648,"valuesMaxCount":2147483647,"isUnique":true,"isIndexable":true}]}]}'
-H 'Content-Type:application/json'
http://ctr-e137-1514896590304-36955-01-03.hwx.site:21000/api/atlas/v2/types/typedefs
| python -m json.tool
{
"classificationDefs": [],
"entityDefs": [],
"enumDefs": [],
"relationshipDefs": [],
"structDefs": [
{
"attributeDefs": [
{
"cardinality": "SINGLE",
"isIndexable": true,
"isOptional": false,
"isUnique": true,
"name": "string",
"typeName": "string",
"valuesMaxCount": 1,
"valuesMinCount": 1
}
],
"category": "STRUCT",
"createTime": 1517386162718,
"createdBy": "admin",
"description": "struct_sample4",
"guid": "2daf9320-a668-4543-a287-806923360b3e",
"name": "struct_sample4",
"typeVersion": "1.0",
"updateTime": 1517386162718,
"updatedBy": "admin",
"version": 1
}
]
}
{noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (ATLAS-2202) Duplicate kafka notifications observed for external table
[
https://issues.apache.org/jira/browse/ATLAS-2202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16200058#comment-16200058
]
Ayub Pathan commented on ATLAS-2202:
Old behavior before ATLAS-2198
* Hive db and external table notif
{noformat}
{"msgSourceIP":"172.27.25.128","msgCreatedBy":"hrt_qa","msgCreationTime":1507716386984,"message":{"entities":[{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Reference","id":{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Id","id":"-1576713171604213","version":0,"typeName":"hive_db","state":"ACTIVE"},"typeName":"hive_db","values":{"name":"default","location":"hdfs://ctr-e134-1499953498516-209460-01-04.hwx.site:8020/apps/hive/warehouse","description":"Default
Hive
database","ownerType":2,"qualifiedName":"default@cl1","owner":"public","clusterName":"cl1","parameters":{}},"traitNames":[],"traits":{},"systemAttributes":{}},{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Reference","id":{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Id","id":"-1576713171604212","version":0,"typeName":"hive_table","state":"ACTIVE"},"typeName":"hive_table","values":{"tableType":"EXTERNAL_TABLE","name":"externaltable_1","createTime":"2017-10-11T10:06:26.000Z","temporary":false,"db":{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Reference","id":{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Id","id":"-1576713171604213","version":0,"typeName":"hive_db","state":"ACTIVE"},"typeName":"hive_db","values":{"name":"default","location":"hdfs://ctr-e134-1499953498516-209460-01-04.hwx.site:8020/apps/hive/warehouse","description":"Default
Hive
database","ownerType":2,"qualifiedName":"default@cl1","owner":"public","clusterName":"cl1","parameters":{}},"traitNames":[],"traits":{},"systemAttributes":{}},"retention":0,"qualifiedName":"default.externaltable_1@cl1","columns":[{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Reference","id":{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Id","id":"-1576713171604210","version":0,"typeName":"hive_column","state":"ACTIVE"},"typeName":"hive_column","values":{"name":"id","qualifiedName":"default.externaltable_1.id@cl1","position":0,"owner":"hrt_qa","type":"int","table":{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Id","id":"-1576713171604212","version":0,"typeName":"hive_table","state":"ACTIVE"}},"traitNames":[],"traits":{},"systemAttributes":{}},{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Reference","id":{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Id","id":"-1576713171604209","version":0,"typeName":"hive_column","state":"ACTIVE"},"typeName":"hive_column","values":{"name":"name","qualifiedName":"default.externaltable_1.name@cl1","position":1,"owner":"hrt_qa","type":"string","table":{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Id","id":"-1576713171604212","version":0,"typeName":"hive_table","state":"ACTIVE"}},"traitNames":[],"traits":{},"systemAttributes":{}}],"lastAccessTime":"2017-10-11T10:06:26.000Z","owner":"hrt_qa","sd":{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Reference","id":{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Id","id":"-1576713171604211","version":0,"typeName":"hive_storagedesc","state":"ACTIVE"},"typeName":"hive_storagedesc","values":{"location":"hdfs://ctr-e134-1499953498516-209460-01-04.hwx.site:8020/apps/hive/warehouse/externaltable_1","serdeInfo":{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Struct","typeName":"hive_serde","values":{"serializationLib":"org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe","parameters":{"serialization.format":"1"}}},"qualifiedName":"default.externaltable_1@cl1_storage","outputFormat":"org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat","compressed":false,"numBuckets":-1,"inputFormat":"org.apache.hadoop.mapred.TextInputFormat","parameters":{},"storedAsSubDirectories":false,"table":{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Id","id":"-1576713171604212","version":0,"typeName":"hive_table","state":"ACTIVE"}},"traitNames":[],"traits":{},"systemAttributes":{}},"parameters":{"rawDataSize":"0","numFiles":"0","transient_lastDdlTime":"1507716386","totalSize":"0","EXTERNAL":"TRUE","COLUMN_STATS_ACCURATE":"{\"BASIC_STATS\":\"true\"}","numRows":"0"},"partitionKeys":[]},"traitNames":[],"traits":{},"systemAttributes":{}}],"type":"ENTITY_FULL_UPDATE","user":"hrt_qa"},"version":{"version":"1.0.0"},"msgCompressionKind":"NONE","msgSplitIdx":1,"msgSplitCount":1}
{noformat}
* Hive process notif
{noformat}
[jira] [Comment Edited] (ATLAS-2202) Duplicate kafka notifications observed for external table
[
https://issues.apache.org/jira/browse/ATLAS-2202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16200058#comment-16200058
]
Ayub Pathan edited comment on ATLAS-2202 at 10/11/17 10:08 AM:
---
Old behavior before ATLAS-2198
* Hive db and external table notif
{noformat}
{"msgSourceIP":"172.27.25.128","msgCreatedBy":"hrt_qa","msgCreationTime":1507716386984,"message":{"entities":[{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Reference","id":{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Id","id":"-1576713171604213","version":0,"typeName":"hive_db","state":"ACTIVE"},"typeName":"hive_db","values":{"name":"default","location":"hdfs://ctr-e134-1499953498516-209460-01-04.hwx.site:8020/apps/hive/warehouse","description":"Default
Hive
database","ownerType":2,"qualifiedName":"default@cl1","owner":"public","clusterName":"cl1","parameters":{}},"traitNames":[],"traits":{},"systemAttributes":{}},{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Reference","id":{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Id","id":"-1576713171604212","version":0,"typeName":"hive_table","state":"ACTIVE"},"typeName":"hive_table","values":{"tableType":"EXTERNAL_TABLE","name":"externaltable_1","createTime":"2017-10-11T10:06:26.000Z","temporary":false,"db":{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Reference","id":{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Id","id":"-1576713171604213","version":0,"typeName":"hive_db","state":"ACTIVE"},"typeName":"hive_db","values":{"name":"default","location":"hdfs://ctr-e134-1499953498516-209460-01-04.hwx.site:8020/apps/hive/warehouse","description":"Default
Hive
database","ownerType":2,"qualifiedName":"default@cl1","owner":"public","clusterName":"cl1","parameters":{}},"traitNames":[],"traits":{},"systemAttributes":{}},"retention":0,"qualifiedName":"default.externaltable_1@cl1","columns":[{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Reference","id":{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Id","id":"-1576713171604210","version":0,"typeName":"hive_column","state":"ACTIVE"},"typeName":"hive_column","values":{"name":"id","qualifiedName":"default.externaltable_1.id@cl1","position":0,"owner":"hrt_qa","type":"int","table":{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Id","id":"-1576713171604212","version":0,"typeName":"hive_table","state":"ACTIVE"}},"traitNames":[],"traits":{},"systemAttributes":{}},{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Reference","id":{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Id","id":"-1576713171604209","version":0,"typeName":"hive_column","state":"ACTIVE"},"typeName":"hive_column","values":{"name":"name","qualifiedName":"default.externaltable_1.name@cl1","position":1,"owner":"hrt_qa","type":"string","table":{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Id","id":"-1576713171604212","version":0,"typeName":"hive_table","state":"ACTIVE"}},"traitNames":[],"traits":{},"systemAttributes":{}}],"lastAccessTime":"2017-10-11T10:06:26.000Z","owner":"hrt_qa","sd":{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Reference","id":{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Id","id":"-1576713171604211","version":0,"typeName":"hive_storagedesc","state":"ACTIVE"},"typeName":"hive_storagedesc","values":{"location":"hdfs://ctr-e134-1499953498516-209460-01-04.hwx.site:8020/apps/hive/warehouse/externaltable_1","serdeInfo":{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Struct","typeName":"hive_serde","values":{"serializationLib":"org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe","parameters":{"serialization.format":"1"}}},"qualifiedName":"default.externaltable_1@cl1_storage","outputFormat":"org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat","compressed":false,"numBuckets":-1,"inputFormat":"org.apache.hadoop.mapred.TextInputFormat","parameters":{},"storedAsSubDirectories":false,"table":{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Id","id":"-1576713171604212","version":0,"typeName":"hive_table","state":"ACTIVE"}},"traitNames":[],"traits":{},"systemAttributes":{}},"parameters":{"rawDataSize":"0","numFiles":"0","transient_lastDdlTime":"1507716386","totalSize":"0","EXTERNAL":"TRUE","COLUMN_STATS_ACCURATE":"{\"BASIC_STATS\":\"true\"}","numRows":"0"},"partitionKeys":[]},"traitNames":[],"traits":{},"systemAttributes":{}}],"type":"ENTITY_FULL_UPDATE","user":"hrt_qa"},"version":{"version":"1.0.0"},"msgCompressionKind":"NONE","msgSplitIdx":1,"msgSplitCount":1}
{noformat}
* Hive process notif
{noformat}
[jira] [Created] (ATLAS-2202) Duplicate kafka notifications observed for external table
Ayub Pathan created ATLAS-2202:
--
Summary: Duplicate kafka notifications observed for external table
Key: ATLAS-2202
URL: https://issues.apache.org/jira/browse/ATLAS-2202
Project: Atlas
Issue Type: Bug
Components: atlas-intg
Affects Versions: 0.8.2
Reporter: Ayub Pathan
Fix For: 0.8.2
Steps to reproduce: Create external table and observe the kafka notifications.
* This notification includes hive_db and external hive_table notification
{noformat}
{"msgSourceIP":"172.27.39.68","msgCreatedBy":"hrt_qa","msgCreationTime":1507714828002,"message":{"entities":[{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Reference","id":{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Id","id":"-1706538104304055","version":0,"typeName":"hive_db","state":"ACTIVE"},"typeName":"hive_db","values":{"name":"default","location":"hdfs://ctr-e134-1499953498516-214922-01-03.hwx.site:8020/apps/hive/warehouse","description":"Default
Hive
database","ownerType":2,"qualifiedName":"default@cl1","owner":"public","clusterName":"cl1","parameters":{}},"traitNames":[],"traits":{},"systemAttributes":{}},{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Reference","id":{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Id","id":"-1706538104304054","version":0,"typeName":"hive_table","state":"ACTIVE"},"typeName":"hive_table","values":{"tableType":"EXTERNAL_TABLE","name":"externaltable_2","createTime":"2017-10-11T09:40:27.000Z","temporary":false,"db":{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Reference","id":{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Id","id":"-1706538104304055","version":0,"typeName":"hive_db","state":"ACTIVE"},"typeName":"hive_db","values":{"name":"default","location":"hdfs://ctr-e134-1499953498516-214922-01-03.hwx.site:8020/apps/hive/warehouse","description":"Default
Hive
[jira] [Updated] (ATLAS-2198) HiveHook fails with OOM when hive view with 500 columns are created.
[
https://issues.apache.org/jira/browse/ATLAS-2198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ayub Pathan updated ATLAS-2198:
---
Priority: Blocker (was: Critical)
> HiveHook fails with OOM when hive view with 500 columns are created.
>
>
> Key: ATLAS-2198
> URL: https://issues.apache.org/jira/browse/ATLAS-2198
> Project: Atlas
> Issue Type: Bug
> Components: atlas-intg
>Affects Versions: 0.8.2
>Reporter: Ayub Pathan
>Assignee: Apoorv Naik
>Priority: Blocker
> Fix For: 0.8.2
>
> Attachments: 0001-ATLAS-2198-Hive-Hook-OOM-fix.patch
>
>
> create below two commands with hivehook enabled, hiveHook fails with OOM.
> {noformat}
> create table t1_500(col0 string, col1 string, col2 string, col3 string, col4
> string, col5 string, col6 string, col7 string, col8 string, col9 string,
> col10 string, col11 string, col12 string, col13 string, col14 string, col15
> string, col16 string, col17 string, col18 string, col19 string, col20 string,
> col21 string, col22 string, col23 string, col24 string, col25 string, col26
> string, col27 string, col28 string, col29 string, col30 string, col31 string,
> col32 string, col33 string, col34 string, col35 string, col36 string, col37
> string, col38 string, col39 string, col40 string, col41 string, col42 string,
> col43 string, col44 string, col45 string, col46 string, col47 string, col48
> string, col49 string, col50 string, col51 string, col52 string, col53 string,
> col54 string, col55 string, col56 string, col57 string, col58 string, col59
> string, col60 string, col61 string, col62 string, col63 string, col64 string,
> col65 string, col66 string, col67 string, col68 string, col69 string, col70
> string, col71 string, col72 string, col73 string, col74 string, col75 string,
> col76 string, col77 string, col78 string, col79 string, col80 string, col81
> string, col82 string, col83 string, col84 string, col85 string, col86 string,
> col87 string, col88 string, col89 string, col90 string, col91 string, col92
> string, col93 string, col94 string, col95 string, col96 string, col97 string,
> col98 string, col99 string, col100 string, col101 string, col102 string,
> col103 string, col104 string, col105 string, col106 string, col107 string,
> col108 string, col109 string, col110 string, col111 string, col112 string,
> col113 string, col114 string, col115 string, col116 string, col117 string,
> col118 string, col119 string, col120 string, col121 string, col122 string,
> col123 string, col124 string, col125 string, col126 string, col127 string,
> col128 string, col129 string, col130 string, col131 string, col132 string,
> col133 string, col134 string, col135 string, col136 string, col137 string,
> col138 string, col139 string, col140 string, col141 string, col142 string,
> col143 string, col144 string, col145 string, col146 string, col147 string,
> col148 string, col149 string, col150 string, col151 string, col152 string,
> col153 string, col154 string, col155 string, col156 string, col157 string,
> col158 string, col159 string, col160 string, col161 string, col162 string,
> col163 string, col164 string, col165 string, col166 string, col167 string,
> col168 string, col169 string, col170 string, col171 string, col172 string,
> col173 string, col174 string, col175 string, col176 string, col177 string,
> col178 string, col179 string, col180 string, col181 string, col182 string,
> col183 string, col184 string, col185 string, col186 string, col187 string,
> col188 string, col189 string, col190 string, col191 string, col192 string,
> col193 string, col194 string, col195 string, col196 string, col197 string,
> col198 string, col199 string, col200 string, col201 string, col202 string,
> col203 string, col204 string, col205 string, col206 string, col207 string,
> col208 string, col209 string, col210 string, col211 string, col212 string,
> col213 string, col214 string, col215 string, col216 string, col217 string,
> col218 string, col219 string, col220 string, col221 string, col222 string,
> col223 string, col224 string, col225 string, col226 string, col227 string,
> col228 string, col229 string, col230 string, col231 string, col232 string,
> col233 string, col234 string, col235 string, col236 string, col237 string,
> col238 string, col239 string, col240 string, col241 string, col242 string,
> col243 string, col244 string, col245 string, col246 string, col247 string,
> col248 string, col249 string, col250 string, col251 string, col252 string,
> col253 string, col254 string, col255 string, col256 string, col257 string,
> col258 string, col259 string, col260 string, col261 string, col262 string,
> col263 string, col264 string, col265 string, col266 string, col267 string,
> col268 string, col269
[jira] [Updated] (ATLAS-2199) DeadLock in atlas while producing metadata with hive view > 300 cols.
[
https://issues.apache.org/jira/browse/ATLAS-2199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ayub Pathan updated ATLAS-2199:
---
Priority: Blocker (was: Critical)
> DeadLock in atlas while producing metadata with hive view > 300 cols.
> -
>
> Key: ATLAS-2199
> URL: https://issues.apache.org/jira/browse/ATLAS-2199
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Affects Versions: 0.8.2
>Reporter: Ayub Pathan
>Priority: Blocker
> Fix For: 0.8.2
>
> Attachments: application.log, atlas.20171009-192118.err,
> atlas.20171009-192118.out, thread-dump.txt
>
>
> Steps to reproduce:
> 1. Produce messages using hive hook - create view with 300 columns. Example,
> command below
> {noformat}
> create table t1_300(col0 string, col1 string, col2 string, col3 string, col4
> string, col5 string, col6 string, col7 string, col8 string, col9 string,
> col10 string, col11 string, col12 string, col13 string, col14 string, col15
> string, col16 string, col17 string, col18 string, col19 string, col20 string,
> col21 string, col22 string, col23 string, col24 string, col25 string, col26
> string, col27 string, col28 string, col29 string, col30 string, col31 string,
> col32 string, col33 string, col34 string, col35 string, col36 string, col37
> string, col38 string, col39 string, col40 string, col41 string, col42 string,
> col43 string, col44 string, col45 string, col46 string, col47 string, col48
> string, col49 string, col50 string, col51 string, col52 string, col53 string,
> col54 string, col55 string, col56 string, col57 string, col58 string, col59
> string, col60 string, col61 string, col62 string, col63 string, col64 string,
> col65 string, col66 string, col67 string, col68 string, col69 string, col70
> string, col71 string, col72 string, col73 string, col74 string, col75 string,
> col76 string, col77 string, col78 string, col79 string, col80 string, col81
> string, col82 string, col83 string, col84 string, col85 string, col86 string,
> col87 string, col88 string, col89 string, col90 string, col91 string, col92
> string, col93 string, col94 string, col95 string, col96 string, col97 string,
> col98 string, col99 string, col100 string, col101 string, col102 string,
> col103 string, col104 string, col105 string, col106 string, col107 string,
> col108 string, col109 string, col110 string, col111 string, col112 string,
> col113 string, col114 string, col115 string, col116 string, col117 string,
> col118 string, col119 string, col120 string, col121 string, col122 string,
> col123 string, col124 string, col125 string, col126 string, col127 string,
> col128 string, col129 string, col130 string, col131 string, col132 string,
> col133 string, col134 string, col135 string, col136 string, col137 string,
> col138 string, col139 string, col140 string, col141 string, col142 string,
> col143 string, col144 string, col145 string, col146 string, col147 string,
> col148 string, col149 string, col150 string, col151 string, col152 string,
> col153 string, col154 string, col155 string, col156 string, col157 string,
> col158 string, col159 string, col160 string, col161 string, col162 string,
> col163 string, col164 string, col165 string, col166 string, col167 string,
> col168 string, col169 string, col170 string, col171 string, col172 string,
> col173 string, col174 string, col175 string, col176 string, col177 string,
> col178 string, col179 string, col180 string, col181 string, col182 string,
> col183 string, col184 string, col185 string, col186 string, col187 string,
> col188 string, col189 string, col190 string, col191 string, col192 string,
> col193 string, col194 string, col195 string, col196 string, col197 string,
> col198 string, col199 string, col200 string, col201 string, col202 string,
> col203 string, col204 string, col205 string, col206 string, col207 string,
> col208 string, col209 string, col210 string, col211 string, col212 string,
> col213 string, col214 string, col215 string, col216 string, col217 string,
> col218 string, col219 string, col220 string, col221 string, col222 string,
> col223 string, col224 string, col225 string, col226 string, col227 string,
> col228 string, col229 string, col230 string, col231 string, col232 string,
> col233 string, col234 string, col235 string, col236 string, col237 string,
> col238 string, col239 string, col240 string, col241 string, col242 string,
> col243 string, col244 string, col245 string, col246 string, col247 string,
> col248 string, col249 string, col250 string, col251 string, col252 string,
> col253 string, col254 string, col255 string, col256 string, col257 string,
> col258 string, col259 string, col260 string, col261 string, col262 string,
> col263 string, col264 string, col265
[jira] [Created] (ATLAS-2199) DeadLock in atlas while producing metadata with hive view > 300 cols.
Ayub Pathan created ATLAS-2199:
--
Summary: DeadLock in atlas while producing metadata with hive view
> 300 cols.
Key: ATLAS-2199
URL: https://issues.apache.org/jira/browse/ATLAS-2199
Project: Atlas
Issue Type: Bug
Components: atlas-core
Affects Versions: 0.8.2
Reporter: Ayub Pathan
Priority: Critical
Fix For: 0.8.2
Attachments: application.log, atlas.20171009-192118.err,
atlas.20171009-192118.out, thread-dump.txt
Steps to reproduce:
1. Produce messages using hive hook - create view with 300 columns. Example,
command below
{noformat}
create table t1_300(col0 string, col1 string, col2 string, col3 string, col4
string, col5 string, col6 string, col7 string, col8 string, col9 string, col10
string, col11 string, col12 string, col13 string, col14 string, col15 string,
col16 string, col17 string, col18 string, col19 string, col20 string, col21
string, col22 string, col23 string, col24 string, col25 string, col26 string,
col27 string, col28 string, col29 string, col30 string, col31 string, col32
string, col33 string, col34 string, col35 string, col36 string, col37 string,
col38 string, col39 string, col40 string, col41 string, col42 string, col43
string, col44 string, col45 string, col46 string, col47 string, col48 string,
col49 string, col50 string, col51 string, col52 string, col53 string, col54
string, col55 string, col56 string, col57 string, col58 string, col59 string,
col60 string, col61 string, col62 string, col63 string, col64 string, col65
string, col66 string, col67 string, col68 string, col69 string, col70 string,
col71 string, col72 string, col73 string, col74 string, col75 string, col76
string, col77 string, col78 string, col79 string, col80 string, col81 string,
col82 string, col83 string, col84 string, col85 string, col86 string, col87
string, col88 string, col89 string, col90 string, col91 string, col92 string,
col93 string, col94 string, col95 string, col96 string, col97 string, col98
string, col99 string, col100 string, col101 string, col102 string, col103
string, col104 string, col105 string, col106 string, col107 string, col108
string, col109 string, col110 string, col111 string, col112 string, col113
string, col114 string, col115 string, col116 string, col117 string, col118
string, col119 string, col120 string, col121 string, col122 string, col123
string, col124 string, col125 string, col126 string, col127 string, col128
string, col129 string, col130 string, col131 string, col132 string, col133
string, col134 string, col135 string, col136 string, col137 string, col138
string, col139 string, col140 string, col141 string, col142 string, col143
string, col144 string, col145 string, col146 string, col147 string, col148
string, col149 string, col150 string, col151 string, col152 string, col153
string, col154 string, col155 string, col156 string, col157 string, col158
string, col159 string, col160 string, col161 string, col162 string, col163
string, col164 string, col165 string, col166 string, col167 string, col168
string, col169 string, col170 string, col171 string, col172 string, col173
string, col174 string, col175 string, col176 string, col177 string, col178
string, col179 string, col180 string, col181 string, col182 string, col183
string, col184 string, col185 string, col186 string, col187 string, col188
string, col189 string, col190 string, col191 string, col192 string, col193
string, col194 string, col195 string, col196 string, col197 string, col198
string, col199 string, col200 string, col201 string, col202 string, col203
string, col204 string, col205 string, col206 string, col207 string, col208
string, col209 string, col210 string, col211 string, col212 string, col213
string, col214 string, col215 string, col216 string, col217 string, col218
string, col219 string, col220 string, col221 string, col222 string, col223
string, col224 string, col225 string, col226 string, col227 string, col228
string, col229 string, col230 string, col231 string, col232 string, col233
string, col234 string, col235 string, col236 string, col237 string, col238
string, col239 string, col240 string, col241 string, col242 string, col243
string, col244 string, col245 string, col246 string, col247 string, col248
string, col249 string, col250 string, col251 string, col252 string, col253
string, col254 string, col255 string, col256 string, col257 string, col258
string, col259 string, col260 string, col261 string, col262 string, col263
string, col264 string, col265 string, col266 string, col267 string, col268
string, col269 string, col270 string, col271 string, col272 string, col273
string, col274 string, col275 string, col276 string, col277 string, col278
string, col279 string, col280 string, col281 string, col282 string, col283
string, col284 string, col285 string, col286 string, col287 string,
[jira] [Created] (ATLAS-2198) HiveHook fails with OOM when hive view with 500 columns are created.
Ayub Pathan created ATLAS-2198:
--
Summary: HiveHook fails with OOM when hive view with 500 columns
are created.
Key: ATLAS-2198
URL: https://issues.apache.org/jira/browse/ATLAS-2198
Project: Atlas
Issue Type: Bug
Components: atlas-intg
Affects Versions: 0.8.2
Reporter: Ayub Pathan
Priority: Critical
Fix For: 0.8.2
create below two commands with hivehook enabled, hiveHook fails with OOM.
{noformat}
create table t1_500(col0 string, col1 string, col2 string, col3 string, col4
string, col5 string, col6 string, col7 string, col8 string, col9 string, col10
string, col11 string, col12 string, col13 string, col14 string, col15 string,
col16 string, col17 string, col18 string, col19 string, col20 string, col21
string, col22 string, col23 string, col24 string, col25 string, col26 string,
col27 string, col28 string, col29 string, col30 string, col31 string, col32
string, col33 string, col34 string, col35 string, col36 string, col37 string,
col38 string, col39 string, col40 string, col41 string, col42 string, col43
string, col44 string, col45 string, col46 string, col47 string, col48 string,
col49 string, col50 string, col51 string, col52 string, col53 string, col54
string, col55 string, col56 string, col57 string, col58 string, col59 string,
col60 string, col61 string, col62 string, col63 string, col64 string, col65
string, col66 string, col67 string, col68 string, col69 string, col70 string,
col71 string, col72 string, col73 string, col74 string, col75 string, col76
string, col77 string, col78 string, col79 string, col80 string, col81 string,
col82 string, col83 string, col84 string, col85 string, col86 string, col87
string, col88 string, col89 string, col90 string, col91 string, col92 string,
col93 string, col94 string, col95 string, col96 string, col97 string, col98
string, col99 string, col100 string, col101 string, col102 string, col103
string, col104 string, col105 string, col106 string, col107 string, col108
string, col109 string, col110 string, col111 string, col112 string, col113
string, col114 string, col115 string, col116 string, col117 string, col118
string, col119 string, col120 string, col121 string, col122 string, col123
string, col124 string, col125 string, col126 string, col127 string, col128
string, col129 string, col130 string, col131 string, col132 string, col133
string, col134 string, col135 string, col136 string, col137 string, col138
string, col139 string, col140 string, col141 string, col142 string, col143
string, col144 string, col145 string, col146 string, col147 string, col148
string, col149 string, col150 string, col151 string, col152 string, col153
string, col154 string, col155 string, col156 string, col157 string, col158
string, col159 string, col160 string, col161 string, col162 string, col163
string, col164 string, col165 string, col166 string, col167 string, col168
string, col169 string, col170 string, col171 string, col172 string, col173
string, col174 string, col175 string, col176 string, col177 string, col178
string, col179 string, col180 string, col181 string, col182 string, col183
string, col184 string, col185 string, col186 string, col187 string, col188
string, col189 string, col190 string, col191 string, col192 string, col193
string, col194 string, col195 string, col196 string, col197 string, col198
string, col199 string, col200 string, col201 string, col202 string, col203
string, col204 string, col205 string, col206 string, col207 string, col208
string, col209 string, col210 string, col211 string, col212 string, col213
string, col214 string, col215 string, col216 string, col217 string, col218
string, col219 string, col220 string, col221 string, col222 string, col223
string, col224 string, col225 string, col226 string, col227 string, col228
string, col229 string, col230 string, col231 string, col232 string, col233
string, col234 string, col235 string, col236 string, col237 string, col238
string, col239 string, col240 string, col241 string, col242 string, col243
string, col244 string, col245 string, col246 string, col247 string, col248
string, col249 string, col250 string, col251 string, col252 string, col253
string, col254 string, col255 string, col256 string, col257 string, col258
string, col259 string, col260 string, col261 string, col262 string, col263
string, col264 string, col265 string, col266 string, col267 string, col268
string, col269 string, col270 string, col271 string, col272 string, col273
string, col274 string, col275 string, col276 string, col277 string, col278
string, col279 string, col280 string, col281 string, col282 string, col283
string, col284 string, col285 string, col286 string, col287 string, col288
string, col289 string, col290 string, col291 string, col292 string, col293
string, col294 string, col295 string, col296 string, col297 string,
[jira] [Created] (ATLAS-2195) Stale transaction eviction errors observed in atlas application logs
Ayub Pathan created ATLAS-2195:
--
Summary: Stale transaction eviction errors observed in atlas
application logs
Key: ATLAS-2195
URL: https://issues.apache.org/jira/browse/ATLAS-2195
Project: Atlas
Issue Type: Bug
Components: atlas-core
Affects Versions: 0.8.2
Reporter: Ayub Pathan
Priority: Critical
Fix For: 0.8.2
{noformat}
2017-10-09 20:18:20,728 ERROR - [Thread-371:] ~ Evicted
[258@ac1b1980131887-ctr-e134-1499953498516-209460-01-04-hwx-site1] from
cache but waiting too long for transactions to close. Stale transaction alert
on: [standardtitantx[null], standardtitantx[null]] (ManagementLogger:189)
2017-10-09 20:18:20,738 ERROR - [Thread-370:] ~ Evicted
[257@ac1b1980131887-ctr-e134-1499953498516-209460-01-04-hwx-site1] from
cache but waiting too long for transactions to close. Stale transaction alert
on: [standardtitantx[null], standardtitantx[null]] (ManagementLogger:189)
2017-10-09 20:18:20,808 ERROR - [Thread-372:] ~ Evicted
[259@ac1b1980131887-ctr-e134-1499953498516-209460-01-04-hwx-site1] from
cache but waiting too long for transactions to close. Stale transaction alert
on: [standardtitantx[null], standardtitantx[null]] (ManagementLogger:189)
2017-10-09 20:18:25,658 ERROR - [Thread-373:] ~ Evicted
[260@ac1b1980131887-ctr-e134-1499953498516-209460-01-04-hwx-site1] from
cache but waiting too long for transactions to close. Stale transaction alert
on: [standardtitantx[null], standardtitantx[null]] (ManagementLogger:189)
2017-10-09 20:18:25,698 ERROR - [Thread-374:] ~ Evicted
[261@ac1b1980131887-ctr-e134-1499953498516-209460-01-04-hwx-site1] from
cache but waiting too long for transactions to close. Stale transaction alert
on: [standardtitantx[null], standardtitantx[null]] (ManagementLogger:189)
2017-10-09 20:18:25,721 ERROR - [Thread-375:] ~ Evicted
[262@ac1b1980131887-ctr-e134-1499953498516-209460-01-04-hwx-site1] from
cache but waiting too long for transactions to close. Stale transaction alert
on: [standardtitantx[null], standardtitantx[null]] (ManagementLogger:189)
2017-10-09 20:18:25,739 ERROR - [Thread-376:] ~ Evicted
[263@ac1b1980131887-ctr-e134-1499953498516-209460-01-04-hwx-site1] from
cache but waiting too long for transactions to close. Stale transaction alert
on: [standardtitantx[null], standardtitantx[null]] (ManagementLogger:189)
2017-10-09 20:18:25,778 ERROR - [Thread-377:] ~ Evicted
[264@ac1b1980131887-ctr-e134-1499953498516-209460-01-04-hwx-site1] from
cache but waiting too long for transactions to close. Stale transaction alert
on: [standardtitantx[null], standardtitantx[null]] (ManagementLogger:189)
2017-10-09 20:18:30,661 ERROR - [Thread-378:] ~ Evicted
[265@ac1b1980131887-ctr-e134-1499953498516-209460-01-04-hwx-site1] from
cache but waiting too long for transactions to close. Stale transaction alert
on: [standardtitantx[null], standardtitantx[null]] (ManagementLogger:189)
2017-10-09 20:18:30,687 ERROR - [Thread-379:] ~ Evicted
[266@ac1b1980131887-ctr-e134-1499953498516-209460-01-04-hwx-site1] from
cache but waiting too long for transactions to close. Stale transaction alert
on: [standardtitantx[null], standardtitantx[null]] (ManagementLogger:189)
{noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (ATLAS-2190) Remove duplicate dependency blocks with different versions, resulting in atlas startup failure with signature mismatch exception
[
https://issues.apache.org/jira/browse/ATLAS-2190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ayub Pathan updated ATLAS-2190:
---
Attachment: ATLAS-2190-branch-0.8.patch
> Remove duplicate dependency blocks with different versions, resulting in
> atlas startup failure with signature mismatch exception
>
>
> Key: ATLAS-2190
> URL: https://issues.apache.org/jira/browse/ATLAS-2190
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Affects Versions: 0.8.2
>Reporter: Ayub Pathan
>Assignee: Ayub Pathan
>Priority: Blocker
> Fix For: 0.8.2
>
> Attachments: ATLAS-2190-branch-0.8.patch
>
>
> In the current pom, below two are duplicate entries resulting in atlas
> startup failure
> {noformat}
> -
> -com.sun.jersey
> -jersey-servlet
> -${jersey.version}
> -
> -
> -
> -javax.servlet.jsp
> -jsp-api
> -2.0
> -
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Assigned] (ATLAS-2190) Remove duplicate dependency blocks with different versions, resulting in atlas startup failure with signature mismatch exception
[
https://issues.apache.org/jira/browse/ATLAS-2190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ayub Pathan reassigned ATLAS-2190:
--
Assignee: Ayub Pathan
> Remove duplicate dependency blocks with different versions, resulting in
> atlas startup failure with signature mismatch exception
>
>
> Key: ATLAS-2190
> URL: https://issues.apache.org/jira/browse/ATLAS-2190
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Affects Versions: 0.8.2
>Reporter: Ayub Pathan
>Assignee: Ayub Pathan
>Priority: Blocker
> Fix For: 0.8.2
>
>
> In the current pom, below two are duplicate entries resulting in atlas
> startup failure
> {noformat}
> -
> -com.sun.jersey
> -jersey-servlet
> -${jersey.version}
> -
> -
> -
> -javax.servlet.jsp
> -jsp-api
> -2.0
> -
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (ATLAS-2190) Remove duplicate dependency blocks with different versions, resulting in atlas startup failure with signature mismatch exception
[
https://issues.apache.org/jira/browse/ATLAS-2190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ayub Pathan updated ATLAS-2190:
---
Affects Version/s: 0.8.2
Priority: Blocker (was: Major)
Fix Version/s: 0.8.2
Description:
In the current pom, below two are duplicate entries resulting in atlas startup
failure
{noformat}
-
-com.sun.jersey
-jersey-servlet
-${jersey.version}
-
-
-
-javax.servlet.jsp
-jsp-api
-2.0
-
{noformat}
Component/s: atlas-core
Summary: Remove duplicate dependency blocks with different
versions, resulting in atlas startup failure with signature mismatch exception
(was: Remove duplicate dependency blocks, resulting in signature )
> Remove duplicate dependency blocks with different versions, resulting in
> atlas startup failure with signature mismatch exception
>
>
> Key: ATLAS-2190
> URL: https://issues.apache.org/jira/browse/ATLAS-2190
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Affects Versions: 0.8.2
>Reporter: Ayub Pathan
>Priority: Blocker
> Fix For: 0.8.2
>
>
> In the current pom, below two are duplicate entries resulting in atlas
> startup failure
> {noformat}
> -
> -com.sun.jersey
> -jersey-servlet
> -${jersey.version}
> -
> -
> -
> -javax.servlet.jsp
> -jsp-api
> -2.0
> -
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (ATLAS-2190) Remove duplicate dependency blocks, resulting in signature
Ayub Pathan created ATLAS-2190: -- Summary: Remove duplicate dependency blocks, resulting in signature Key: ATLAS-2190 URL: https://issues.apache.org/jira/browse/ATLAS-2190 Project: Atlas Issue Type: Bug Reporter: Ayub Pathan -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (ATLAS-2188) UI can trim whitespaces at the beginning or end of search query
Ayub Pathan created ATLAS-2188:
--
Summary: UI can trim whitespaces at the beginning or end of search
query
Key: ATLAS-2188
URL: https://issues.apache.org/jira/browse/ATLAS-2188
Project: Atlas
Issue Type: Bug
Components: atlas-webui
Affects Versions: 0.8.2
Reporter: Ayub Pathan
For example, the below query has trailing space in the query resulting to not
returning anything, but if you see semantically this query seems to be correct.
If a user by mistake enters a whitespace at the beginning or end of search
query, this will result in null response, which seems to be very strict.
Instead, UI can trim the whitespaces at the beginning or end of the query.
Please let me know your thoughts.
query json sample:
{noformat}
{"excludeDeletedEntities":true,"entityFilters":{"condition":"AND","criterion":[{"attributeName":"name","operator":"neq","attributeValue":"123"},{"attributeName":"name","operator":"startsWith","attributeValue":"
hbase"},{"attributeName":"owner","operator":"startsWith","attributeValue":"hrt_qa"}]},"tagFilters":null,"attributes":[],"query":null,"limit":25,"offset":0,"typeName":"hbase_table","classification":null}
{noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (ATLAS-2186) Atlas not responding for long time after startup
Ayub Pathan created ATLAS-2186: -- Summary: Atlas not responding for long time after startup Key: ATLAS-2186 URL: https://issues.apache.org/jira/browse/ATLAS-2186 Project: Atlas Issue Type: Bug Components: atlas-core Affects Versions: 0.8.1 Reporter: Ayub Pathan Priority: Blocker Fix For: 0.8.1 Atlas with latest bits is not responding for any query, the request times out. This behavior can be seen for more than ~15 minutes after atlas startup and everything will be back to normal after this period. I suspect one of the threads is looping or hanging which is resulting in this state. Atlas logs does not show any suspicious information (checked with debug logs as well). I took a thread dump of Atlas, attaching the dump file and report below. I strongly suspect, this might be an issue with newer jetty version introduced recently. Thread dump report: http://fastthread.io/my-thread-report.jsp?p=c2hhcmVkLzIwMTcvMTAvMy8tLWF0bGFzLjIwMTcxMDAzLTEyMjUzMS5vdXQtLTEyLTE2LTQ2 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (ATLAS-2177) offset is not reset when search by type is changed, leads to undesired results.
Ayub Pathan created ATLAS-2177: -- Summary: offset is not reset when search by type is changed, leads to undesired results. Key: ATLAS-2177 URL: https://issues.apache.org/jira/browse/ATLAS-2177 Project: Atlas Issue Type: Bug Components: atlas-webui Affects Versions: 0.9-incubating Reporter: Ayub Pathan Priority: Critical Fix For: 0.9-incubating Attachments: bug-snap2.mp4 Attaching recording for reference. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (ATLAS-2175) Infinite refresh scroll appears when a search query is saved.
Ayub Pathan created ATLAS-2175: -- Summary: Infinite refresh scroll appears when a search query is saved. Key: ATLAS-2175 URL: https://issues.apache.org/jira/browse/ATLAS-2175 Project: Atlas Issue Type: Bug Components: atlas-webui Affects Versions: 0.9-incubating Reporter: Ayub Pathan Fix For: 0.9-incubating Attachments: bug-snap.mp4 Attaching a recording for reference. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (ATLAS-2155) Attributes with null values are not filtered using search parameters
Ayub Pathan created ATLAS-2155:
--
Summary: Attributes with null values are not filtered using search
parameters
Key: ATLAS-2155
URL: https://issues.apache.org/jira/browse/ATLAS-2155
Project: Atlas
Issue Type: Bug
Components: atlas-core
Affects Versions: 0.9.1
Reporter: Ayub Pathan
Fix For: 0.9.1
For example: Below example shows attrbutes like "Description" has nullvalues
{noformat}
apathan@Ayub-sys-101:~/Hortonworks > curl
'http://localhost:21000/api/atlas/v2/search/basic' -H 'Origin:
http://localhost:21000' -H 'Accept-Encoding: gzip, deflate, br' -H
'X-XSRF-HEADER: ""' -H 'Authorization: Basic YWRtaW46YWRtaW4=' -H
'Accept-Language: en-US,en;q=0.8' -H 'Cookie:
ATLASSESSIONID=9hm04vl07k7f1vmcmeq9q6kg9' -H 'Connection: keep-alive' -H
'Pragma: no-cache' -H 'X-Requested-With: XMLHttpRequest' -H 'User-Agent:
Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_6) AppleWebKit/537.36 (KHTML, like
Gecko) Chrome/60.0.3112.113 Safari/537.36' -H 'Content-Type: application/json'
-H 'Accept: application/json, text/javascript, */*; q=0.01' -H 'Cache-Control:
no-cache' -H 'Referer: http://localhost:21000/index.html' --data-binary
'{"excludeDeletedEntities":true,"entityFilters":{"condition":"AND","criterion":[{"attributeName":"owner","operator":"eq","attributeValue":"apathan"}]},"tagFilters":null,"attributes":["qualifiedName"],"query":null,"limit":25,"offset":0,"typeName":"hive_table","classification":null}'
--compressed | python -m json.tool
% Total% Received % Xferd Average Speed TimeTime Time Current
Dload Upload Total SpentLeft Speed
100 11050 825 100 280 13953 4735 --:--:-- --:--:-- --:--:-- 14224
{
"entities": [
{
"attributes": {
"createTime": 1505475604641,
"description": null,
"name": "h5",
"owner": "apathan",
"qualifiedName": "default.h5@primary"
},
"classificationNames": [],
"displayText": "h5",
"guid": "7a8955bf-7395-454f-9d67-ea3ba01529bf",
"status": "ACTIVE",
"typeName": "hive_table"
},
{
"attributes": {
"createTime": 1505475721717,
"description": null,
"name": "h7",
"owner": "apathan",
"qualifiedName": "default.h7@primary"
},
"classificationNames": [
"abc",
"dsfads",
"sdf"
],
"displayText": "h7",
"guid": "82d6de4e-d808-4c88-8207-be5a58fbbb4e",
"status": "ACTIVE",
"typeName": "hive_table"
}
],
"queryType": "BASIC",
"searchParameters": {
"attributes": [
"qualifiedName"
],
"entityFilters": {
"condition": "AND",
"criterion": [
{
"attributeName": "owner",
"attributeValue": "apathan",
"operator": "eq"
}
]
},
"excludeDeletedEntities": true,
"limit": 25,
"offset": 0,
"typeName": "hive_table"
}
}
{noformat}
*Filtering those attributes using any operand will not take effect and it will
always result empty response*
example:
{noformat}
apathan@Ayub-sys-101:~/Hortonworks > curl
'http://localhost:21000/api/atlas/v2/search/basic' -H 'Origin:
http://localhost:21000' -H 'Accept-Encoding: gzip, deflate, br' -H
'X-XSRF-HEADER: ""' -H 'Authorization: Basic YWRtaW46YWRtaW4=' -H
'Accept-Language: en-US,en;q=0.8' -H 'Cookie:
ATLASSESSIONID=9hm04vl07k7f1vmcmeq9q6kg9' -H 'Connection: keep-alive' -H
'Pragma: no-cache' -H 'X-Requested-With: XMLHttpRequest' -H 'User-Agent:
Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_6) AppleWebKit/537.36 (KHTML, like
Gecko) Chrome/60.0.3112.113 Safari/537.36' -H 'Content-Type: application/json'
-H 'Accept: application/json, text/javascript, */*; q=0.01' -H 'Cache-Control:
no-cache' -H 'Referer: http://localhost:21000/index.html' --data-binary
'{"excludeDeletedEntities":true,"entityFilters":{"condition":"AND","criterion":[{"attributeName":"description","operator":"neq","attributeValue":"ac"}]},"tagFilters":null,"attributes":["qualifiedName"],"query":null,"limit":25,"offset":0,"typeName":"hive_table","classification":null}'
--compressed | python -m json.tool
% Total% Received % Xferd Average Speed TimeTime Time Current
Dload Upload Total SpentLeft Speed
100 5520 270 100 282 4562 4765 --:--:-- --:--:-- --:--:-- 4862
{
"queryType": "BASIC",
"searchParameters": {
"attributes": [
"qualifiedName"
],
"entityFilters": {
[jira] [Updated] (ATLAS-2152) Saved Search: UI: Atlas UI not able to load the saved search
[
https://issues.apache.org/jira/browse/ATLAS-2152?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ayub Pathan updated ATLAS-2152:
---
Attachment: SavedSearchSample.json
Attaching the sample savedsearch.json
[~kevalbhatt] [~madhan.neethiraj]
> Saved Search: UI: Atlas UI not able to load the saved search
>
>
> Key: ATLAS-2152
> URL: https://issues.apache.org/jira/browse/ATLAS-2152
> Project: Atlas
> Issue Type: Bug
> Components: atlas-webui
>Affects Versions: 0.9.1
>Reporter: Ayub Pathan
>Priority: Critical
> Fix For: 0.9.1
>
> Attachments: SavedSearchSample.json
>
>
> After saving the search query, Atlas UI unable to load the saved search.
> Below exception is observed on the browser console
> {noformat}
> CommonViewFunction.js:487 Uncaught TypeError: Cannot read property 'toJSON'
> of undefined
> at CommonViewFunction.js:487
> at Function.s.each.s.forEach (underscore-min.js:5)
> at Object.g.generateUrlFromSaveSearchObject (CommonViewFunction.js:476)
> at L.d.applyValue (SearchLayoutView.js:155)
> at L.d.stateChange (SaveSearchItemView.js:60)
> at HTMLLIElement.dispatch (jquery.min.js:3)
> at HTMLLIElement.q.handle (jquery.min.js:3)
> {noformat}
> Tested on master.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (ATLAS-2152) Saved Search: UI: Atlas UI not able to load the saved search
Ayub Pathan created ATLAS-2152:
--
Summary: Saved Search: UI: Atlas UI not able to load the saved
search
Key: ATLAS-2152
URL: https://issues.apache.org/jira/browse/ATLAS-2152
Project: Atlas
Issue Type: Bug
Components: atlas-webui
Affects Versions: 0.9.1
Reporter: Ayub Pathan
Priority: Critical
Fix For: 0.9.1
After saving the search query, Atlas UI unable to load the saved search.
Below exception is observed on the browser console
{noformat}
CommonViewFunction.js:487 Uncaught TypeError: Cannot read property 'toJSON' of
undefined
at CommonViewFunction.js:487
at Function.s.each.s.forEach (underscore-min.js:5)
at Object.g.generateUrlFromSaveSearchObject (CommonViewFunction.js:476)
at L.d.applyValue (SearchLayoutView.js:155)
at L.d.stateChange (SaveSearchItemView.js:60)
at HTMLLIElement.dispatch (jquery.min.js:3)
at HTMLLIElement.q.handle (jquery.min.js:3)
{noformat}
Tested on master.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (ATLAS-2147) querying supertype "Asset" with multiple filters fails with "unresolved datatype. Can not resolve due to unresolved children"
Ayub Pathan created ATLAS-2147:
--
Summary: querying supertype "Asset" with multiple filters fails
with "unresolved datatype. Can not resolve due to unresolved children"
Key: ATLAS-2147
URL: https://issues.apache.org/jira/browse/ATLAS-2147
Project: Atlas
Issue Type: Bug
Components: atlas-core
Affects Versions: 0.9.1
Reporter: Ayub Pathan
Fix For: 0.9.1
Example query with multiple filters: Asset where (name="") or
(qualifiesName="")
Stacktrace:
{noformat}
2017-09-19 14:37:15,166 ERROR - [pool-1-thread-9 -
1ca16b19-634e-48bb-b872-20025c210b75:] ~ graph rollback due to exception
(GraphTransactionInterceptor:71)
org.apache.atlas.query.Expressions$UnresolvedException: Unresolved datatype.
Can not resolve due to unresolved children, expression: Asset where (name =
"h7") or (qualifiedName = "default.h7@primary")
at
org.apache.atlas.query.Expressions$FilterExpression.dataType$lzycompute(Expressions.scala:708)
at
org.apache.atlas.query.Expressions$FilterExpression.dataType(Expressions.scala:706)
at org.apache.atlas.query.Resolver.apply(Resolver.scala:48)
at org.apache.atlas.query.Resolver.apply(Resolver.scala:25)
at scala.PartialFunction$class.applyOrElse(PartialFunction.scala:123)
at org.apache.atlas.query.Resolver.applyOrElse(Resolver.scala:25)
at
org.apache.atlas.query.Expressions$Expression$class.transformUp(Expressions.scala:201)
at
org.apache.atlas.query.Expressions$IdExpression.transformUp(Expressions.scala:404)
at
org.apache.atlas.query.Expressions$Expression$$anonfun$4.apply(Expressions.scala:211)
at scala.collection.Iterator$$anon$11.next(Iterator.scala:409)
at scala.collection.Iterator$class.foreach(Iterator.scala:893)
at scala.collection.AbstractIterator.foreach(Iterator.scala:1336)
at
scala.collection.generic.Growable$class.$plus$plus$eq(Growable.scala:59)
at
scala.collection.mutable.ArrayBuffer.$plus$plus$eq(ArrayBuffer.scala:104)
at
scala.collection.mutable.ArrayBuffer.$plus$plus$eq(ArrayBuffer.scala:48)
at scala.collection.TraversableOnce$class.to(TraversableOnce.scala:310)
at scala.collection.AbstractIterator.to(Iterator.scala:1336)
at
scala.collection.TraversableOnce$class.toBuffer(TraversableOnce.scala:302)
at scala.collection.AbstractIterator.toBuffer(Iterator.scala:1336)
at
scala.collection.TraversableOnce$class.toArray(TraversableOnce.scala:289)
at scala.collection.AbstractIterator.toArray(Iterator.scala:1336)
at
org.apache.atlas.query.Expressions$Expression$class.transformChildrenUp(Expressions.scala:240)
at
org.apache.atlas.query.Expressions$BinaryExpression.transformChildrenUp(Expressions.scala:367)
at
org.apache.atlas.query.Expressions$Expression$class.transformUp(Expressions.scala:199)
at
org.apache.atlas.query.Expressions$BinaryExpression.transformUp(Expressions.scala:367)
at
org.apache.atlas.query.Expressions$Expression$$anonfun$4$$anonfun$apply$3.apply(Expressions.scala:229)
at
scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
at
scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
at scala.collection.immutable.List.foreach(List.scala:381)
at scala.collection.TraversableLike$class.map(TraversableLike.scala:234)
at scala.collection.immutable.List.map(List.scala:285)
at
org.apache.atlas.query.Expressions$Expression$$anonfun$4.apply(Expressions.scala:227)
at scala.collection.Iterator$$anon$11.next(Iterator.scala:409)
at scala.collection.Iterator$class.foreach(Iterator.scala:893)
at scala.collection.AbstractIterator.foreach(Iterator.scala:1336)
at
scala.collection.generic.Growable$class.$plus$plus$eq(Growable.scala:59)
at
scala.collection.mutable.ArrayBuffer.$plus$plus$eq(ArrayBuffer.scala:104)
at
scala.collection.mutable.ArrayBuffer.$plus$plus$eq(ArrayBuffer.scala:48)
at scala.collection.TraversableOnce$class.to(TraversableOnce.scala:310)
at scala.collection.AbstractIterator.to(Iterator.scala:1336)
at
scala.collection.TraversableOnce$class.toBuffer(TraversableOnce.scala:302)
at scala.collection.AbstractIterator.toBuffer(Iterator.scala:1336)
at
scala.collection.TraversableOnce$class.toArray(TraversableOnce.scala:289)
at scala.collection.AbstractIterator.toArray(Iterator.scala:1336)
at
org.apache.atlas.query.Expressions$Expression$class.transformChildrenUp(Expressions.scala:240)
at
org.apache.atlas.query.Expressions$LogicalExpression.transformChildrenUp(Expressions.scala:684)
at
org.apache.atlas.query.Expressions$Expression$class.transformUp(Expressions.scala:199)
at
[jira] [Assigned] (ATLAS-1941) HBaseStoreManager is copying host list from root.storage.hostname to hbase.zookeeper.quorum, which has one incorrect zookeeper host
[
https://issues.apache.org/jira/browse/ATLAS-1941?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ayub Pathan reassigned ATLAS-1941:
--
Assignee: Ayub Pathan
> HBaseStoreManager is copying host list from root.storage.hostname to
> hbase.zookeeper.quorum, which has one incorrect zookeeper host
> ---
>
> Key: ATLAS-1941
> URL: https://issues.apache.org/jira/browse/ATLAS-1941
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Affects Versions: 0.9-incubating
>Reporter: Ayub Pathan
>Assignee: Ayub Pathan
>Priority: Minor
> Fix For: 0.9-incubating
>
>
> While Atlas startup, HBaseStoreManager is copying host list from
> root.storage.hostname to hbase.zookeeper.quorum, which has one incorrect
> zookeeper host as well.
> {noformat}
> 2017-07-11 12:13:51,341 INFO - [main:] ~ Not running setup per configuration
> atlas.server.run.setup.on.start. (SetupSteps:189)
> 2017-07-11 12:13:52,332 INFO - [main:] ~ JSR-330 'javax.inject.Inject'
> annotation found and supported for autowiring
> (AutowiredAnnotationBeanPostProcessor:155)
> 2017-07-11 12:13:53,038 INFO - [main:] ~ Instantiated HBase compatibility
> layer supporting runtime HBase version 1.1.2.2.6.2.0-71:
> com.thinkaurelius.titan.diskstorage.hbase.HBaseCompat1_1
> (HBaseCompatLoader:69)
> 2017-07-11 12:13:53,108 INFO - [main:] ~ Copied host list from
> root.storage.hostname to hbase.zookeeper.quorum:
> ctr-e133-1493418528701-179285-01-05.hwx.site,ctr-e133-1493418528701-179285-01-04.hwx.site,ctr-e133-1493418528701-179285-01-02.hwx.site
> (HBaseStoreManager:320)
> {noformat}
> In the above log, the correct zookeeper hosts are
> ctr-e133-1493418528701-179285-01-05.hwx.site,ctr-e133-1493418528701-179285-01-02.hwx.site.
> But, as we can see HBaseStoreManager is adding
> ctr-e133-1493418528701-179285-01-04.hwx.site as well which is incorrect.
> As an outcome of this, zookeeper connection to this host fails.
> {noformat}
> 2017-07-11 12:13:53,399 INFO -
> [main-SendThread(ctr-e133-1493418528701-179285-01-04.hwx.site:2181):] ~
> Opening socket connection to server
> ctr-e133-1493418528701-179285-01-04.hwx.site/172.27.9.2:2181. Will
> attempt to SASL-authenticate using Login Context section 'Client'
> (ClientCnxn:1019)
> 2017-07-11 12:13:53,412 WARN -
> [main-SendThread(ctr-e133-1493418528701-179285-01-04.hwx.site:2181):] ~
> Session 0x0 for server null, unexpected error, closing socket connection and
> attempting reconnect (ClientCnxn:1146)
> java.net.ConnectException: Connection refused
> at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
> at
> sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
> at
> org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:361)
> at
> org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1125)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Resolved] (ATLAS-1941) HBaseStoreManager is copying host list from root.storage.hostname to hbase.zookeeper.quorum, which has one incorrect zookeeper host
[
https://issues.apache.org/jira/browse/ATLAS-1941?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ayub Pathan resolved ATLAS-1941.
Resolution: Duplicate
This issue is addressed as part of AMBARI-21902
> HBaseStoreManager is copying host list from root.storage.hostname to
> hbase.zookeeper.quorum, which has one incorrect zookeeper host
> ---
>
> Key: ATLAS-1941
> URL: https://issues.apache.org/jira/browse/ATLAS-1941
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Affects Versions: 0.9-incubating
>Reporter: Ayub Pathan
>Priority: Minor
> Fix For: 0.9-incubating
>
>
> While Atlas startup, HBaseStoreManager is copying host list from
> root.storage.hostname to hbase.zookeeper.quorum, which has one incorrect
> zookeeper host as well.
> {noformat}
> 2017-07-11 12:13:51,341 INFO - [main:] ~ Not running setup per configuration
> atlas.server.run.setup.on.start. (SetupSteps:189)
> 2017-07-11 12:13:52,332 INFO - [main:] ~ JSR-330 'javax.inject.Inject'
> annotation found and supported for autowiring
> (AutowiredAnnotationBeanPostProcessor:155)
> 2017-07-11 12:13:53,038 INFO - [main:] ~ Instantiated HBase compatibility
> layer supporting runtime HBase version 1.1.2.2.6.2.0-71:
> com.thinkaurelius.titan.diskstorage.hbase.HBaseCompat1_1
> (HBaseCompatLoader:69)
> 2017-07-11 12:13:53,108 INFO - [main:] ~ Copied host list from
> root.storage.hostname to hbase.zookeeper.quorum:
> ctr-e133-1493418528701-179285-01-05.hwx.site,ctr-e133-1493418528701-179285-01-04.hwx.site,ctr-e133-1493418528701-179285-01-02.hwx.site
> (HBaseStoreManager:320)
> {noformat}
> In the above log, the correct zookeeper hosts are
> ctr-e133-1493418528701-179285-01-05.hwx.site,ctr-e133-1493418528701-179285-01-02.hwx.site.
> But, as we can see HBaseStoreManager is adding
> ctr-e133-1493418528701-179285-01-04.hwx.site as well which is incorrect.
> As an outcome of this, zookeeper connection to this host fails.
> {noformat}
> 2017-07-11 12:13:53,399 INFO -
> [main-SendThread(ctr-e133-1493418528701-179285-01-04.hwx.site:2181):] ~
> Opening socket connection to server
> ctr-e133-1493418528701-179285-01-04.hwx.site/172.27.9.2:2181. Will
> attempt to SASL-authenticate using Login Context section 'Client'
> (ClientCnxn:1019)
> 2017-07-11 12:13:53,412 WARN -
> [main-SendThread(ctr-e133-1493418528701-179285-01-04.hwx.site:2181):] ~
> Session 0x0 for server null, unexpected error, closing socket connection and
> attempting reconnect (ClientCnxn:1146)
> java.net.ConnectException: Connection refused
> at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
> at
> sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
> at
> org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:361)
> at
> org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1125)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Comment Edited] (ATLAS-2128) UI Regression : DSL Results pagination doesn't spawn more than 2 pages
[
https://issues.apache.org/jira/browse/ATLAS-2128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16161000#comment-16161000
]
Ayub Pathan edited comment on ATLAS-2128 at 9/11/17 10:02 AM:
--
Looks like, this issue is introduced as part of
https://github.com/apache/atlas/commit/d0beadf2e4a83bce17b8c42e7a584a035cf416c2
this commit.
To be more precise, this line might be causing the issue.:
https://github.com/apache/atlas/blob/master/dashboardv2/public/js/views/search/SearchResultLayoutView.js#L366
{code}
if (!model.entities && that.offset > that.limit) {
that.ui.nextData.attr('disabled', true);
that.offset = that.offset - that.limit;
that.hideLoader();
return;
}
{code}
CC [~kevalbhatt]
was (Author: ayubkhan):
This issue is introduced as part of
https://github.com/apache/atlas/commit/d0beadf2e4a83bce17b8c42e7a584a035cf416c2
this commit.
To be more precise, this line might be causing the issue.:
https://github.com/apache/atlas/blob/master/dashboardv2/public/js/views/search/SearchResultLayoutView.js#L366
{code}
if (!model.entities && that.offset > that.limit) {
that.ui.nextData.attr('disabled', true);
that.offset = that.offset - that.limit;
that.hideLoader();
return;
}
{code}
CC [~kevalbhatt]
> UI Regression : DSL Results pagination doesn't spawn more than 2 pages
> --
>
> Key: ATLAS-2128
> URL: https://issues.apache.org/jira/browse/ATLAS-2128
> Project: Atlas
> Issue Type: Bug
> Components: atlas-webui
>Affects Versions: 0.9-incubating, 0.8.1-incubating
>Reporter: Sharmadha Sainath
>Priority: Blocker
>
> 1. Created 300 hive_tables.
> 2. On Basic Search in UI , fired search typename= hive_table
> 3. hive_table results were fetched across 12 pages - 25 entities in each page
> - which is the correct behavior.
> 4.On Advanced Search in UI , fired search typename = hive_table
> * In the 1st page , 1- 25 results were shown.
> * On clicking next , in the 2nd page , 26 - 50 results were shown
> * On clicking next , Expected to land in 3rd page with 51 - 75
> results , but , 26-50 results were loaded again and the Next Button was
> disabled.
> Calls made to the Atlas backend seem to be correct. On clicking "Next" each
> time , correct set of results were fetched. For example , call was made to
> fetch entities from offset 50 , limit = 25 , but the UI still showed 26-50
> results and disabled "Next" button.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (ATLAS-2128) UI Regression : DSL Results pagination doesn't spawn more than 2 pages
[
https://issues.apache.org/jira/browse/ATLAS-2128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16161000#comment-16161000
]
Ayub Pathan commented on ATLAS-2128:
This issue is introduced as part of
https://github.com/apache/atlas/commit/d0beadf2e4a83bce17b8c42e7a584a035cf416c2
this commit.
To be more precise, this line might be causing the issue.:
https://github.com/apache/atlas/blob/master/dashboardv2/public/js/views/search/SearchResultLayoutView.js#L366
{code}
if (!model.entities && that.offset > that.limit) {
that.ui.nextData.attr('disabled', true);
that.offset = that.offset - that.limit;
that.hideLoader();
return;
}
{code}
CC [~kevalbhatt]
> UI Regression : DSL Results pagination doesn't spawn more than 2 pages
> --
>
> Key: ATLAS-2128
> URL: https://issues.apache.org/jira/browse/ATLAS-2128
> Project: Atlas
> Issue Type: Bug
> Components: atlas-webui
>Affects Versions: 0.9-incubating, 0.8.1-incubating
>Reporter: Sharmadha Sainath
>Priority: Blocker
>
> 1. Created 300 hive_tables.
> 2. On Basic Search in UI , fired search typename= hive_table
> 3. hive_table results were fetched across 12 pages - 25 entities in each page
> - which is the correct behavior.
> 4.On Advanced Search in UI , fired search typename = hive_table
> * In the 1st page , 1- 25 results were shown.
> * On clicking next , in the 2nd page , 26 - 50 results were shown
> * On clicking next , Expected to land in 3rd page with 51 - 75
> results , but , 26-50 results were loaded again and the Next Button was
> disabled.
> Calls made to the Atlas backend seem to be correct. On clicking "Next" each
> time , correct set of results were fetched. For example , call was made to
> fetch entities from offset 50 , limit = 25 , but the UI still showed 26-50
> results and disabled "Next" button.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (ATLAS-633) "kafka.common.NoReplicaOnlineException" exception on atlas startup
[
https://issues.apache.org/jira/browse/ATLAS-633?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16128277#comment-16128277
]
Ayub Pathan commented on ATLAS-633:
---
[~bill] This issue is seen in embedded mode of Atlas and I believe this is a
limitation from ATlas-Kafka integration. Since I am using non-embedded mode of
Atlas, I didn't pursue on this route.
> "kafka.common.NoReplicaOnlineException" exception on atlas startup
> --
>
> Key: ATLAS-633
> URL: https://issues.apache.org/jira/browse/ATLAS-633
> Project: Atlas
> Issue Type: Bug
>Affects Versions: trunk
>Reporter: Ayub Pathan
>
> "kafka.common.NoReplicaOnlineException" exception on atlas startup
> {noformat}
> 2016-04-05 11:01:54,811 INFO - [main:] ~ [Socket Server on Broker 1],
> Started 1 acceptor threads (SocketServer:68)
> 2016-04-05 11:01:54,828 INFO - [ExpirationReaper-1:] ~ [ExpirationReaper-1],
> Starting (DelayedOperationPurgatory$ExpiredOperationReaper:68)
> 2016-04-05 11:01:54,828 INFO - [ExpirationReaper-1:] ~ [ExpirationReaper-1],
> Starting (DelayedOperationPurgatory$ExpiredOperationReaper:68)
> 2016-04-05 11:01:54,852 INFO - [main:] ~ [Controller 1]: Controller starting
> up (KafkaController:68)
> 2016-04-05 11:01:54,862 INFO - [main:] ~ Creating /controller (is it secure?
> false) (ZKCheckedEphemeral:68)
> 2016-04-05 11:01:54,867 INFO - [main:] ~ Result of znode creation is: OK
> (ZKCheckedEphemeral:68)
> 2016-04-05 11:01:54,868 INFO - [main:] ~ 1 successfully elected as leader
> (ZookeeperLeaderElector:68)
> 2016-04-05 11:01:54,868 INFO - [main:] ~ [Controller 1]: Broker 1 starting
> become controller state transition (KafkaController:68)
> 2016-04-05 11:01:54,870 INFO - [main:] ~ [Controller 1]: Initialized
> controller epoch to 1 and zk version 0 (KafkaController:68)
> 2016-04-05 11:01:54,872 INFO - [main:] ~ [Controller 1]: Controller 1
> incremented epoch to 2 (KafkaController:68)
> 2016-04-05 11:01:54,939 INFO - [main:] ~ [Controller 1]: Partitions
> undergoing preferred replica election: (KafkaController:68)
> 2016-04-05 11:01:54,939 INFO - [main:] ~ [Controller 1]: Partitions that
> completed preferred replica election: (KafkaController:68)
> 2016-04-05 11:01:54,940 INFO - [main:] ~ [Controller 1]: Resuming preferred
> replica election for partitions: (KafkaController:68)
> 2016-04-05 11:01:54,943 INFO - [main:] ~ [Controller 1]: Partitions being
> reassigned: Map() (KafkaController:68)
> 2016-04-05 11:01:54,944 INFO - [main:] ~ [Controller 1]: Partitions already
> reassigned: List() (KafkaController:68)
> 2016-04-05 11:01:54,945 INFO - [main:] ~ [Controller 1]: Resuming
> reassignment of partitions: Map() (KafkaController:68)
> 2016-04-05 11:01:54,949 INFO - [main:] ~ [Controller 1]: List of topics to
> be deleted: (KafkaController:68)
> 2016-04-05 11:01:54,949 INFO - [main:] ~ [Controller 1]: List of topics
> ineligible for deletion: ATLAS_HOOK,ATLAS_ENTITIES (KafkaController:68)
> 2016-04-05 11:01:54,951 INFO - [main:] ~ [Controller 1]: Currently active
> brokers in the cluster: Set() (KafkaController:68)
> 2016-04-05 11:01:54,952 INFO - [main:] ~ [Controller 1]: Currently shutting
> brokers in the cluster: Set() (KafkaController:68)
> 2016-04-05 11:01:54,952 INFO - [main:] ~ [Controller 1]: Current list of
> topics in the cluster: Set(ATLAS_ENTITIES, ATLAS_HOOK) (KafkaController:68)
> 2016-04-05 11:01:54,956 INFO - [main:] ~ [Replica state machine on
> controller 1]: Started replica state machine with initial state ->
> Map([Topic=ATLAS_ENTITIES,Partition=0,Replica=1] ->
> ReplicaDeletionIneligible, [Topic=ATLAS_HOOK,Partition=0,Replica=1] ->
> ReplicaDeletionIneligible) (ReplicaStateMachine:68)
> 2016-04-05 11:01:54,977 ERROR - [main:] ~ Controller 1 epoch 2 initiated
> state change for partition [ATLAS_HOOK,0] from OfflinePartition to
> OnlinePartition failed (logger:103)
> kafka.common.NoReplicaOnlineException: No replica for partition
> [ATLAS_HOOK,0] is alive. Live brokers are: [Set()], Assigned replicas are:
> [List(1)]
> at
> kafka.controller.OfflinePartitionLeaderSelector.selectLeader(PartitionLeaderSelector.scala:75)
> at
> kafka.controller.PartitionStateMachine.electLeaderForPartition(PartitionStateMachine.scala:345)
> at
> kafka.controller.PartitionStateMachine.kafka$controller$PartitionStateMachine$$handleStateChange(PartitionStateMachine.scala:205)
> at
> kafka.controller.PartitionStateMachine$$anonfun$triggerOnlinePartitionStateChange$3.apply(PartitionStateMachine.scala:120)
> at
> kafka.controller.PartitionStateMachine$$anonfun$triggerOnlinePartitionStateChange$3.apply(PartitionStateMachine.scala:117)
> at
> scala.collection.TraversableLike$WithFilter$$anonfun$foreach$1.apply(TraversableLike.scala:772)
>
[jira] [Commented] (ATLAS-2022) Regression : Empty results fetched from GET Basic search query
[
https://issues.apache.org/jira/browse/ATLAS-2022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16114315#comment-16114315
]
Ayub Pathan commented on ATLAS-2022:
+1. looks good to me
> Regression : Empty results fetched from GET Basic search query
> --
>
> Key: ATLAS-2022
> URL: https://issues.apache.org/jira/browse/ATLAS-2022
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Affects Versions: 0.9-incubating
>Reporter: Sharmadha Sainath
>Assignee: Sharmadha Sainath
>Priority: Blocker
> Attachments: ATLAS-2022.patch
>
>
> Basic search query fired as a POST request with attribute , tag filters
> fetches correct results.
> But Basic query fired as a GET request with query params encoded in the URL ,
> returns empty results.
> For example : Basic query : typeName = "hive_table" query = hive_table in Atlas"
> http://localhost:21000/api/atlas/v2/search/basic?typeName=hive_table=employee
> returns
> {code}
> {
> queryType: "BASIC",
> searchParameters: {
> query: "employee",
> typeName: "hive_table",
> excludeDeletedEntities: false,
> limit: 0,
> offset: 0
> },
> queryText: "employee"
> }
> {code}
> Few commits back , following was the response :
> {code}
> {
> queryType: "BASIC",
> type: "hive_table",
> entities: [
> {
> typeName: "hive_table",
> attributes: {
> owner: "admin",
> qualifiedName: "default.employee@cl1",
> name: "employee",
> description: null
> },
> guid: "253aa208-0415-4e86-8611-3858fad78ede",
> status: "ACTIVE",
> displayText: "employee",
> classificationNames: [ ]
> }
> ]
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Assigned] (ATLAS-2022) Regression : Empty results fetched from GET Basic search query
[
https://issues.apache.org/jira/browse/ATLAS-2022?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ayub Pathan reassigned ATLAS-2022:
--
Assignee: Sharmadha Sainath
> Regression : Empty results fetched from GET Basic search query
> --
>
> Key: ATLAS-2022
> URL: https://issues.apache.org/jira/browse/ATLAS-2022
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Affects Versions: 0.9-incubating
>Reporter: Sharmadha Sainath
>Assignee: Sharmadha Sainath
>Priority: Blocker
> Attachments: ATLAS-2022.patch
>
>
> Basic search query fired as a POST request with attribute , tag filters
> fetches correct results.
> But Basic query fired as a GET request with query params encoded in the URL ,
> returns empty results.
> For example : Basic query : typeName = "hive_table" query = hive_table in Atlas"
> http://localhost:21000/api/atlas/v2/search/basic?typeName=hive_table=employee
> returns
> {code}
> {
> queryType: "BASIC",
> searchParameters: {
> query: "employee",
> typeName: "hive_table",
> excludeDeletedEntities: false,
> limit: 0,
> offset: 0
> },
> queryText: "employee"
> }
> {code}
> Few commits back , following was the response :
> {code}
> {
> queryType: "BASIC",
> type: "hive_table",
> entities: [
> {
> typeName: "hive_table",
> attributes: {
> owner: "admin",
> qualifiedName: "default.employee@cl1",
> name: "employee",
> description: null
> },
> guid: "253aa208-0415-4e86-8611-3858fad78ede",
> status: "ACTIVE",
> displayText: "employee",
> classificationNames: [ ]
> }
> ]
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (ATLAS-1951) Regression: Any REST API request without user credentials results in 302 redirect to login.jsp. Actually, the correct response should be 401.
Ayub Pathan created ATLAS-1951:
--
Summary: Regression: Any REST API request without user credentials
results in 302 redirect to login.jsp. Actually, the correct response should be
401.
Key: ATLAS-1951
URL: https://issues.apache.org/jira/browse/ATLAS-1951
Project: Atlas
Issue Type: Bug
Components: atlas-core
Affects Versions: 0.9-incubating
Reporter: Ayub Pathan
Priority: Critical
Fix For: 0.9-incubating
Regression: Any REST API request to atlas without user credentials results in
302 redirect( login.jsp), but the actual response code should be 401
unauthorized.
This issue could have been introduced as part of new spring changes.
For example:
{noformat}
curl -v -X GET "http://ctr-e133-1493418528701-181199-01-02.hwx.site:21000;
* Rebuilt URL to: http://ctr-e133-1493418528701-181199-01-02.hwx.site:21000/
* Trying 172.27.56.2...
* Connected to ctr-e133-1493418528701-181199-01-02.hwx.site (172.27.56.2)
port 21000 (#0)
> GET / HTTP/1.1
> Host: ctr-e133-1493418528701-181199-01-02.hwx.site:21000
> User-Agent: curl/7.43.0
> Accept: */*
>
< HTTP/1.1 302 Found
< Date: Fri, 14 Jul 2017 11:16:42 GMT
< Set-Cookie: ATLASSESSIONID=1i0rxnm66dd3h17xyhvstk0vck;Path=/;HttpOnly
< Expires: Thu, 01 Jan 1970 00:00:00 GMT
< X-Frame-Options: DENY
< Location:
http://ctr-e133-1493418528701-181199-01-02.hwx.site:21000/login.jsp
< Content-Length: 0
< Server: Jetty(9.2.12.v20150709)
<
* Connection #0 to host ctr-e133-1493418528701-181199-01-02.hwx.site left
intact
{noformat}
CC [~skoneru]
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (ATLAS-1946) LDAPS(LDAP over SSL) support in Atlas
[ https://issues.apache.org/jira/browse/ATLAS-1946?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayub Pathan updated ATLAS-1946: --- Issue Type: Improvement (was: Bug) > LDAPS(LDAP over SSL) support in Atlas > - > > Key: ATLAS-1946 > URL: https://issues.apache.org/jira/browse/ATLAS-1946 > Project: Atlas > Issue Type: Improvement > Components: atlas-intg >Affects Versions: 0.9-incubating >Reporter: Ayub Pathan > > Add support for LDAPS in Atlas. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (ATLAS-1946) LDAPS(LDAP over SSL) support in Atlas
Ayub Pathan created ATLAS-1946: -- Summary: LDAPS(LDAP over SSL) support in Atlas Key: ATLAS-1946 URL: https://issues.apache.org/jira/browse/ATLAS-1946 Project: Atlas Issue Type: Bug Components: atlas-intg Affects Versions: 0.9-incubating Reporter: Ayub Pathan Add support for LDAPS in Atlas. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (ATLAS-1944) NotificationHookConsumer throws exception while shutting down the consumer thread
Ayub Pathan created ATLAS-1944:
--
Summary: NotificationHookConsumer throws exception while shutting
down the consumer thread
Key: ATLAS-1944
URL: https://issues.apache.org/jira/browse/ATLAS-1944
Project: Atlas
Issue Type: Bug
Components: atlas-core
Affects Versions: 0.9-incubating
Reporter: Ayub Pathan
Priority: Critical
Fix For: 0.9-incubating
NotificationHookConsumer throws below exception while shutting down the
consumer thread, this issue is possibly after this commit.
https://github.com/apache/incubator-atlas/commit/0e7f8ea4603c858cc295259bbd1a22314b732f62
CC [~nrodrigues]
{noformat}
2017-07-12 01:26:09,743 WARN - [pool-1-thread-1:] ~ Error stopping service
org.apache.atlas.notification.NotificationHookConsumer (Services:69)
java.util.ConcurrentModificationException: KafkaConsumer is not safe for
multi-threaded access
at
org.apache.kafka.clients.consumer.KafkaConsumer.acquire(KafkaConsumer.java:1557)
at
org.apache.kafka.clients.consumer.KafkaConsumer.close(KafkaConsumer.java:1467)
at
org.apache.atlas.kafka.AtlasKafkaConsumer.close(AtlasKafkaConsumer.java:88)
at
org.apache.atlas.notification.NotificationHookConsumer$HookConsumer.stop(NotificationHookConsumer.java:384)
at
org.apache.atlas.notification.NotificationHookConsumer.stopConsumerThreads(NotificationHookConsumer.java:172)
at
org.apache.atlas.notification.NotificationHookConsumer.stop(NotificationHookConsumer.java:155)
at org.apache.atlas.service.Services.stop(Services.java:67)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor$LifecycleElement.invoke(InitDestroyAnnotationBeanPostProcessor.java:366)
at
org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor$LifecycleMetadata.invokeDestroyMethods(InitDestroyAnnotationBeanPostProcessor.java:325)
at
org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor.postProcessBeforeDestruction(InitDestroyAnnotationBeanPostProcessor.java:154)
at
org.springframework.beans.factory.support.DisposableBeanAdapter.destroy(DisposableBeanAdapter.java:253)
at
org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.destroyBean(DefaultSingletonBeanRegistry.java:578)
at
org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.destroySingleton(DefaultSingletonBeanRegistry.java:554)
at
org.springframework.beans.factory.support.DefaultListableBeanFactory.destroySingleton(DefaultListableBeanFactory.java:961)
at
org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.destroySingletons(DefaultSingletonBeanRegistry.java:523)
at
org.springframework.beans.factory.support.DefaultListableBeanFactory.destroySingletons(DefaultListableBeanFactory.java:968)
at
org.springframework.context.support.AbstractApplicationContext.destroyBeans(AbstractApplicationContext.java:1033)
at
org.springframework.context.support.AbstractApplicationContext.doClose(AbstractApplicationContext.java:1009)
at
org.springframework.context.support.AbstractApplicationContext.close(AbstractApplicationContext.java:961)
at
org.springframework.web.context.ContextLoader.closeWebApplicationContext(ContextLoader.java:583)
at
org.springframework.web.context.ContextLoaderListener.contextDestroyed(ContextLoaderListener.java:116)
at
org.eclipse.jetty.server.handler.ContextHandler.callContextDestroyed(ContextHandler.java:808)
at
org.eclipse.jetty.servlet.ServletContextHandler.callContextDestroyed(ServletContextHandler.java:457)
at
org.eclipse.jetty.server.handler.ContextHandler.doStop(ContextHandler.java:842)
at
org.eclipse.jetty.servlet.ServletContextHandler.doStop(ServletContextHandler.java:215)
at org.eclipse.jetty.webapp.WebAppContext.doStop(WebAppContext.java:529)
at
org.eclipse.jetty.util.component.AbstractLifeCycle.stop(AbstractLifeCycle.java:89)
at
org.eclipse.jetty.util.component.ContainerLifeCycle.stop(ContainerLifeCycle.java:143)
at
org.eclipse.jetty.util.component.ContainerLifeCycle.doStop(ContainerLifeCycle.java:162)
at
org.eclipse.jetty.server.handler.AbstractHandler.doStop(AbstractHandler.java:73)
at org.eclipse.jetty.server.Server.doStop(Server.java:456)
at
org.eclipse.jetty.util.component.AbstractLifeCycle.stop(AbstractLifeCycle.java:89)
{noformat}
--
This message was
[jira] [Commented] (ATLAS-1874) V2 DSL search query does not support for count() but v1 do.
[
https://issues.apache.org/jira/browse/ATLAS-1874?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16083284#comment-16083284
]
Ayub Pathan commented on ATLAS-1874:
[~grahamwallis] DSL query works fine for the above query you mentioned, it's
only that UI does not display anything, because query is requesting for
specific attribute (in this case, owner) of that entity. Please check the below
response for same query.
Response for query: "`hive_table` where db.name=\"default\" select owner"
{noformat}
{
"queryType":"DSL",
"queryText":"`hive_table` where db.name=\"default\" select owner",
"attributes":{
"name":[
"owner"
],
"values":[
[
"hive"
]
]
}
}
{noformat}
> V2 DSL search query does not support for count() but v1 do.
> ---
>
> Key: ATLAS-1874
> URL: https://issues.apache.org/jira/browse/ATLAS-1874
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Affects Versions: 0.8.1-incubating
>Reporter: Ayub Pathan
> Fix For: 0.9-incubating, 0.8.1-incubating
>
>
> v1 DSL query supports count()
> {noformat}
> curl -u admin:admin
> 'http://ctr-e133-1493418528701-113468-01-02.hwx.site:21000/api/atlas/discovery/search/dsl?query=hive_table%20where%20db.name%3D%22default%22%20select%20count()%20as%20%27count%27'
> | python -m json.tool
> % Total% Received % Xferd Average Speed TimeTime Time
> Current
> Dload Upload Total SpentLeft Speed
> 100 5370 5370 0339 0 --:--:-- 0:00:01 --:--:-- 339
> {
> "count": 1,
> "dataType": {
> "attributeDefinitions": [
> {
> "dataTypeName": "long",
> "isComposite": false,
> "isIndexable": false,
> "isUnique": false,
> "multiplicity": {
> "isUnique": false,
> "lower": 0,
> "upper": 1
> },
> "name": "count",
> "reverseAttributeName": null
> }
> ],
> "typeDescription": null,
> "typeName": "__tempQueryResultStruct286",
> "typeVersion": "1.0"
> },
> "query": "hive_table where db.name=\"default\" select count() as 'count'",
> "queryType": "dsl",
> "requestId": "pool-2-thread-9 - 47389e3f-bbf1-4209-8e50-8a3235a7e5a9",
> "results": [
> {
> "$typeName$": "__tempQueryResultStruct286",
> "count": 68
> }
> ]
> }
> {noformat}
> v2 DSL search query does not
> {noformat}
> curl -u admin:admin
> 'http://ctr-e133-1493418528701-113468-01-02.hwx.site:21000/api/atlas/v2/search/dsl?limit=25=true=where+db.name%3D%22default%22+select+count()+as+%27count%27=hive_table&_=1497434602692'
> | python -m json.tool
> {
> "queryText": "`hive_table` where db.name=\"default\" select count() as
> 'count'",
> "queryType": "DSL"
> }
> {noformat}
> *From the initial analysis, it seems like, V2 API response does not have the
> count attribute which is cause for this failure.* might want to consider
> adding the count in V2?
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (ATLAS-1941) HBaseStoreManager is overwriting the hbase.zookeeper.quorum value to incorrect one
[
https://issues.apache.org/jira/browse/ATLAS-1941?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ayub Pathan updated ATLAS-1941:
---
Priority: Minor (was: Major)
> HBaseStoreManager is overwriting the hbase.zookeeper.quorum value to
> incorrect one
> --
>
> Key: ATLAS-1941
> URL: https://issues.apache.org/jira/browse/ATLAS-1941
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Affects Versions: 0.9-incubating
>Reporter: Ayub Pathan
>Priority: Minor
> Fix For: 0.9-incubating
>
>
> While Atlas startup, HBaseStoreManager is copying host list from
> root.storage.hostname to hbase.zookeeper.quorum, which has one incorrect
> zookeeper host as well.
> {noformat}
> 2017-07-11 12:13:51,341 INFO - [main:] ~ Not running setup per configuration
> atlas.server.run.setup.on.start. (SetupSteps:189)
> 2017-07-11 12:13:52,332 INFO - [main:] ~ JSR-330 'javax.inject.Inject'
> annotation found and supported for autowiring
> (AutowiredAnnotationBeanPostProcessor:155)
> 2017-07-11 12:13:53,038 INFO - [main:] ~ Instantiated HBase compatibility
> layer supporting runtime HBase version 1.1.2.2.6.2.0-71:
> com.thinkaurelius.titan.diskstorage.hbase.HBaseCompat1_1
> (HBaseCompatLoader:69)
> 2017-07-11 12:13:53,108 INFO - [main:] ~ Copied host list from
> root.storage.hostname to hbase.zookeeper.quorum:
> ctr-e133-1493418528701-179285-01-05.hwx.site,ctr-e133-1493418528701-179285-01-04.hwx.site,ctr-e133-1493418528701-179285-01-02.hwx.site
> (HBaseStoreManager:320)
> {noformat}
> In the above log, the correct zookeeper hosts are
> ctr-e133-1493418528701-179285-01-05.hwx.site,ctr-e133-1493418528701-179285-01-02.hwx.site.
> But, as we can see HBaseStoreManager is adding
> ctr-e133-1493418528701-179285-01-04.hwx.site as well which is incorrect.
> As an outcome of this, zookeeper connection to this host fails.
> {noformat}
> 2017-07-11 12:13:53,399 INFO -
> [main-SendThread(ctr-e133-1493418528701-179285-01-04.hwx.site:2181):] ~
> Opening socket connection to server
> ctr-e133-1493418528701-179285-01-04.hwx.site/172.27.9.2:2181. Will
> attempt to SASL-authenticate using Login Context section 'Client'
> (ClientCnxn:1019)
> 2017-07-11 12:13:53,412 WARN -
> [main-SendThread(ctr-e133-1493418528701-179285-01-04.hwx.site:2181):] ~
> Session 0x0 for server null, unexpected error, closing socket connection and
> attempting reconnect (ClientCnxn:1146)
> java.net.ConnectException: Connection refused
> at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
> at
> sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
> at
> org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:361)
> at
> org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1125)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (ATLAS-1941) HBaseStoreManager is copying host list from root.storage.hostname to hbase.zookeeper.quorum, which has one incorrect zookeeper host
[
https://issues.apache.org/jira/browse/ATLAS-1941?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ayub Pathan updated ATLAS-1941:
---
Summary: HBaseStoreManager is copying host list from root.storage.hostname
to hbase.zookeeper.quorum, which has one incorrect zookeeper host (was:
HBaseStoreManager is overwriting the hbase.zookeeper.quorum value to incorrect
one)
> HBaseStoreManager is copying host list from root.storage.hostname to
> hbase.zookeeper.quorum, which has one incorrect zookeeper host
> ---
>
> Key: ATLAS-1941
> URL: https://issues.apache.org/jira/browse/ATLAS-1941
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Affects Versions: 0.9-incubating
>Reporter: Ayub Pathan
>Priority: Minor
> Fix For: 0.9-incubating
>
>
> While Atlas startup, HBaseStoreManager is copying host list from
> root.storage.hostname to hbase.zookeeper.quorum, which has one incorrect
> zookeeper host as well.
> {noformat}
> 2017-07-11 12:13:51,341 INFO - [main:] ~ Not running setup per configuration
> atlas.server.run.setup.on.start. (SetupSteps:189)
> 2017-07-11 12:13:52,332 INFO - [main:] ~ JSR-330 'javax.inject.Inject'
> annotation found and supported for autowiring
> (AutowiredAnnotationBeanPostProcessor:155)
> 2017-07-11 12:13:53,038 INFO - [main:] ~ Instantiated HBase compatibility
> layer supporting runtime HBase version 1.1.2.2.6.2.0-71:
> com.thinkaurelius.titan.diskstorage.hbase.HBaseCompat1_1
> (HBaseCompatLoader:69)
> 2017-07-11 12:13:53,108 INFO - [main:] ~ Copied host list from
> root.storage.hostname to hbase.zookeeper.quorum:
> ctr-e133-1493418528701-179285-01-05.hwx.site,ctr-e133-1493418528701-179285-01-04.hwx.site,ctr-e133-1493418528701-179285-01-02.hwx.site
> (HBaseStoreManager:320)
> {noformat}
> In the above log, the correct zookeeper hosts are
> ctr-e133-1493418528701-179285-01-05.hwx.site,ctr-e133-1493418528701-179285-01-02.hwx.site.
> But, as we can see HBaseStoreManager is adding
> ctr-e133-1493418528701-179285-01-04.hwx.site as well which is incorrect.
> As an outcome of this, zookeeper connection to this host fails.
> {noformat}
> 2017-07-11 12:13:53,399 INFO -
> [main-SendThread(ctr-e133-1493418528701-179285-01-04.hwx.site:2181):] ~
> Opening socket connection to server
> ctr-e133-1493418528701-179285-01-04.hwx.site/172.27.9.2:2181. Will
> attempt to SASL-authenticate using Login Context section 'Client'
> (ClientCnxn:1019)
> 2017-07-11 12:13:53,412 WARN -
> [main-SendThread(ctr-e133-1493418528701-179285-01-04.hwx.site:2181):] ~
> Session 0x0 for server null, unexpected error, closing socket connection and
> attempting reconnect (ClientCnxn:1146)
> java.net.ConnectException: Connection refused
> at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
> at
> sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
> at
> org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:361)
> at
> org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1125)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (ATLAS-1923) Tag creation: hive_principal_type and file_action are shown in the drop down while adding attributes to a tag.
Ayub Pathan created ATLAS-1923: -- Summary: Tag creation: hive_principal_type and file_action are shown in the drop down while adding attributes to a tag. Key: ATLAS-1923 URL: https://issues.apache.org/jira/browse/ATLAS-1923 Project: Atlas Issue Type: Bug Components: atlas-webui Affects Versions: 0.9-incubating Reporter: Ayub Pathan Fix For: 0.9-incubating Attachments: Screen Shot 2017-07-05 at 10.38.50 AM.png Tag creation: hive_principal_type and file_action are shown in the drop down while adding attributes to a tag. Attaching screenshot for reference. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (ATLAS-1919) Export of hive_table with fetchType "connected" fails with duplicate entry. Seems like, same edge(asset) is traversed twice.
[
https://issues.apache.org/jira/browse/ATLAS-1919?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ayub Pathan updated ATLAS-1919:
---
Description:
Export of hive_table with fetchType "connected" fails with duplicate entry.
Seems like, same edge(asset) is traversed twice. Possibly a loop condition.
Export request payload
{noformat}
{
"itemsToExport": [
{
"typeName": "hive_table",
"uniqueAttributes": {
"qualifiedName": "any_random_table@cluster"
}
}
],
"options": {
"fetchType" : "connected"
}
}
{noformat}
stacktrace
{noformat}
2017-07-04 08:44:19,159 INFO - [pool-2-thread-9 -
8bc8d958-e650-4351-acf0-24c28d15627a:] ~ export(item=AtlasObjectId{guid='null',
typeName='hive_table', uniqueAttributes={qualifiedName:db1.table1@cl1}};
matchType=null, fetchType=CONNECTED): found 1 entities (ExportService:280)
2017-07-04 08:44:19,794 ERROR - [pool-2-thread-9 -
8bc8d958-e650-4351-acf0-24c28d15627a:] ~ Fetching entity failed for:
AtlasObjectId{guid='null', typeName='hive_table',
uniqueAttributes={qualifiedName:db1.table1@cl1}} (ExportService:207)
org.apache.atlas.exception.AtlasBaseException: Error writing file
75a90231-3eca-4ba7-9e21-93e7ec2b0df2.
at org.apache.atlas.repository.impexp.ZipSink.saveToZip(ZipSink.java:91)
at org.apache.atlas.repository.impexp.ZipSink.add(ZipSink.java:50)
at
org.apache.atlas.repository.impexp.ExportService.addEntity(ExportService.java:433)
at
org.apache.atlas.repository.impexp.ExportService.processEntity(ExportService.java:297)
at
org.apache.atlas.repository.impexp.ExportService.processObjectId(ExportService.java:198)
at
org.apache.atlas.repository.impexp.ExportService.processItems(ExportService.java:162)
at
org.apache.atlas.repository.impexp.ExportService.run(ExportService.java:95)
at
org.apache.atlas.web.resources.AdminResource.export(AdminResource.java:325)
at sun.reflect.GeneratedMethodAccessor238.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60)
at
com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$ResponseOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:205)
at
com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75)
at
com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:302)
at
com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
at
com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:108)
at
com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
at
com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:84)
at
com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1542)
at
com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1473)
at
com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1419)
at
com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1409)
at
com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:409)
at
com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:558)
at
com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:733)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
at
com.google.inject.servlet.ServletDefinition.doServiceImpl(ServletDefinition.java:286)
at
com.google.inject.servlet.ServletDefinition.doService(ServletDefinition.java:276)
at
com.google.inject.servlet.ServletDefinition.service(ServletDefinition.java:181)
at
com.google.inject.servlet.ManagedServletPipeline.service(ManagedServletPipeline.java:91)
at
com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:85)
at
org.apache.atlas.web.filters.AuditFilter.doFilter(AuditFilter.java:78)
at
com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82)
at
org.apache.atlas.web.filters.StaleTransactionCleanupFilter.doFilter(StaleTransactionCleanupFilter.java:56)
at
com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82)
at
[jira] [Created] (ATLAS-1919) Export of hive_table with fetchType "connected" fails with duplicate entry. Seems like, same edge(asset) is traversed twice.
Ayub Pathan created ATLAS-1919:
--
Summary: Export of hive_table with fetchType "connected" fails
with duplicate entry. Seems like, same edge(asset) is traversed twice.
Key: ATLAS-1919
URL: https://issues.apache.org/jira/browse/ATLAS-1919
Project: Atlas
Issue Type: Bug
Components: atlas-core
Affects Versions: 0.9-incubating
Reporter: Ayub Pathan
Priority: Critical
Fix For: 0.9-incubating
Export of hive_table with fetchType "connected" fails with duplicate entry.
Seems like, same edge(asset) is traversed twice. Possibly a loop condition.
Export request payload
{noformat}
{
"itemsToExport": [
{
"typeName": "hive_table",
"uniqueAttributes": {
"qualifiedName": "any_random_table@cluster"
}
}
],
"options": {
"fetchType" : "connected"
}
}
{noformat}
stacktrace
{noformat}
2017-07-04 08:44:19,159 INFO - [pool-2-thread-9 -
8bc8d958-e650-4351-acf0-24c28d15627a:] ~ export(item=AtlasObjectId{guid='null',
typeName='hive_table', uniqueAttributes={qualifiedName:db1.table1@cl1}};
matchType=null, fetchType=CONNECTED): found 1 entities (ExportService:280)
2017-07-04 08:44:19,794 ERROR - [pool-2-thread-9 -
8bc8d958-e650-4351-acf0-24c28d15627a:] ~ Fetching entity failed for:
AtlasObjectId{guid='null', typeName='hive_table',
uniqueAttributes={qualifiedName:db1.table1@cl1}} (ExportService:207)
org.apache.atlas.exception.AtlasBaseException: Error writing file
75a90231-3eca-4ba7-9e21-93e7ec2b0df2.
at org.apache.atlas.repository.impexp.ZipSink.saveToZip(ZipSink.java:91)
at org.apache.atlas.repository.impexp.ZipSink.add(ZipSink.java:50)
at
org.apache.atlas.repository.impexp.ExportService.addEntity(ExportService.java:433)
at
org.apache.atlas.repository.impexp.ExportService.processEntity(ExportService.java:297)
at
org.apache.atlas.repository.impexp.ExportService.processObjectId(ExportService.java:198)
at
org.apache.atlas.repository.impexp.ExportService.processItems(ExportService.java:162)
at
org.apache.atlas.repository.impexp.ExportService.run(ExportService.java:95)
at
org.apache.atlas.web.resources.AdminResource.export(AdminResource.java:325)
at sun.reflect.GeneratedMethodAccessor238.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60)
at
com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$ResponseOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:205)
at
com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75)
at
com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:302)
at
com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
at
com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:108)
at
com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
at
com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:84)
at
com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1542)
at
com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1473)
at
com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1419)
at
com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1409)
at
com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:409)
at
com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:558)
at
com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:733)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
at
com.google.inject.servlet.ServletDefinition.doServiceImpl(ServletDefinition.java:286)
at
com.google.inject.servlet.ServletDefinition.doService(ServletDefinition.java:276)
at
com.google.inject.servlet.ServletDefinition.service(ServletDefinition.java:181)
at
com.google.inject.servlet.ManagedServletPipeline.service(ManagedServletPipeline.java:91)
at
com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:85)
at
org.apache.atlas.web.filters.AuditFilter.doFilter(AuditFilter.java:78)
at
