[jira] [Created] (ATLAS-1919) Export of hive_table with fetchType "connected" fails with duplicate entry. Seems like, same edge(asset) is traversed twice.
Ayub Pathan created ATLAS-1919: -- Summary: Export of hive_table with fetchType "connected" fails with duplicate entry. Seems like, same edge(asset) is traversed twice. Key: ATLAS-1919 URL: https://issues.apache.org/jira/browse/ATLAS-1919 Project: Atlas Issue Type: Bug Components: atlas-core Affects Versions: 0.9-incubating Reporter: Ayub Pathan Priority: Critical Fix For: 0.9-incubating Export of hive_table with fetchType "connected" fails with duplicate entry. Seems like, same edge(asset) is traversed twice. Possibly a loop condition. Export request payload {noformat} { "itemsToExport": [ { "typeName": "hive_table", "uniqueAttributes": { "qualifiedName": "any_random_table@cluster" } } ], "options": { "fetchType" : "connected" } } {noformat} stacktrace {noformat} 2017-07-04 08:44:19,159 INFO - [pool-2-thread-9 - 8bc8d958-e650-4351-acf0-24c28d15627a:] ~ export(item=AtlasObjectId{guid='null', typeName='hive_table', uniqueAttributes={qualifiedName:db1.table1@cl1}}; matchType=null, fetchType=CONNECTED): found 1 entities (ExportService:280) 2017-07-04 08:44:19,794 ERROR - [pool-2-thread-9 - 8bc8d958-e650-4351-acf0-24c28d15627a:] ~ Fetching entity failed for: AtlasObjectId{guid='null', typeName='hive_table', uniqueAttributes={qualifiedName:db1.table1@cl1}} (ExportService:207) org.apache.atlas.exception.AtlasBaseException: Error writing file 75a90231-3eca-4ba7-9e21-93e7ec2b0df2. at org.apache.atlas.repository.impexp.ZipSink.saveToZip(ZipSink.java:91) at org.apache.atlas.repository.impexp.ZipSink.add(ZipSink.java:50) at org.apache.atlas.repository.impexp.ExportService.addEntity(ExportService.java:433) at org.apache.atlas.repository.impexp.ExportService.processEntity(ExportService.java:297) at org.apache.atlas.repository.impexp.ExportService.processObjectId(ExportService.java:198) at org.apache.atlas.repository.impexp.ExportService.processItems(ExportService.java:162) at org.apache.atlas.repository.impexp.ExportService.run(ExportService.java:95) at org.apache.atlas.web.resources.AdminResource.export(AdminResource.java:325) at sun.reflect.GeneratedMethodAccessor238.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60) at com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$ResponseOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:205) at com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75) at com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:302) at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147) at com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:108) at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147) at com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:84) at com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1542) at com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1473) at com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1419) at com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1409) at com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:409) at com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:558) at com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:733) at javax.servlet.http.HttpServlet.service(HttpServlet.java:790) at com.google.inject.servlet.ServletDefinition.doServiceImpl(ServletDefinition.java:286) at com.google.inject.servlet.ServletDefinition.doService(ServletDefinition.java:276) at com.google.inject.servlet.ServletDefinition.service(ServletDefinition.java:181) at com.google.inject.servlet.ManagedServletPipeline.service(ManagedServletPipeline.java:91) at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:85) at org.apache.atlas.web.filters.AuditFilter.doFilter(AuditFilter.java:78) at com.google.inject.servlet.Filt
[jira] [Updated] (ATLAS-1919) Export of hive_table with fetchType "connected" fails with duplicate entry. Seems like, same edge(asset) is traversed twice.
[ https://issues.apache.org/jira/browse/ATLAS-1919?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayub Pathan updated ATLAS-1919: --- Description: Export of hive_table with fetchType "connected" fails with duplicate entry. Seems like, same edge(asset) is traversed twice. Possibly a loop condition. Export request payload {noformat} { "itemsToExport": [ { "typeName": "hive_table", "uniqueAttributes": { "qualifiedName": "any_random_table@cluster" } } ], "options": { "fetchType" : "connected" } } {noformat} stacktrace {noformat} 2017-07-04 08:44:19,159 INFO - [pool-2-thread-9 - 8bc8d958-e650-4351-acf0-24c28d15627a:] ~ export(item=AtlasObjectId{guid='null', typeName='hive_table', uniqueAttributes={qualifiedName:db1.table1@cl1}}; matchType=null, fetchType=CONNECTED): found 1 entities (ExportService:280) 2017-07-04 08:44:19,794 ERROR - [pool-2-thread-9 - 8bc8d958-e650-4351-acf0-24c28d15627a:] ~ Fetching entity failed for: AtlasObjectId{guid='null', typeName='hive_table', uniqueAttributes={qualifiedName:db1.table1@cl1}} (ExportService:207) org.apache.atlas.exception.AtlasBaseException: Error writing file 75a90231-3eca-4ba7-9e21-93e7ec2b0df2. at org.apache.atlas.repository.impexp.ZipSink.saveToZip(ZipSink.java:91) at org.apache.atlas.repository.impexp.ZipSink.add(ZipSink.java:50) at org.apache.atlas.repository.impexp.ExportService.addEntity(ExportService.java:433) at org.apache.atlas.repository.impexp.ExportService.processEntity(ExportService.java:297) at org.apache.atlas.repository.impexp.ExportService.processObjectId(ExportService.java:198) at org.apache.atlas.repository.impexp.ExportService.processItems(ExportService.java:162) at org.apache.atlas.repository.impexp.ExportService.run(ExportService.java:95) at org.apache.atlas.web.resources.AdminResource.export(AdminResource.java:325) at sun.reflect.GeneratedMethodAccessor238.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60) at com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$ResponseOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:205) at com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75) at com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:302) at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147) at com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:108) at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147) at com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:84) at com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1542) at com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1473) at com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1419) at com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1409) at com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:409) at com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:558) at com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:733) at javax.servlet.http.HttpServlet.service(HttpServlet.java:790) at com.google.inject.servlet.ServletDefinition.doServiceImpl(ServletDefinition.java:286) at com.google.inject.servlet.ServletDefinition.doService(ServletDefinition.java:276) at com.google.inject.servlet.ServletDefinition.service(ServletDefinition.java:181) at com.google.inject.servlet.ManagedServletPipeline.service(ManagedServletPipeline.java:91) at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:85) at org.apache.atlas.web.filters.AuditFilter.doFilter(AuditFilter.java:78) at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82) at org.apache.atlas.web.filters.StaleTransactionCleanupFilter.doFilter(StaleTransactionCleanupFilter.java:56) at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82) at com.google.inject.servlet.ManagedFilterPipeline.di
[jira] [Created] (ATLAS-1923) Tag creation: hive_principal_type and file_action are shown in the drop down while adding attributes to a tag.
Ayub Pathan created ATLAS-1923: -- Summary: Tag creation: hive_principal_type and file_action are shown in the drop down while adding attributes to a tag. Key: ATLAS-1923 URL: https://issues.apache.org/jira/browse/ATLAS-1923 Project: Atlas Issue Type: Bug Components: atlas-webui Affects Versions: 0.9-incubating Reporter: Ayub Pathan Fix For: 0.9-incubating Attachments: Screen Shot 2017-07-05 at 10.38.50 AM.png Tag creation: hive_principal_type and file_action are shown in the drop down while adding attributes to a tag. Attaching screenshot for reference. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (ATLAS-1923) Tag creation: hive_principal_type and file_action are shown in the drop down while adding attributes to a tag.
[ https://issues.apache.org/jira/browse/ATLAS-1923?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayub Pathan updated ATLAS-1923: --- Priority: Minor (was: Major) > Tag creation: hive_principal_type and file_action are shown in the drop down > while adding attributes to a tag. > -- > > Key: ATLAS-1923 > URL: https://issues.apache.org/jira/browse/ATLAS-1923 > Project: Atlas > Issue Type: Bug > Components: atlas-webui >Affects Versions: 0.9-incubating >Reporter: Ayub Pathan >Priority: Minor > Fix For: 0.9-incubating > > Attachments: Screen Shot 2017-07-05 at 10.38.50 AM.png > > > Tag creation: hive_principal_type and file_action are shown in the drop down > while adding attributes to a tag. > Attaching screenshot for reference. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (ATLAS-1923) Tag creation: hive_principal_type and file_action are shown in the drop down while adding attributes to a tag.
[ https://issues.apache.org/jira/browse/ATLAS-1923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16074264#comment-16074264 ] Ayub Pathan commented on ATLAS-1923: I see that, any ENUm type is shown in the drop down while tag creation. Is there specific use-case for this? Looks like this is coming from ATLAS-1718 and ATLAS-1719 > Tag creation: hive_principal_type and file_action are shown in the drop down > while adding attributes to a tag. > -- > > Key: ATLAS-1923 > URL: https://issues.apache.org/jira/browse/ATLAS-1923 > Project: Atlas > Issue Type: Bug > Components: atlas-webui >Affects Versions: 0.9-incubating >Reporter: Ayub Pathan >Priority: Minor > Fix For: 0.9-incubating > > Attachments: Screen Shot 2017-07-05 at 10.38.50 AM.png > > > Tag creation: hive_principal_type and file_action are shown in the drop down > while adding attributes to a tag. > Attaching screenshot for reference. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (ATLAS-1874) V2 DSL search query does not support for count() but v1 do.
[ https://issues.apache.org/jira/browse/ATLAS-1874?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16081671#comment-16081671 ] Ayub Pathan commented on ATLAS-1874: [~grahamwallis] This "'_=1497434602692" comes from UI, whenever it makes any REST call, it attaches a unique ID. > V2 DSL search query does not support for count() but v1 do. > --- > > Key: ATLAS-1874 > URL: https://issues.apache.org/jira/browse/ATLAS-1874 > Project: Atlas > Issue Type: Bug > Components: atlas-core >Affects Versions: 0.8.1-incubating >Reporter: Ayub Pathan > Fix For: 0.9-incubating, 0.8.1-incubating > > > v1 DSL query supports count() > {noformat} > curl -u admin:admin > 'http://ctr-e133-1493418528701-113468-01-02.hwx.site:21000/api/atlas/discovery/search/dsl?query=hive_table%20where%20db.name%3D%22default%22%20select%20count()%20as%20%27count%27' > | python -m json.tool > % Total% Received % Xferd Average Speed TimeTime Time > Current > Dload Upload Total SpentLeft Speed > 100 5370 5370 0339 0 --:--:-- 0:00:01 --:--:-- 339 > { > "count": 1, > "dataType": { > "attributeDefinitions": [ > { > "dataTypeName": "long", > "isComposite": false, > "isIndexable": false, > "isUnique": false, > "multiplicity": { > "isUnique": false, > "lower": 0, > "upper": 1 > }, > "name": "count", > "reverseAttributeName": null > } > ], > "typeDescription": null, > "typeName": "__tempQueryResultStruct286", > "typeVersion": "1.0" > }, > "query": "hive_table where db.name=\"default\" select count() as 'count'", > "queryType": "dsl", > "requestId": "pool-2-thread-9 - 47389e3f-bbf1-4209-8e50-8a3235a7e5a9", > "results": [ > { > "$typeName$": "__tempQueryResultStruct286", > "count": 68 > } > ] > } > {noformat} > v2 DSL search query does not > {noformat} > curl -u admin:admin > 'http://ctr-e133-1493418528701-113468-01-02.hwx.site:21000/api/atlas/v2/search/dsl?limit=25&excludeDeletedEntities=true&query=where+db.name%3D%22default%22+select+count()+as+%27count%27&typeName=hive_table&_=1497434602692' > | python -m json.tool > { > "queryText": "`hive_table` where db.name=\"default\" select count() as > 'count'", > "queryType": "DSL" > } > {noformat} > *From the initial analysis, it seems like, V2 API response does not have the > count attribute which is cause for this failure.* might want to consider > adding the count in V2? -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (ATLAS-1941) HBaseStoreManager is overwriting the hbase.zookeeper.quorum value to incorrect one
Ayub Pathan created ATLAS-1941: -- Summary: HBaseStoreManager is overwriting the hbase.zookeeper.quorum value to incorrect one Key: ATLAS-1941 URL: https://issues.apache.org/jira/browse/ATLAS-1941 Project: Atlas Issue Type: Bug Components: atlas-core Affects Versions: 0.9-incubating Reporter: Ayub Pathan Fix For: 0.9-incubating While Atlas startup, HBaseStoreManager is copying host list from root.storage.hostname to hbase.zookeeper.quorum, which has one incorrect zookeeper host as well. {noformat} 2017-07-11 12:13:51,341 INFO - [main:] ~ Not running setup per configuration atlas.server.run.setup.on.start. (SetupSteps:189) 2017-07-11 12:13:52,332 INFO - [main:] ~ JSR-330 'javax.inject.Inject' annotation found and supported for autowiring (AutowiredAnnotationBeanPostProcessor:155) 2017-07-11 12:13:53,038 INFO - [main:] ~ Instantiated HBase compatibility layer supporting runtime HBase version 1.1.2.2.6.2.0-71: com.thinkaurelius.titan.diskstorage.hbase.HBaseCompat1_1 (HBaseCompatLoader:69) 2017-07-11 12:13:53,108 INFO - [main:] ~ Copied host list from root.storage.hostname to hbase.zookeeper.quorum: ctr-e133-1493418528701-179285-01-05.hwx.site,ctr-e133-1493418528701-179285-01-04.hwx.site,ctr-e133-1493418528701-179285-01-02.hwx.site (HBaseStoreManager:320) {noformat} In the above log, the correct zookeeper hosts are ctr-e133-1493418528701-179285-01-05.hwx.site,ctr-e133-1493418528701-179285-01-02.hwx.site. But, as we can see HBaseStoreManager is adding ctr-e133-1493418528701-179285-01-04.hwx.site as well which is incorrect. As an outcome of this, zookeeper connection to this host fails. {noformat} 2017-07-11 12:13:53,399 INFO - [main-SendThread(ctr-e133-1493418528701-179285-01-04.hwx.site:2181):] ~ Opening socket connection to server ctr-e133-1493418528701-179285-01-04.hwx.site/172.27.9.2:2181. Will attempt to SASL-authenticate using Login Context section 'Client' (ClientCnxn:1019) 2017-07-11 12:13:53,412 WARN - [main-SendThread(ctr-e133-1493418528701-179285-01-04.hwx.site:2181):] ~ Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect (ClientCnxn:1146) java.net.ConnectException: Connection refused at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717) at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:361) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1125) {noformat} -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (ATLAS-1940) Atlas server fails to initialize types on startup from graph store due to duplicate type registered.
[ https://issues.apache.org/jira/browse/ATLAS-1940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16082384#comment-16082384 ] Ayub Pathan commented on ATLAS-1940: This seems similar to ATLAS-1472, which is committed in atlas 0.8. > Atlas server fails to initialize types on startup from graph store due to > duplicate type registered. > > > Key: ATLAS-1940 > URL: https://issues.apache.org/jira/browse/ATLAS-1940 > Project: Atlas > Issue Type: Bug > Components: atlas-intg >Affects Versions: 0.8-incubating >Reporter: Nixon Rodrigues >Assignee: Nixon Rodrigues >Priority: Blocker > Fix For: 0.9-incubating, 0.8.1-incubating > > > Types are not getting restored from typesRegistry during Atlas startup due > duplicate type issue. > Below is exception trace. > {code} > 2017-07-07 13:35:23,088 INFO - [main:] ~ Restoring type CLASS.Process.null > (GraphBackedTypeStore:267) > 2017-07-07 13:35:23,099 INFO - [main:] ~ Restoring type > CLASS.falcon_process.null (GraphBackedTypeStore:267) > 2017-07-07 13:35:23,122 INFO - [main:] ~ Restoring type TRAIT.Metric.Metric > Trait (GraphBackedTypeStore:267) > 2017-07-07 13:35:23,132 INFO - [main:] ~ Restoring type > CLASS.Referenceable.null (GraphBackedTypeStore:267) > 2017-07-07 13:35:23,140 INFO - [main:] ~ Restoring type > CLASS.hive_storagedesc.null (GraphBackedTypeStore:267) > 2017-07-07 13:35:23,203 INFO - [main:] ~ Restoring type > CLASS.hive_table.null (GraphBackedTypeStore:267) > 2017-07-07 13:35:23,276 INFO - [main:] ~ Restoring type > CLASS.falcon_feed_creation.null (GraphBackedTypeStore:267) > 2017-07-07 13:35:23,288 INFO - [main:] ~ Restoring type > CLASS.sqoop_dbdatastore.null (GraphBackedTypeStore:267) > 2017-07-07 13:35:23,399 INFO - [main:] ~ Number of types got from transient > type system: 49 (DefaultMetadataService:168) > 2017-07-07 13:35:23,406 INFO - [main:] ~ Restored type system from the store > (DefaultMetadataService:161) > 2017-07-07 13:35:23,407 INFO - [main:] ~ Global indexes already exist for > graph (GraphBackedSearchIndexer:141) > 2017-07-07 13:35:23,753 WARN - [main:] ~ Invalid constraint ownedRef for > attribute hive_storagedesc.serdeInfo (AtlasTypeDefGraphStore:689) > 2017-07-07 13:35:23,754 WARN - [main:] ~ Invalid constraint ownedRef for > attribute hive_table.aliases (AtlasTypeDefGraphStore:689) > 2017-07-07 13:35:23,759 ERROR - [main:] ~ failed to initialize types from > graph store (AtlasTypeDefGraphStoreV1:77) > org.apache.atlas.exception.AtlasBaseException: Given type TaxonomyTerm > already exists > at > org.apache.atlas.type.AtlasTypeRegistry$AtlasTransientTypeRegistry.addTypeWithNoRefResolve(AtlasTypeRegistry.java:573) > at > org.apache.atlas.type.AtlasTypeRegistry$AtlasTransientTypeRegistry.addTypesWithNoRefResolve(AtlasTypeRegistry.java:611) > at > org.apache.atlas.type.AtlasTypeRegistry$AtlasTransientTypeRegistry.addTypes(AtlasTypeRegistry.java:407) > at > org.apache.atlas.repository.store.graph.AtlasTypeDefGraphStore.init(AtlasTypeDefGraphStore.java:105) > at > org.apache.atlas.repository.store.graph.v1.AtlasTypeDefGraphStoreV1.init(AtlasTypeDefGraphStoreV1.java:107) > at > org.apache.atlas.repository.store.graph.v1.AtlasTypeDefGraphStoreV1.(AtlasTypeDefGraphStoreV1.java:75) > at > org.apache.atlas.repository.store.graph.v1.AtlasTypeDefGraphStoreV1$$EnhancerByGuice$$21ccc5be.() > at > org.apache.atlas.repository.store.graph.v1.AtlasTypeDefGraphStoreV1$$EnhancerByGuice$$21ccc5be$$FastClassByGuice$$a11fb32d.newInstance() > at > com.google.inject.internal.ProxyFactory$ProxyConstructor.newInstance(ProxyFactory.java:265) > at > com.google.inject.internal.ConstructorInjector.provision(ConstructorInjector.java:111) > at > com.google.inject.internal.ConstructorInjector.construct(ConstructorInjector.java:90) > at > com.google.inject.internal.ConstructorBindingImpl$Factory.get(ConstructorBindingImpl.java:268) > at com.google.inject.internal.FactoryProxy.get(FactoryProxy.java:56) > at > com.google.inject.internal.ProviderToInternalFactoryAdapter$1.call(ProviderToInternalFactoryAdapter.java:46) > at > com.google.inject.internal.InjectorImpl.callInContext(InjectorImpl.java:1092) > at > com.google.inject.internal.ProviderToInternalFactoryAdapter.get(ProviderToInternalFactoryAdapter.java:40) > at > com.google.inject.internal.SingletonScope$1.get(SingletonScope.java:194) > at > com.google.inject.internal.InternalFactoryToProviderAdapter.get(InternalFactoryToProviderAdapter.java:41) > at > com.google.inject.internal.InternalInjectorCreator$1.call(InternalInjectorCreator.java:205) > at > co
[jira] [Updated] (ATLAS-1941) HBaseStoreManager is copying host list from root.storage.hostname to hbase.zookeeper.quorum, which has one incorrect zookeeper host
[ https://issues.apache.org/jira/browse/ATLAS-1941?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayub Pathan updated ATLAS-1941: --- Summary: HBaseStoreManager is copying host list from root.storage.hostname to hbase.zookeeper.quorum, which has one incorrect zookeeper host (was: HBaseStoreManager is overwriting the hbase.zookeeper.quorum value to incorrect one) > HBaseStoreManager is copying host list from root.storage.hostname to > hbase.zookeeper.quorum, which has one incorrect zookeeper host > --- > > Key: ATLAS-1941 > URL: https://issues.apache.org/jira/browse/ATLAS-1941 > Project: Atlas > Issue Type: Bug > Components: atlas-core >Affects Versions: 0.9-incubating >Reporter: Ayub Pathan >Priority: Minor > Fix For: 0.9-incubating > > > While Atlas startup, HBaseStoreManager is copying host list from > root.storage.hostname to hbase.zookeeper.quorum, which has one incorrect > zookeeper host as well. > {noformat} > 2017-07-11 12:13:51,341 INFO - [main:] ~ Not running setup per configuration > atlas.server.run.setup.on.start. (SetupSteps:189) > 2017-07-11 12:13:52,332 INFO - [main:] ~ JSR-330 'javax.inject.Inject' > annotation found and supported for autowiring > (AutowiredAnnotationBeanPostProcessor:155) > 2017-07-11 12:13:53,038 INFO - [main:] ~ Instantiated HBase compatibility > layer supporting runtime HBase version 1.1.2.2.6.2.0-71: > com.thinkaurelius.titan.diskstorage.hbase.HBaseCompat1_1 > (HBaseCompatLoader:69) > 2017-07-11 12:13:53,108 INFO - [main:] ~ Copied host list from > root.storage.hostname to hbase.zookeeper.quorum: > ctr-e133-1493418528701-179285-01-05.hwx.site,ctr-e133-1493418528701-179285-01-04.hwx.site,ctr-e133-1493418528701-179285-01-02.hwx.site > (HBaseStoreManager:320) > {noformat} > In the above log, the correct zookeeper hosts are > ctr-e133-1493418528701-179285-01-05.hwx.site,ctr-e133-1493418528701-179285-01-02.hwx.site. > But, as we can see HBaseStoreManager is adding > ctr-e133-1493418528701-179285-01-04.hwx.site as well which is incorrect. > As an outcome of this, zookeeper connection to this host fails. > {noformat} > 2017-07-11 12:13:53,399 INFO - > [main-SendThread(ctr-e133-1493418528701-179285-01-04.hwx.site:2181):] ~ > Opening socket connection to server > ctr-e133-1493418528701-179285-01-04.hwx.site/172.27.9.2:2181. Will > attempt to SASL-authenticate using Login Context section 'Client' > (ClientCnxn:1019) > 2017-07-11 12:13:53,412 WARN - > [main-SendThread(ctr-e133-1493418528701-179285-01-04.hwx.site:2181):] ~ > Session 0x0 for server null, unexpected error, closing socket connection and > attempting reconnect (ClientCnxn:1146) > java.net.ConnectException: Connection refused > at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) > at > sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717) > at > org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:361) > at > org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1125) > {noformat} -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (ATLAS-1941) HBaseStoreManager is overwriting the hbase.zookeeper.quorum value to incorrect one
[ https://issues.apache.org/jira/browse/ATLAS-1941?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayub Pathan updated ATLAS-1941: --- Priority: Minor (was: Major) > HBaseStoreManager is overwriting the hbase.zookeeper.quorum value to > incorrect one > -- > > Key: ATLAS-1941 > URL: https://issues.apache.org/jira/browse/ATLAS-1941 > Project: Atlas > Issue Type: Bug > Components: atlas-core >Affects Versions: 0.9-incubating >Reporter: Ayub Pathan >Priority: Minor > Fix For: 0.9-incubating > > > While Atlas startup, HBaseStoreManager is copying host list from > root.storage.hostname to hbase.zookeeper.quorum, which has one incorrect > zookeeper host as well. > {noformat} > 2017-07-11 12:13:51,341 INFO - [main:] ~ Not running setup per configuration > atlas.server.run.setup.on.start. (SetupSteps:189) > 2017-07-11 12:13:52,332 INFO - [main:] ~ JSR-330 'javax.inject.Inject' > annotation found and supported for autowiring > (AutowiredAnnotationBeanPostProcessor:155) > 2017-07-11 12:13:53,038 INFO - [main:] ~ Instantiated HBase compatibility > layer supporting runtime HBase version 1.1.2.2.6.2.0-71: > com.thinkaurelius.titan.diskstorage.hbase.HBaseCompat1_1 > (HBaseCompatLoader:69) > 2017-07-11 12:13:53,108 INFO - [main:] ~ Copied host list from > root.storage.hostname to hbase.zookeeper.quorum: > ctr-e133-1493418528701-179285-01-05.hwx.site,ctr-e133-1493418528701-179285-01-04.hwx.site,ctr-e133-1493418528701-179285-01-02.hwx.site > (HBaseStoreManager:320) > {noformat} > In the above log, the correct zookeeper hosts are > ctr-e133-1493418528701-179285-01-05.hwx.site,ctr-e133-1493418528701-179285-01-02.hwx.site. > But, as we can see HBaseStoreManager is adding > ctr-e133-1493418528701-179285-01-04.hwx.site as well which is incorrect. > As an outcome of this, zookeeper connection to this host fails. > {noformat} > 2017-07-11 12:13:53,399 INFO - > [main-SendThread(ctr-e133-1493418528701-179285-01-04.hwx.site:2181):] ~ > Opening socket connection to server > ctr-e133-1493418528701-179285-01-04.hwx.site/172.27.9.2:2181. Will > attempt to SASL-authenticate using Login Context section 'Client' > (ClientCnxn:1019) > 2017-07-11 12:13:53,412 WARN - > [main-SendThread(ctr-e133-1493418528701-179285-01-04.hwx.site:2181):] ~ > Session 0x0 for server null, unexpected error, closing socket connection and > attempting reconnect (ClientCnxn:1146) > java.net.ConnectException: Connection refused > at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) > at > sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717) > at > org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:361) > at > org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1125) > {noformat} -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (ATLAS-1874) V2 DSL search query does not support for count() but v1 do.
[ https://issues.apache.org/jira/browse/ATLAS-1874?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16083284#comment-16083284 ] Ayub Pathan commented on ATLAS-1874: [~grahamwallis] DSL query works fine for the above query you mentioned, it's only that UI does not display anything, because query is requesting for specific attribute (in this case, owner) of that entity. Please check the below response for same query. Response for query: "`hive_table` where db.name=\"default\" select owner" {noformat} { "queryType":"DSL", "queryText":"`hive_table` where db.name=\"default\" select owner", "attributes":{ "name":[ "owner" ], "values":[ [ "hive" ] ] } } {noformat} > V2 DSL search query does not support for count() but v1 do. > --- > > Key: ATLAS-1874 > URL: https://issues.apache.org/jira/browse/ATLAS-1874 > Project: Atlas > Issue Type: Bug > Components: atlas-core >Affects Versions: 0.8.1-incubating >Reporter: Ayub Pathan > Fix For: 0.9-incubating, 0.8.1-incubating > > > v1 DSL query supports count() > {noformat} > curl -u admin:admin > 'http://ctr-e133-1493418528701-113468-01-02.hwx.site:21000/api/atlas/discovery/search/dsl?query=hive_table%20where%20db.name%3D%22default%22%20select%20count()%20as%20%27count%27' > | python -m json.tool > % Total% Received % Xferd Average Speed TimeTime Time > Current > Dload Upload Total SpentLeft Speed > 100 5370 5370 0339 0 --:--:-- 0:00:01 --:--:-- 339 > { > "count": 1, > "dataType": { > "attributeDefinitions": [ > { > "dataTypeName": "long", > "isComposite": false, > "isIndexable": false, > "isUnique": false, > "multiplicity": { > "isUnique": false, > "lower": 0, > "upper": 1 > }, > "name": "count", > "reverseAttributeName": null > } > ], > "typeDescription": null, > "typeName": "__tempQueryResultStruct286", > "typeVersion": "1.0" > }, > "query": "hive_table where db.name=\"default\" select count() as 'count'", > "queryType": "dsl", > "requestId": "pool-2-thread-9 - 47389e3f-bbf1-4209-8e50-8a3235a7e5a9", > "results": [ > { > "$typeName$": "__tempQueryResultStruct286", > "count": 68 > } > ] > } > {noformat} > v2 DSL search query does not > {noformat} > curl -u admin:admin > 'http://ctr-e133-1493418528701-113468-01-02.hwx.site:21000/api/atlas/v2/search/dsl?limit=25&excludeDeletedEntities=true&query=where+db.name%3D%22default%22+select+count()+as+%27count%27&typeName=hive_table&_=1497434602692' > | python -m json.tool > { > "queryText": "`hive_table` where db.name=\"default\" select count() as > 'count'", > "queryType": "DSL" > } > {noformat} > *From the initial analysis, it seems like, V2 API response does not have the > count attribute which is cause for this failure.* might want to consider > adding the count in V2? -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Comment Edited] (ATLAS-1874) V2 DSL search query does not support for count() but v1 do.
[ https://issues.apache.org/jira/browse/ATLAS-1874?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16083284#comment-16083284 ] Ayub Pathan edited comment on ATLAS-1874 at 7/12/17 1:22 AM: - [~grahamwallis] DSL query works fine for the above query you mentioned, it's only that UI does not display anything, because query is requesting for specific attribute (in this case, owner) of that entity. Please check the below response for same query. Response for query: "`hive_table` where db.name=\"default\" select owner" {noformat} { "queryType":"DSL", "queryText":"`hive_table` where db.name=\"default\" select owner", "attributes":{ "name":[ "owner" ], "values":[ [ "hive" ] ] } } {noformat} Where as for the count query, response is empty. for example: {noformat} curl -u admin:admin 'http://ctr-e133-1493418528701-113468-01-02.hwx.site:21000/api/atlas/v2/search/dsl?limit=25&excludeDeletedEntities=true&query=where+db.name%3D%22default%22+select+count()+as+%27count%27&typeName=hive_table&_=1497434602692' | python -m json.tool { "queryText": "`hive_table` where db.name=\"default\" select count() as 'count'", "queryType": "DSL" } {noformat} was (Author: ayubkhan): [~grahamwallis] DSL query works fine for the above query you mentioned, it's only that UI does not display anything, because query is requesting for specific attribute (in this case, owner) of that entity. Please check the below response for same query. Response for query: "`hive_table` where db.name=\"default\" select owner" {noformat} { "queryType":"DSL", "queryText":"`hive_table` where db.name=\"default\" select owner", "attributes":{ "name":[ "owner" ], "values":[ [ "hive" ] ] } } {noformat} > V2 DSL search query does not support for count() but v1 do. > --- > > Key: ATLAS-1874 > URL: https://issues.apache.org/jira/browse/ATLAS-1874 > Project: Atlas > Issue Type: Bug > Components: atlas-core >Affects Versions: 0.8.1-incubating >Reporter: Ayub Pathan > Fix For: 0.9-incubating, 0.8.1-incubating > > > v1 DSL query supports count() > {noformat} > curl -u admin:admin > 'http://ctr-e133-1493418528701-113468-01-02.hwx.site:21000/api/atlas/discovery/search/dsl?query=hive_table%20where%20db.name%3D%22default%22%20select%20count()%20as%20%27count%27' > | python -m json.tool > % Total% Received % Xferd Average Speed TimeTime Time > Current > Dload Upload Total SpentLeft Speed > 100 5370 5370 0339 0 --:--:-- 0:00:01 --:--:-- 339 > { > "count": 1, > "dataType": { > "attributeDefinitions": [ > { > "dataTypeName": "long", > "isComposite": false, > "isIndexable": false, > "isUnique": false, > "multiplicity": { > "isUnique": false, > "lower": 0, > "upper": 1 > }, > "name": "count", > "reverseAttributeName": null > } > ], > "typeDescription": null, > "typeName": "__tempQueryResultStruct286", > "typeVersion": "1.0" > }, > "query": "hive_table where db.name=\"default\" select count() as 'count'", > "queryType": "dsl", > "requestId": "pool-2-thread-9 - 47389e3f-bbf1-4209-8e50-8a3235a7e5a9", > "results": [ > { > "$typeName$": "__tempQueryResultStruct286", > "count": 68 > } > ] > } > {noformat} > v2 DSL search query does not > {noformat} > curl -u admin:admin > 'http://ctr-e133-1493418528701-113468-01-02.hwx.site:21000/api/atlas/v2/search/dsl?limit=25&excludeDeletedEntities=true&query=where+db.name%3D%22default%22+select+count()+as+%27count%27&typeName=hive_table&_=1497434602692' > | python -m json.tool > { > "queryText": "`hive_table` where db.name=\"default\" select count() as > 'count'", > "queryType": "DSL" > } > {noformat} > *From the initial analysis, it seems like, V2 API response does not have the > count attribute which is cause for this failure.* might want to consider > adding the count in V2? -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (ATLAS-1944) NotificationHookConsumer throws exception while shutting down the consumer thread
Ayub Pathan created ATLAS-1944: -- Summary: NotificationHookConsumer throws exception while shutting down the consumer thread Key: ATLAS-1944 URL: https://issues.apache.org/jira/browse/ATLAS-1944 Project: Atlas Issue Type: Bug Components: atlas-core Affects Versions: 0.9-incubating Reporter: Ayub Pathan Priority: Critical Fix For: 0.9-incubating NotificationHookConsumer throws below exception while shutting down the consumer thread, this issue is possibly after this commit. https://github.com/apache/incubator-atlas/commit/0e7f8ea4603c858cc295259bbd1a22314b732f62 CC [~nrodrigues] {noformat} 2017-07-12 01:26:09,743 WARN - [pool-1-thread-1:] ~ Error stopping service org.apache.atlas.notification.NotificationHookConsumer (Services:69) java.util.ConcurrentModificationException: KafkaConsumer is not safe for multi-threaded access at org.apache.kafka.clients.consumer.KafkaConsumer.acquire(KafkaConsumer.java:1557) at org.apache.kafka.clients.consumer.KafkaConsumer.close(KafkaConsumer.java:1467) at org.apache.atlas.kafka.AtlasKafkaConsumer.close(AtlasKafkaConsumer.java:88) at org.apache.atlas.notification.NotificationHookConsumer$HookConsumer.stop(NotificationHookConsumer.java:384) at org.apache.atlas.notification.NotificationHookConsumer.stopConsumerThreads(NotificationHookConsumer.java:172) at org.apache.atlas.notification.NotificationHookConsumer.stop(NotificationHookConsumer.java:155) at org.apache.atlas.service.Services.stop(Services.java:67) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor$LifecycleElement.invoke(InitDestroyAnnotationBeanPostProcessor.java:366) at org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor$LifecycleMetadata.invokeDestroyMethods(InitDestroyAnnotationBeanPostProcessor.java:325) at org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor.postProcessBeforeDestruction(InitDestroyAnnotationBeanPostProcessor.java:154) at org.springframework.beans.factory.support.DisposableBeanAdapter.destroy(DisposableBeanAdapter.java:253) at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.destroyBean(DefaultSingletonBeanRegistry.java:578) at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.destroySingleton(DefaultSingletonBeanRegistry.java:554) at org.springframework.beans.factory.support.DefaultListableBeanFactory.destroySingleton(DefaultListableBeanFactory.java:961) at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.destroySingletons(DefaultSingletonBeanRegistry.java:523) at org.springframework.beans.factory.support.DefaultListableBeanFactory.destroySingletons(DefaultListableBeanFactory.java:968) at org.springframework.context.support.AbstractApplicationContext.destroyBeans(AbstractApplicationContext.java:1033) at org.springframework.context.support.AbstractApplicationContext.doClose(AbstractApplicationContext.java:1009) at org.springframework.context.support.AbstractApplicationContext.close(AbstractApplicationContext.java:961) at org.springframework.web.context.ContextLoader.closeWebApplicationContext(ContextLoader.java:583) at org.springframework.web.context.ContextLoaderListener.contextDestroyed(ContextLoaderListener.java:116) at org.eclipse.jetty.server.handler.ContextHandler.callContextDestroyed(ContextHandler.java:808) at org.eclipse.jetty.servlet.ServletContextHandler.callContextDestroyed(ServletContextHandler.java:457) at org.eclipse.jetty.server.handler.ContextHandler.doStop(ContextHandler.java:842) at org.eclipse.jetty.servlet.ServletContextHandler.doStop(ServletContextHandler.java:215) at org.eclipse.jetty.webapp.WebAppContext.doStop(WebAppContext.java:529) at org.eclipse.jetty.util.component.AbstractLifeCycle.stop(AbstractLifeCycle.java:89) at org.eclipse.jetty.util.component.ContainerLifeCycle.stop(ContainerLifeCycle.java:143) at org.eclipse.jetty.util.component.ContainerLifeCycle.doStop(ContainerLifeCycle.java:162) at org.eclipse.jetty.server.handler.AbstractHandler.doStop(AbstractHandler.java:73) at org.eclipse.jetty.server.Server.doStop(Server.java:456) at org.eclipse.jetty.util.component.AbstractLifeCycle.stop(AbstractLifeCycle.java:89) {noformat} -- This message was se
[jira] [Created] (ATLAS-1946) LDAPS(LDAP over SSL) support in Atlas
Ayub Pathan created ATLAS-1946: -- Summary: LDAPS(LDAP over SSL) support in Atlas Key: ATLAS-1946 URL: https://issues.apache.org/jira/browse/ATLAS-1946 Project: Atlas Issue Type: Bug Components: atlas-intg Affects Versions: 0.9-incubating Reporter: Ayub Pathan Add support for LDAPS in Atlas. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (ATLAS-1946) LDAPS(LDAP over SSL) support in Atlas
[ https://issues.apache.org/jira/browse/ATLAS-1946?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayub Pathan updated ATLAS-1946: --- Issue Type: Improvement (was: Bug) > LDAPS(LDAP over SSL) support in Atlas > - > > Key: ATLAS-1946 > URL: https://issues.apache.org/jira/browse/ATLAS-1946 > Project: Atlas > Issue Type: Improvement > Components: atlas-intg >Affects Versions: 0.9-incubating >Reporter: Ayub Pathan > > Add support for LDAPS in Atlas. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (ATLAS-1951) Regression: Any REST API request without user credentials results in 302 redirect to login.jsp. Actually, the correct response should be 401.
Ayub Pathan created ATLAS-1951: -- Summary: Regression: Any REST API request without user credentials results in 302 redirect to login.jsp. Actually, the correct response should be 401. Key: ATLAS-1951 URL: https://issues.apache.org/jira/browse/ATLAS-1951 Project: Atlas Issue Type: Bug Components: atlas-core Affects Versions: 0.9-incubating Reporter: Ayub Pathan Priority: Critical Fix For: 0.9-incubating Regression: Any REST API request to atlas without user credentials results in 302 redirect( login.jsp), but the actual response code should be 401 unauthorized. This issue could have been introduced as part of new spring changes. For example: {noformat} curl -v -X GET "http://ctr-e133-1493418528701-181199-01-02.hwx.site:21000"; * Rebuilt URL to: http://ctr-e133-1493418528701-181199-01-02.hwx.site:21000/ * Trying 172.27.56.2... * Connected to ctr-e133-1493418528701-181199-01-02.hwx.site (172.27.56.2) port 21000 (#0) > GET / HTTP/1.1 > Host: ctr-e133-1493418528701-181199-01-02.hwx.site:21000 > User-Agent: curl/7.43.0 > Accept: */* > < HTTP/1.1 302 Found < Date: Fri, 14 Jul 2017 11:16:42 GMT < Set-Cookie: ATLASSESSIONID=1i0rxnm66dd3h17xyhvstk0vck;Path=/;HttpOnly < Expires: Thu, 01 Jan 1970 00:00:00 GMT < X-Frame-Options: DENY < Location: http://ctr-e133-1493418528701-181199-01-02.hwx.site:21000/login.jsp < Content-Length: 0 < Server: Jetty(9.2.12.v20150709) < * Connection #0 to host ctr-e133-1493418528701-181199-01-02.hwx.site left intact {noformat} CC [~skoneru] -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (ATLAS-1951) Regression: Any REST API request without user credentials results in 302 redirect to login.jsp. Actually, the correct response should be 401.
[ https://issues.apache.org/jira/browse/ATLAS-1951?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayub Pathan updated ATLAS-1951: --- Description: Regression: Any REST API request to atlas without user credentials results in 302 redirect( login.jsp), but the actual response code should be 401 unauthorized. This issue could have been introduced as part of new spring changes. For example: {noformat} curl -v -X GET "http://ctr-e133-1493418528701-181199-01-02.hwx.site:21000"; * Rebuilt URL to: http://ctr-e133-1493418528701-181199-01-02.hwx.site:21000/ * Trying 172.27.56.2... * Connected to ctr-e133-1493418528701-181199-01-02.hwx.site (172.27.56.2) port 21000 (#0) > GET / HTTP/1.1 > Host: ctr-e133-1493418528701-181199-01-02.hwx.site:21000 > User-Agent: curl/7.43.0 > Accept: */* > < HTTP/1.1 302 Found < Date: Fri, 14 Jul 2017 11:16:42 GMT < Set-Cookie: ATLASSESSIONID=1i0rxnm66dd3h17xyhvstk0vck;Path=/;HttpOnly < Expires: Thu, 01 Jan 1970 00:00:00 GMT < X-Frame-Options: DENY < Location: http://ctr-e133-1493418528701-181199-01-02.hwx.site:21000/login.jsp < Content-Length: 0 < Server: Jetty(9.2.12.v20150709) < * Connection #0 to host ctr-e133-1493418528701-181199-01-02.hwx.site left intact {noformat} was: Regression: Any REST API request to atlas without user credentials results in 302 redirect( login.jsp), but the actual response code should be 401 unauthorized. This issue could have been introduced as part of new spring changes. For example: {noformat} curl -v -X GET "http://ctr-e133-1493418528701-181199-01-02.hwx.site:21000"; * Rebuilt URL to: http://ctr-e133-1493418528701-181199-01-02.hwx.site:21000/ * Trying 172.27.56.2... * Connected to ctr-e133-1493418528701-181199-01-02.hwx.site (172.27.56.2) port 21000 (#0) > GET / HTTP/1.1 > Host: ctr-e133-1493418528701-181199-01-02.hwx.site:21000 > User-Agent: curl/7.43.0 > Accept: */* > < HTTP/1.1 302 Found < Date: Fri, 14 Jul 2017 11:16:42 GMT < Set-Cookie: ATLASSESSIONID=1i0rxnm66dd3h17xyhvstk0vck;Path=/;HttpOnly < Expires: Thu, 01 Jan 1970 00:00:00 GMT < X-Frame-Options: DENY < Location: http://ctr-e133-1493418528701-181199-01-02.hwx.site:21000/login.jsp < Content-Length: 0 < Server: Jetty(9.2.12.v20150709) < * Connection #0 to host ctr-e133-1493418528701-181199-01-02.hwx.site left intact {noformat} CC [~skoneru] > Regression: Any REST API request without user credentials results in 302 > redirect to login.jsp. Actually, the correct response should be 401. > - > > Key: ATLAS-1951 > URL: https://issues.apache.org/jira/browse/ATLAS-1951 > Project: Atlas > Issue Type: Bug > Components: atlas-core >Affects Versions: 0.9-incubating >Reporter: Ayub Pathan >Priority: Critical > Fix For: 0.9-incubating > > > Regression: Any REST API request to atlas without user credentials results in > 302 redirect( login.jsp), but the actual response code should be 401 > unauthorized. > This issue could have been introduced as part of new spring changes. > For example: > {noformat} > curl -v -X GET "http://ctr-e133-1493418528701-181199-01-02.hwx.site:21000"; > * Rebuilt URL to: > http://ctr-e133-1493418528701-181199-01-02.hwx.site:21000/ > * Trying 172.27.56.2... > * Connected to ctr-e133-1493418528701-181199-01-02.hwx.site (172.27.56.2) > port 21000 (#0) > > GET / HTTP/1.1 > > Host: ctr-e133-1493418528701-181199-01-02.hwx.site:21000 > > User-Agent: curl/7.43.0 > > Accept: */* > > > < HTTP/1.1 302 Found > < Date: Fri, 14 Jul 2017 11:16:42 GMT > < Set-Cookie: ATLASSESSIONID=1i0rxnm66dd3h17xyhvstk0vck;Path=/;HttpOnly > < Expires: Thu, 01 Jan 1970 00:00:00 GMT > < X-Frame-Options: DENY > < Location: > http://ctr-e133-1493418528701-181199-01-02.hwx.site:21000/login.jsp > < Content-Length: 0 > < Server: Jetty(9.2.12.v20150709) > < > * Connection #0 to host ctr-e133-1493418528701-181199-01-02.hwx.site left > intact > {noformat} -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (ATLAS-2612) Atlas returns term definition with "/glossary/" API. Is this accurate? The correct API should be /glossary/term/
Ayub Pathan created ATLAS-2612: -- Summary: Atlas returns term definition with "/glossary/" API. Is this accurate? The correct API should be /glossary/term/ Key: ATLAS-2612 URL: https://issues.apache.org/jira/browse/ATLAS-2612 Project: Atlas Issue Type: Bug Components: atlas-core Affects Versions: 1.0.0 Reporter: Ayub Pathan Fix For: 1.0.0 * Below are the list of glossary and terms {noformat} curl -k -u admin:admin 'https://ctr-e138-1518143905142-256977-01-04.hwx.site:21443/api/atlas/v2/glossary/'| python -m json.tool % Total% Received % Xferd Average Speed TimeTime Time Current Dload Upload Total SpentLeft Speed 100 5640 5640 0224 0 --:--:-- 0:00:02 --:--:-- 224 [ { "guid": "7fd4da3a-e6df-402f-a801-15883567d762", "displayName": "Org", "shortDescription": "hortonworks", "longDescription": "HWX", "qualifiedName": "Org", "terms": [ { "termGuid": "eac56951-c61f-4b49-a578-8be5d37d2679", "relationGuid": "fb87ff3e-50e1-461f-be56-e0ab3d701c15", "displayText": "apache" }, { "termGuid": "2cf05982-2e39-41c5-a656-b3703dbc8b4c", "relationGuid": "a870627e-febe-474e-994e-31dc37aac716", "displayText": "abc" } ] }, { "guid": "1627ad45-84a3-4189-b708-54939fa18afa", "displayName": "apache", "shortDescription": "apache", "longDescription": "apache", "qualifiedName": "apache" } ] {noformat} * */glossary/ API returns term definition* {noformat} curl -k -u admin:admin 'https://ctr-e138-1518143905142-256977-01-04.hwx.site:21443/api/atlas/v2/glossary/2cf05982-2e39-41c5-a656-b3703dbc8b4c' | python -m json.tool % Total% Received % Xferd Average Speed TimeTime Time Current Dload Upload Total SpentLeft Speed 100 1420 1420 0 57 0 --:--:-- 0:00:02 --:--:--57 { "guid": "2cf05982-2e39-41c5-a656-b3703dbc8b4c", "displayName": "abc", "shortDescription": "abc", "longDescription": "abc", "qualifiedName": "abc@Org" } {noformat} atlas version {noformat} { "Description": "Metadata Management and Data Governance Platform over Hadoop", "Version": "1.0.0.3.0.0.0-1242-r9b7265e3c2e483a3942f42f111c25b38885259fb", "Name": "apache-atlas" } {noformat} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (ATLAS-2613) Submitting a glossary with guid=-1, atlas returns 404 but actually creates the glossary
Ayub Pathan created ATLAS-2613: -- Summary: Submitting a glossary with guid=-1, atlas returns 404 but actually creates the glossary Key: ATLAS-2613 URL: https://issues.apache.org/jira/browse/ATLAS-2613 Project: Atlas Issue Type: Bug Components: atlas-core Affects Versions: 1.0.0 Reporter: Ayub Pathan Fix For: 1.0.0 Submitting below glossary returns 404 with error "Given instance guid -1 is invalid/not found" but actually glossary is created. {noformat} https://ctr-e138-1518143905142-256977-01-04.hwx.site:21443/api/atlas/v2/glossary { "guid" : "-1", "qualifiedName" : "HortoniaBank-8", "displayName": "Banking", "shortDescription": "Glossary of bank", "longDescription": "Glossary of bank - long description", "language": "English", "usage": "N/A" } {noformat} Atlas log also show graph rollback {noformat} 2018-04-25 10:38:05,551 ERROR - [pool-2-thread-10 - 8ef9844f-deac-4df0-969c-e077f4664ed3:] ~ graph rollback due to exception AtlasBaseException:Given instance guid -1 is invalid/not found (GraphTransactionInterceptor:73) {noformat} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (ATLAS-2614) Uniqueness of qualified names is not honored while updating a glossary
Ayub Pathan created ATLAS-2614: -- Summary: Uniqueness of qualified names is not honored while updating a glossary Key: ATLAS-2614 URL: https://issues.apache.org/jira/browse/ATLAS-2614 Project: Atlas Issue Type: Bug Components: atlas-core Affects Versions: 1.0.0 Reporter: Ayub Pathan Fix For: 1.0.0 Uniqueness of qualified names is not honored while updating a glossary * creating 2 glossaries with unique names {noformat} apathan@Ayub-sys-101:~ > curl -k -H "Content-Type: application/json" -X POST -u admin:admin https://ctr-e138-1518143905142-256977-01-04.hwx.site:21443/api/atlas/v2/glossary/ -d '{ "qualifiedName" : "HortoniaBank-11", "displayName": "Banking-11" }' | python -m json.tool % Total% Received % Xferd Average Speed TimeTime Time Current Dload Upload Total SpentLeft Speed 100 1760 108 10068 23 14 0:00:04 0:00:04 --:--:--43 { "guid": "c445e6c3-1113-4d55-acbd-d421c8c98db1", "displayName": "Banking-11", "qualifiedName": "HortoniaBank-11" } apathan@Ayub-sys-101:~ > curl -k -H "Content-Type: application/json" -X POST -u admin:admin https://ctr-e138-1518143905142-256977-01-04.hwx.site:21443/api/atlas/v2/glossary/ -d '{ "qualifiedName" : "HortoniaBank-12", "displayName": "Banking-12" }' | python -m json.tool % Total% Received % Xferd Average Speed TimeTime Time Current Dload Upload Total SpentLeft Speed 100 1760 108 10068 32 20 0:00:03 0:00:03 --:--:--52 { "guid": "bf43f063-f499-472c-b526-0f81de685e02", "displayName": "Banking-12", "qualifiedName": "HortoniaBank-12" } apathan@Ayub-sys-101:~ > curl -k -u admin:admin https://ctr-e138-1518143905142-256977-01-04.hwx.site:21443/api/atlas/v2/glossary/ | python -m json.tool % Total% Received % Xferd Average Speed TimeTime Time Current Dload Upload Total SpentLeft Speed 100 2190 2190 0 64 0 --:--:-- 0:00:03 --:--:--64 [ { "guid": "bf43f063-f499-472c-b526-0f81de685e02", "displayName": "Banking-12", "qualifiedName": "HortoniaBank-12" }, { "guid": "c445e6c3-1113-4d55-acbd-d421c8c98db1", "displayName": "Banking-11", "qualifiedName": "HortoniaBank-11" } ] {noformat} * Updating glossary-2 with glossary-1's qualified name, which is allowed. {noformat} curl -k -H "Content-Type: application/json" -X PUT -u admin:admin https://ctr-e138-1518143905142-256977-01-04.hwx.site:21443/api/atlas/v2/glossary/bf43f063-f499-472c-b526-0f81de685e02 -d '{ "qualifiedName" : "HortoniaBank-11", "displayName": "Banking-11" }' | python -m json.tool % Total% Received % Xferd Average Speed TimeTime Time Current Dload Upload Total SpentLeft Speed 100 1760 108 10068 38 23 0:00:02 0:00:02 --:--:--62 { "guid": "bf43f063-f499-472c-b526-0f81de685e02", "displayName": "Banking-11", "qualifiedName": "HortoniaBank-11" } {noformat} * Now, you can see both the entities have same qualifedName {noformat} apathan@Ayub-sys-101:~ > curl -k -u admin:admin https://ctr-e138-1518143905142-256977-01-04.hwx.site:21443/api/atlas/v2/glossary/ | python -m json.tool % Total% Received % Xferd Average Speed TimeTime Time Current Dload Upload Total SpentLeft Speed 100 2190 2190 0141 0 --:--:-- 0:00:01 --:--:-- 141 [ { "guid": "bf43f063-f499-472c-b526-0f81de685e02", "displayName": "Banking-11", "qualifiedName": "HortoniaBank-11" }, { "guid": "c445e6c3-1113-4d55-acbd-d421c8c98db1", "displayName": "Banking-11", "qualifiedName": "HortoniaBank-11" } ] apathan@Ayub-sys-101:~ > {noformat} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (ATLAS-2615) Bulk glossary creation support
Ayub Pathan created ATLAS-2615: -- Summary: Bulk glossary creation support Key: ATLAS-2615 URL: https://issues.apache.org/jira/browse/ATLAS-2615 Project: Atlas Issue Type: Improvement Components: atlas-core Affects Versions: 1.0.0 Reporter: Ayub Pathan Fix For: 1.0.0 With current implementation, there is no support to create multiple glossaries with single REST call. This requires multiple REST call, with each call creating one glossary. >From user's point, creating bulk glossaries in one shot could be helpful. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (ATLAS-2616) NPE for GET call of /glossary//categories, when there are no categories associated to glossary
Ayub Pathan created ATLAS-2616: -- Summary: NPE for GET call of /glossary//categories, when there are no categories associated to glossary Key: ATLAS-2616 URL: https://issues.apache.org/jira/browse/ATLAS-2616 Project: Atlas Issue Type: Improvement Components: atlas-core Affects Versions: 1.0.0 Reporter: Ayub Pathan Fix For: 1.0.0 * Two glossaries configured with one glossary having categories {noformat} apathan@Ayub-sys-101:~ > curl -k -u admin:admin https://ctr-e138-1518143905142-256977-01-04.hwx.site:21443/api/atlas/v2/glossary/ | python -m json.tool % Total% Received % Xferd Average Speed TimeTime Time Current Dload Upload Total SpentLeft Speed 100 6330 6330 0386 0 --:--:-- 0:00:01 --:--:-- 386 [ { "guid": "bf43f063-f499-472c-b526-0f81de685e02", "displayName": "true", "shortDescription": "asd", "longDescription": "Glossary of bank - long description", "qualifiedName": "HortoniaBank-11" }, { "guid": "c445e6c3-1113-4d55-acbd-d421c8c98db1", "displayName": "Banking-11", "shortDescription": "sdfds", "longDescription": "sdfdsf", "qualifiedName": "HortoniaBank-11", "categories": [ { "categoryGuid": "59295311-2ef8-400c-9188-ea2adfdca65b", "relationGuid": "39113069-211b-44e1-b924-b0402c997a20", "displayText": "Bank accounts" }, { "categoryGuid": "625e13a6-e7ef-49d2-bfe3-bb31e8b63ef0", "relationGuid": "4366623b-2f7e-43df-83f2-2d438f3fd120", "displayText": "sub" } ] } ] {noformat} * GET request on categories is fine when there are categories associated. {noformat} apathan@Ayub-sys-101:~ > apathan@Ayub-sys-101:~ > curl -k -u admin:admin https://ctr-e138-1518143905142-256977-01-04.hwx.site:21443/api/atlas/v2/glossary/c445e6c3-1113-4d55-acbd-d421c8c98db1/categories | python -m json.tool % Total% Received % Xferd Average Speed TimeTime Time Current Dload Upload Total SpentLeft Speed 100 2710 2710 0 97 0 --:--:-- 0:00:02 --:--:--97 [ { "categoryGuid": "59295311-2ef8-400c-9188-ea2adfdca65b", "relationGuid": "39113069-211b-44e1-b924-b0402c997a20", "displayText": "Bank accounts" }, { "categoryGuid": "625e13a6-e7ef-49d2-bfe3-bb31e8b63ef0", "relationGuid": "4366623b-2f7e-43df-83f2-2d438f3fd120", "displayText": "sub" } ] {noformat} * GET request on categories returns 500 with NPE when there are no categories associated to glossary. {noformat} apathan@Ayub-sys-101:~ > curl -k -u admin:admin https://ctr-e138-1518143905142-256977-01-04.hwx.site:21443/api/atlas/v2/glossary/bf43f063-f499-472c-b526-0f81de685e02/categories | pj % Total% Received % Xferd Average Speed TimeTime Time Current Dload Upload Total SpentLeft Speed 100850850 0 33 0 --:--:-- 0:00:02 --:--:--33 Expecting value: line 1 column 1 (char 0) apathan@Ayub-sys-101:~ > {noformat} {noformat} 2018-04-26 11:59:39,528 ERROR - [pool-2-thread-4 - 29e30189-d093-4245-a623-1c50c5aad766:] ~ Error handling a request: 935035fa3fbef294 (ExceptionMapperUtil:32) java.lang.NullPointerException at java.util.ArrayList.(ArrayList.java:178) at org.apache.atlas.glossary.GlossaryService.getGlossaryCategories(GlossaryService.java:618) at org.apache.atlas.web.rest.GlossaryREST.getGlossaryCategories(GlossaryREST.java:645) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60) at com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$TypeOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:185) at com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75) at com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:302) at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147) at com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:108) at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(Ri
[jira] [Created] (ATLAS-2636) Business catalog: Creating a glossary with incorrect GUID, fails first time. When the same request with the same payload returns "409 conflict" saying "Glossary category
Ayub Pathan created ATLAS-2636: -- Summary: Business catalog: Creating a glossary with incorrect GUID, fails first time. When the same request with the same payload returns "409 conflict" saying "Glossary category with qualifiedName category1 already exists" Key: ATLAS-2636 URL: https://issues.apache.org/jira/browse/ATLAS-2636 Project: Atlas Issue Type: Bug Components: atlas-core Affects Versions: 1.0.0 Reporter: Ayub Pathan Fix For: 1.0.0 * Create a category with below payload, this will fail with 500 internal error. {noformat} { "qualifiedName": "category1", "displayName": "Bank accounts", "anchor": { "glossaryGuid": "-1" } } {noformat} * Repeat the same request again, now atlas throws below error {noformat} { "errorCode": "ATLAS-409-00-00A", "errorMessage": "Glossary category with qualifiedName category1 already exists" } {noformat} Does this mean, the first request succeeded to register the category? -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (ATLAS-2639) UI: Associating a classification to entity from term page results in adding tag to term rather than entity
Ayub Pathan created ATLAS-2639: -- Summary: UI: Associating a classification to entity from term page results in adding tag to term rather than entity Key: ATLAS-2639 URL: https://issues.apache.org/jira/browse/ATLAS-2639 Project: Atlas Issue Type: Bug Components: atlas-core Affects Versions: 1.0.0 Reporter: Ayub Pathan Fix For: 1.0.0 * Create a term(term1) and classification(tag1) * Create an entity(E1) and associate term1 to E1. * In the term page, entity association can be seen, associate tag1 to E1. * This results in associating tag1 to term instead of E1. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (ATLAS-2311) Relationship GET api fails
Ayub Pathan created ATLAS-2311: -- Summary: Relationship GET api fails Key: ATLAS-2311 URL: https://issues.apache.org/jira/browse/ATLAS-2311 Project: Atlas Issue Type: Bug Components: atlas-core Affects Versions: 1.0.0 Reporter: Ayub Pathan Priority: Critical Fix For: 1.0.0 Relationship GET api fails * create simple hive table {noformat} apathan@Ayub-sys-101:~/Downloads > curl 'http://ctr-e135-1512069032975-24737-01-03.hwx.site:21000/api/atlas/v2/entity/guid/4dcd5ddb-b989-4b11-bcda-54b518e60b98?_=1513162047823' -H 'Accept-Encoding: gzip, deflate' -H 'Accept-Language: en-US,en;q=0.9' -H 'Upgrade-Insecure-Requests: 1' -H 'User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/63.0.3239.84 Safari/537.36' -H 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8' -H 'Cache-Control: max-age=0' -H 'Cookie: ATLASSESSIONID=8zvyxbzmb16l1tdpb2w1qzxrp' -H 'Connection: keep-alive' --compressed | python -m json.tool % Total% Received % Xferd Average Speed TimeTime Time Current Dload Upload Total SpentLeft Speed 100 60170 60170 0 5450 0 --:--:-- 0:00:01 --:--:-- 5455 { "entity": { "attributes": { "aliases": null, "columns": [ { "guid": "e3832ba0-2d5e-4c9b-b33b-81b73064798f", "typeName": "hive_column" }, { "guid": "069af03f-d257-441e-9993-98ee463076c6", "typeName": "hive_column" } ], "comment": null, "createTime": 1513160887000, "db": { "guid": "e4e295df-f002-484e-b53e-bbdc9dd920e0", "typeName": "hive_db" }, "description": null, "lastAccessTime": 1513160887000, "name": "t2", "owner": "anonymous", "parameters": { "COLUMN_STATS_ACCURATE": "{\"BASIC_STATS\":\"true\"}", "numFiles": "0", "numRows": "0", "rawDataSize": "0", "totalSize": "0", "transient_lastDdlTime": "1513160887" }, "partitionKeys": null, "qualifiedName": "default.t2@cl1", "retention": 0, "sd": { "guid": "07c32764-9c1a-4360-be12-6b681409c2eb", "typeName": "hive_storagedesc" }, "tableType": "MANAGED_TABLE", "temporary": false, "viewExpandedText": null, "viewOriginalText": null }, "classifications": [], "createTime": 1513160888355, "createdBy": "anonymous", "guid": "4dcd5ddb-b989-4b11-bcda-54b518e60b98", "relationshipAttributes": { "alsoKnownAs": [], "columns": [ { "displayText": "a", "guid": "e3832ba0-2d5e-4c9b-b33b-81b73064798f", "relationshipAttributes": { "typeName": "hive_table_columns" }, "relationshipGuid": "d8882a22-ce59-4930-80b1-f0a21c8b565a", "typeName": "hive_column" }, { "displayText": "b", "guid": "069af03f-d257-441e-9993-98ee463076c6", "relationshipAttributes": { "typeName": "hive_table_columns" }, "relationshipGuid": "825586ff-40a2-4574-93b1-8178227d13bf", "typeName": "hive_column" } ], "db": { "displayText": "default", "guid": "e4e295df-f002-484e-b53e-bbdc9dd920e0", "relationshipAttributes": { "typeName": "hive_db_tables" }, "relationshipGuid": "f2c19451-95d9-4f94-b18d-00d37222e60a", "typeName": "hive_db" }, "embeddedMetadata": [], "externalReferences": [], "knownLocations": [], "managedResources": [], "meanings": [], "partitionKeys": [], "relatedEntity": null, "sd": { "displayText": "default.t2@cl1_storage", "guid": "07c32764-9c1a-4360-be12-6b681409c2eb", "relationshipAttributes": { "typeName": "hive_table_storagedesc" }, "relationshipGuid": "7885a093-582d-45e3-9218-ff584bea7dcd", "typeName": "hive_storagedesc" }, "sinkFromProcesses": [
[jira] [Created] (ATLAS-2429) ValuesMinCount and ValuesMaxCount in attributeDef always defaults to 1 for any value.
Ayub Pathan created ATLAS-2429: -- Summary: ValuesMinCount and ValuesMaxCount in attributeDef always defaults to 1 for any value. Key: ATLAS-2429 URL: https://issues.apache.org/jira/browse/ATLAS-2429 Project: Atlas Issue Type: Bug Components: atlas-core Affects Versions: 1.0.0 Reporter: Ayub Pathan Fix For: 1.0.0 {noformat} curl -u 'admin:admin' -X POST -d '{"structDefs":[{"category":"STRUCT","name":"struct_sample4","attributeDefs":[{"name":"string","typeName":"string","isOptional":false,"cardinality":"SINGLE","valuesMinCount":-2147483648,"valuesMaxCount":2147483647,"isUnique":true,"isIndexable":true}]}]}' -H 'Content-Type:application/json' http://ctr-e137-1514896590304-36955-01-03.hwx.site:21000/api/atlas/v2/types/typedefs | python -m json.tool { "classificationDefs": [], "entityDefs": [], "enumDefs": [], "relationshipDefs": [], "structDefs": [ { "attributeDefs": [ { "cardinality": "SINGLE", "isIndexable": true, "isOptional": false, "isUnique": true, "name": "string", "typeName": "string", "valuesMaxCount": 1, "valuesMinCount": 1 } ], "category": "STRUCT", "createTime": 1517386162718, "createdBy": "admin", "description": "struct_sample4", "guid": "2daf9320-a668-4543-a287-806923360b3e", "name": "struct_sample4", "typeVersion": "1.0", "updateTime": 1517386162718, "updatedBy": "admin", "version": 1 } ] } {noformat} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (ATLAS-2022) Regression : Empty results fetched from GET Basic search query
[ https://issues.apache.org/jira/browse/ATLAS-2022?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayub Pathan reassigned ATLAS-2022: -- Assignee: Sharmadha Sainath > Regression : Empty results fetched from GET Basic search query > -- > > Key: ATLAS-2022 > URL: https://issues.apache.org/jira/browse/ATLAS-2022 > Project: Atlas > Issue Type: Bug > Components: atlas-core >Affects Versions: 0.9-incubating >Reporter: Sharmadha Sainath >Assignee: Sharmadha Sainath >Priority: Blocker > Attachments: ATLAS-2022.patch > > > Basic search query fired as a POST request with attribute , tag filters > fetches correct results. > But Basic query fired as a GET request with query params encoded in the URL , > returns empty results. > For example : Basic query : typeName = "hive_table" query = hive_table in Atlas" > http://localhost:21000/api/atlas/v2/search/basic?typeName=hive_table&query=employee > returns > {code} > { > queryType: "BASIC", > searchParameters: { > query: "employee", > typeName: "hive_table", > excludeDeletedEntities: false, > limit: 0, > offset: 0 > }, > queryText: "employee" > } > {code} > Few commits back , following was the response : > {code} > { > queryType: "BASIC", > type: "hive_table", > entities: [ > { > typeName: "hive_table", > attributes: { > owner: "admin", > qualifiedName: "default.employee@cl1", > name: "employee", > description: null > }, > guid: "253aa208-0415-4e86-8611-3858fad78ede", > status: "ACTIVE", > displayText: "employee", > classificationNames: [ ] > } > ] > } > {code} -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (ATLAS-2022) Regression : Empty results fetched from GET Basic search query
[ https://issues.apache.org/jira/browse/ATLAS-2022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16114315#comment-16114315 ] Ayub Pathan commented on ATLAS-2022: +1. looks good to me > Regression : Empty results fetched from GET Basic search query > -- > > Key: ATLAS-2022 > URL: https://issues.apache.org/jira/browse/ATLAS-2022 > Project: Atlas > Issue Type: Bug > Components: atlas-core >Affects Versions: 0.9-incubating >Reporter: Sharmadha Sainath >Assignee: Sharmadha Sainath >Priority: Blocker > Attachments: ATLAS-2022.patch > > > Basic search query fired as a POST request with attribute , tag filters > fetches correct results. > But Basic query fired as a GET request with query params encoded in the URL , > returns empty results. > For example : Basic query : typeName = "hive_table" query = hive_table in Atlas" > http://localhost:21000/api/atlas/v2/search/basic?typeName=hive_table&query=employee > returns > {code} > { > queryType: "BASIC", > searchParameters: { > query: "employee", > typeName: "hive_table", > excludeDeletedEntities: false, > limit: 0, > offset: 0 > }, > queryText: "employee" > } > {code} > Few commits back , following was the response : > {code} > { > queryType: "BASIC", > type: "hive_table", > entities: [ > { > typeName: "hive_table", > attributes: { > owner: "admin", > qualifiedName: "default.employee@cl1", > name: "employee", > description: null > }, > guid: "253aa208-0415-4e86-8611-3858fad78ede", > status: "ACTIVE", > displayText: "employee", > classificationNames: [ ] > } > ] > } > {code} -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (ATLAS-633) "kafka.common.NoReplicaOnlineException" exception on atlas startup
[ https://issues.apache.org/jira/browse/ATLAS-633?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16128277#comment-16128277 ] Ayub Pathan commented on ATLAS-633: --- [~bill] This issue is seen in embedded mode of Atlas and I believe this is a limitation from ATlas-Kafka integration. Since I am using non-embedded mode of Atlas, I didn't pursue on this route. > "kafka.common.NoReplicaOnlineException" exception on atlas startup > -- > > Key: ATLAS-633 > URL: https://issues.apache.org/jira/browse/ATLAS-633 > Project: Atlas > Issue Type: Bug >Affects Versions: trunk >Reporter: Ayub Pathan > > "kafka.common.NoReplicaOnlineException" exception on atlas startup > {noformat} > 2016-04-05 11:01:54,811 INFO - [main:] ~ [Socket Server on Broker 1], > Started 1 acceptor threads (SocketServer:68) > 2016-04-05 11:01:54,828 INFO - [ExpirationReaper-1:] ~ [ExpirationReaper-1], > Starting (DelayedOperationPurgatory$ExpiredOperationReaper:68) > 2016-04-05 11:01:54,828 INFO - [ExpirationReaper-1:] ~ [ExpirationReaper-1], > Starting (DelayedOperationPurgatory$ExpiredOperationReaper:68) > 2016-04-05 11:01:54,852 INFO - [main:] ~ [Controller 1]: Controller starting > up (KafkaController:68) > 2016-04-05 11:01:54,862 INFO - [main:] ~ Creating /controller (is it secure? > false) (ZKCheckedEphemeral:68) > 2016-04-05 11:01:54,867 INFO - [main:] ~ Result of znode creation is: OK > (ZKCheckedEphemeral:68) > 2016-04-05 11:01:54,868 INFO - [main:] ~ 1 successfully elected as leader > (ZookeeperLeaderElector:68) > 2016-04-05 11:01:54,868 INFO - [main:] ~ [Controller 1]: Broker 1 starting > become controller state transition (KafkaController:68) > 2016-04-05 11:01:54,870 INFO - [main:] ~ [Controller 1]: Initialized > controller epoch to 1 and zk version 0 (KafkaController:68) > 2016-04-05 11:01:54,872 INFO - [main:] ~ [Controller 1]: Controller 1 > incremented epoch to 2 (KafkaController:68) > 2016-04-05 11:01:54,939 INFO - [main:] ~ [Controller 1]: Partitions > undergoing preferred replica election: (KafkaController:68) > 2016-04-05 11:01:54,939 INFO - [main:] ~ [Controller 1]: Partitions that > completed preferred replica election: (KafkaController:68) > 2016-04-05 11:01:54,940 INFO - [main:] ~ [Controller 1]: Resuming preferred > replica election for partitions: (KafkaController:68) > 2016-04-05 11:01:54,943 INFO - [main:] ~ [Controller 1]: Partitions being > reassigned: Map() (KafkaController:68) > 2016-04-05 11:01:54,944 INFO - [main:] ~ [Controller 1]: Partitions already > reassigned: List() (KafkaController:68) > 2016-04-05 11:01:54,945 INFO - [main:] ~ [Controller 1]: Resuming > reassignment of partitions: Map() (KafkaController:68) > 2016-04-05 11:01:54,949 INFO - [main:] ~ [Controller 1]: List of topics to > be deleted: (KafkaController:68) > 2016-04-05 11:01:54,949 INFO - [main:] ~ [Controller 1]: List of topics > ineligible for deletion: ATLAS_HOOK,ATLAS_ENTITIES (KafkaController:68) > 2016-04-05 11:01:54,951 INFO - [main:] ~ [Controller 1]: Currently active > brokers in the cluster: Set() (KafkaController:68) > 2016-04-05 11:01:54,952 INFO - [main:] ~ [Controller 1]: Currently shutting > brokers in the cluster: Set() (KafkaController:68) > 2016-04-05 11:01:54,952 INFO - [main:] ~ [Controller 1]: Current list of > topics in the cluster: Set(ATLAS_ENTITIES, ATLAS_HOOK) (KafkaController:68) > 2016-04-05 11:01:54,956 INFO - [main:] ~ [Replica state machine on > controller 1]: Started replica state machine with initial state -> > Map([Topic=ATLAS_ENTITIES,Partition=0,Replica=1] -> > ReplicaDeletionIneligible, [Topic=ATLAS_HOOK,Partition=0,Replica=1] -> > ReplicaDeletionIneligible) (ReplicaStateMachine:68) > 2016-04-05 11:01:54,977 ERROR - [main:] ~ Controller 1 epoch 2 initiated > state change for partition [ATLAS_HOOK,0] from OfflinePartition to > OnlinePartition failed (logger:103) > kafka.common.NoReplicaOnlineException: No replica for partition > [ATLAS_HOOK,0] is alive. Live brokers are: [Set()], Assigned replicas are: > [List(1)] > at > kafka.controller.OfflinePartitionLeaderSelector.selectLeader(PartitionLeaderSelector.scala:75) > at > kafka.controller.PartitionStateMachine.electLeaderForPartition(PartitionStateMachine.scala:345) > at > kafka.controller.PartitionStateMachine.kafka$controller$PartitionStateMachine$$handleStateChange(PartitionStateMachine.scala:205) > at > kafka.controller.PartitionStateMachine$$anonfun$triggerOnlinePartitionStateChange$3.apply(PartitionStateMachine.scala:120) > at > kafka.controller.PartitionStateMachine$$anonfun$triggerOnlinePartitionStateChange$3.apply(PartitionStateMachine.scala:117) > at > scala.collection.TraversableLike$WithFilter$$anonfun$foreach$1.apply(TraversableLike.scal
[jira] [Commented] (ATLAS-2128) UI Regression : DSL Results pagination doesn't spawn more than 2 pages
[ https://issues.apache.org/jira/browse/ATLAS-2128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16161000#comment-16161000 ] Ayub Pathan commented on ATLAS-2128: This issue is introduced as part of https://github.com/apache/atlas/commit/d0beadf2e4a83bce17b8c42e7a584a035cf416c2 this commit. To be more precise, this line might be causing the issue.: https://github.com/apache/atlas/blob/master/dashboardv2/public/js/views/search/SearchResultLayoutView.js#L366 {code} if (!model.entities && that.offset > that.limit) { that.ui.nextData.attr('disabled', true); that.offset = that.offset - that.limit; that.hideLoader(); return; } {code} CC [~kevalbhatt] > UI Regression : DSL Results pagination doesn't spawn more than 2 pages > -- > > Key: ATLAS-2128 > URL: https://issues.apache.org/jira/browse/ATLAS-2128 > Project: Atlas > Issue Type: Bug > Components: atlas-webui >Affects Versions: 0.9-incubating, 0.8.1-incubating >Reporter: Sharmadha Sainath >Priority: Blocker > > 1. Created 300 hive_tables. > 2. On Basic Search in UI , fired search typename= hive_table > 3. hive_table results were fetched across 12 pages - 25 entities in each page > - which is the correct behavior. > 4.On Advanced Search in UI , fired search typename = hive_table > * In the 1st page , 1- 25 results were shown. > * On clicking next , in the 2nd page , 26 - 50 results were shown > * On clicking next , Expected to land in 3rd page with 51 - 75 > results , but , 26-50 results were loaded again and the Next Button was > disabled. > Calls made to the Atlas backend seem to be correct. On clicking "Next" each > time , correct set of results were fetched. For example , call was made to > fetch entities from offset 50 , limit = 25 , but the UI still showed 26-50 > results and disabled "Next" button. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Comment Edited] (ATLAS-2128) UI Regression : DSL Results pagination doesn't spawn more than 2 pages
[ https://issues.apache.org/jira/browse/ATLAS-2128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16161000#comment-16161000 ] Ayub Pathan edited comment on ATLAS-2128 at 9/11/17 10:02 AM: -- Looks like, this issue is introduced as part of https://github.com/apache/atlas/commit/d0beadf2e4a83bce17b8c42e7a584a035cf416c2 this commit. To be more precise, this line might be causing the issue.: https://github.com/apache/atlas/blob/master/dashboardv2/public/js/views/search/SearchResultLayoutView.js#L366 {code} if (!model.entities && that.offset > that.limit) { that.ui.nextData.attr('disabled', true); that.offset = that.offset - that.limit; that.hideLoader(); return; } {code} CC [~kevalbhatt] was (Author: ayubkhan): This issue is introduced as part of https://github.com/apache/atlas/commit/d0beadf2e4a83bce17b8c42e7a584a035cf416c2 this commit. To be more precise, this line might be causing the issue.: https://github.com/apache/atlas/blob/master/dashboardv2/public/js/views/search/SearchResultLayoutView.js#L366 {code} if (!model.entities && that.offset > that.limit) { that.ui.nextData.attr('disabled', true); that.offset = that.offset - that.limit; that.hideLoader(); return; } {code} CC [~kevalbhatt] > UI Regression : DSL Results pagination doesn't spawn more than 2 pages > -- > > Key: ATLAS-2128 > URL: https://issues.apache.org/jira/browse/ATLAS-2128 > Project: Atlas > Issue Type: Bug > Components: atlas-webui >Affects Versions: 0.9-incubating, 0.8.1-incubating >Reporter: Sharmadha Sainath >Priority: Blocker > > 1. Created 300 hive_tables. > 2. On Basic Search in UI , fired search typename= hive_table > 3. hive_table results were fetched across 12 pages - 25 entities in each page > - which is the correct behavior. > 4.On Advanced Search in UI , fired search typename = hive_table > * In the 1st page , 1- 25 results were shown. > * On clicking next , in the 2nd page , 26 - 50 results were shown > * On clicking next , Expected to land in 3rd page with 51 - 75 > results , but , 26-50 results were loaded again and the Next Button was > disabled. > Calls made to the Atlas backend seem to be correct. On clicking "Next" each > time , correct set of results were fetched. For example , call was made to > fetch entities from offset 50 , limit = 25 , but the UI still showed 26-50 > results and disabled "Next" button. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Resolved] (ATLAS-1941) HBaseStoreManager is copying host list from root.storage.hostname to hbase.zookeeper.quorum, which has one incorrect zookeeper host
[ https://issues.apache.org/jira/browse/ATLAS-1941?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayub Pathan resolved ATLAS-1941. Resolution: Duplicate This issue is addressed as part of AMBARI-21902 > HBaseStoreManager is copying host list from root.storage.hostname to > hbase.zookeeper.quorum, which has one incorrect zookeeper host > --- > > Key: ATLAS-1941 > URL: https://issues.apache.org/jira/browse/ATLAS-1941 > Project: Atlas > Issue Type: Bug > Components: atlas-core >Affects Versions: 0.9-incubating >Reporter: Ayub Pathan >Priority: Minor > Fix For: 0.9-incubating > > > While Atlas startup, HBaseStoreManager is copying host list from > root.storage.hostname to hbase.zookeeper.quorum, which has one incorrect > zookeeper host as well. > {noformat} > 2017-07-11 12:13:51,341 INFO - [main:] ~ Not running setup per configuration > atlas.server.run.setup.on.start. (SetupSteps:189) > 2017-07-11 12:13:52,332 INFO - [main:] ~ JSR-330 'javax.inject.Inject' > annotation found and supported for autowiring > (AutowiredAnnotationBeanPostProcessor:155) > 2017-07-11 12:13:53,038 INFO - [main:] ~ Instantiated HBase compatibility > layer supporting runtime HBase version 1.1.2.2.6.2.0-71: > com.thinkaurelius.titan.diskstorage.hbase.HBaseCompat1_1 > (HBaseCompatLoader:69) > 2017-07-11 12:13:53,108 INFO - [main:] ~ Copied host list from > root.storage.hostname to hbase.zookeeper.quorum: > ctr-e133-1493418528701-179285-01-05.hwx.site,ctr-e133-1493418528701-179285-01-04.hwx.site,ctr-e133-1493418528701-179285-01-02.hwx.site > (HBaseStoreManager:320) > {noformat} > In the above log, the correct zookeeper hosts are > ctr-e133-1493418528701-179285-01-05.hwx.site,ctr-e133-1493418528701-179285-01-02.hwx.site. > But, as we can see HBaseStoreManager is adding > ctr-e133-1493418528701-179285-01-04.hwx.site as well which is incorrect. > As an outcome of this, zookeeper connection to this host fails. > {noformat} > 2017-07-11 12:13:53,399 INFO - > [main-SendThread(ctr-e133-1493418528701-179285-01-04.hwx.site:2181):] ~ > Opening socket connection to server > ctr-e133-1493418528701-179285-01-04.hwx.site/172.27.9.2:2181. Will > attempt to SASL-authenticate using Login Context section 'Client' > (ClientCnxn:1019) > 2017-07-11 12:13:53,412 WARN - > [main-SendThread(ctr-e133-1493418528701-179285-01-04.hwx.site:2181):] ~ > Session 0x0 for server null, unexpected error, closing socket connection and > attempting reconnect (ClientCnxn:1146) > java.net.ConnectException: Connection refused > at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) > at > sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717) > at > org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:361) > at > org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1125) > {noformat} -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Assigned] (ATLAS-1941) HBaseStoreManager is copying host list from root.storage.hostname to hbase.zookeeper.quorum, which has one incorrect zookeeper host
[ https://issues.apache.org/jira/browse/ATLAS-1941?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayub Pathan reassigned ATLAS-1941: -- Assignee: Ayub Pathan > HBaseStoreManager is copying host list from root.storage.hostname to > hbase.zookeeper.quorum, which has one incorrect zookeeper host > --- > > Key: ATLAS-1941 > URL: https://issues.apache.org/jira/browse/ATLAS-1941 > Project: Atlas > Issue Type: Bug > Components: atlas-core >Affects Versions: 0.9-incubating >Reporter: Ayub Pathan >Assignee: Ayub Pathan >Priority: Minor > Fix For: 0.9-incubating > > > While Atlas startup, HBaseStoreManager is copying host list from > root.storage.hostname to hbase.zookeeper.quorum, which has one incorrect > zookeeper host as well. > {noformat} > 2017-07-11 12:13:51,341 INFO - [main:] ~ Not running setup per configuration > atlas.server.run.setup.on.start. (SetupSteps:189) > 2017-07-11 12:13:52,332 INFO - [main:] ~ JSR-330 'javax.inject.Inject' > annotation found and supported for autowiring > (AutowiredAnnotationBeanPostProcessor:155) > 2017-07-11 12:13:53,038 INFO - [main:] ~ Instantiated HBase compatibility > layer supporting runtime HBase version 1.1.2.2.6.2.0-71: > com.thinkaurelius.titan.diskstorage.hbase.HBaseCompat1_1 > (HBaseCompatLoader:69) > 2017-07-11 12:13:53,108 INFO - [main:] ~ Copied host list from > root.storage.hostname to hbase.zookeeper.quorum: > ctr-e133-1493418528701-179285-01-05.hwx.site,ctr-e133-1493418528701-179285-01-04.hwx.site,ctr-e133-1493418528701-179285-01-02.hwx.site > (HBaseStoreManager:320) > {noformat} > In the above log, the correct zookeeper hosts are > ctr-e133-1493418528701-179285-01-05.hwx.site,ctr-e133-1493418528701-179285-01-02.hwx.site. > But, as we can see HBaseStoreManager is adding > ctr-e133-1493418528701-179285-01-04.hwx.site as well which is incorrect. > As an outcome of this, zookeeper connection to this host fails. > {noformat} > 2017-07-11 12:13:53,399 INFO - > [main-SendThread(ctr-e133-1493418528701-179285-01-04.hwx.site:2181):] ~ > Opening socket connection to server > ctr-e133-1493418528701-179285-01-04.hwx.site/172.27.9.2:2181. Will > attempt to SASL-authenticate using Login Context section 'Client' > (ClientCnxn:1019) > 2017-07-11 12:13:53,412 WARN - > [main-SendThread(ctr-e133-1493418528701-179285-01-04.hwx.site:2181):] ~ > Session 0x0 for server null, unexpected error, closing socket connection and > attempting reconnect (ClientCnxn:1146) > java.net.ConnectException: Connection refused > at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) > at > sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717) > at > org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:361) > at > org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1125) > {noformat} -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (ATLAS-2147) querying supertype "Asset" with multiple filters fails with "unresolved datatype. Can not resolve due to unresolved children"
Ayub Pathan created ATLAS-2147: -- Summary: querying supertype "Asset" with multiple filters fails with "unresolved datatype. Can not resolve due to unresolved children" Key: ATLAS-2147 URL: https://issues.apache.org/jira/browse/ATLAS-2147 Project: Atlas Issue Type: Bug Components: atlas-core Affects Versions: 0.9.1 Reporter: Ayub Pathan Fix For: 0.9.1 Example query with multiple filters: Asset where (name="") or (qualifiesName="") Stacktrace: {noformat} 2017-09-19 14:37:15,166 ERROR - [pool-1-thread-9 - 1ca16b19-634e-48bb-b872-20025c210b75:] ~ graph rollback due to exception (GraphTransactionInterceptor:71) org.apache.atlas.query.Expressions$UnresolvedException: Unresolved datatype. Can not resolve due to unresolved children, expression: Asset where (name = "h7") or (qualifiedName = "default.h7@primary") at org.apache.atlas.query.Expressions$FilterExpression.dataType$lzycompute(Expressions.scala:708) at org.apache.atlas.query.Expressions$FilterExpression.dataType(Expressions.scala:706) at org.apache.atlas.query.Resolver.apply(Resolver.scala:48) at org.apache.atlas.query.Resolver.apply(Resolver.scala:25) at scala.PartialFunction$class.applyOrElse(PartialFunction.scala:123) at org.apache.atlas.query.Resolver.applyOrElse(Resolver.scala:25) at org.apache.atlas.query.Expressions$Expression$class.transformUp(Expressions.scala:201) at org.apache.atlas.query.Expressions$IdExpression.transformUp(Expressions.scala:404) at org.apache.atlas.query.Expressions$Expression$$anonfun$4.apply(Expressions.scala:211) at scala.collection.Iterator$$anon$11.next(Iterator.scala:409) at scala.collection.Iterator$class.foreach(Iterator.scala:893) at scala.collection.AbstractIterator.foreach(Iterator.scala:1336) at scala.collection.generic.Growable$class.$plus$plus$eq(Growable.scala:59) at scala.collection.mutable.ArrayBuffer.$plus$plus$eq(ArrayBuffer.scala:104) at scala.collection.mutable.ArrayBuffer.$plus$plus$eq(ArrayBuffer.scala:48) at scala.collection.TraversableOnce$class.to(TraversableOnce.scala:310) at scala.collection.AbstractIterator.to(Iterator.scala:1336) at scala.collection.TraversableOnce$class.toBuffer(TraversableOnce.scala:302) at scala.collection.AbstractIterator.toBuffer(Iterator.scala:1336) at scala.collection.TraversableOnce$class.toArray(TraversableOnce.scala:289) at scala.collection.AbstractIterator.toArray(Iterator.scala:1336) at org.apache.atlas.query.Expressions$Expression$class.transformChildrenUp(Expressions.scala:240) at org.apache.atlas.query.Expressions$BinaryExpression.transformChildrenUp(Expressions.scala:367) at org.apache.atlas.query.Expressions$Expression$class.transformUp(Expressions.scala:199) at org.apache.atlas.query.Expressions$BinaryExpression.transformUp(Expressions.scala:367) at org.apache.atlas.query.Expressions$Expression$$anonfun$4$$anonfun$apply$3.apply(Expressions.scala:229) at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234) at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234) at scala.collection.immutable.List.foreach(List.scala:381) at scala.collection.TraversableLike$class.map(TraversableLike.scala:234) at scala.collection.immutable.List.map(List.scala:285) at org.apache.atlas.query.Expressions$Expression$$anonfun$4.apply(Expressions.scala:227) at scala.collection.Iterator$$anon$11.next(Iterator.scala:409) at scala.collection.Iterator$class.foreach(Iterator.scala:893) at scala.collection.AbstractIterator.foreach(Iterator.scala:1336) at scala.collection.generic.Growable$class.$plus$plus$eq(Growable.scala:59) at scala.collection.mutable.ArrayBuffer.$plus$plus$eq(ArrayBuffer.scala:104) at scala.collection.mutable.ArrayBuffer.$plus$plus$eq(ArrayBuffer.scala:48) at scala.collection.TraversableOnce$class.to(TraversableOnce.scala:310) at scala.collection.AbstractIterator.to(Iterator.scala:1336) at scala.collection.TraversableOnce$class.toBuffer(TraversableOnce.scala:302) at scala.collection.AbstractIterator.toBuffer(Iterator.scala:1336) at scala.collection.TraversableOnce$class.toArray(TraversableOnce.scala:289) at scala.collection.AbstractIterator.toArray(Iterator.scala:1336) at org.apache.atlas.query.Expressions$Expression$class.transformChildrenUp(Expressions.scala:240) at org.apache.atlas.query.Expressions$LogicalExpression.transformChildrenUp(Expressions.scala:684) at org.apache.atlas.query.Expressions$Expression$class.transformUp(Expressions.scala:199) at org.apache.at
[jira] [Created] (ATLAS-2152) Saved Search: UI: Atlas UI not able to load the saved search
Ayub Pathan created ATLAS-2152: -- Summary: Saved Search: UI: Atlas UI not able to load the saved search Key: ATLAS-2152 URL: https://issues.apache.org/jira/browse/ATLAS-2152 Project: Atlas Issue Type: Bug Components: atlas-webui Affects Versions: 0.9.1 Reporter: Ayub Pathan Priority: Critical Fix For: 0.9.1 After saving the search query, Atlas UI unable to load the saved search. Below exception is observed on the browser console {noformat} CommonViewFunction.js:487 Uncaught TypeError: Cannot read property 'toJSON' of undefined at CommonViewFunction.js:487 at Function.s.each.s.forEach (underscore-min.js:5) at Object.g.generateUrlFromSaveSearchObject (CommonViewFunction.js:476) at L.d.applyValue (SearchLayoutView.js:155) at L.d.stateChange (SaveSearchItemView.js:60) at HTMLLIElement.dispatch (jquery.min.js:3) at HTMLLIElement.q.handle (jquery.min.js:3) {noformat} Tested on master. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (ATLAS-2152) Saved Search: UI: Atlas UI not able to load the saved search
[ https://issues.apache.org/jira/browse/ATLAS-2152?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayub Pathan updated ATLAS-2152: --- Attachment: SavedSearchSample.json Attaching the sample savedsearch.json [~kevalbhatt] [~madhan.neethiraj] > Saved Search: UI: Atlas UI not able to load the saved search > > > Key: ATLAS-2152 > URL: https://issues.apache.org/jira/browse/ATLAS-2152 > Project: Atlas > Issue Type: Bug > Components: atlas-webui >Affects Versions: 0.9.1 >Reporter: Ayub Pathan >Priority: Critical > Fix For: 0.9.1 > > Attachments: SavedSearchSample.json > > > After saving the search query, Atlas UI unable to load the saved search. > Below exception is observed on the browser console > {noformat} > CommonViewFunction.js:487 Uncaught TypeError: Cannot read property 'toJSON' > of undefined > at CommonViewFunction.js:487 > at Function.s.each.s.forEach (underscore-min.js:5) > at Object.g.generateUrlFromSaveSearchObject (CommonViewFunction.js:476) > at L.d.applyValue (SearchLayoutView.js:155) > at L.d.stateChange (SaveSearchItemView.js:60) > at HTMLLIElement.dispatch (jquery.min.js:3) > at HTMLLIElement.q.handle (jquery.min.js:3) > {noformat} > Tested on master. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (ATLAS-2155) Attributes with null values are not filtered using search parameters
Ayub Pathan created ATLAS-2155: -- Summary: Attributes with null values are not filtered using search parameters Key: ATLAS-2155 URL: https://issues.apache.org/jira/browse/ATLAS-2155 Project: Atlas Issue Type: Bug Components: atlas-core Affects Versions: 0.9.1 Reporter: Ayub Pathan Fix For: 0.9.1 For example: Below example shows attrbutes like "Description" has nullvalues {noformat} apathan@Ayub-sys-101:~/Hortonworks > curl 'http://localhost:21000/api/atlas/v2/search/basic' -H 'Origin: http://localhost:21000' -H 'Accept-Encoding: gzip, deflate, br' -H 'X-XSRF-HEADER: ""' -H 'Authorization: Basic YWRtaW46YWRtaW4=' -H 'Accept-Language: en-US,en;q=0.8' -H 'Cookie: ATLASSESSIONID=9hm04vl07k7f1vmcmeq9q6kg9' -H 'Connection: keep-alive' -H 'Pragma: no-cache' -H 'X-Requested-With: XMLHttpRequest' -H 'User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/60.0.3112.113 Safari/537.36' -H 'Content-Type: application/json' -H 'Accept: application/json, text/javascript, */*; q=0.01' -H 'Cache-Control: no-cache' -H 'Referer: http://localhost:21000/index.html' --data-binary '{"excludeDeletedEntities":true,"entityFilters":{"condition":"AND","criterion":[{"attributeName":"owner","operator":"eq","attributeValue":"apathan"}]},"tagFilters":null,"attributes":["qualifiedName"],"query":null,"limit":25,"offset":0,"typeName":"hive_table","classification":null}' --compressed | python -m json.tool % Total% Received % Xferd Average Speed TimeTime Time Current Dload Upload Total SpentLeft Speed 100 11050 825 100 280 13953 4735 --:--:-- --:--:-- --:--:-- 14224 { "entities": [ { "attributes": { "createTime": 1505475604641, "description": null, "name": "h5", "owner": "apathan", "qualifiedName": "default.h5@primary" }, "classificationNames": [], "displayText": "h5", "guid": "7a8955bf-7395-454f-9d67-ea3ba01529bf", "status": "ACTIVE", "typeName": "hive_table" }, { "attributes": { "createTime": 1505475721717, "description": null, "name": "h7", "owner": "apathan", "qualifiedName": "default.h7@primary" }, "classificationNames": [ "abc", "dsfads", "sdf" ], "displayText": "h7", "guid": "82d6de4e-d808-4c88-8207-be5a58fbbb4e", "status": "ACTIVE", "typeName": "hive_table" } ], "queryType": "BASIC", "searchParameters": { "attributes": [ "qualifiedName" ], "entityFilters": { "condition": "AND", "criterion": [ { "attributeName": "owner", "attributeValue": "apathan", "operator": "eq" } ] }, "excludeDeletedEntities": true, "limit": 25, "offset": 0, "typeName": "hive_table" } } {noformat} *Filtering those attributes using any operand will not take effect and it will always result empty response* example: {noformat} apathan@Ayub-sys-101:~/Hortonworks > curl 'http://localhost:21000/api/atlas/v2/search/basic' -H 'Origin: http://localhost:21000' -H 'Accept-Encoding: gzip, deflate, br' -H 'X-XSRF-HEADER: ""' -H 'Authorization: Basic YWRtaW46YWRtaW4=' -H 'Accept-Language: en-US,en;q=0.8' -H 'Cookie: ATLASSESSIONID=9hm04vl07k7f1vmcmeq9q6kg9' -H 'Connection: keep-alive' -H 'Pragma: no-cache' -H 'X-Requested-With: XMLHttpRequest' -H 'User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/60.0.3112.113 Safari/537.36' -H 'Content-Type: application/json' -H 'Accept: application/json, text/javascript, */*; q=0.01' -H 'Cache-Control: no-cache' -H 'Referer: http://localhost:21000/index.html' --data-binary '{"excludeDeletedEntities":true,"entityFilters":{"condition":"AND","criterion":[{"attributeName":"description","operator":"neq","attributeValue":"ac"}]},"tagFilters":null,"attributes":["qualifiedName"],"query":null,"limit":25,"offset":0,"typeName":"hive_table","classification":null}' --compressed | python -m json.tool % Total% Received % Xferd Average Speed TimeTime Time Current Dload Upload Total SpentLeft Speed 100 5520 270 100 282 4562 4765 --:--:-- --:--:-- --:--:-- 4862 { "queryType": "BASIC", "searchParameters": { "attributes": [ "qualifiedName" ], "entityFilters": {
[jira] [Commented] (ATLAS-2164) Unable to execute the saved search query using its GUID
[ https://issues.apache.org/jira/browse/ATLAS-2164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16176219#comment-16176219 ] Ayub Pathan commented on ATLAS-2164: [~ssainath] I dont see any reference to below API {noformat} http://localhost:21000/api/atlas/v2/search/saved/execute/ {noformat} Can you please check? > Unable to execute the saved search query using its GUID > --- > > Key: ATLAS-2164 > URL: https://issues.apache.org/jira/browse/ATLAS-2164 > Project: Atlas > Issue Type: Bug > Components: atlas-core >Affects Versions: 0.9.1 >Reporter: Sharmadha Sainath >Priority: Critical > > 1.Created and saved 2 searches - one basic , one Advanced. > Following is the response of /api/atlas/v2/search/saved : > {code} > [ >{ > "guid":"153e89bb-eabb-4cc6-9cda-07c7ed084bac", > "ownerName":"admin", > "name":"basic_search1", > "searchType":"BASIC", > "searchParameters":{ > "query":"default", > "typeName":"Asset", > "classification":"tag3", > "excludeDeletedEntities":true, > "limit":0, > "offset":0 > } >}, >{ > "guid":"19be50af-bba1-424f-80e1-c49681b2bdeb", > "ownerName":"admin", > "name":"dsl_search_1", > "searchType":"ADVANCED", > "searchParameters":{ > "query":"where name=\"h1\"", > "typeName":"hdfs_path", > "excludeDeletedEntities":true, > "limit":0, > "offset":0 > } >} > ] > {code} > 2.Fired the following to execute basic saved query using its GUID: > {code} > http://localhost:21000/api/atlas/v2/search/saved/execute/153e89bb-eabb-4cc6-9cda-07c7ed084bac > {code} > Response : > {code} > { > errorCode: "ATLAS-404-00-009", > errorMessage: "Instance __AtlasUserSavedSearch with unique attribute > {uniqueName=admin:153e89bb-eabb-4cc6-9cda-07c7ed084bac:BASIC} does not exist" > } > {code} > Note : Logged in as admin while executing the "execute" query. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Issue Comment Deleted] (ATLAS-2164) Unable to execute the saved search query using its GUID
[ https://issues.apache.org/jira/browse/ATLAS-2164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayub Pathan updated ATLAS-2164: --- Comment: was deleted (was: [~ssainath] I dont see any reference to below API {noformat} http://localhost:21000/api/atlas/v2/search/saved/execute/ {noformat} Can you please check?) > Unable to execute the saved search query using its GUID > --- > > Key: ATLAS-2164 > URL: https://issues.apache.org/jira/browse/ATLAS-2164 > Project: Atlas > Issue Type: Bug > Components: atlas-core >Affects Versions: 0.9.1 >Reporter: Sharmadha Sainath >Priority: Critical > > 1.Created and saved 2 searches - one basic , one Advanced. > Following is the response of /api/atlas/v2/search/saved : > {code} > [ >{ > "guid":"153e89bb-eabb-4cc6-9cda-07c7ed084bac", > "ownerName":"admin", > "name":"basic_search1", > "searchType":"BASIC", > "searchParameters":{ > "query":"default", > "typeName":"Asset", > "classification":"tag3", > "excludeDeletedEntities":true, > "limit":0, > "offset":0 > } >}, >{ > "guid":"19be50af-bba1-424f-80e1-c49681b2bdeb", > "ownerName":"admin", > "name":"dsl_search_1", > "searchType":"ADVANCED", > "searchParameters":{ > "query":"where name=\"h1\"", > "typeName":"hdfs_path", > "excludeDeletedEntities":true, > "limit":0, > "offset":0 > } >} > ] > {code} > 2.Fired the following to execute basic saved query using its GUID: > {code} > http://localhost:21000/api/atlas/v2/search/saved/execute/153e89bb-eabb-4cc6-9cda-07c7ed084bac > {code} > Response : > {code} > { > errorCode: "ATLAS-404-00-009", > errorMessage: "Instance __AtlasUserSavedSearch with unique attribute > {uniqueName=admin:153e89bb-eabb-4cc6-9cda-07c7ed084bac:BASIC} does not exist" > } > {code} > Note : Logged in as admin while executing the "execute" query. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (ATLAS-2175) Infinite refresh scroll appears when a search query is saved.
Ayub Pathan created ATLAS-2175: -- Summary: Infinite refresh scroll appears when a search query is saved. Key: ATLAS-2175 URL: https://issues.apache.org/jira/browse/ATLAS-2175 Project: Atlas Issue Type: Bug Components: atlas-webui Affects Versions: 0.9-incubating Reporter: Ayub Pathan Fix For: 0.9-incubating Attachments: bug-snap.mp4 Attaching a recording for reference. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (ATLAS-2177) offset is not reset when search by type is changed, leads to undesired results.
Ayub Pathan created ATLAS-2177: -- Summary: offset is not reset when search by type is changed, leads to undesired results. Key: ATLAS-2177 URL: https://issues.apache.org/jira/browse/ATLAS-2177 Project: Atlas Issue Type: Bug Components: atlas-webui Affects Versions: 0.9-incubating Reporter: Ayub Pathan Priority: Critical Fix For: 0.9-incubating Attachments: bug-snap2.mp4 Attaching recording for reference. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (ATLAS-2186) Atlas not responding for long time after startup
Ayub Pathan created ATLAS-2186: -- Summary: Atlas not responding for long time after startup Key: ATLAS-2186 URL: https://issues.apache.org/jira/browse/ATLAS-2186 Project: Atlas Issue Type: Bug Components: atlas-core Affects Versions: 0.8.1 Reporter: Ayub Pathan Priority: Blocker Fix For: 0.8.1 Atlas with latest bits is not responding for any query, the request times out. This behavior can be seen for more than ~15 minutes after atlas startup and everything will be back to normal after this period. I suspect one of the threads is looping or hanging which is resulting in this state. Atlas logs does not show any suspicious information (checked with debug logs as well). I took a thread dump of Atlas, attaching the dump file and report below. I strongly suspect, this might be an issue with newer jetty version introduced recently. Thread dump report: http://fastthread.io/my-thread-report.jsp?p=c2hhcmVkLzIwMTcvMTAvMy8tLWF0bGFzLjIwMTcxMDAzLTEyMjUzMS5vdXQtLTEyLTE2LTQ2 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (ATLAS-2186) Atlas not responding for long time after startup
[ https://issues.apache.org/jira/browse/ATLAS-2186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayub Pathan updated ATLAS-2186: --- Attachment: atlas.20171003-122531.out Attaching thread dump file for reference. > Atlas not responding for long time after startup > > > Key: ATLAS-2186 > URL: https://issues.apache.org/jira/browse/ATLAS-2186 > Project: Atlas > Issue Type: Bug > Components: atlas-core >Affects Versions: 0.8.1 >Reporter: Ayub Pathan >Priority: Blocker > Fix For: 0.8.1 > > Attachments: atlas.20171003-122531.out > > > Atlas with latest bits is not responding for any query, the request times > out. This behavior can be seen for more than ~15 minutes after atlas startup > and everything will be back to normal after this period. > I suspect one of the threads is looping or hanging which is resulting in this > state. Atlas logs does not show any suspicious information (checked with > debug logs as well). I took a thread dump of Atlas, attaching the dump file > and report below. I strongly suspect, this might be an issue with newer jetty > version introduced recently. > Thread dump report: > http://fastthread.io/my-thread-report.jsp?p=c2hhcmVkLzIwMTcvMTAvMy8tLWF0bGFzLjIwMTcxMDAzLTEyMjUzMS5vdXQtLTEyLTE2LTQ2 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (ATLAS-2188) UI can trim whitespaces at the beginning or end of search query
Ayub Pathan created ATLAS-2188: -- Summary: UI can trim whitespaces at the beginning or end of search query Key: ATLAS-2188 URL: https://issues.apache.org/jira/browse/ATLAS-2188 Project: Atlas Issue Type: Bug Components: atlas-webui Affects Versions: 0.8.2 Reporter: Ayub Pathan For example, the below query has trailing space in the query resulting to not returning anything, but if you see semantically this query seems to be correct. If a user by mistake enters a whitespace at the beginning or end of search query, this will result in null response, which seems to be very strict. Instead, UI can trim the whitespaces at the beginning or end of the query. Please let me know your thoughts. query json sample: {noformat} {"excludeDeletedEntities":true,"entityFilters":{"condition":"AND","criterion":[{"attributeName":"name","operator":"neq","attributeValue":"123"},{"attributeName":"name","operator":"startsWith","attributeValue":" hbase"},{"attributeName":"owner","operator":"startsWith","attributeValue":"hrt_qa"}]},"tagFilters":null,"attributes":[],"query":null,"limit":25,"offset":0,"typeName":"hbase_table","classification":null} {noformat} -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (ATLAS-2190) Remove duplicate dependency blocks, resulting in signature
Ayub Pathan created ATLAS-2190: -- Summary: Remove duplicate dependency blocks, resulting in signature Key: ATLAS-2190 URL: https://issues.apache.org/jira/browse/ATLAS-2190 Project: Atlas Issue Type: Bug Reporter: Ayub Pathan -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (ATLAS-2190) Remove duplicate dependency blocks with different versions, resulting in atlas startup failure with signature mismatch exception
[ https://issues.apache.org/jira/browse/ATLAS-2190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayub Pathan updated ATLAS-2190: --- Affects Version/s: 0.8.2 Priority: Blocker (was: Major) Fix Version/s: 0.8.2 Description: In the current pom, below two are duplicate entries resulting in atlas startup failure {noformat} - -com.sun.jersey -jersey-servlet -${jersey.version} - - - -javax.servlet.jsp -jsp-api -2.0 - {noformat} Component/s: atlas-core Summary: Remove duplicate dependency blocks with different versions, resulting in atlas startup failure with signature mismatch exception (was: Remove duplicate dependency blocks, resulting in signature ) > Remove duplicate dependency blocks with different versions, resulting in > atlas startup failure with signature mismatch exception > > > Key: ATLAS-2190 > URL: https://issues.apache.org/jira/browse/ATLAS-2190 > Project: Atlas > Issue Type: Bug > Components: atlas-core >Affects Versions: 0.8.2 >Reporter: Ayub Pathan >Priority: Blocker > Fix For: 0.8.2 > > > In the current pom, below two are duplicate entries resulting in atlas > startup failure > {noformat} > - > -com.sun.jersey > -jersey-servlet > -${jersey.version} > - > - > - > -javax.servlet.jsp > -jsp-api > -2.0 > - > {noformat} -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Assigned] (ATLAS-2190) Remove duplicate dependency blocks with different versions, resulting in atlas startup failure with signature mismatch exception
[ https://issues.apache.org/jira/browse/ATLAS-2190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayub Pathan reassigned ATLAS-2190: -- Assignee: Ayub Pathan > Remove duplicate dependency blocks with different versions, resulting in > atlas startup failure with signature mismatch exception > > > Key: ATLAS-2190 > URL: https://issues.apache.org/jira/browse/ATLAS-2190 > Project: Atlas > Issue Type: Bug > Components: atlas-core >Affects Versions: 0.8.2 >Reporter: Ayub Pathan >Assignee: Ayub Pathan >Priority: Blocker > Fix For: 0.8.2 > > > In the current pom, below two are duplicate entries resulting in atlas > startup failure > {noformat} > - > -com.sun.jersey > -jersey-servlet > -${jersey.version} > - > - > - > -javax.servlet.jsp > -jsp-api > -2.0 > - > {noformat} -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (ATLAS-2190) Remove duplicate dependency blocks with different versions, resulting in atlas startup failure with signature mismatch exception
[ https://issues.apache.org/jira/browse/ATLAS-2190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayub Pathan updated ATLAS-2190: --- Attachment: ATLAS-2190-branch-0.8.patch > Remove duplicate dependency blocks with different versions, resulting in > atlas startup failure with signature mismatch exception > > > Key: ATLAS-2190 > URL: https://issues.apache.org/jira/browse/ATLAS-2190 > Project: Atlas > Issue Type: Bug > Components: atlas-core >Affects Versions: 0.8.2 >Reporter: Ayub Pathan >Assignee: Ayub Pathan >Priority: Blocker > Fix For: 0.8.2 > > Attachments: ATLAS-2190-branch-0.8.patch > > > In the current pom, below two are duplicate entries resulting in atlas > startup failure > {noformat} > - > -com.sun.jersey > -jersey-servlet > -${jersey.version} > - > - > - > -javax.servlet.jsp > -jsp-api > -2.0 > - > {noformat} -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (ATLAS-2195) Stale transaction eviction errors observed in atlas application logs
Ayub Pathan created ATLAS-2195: -- Summary: Stale transaction eviction errors observed in atlas application logs Key: ATLAS-2195 URL: https://issues.apache.org/jira/browse/ATLAS-2195 Project: Atlas Issue Type: Bug Components: atlas-core Affects Versions: 0.8.2 Reporter: Ayub Pathan Priority: Critical Fix For: 0.8.2 {noformat} 2017-10-09 20:18:20,728 ERROR - [Thread-371:] ~ Evicted [258@ac1b1980131887-ctr-e134-1499953498516-209460-01-04-hwx-site1] from cache but waiting too long for transactions to close. Stale transaction alert on: [standardtitantx[null], standardtitantx[null]] (ManagementLogger:189) 2017-10-09 20:18:20,738 ERROR - [Thread-370:] ~ Evicted [257@ac1b1980131887-ctr-e134-1499953498516-209460-01-04-hwx-site1] from cache but waiting too long for transactions to close. Stale transaction alert on: [standardtitantx[null], standardtitantx[null]] (ManagementLogger:189) 2017-10-09 20:18:20,808 ERROR - [Thread-372:] ~ Evicted [259@ac1b1980131887-ctr-e134-1499953498516-209460-01-04-hwx-site1] from cache but waiting too long for transactions to close. Stale transaction alert on: [standardtitantx[null], standardtitantx[null]] (ManagementLogger:189) 2017-10-09 20:18:25,658 ERROR - [Thread-373:] ~ Evicted [260@ac1b1980131887-ctr-e134-1499953498516-209460-01-04-hwx-site1] from cache but waiting too long for transactions to close. Stale transaction alert on: [standardtitantx[null], standardtitantx[null]] (ManagementLogger:189) 2017-10-09 20:18:25,698 ERROR - [Thread-374:] ~ Evicted [261@ac1b1980131887-ctr-e134-1499953498516-209460-01-04-hwx-site1] from cache but waiting too long for transactions to close. Stale transaction alert on: [standardtitantx[null], standardtitantx[null]] (ManagementLogger:189) 2017-10-09 20:18:25,721 ERROR - [Thread-375:] ~ Evicted [262@ac1b1980131887-ctr-e134-1499953498516-209460-01-04-hwx-site1] from cache but waiting too long for transactions to close. Stale transaction alert on: [standardtitantx[null], standardtitantx[null]] (ManagementLogger:189) 2017-10-09 20:18:25,739 ERROR - [Thread-376:] ~ Evicted [263@ac1b1980131887-ctr-e134-1499953498516-209460-01-04-hwx-site1] from cache but waiting too long for transactions to close. Stale transaction alert on: [standardtitantx[null], standardtitantx[null]] (ManagementLogger:189) 2017-10-09 20:18:25,778 ERROR - [Thread-377:] ~ Evicted [264@ac1b1980131887-ctr-e134-1499953498516-209460-01-04-hwx-site1] from cache but waiting too long for transactions to close. Stale transaction alert on: [standardtitantx[null], standardtitantx[null]] (ManagementLogger:189) 2017-10-09 20:18:30,661 ERROR - [Thread-378:] ~ Evicted [265@ac1b1980131887-ctr-e134-1499953498516-209460-01-04-hwx-site1] from cache but waiting too long for transactions to close. Stale transaction alert on: [standardtitantx[null], standardtitantx[null]] (ManagementLogger:189) 2017-10-09 20:18:30,687 ERROR - [Thread-379:] ~ Evicted [266@ac1b1980131887-ctr-e134-1499953498516-209460-01-04-hwx-site1] from cache but waiting too long for transactions to close. Stale transaction alert on: [standardtitantx[null], standardtitantx[null]] (ManagementLogger:189) {noformat} -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (ATLAS-2196) HiveHook fails with OOM when hive view with 500 columns are created.
Ayub Pathan created ATLAS-2196: -- Summary: HiveHook fails with OOM when hive view with 500 columns are created. Key: ATLAS-2196 URL: https://issues.apache.org/jira/browse/ATLAS-2196 Project: Atlas Issue Type: Bug Components: atlas-intg Affects Versions: 0.8.2 Reporter: Ayub Pathan Fix For: 0.8.2 create below two commands with hivehook enabled, hiveHook fails with OOM. {noformat} create table t1_500(col0 string, col1 string, col2 string, col3 string, col4 string, col5 string, col6 string, col7 string, col8 string, col9 string, col10 string, col11 string, col12 string, col13 string, col14 string, col15 string, col16 string, col17 string, col18 string, col19 string, col20 string, col21 string, col22 string, col23 string, col24 string, col25 string, col26 string, col27 string, col28 string, col29 string, col30 string, col31 string, col32 string, col33 string, col34 string, col35 string, col36 string, col37 string, col38 string, col39 string, col40 string, col41 string, col42 string, col43 string, col44 string, col45 string, col46 string, col47 string, col48 string, col49 string, col50 string, col51 string, col52 string, col53 string, col54 string, col55 string, col56 string, col57 string, col58 string, col59 string, col60 string, col61 string, col62 string, col63 string, col64 string, col65 string, col66 string, col67 string, col68 string, col69 string, col70 string, col71 string, col72 string, col73 string, col74 string, col75 string, col76 string, col77 string, col78 string, col79 string, col80 string, col81 string, col82 string, col83 string, col84 string, col85 string, col86 string, col87 string, col88 string, col89 string, col90 string, col91 string, col92 string, col93 string, col94 string, col95 string, col96 string, col97 string, col98 string, col99 string, col100 string, col101 string, col102 string, col103 string, col104 string, col105 string, col106 string, col107 string, col108 string, col109 string, col110 string, col111 string, col112 string, col113 string, col114 string, col115 string, col116 string, col117 string, col118 string, col119 string, col120 string, col121 string, col122 string, col123 string, col124 string, col125 string, col126 string, col127 string, col128 string, col129 string, col130 string, col131 string, col132 string, col133 string, col134 string, col135 string, col136 string, col137 string, col138 string, col139 string, col140 string, col141 string, col142 string, col143 string, col144 string, col145 string, col146 string, col147 string, col148 string, col149 string, col150 string, col151 string, col152 string, col153 string, col154 string, col155 string, col156 string, col157 string, col158 string, col159 string, col160 string, col161 string, col162 string, col163 string, col164 string, col165 string, col166 string, col167 string, col168 string, col169 string, col170 string, col171 string, col172 string, col173 string, col174 string, col175 string, col176 string, col177 string, col178 string, col179 string, col180 string, col181 string, col182 string, col183 string, col184 string, col185 string, col186 string, col187 string, col188 string, col189 string, col190 string, col191 string, col192 string, col193 string, col194 string, col195 string, col196 string, col197 string, col198 string, col199 string, col200 string, col201 string, col202 string, col203 string, col204 string, col205 string, col206 string, col207 string, col208 string, col209 string, col210 string, col211 string, col212 string, col213 string, col214 string, col215 string, col216 string, col217 string, col218 string, col219 string, col220 string, col221 string, col222 string, col223 string, col224 string, col225 string, col226 string, col227 string, col228 string, col229 string, col230 string, col231 string, col232 string, col233 string, col234 string, col235 string, col236 string, col237 string, col238 string, col239 string, col240 string, col241 string, col242 string, col243 string, col244 string, col245 string, col246 string, col247 string, col248 string, col249 string, col250 string, col251 string, col252 string, col253 string, col254 string, col255 string, col256 string, col257 string, col258 string, col259 string, col260 string, col261 string, col262 string, col263 string, col264 string, col265 string, col266 string, col267 string, col268 string, col269 string, col270 string, col271 string, col272 string, col273 string, col274 string, col275 string, col276 string, col277 string, col278 string, col279 string, col280 string, col281 string, col282 string, col283 string, col284 string, col285 string, col286 string, col287 string, col288 string, col289 string, col290 string, col291 string, col292 string, col293 string, col294 string, col295 string, col296 string, col297 string, col298 string, col299 string, co
[jira] [Created] (ATLAS-2198) HiveHook fails with OOM when hive view with 500 columns are created.
Ayub Pathan created ATLAS-2198: -- Summary: HiveHook fails with OOM when hive view with 500 columns are created. Key: ATLAS-2198 URL: https://issues.apache.org/jira/browse/ATLAS-2198 Project: Atlas Issue Type: Bug Components: atlas-intg Affects Versions: 0.8.2 Reporter: Ayub Pathan Priority: Critical Fix For: 0.8.2 create below two commands with hivehook enabled, hiveHook fails with OOM. {noformat} create table t1_500(col0 string, col1 string, col2 string, col3 string, col4 string, col5 string, col6 string, col7 string, col8 string, col9 string, col10 string, col11 string, col12 string, col13 string, col14 string, col15 string, col16 string, col17 string, col18 string, col19 string, col20 string, col21 string, col22 string, col23 string, col24 string, col25 string, col26 string, col27 string, col28 string, col29 string, col30 string, col31 string, col32 string, col33 string, col34 string, col35 string, col36 string, col37 string, col38 string, col39 string, col40 string, col41 string, col42 string, col43 string, col44 string, col45 string, col46 string, col47 string, col48 string, col49 string, col50 string, col51 string, col52 string, col53 string, col54 string, col55 string, col56 string, col57 string, col58 string, col59 string, col60 string, col61 string, col62 string, col63 string, col64 string, col65 string, col66 string, col67 string, col68 string, col69 string, col70 string, col71 string, col72 string, col73 string, col74 string, col75 string, col76 string, col77 string, col78 string, col79 string, col80 string, col81 string, col82 string, col83 string, col84 string, col85 string, col86 string, col87 string, col88 string, col89 string, col90 string, col91 string, col92 string, col93 string, col94 string, col95 string, col96 string, col97 string, col98 string, col99 string, col100 string, col101 string, col102 string, col103 string, col104 string, col105 string, col106 string, col107 string, col108 string, col109 string, col110 string, col111 string, col112 string, col113 string, col114 string, col115 string, col116 string, col117 string, col118 string, col119 string, col120 string, col121 string, col122 string, col123 string, col124 string, col125 string, col126 string, col127 string, col128 string, col129 string, col130 string, col131 string, col132 string, col133 string, col134 string, col135 string, col136 string, col137 string, col138 string, col139 string, col140 string, col141 string, col142 string, col143 string, col144 string, col145 string, col146 string, col147 string, col148 string, col149 string, col150 string, col151 string, col152 string, col153 string, col154 string, col155 string, col156 string, col157 string, col158 string, col159 string, col160 string, col161 string, col162 string, col163 string, col164 string, col165 string, col166 string, col167 string, col168 string, col169 string, col170 string, col171 string, col172 string, col173 string, col174 string, col175 string, col176 string, col177 string, col178 string, col179 string, col180 string, col181 string, col182 string, col183 string, col184 string, col185 string, col186 string, col187 string, col188 string, col189 string, col190 string, col191 string, col192 string, col193 string, col194 string, col195 string, col196 string, col197 string, col198 string, col199 string, col200 string, col201 string, col202 string, col203 string, col204 string, col205 string, col206 string, col207 string, col208 string, col209 string, col210 string, col211 string, col212 string, col213 string, col214 string, col215 string, col216 string, col217 string, col218 string, col219 string, col220 string, col221 string, col222 string, col223 string, col224 string, col225 string, col226 string, col227 string, col228 string, col229 string, col230 string, col231 string, col232 string, col233 string, col234 string, col235 string, col236 string, col237 string, col238 string, col239 string, col240 string, col241 string, col242 string, col243 string, col244 string, col245 string, col246 string, col247 string, col248 string, col249 string, col250 string, col251 string, col252 string, col253 string, col254 string, col255 string, col256 string, col257 string, col258 string, col259 string, col260 string, col261 string, col262 string, col263 string, col264 string, col265 string, col266 string, col267 string, col268 string, col269 string, col270 string, col271 string, col272 string, col273 string, col274 string, col275 string, col276 string, col277 string, col278 string, col279 string, col280 string, col281 string, col282 string, col283 string, col284 string, col285 string, col286 string, col287 string, col288 string, col289 string, col290 string, col291 string, col292 string, col293 string, col294 string, col295 string, col296 string, col297 string, co
[jira] [Created] (ATLAS-2199) DeadLock in atlas while producing metadata with hive view > 300 cols.
Ayub Pathan created ATLAS-2199: -- Summary: DeadLock in atlas while producing metadata with hive view > 300 cols. Key: ATLAS-2199 URL: https://issues.apache.org/jira/browse/ATLAS-2199 Project: Atlas Issue Type: Bug Components: atlas-core Affects Versions: 0.8.2 Reporter: Ayub Pathan Priority: Critical Fix For: 0.8.2 Attachments: application.log, atlas.20171009-192118.err, atlas.20171009-192118.out, thread-dump.txt Steps to reproduce: 1. Produce messages using hive hook - create view with 300 columns. Example, command below {noformat} create table t1_300(col0 string, col1 string, col2 string, col3 string, col4 string, col5 string, col6 string, col7 string, col8 string, col9 string, col10 string, col11 string, col12 string, col13 string, col14 string, col15 string, col16 string, col17 string, col18 string, col19 string, col20 string, col21 string, col22 string, col23 string, col24 string, col25 string, col26 string, col27 string, col28 string, col29 string, col30 string, col31 string, col32 string, col33 string, col34 string, col35 string, col36 string, col37 string, col38 string, col39 string, col40 string, col41 string, col42 string, col43 string, col44 string, col45 string, col46 string, col47 string, col48 string, col49 string, col50 string, col51 string, col52 string, col53 string, col54 string, col55 string, col56 string, col57 string, col58 string, col59 string, col60 string, col61 string, col62 string, col63 string, col64 string, col65 string, col66 string, col67 string, col68 string, col69 string, col70 string, col71 string, col72 string, col73 string, col74 string, col75 string, col76 string, col77 string, col78 string, col79 string, col80 string, col81 string, col82 string, col83 string, col84 string, col85 string, col86 string, col87 string, col88 string, col89 string, col90 string, col91 string, col92 string, col93 string, col94 string, col95 string, col96 string, col97 string, col98 string, col99 string, col100 string, col101 string, col102 string, col103 string, col104 string, col105 string, col106 string, col107 string, col108 string, col109 string, col110 string, col111 string, col112 string, col113 string, col114 string, col115 string, col116 string, col117 string, col118 string, col119 string, col120 string, col121 string, col122 string, col123 string, col124 string, col125 string, col126 string, col127 string, col128 string, col129 string, col130 string, col131 string, col132 string, col133 string, col134 string, col135 string, col136 string, col137 string, col138 string, col139 string, col140 string, col141 string, col142 string, col143 string, col144 string, col145 string, col146 string, col147 string, col148 string, col149 string, col150 string, col151 string, col152 string, col153 string, col154 string, col155 string, col156 string, col157 string, col158 string, col159 string, col160 string, col161 string, col162 string, col163 string, col164 string, col165 string, col166 string, col167 string, col168 string, col169 string, col170 string, col171 string, col172 string, col173 string, col174 string, col175 string, col176 string, col177 string, col178 string, col179 string, col180 string, col181 string, col182 string, col183 string, col184 string, col185 string, col186 string, col187 string, col188 string, col189 string, col190 string, col191 string, col192 string, col193 string, col194 string, col195 string, col196 string, col197 string, col198 string, col199 string, col200 string, col201 string, col202 string, col203 string, col204 string, col205 string, col206 string, col207 string, col208 string, col209 string, col210 string, col211 string, col212 string, col213 string, col214 string, col215 string, col216 string, col217 string, col218 string, col219 string, col220 string, col221 string, col222 string, col223 string, col224 string, col225 string, col226 string, col227 string, col228 string, col229 string, col230 string, col231 string, col232 string, col233 string, col234 string, col235 string, col236 string, col237 string, col238 string, col239 string, col240 string, col241 string, col242 string, col243 string, col244 string, col245 string, col246 string, col247 string, col248 string, col249 string, col250 string, col251 string, col252 string, col253 string, col254 string, col255 string, col256 string, col257 string, col258 string, col259 string, col260 string, col261 string, col262 string, col263 string, col264 string, col265 string, col266 string, col267 string, col268 string, col269 string, col270 string, col271 string, col272 string, col273 string, col274 string, col275 string, col276 string, col277 string, col278 string, col279 string, col280 string, col281 string, col282 string, col283 string, col284 string, col285 string, col286 string, col287 string, col288
[jira] [Updated] (ATLAS-2199) DeadLock in atlas while producing metadata with hive view > 300 cols.
[ https://issues.apache.org/jira/browse/ATLAS-2199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayub Pathan updated ATLAS-2199: --- Priority: Blocker (was: Critical) > DeadLock in atlas while producing metadata with hive view > 300 cols. > - > > Key: ATLAS-2199 > URL: https://issues.apache.org/jira/browse/ATLAS-2199 > Project: Atlas > Issue Type: Bug > Components: atlas-core >Affects Versions: 0.8.2 >Reporter: Ayub Pathan >Priority: Blocker > Fix For: 0.8.2 > > Attachments: application.log, atlas.20171009-192118.err, > atlas.20171009-192118.out, thread-dump.txt > > > Steps to reproduce: > 1. Produce messages using hive hook - create view with 300 columns. Example, > command below > {noformat} > create table t1_300(col0 string, col1 string, col2 string, col3 string, col4 > string, col5 string, col6 string, col7 string, col8 string, col9 string, > col10 string, col11 string, col12 string, col13 string, col14 string, col15 > string, col16 string, col17 string, col18 string, col19 string, col20 string, > col21 string, col22 string, col23 string, col24 string, col25 string, col26 > string, col27 string, col28 string, col29 string, col30 string, col31 string, > col32 string, col33 string, col34 string, col35 string, col36 string, col37 > string, col38 string, col39 string, col40 string, col41 string, col42 string, > col43 string, col44 string, col45 string, col46 string, col47 string, col48 > string, col49 string, col50 string, col51 string, col52 string, col53 string, > col54 string, col55 string, col56 string, col57 string, col58 string, col59 > string, col60 string, col61 string, col62 string, col63 string, col64 string, > col65 string, col66 string, col67 string, col68 string, col69 string, col70 > string, col71 string, col72 string, col73 string, col74 string, col75 string, > col76 string, col77 string, col78 string, col79 string, col80 string, col81 > string, col82 string, col83 string, col84 string, col85 string, col86 string, > col87 string, col88 string, col89 string, col90 string, col91 string, col92 > string, col93 string, col94 string, col95 string, col96 string, col97 string, > col98 string, col99 string, col100 string, col101 string, col102 string, > col103 string, col104 string, col105 string, col106 string, col107 string, > col108 string, col109 string, col110 string, col111 string, col112 string, > col113 string, col114 string, col115 string, col116 string, col117 string, > col118 string, col119 string, col120 string, col121 string, col122 string, > col123 string, col124 string, col125 string, col126 string, col127 string, > col128 string, col129 string, col130 string, col131 string, col132 string, > col133 string, col134 string, col135 string, col136 string, col137 string, > col138 string, col139 string, col140 string, col141 string, col142 string, > col143 string, col144 string, col145 string, col146 string, col147 string, > col148 string, col149 string, col150 string, col151 string, col152 string, > col153 string, col154 string, col155 string, col156 string, col157 string, > col158 string, col159 string, col160 string, col161 string, col162 string, > col163 string, col164 string, col165 string, col166 string, col167 string, > col168 string, col169 string, col170 string, col171 string, col172 string, > col173 string, col174 string, col175 string, col176 string, col177 string, > col178 string, col179 string, col180 string, col181 string, col182 string, > col183 string, col184 string, col185 string, col186 string, col187 string, > col188 string, col189 string, col190 string, col191 string, col192 string, > col193 string, col194 string, col195 string, col196 string, col197 string, > col198 string, col199 string, col200 string, col201 string, col202 string, > col203 string, col204 string, col205 string, col206 string, col207 string, > col208 string, col209 string, col210 string, col211 string, col212 string, > col213 string, col214 string, col215 string, col216 string, col217 string, > col218 string, col219 string, col220 string, col221 string, col222 string, > col223 string, col224 string, col225 string, col226 string, col227 string, > col228 string, col229 string, col230 string, col231 string, col232 string, > col233 string, col234 string, col235 string, col236 string, col237 string, > col238 string, col239 string, col240 string, col241 string, col242 string, > col243 string, col244 string, col245 string, col246 string, col247 string, > col248 string, col249 string, col250 string, col251 string, col252 string, > col253 string, col254 string, col255 string, col256 string, col257 string, > col258 string, col259 string, col260 string, col261 string, col262 string, > col263 string, col264 string, col265 str
[jira] [Updated] (ATLAS-2198) HiveHook fails with OOM when hive view with 500 columns are created.
[ https://issues.apache.org/jira/browse/ATLAS-2198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayub Pathan updated ATLAS-2198: --- Priority: Blocker (was: Critical) > HiveHook fails with OOM when hive view with 500 columns are created. > > > Key: ATLAS-2198 > URL: https://issues.apache.org/jira/browse/ATLAS-2198 > Project: Atlas > Issue Type: Bug > Components: atlas-intg >Affects Versions: 0.8.2 >Reporter: Ayub Pathan >Assignee: Apoorv Naik >Priority: Blocker > Fix For: 0.8.2 > > Attachments: 0001-ATLAS-2198-Hive-Hook-OOM-fix.patch > > > create below two commands with hivehook enabled, hiveHook fails with OOM. > {noformat} > create table t1_500(col0 string, col1 string, col2 string, col3 string, col4 > string, col5 string, col6 string, col7 string, col8 string, col9 string, > col10 string, col11 string, col12 string, col13 string, col14 string, col15 > string, col16 string, col17 string, col18 string, col19 string, col20 string, > col21 string, col22 string, col23 string, col24 string, col25 string, col26 > string, col27 string, col28 string, col29 string, col30 string, col31 string, > col32 string, col33 string, col34 string, col35 string, col36 string, col37 > string, col38 string, col39 string, col40 string, col41 string, col42 string, > col43 string, col44 string, col45 string, col46 string, col47 string, col48 > string, col49 string, col50 string, col51 string, col52 string, col53 string, > col54 string, col55 string, col56 string, col57 string, col58 string, col59 > string, col60 string, col61 string, col62 string, col63 string, col64 string, > col65 string, col66 string, col67 string, col68 string, col69 string, col70 > string, col71 string, col72 string, col73 string, col74 string, col75 string, > col76 string, col77 string, col78 string, col79 string, col80 string, col81 > string, col82 string, col83 string, col84 string, col85 string, col86 string, > col87 string, col88 string, col89 string, col90 string, col91 string, col92 > string, col93 string, col94 string, col95 string, col96 string, col97 string, > col98 string, col99 string, col100 string, col101 string, col102 string, > col103 string, col104 string, col105 string, col106 string, col107 string, > col108 string, col109 string, col110 string, col111 string, col112 string, > col113 string, col114 string, col115 string, col116 string, col117 string, > col118 string, col119 string, col120 string, col121 string, col122 string, > col123 string, col124 string, col125 string, col126 string, col127 string, > col128 string, col129 string, col130 string, col131 string, col132 string, > col133 string, col134 string, col135 string, col136 string, col137 string, > col138 string, col139 string, col140 string, col141 string, col142 string, > col143 string, col144 string, col145 string, col146 string, col147 string, > col148 string, col149 string, col150 string, col151 string, col152 string, > col153 string, col154 string, col155 string, col156 string, col157 string, > col158 string, col159 string, col160 string, col161 string, col162 string, > col163 string, col164 string, col165 string, col166 string, col167 string, > col168 string, col169 string, col170 string, col171 string, col172 string, > col173 string, col174 string, col175 string, col176 string, col177 string, > col178 string, col179 string, col180 string, col181 string, col182 string, > col183 string, col184 string, col185 string, col186 string, col187 string, > col188 string, col189 string, col190 string, col191 string, col192 string, > col193 string, col194 string, col195 string, col196 string, col197 string, > col198 string, col199 string, col200 string, col201 string, col202 string, > col203 string, col204 string, col205 string, col206 string, col207 string, > col208 string, col209 string, col210 string, col211 string, col212 string, > col213 string, col214 string, col215 string, col216 string, col217 string, > col218 string, col219 string, col220 string, col221 string, col222 string, > col223 string, col224 string, col225 string, col226 string, col227 string, > col228 string, col229 string, col230 string, col231 string, col232 string, > col233 string, col234 string, col235 string, col236 string, col237 string, > col238 string, col239 string, col240 string, col241 string, col242 string, > col243 string, col244 string, col245 string, col246 string, col247 string, > col248 string, col249 string, col250 string, col251 string, col252 string, > col253 string, col254 string, col255 string, col256 string, col257 string, > col258 string, col259 string, col260 string, col261 string, col262 string, > col263 string, col264 string, col265 string, col266 string, col267 string, > col268 string, col269
[jira] [Created] (ATLAS-2202) Duplicate kafka notifications observed for external table
Ayub Pathan created ATLAS-2202: -- Summary: Duplicate kafka notifications observed for external table Key: ATLAS-2202 URL: https://issues.apache.org/jira/browse/ATLAS-2202 Project: Atlas Issue Type: Bug Components: atlas-intg Affects Versions: 0.8.2 Reporter: Ayub Pathan Fix For: 0.8.2 Steps to reproduce: Create external table and observe the kafka notifications. * This notification includes hive_db and external hive_table notification {noformat} {"msgSourceIP":"172.27.39.68","msgCreatedBy":"hrt_qa","msgCreationTime":1507714828002,"message":{"entities":[{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Reference","id":{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Id","id":"-1706538104304055","version":0,"typeName":"hive_db","state":"ACTIVE"},"typeName":"hive_db","values":{"name":"default","location":"hdfs://ctr-e134-1499953498516-214922-01-03.hwx.site:8020/apps/hive/warehouse","description":"Default Hive database","ownerType":2,"qualifiedName":"default@cl1","owner":"public","clusterName":"cl1","parameters":{}},"traitNames":[],"traits":{},"systemAttributes":{}},{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Reference","id":{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Id","id":"-1706538104304054","version":0,"typeName":"hive_table","state":"ACTIVE"},"typeName":"hive_table","values":{"tableType":"EXTERNAL_TABLE","name":"externaltable_2","createTime":"2017-10-11T09:40:27.000Z","temporary":false,"db":{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Reference","id":{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Id","id":"-1706538104304055","version":0,"typeName":"hive_db","state":"ACTIVE"},"typeName":"hive_db","values":{"name":"default","location":"hdfs://ctr-e134-1499953498516-214922-01-03.hwx.site:8020/apps/hive/warehouse","description":"Default Hive database","ownerType":2,"qualifiedName":"default@cl1","owner":"public","clusterName":"cl1","parameters":{}},"traitNames":[],"traits":{},"systemAttributes":{}},"retention":0,"qualifiedName":"default.externaltable_2@cl1","columns":[{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Reference","id":{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Id","id":"-1706538104304052","version":0,"typeName":"hive_column","state":"ACTIVE"},"typeName":"hive_column","values":{"name":"id","qualifiedName":"default.externaltable_2.id@cl1","position":0,"owner":"hrt_qa","type":"int","table":{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Id","id":"-1706538104304054","version":0,"typeName":"hive_table","state":"ACTIVE"}},"traitNames":[],"traits":{},"systemAttributes":{}},{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Reference","id":{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Id","id":"-1706538104304051","version":0,"typeName":"hive_column","state":"ACTIVE"},"typeName":"hive_column","values":{"name":"name","qualifiedName":"default.externaltable_2.name@cl1","position":1,"owner":"hrt_qa","type":"string","table":{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Id","id":"-1706538104304054","version":0,"typeName":"hive_table","state":"ACTIVE"}},"traitNames":[],"traits":{},"systemAttributes":{}}],"lastAccessTime":"2017-10-11T09:40:27.000Z","owner":"hrt_qa","sd":{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Reference","id":{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Id","id":"-1706538104304053","version":0,"typeName":"hive_storagedesc","state":"ACTIVE"},"typeName":"hive_storagedesc","values":{"location":"hdfs://ctr-e134-1499953498516-214922-01-03.hwx.site:8020/apps/hive/warehouse/externaltable_2","serdeInfo":{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Struct","typeName":"hive_serde","values":{"serializationLib":"org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe","parameters":{"serialization.format":"1"}}},"qualifiedName":"default.externaltable_2@cl1_storage","outputFormat":"org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat","compressed":false,"numBuckets":-1,"inputFormat":"org.apache.hadoop.mapred.TextInputFormat","parameters":{},"storedAsSubDirectories":false,"table":{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Id","id":"-1706538104304054","version":0,"typeName":"hive_table","state":"ACTIVE"}},"traitNames":[],"traits":{},"systemAttributes":{}},"parameters":{"rawDataSize":"0","numFiles":"0","transient_lastDdlTime":"1507714827","totalSize":"0","EXTERNAL":"TRUE","COLUMN_STATS_ACCURATE":"{\"BASIC_STATS\":\"true\"}","numRows":"0"},"partitionKeys":[]},"traitNames":[],"traits":{},"systemAttributes":{}}],"type":"ENTITY_FULL_UPDATE","user":"hrt_qa"},"version":{"version":
[jira] [Comment Edited] (ATLAS-2202) Duplicate kafka notifications observed for external table
[ https://issues.apache.org/jira/browse/ATLAS-2202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16200058#comment-16200058 ] Ayub Pathan edited comment on ATLAS-2202 at 10/11/17 10:08 AM: --- Old behavior before ATLAS-2198 * Hive db and external table notif {noformat} {"msgSourceIP":"172.27.25.128","msgCreatedBy":"hrt_qa","msgCreationTime":1507716386984,"message":{"entities":[{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Reference","id":{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Id","id":"-1576713171604213","version":0,"typeName":"hive_db","state":"ACTIVE"},"typeName":"hive_db","values":{"name":"default","location":"hdfs://ctr-e134-1499953498516-209460-01-04.hwx.site:8020/apps/hive/warehouse","description":"Default Hive database","ownerType":2,"qualifiedName":"default@cl1","owner":"public","clusterName":"cl1","parameters":{}},"traitNames":[],"traits":{},"systemAttributes":{}},{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Reference","id":{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Id","id":"-1576713171604212","version":0,"typeName":"hive_table","state":"ACTIVE"},"typeName":"hive_table","values":{"tableType":"EXTERNAL_TABLE","name":"externaltable_1","createTime":"2017-10-11T10:06:26.000Z","temporary":false,"db":{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Reference","id":{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Id","id":"-1576713171604213","version":0,"typeName":"hive_db","state":"ACTIVE"},"typeName":"hive_db","values":{"name":"default","location":"hdfs://ctr-e134-1499953498516-209460-01-04.hwx.site:8020/apps/hive/warehouse","description":"Default Hive database","ownerType":2,"qualifiedName":"default@cl1","owner":"public","clusterName":"cl1","parameters":{}},"traitNames":[],"traits":{},"systemAttributes":{}},"retention":0,"qualifiedName":"default.externaltable_1@cl1","columns":[{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Reference","id":{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Id","id":"-1576713171604210","version":0,"typeName":"hive_column","state":"ACTIVE"},"typeName":"hive_column","values":{"name":"id","qualifiedName":"default.externaltable_1.id@cl1","position":0,"owner":"hrt_qa","type":"int","table":{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Id","id":"-1576713171604212","version":0,"typeName":"hive_table","state":"ACTIVE"}},"traitNames":[],"traits":{},"systemAttributes":{}},{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Reference","id":{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Id","id":"-1576713171604209","version":0,"typeName":"hive_column","state":"ACTIVE"},"typeName":"hive_column","values":{"name":"name","qualifiedName":"default.externaltable_1.name@cl1","position":1,"owner":"hrt_qa","type":"string","table":{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Id","id":"-1576713171604212","version":0,"typeName":"hive_table","state":"ACTIVE"}},"traitNames":[],"traits":{},"systemAttributes":{}}],"lastAccessTime":"2017-10-11T10:06:26.000Z","owner":"hrt_qa","sd":{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Reference","id":{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Id","id":"-1576713171604211","version":0,"typeName":"hive_storagedesc","state":"ACTIVE"},"typeName":"hive_storagedesc","values":{"location":"hdfs://ctr-e134-1499953498516-209460-01-04.hwx.site:8020/apps/hive/warehouse/externaltable_1","serdeInfo":{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Struct","typeName":"hive_serde","values":{"serializationLib":"org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe","parameters":{"serialization.format":"1"}}},"qualifiedName":"default.externaltable_1@cl1_storage","outputFormat":"org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat","compressed":false,"numBuckets":-1,"inputFormat":"org.apache.hadoop.mapred.TextInputFormat","parameters":{},"storedAsSubDirectories":false,"table":{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Id","id":"-1576713171604212","version":0,"typeName":"hive_table","state":"ACTIVE"}},"traitNames":[],"traits":{},"systemAttributes":{}},"parameters":{"rawDataSize":"0","numFiles":"0","transient_lastDdlTime":"1507716386","totalSize":"0","EXTERNAL":"TRUE","COLUMN_STATS_ACCURATE":"{\"BASIC_STATS\":\"true\"}","numRows":"0"},"partitionKeys":[]},"traitNames":[],"traits":{},"systemAttributes":{}}],"type":"ENTITY_FULL_UPDATE","user":"hrt_qa"},"version":{"version":"1.0.0"},"msgCompressionKind":"NONE","msgSplitIdx":1,"msgSplitCount":1} {noformat} * Hive process notif {noformat} {"msgSourceIP":"172.27.25.128","msgCreatedBy":"hrt_qa","msgCreationTime":1507716386985,"message":{
[jira] [Commented] (ATLAS-2202) Duplicate kafka notifications observed for external table
[ https://issues.apache.org/jira/browse/ATLAS-2202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16200058#comment-16200058 ] Ayub Pathan commented on ATLAS-2202: Old behavior before ATLAS-2198 * Hive db and external table notif {noformat} {"msgSourceIP":"172.27.25.128","msgCreatedBy":"hrt_qa","msgCreationTime":1507716386984,"message":{"entities":[{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Reference","id":{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Id","id":"-1576713171604213","version":0,"typeName":"hive_db","state":"ACTIVE"},"typeName":"hive_db","values":{"name":"default","location":"hdfs://ctr-e134-1499953498516-209460-01-04.hwx.site:8020/apps/hive/warehouse","description":"Default Hive database","ownerType":2,"qualifiedName":"default@cl1","owner":"public","clusterName":"cl1","parameters":{}},"traitNames":[],"traits":{},"systemAttributes":{}},{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Reference","id":{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Id","id":"-1576713171604212","version":0,"typeName":"hive_table","state":"ACTIVE"},"typeName":"hive_table","values":{"tableType":"EXTERNAL_TABLE","name":"externaltable_1","createTime":"2017-10-11T10:06:26.000Z","temporary":false,"db":{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Reference","id":{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Id","id":"-1576713171604213","version":0,"typeName":"hive_db","state":"ACTIVE"},"typeName":"hive_db","values":{"name":"default","location":"hdfs://ctr-e134-1499953498516-209460-01-04.hwx.site:8020/apps/hive/warehouse","description":"Default Hive database","ownerType":2,"qualifiedName":"default@cl1","owner":"public","clusterName":"cl1","parameters":{}},"traitNames":[],"traits":{},"systemAttributes":{}},"retention":0,"qualifiedName":"default.externaltable_1@cl1","columns":[{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Reference","id":{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Id","id":"-1576713171604210","version":0,"typeName":"hive_column","state":"ACTIVE"},"typeName":"hive_column","values":{"name":"id","qualifiedName":"default.externaltable_1.id@cl1","position":0,"owner":"hrt_qa","type":"int","table":{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Id","id":"-1576713171604212","version":0,"typeName":"hive_table","state":"ACTIVE"}},"traitNames":[],"traits":{},"systemAttributes":{}},{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Reference","id":{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Id","id":"-1576713171604209","version":0,"typeName":"hive_column","state":"ACTIVE"},"typeName":"hive_column","values":{"name":"name","qualifiedName":"default.externaltable_1.name@cl1","position":1,"owner":"hrt_qa","type":"string","table":{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Id","id":"-1576713171604212","version":0,"typeName":"hive_table","state":"ACTIVE"}},"traitNames":[],"traits":{},"systemAttributes":{}}],"lastAccessTime":"2017-10-11T10:06:26.000Z","owner":"hrt_qa","sd":{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Reference","id":{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Id","id":"-1576713171604211","version":0,"typeName":"hive_storagedesc","state":"ACTIVE"},"typeName":"hive_storagedesc","values":{"location":"hdfs://ctr-e134-1499953498516-209460-01-04.hwx.site:8020/apps/hive/warehouse/externaltable_1","serdeInfo":{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Struct","typeName":"hive_serde","values":{"serializationLib":"org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe","parameters":{"serialization.format":"1"}}},"qualifiedName":"default.externaltable_1@cl1_storage","outputFormat":"org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat","compressed":false,"numBuckets":-1,"inputFormat":"org.apache.hadoop.mapred.TextInputFormat","parameters":{},"storedAsSubDirectories":false,"table":{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Id","id":"-1576713171604212","version":0,"typeName":"hive_table","state":"ACTIVE"}},"traitNames":[],"traits":{},"systemAttributes":{}},"parameters":{"rawDataSize":"0","numFiles":"0","transient_lastDdlTime":"1507716386","totalSize":"0","EXTERNAL":"TRUE","COLUMN_STATS_ACCURATE":"{\"BASIC_STATS\":\"true\"}","numRows":"0"},"partitionKeys":[]},"traitNames":[],"traits":{},"systemAttributes":{}}],"type":"ENTITY_FULL_UPDATE","user":"hrt_qa"},"version":{"version":"1.0.0"},"msgCompressionKind":"NONE","msgSplitIdx":1,"msgSplitCount":1} {noformat} * Hive process notif {noformat} {"msgSourceIP":"172.27.25.128","msgCreatedBy":"hrt_qa","msgCreationTime":1507716386985,"message":{"entities":[{"jsonClass":"org.apache.atlas.typesystem
[jira] [Commented] (ATLAS-3545) NullPointerException while trying to delete classification
[ https://issues.apache.org/jira/browse/ATLAS-3545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16987104#comment-16987104 ] Ayub Pathan commented on ATLAS-3545: [~bolke] Can you please add the repro steps here? > NullPointerException while trying to delete classification > -- > > Key: ATLAS-3545 > URL: https://issues.apache.org/jira/browse/ATLAS-3545 > Project: Atlas > Issue Type: Bug >Affects Versions: trunk >Reporter: Bolke de Bruin >Priority: Critical > Attachments: Screenshot 2019-11-30 at 22.16.44.png, Screenshot > 2019-11-30 at 22.28.00.png, Screenshot 2019-12-02 at 21.43.29.png > > > We see an issue where there is a NullPointerException while trying to delete > a classification that is propagating. > It seems (stack trace is as of yet unavailable) that it is caused due to a > typeName being NULL in AtlasTypeRegistry.getType. The UI looks fishy too see > screenshots. > -- This message was sent by Atlassian Jira (v8.3.4#803005)