[jira] [Resolved] (ATLAS-2249) Upgrade Atlas Client to use Jersey 2
[ https://issues.apache.org/jira/browse/ATLAS-2249?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Padovani resolved ATLAS-2249. Resolution: Won't Fix > Upgrade Atlas Client to use Jersey 2 > > > Key: ATLAS-2249 > URL: https://issues.apache.org/jira/browse/ATLAS-2249 > Project: Atlas > Issue Type: Improvement > Components: atlas-intg >Affects Versions: 1.0.0 >Reporter: Pierre Padovani >Priority: Major > Labels: patch-available > Attachments: client-jersey2-V2.patch, client-jersey2-V3.patch, > client-jersey2-V4.patch, client-jersey2.patch > > > Using the existing Atlas client within a Jersey 2 service causes conflicts. I > recommend upgrading just the client code to Jersey 2. The enclosed patch > fully upgrades the V1 and V2 client to Jersey 2, and we have tested this > against our own installation of 1.0.0-SNAPSHOT. > NOTE: We have not tested SSL/Kerberos authentication/security -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (ATLAS-2249) Upgrade Atlas Client to use Jersey 2
[ https://issues.apache.org/jira/browse/ATLAS-2249?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16688558#comment-16688558 ] Pierre Padovani commented on ATLAS-2249: My apologies for taking so long to answer this it got buried in email. I would actually not recommend using this patch due to an issue in terms of json serialization around nulls we discovered. Our solution/workaround was to uber-jar the client changing all of the packaging of the underlying Jersey 1 code to avoid conflicts. I believe that only by updating the entire Atlas code base to Jersey 2 can this be resolved in a correct manner. > Upgrade Atlas Client to use Jersey 2 > > > Key: ATLAS-2249 > URL: https://issues.apache.org/jira/browse/ATLAS-2249 > Project: Atlas > Issue Type: Improvement > Components: atlas-intg >Affects Versions: 1.0.0 >Reporter: Pierre Padovani >Priority: Major > Labels: patch-available > Attachments: client-jersey2-V2.patch, client-jersey2-V3.patch, > client-jersey2-V4.patch, client-jersey2.patch > > > Using the existing Atlas client within a Jersey 2 service causes conflicts. I > recommend upgrading just the client code to Jersey 2. The enclosed patch > fully upgrades the V1 and V2 client to Jersey 2, and we have tested this > against our own installation of 1.0.0-SNAPSHOT. > NOTE: We have not tested SSL/Kerberos authentication/security -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Closed] (ATLAS-2249) Upgrade Atlas Client to use Jersey 2
[ https://issues.apache.org/jira/browse/ATLAS-2249?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Padovani closed ATLAS-2249. -- > Upgrade Atlas Client to use Jersey 2 > > > Key: ATLAS-2249 > URL: https://issues.apache.org/jira/browse/ATLAS-2249 > Project: Atlas > Issue Type: Improvement > Components: atlas-intg >Affects Versions: 1.0.0 >Reporter: Pierre Padovani >Priority: Major > Labels: patch-available > Attachments: client-jersey2-V2.patch, client-jersey2-V3.patch, > client-jersey2-V4.patch, client-jersey2.patch > > > Using the existing Atlas client within a Jersey 2 service causes conflicts. I > recommend upgrading just the client code to Jersey 2. The enclosed patch > fully upgrades the V1 and V2 client to Jersey 2, and we have tested this > against our own installation of 1.0.0-SNAPSHOT. > NOTE: We have not tested SSL/Kerberos authentication/security -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (ATLAS-2752) Atlas start up fail with error "Error creating bean with name 'graphTransactionAdvisor'
[ https://issues.apache.org/jira/browse/ATLAS-2752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16509682#comment-16509682 ] Pierre Padovani commented on ATLAS-2752: This error sounds like a jar is missing. How did you get the installation? Did you build it, or pull the binary distribution? Check $ATLAS_HOME/server/webapp/atlas/WEB-INF/lib You should have these solr jars: * server/webapp/atlas/WEB-INF/lib/janusgraph-solr-0.2.0.jar * server/webapp/atlas/WEB-INF/lib/solr-solrj-7.0.0.jar * server/webapp/atlas/WEB-INF/lib/solr-core-7.0.0.jar If you do have those jars, then your issue is outside my knowledge area. I would close this issue, and open another specific to solr, and an expert in that area will pick up the issue and help you out. Pierre > Atlas start up fail with error "Error creating bean with name > 'graphTransactionAdvisor' > --- > > Key: ATLAS-2752 > URL: https://issues.apache.org/jira/browse/ATLAS-2752 > Project: Atlas > Issue Type: Bug > Components: atlas-core, atlas-webui >Affects Versions: 1.0.0 > Environment: test >Reporter: Mahesh Kakol >Priority: Major > > I am getting below while starting atlas and graph stoage as cassandra > > > 2018-06-11 11:54:21,226 WARN - [main:] ~ Using @Deprecated Class > org.springframework.web.util.Log4jConfigListener (DeprecationWarning:43) > 2018-06-11 11:54:23,400 WARN - [main:] ~ Exception encountered during context > initialization - cancelling refresh attempt: > org.springframework.beans.factory.BeanCreationExceptio > n: Error creating bean with name > 'org.springframework.context.event.internalEventListenerProcessor': > BeanPostProcessor before instantiation of bean failed; nested exception is or > g.springframework.beans.factory.UnsatisfiedDependencyException: Error > creating bean with name 'graphTransactionAdvisor' defined in URL > [jar:[file:/data/apache-atlas/apache-atlas-s|file:///data/apache-atlas/apache-atlas-s] > ources-1.0.0/distro/target/apache-atlas-1.0.0-bin/apache-atlas-1.0.0/server/webapp/atlas/WEB-INF/lib/atlas-repository-1.0.0.jar!/org/apache/atlas/GraphTransactionAdvisor.class]: > Unsatisfied dependency expressed through constructor parameter 0; nested > exception is > org.springframework.beans.factory.UnsatisfiedDependencyException: Error > creating bean with n > ame 'graphTransactionInterceptor' defined in URL > [jar:[file:/data/apache-atlas/apache-atlas-sources-1.0.0/distro/target/apache-atlas-1.0.0-bin/apache-atlas-1.0.0/server/webapp/atl|file:///data/apache-atlas/apache-atlas-sources-1.0.0/distro/target/apache-atlas-1.0.0-bin/apache-atlas-1.0.0/server/webapp/atl] > as/WEB-INF/lib/atlas-repository-1.0.0.jar!/org/apache/atlas/GraphTransactionInterceptor.class]: > Unsatisfied dependency expressed through constructor parameter 0; nested > exception > is org.springframework.beans.factory.BeanCreationException: Error creating > bean with name 'get' defined in > org.apache.atlas.repository.graph.AtlasGraphProvider: Bean instantiati > on via factory method failed; nested exception is > org.springframework.beans.BeanInstantiationException: Failed to instantiate > [org.apache.atlas.repository.graphdb.AtlasGraph]: Fa > ctory method 'get' threw exception; nested exception is > java.lang.IllegalArgumentException: Could not instantiate implementation: > org.janusgraph.diskstorage.cassandra.astyanax.As > tyanaxStoreManager (AbstractApplicationContext:551) > 2018-06-11 11:54:23,402 ERROR - [main:] ~ Context initialization failed > (ContextLoader:350) > org.springframework.beans.factory.BeanCreationException: Error creating bean > with name 'org.springframework.context.event.internalEventListenerProcessor': > BeanPostProcessor befor > e instantiation of bean failed; nested exception is > org.springframework.beans.factory.UnsatisfiedDependencyException: Error > creating bean with name 'graphTransactionAdvisor' defi > ned in URL > [jar:[file:/data/apache-atlas/apache-atlas-sources-1.0.0/distro/target/apache-atlas-1.0.0-bin/apache-atlas-1.0.0/server/webapp/atlas/WEB-INF/lib/atlas-repository-1.0.0|file:///data/apache-atlas/apache-atlas-sources-1.0.0/distro/target/apache-atlas-1.0.0-bin/apache-atlas-1.0.0/server/webapp/atlas/WEB-INF/lib/atlas-repository-1.0.0]. > jar!/org/apache/atlas/GraphTransactionAdvisor.class]: Unsatisfied dependency > expressed through constructor parameter 0; nested exception is > org.springframework.beans.factory.Unsa > tisfiedDependencyException: Error creating bean with name > 'graphTransactionInterceptor' defined in URL > [jar:[file:/data/apache-atlas/apache-atlas-sources-1.0.0/distro/target/apach|file:///data/apache-atlas/apache-atlas-sources-1.0.0/distro/target/apach] > e-atlas-1.0.0-bin/apache-atlas-1.0.0
[jira] [Commented] (ATLAS-2752) Atlas start up fail with error "Error creating bean with name 'graphTransactionAdvisor'
[ https://issues.apache.org/jira/browse/ATLAS-2752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16508430#comment-16508430 ] Pierre Padovani commented on ATLAS-2752: Your issue is the " atlas.graph.storage.port" setting for the backend. Port 9042 is for the native transport not thrift, so the data frames used are different. For my company's installation (Cassandra + Elasticsearch) we just leave the port setting blank, and the default thrift port is automatically used. If you have configured your cluster with a custom thrift port, that should be the value used in the " atlas.graph.storage.port" settings. Pierre > Atlas start up fail with error "Error creating bean with name > 'graphTransactionAdvisor' > --- > > Key: ATLAS-2752 > URL: https://issues.apache.org/jira/browse/ATLAS-2752 > Project: Atlas > Issue Type: Bug > Components: atlas-core, atlas-webui >Affects Versions: 1.0.0 > Environment: test >Reporter: Mahesh Kakol >Priority: Major > > I am getting below while starting atlas and graph stoage as cassandra > > > 2018-06-11 11:54:21,226 WARN - [main:] ~ Using @Deprecated Class > org.springframework.web.util.Log4jConfigListener (DeprecationWarning:43) > 2018-06-11 11:54:23,400 WARN - [main:] ~ Exception encountered during context > initialization - cancelling refresh attempt: > org.springframework.beans.factory.BeanCreationExceptio > n: Error creating bean with name > 'org.springframework.context.event.internalEventListenerProcessor': > BeanPostProcessor before instantiation of bean failed; nested exception is or > g.springframework.beans.factory.UnsatisfiedDependencyException: Error > creating bean with name 'graphTransactionAdvisor' defined in URL > [jar:[file:/data/apache-atlas/apache-atlas-s|file:///data/apache-atlas/apache-atlas-s] > ources-1.0.0/distro/target/apache-atlas-1.0.0-bin/apache-atlas-1.0.0/server/webapp/atlas/WEB-INF/lib/atlas-repository-1.0.0.jar!/org/apache/atlas/GraphTransactionAdvisor.class]: > Unsatisfied dependency expressed through constructor parameter 0; nested > exception is > org.springframework.beans.factory.UnsatisfiedDependencyException: Error > creating bean with n > ame 'graphTransactionInterceptor' defined in URL > [jar:[file:/data/apache-atlas/apache-atlas-sources-1.0.0/distro/target/apache-atlas-1.0.0-bin/apache-atlas-1.0.0/server/webapp/atl|file:///data/apache-atlas/apache-atlas-sources-1.0.0/distro/target/apache-atlas-1.0.0-bin/apache-atlas-1.0.0/server/webapp/atl] > as/WEB-INF/lib/atlas-repository-1.0.0.jar!/org/apache/atlas/GraphTransactionInterceptor.class]: > Unsatisfied dependency expressed through constructor parameter 0; nested > exception > is org.springframework.beans.factory.BeanCreationException: Error creating > bean with name 'get' defined in > org.apache.atlas.repository.graph.AtlasGraphProvider: Bean instantiati > on via factory method failed; nested exception is > org.springframework.beans.BeanInstantiationException: Failed to instantiate > [org.apache.atlas.repository.graphdb.AtlasGraph]: Fa > ctory method 'get' threw exception; nested exception is > java.lang.IllegalArgumentException: Could not instantiate implementation: > org.janusgraph.diskstorage.cassandra.astyanax.As > tyanaxStoreManager (AbstractApplicationContext:551) > 2018-06-11 11:54:23,402 ERROR - [main:] ~ Context initialization failed > (ContextLoader:350) > org.springframework.beans.factory.BeanCreationException: Error creating bean > with name 'org.springframework.context.event.internalEventListenerProcessor': > BeanPostProcessor befor > e instantiation of bean failed; nested exception is > org.springframework.beans.factory.UnsatisfiedDependencyException: Error > creating bean with name 'graphTransactionAdvisor' defi > ned in URL > [jar:[file:/data/apache-atlas/apache-atlas-sources-1.0.0/distro/target/apache-atlas-1.0.0-bin/apache-atlas-1.0.0/server/webapp/atlas/WEB-INF/lib/atlas-repository-1.0.0|file:///data/apache-atlas/apache-atlas-sources-1.0.0/distro/target/apache-atlas-1.0.0-bin/apache-atlas-1.0.0/server/webapp/atlas/WEB-INF/lib/atlas-repository-1.0.0]. > jar!/org/apache/atlas/GraphTransactionAdvisor.class]: Unsatisfied dependency > expressed through constructor parameter 0; nested exception is > org.springframework.beans.factory.Unsa > tisfiedDependencyException: Error creating bean with name > 'graphTransactionInterceptor' defined in URL > [jar:[file:/data/apache-atlas/apache-atlas-sources-1.0.0/distro/target/apach|file:///data/apache-atlas/apache-atlas-sources-1.0.0/distro/target/apach] > e-atlas-1.0.0-bin/apache-atlas-1.0.0/server/webapp/atlas/WEB-INF/lib/atlas-repository-1.0.0.jar!/org/apache/atlas/GraphTransactionInterceptor.class]: > Unsatisfied dependency
[jira] [Comment Edited] (ATLAS-2752) Atlas start up fail with error "Error creating bean with name 'graphTransactionAdvisor'
[ https://issues.apache.org/jira/browse/ATLAS-2752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16508176#comment-16508176 ] Pierre Padovani edited comment on ATLAS-2752 at 6/11/18 2:56 PM: - A few questions for you: * What version of Cassandra are you using? * Did you enable thrift on the Cassandra cluster? Pierre was (Author: ppadovani): A few questions for you: * What version of Cassandra are you using? * Did you enable thrift? Pierre > Atlas start up fail with error "Error creating bean with name > 'graphTransactionAdvisor' > --- > > Key: ATLAS-2752 > URL: https://issues.apache.org/jira/browse/ATLAS-2752 > Project: Atlas > Issue Type: Bug > Components: atlas-core, atlas-webui >Affects Versions: 1.0.0 > Environment: test >Reporter: Mahesh Kakol >Priority: Major > > I am getting below while starting atlas and graph stoage as cassandra > > > 2018-06-11 11:54:21,226 WARN - [main:] ~ Using @Deprecated Class > org.springframework.web.util.Log4jConfigListener (DeprecationWarning:43) > 2018-06-11 11:54:23,400 WARN - [main:] ~ Exception encountered during context > initialization - cancelling refresh attempt: > org.springframework.beans.factory.BeanCreationExceptio > n: Error creating bean with name > 'org.springframework.context.event.internalEventListenerProcessor': > BeanPostProcessor before instantiation of bean failed; nested exception is or > g.springframework.beans.factory.UnsatisfiedDependencyException: Error > creating bean with name 'graphTransactionAdvisor' defined in URL > [jar:[file:/data/apache-atlas/apache-atlas-s|file:///data/apache-atlas/apache-atlas-s] > ources-1.0.0/distro/target/apache-atlas-1.0.0-bin/apache-atlas-1.0.0/server/webapp/atlas/WEB-INF/lib/atlas-repository-1.0.0.jar!/org/apache/atlas/GraphTransactionAdvisor.class]: > Unsatisfied dependency expressed through constructor parameter 0; nested > exception is > org.springframework.beans.factory.UnsatisfiedDependencyException: Error > creating bean with n > ame 'graphTransactionInterceptor' defined in URL > [jar:[file:/data/apache-atlas/apache-atlas-sources-1.0.0/distro/target/apache-atlas-1.0.0-bin/apache-atlas-1.0.0/server/webapp/atl|file:///data/apache-atlas/apache-atlas-sources-1.0.0/distro/target/apache-atlas-1.0.0-bin/apache-atlas-1.0.0/server/webapp/atl] > as/WEB-INF/lib/atlas-repository-1.0.0.jar!/org/apache/atlas/GraphTransactionInterceptor.class]: > Unsatisfied dependency expressed through constructor parameter 0; nested > exception > is org.springframework.beans.factory.BeanCreationException: Error creating > bean with name 'get' defined in > org.apache.atlas.repository.graph.AtlasGraphProvider: Bean instantiati > on via factory method failed; nested exception is > org.springframework.beans.BeanInstantiationException: Failed to instantiate > [org.apache.atlas.repository.graphdb.AtlasGraph]: Fa > ctory method 'get' threw exception; nested exception is > java.lang.IllegalArgumentException: Could not instantiate implementation: > org.janusgraph.diskstorage.cassandra.astyanax.As > tyanaxStoreManager (AbstractApplicationContext:551) > 2018-06-11 11:54:23,402 ERROR - [main:] ~ Context initialization failed > (ContextLoader:350) > org.springframework.beans.factory.BeanCreationException: Error creating bean > with name 'org.springframework.context.event.internalEventListenerProcessor': > BeanPostProcessor befor > e instantiation of bean failed; nested exception is > org.springframework.beans.factory.UnsatisfiedDependencyException: Error > creating bean with name 'graphTransactionAdvisor' defi > ned in URL > [jar:[file:/data/apache-atlas/apache-atlas-sources-1.0.0/distro/target/apache-atlas-1.0.0-bin/apache-atlas-1.0.0/server/webapp/atlas/WEB-INF/lib/atlas-repository-1.0.0|file:///data/apache-atlas/apache-atlas-sources-1.0.0/distro/target/apache-atlas-1.0.0-bin/apache-atlas-1.0.0/server/webapp/atlas/WEB-INF/lib/atlas-repository-1.0.0]. > jar!/org/apache/atlas/GraphTransactionAdvisor.class]: Unsatisfied dependency > expressed through constructor parameter 0; nested exception is > org.springframework.beans.factory.Unsa > tisfiedDependencyException: Error creating bean with name > 'graphTransactionInterceptor' defined in URL > [jar:[file:/data/apache-atlas/apache-atlas-sources-1.0.0/distro/target/apach|file:///data/apache-atlas/apache-atlas-sources-1.0.0/distro/target/apach] > e-atlas-1.0.0-bin/apache-atlas-1.0.0/server/webapp/atlas/WEB-INF/lib/atlas-repository-1.0.0.jar!/org/apache/atlas/GraphTransactionInterceptor.class]: > Unsatisfied dependency expre > ssed through constructor parameter 0; nested exception is > org.springframework.beans.factory.BeanCreationException: Error creating bean >
[jira] [Commented] (ATLAS-2752) Atlas start up fail with error "Error creating bean with name 'graphTransactionAdvisor'
[ https://issues.apache.org/jira/browse/ATLAS-2752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16508176#comment-16508176 ] Pierre Padovani commented on ATLAS-2752: A few questions for you: * What version of Cassandra are you using? * Did you enable thrift? Pierre > Atlas start up fail with error "Error creating bean with name > 'graphTransactionAdvisor' > --- > > Key: ATLAS-2752 > URL: https://issues.apache.org/jira/browse/ATLAS-2752 > Project: Atlas > Issue Type: Bug > Components: atlas-core, atlas-webui >Affects Versions: 1.0.0 > Environment: test >Reporter: Mahesh Kakol >Priority: Major > > I am getting below while starting atlas and graph stoage as cassandra > > > 2018-06-11 11:54:21,226 WARN - [main:] ~ Using @Deprecated Class > org.springframework.web.util.Log4jConfigListener (DeprecationWarning:43) > 2018-06-11 11:54:23,400 WARN - [main:] ~ Exception encountered during context > initialization - cancelling refresh attempt: > org.springframework.beans.factory.BeanCreationExceptio > n: Error creating bean with name > 'org.springframework.context.event.internalEventListenerProcessor': > BeanPostProcessor before instantiation of bean failed; nested exception is or > g.springframework.beans.factory.UnsatisfiedDependencyException: Error > creating bean with name 'graphTransactionAdvisor' defined in URL > [jar:[file:/data/apache-atlas/apache-atlas-s|file:///data/apache-atlas/apache-atlas-s] > ources-1.0.0/distro/target/apache-atlas-1.0.0-bin/apache-atlas-1.0.0/server/webapp/atlas/WEB-INF/lib/atlas-repository-1.0.0.jar!/org/apache/atlas/GraphTransactionAdvisor.class]: > Unsatisfied dependency expressed through constructor parameter 0; nested > exception is > org.springframework.beans.factory.UnsatisfiedDependencyException: Error > creating bean with n > ame 'graphTransactionInterceptor' defined in URL > [jar:[file:/data/apache-atlas/apache-atlas-sources-1.0.0/distro/target/apache-atlas-1.0.0-bin/apache-atlas-1.0.0/server/webapp/atl|file:///data/apache-atlas/apache-atlas-sources-1.0.0/distro/target/apache-atlas-1.0.0-bin/apache-atlas-1.0.0/server/webapp/atl] > as/WEB-INF/lib/atlas-repository-1.0.0.jar!/org/apache/atlas/GraphTransactionInterceptor.class]: > Unsatisfied dependency expressed through constructor parameter 0; nested > exception > is org.springframework.beans.factory.BeanCreationException: Error creating > bean with name 'get' defined in > org.apache.atlas.repository.graph.AtlasGraphProvider: Bean instantiati > on via factory method failed; nested exception is > org.springframework.beans.BeanInstantiationException: Failed to instantiate > [org.apache.atlas.repository.graphdb.AtlasGraph]: Fa > ctory method 'get' threw exception; nested exception is > java.lang.IllegalArgumentException: Could not instantiate implementation: > org.janusgraph.diskstorage.cassandra.astyanax.As > tyanaxStoreManager (AbstractApplicationContext:551) > 2018-06-11 11:54:23,402 ERROR - [main:] ~ Context initialization failed > (ContextLoader:350) > org.springframework.beans.factory.BeanCreationException: Error creating bean > with name 'org.springframework.context.event.internalEventListenerProcessor': > BeanPostProcessor befor > e instantiation of bean failed; nested exception is > org.springframework.beans.factory.UnsatisfiedDependencyException: Error > creating bean with name 'graphTransactionAdvisor' defi > ned in URL > [jar:[file:/data/apache-atlas/apache-atlas-sources-1.0.0/distro/target/apache-atlas-1.0.0-bin/apache-atlas-1.0.0/server/webapp/atlas/WEB-INF/lib/atlas-repository-1.0.0|file:///data/apache-atlas/apache-atlas-sources-1.0.0/distro/target/apache-atlas-1.0.0-bin/apache-atlas-1.0.0/server/webapp/atlas/WEB-INF/lib/atlas-repository-1.0.0]. > jar!/org/apache/atlas/GraphTransactionAdvisor.class]: Unsatisfied dependency > expressed through constructor parameter 0; nested exception is > org.springframework.beans.factory.Unsa > tisfiedDependencyException: Error creating bean with name > 'graphTransactionInterceptor' defined in URL > [jar:[file:/data/apache-atlas/apache-atlas-sources-1.0.0/distro/target/apach|file:///data/apache-atlas/apache-atlas-sources-1.0.0/distro/target/apach] > e-atlas-1.0.0-bin/apache-atlas-1.0.0/server/webapp/atlas/WEB-INF/lib/atlas-repository-1.0.0.jar!/org/apache/atlas/GraphTransactionInterceptor.class]: > Unsatisfied dependency expre > ssed through constructor parameter 0; nested exception is > org.springframework.beans.factory.BeanCreationException: Error creating bean > with name 'get' defined in org.apache.atlas > .repository.graph.AtlasGraphProvider: Bean instantiation via factory method > failed; nested exception is > org.springframework.beans.BeanInstantiationExcepti
[jira] [Resolved] (ATLAS-2727) Cannot delete a child of a COMPOSITION/AGGREGATION relationship
[
https://issues.apache.org/jira/browse/ATLAS-2727?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pierre Padovani resolved ATLAS-2727.
Resolution: Fixed
Fix Version/s: 1.0.0
> Cannot delete a child of a COMPOSITION/AGGREGATION relationship
> ---
>
> Key: ATLAS-2727
> URL: https://issues.apache.org/jira/browse/ATLAS-2727
> Project: Atlas
> Issue Type: Bug
>Affects Versions: 1.0.0
>Reporter: Pierre Padovani
>Priority: Critical
> Fix For: 1.0.0
>
>
> I am unable to delete a child of a composition/aggregation relationship. I
> get this exception:
> {code:java}
> java.lang.IllegalArgumentException: Invalid edge label r:DiscoveryPackTables:
> expected 2 or 3 label components but found 1 at
> org.apache.atlas.repository.graph.AtlasEdgeLabel.(AtlasEdgeLabel.java:37)
> at
> org.apache.atlas.repository.store.graph.v1.DeleteHandlerV1.getAttributeForEdge(DeleteHandlerV1.java:722)
> at
> org.apache.atlas.repository.store.graph.v1.DeleteHandlerV1.deleteVertex(DeleteHandlerV1.java:865)
> at
> org.apache.atlas.repository.store.graph.v1.DeleteHandlerV1.deleteTypeVertex(DeleteHandlerV1.java:718)
> at
> org.apache.atlas.repository.store.graph.v1.DeleteHandlerV1.deleteEntities(DeleteHandlerV1.java:140)
> at
> org.apache.atlas.repository.store.graph.v1.AtlasEntityStoreV1.deleteVertices(AtlasEntityStoreV1.java:704)
> at
> org.apache.atlas.repository.store.graph.v1.AtlasEntityStoreV1.deleteById(AtlasEntityStoreV1.java:297)
> at
> org.apache.atlas.repository.store.graph.v1.AtlasEntityStoreV1$$FastClassBySpringCGLIB$$80c00649.invoke()
> at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
> at
> org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:738)
> at
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
> at
> org.apache.atlas.GraphTransactionInterceptor.invoke(GraphTransactionInterceptor.java:75)
> at
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
> at
> org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:673)
> at
> org.apache.atlas.repository.store.graph.v1.AtlasEntityStoreV1$$EnhancerBySpringCGLIB$$2072786c.deleteById()
> at org.apache.atlas.web.rest.EntityREST.deleteByGuid(EntityREST.java:327) at
> sun.reflect.GeneratedMethodAccessor231.invoke(Unknown Source) at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498) at
> com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60)
> {code}
> Type system:
> {code:java}
> Entites:
> {
> "name": "DiscoveryPack",
> "superTypes": [
> "DataSet"
> ],
> "typeVersion": "0.1",
> "attributeDefs": [
> ]
> },
> {
> "name": "DiscoveryPackTable",
> "superTypes": [
> "Table"
> ],
> "typeVersion": "0.1",
> "attributeDefs": [
> ]
> },
> {
> "name": "Table",
> "superTypes": [
> "DataSet"
> ],
> "typeVersion": "0.1",
> "attributeDefs": [
> ]
> }
> relationship:
> {
> "endDef1": {
> "type": "DiscoveryPack",
> "name": "tables",
> "isContainer": true,
> "cardinality": "SET",
> "isLegacyAttribute": false
> },
> "endDef2": {
> "type": "DiscoveryPackTable",
> "name": "discoveryPack",
> "isContainer": false,
> "cardinality": "SINGLE",
> "isLegacyAttribute": false
> },
> "name": "DiscoveryPackTables",
> "propagateTags": "ONE_TO_TWO",
> "relationshipCategory": "COMPOSITION",
> "typeVersion": "1.0"
> }{code}
>
> * Create a DiscoveryPack
> * Create a DiscoveryPackTable.
> * Create a relationship between them
> * Delete the DiscoveryPackTable created previously
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Closed] (ATLAS-2727) Cannot delete a child of a COMPOSITION/AGGREGATION relationship
[
https://issues.apache.org/jira/browse/ATLAS-2727?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pierre Padovani closed ATLAS-2727.
--
> Cannot delete a child of a COMPOSITION/AGGREGATION relationship
> ---
>
> Key: ATLAS-2727
> URL: https://issues.apache.org/jira/browse/ATLAS-2727
> Project: Atlas
> Issue Type: Bug
>Affects Versions: 1.0.0
>Reporter: Pierre Padovani
>Priority: Critical
> Fix For: 1.0.0
>
>
> I am unable to delete a child of a composition/aggregation relationship. I
> get this exception:
> {code:java}
> java.lang.IllegalArgumentException: Invalid edge label r:DiscoveryPackTables:
> expected 2 or 3 label components but found 1 at
> org.apache.atlas.repository.graph.AtlasEdgeLabel.(AtlasEdgeLabel.java:37)
> at
> org.apache.atlas.repository.store.graph.v1.DeleteHandlerV1.getAttributeForEdge(DeleteHandlerV1.java:722)
> at
> org.apache.atlas.repository.store.graph.v1.DeleteHandlerV1.deleteVertex(DeleteHandlerV1.java:865)
> at
> org.apache.atlas.repository.store.graph.v1.DeleteHandlerV1.deleteTypeVertex(DeleteHandlerV1.java:718)
> at
> org.apache.atlas.repository.store.graph.v1.DeleteHandlerV1.deleteEntities(DeleteHandlerV1.java:140)
> at
> org.apache.atlas.repository.store.graph.v1.AtlasEntityStoreV1.deleteVertices(AtlasEntityStoreV1.java:704)
> at
> org.apache.atlas.repository.store.graph.v1.AtlasEntityStoreV1.deleteById(AtlasEntityStoreV1.java:297)
> at
> org.apache.atlas.repository.store.graph.v1.AtlasEntityStoreV1$$FastClassBySpringCGLIB$$80c00649.invoke()
> at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
> at
> org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:738)
> at
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
> at
> org.apache.atlas.GraphTransactionInterceptor.invoke(GraphTransactionInterceptor.java:75)
> at
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
> at
> org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:673)
> at
> org.apache.atlas.repository.store.graph.v1.AtlasEntityStoreV1$$EnhancerBySpringCGLIB$$2072786c.deleteById()
> at org.apache.atlas.web.rest.EntityREST.deleteByGuid(EntityREST.java:327) at
> sun.reflect.GeneratedMethodAccessor231.invoke(Unknown Source) at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498) at
> com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60)
> {code}
> Type system:
> {code:java}
> Entites:
> {
> "name": "DiscoveryPack",
> "superTypes": [
> "DataSet"
> ],
> "typeVersion": "0.1",
> "attributeDefs": [
> ]
> },
> {
> "name": "DiscoveryPackTable",
> "superTypes": [
> "Table"
> ],
> "typeVersion": "0.1",
> "attributeDefs": [
> ]
> },
> {
> "name": "Table",
> "superTypes": [
> "DataSet"
> ],
> "typeVersion": "0.1",
> "attributeDefs": [
> ]
> }
> relationship:
> {
> "endDef1": {
> "type": "DiscoveryPack",
> "name": "tables",
> "isContainer": true,
> "cardinality": "SET",
> "isLegacyAttribute": false
> },
> "endDef2": {
> "type": "DiscoveryPackTable",
> "name": "discoveryPack",
> "isContainer": false,
> "cardinality": "SINGLE",
> "isLegacyAttribute": false
> },
> "name": "DiscoveryPackTables",
> "propagateTags": "ONE_TO_TWO",
> "relationshipCategory": "COMPOSITION",
> "typeVersion": "1.0"
> }{code}
>
> * Create a DiscoveryPack
> * Create a DiscoveryPackTable.
> * Create a relationship between them
> * Delete the DiscoveryPackTable created previously
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (ATLAS-2727) Cannot delete a child of a COMPOSITION/AGGREGATION relationship
Pierre Padovani created ATLAS-2727:
--
Summary: Cannot delete a child of a COMPOSITION/AGGREGATION
relationship
Key: ATLAS-2727
URL: https://issues.apache.org/jira/browse/ATLAS-2727
Project: Atlas
Issue Type: Bug
Affects Versions: 1.0.0
Reporter: Pierre Padovani
I am unable to delete a child of a composition/aggregation relationship. I get
this exception:
{code:java}
java.lang.IllegalArgumentException: Invalid edge label r:DiscoveryPackTables:
expected 2 or 3 label components but found 1 at
org.apache.atlas.repository.graph.AtlasEdgeLabel.(AtlasEdgeLabel.java:37)
at
org.apache.atlas.repository.store.graph.v1.DeleteHandlerV1.getAttributeForEdge(DeleteHandlerV1.java:722)
at
org.apache.atlas.repository.store.graph.v1.DeleteHandlerV1.deleteVertex(DeleteHandlerV1.java:865)
at
org.apache.atlas.repository.store.graph.v1.DeleteHandlerV1.deleteTypeVertex(DeleteHandlerV1.java:718)
at
org.apache.atlas.repository.store.graph.v1.DeleteHandlerV1.deleteEntities(DeleteHandlerV1.java:140)
at
org.apache.atlas.repository.store.graph.v1.AtlasEntityStoreV1.deleteVertices(AtlasEntityStoreV1.java:704)
at
org.apache.atlas.repository.store.graph.v1.AtlasEntityStoreV1.deleteById(AtlasEntityStoreV1.java:297)
at
org.apache.atlas.repository.store.graph.v1.AtlasEntityStoreV1$$FastClassBySpringCGLIB$$80c00649.invoke()
at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204) at
org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:738)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
at
org.apache.atlas.GraphTransactionInterceptor.invoke(GraphTransactionInterceptor.java:75)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
at
org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:673)
at
org.apache.atlas.repository.store.graph.v1.AtlasEntityStoreV1$$EnhancerBySpringCGLIB$$2072786c.deleteById()
at org.apache.atlas.web.rest.EntityREST.deleteByGuid(EntityREST.java:327) at
sun.reflect.GeneratedMethodAccessor231.invoke(Unknown Source) at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498) at
com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60)
{code}
Type system:
{code:java}
Entites:
{
"name": "DiscoveryPack",
"superTypes": [
"DataSet"
],
"typeVersion": "0.1",
"attributeDefs": [
]
},
{
"name": "DiscoveryPackTable",
"superTypes": [
"Table"
],
"typeVersion": "0.1",
"attributeDefs": [
]
},
{
"name": "Table",
"superTypes": [
"DataSet"
],
"typeVersion": "0.1",
"attributeDefs": [
]
}
relationship:
{
"endDef1": {
"type": "DiscoveryPack",
"name": "tables",
"isContainer": true,
"cardinality": "SET",
"isLegacyAttribute": false
},
"endDef2": {
"type": "DiscoveryPackTable",
"name": "discoveryPack",
"isContainer": false,
"cardinality": "SINGLE",
"isLegacyAttribute": false
},
"name": "DiscoveryPackTables",
"propagateTags": "ONE_TO_TWO",
"relationshipCategory": "COMPOSITION",
"typeVersion": "1.0"
}{code}
* Create a DiscoveryPack
* Create a DiscoveryPackTable.
* Create a relationship between them
* Delete the DiscoveryPackTable created previously
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (ATLAS-2564) AtlasBaseClient.java info logging too verbose
[ https://issues.apache.org/jira/browse/ATLAS-2564?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Padovani updated ATLAS-2564: --- Attachment: ATLAS-2564.patch > AtlasBaseClient.java info logging too verbose > - > > Key: ATLAS-2564 > URL: https://issues.apache.org/jira/browse/ATLAS-2564 > Project: Atlas > Issue Type: Bug >Affects Versions: 1.0.0-alpha, 1.0.0 >Reporter: Pierre Padovani >Assignee: Pierre Padovani >Priority: Major > Fix For: 1.0.0 > > Attachments: ATLAS-2564.patch > > Original Estimate: 1m > Remaining Estimate: 1m > > The AtlasBaseClient.java logs the full payload at the info level. For large > entities, this can fill up log files very quickly. These log lines should be > moved to log level debug, and a new log line put in place that provides a > single line summary of the operation with out printing out the > request/response payloads. This also solves a potential issue with PII if > someone decides to put PII into an entity. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (ATLAS-2564) AtlasBaseClient.java info logging too verbose
Pierre Padovani created ATLAS-2564: -- Summary: AtlasBaseClient.java info logging too verbose Key: ATLAS-2564 URL: https://issues.apache.org/jira/browse/ATLAS-2564 Project: Atlas Issue Type: Bug Affects Versions: 1.0.0-alpha, 1.0.0 Reporter: Pierre Padovani Assignee: Pierre Padovani Fix For: 1.0.0 The AtlasBaseClient.java logs the full payload at the info level. For large entities, this can fill up log files very quickly. These log lines should be moved to log level debug, and a new log line put in place that provides a single line summary of the operation with out printing out the request/response payloads. This also solves a potential issue with PII if someone decides to put PII into an entity. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Closed] (ATLAS-2470) Basic support for Cassandra
[ https://issues.apache.org/jira/browse/ATLAS-2470?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Padovani closed ATLAS-2470. -- > Basic support for Cassandra > > > Key: ATLAS-2470 > URL: https://issues.apache.org/jira/browse/ATLAS-2470 > Project: Atlas > Issue Type: Sub-task >Affects Versions: 1.0.0 >Reporter: Pierre Padovani >Assignee: Pierre Padovani >Priority: Major > Fix For: 1.0.0 > > Attachments: ATLAS-2470-1.patch, ATLAS-2470-2.patch, > ATLAS-2470-3.patch, ATLAS-2470-5.patch, ATLAS-2470.patch > > > Add the basic build support, and ability to run embedded Cassandra and solr > using a profile build: > -Pdist,embedded-cassandra-solr -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (ATLAS-2473) Add Audit support w/o HBase
[ https://issues.apache.org/jira/browse/ATLAS-2473?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Padovani reassigned ATLAS-2473: -- Assignee: Pierre Padovani > Add Audit support w/o HBase > --- > > Key: ATLAS-2473 > URL: https://issues.apache.org/jira/browse/ATLAS-2473 > Project: Atlas > Issue Type: Sub-task >Reporter: Pierre Padovani >Assignee: Pierre Padovani >Priority: Major > Fix For: 1.0.0 > > > Currently you can only have no audit, or audit on hbase. Need to add an audit > handler that can talk Cassandra or find another alternative. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (ATLAS-2547) Fix CassandraAuditRepositoryTest.java to wait for Cassandra server start
[ https://issues.apache.org/jira/browse/ATLAS-2547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Padovani resolved ATLAS-2547. Resolution: Fixed Fix Version/s: 1.0.0 > Fix CassandraAuditRepositoryTest.java to wait for Cassandra server start > > > Key: ATLAS-2547 > URL: https://issues.apache.org/jira/browse/ATLAS-2547 > Project: Atlas > Issue Type: Sub-task >Reporter: Pierre Padovani >Assignee: Pierre Padovani >Priority: Major > Fix For: 1.0.0 > > > Remove hard coded Thread.sleep and instead provide a basic looping mechanism > to test for an active Cassandra server. Loop should abort and throw an > exception if no connection can be achieved within a timeout/number of loops. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Closed] (ATLAS-2473) Add Audit support w/o HBase
[ https://issues.apache.org/jira/browse/ATLAS-2473?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Padovani closed ATLAS-2473. -- > Add Audit support w/o HBase > --- > > Key: ATLAS-2473 > URL: https://issues.apache.org/jira/browse/ATLAS-2473 > Project: Atlas > Issue Type: Sub-task >Reporter: Pierre Padovani >Priority: Major > Fix For: 1.0.0 > > > Currently you can only have no audit, or audit on hbase. Need to add an audit > handler that can talk Cassandra or find another alternative. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Closed] (ATLAS-2547) Fix CassandraAuditRepositoryTest.java to wait for Cassandra server start
[ https://issues.apache.org/jira/browse/ATLAS-2547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Padovani closed ATLAS-2547. -- > Fix CassandraAuditRepositoryTest.java to wait for Cassandra server start > > > Key: ATLAS-2547 > URL: https://issues.apache.org/jira/browse/ATLAS-2547 > Project: Atlas > Issue Type: Sub-task >Reporter: Pierre Padovani >Assignee: Pierre Padovani >Priority: Major > Fix For: 1.0.0 > > > Remove hard coded Thread.sleep and instead provide a basic looping mechanism > to test for an active Cassandra server. Loop should abort and throw an > exception if no connection can be achieved within a timeout/number of loops. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (ATLAS-2547) Fix CassandraAuditRepositoryTest.java to wait for Cassandra server start
Pierre Padovani created ATLAS-2547: -- Summary: Fix CassandraAuditRepositoryTest.java to wait for Cassandra server start Key: ATLAS-2547 URL: https://issues.apache.org/jira/browse/ATLAS-2547 Project: Atlas Issue Type: Sub-task Reporter: Pierre Padovani Assignee: Pierre Padovani Remove hard coded Thread.sleep and instead provide a basic looping mechanism to test for an active Cassandra server. Loop should abort and throw an exception if no connection can be achieved within a timeout/number of loops. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (ATLAS-2473) Add Audit support w/o HBase
[ https://issues.apache.org/jira/browse/ATLAS-2473?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Padovani resolved ATLAS-2473. Resolution: Fixed Fix Version/s: 1.0.0 > Add Audit support w/o HBase > --- > > Key: ATLAS-2473 > URL: https://issues.apache.org/jira/browse/ATLAS-2473 > Project: Atlas > Issue Type: Sub-task >Reporter: Pierre Padovani >Priority: Major > Fix For: 1.0.0 > > > Currently you can only have no audit, or audit on hbase. Need to add an audit > handler that can talk Cassandra or find another alternative. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (ATLAS-2470) Basic support for Cassandra
[ https://issues.apache.org/jira/browse/ATLAS-2470?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Padovani updated ATLAS-2470: --- Attachment: ATLAS-2470-5.patch > Basic support for Cassandra > > > Key: ATLAS-2470 > URL: https://issues.apache.org/jira/browse/ATLAS-2470 > Project: Atlas > Issue Type: Sub-task >Affects Versions: 1.0.0 >Reporter: Pierre Padovani >Assignee: Pierre Padovani >Priority: Major > Fix For: 1.0.0 > > Attachments: ATLAS-2470-1.patch, ATLAS-2470-2.patch, > ATLAS-2470-3.patch, ATLAS-2470-5.patch, ATLAS-2470.patch > > > Add the basic build support, and ability to run embedded Cassandra and solr > using a profile build: > -Pdist,embedded-cassandra-solr -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (ATLAS-2471) Create Dockerfile support for Cassandra
[ https://issues.apache.org/jira/browse/ATLAS-2471?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16406483#comment-16406483 ] Pierre Padovani commented on ATLAS-2471: Before a Dockerfile can be incorporated, the Cassandra code must be merged to master. > Create Dockerfile support for Cassandra > --- > > Key: ATLAS-2471 > URL: https://issues.apache.org/jira/browse/ATLAS-2471 > Project: Atlas > Issue Type: Sub-task >Reporter: Pierre Padovani >Assignee: Pierre Padovani >Priority: Major > > Add an additional Dockerfile that builds a self contained container that runs > on Cassandra -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (ATLAS-2472) Update Atlas documentation for Cassandra support
[ https://issues.apache.org/jira/browse/ATLAS-2472?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Padovani resolved ATLAS-2472. Resolution: Fixed Added the full documentation as part of ATLAS-2470 > Update Atlas documentation for Cassandra support > > > Key: ATLAS-2472 > URL: https://issues.apache.org/jira/browse/ATLAS-2472 > Project: Atlas > Issue Type: Sub-task >Reporter: Pierre Padovani >Assignee: Pierre Padovani >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Closed] (ATLAS-2472) Update Atlas documentation for Cassandra support
[ https://issues.apache.org/jira/browse/ATLAS-2472?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Padovani closed ATLAS-2472. -- > Update Atlas documentation for Cassandra support > > > Key: ATLAS-2472 > URL: https://issues.apache.org/jira/browse/ATLAS-2472 > Project: Atlas > Issue Type: Sub-task >Reporter: Pierre Padovani >Assignee: Pierre Padovani >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (ATLAS-2470) Basic support for Cassandra
[ https://issues.apache.org/jira/browse/ATLAS-2470?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Padovani updated ATLAS-2470: --- Attachment: ATLAS-2470-3.patch > Basic support for Cassandra > > > Key: ATLAS-2470 > URL: https://issues.apache.org/jira/browse/ATLAS-2470 > Project: Atlas > Issue Type: Sub-task >Affects Versions: 1.0.0 >Reporter: Pierre Padovani >Assignee: Pierre Padovani >Priority: Major > Fix For: 1.0.0 > > Attachments: ATLAS-2470-1.patch, ATLAS-2470-2.patch, > ATLAS-2470-3.patch, ATLAS-2470.patch > > > Add the basic build support, and ability to run embedded Cassandra and solr > using a profile build: > -Pdist,embedded-cassandra-solr -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (ATLAS-2478) Elasticsearch support is broken for JanusGraph
[
https://issues.apache.org/jira/browse/ATLAS-2478?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pierre Padovani updated ATLAS-2478:
---
Attachment: ATLAS-2478.patch
> Elasticsearch support is broken for JanusGraph
> --
>
> Key: ATLAS-2478
> URL: https://issues.apache.org/jira/browse/ATLAS-2478
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Affects Versions: 1.0.0-alpha
>Reporter: Pierre Padovani
>Assignee: Pierre Padovani
>Priority: Major
> Fix For: 1.0.0
>
> Attachments: ATLAS-2478.patch
>
>
> With JanusGraph the Elasticsearch support moved to 5.x+. This introduced a
> change where fields that contained '.' (dots) were not allowed unless either
> a specific cluster wide setting was enabled AND the mapping was formatted
> such that each of the fields that contained a '.' could be considered part of
> an object.
> Example:
> {code:java}
> foo.x
> foo.y
> foo.z{code}
> Elasticsearch looks at these fields as if they are truly:
> {code:java}
> foo : {
> x,
> y,
> z
> }{code}
> In the file:
> /atlas/common/src/main/java/org/apache/atlas/repository/Constants.java
> {code:java}
> /**
> * Properties for type store graph.
> */
> public static final String TYPE_CATEGORY_PROPERTY_KEY =
> INTERNAL_PROPERTY_KEY_PREFIX + "type.category";
> public static final String VERTEX_TYPE_PROPERTY_KEY =
> INTERNAL_PROPERTY_KEY_PREFIX + "type";
> public static final String TYPENAME_PROPERTY_KEY =
> INTERNAL_PROPERTY_KEY_PREFIX + "type.name";
> public static final String TYPEDESCRIPTION_PROPERTY_KEY =
> INTERNAL_PROPERTY_KEY_PREFIX + "type.description";
> public static final String TYPEVERSION_PROPERTY_KEY =
> INTERNAL_PROPERTY_KEY_PREFIX + "type.version";
> public static final String TYPEOPTIONS_PROPERTY_KEY =
> INTERNAL_PROPERTY_KEY_PREFIX + "type.options";
> {code}
> These are the only fields that cause Elasticsearch issue. As you can see a
> field called 'type' is created, then additional fields type.name,
> type.description etc. This will cause a mapping conflict exception in
> Elasticsearch and it will refuse to create the mapping.
>
> The easy fix is to simply replace the '.' with an '_' (underscore) but this
> will be a backwards incompatible change for existing customers.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (ATLAS-2478) Elasticsearch support is broken for JanusGraph
[
https://issues.apache.org/jira/browse/ATLAS-2478?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pierre Padovani updated ATLAS-2478:
---
Attachment: (was: ATLAS-2478.patch)
> Elasticsearch support is broken for JanusGraph
> --
>
> Key: ATLAS-2478
> URL: https://issues.apache.org/jira/browse/ATLAS-2478
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Affects Versions: 1.0.0-alpha
>Reporter: Pierre Padovani
>Assignee: Pierre Padovani
>Priority: Major
> Fix For: 1.0.0
>
>
> With JanusGraph the Elasticsearch support moved to 5.x+. This introduced a
> change where fields that contained '.' (dots) were not allowed unless either
> a specific cluster wide setting was enabled AND the mapping was formatted
> such that each of the fields that contained a '.' could be considered part of
> an object.
> Example:
> {code:java}
> foo.x
> foo.y
> foo.z{code}
> Elasticsearch looks at these fields as if they are truly:
> {code:java}
> foo : {
> x,
> y,
> z
> }{code}
> In the file:
> /atlas/common/src/main/java/org/apache/atlas/repository/Constants.java
> {code:java}
> /**
> * Properties for type store graph.
> */
> public static final String TYPE_CATEGORY_PROPERTY_KEY =
> INTERNAL_PROPERTY_KEY_PREFIX + "type.category";
> public static final String VERTEX_TYPE_PROPERTY_KEY =
> INTERNAL_PROPERTY_KEY_PREFIX + "type";
> public static final String TYPENAME_PROPERTY_KEY =
> INTERNAL_PROPERTY_KEY_PREFIX + "type.name";
> public static final String TYPEDESCRIPTION_PROPERTY_KEY =
> INTERNAL_PROPERTY_KEY_PREFIX + "type.description";
> public static final String TYPEVERSION_PROPERTY_KEY =
> INTERNAL_PROPERTY_KEY_PREFIX + "type.version";
> public static final String TYPEOPTIONS_PROPERTY_KEY =
> INTERNAL_PROPERTY_KEY_PREFIX + "type.options";
> {code}
> These are the only fields that cause Elasticsearch issue. As you can see a
> field called 'type' is created, then additional fields type.name,
> type.description etc. This will cause a mapping conflict exception in
> Elasticsearch and it will refuse to create the mapping.
>
> The easy fix is to simply replace the '.' with an '_' (underscore) but this
> will be a backwards incompatible change for existing customers.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (ATLAS-2478) Elasticsearch support is broken for JanusGraph
[
https://issues.apache.org/jira/browse/ATLAS-2478?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pierre Padovani updated ATLAS-2478:
---
Attachment: ATLAS-2478.patch
> Elasticsearch support is broken for JanusGraph
> --
>
> Key: ATLAS-2478
> URL: https://issues.apache.org/jira/browse/ATLAS-2478
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Affects Versions: 1.0.0-alpha
>Reporter: Pierre Padovani
>Assignee: Pierre Padovani
>Priority: Major
> Fix For: 1.0.0
>
> Attachments: ATLAS-2478.patch
>
>
> With JanusGraph the Elasticsearch support moved to 5.x+. This introduced a
> change where fields that contained '.' (dots) were not allowed unless either
> a specific cluster wide setting was enabled AND the mapping was formatted
> such that each of the fields that contained a '.' could be considered part of
> an object.
> Example:
> {code:java}
> foo.x
> foo.y
> foo.z{code}
> Elasticsearch looks at these fields as if they are truly:
> {code:java}
> foo : {
> x,
> y,
> z
> }{code}
> In the file:
> /atlas/common/src/main/java/org/apache/atlas/repository/Constants.java
> {code:java}
> /**
> * Properties for type store graph.
> */
> public static final String TYPE_CATEGORY_PROPERTY_KEY =
> INTERNAL_PROPERTY_KEY_PREFIX + "type.category";
> public static final String VERTEX_TYPE_PROPERTY_KEY =
> INTERNAL_PROPERTY_KEY_PREFIX + "type";
> public static final String TYPENAME_PROPERTY_KEY =
> INTERNAL_PROPERTY_KEY_PREFIX + "type.name";
> public static final String TYPEDESCRIPTION_PROPERTY_KEY =
> INTERNAL_PROPERTY_KEY_PREFIX + "type.description";
> public static final String TYPEVERSION_PROPERTY_KEY =
> INTERNAL_PROPERTY_KEY_PREFIX + "type.version";
> public static final String TYPEOPTIONS_PROPERTY_KEY =
> INTERNAL_PROPERTY_KEY_PREFIX + "type.options";
> {code}
> These are the only fields that cause Elasticsearch issue. As you can see a
> field called 'type' is created, then additional fields type.name,
> type.description etc. This will cause a mapping conflict exception in
> Elasticsearch and it will refuse to create the mapping.
>
> The easy fix is to simply replace the '.' with an '_' (underscore) but this
> will be a backwards incompatible change for existing customers.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (ATLAS-2470) Basic support for Cassandra
[ https://issues.apache.org/jira/browse/ATLAS-2470?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16391565#comment-16391565 ] Pierre Padovani commented on ATLAS-2470: [~davidrad] Added an updated patch that includes: * Audit support for Cassandra as a storage backend * Refactored audit unit tests for audit to support adding a Cassandra unit test * Updated audit unit tests to test the V2 audit code for both in memory and Cassandra * Updates to application.properties and distro pom.xml > Basic support for Cassandra > > > Key: ATLAS-2470 > URL: https://issues.apache.org/jira/browse/ATLAS-2470 > Project: Atlas > Issue Type: Sub-task >Affects Versions: 1.0.0 >Reporter: Pierre Padovani >Assignee: Pierre Padovani >Priority: Major > Fix For: 1.0.0 > > Attachments: ATLAS-2470-1.patch, ATLAS-2470-2.patch, ATLAS-2470.patch > > > Add the basic build support, and ability to run embedded Cassandra and solr > using a profile build: > -Pdist,embedded-cassandra-solr -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (ATLAS-2470) Basic support for Cassandra
[ https://issues.apache.org/jira/browse/ATLAS-2470?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Padovani updated ATLAS-2470: --- Attachment: ATLAS-2470-2.patch > Basic support for Cassandra > > > Key: ATLAS-2470 > URL: https://issues.apache.org/jira/browse/ATLAS-2470 > Project: Atlas > Issue Type: Sub-task >Affects Versions: 1.0.0 >Reporter: Pierre Padovani >Assignee: Pierre Padovani >Priority: Major > Fix For: 1.0.0 > > Attachments: ATLAS-2470-1.patch, ATLAS-2470-2.patch, ATLAS-2470.patch > > > Add the basic build support, and ability to run embedded Cassandra and solr > using a profile build: > -Pdist,embedded-cassandra-solr -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (ATLAS-2478) Elasticsearch support is broken for JanusGraph
[
https://issues.apache.org/jira/browse/ATLAS-2478?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16386101#comment-16386101
]
Pierre Padovani commented on ATLAS-2478:
[~davidrad] Could I get your thoughts on this issue please? Include anyone else
you think might have some input.
Thanks!
> Elasticsearch support is broken for JanusGraph
> --
>
> Key: ATLAS-2478
> URL: https://issues.apache.org/jira/browse/ATLAS-2478
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Affects Versions: 1.0.0-alpha
>Reporter: Pierre Padovani
>Assignee: Pierre Padovani
>Priority: Major
> Fix For: 1.0.0
>
>
> With JanusGraph the Elasticsearch support moved to 5.x+. This introduced a
> change where fields that contained '.' (dots) were not allowed unless either
> a specific cluster wide setting was enabled AND the mapping was formatted
> such that each of the fields that contained a '.' could be considered part of
> an object.
> Example:
> {code:java}
> foo.x
> foo.y
> foo.z{code}
> Elasticsearch looks at these fields as if they are truly:
> {code:java}
> foo : {
> x,
> y,
> z
> }{code}
> In the file:
> /atlas/common/src/main/java/org/apache/atlas/repository/Constants.java
> {code:java}
> /**
> * Properties for type store graph.
> */
> public static final String TYPE_CATEGORY_PROPERTY_KEY =
> INTERNAL_PROPERTY_KEY_PREFIX + "type.category";
> public static final String VERTEX_TYPE_PROPERTY_KEY =
> INTERNAL_PROPERTY_KEY_PREFIX + "type";
> public static final String TYPENAME_PROPERTY_KEY =
> INTERNAL_PROPERTY_KEY_PREFIX + "type.name";
> public static final String TYPEDESCRIPTION_PROPERTY_KEY =
> INTERNAL_PROPERTY_KEY_PREFIX + "type.description";
> public static final String TYPEVERSION_PROPERTY_KEY =
> INTERNAL_PROPERTY_KEY_PREFIX + "type.version";
> public static final String TYPEOPTIONS_PROPERTY_KEY =
> INTERNAL_PROPERTY_KEY_PREFIX + "type.options";
> {code}
> These are the only fields that cause Elasticsearch issue. As you can see a
> field called 'type' is created, then additional fields type.name,
> type.description etc. This will cause a mapping conflict exception in
> Elasticsearch and it will refuse to create the mapping.
>
> The easy fix is to simply replace the '.' with an '_' (underscore) but this
> will be a backwards incompatible change for existing customers.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (ATLAS-2478) Elasticsearch support is broken for JanusGraph
Pierre Padovani created ATLAS-2478:
--
Summary: Elasticsearch support is broken for JanusGraph
Key: ATLAS-2478
URL: https://issues.apache.org/jira/browse/ATLAS-2478
Project: Atlas
Issue Type: Bug
Components: atlas-core
Affects Versions: 1.0.0-alpha
Reporter: Pierre Padovani
Assignee: Pierre Padovani
Fix For: 1.0.0
With JanusGraph the Elasticsearch support moved to 5.x+. This introduced a
change where fields that contained '.' (dots) were not allowed unless either a
specific cluster wide setting was enabled AND the mapping was formatted such
that each of the fields that contained a '.' could be considered part of an
object.
Example:
{code:java}
foo.x
foo.y
foo.z{code}
Elasticsearch looks at these fields as if they are truly:
{code:java}
foo : {
x,
y,
z
}{code}
In the file:
/atlas/common/src/main/java/org/apache/atlas/repository/Constants.java
{code:java}
/**
* Properties for type store graph.
*/
public static final String TYPE_CATEGORY_PROPERTY_KEY =
INTERNAL_PROPERTY_KEY_PREFIX + "type.category";
public static final String VERTEX_TYPE_PROPERTY_KEY =
INTERNAL_PROPERTY_KEY_PREFIX + "type";
public static final String TYPENAME_PROPERTY_KEY = INTERNAL_PROPERTY_KEY_PREFIX
+ "type.name";
public static final String TYPEDESCRIPTION_PROPERTY_KEY =
INTERNAL_PROPERTY_KEY_PREFIX + "type.description";
public static final String TYPEVERSION_PROPERTY_KEY =
INTERNAL_PROPERTY_KEY_PREFIX + "type.version";
public static final String TYPEOPTIONS_PROPERTY_KEY =
INTERNAL_PROPERTY_KEY_PREFIX + "type.options";
{code}
These are the only fields that cause Elasticsearch issue. As you can see a
field called 'type' is created, then additional fields type.name,
type.description etc. This will cause a mapping conflict exception in
Elasticsearch and it will refuse to create the mapping.
The easy fix is to simply replace the '.' with an '_' (underscore) but this
will be a backwards incompatible change for existing customers.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (ATLAS-2472) Update Atlas documentation for Cassandra support
[ https://issues.apache.org/jira/browse/ATLAS-2472?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Padovani reassigned ATLAS-2472: -- Assignee: Pierre Padovani > Update Atlas documentation for Cassandra support > > > Key: ATLAS-2472 > URL: https://issues.apache.org/jira/browse/ATLAS-2472 > Project: Atlas > Issue Type: Sub-task >Reporter: Pierre Padovani >Assignee: Pierre Padovani >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (ATLAS-2471) Create Dockerfile support for Cassandra
[ https://issues.apache.org/jira/browse/ATLAS-2471?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Padovani reassigned ATLAS-2471: -- Assignee: Pierre Padovani > Create Dockerfile support for Cassandra > --- > > Key: ATLAS-2471 > URL: https://issues.apache.org/jira/browse/ATLAS-2471 > Project: Atlas > Issue Type: Sub-task >Reporter: Pierre Padovani >Assignee: Pierre Padovani >Priority: Major > > Add an additional Dockerfile that builds a self contained container that runs > on Cassandra -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (ATLAS-2470) Basic support for Cassandra
[ https://issues.apache.org/jira/browse/ATLAS-2470?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Padovani updated ATLAS-2470: --- Attachment: ATLAS-2470-1.patch > Basic support for Cassandra > > > Key: ATLAS-2470 > URL: https://issues.apache.org/jira/browse/ATLAS-2470 > Project: Atlas > Issue Type: Sub-task >Affects Versions: 1.0.0 >Reporter: Pierre Padovani >Assignee: Pierre Padovani >Priority: Major > Fix For: 1.0.0 > > Attachments: ATLAS-2470-1.patch, ATLAS-2470.patch > > > Add the basic build support, and ability to run embedded Cassandra and solr > using a profile build: > -Pdist,embedded-cassandra-solr -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (ATLAS-2470) Basic support for Cassandra
[ https://issues.apache.org/jira/browse/ATLAS-2470?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Padovani updated ATLAS-2470: --- Attachment: (was: ATLAS-2470-1.patch) > Basic support for Cassandra > > > Key: ATLAS-2470 > URL: https://issues.apache.org/jira/browse/ATLAS-2470 > Project: Atlas > Issue Type: Sub-task >Affects Versions: 1.0.0 >Reporter: Pierre Padovani >Assignee: Pierre Padovani >Priority: Major > Fix For: 1.0.0 > > Attachments: ATLAS-2470-1.patch, ATLAS-2470.patch > > > Add the basic build support, and ability to run embedded Cassandra and solr > using a profile build: > -Pdist,embedded-cassandra-solr -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (ATLAS-2470) Basic support for Cassandra
[ https://issues.apache.org/jira/browse/ATLAS-2470?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16385407#comment-16385407 ] Pierre Padovani commented on ATLAS-2470: Added a new patch version with changes from the code review. > Basic support for Cassandra > > > Key: ATLAS-2470 > URL: https://issues.apache.org/jira/browse/ATLAS-2470 > Project: Atlas > Issue Type: Sub-task >Affects Versions: 1.0.0 >Reporter: Pierre Padovani >Assignee: Pierre Padovani >Priority: Major > Fix For: 1.0.0 > > Attachments: ATLAS-2470-1.patch, ATLAS-2470.patch > > > Add the basic build support, and ability to run embedded Cassandra and solr > using a profile build: > -Pdist,embedded-cassandra-solr -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (ATLAS-2470) Basic support for Cassandra
[ https://issues.apache.org/jira/browse/ATLAS-2470?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Padovani updated ATLAS-2470: --- Attachment: ATLAS-2470-1.patch > Basic support for Cassandra > > > Key: ATLAS-2470 > URL: https://issues.apache.org/jira/browse/ATLAS-2470 > Project: Atlas > Issue Type: Sub-task >Affects Versions: 1.0.0 >Reporter: Pierre Padovani >Assignee: Pierre Padovani >Priority: Major > Fix For: 1.0.0 > > Attachments: ATLAS-2470-1.patch, ATLAS-2470.patch > > > Add the basic build support, and ability to run embedded Cassandra and solr > using a profile build: > -Pdist,embedded-cassandra-solr -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (ATLAS-2470) Basic support for Cassandra
[ https://issues.apache.org/jira/browse/ATLAS-2470?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Padovani updated ATLAS-2470: --- Attachment: ATLAS-2470.patch > Basic support for Cassandra > > > Key: ATLAS-2470 > URL: https://issues.apache.org/jira/browse/ATLAS-2470 > Project: Atlas > Issue Type: Sub-task >Reporter: Pierre Padovani >Assignee: Pierre Padovani >Priority: Major > Attachments: ATLAS-2470.patch > > > Add the basic build support, and ability to run embedded Cassandra and solr > using a profile build: > -Pdist,embedded-cassandra-solr -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (ATLAS-2473) Add Audit support w/o HBase
Pierre Padovani created ATLAS-2473: -- Summary: Add Audit support w/o HBase Key: ATLAS-2473 URL: https://issues.apache.org/jira/browse/ATLAS-2473 Project: Atlas Issue Type: Sub-task Reporter: Pierre Padovani Currently you can only have no audit, or audit on hbase. Need to add an audit handler that can talk Cassandra or find another alternative. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (ATLAS-2470) Basic support for Cassandra
[ https://issues.apache.org/jira/browse/ATLAS-2470?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Padovani reassigned ATLAS-2470: -- Assignee: Pierre Padovani > Basic support for Cassandra > > > Key: ATLAS-2470 > URL: https://issues.apache.org/jira/browse/ATLAS-2470 > Project: Atlas > Issue Type: Sub-task >Reporter: Pierre Padovani >Assignee: Pierre Padovani >Priority: Major > > Add the basic build support, and ability to run embedded Cassandra and solr > using a profile build: > -Pdist,embedded-cassandra-solr -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (ATLAS-2472) Update Atlas documentation for Cassandra support
Pierre Padovani created ATLAS-2472: -- Summary: Update Atlas documentation for Cassandra support Key: ATLAS-2472 URL: https://issues.apache.org/jira/browse/ATLAS-2472 Project: Atlas Issue Type: Sub-task Reporter: Pierre Padovani -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (ATLAS-2470) Basic support for Cassandra
Pierre Padovani created ATLAS-2470: -- Summary: Basic support for Cassandra Key: ATLAS-2470 URL: https://issues.apache.org/jira/browse/ATLAS-2470 Project: Atlas Issue Type: Sub-task Reporter: Pierre Padovani Add the basic build support, and ability to run embedded Cassandra and solr using a profile build: -Pdist,embedded-cassandra-solr -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (ATLAS-2471) Create Dockerfile support for Cassandra
Pierre Padovani created ATLAS-2471: -- Summary: Create Dockerfile support for Cassandra Key: ATLAS-2471 URL: https://issues.apache.org/jira/browse/ATLAS-2471 Project: Atlas Issue Type: Sub-task Reporter: Pierre Padovani Add an additional Dockerfile that builds a self contained container that runs on Cassandra -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (ATLAS-832) atlas_start should start embedded hbase and solr with -Pembedded_services
[
https://issues.apache.org/jira/browse/ATLAS-832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16382510#comment-16382510
]
Pierre Padovani edited comment on ATLAS-832 at 3/1/18 7:30 PM:
---
The issue is this line in atlas_config.py:
{noformat}
ENV_KEYS = ["JAVA_HOME", ATLAS_OPTS, ATLAS_SERVER_OPTS, ATLAS_SERVER_HEAP,
ATLAS_LOG, ATLAS_PID, ATLAS_CONF,
"ATLASCPPATH", ATLAS_DATA, ATLAS_HOME, ATLAS_WEBAPP, HBASE_CONF_DIR,
SOLR_PORT]{noformat}
Is missing
MANAGE_LOCAL_HBASE
MANAGE_LOCAL_SOLR
was (Author: ppadovani):
The issue is this line in config.py:
```
ENV_KEYS = ["JAVA_HOME", ATLAS_OPTS, ATLAS_SERVER_OPTS, ATLAS_SERVER_HEAP,
ATLAS_LOG, ATLAS_PID, ATLAS_CONF,
"ATLASCPPATH", ATLAS_DATA, ATLAS_HOME, ATLAS_WEBAPP, HBASE_CONF_DIR, SOLR_PORT]
```
Is missing
MANAGE_LOCAL_HBASE
MANAGE_LOCAL_SOLR
> atlas_start should start embedded hbase and solr with -Pembedded_services
> -
>
> Key: ATLAS-832
> URL: https://issues.apache.org/jira/browse/ATLAS-832
> Project: Atlas
> Issue Type: Bug
>Reporter: Shwetha G S
>Assignee: Vimal Sharma
>Priority: Major
>
> As part of ATLAS-823, embedded hbase and solr was moved to profile
> 'embedded_services'. After the package is built using '-Pembedded_services',
> the package configuration points to local hbase and solr, but atlas_start
> doesn't start embedded hbase and solr
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (ATLAS-832) atlas_start should start embedded hbase and solr with -Pembedded_services
[ https://issues.apache.org/jira/browse/ATLAS-832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16382510#comment-16382510 ] Pierre Padovani edited comment on ATLAS-832 at 3/1/18 7:30 PM: --- The issue is this line in config.py: ``` ENV_KEYS = ["JAVA_HOME", ATLAS_OPTS, ATLAS_SERVER_OPTS, ATLAS_SERVER_HEAP, ATLAS_LOG, ATLAS_PID, ATLAS_CONF, "ATLASCPPATH", ATLAS_DATA, ATLAS_HOME, ATLAS_WEBAPP, HBASE_CONF_DIR, SOLR_PORT] ``` Is missing MANAGE_LOCAL_HBASE MANAGE_LOCAL_SOLR was (Author: ppadovani): The issue is this line in config.py: ```ENV_KEYS = ["JAVA_HOME", ATLAS_OPTS, ATLAS_SERVER_OPTS, ATLAS_SERVER_HEAP, ATLAS_LOG, ATLAS_PID, ATLAS_CONF, "ATLASCPPATH", ATLAS_DATA, ATLAS_HOME, ATLAS_WEBAPP, HBASE_CONF_DIR, SOLR_PORT]``` Is missing MANAGE_LOCAL_HBASE MANAGE_LOCAL_SOLR > atlas_start should start embedded hbase and solr with -Pembedded_services > - > > Key: ATLAS-832 > URL: https://issues.apache.org/jira/browse/ATLAS-832 > Project: Atlas > Issue Type: Bug >Reporter: Shwetha G S >Assignee: Vimal Sharma >Priority: Major > > As part of ATLAS-823, embedded hbase and solr was moved to profile > 'embedded_services'. After the package is built using '-Pembedded_services', > the package configuration points to local hbase and solr, but atlas_start > doesn't start embedded hbase and solr -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (ATLAS-832) atlas_start should start embedded hbase and solr with -Pembedded_services
[ https://issues.apache.org/jira/browse/ATLAS-832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16382510#comment-16382510 ] Pierre Padovani commented on ATLAS-832: --- The issue is this line in config.py: ```ENV_KEYS = ["JAVA_HOME", ATLAS_OPTS, ATLAS_SERVER_OPTS, ATLAS_SERVER_HEAP, ATLAS_LOG, ATLAS_PID, ATLAS_CONF, "ATLASCPPATH", ATLAS_DATA, ATLAS_HOME, ATLAS_WEBAPP, HBASE_CONF_DIR, SOLR_PORT]``` Is missing MANAGE_LOCAL_HBASE MANAGE_LOCAL_SOLR > atlas_start should start embedded hbase and solr with -Pembedded_services > - > > Key: ATLAS-832 > URL: https://issues.apache.org/jira/browse/ATLAS-832 > Project: Atlas > Issue Type: Bug >Reporter: Shwetha G S >Assignee: Vimal Sharma >Priority: Major > > As part of ATLAS-823, embedded hbase and solr was moved to profile > 'embedded_services'. After the package is built using '-Pembedded_services', > the package configuration points to local hbase and solr, but atlas_start > doesn't start embedded hbase and solr -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (ATLAS-2259) Add Janus Graph Cassandra support
[
https://issues.apache.org/jira/browse/ATLAS-2259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16377284#comment-16377284
]
Pierre Padovani edited comment on ATLAS-2259 at 2/26/18 6:02 PM:
-
Hi [~davidrad],
I'm going to spend some time doing more than just this POM change. There is a
way to run Janus graph with embedded Cassandra, so I was also going to update
the profiles and add another one for embedded Cassandra and solr to complete
this ticket.
Then I am going to open two more tickets.
1 - Add a new Dockerfile for a Cassandra backed docker, which would use the
above embedded cassandra
2 - Fix for Elasticsearch 5.x support
Pierre
was (Author: ppadovani):
Hi [~davidrad],
I'm going to spend some time doing more than just this POM change. There is a
way to run Janus graph with embedded Cassandra, so I was also going to update
the profiles and add another one for embedded Cassandra and solr to complete
this ticket.
Then I am going to open two more tickets.
1 - Add Cassandra backed docker, which would use the above embedded cassandra
2 - Fix for Elasticsearch 5.x support
Pierre
> Add Janus Graph Cassandra support
> -
>
> Key: ATLAS-2259
> URL: https://issues.apache.org/jira/browse/ATLAS-2259
> Project: Atlas
> Issue Type: Improvement
> Components: atlas-core
>Affects Versions: 1.0.0
>Reporter: Pierre Padovani
>Assignee: Pierre Padovani
>Priority: Major
> Fix For: 1.0.0
>
>
> Atlas should have support for Cassandra as a backend for Janus available by
> default. If someone wants this type of configuration, they have to modify the
> pom.xml and rebuild Atlas. Here is the pom.xml modification required to
> enable this support:
> {code:java}
>
> org.janusgraph
> janusgraph-cassandra
> ${janus.version}
>
>
> org.codehaus.jettison
> jettison
>
>
> commons-lang
> commons-lang
>
>
>
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (ATLAS-2259) Add Janus Graph Cassandra support
[
https://issues.apache.org/jira/browse/ATLAS-2259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16377284#comment-16377284
]
Pierre Padovani commented on ATLAS-2259:
Hi [~davidrad],
I'm going to spend some time doing more than just this POM change. There is a
way to run Janus graph with embedded Cassandra, so I was also going to update
the profiles and add another one for embedded Cassandra and solr to complete
this ticket.
Then I am going to open two more tickets.
1 - Add Cassandra backed docker, which would use the above embedded cassandra
2 - Fix for Elasticsearch 5.x support
Pierre
> Add Janus Graph Cassandra support
> -
>
> Key: ATLAS-2259
> URL: https://issues.apache.org/jira/browse/ATLAS-2259
> Project: Atlas
> Issue Type: Improvement
> Components: atlas-core
>Affects Versions: 1.0.0
>Reporter: Pierre Padovani
>Assignee: Pierre Padovani
>Priority: Major
> Fix For: 1.0.0
>
>
> Atlas should have support for Cassandra as a backend for Janus available by
> default. If someone wants this type of configuration, they have to modify the
> pom.xml and rebuild Atlas. Here is the pom.xml modification required to
> enable this support:
> {code:java}
>
> org.janusgraph
> janusgraph-cassandra
> ${janus.version}
>
>
> org.codehaus.jettison
> jettison
>
>
> commons-lang
> commons-lang
>
>
>
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Issue Comment Deleted] (ATLAS-2259) Add Janus Graph Cassandra support
[
https://issues.apache.org/jira/browse/ATLAS-2259?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pierre Padovani updated ATLAS-2259:
---
Comment: was deleted
(was: Any updates on this? Is this going to be picked up and worked on? I don't
have commit privs to Atlas, so there isn't much I can do...)
> Add Janus Graph Cassandra support
> -
>
> Key: ATLAS-2259
> URL: https://issues.apache.org/jira/browse/ATLAS-2259
> Project: Atlas
> Issue Type: Improvement
> Components: atlas-core
>Affects Versions: 1.0.0
>Reporter: Pierre Padovani
>Assignee: Pierre Padovani
>Priority: Major
> Fix For: 1.0.0
>
>
> Atlas should have support for Cassandra as a backend for Janus available by
> default. If someone wants this type of configuration, they have to modify the
> pom.xml and rebuild Atlas. Here is the pom.xml modification required to
> enable this support:
> {code:java}
>
> org.janusgraph
> janusgraph-cassandra
> ${janus.version}
>
>
> org.codehaus.jettison
> jettison
>
>
> commons-lang
> commons-lang
>
>
>
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (ATLAS-2259) Add Janus Graph Cassandra support
[
https://issues.apache.org/jira/browse/ATLAS-2259?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pierre Padovani updated ATLAS-2259:
---
Summary: Add Janus Graph Cassandra support (was: Add Janus Graph Cassandra
support to the default build)
> Add Janus Graph Cassandra support
> -
>
> Key: ATLAS-2259
> URL: https://issues.apache.org/jira/browse/ATLAS-2259
> Project: Atlas
> Issue Type: Improvement
> Components: atlas-core
>Affects Versions: 1.0.0
>Reporter: Pierre Padovani
>Assignee: Pierre Padovani
>Priority: Major
> Fix For: 1.0.0
>
>
> Atlas should have support for Cassandra as a backend for Janus available by
> default. If someone wants this type of configuration, they have to modify the
> pom.xml and rebuild Atlas. Here is the pom.xml modification required to
> enable this support:
> {code:java}
>
> org.janusgraph
> janusgraph-cassandra
> ${janus.version}
>
>
> org.codehaus.jettison
> jettison
>
>
> commons-lang
> commons-lang
>
>
>
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Issue Comment Deleted] (ATLAS-2259) Add Janus Graph Cassandra support
[
https://issues.apache.org/jira/browse/ATLAS-2259?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pierre Padovani updated ATLAS-2259:
---
Comment: was deleted
(was: The above is not currently working after the update to Janus 0.2.0. At
the very least it is missing:
org.janusgraph
janusgraph-cql
${janus.version}
I'll update this ticket once I have it fully working.)
> Add Janus Graph Cassandra support
> -
>
> Key: ATLAS-2259
> URL: https://issues.apache.org/jira/browse/ATLAS-2259
> Project: Atlas
> Issue Type: Improvement
> Components: atlas-core
>Affects Versions: 1.0.0
>Reporter: Pierre Padovani
>Assignee: Pierre Padovani
>Priority: Major
> Fix For: 1.0.0
>
>
> Atlas should have support for Cassandra as a backend for Janus available by
> default. If someone wants this type of configuration, they have to modify the
> pom.xml and rebuild Atlas. Here is the pom.xml modification required to
> enable this support:
> {code:java}
>
> org.janusgraph
> janusgraph-cassandra
> ${janus.version}
>
>
> org.codehaus.jettison
> jettison
>
>
> commons-lang
> commons-lang
>
>
>
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (ATLAS-2259) Add Janus Graph Cassandra support to the default build
[
https://issues.apache.org/jira/browse/ATLAS-2259?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pierre Padovani reassigned ATLAS-2259:
--
Assignee: Pierre Padovani
> Add Janus Graph Cassandra support to the default build
> --
>
> Key: ATLAS-2259
> URL: https://issues.apache.org/jira/browse/ATLAS-2259
> Project: Atlas
> Issue Type: Improvement
> Components: atlas-core
>Affects Versions: 1.0.0
>Reporter: Pierre Padovani
>Assignee: Pierre Padovani
>Priority: Major
> Fix For: 1.0.0
>
>
> Atlas should have support for Cassandra as a backend for Janus available by
> default. If someone wants this type of configuration, they have to modify the
> pom.xml and rebuild Atlas. Here is the pom.xml modification required to
> enable this support:
> {code:java}
>
> org.janusgraph
> janusgraph-cassandra
> ${janus.version}
>
>
> org.codehaus.jettison
> jettison
>
>
> commons-lang
> commons-lang
>
>
>
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (ATLAS-2270) Supported combinations of persistent store and index backend
[
https://issues.apache.org/jira/browse/ATLAS-2270?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16373048#comment-16373048
]
Pierre Padovani commented on ATLAS-2270:
[~davidrad] I'd be happy to be a contributor and get this in. How does one go
about becoming one? Are there a set of guidelines somewhere specific to this
project?
We've been running the Cassandra + ES 5.x flavor of Atlas as both a self
contained docker container for dev purposes, as well as a full blown HA cluster
for at least the last two months. So I think it looks like a pretty stable
setup.
> Supported combinations of persistent store and index backend
>
>
> Key: ATLAS-2270
> URL: https://issues.apache.org/jira/browse/ATLAS-2270
> Project: Atlas
> Issue Type: Bug
>Reporter: Graham Wallis
>Priority: Major
>
> We need to discuss and decide which combinations of persistent store and
> indexing backend Atlas 1.0.0 (master) should support. This includes
> building/running Atlas as a standalone package and running UTs/ITs as part of
> the Atlas build.
> This JIRA focusses on titan0 and janusgraph 0.2.0, as they are the graph
> databases that will be supported in master/1.0.0. This JIRA deliberately
> ignores titan1 and janusgraph 0.1.1 as the former should be
> deprecated/removed and the other is a transient state as we get to janusgraph
> 0.2.0.
> With titan0 as the graph provider, Atlas has supported the following
> combinations of persistent store and indexer. It is suggested that this set
> is kept unchanged:
> {{
> titan0 solr es
>
> berkeley 0 1
> hbase 1 0
> cassandra0 0
> }}
> With janusgraph (0.2.0) as the graph provider, Atlas *could* support
> additional combinations. Cassandra is included in this discussion pending
> response to ATLAS-2259.
> {{
> janus 0.2.0 solr es
>
> berkeley ? 1
> hbase 1 ?
> cassandra? ?
> }}
> It is suggested that the combinations marked with '1' should be continued and
> the remaining 4 combinations, marked with '?', should be considered. There
> seems to be evidence of people using all 4 of these combinations, although
> not necessarily with Atlas.
> Depending on the decision made above, we need to ensure that it is possible
> to build Atlas as a standalone package with any of the combinations - i.e.
> that they are mutually exclusive and do not interfere with one another. They
> currently interfere which makes it impossible to build Atlas with
> -Pdist,berkeley-elasticsearch because the 'dist' profile will exclude jars
> that are needed by the berkeley-elasticsearch profile - which leads to class
> not found exceptions when the Atlas server is started. The solution to this
> could be very simple, or slightly more sophisticated, depending on how many
> of the combinations we choose to support.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (ATLAS-2259) Add Janus Graph Cassandra support to the default build
[
https://issues.apache.org/jira/browse/ATLAS-2259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16322731#comment-16322731
]
Pierre Padovani commented on ATLAS-2259:
Any updates on this? Is this going to be picked up and worked on? I don't have
commit privs to Atlas, so there isn't much I can do...
> Add Janus Graph Cassandra support to the default build
> --
>
> Key: ATLAS-2259
> URL: https://issues.apache.org/jira/browse/ATLAS-2259
> Project: Atlas
> Issue Type: Improvement
> Components: atlas-core
>Affects Versions: 1.0.0
>Reporter: Pierre Padovani
> Fix For: 1.0.0
>
>
> Atlas should have support for Cassandra as a backend for Janus available by
> default. If someone wants this type of configuration, they have to modify the
> pom.xml and rebuild Atlas. Here is the pom.xml modification required to
> enable this support:
> {code:java}
>
> org.janusgraph
> janusgraph-cassandra
> ${janus.version}
>
>
> org.codehaus.jettison
> jettison
>
>
> commons-lang
> commons-lang
>
>
>
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (ATLAS-2249) Upgrade Atlas Client to use Jersey 2
[ https://issues.apache.org/jira/browse/ATLAS-2249?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Padovani updated ATLAS-2249: --- Attachment: client-jersey2-V4.patch Updates to the patch based on current master changes. > Upgrade Atlas Client to use Jersey 2 > > > Key: ATLAS-2249 > URL: https://issues.apache.org/jira/browse/ATLAS-2249 > Project: Atlas > Issue Type: Improvement > Components: atlas-intg >Affects Versions: 1.0.0 >Reporter: Pierre Padovani > Labels: patch-available > Fix For: 1.0.0 > > Attachments: client-jersey2-V2.patch, client-jersey2-V3.patch, > client-jersey2-V4.patch, client-jersey2.patch > > > Using the existing Atlas client within a Jersey 2 service causes conflicts. I > recommend upgrading just the client code to Jersey 2. The enclosed patch > fully upgrades the V1 and V2 client to Jersey 2, and we have tested this > against our own installation of 1.0.0-SNAPSHOT. > NOTE: We have not tested SSL/Kerberos authentication/security -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Comment Edited] (ATLAS-2259) Add Janus Graph Cassandra support to the default build
[
https://issues.apache.org/jira/browse/ATLAS-2259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16269278#comment-16269278
]
Pierre Padovani edited comment on ATLAS-2259 at 11/28/17 7:10 PM:
--
Here is the current working pom.xml dependency additions required.
{code:java}
org.janusgraph
janusgraph-cassandra
${janus.version}
org.codehaus.jettison
jettison
commons-lang
commons-lang
org.janusgraph
janusgraph-cql
${janus.version}
{code}
This does not support Elasticsearch as there is a separate issue with that, as
being discussed here:
[ATLAS-2270|https://issues.apache.org/jira/browse/ATLAS-2270]
was (Author: ppadovani):
Here is the current working pom.xml dependency additions required.
{code:java}
org.janusgraph
janusgraph-cassandra
${janus.version}
org.codehaus.jettison
jettison
commons-lang
commons-lang
org.janusgraph
janusgraph-cql
${janus.version}
{code}
> Add Janus Graph Cassandra support to the default build
> --
>
> Key: ATLAS-2259
> URL: https://issues.apache.org/jira/browse/ATLAS-2259
> Project: Atlas
> Issue Type: Improvement
> Components: atlas-core
>Affects Versions: 1.0.0
>Reporter: Pierre Padovani
> Fix For: 1.0.0
>
>
> Atlas should have support for Cassandra as a backend for Janus available by
> default. If someone wants this type of configuration, they have to modify the
> pom.xml and rebuild Atlas. Here is the pom.xml modification required to
> enable this support:
> {code:java}
>
> org.janusgraph
> janusgraph-cassandra
> ${janus.version}
>
>
> org.codehaus.jettison
> jettison
>
>
> commons-lang
> commons-lang
>
>
>
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (ATLAS-2259) Add Janus Graph Cassandra support to the default build
[
https://issues.apache.org/jira/browse/ATLAS-2259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16269278#comment-16269278
]
Pierre Padovani commented on ATLAS-2259:
Here is the current working pom.xml dependency additions required.
{code:java}
org.janusgraph
janusgraph-cassandra
${janus.version}
org.codehaus.jettison
jettison
commons-lang
commons-lang
org.janusgraph
janusgraph-cql
${janus.version}
{code}
> Add Janus Graph Cassandra support to the default build
> --
>
> Key: ATLAS-2259
> URL: https://issues.apache.org/jira/browse/ATLAS-2259
> Project: Atlas
> Issue Type: Improvement
> Components: atlas-core
>Affects Versions: 1.0.0
>Reporter: Pierre Padovani
> Fix For: 1.0.0
>
>
> Atlas should have support for Cassandra as a backend for Janus available by
> default. If someone wants this type of configuration, they have to modify the
> pom.xml and rebuild Atlas. Here is the pom.xml modification required to
> enable this support:
> {code:java}
>
> org.janusgraph
> janusgraph-cassandra
> ${janus.version}
>
>
> org.codehaus.jettison
> jettison
>
>
> commons-lang
> commons-lang
>
>
>
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Comment Edited] (ATLAS-2270) Supported combinations of persistent store and index backend
[
https://issues.apache.org/jira/browse/ATLAS-2270?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16269246#comment-16269246
]
Pierre Padovani edited comment on ATLAS-2270 at 11/28/17 6:48 PM:
--
Hi [~grahamwallis]
IMHO - In terms of the Janus-ES topology support:
* Fix the janus pom.xml and include the es rest client dependency and ship that
with the atlas distribution. Otherwise implementors will have to copy that jar
into the lib of Atlas before starting.
* Require anyone wanting to use a janus-es topology to stand up their own es
cluster and supply the hostnames in the atlas properties. This isn't all that
hard: download ES then start it (for a single node config).
* Include a docker file (I'll gladly contribute mine as an alternative to the
current one.) that provides this topology for development/example purposes. (We
exclusively use this docker for dev/debug at this point, it is just a single
Atlas node.)
We are in the middle of our development cycle in terms of metadata support, and
our infrastructure lacks any Hadoop support, and will likely not contain it in
the future. As we evaluated possible metadata products, Atlas stood out,
however using Titan was not something we wanted. Seeing the move to Janus was
huge for us, we intend to deploy Atlas Janus 0.2.0 on Cassandra + ES.
*EDIT* To be clear I implemented all of the above changes, and have built and
deployed an Atlas docker container running Janus 0.2.0 using Cassandra and
Elasticsearch. As far as we can tell it works very well.
Thanks!
Pierre
was (Author: ppadovani):
Hi [~grahamwallis]
IMHO - In terms of the Janus-ES topology support:
* Fix the janus pom.xml and include the es rest client dependency and ship that
with the atlas distribution. Otherwise implementors will have to copy that jar
into the lib of Atlas before starting.
* Require anyone wanting to use a janus-es topology to stand up their own es
cluster and supply the hostnames in the atlas properties. This isn't all that
hard: download ES then start it (for a single node config).
* Include a docker file (I'll gladly contribute mine as an alternative to the
current one.) that provides this topology for development/example purposes. (We
exclusively use this docker for dev/debug at this point, it is just a single
Atlas node.)
We are in the middle of our development cycle in terms of metadata support, and
our infrastructure lacks any Hadoop support, and will likely not contain it in
the future. As we evaluated possible metadata products, Atlas stood out,
however using Titan was not something we wanted. Seeing the move to Janus was
huge for us, we intend to deploy Atlas Janus 0.2.0 on Cassandra + ES.
Thanks!
Pierre
> Supported combinations of persistent store and index backend
>
>
> Key: ATLAS-2270
> URL: https://issues.apache.org/jira/browse/ATLAS-2270
> Project: Atlas
> Issue Type: Bug
>Reporter: Graham Wallis
>
> We need to discuss and decide which combinations of persistent store and
> indexing backend Atlas 1.0.0 (master) should support. This includes
> building/running Atlas as a standalone package and running UTs/ITs as part of
> the Atlas build.
> This JIRA focusses on titan0 and janusgraph 0.2.0, as they are the graph
> databases that will be supported in master/1.0.0. This JIRA deliberately
> ignores titan1 and janusgraph 0.1.1 as the former should be
> deprecated/removed and the other is a transient state as we get to janusgraph
> 0.2.0.
> With titan0 as the graph provider, Atlas has supported the following
> combinations of persistent store and indexer. It is suggested that this set
> is kept unchanged:
> {{
> titan0 solr es
>
> berkeley 0 1
> hbase 1 0
> cassandra0 0
> }}
> With janusgraph (0.2.0) as the graph provider, Atlas *could* support
> additional combinations. Cassandra is included in this discussion pending
> response to ATLAS-2259.
> {{
> janus 0.2.0 solr es
>
> berkeley ? 1
> hbase 1 ?
> cassandra? ?
> }}
> It is suggested that the combinations marked with '1' should be continued and
> the remaining 4 combinations, marked with '?', should be considered. There
> seems to be evidence of people using all 4 of these combinations, although
> not necessarily with Atlas.
> Depending on the decision made above, we need to ensure that it is possible
> to build Atlas as a standalone package with any of the combinations - i.e.
> that they are mutually exclusive and do not interfere with one another. They
> currently interfere which makes it i
[jira] [Commented] (ATLAS-2270) Supported combinations of persistent store and index backend
[
https://issues.apache.org/jira/browse/ATLAS-2270?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16269246#comment-16269246
]
Pierre Padovani commented on ATLAS-2270:
Hi [~grahamwallis]
IMHO - In terms of the Janus-ES topology support:
* Fix the janus pom.xml and include the es rest client dependency and ship that
with the atlas distribution. Otherwise implementors will have to copy that jar
into the lib of Atlas before starting.
* Require anyone wanting to use a janus-es topology to stand up their own es
cluster and supply the hostnames in the atlas properties. This isn't all that
hard: download ES then start it (for a single node config).
* Include a docker file (I'll gladly contribute mine as an alternative to the
current one.) that provides this topology for development/example purposes. (We
exclusively use this docker for dev/debug at this point, it is just a single
Atlas node.)
We are in the middle of our development cycle in terms of metadata support, and
our infrastructure lacks any Hadoop support, and will likely not contain it in
the future. As we evaluated possible metadata products, Atlas stood out,
however using Titan was not something we wanted. Seeing the move to Janus was
huge for us, we intend to deploy Atlas Janus 0.2.0 on Cassandra + ES.
Thanks!
Pierre
> Supported combinations of persistent store and index backend
>
>
> Key: ATLAS-2270
> URL: https://issues.apache.org/jira/browse/ATLAS-2270
> Project: Atlas
> Issue Type: Bug
>Reporter: Graham Wallis
>
> We need to discuss and decide which combinations of persistent store and
> indexing backend Atlas 1.0.0 (master) should support. This includes
> building/running Atlas as a standalone package and running UTs/ITs as part of
> the Atlas build.
> This JIRA focusses on titan0 and janusgraph 0.2.0, as they are the graph
> databases that will be supported in master/1.0.0. This JIRA deliberately
> ignores titan1 and janusgraph 0.1.1 as the former should be
> deprecated/removed and the other is a transient state as we get to janusgraph
> 0.2.0.
> With titan0 as the graph provider, Atlas has supported the following
> combinations of persistent store and indexer. It is suggested that this set
> is kept unchanged:
> {{
> titan0 solr es
>
> berkeley 0 1
> hbase 1 0
> cassandra0 0
> }}
> With janusgraph (0.2.0) as the graph provider, Atlas *could* support
> additional combinations. Cassandra is included in this discussion pending
> response to ATLAS-2259.
> {{
> janus 0.2.0 solr es
>
> berkeley ? 1
> hbase 1 ?
> cassandra? ?
> }}
> It is suggested that the combinations marked with '1' should be continued and
> the remaining 4 combinations, marked with '?', should be considered. There
> seems to be evidence of people using all 4 of these combinations, although
> not necessarily with Atlas.
> Depending on the decision made above, we need to ensure that it is possible
> to build Atlas as a standalone package with any of the combinations - i.e.
> that they are mutually exclusive and do not interfere with one another. They
> currently interfere which makes it impossible to build Atlas with
> -Pdist,berkeley-elasticsearch because the 'dist' profile will exclude jars
> that are needed by the berkeley-elasticsearch profile - which leads to class
> not found exceptions when the Atlas server is started. The solution to this
> could be very simple, or slightly more sophisticated, depending on how many
> of the combinations we choose to support.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Comment Edited] (ATLAS-2270) Supported combinations of persistent store and index backend
[
https://issues.apache.org/jira/browse/ATLAS-2270?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16269001#comment-16269001
]
Pierre Padovani edited comment on ATLAS-2270 at 11/28/17 5:24 PM:
--
[~davidrad] Per the JanusGraph documentation, Cassandra and Elasticsearch are
fully supported.
After some debugging, it seems that the main issue with supporting
Elasticsearch has to do with two lines in
org.apache.atlas.repository.Constants.java:
{code:java}
public static final String VERTEX_TYPE_PROPERTY_KEY =
INTERNAL_PROPERTY_KEY_PREFIX + "type";
public static final String TYPENAME_PROPERTY_KEY =
INTERNAL_PROPERTY_KEY_PREFIX + "type.name";
{code}
Those constants are used in this code:
{code:java}
private void createTypeStoreIndexes(AtlasGraphManagement management) {
//Create unique index on typeName
createIndexes(management, Constants.TYPENAME_PROPERTY_KEY,
String.class, true, AtlasCardinality.SINGLE,
true, true);
//create index on vertex type
createIndexes(management, Constants.VERTEX_TYPE_PROPERTY_KEY,
String.class, false, AtlasCardinality.SINGLE,
true, true);
}
{code}
In Elasticsearch 2.x and above, creating two fields named as above is
disallowed. The reason for this has to do with the treatment of a '.' in the
name as a path within a sub object. So the above code would attempt to create
the following:
1) { "type" : { "name" : "string"}}
2) { "type": "string"}
This fails because the update is attempting to change an object to a field
index. The simple fix would be to change the '.' to '_', however this may cause
backward incompatibilities for existing systems.
*Update:* - Changing the '.' to '_' solves the issues, and I have a fully
operational Atlas with Cassandra and Elasticsearch.
was (Author: ppadovani):
[~davidrad] Per the JanusGraph documentation, Cassandra and Elasticsearch are
fully supported.
After some debugging, it seems that the main issue with supporting
Elasticsearch has to do with two lines in
org.apache.atlas.repository.Constants.java:
{code:java}
public static final String VERTEX_TYPE_PROPERTY_KEY =
INTERNAL_PROPERTY_KEY_PREFIX + "type";
public static final String TYPENAME_PROPERTY_KEY =
INTERNAL_PROPERTY_KEY_PREFIX + "type.name";
{code}
Those constants are used in this code:
{code:java}
private void createTypeStoreIndexes(AtlasGraphManagement management) {
//Create unique index on typeName
createIndexes(management, Constants.TYPENAME_PROPERTY_KEY,
String.class, true, AtlasCardinality.SINGLE,
true, true);
//create index on vertex type
createIndexes(management, Constants.VERTEX_TYPE_PROPERTY_KEY,
String.class, false, AtlasCardinality.SINGLE,
true, true);
}
{code}
In Elasticsearch 2.x and above, creating two fields named as above is
disallowed. The reason for this has to do with the treatment of a '.' in the
name as a path within a sub object. So the above code would attempt to create
the following:
1) { "type" : { "name" : "string"}}
2) { "type": "string"}
This fails because the update is attempting to change an object to a field
index. The simple fix would be to change the '.' to '_', however this may cause
backward incompatibilities for existing systems.
> Supported combinations of persistent store and index backend
>
>
> Key: ATLAS-2270
> URL: https://issues.apache.org/jira/browse/ATLAS-2270
> Project: Atlas
> Issue Type: Bug
>Reporter: Graham Wallis
>
> We need to discuss and decide which combinations of persistent store and
> indexing backend Atlas 1.0.0 (master) should support. This includes
> building/running Atlas as a standalone package and running UTs/ITs as part of
> the Atlas build.
> This JIRA focusses on titan0 and janusgraph 0.2.0, as they are the graph
> databases that will be supported in master/1.0.0. This JIRA deliberately
> ignores titan1 and janusgraph 0.1.1 as the former should be
> deprecated/removed and the other is a transient state as we get to janusgraph
> 0.2.0.
> With titan0 as the graph provider, Atlas has supported the following
> combinations of persistent store and indexer. It is suggested that this set
> is kept unchanged:
> {{
> titan0 solr es
>
> berkeley 0 1
> hbase 1 0
> cassandra0 0
> }}
> With janusgraph (0.2.0) as the graph provider, Atlas *could* support
> additional combinations. Cassandra is included in this discussion pending
> response to ATLAS-2259.
> {{
> janus 0.2.0 solr es
>
> berkeley ? 1
[jira] [Commented] (ATLAS-2270) Supported combinations of persistent store and index backend
[
https://issues.apache.org/jira/browse/ATLAS-2270?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16269001#comment-16269001
]
Pierre Padovani commented on ATLAS-2270:
[~davidrad] Per the JanusGraph documentation, Cassandra and Elasticsearch are
fully supported.
After some debugging, it seems that the main issue with supporting
Elasticsearch has to do with two lines in
org.apache.atlas.repository.Constants.java:
{code:java}
public static final String VERTEX_TYPE_PROPERTY_KEY =
INTERNAL_PROPERTY_KEY_PREFIX + "type";
public static final String TYPENAME_PROPERTY_KEY =
INTERNAL_PROPERTY_KEY_PREFIX + "type.name";
{code}
Those constants are used in this code:
{code:java}
private void createTypeStoreIndexes(AtlasGraphManagement management) {
//Create unique index on typeName
createIndexes(management, Constants.TYPENAME_PROPERTY_KEY,
String.class, true, AtlasCardinality.SINGLE,
true, true);
//create index on vertex type
createIndexes(management, Constants.VERTEX_TYPE_PROPERTY_KEY,
String.class, false, AtlasCardinality.SINGLE,
true, true);
}
{code}
In Elasticsearch 2.x and above, creating two fields named as above is
disallowed. The reason for this has to do with the treatment of a '.' in the
name as a path within a sub object. So the above code would attempt to create
the following:
1) { "type" : { "name" : "string"}}
2) { "type": "string"}
This fails because the update is attempting to change an object to a field
index. The simple fix would be to change the '.' to '_', however this may cause
backward incompatibilities for existing systems.
> Supported combinations of persistent store and index backend
>
>
> Key: ATLAS-2270
> URL: https://issues.apache.org/jira/browse/ATLAS-2270
> Project: Atlas
> Issue Type: Bug
>Reporter: Graham Wallis
>
> We need to discuss and decide which combinations of persistent store and
> indexing backend Atlas 1.0.0 (master) should support. This includes
> building/running Atlas as a standalone package and running UTs/ITs as part of
> the Atlas build.
> This JIRA focusses on titan0 and janusgraph 0.2.0, as they are the graph
> databases that will be supported in master/1.0.0. This JIRA deliberately
> ignores titan1 and janusgraph 0.1.1 as the former should be
> deprecated/removed and the other is a transient state as we get to janusgraph
> 0.2.0.
> With titan0 as the graph provider, Atlas has supported the following
> combinations of persistent store and indexer. It is suggested that this set
> is kept unchanged:
> {{
> titan0 solr es
>
> berkeley 0 1
> hbase 1 0
> cassandra0 0
> }}
> With janusgraph (0.2.0) as the graph provider, Atlas *could* support
> additional combinations. Cassandra is included in this discussion pending
> response to ATLAS-2259.
> {{
> janus 0.2.0 solr es
>
> berkeley ? 1
> hbase 1 ?
> cassandra? ?
> }}
> It is suggested that the combinations marked with '1' should be continued and
> the remaining 4 combinations, marked with '?', should be considered. There
> seems to be evidence of people using all 4 of these combinations, although
> not necessarily with Atlas.
> Depending on the decision made above, we need to ensure that it is possible
> to build Atlas as a standalone package with any of the combinations - i.e.
> that they are mutually exclusive and do not interfere with one another. They
> currently interfere which makes it impossible to build Atlas with
> -Pdist,berkeley-elasticsearch because the 'dist' profile will exclude jars
> that are needed by the berkeley-elasticsearch profile - which leads to class
> not found exceptions when the Atlas server is started. The solution to this
> could be very simple, or slightly more sophisticated, depending on how many
> of the combinations we choose to support.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Issue Comment Deleted] (ATLAS-2270) Supported combinations of persistent store and index backend
[
https://issues.apache.org/jira/browse/ATLAS-2270?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pierre Padovani updated ATLAS-2270:
---
Comment: was deleted
(was: [~davidrad] Looking at the Janus documentation it indicates full support
for Cassandra and Elasticsearch. It just requires particular setup steps.
In terms of Atlas, what I can tell the only issue seems to be, is a bug in the
Atlas support of Elasticsearch. It looks like the mapping for Elasticsearch is
created in one form, then an attempt to update it in a way that is incompatible
with the previous creation causes the bean creation lifecycle to fail.
I'll use the ticket I referenced above to put a full patch forward that:
* Adds Cassandra support
* Fixes Elasticsearch support
* Adds a new Dockerfile that creates a standalone Cassandra + Elasticsearch
configuration.
)
> Supported combinations of persistent store and index backend
>
>
> Key: ATLAS-2270
> URL: https://issues.apache.org/jira/browse/ATLAS-2270
> Project: Atlas
> Issue Type: Bug
>Reporter: Graham Wallis
>
> We need to discuss and decide which combinations of persistent store and
> indexing backend Atlas 1.0.0 (master) should support. This includes
> building/running Atlas as a standalone package and running UTs/ITs as part of
> the Atlas build.
> This JIRA focusses on titan0 and janusgraph 0.2.0, as they are the graph
> databases that will be supported in master/1.0.0. This JIRA deliberately
> ignores titan1 and janusgraph 0.1.1 as the former should be
> deprecated/removed and the other is a transient state as we get to janusgraph
> 0.2.0.
> With titan0 as the graph provider, Atlas has supported the following
> combinations of persistent store and indexer. It is suggested that this set
> is kept unchanged:
> {{
> titan0 solr es
>
> berkeley 0 1
> hbase 1 0
> cassandra0 0
> }}
> With janusgraph (0.2.0) as the graph provider, Atlas *could* support
> additional combinations. Cassandra is included in this discussion pending
> response to ATLAS-2259.
> {{
> janus 0.2.0 solr es
>
> berkeley ? 1
> hbase 1 ?
> cassandra? ?
> }}
> It is suggested that the combinations marked with '1' should be continued and
> the remaining 4 combinations, marked with '?', should be considered. There
> seems to be evidence of people using all 4 of these combinations, although
> not necessarily with Atlas.
> Depending on the decision made above, we need to ensure that it is possible
> to build Atlas as a standalone package with any of the combinations - i.e.
> that they are mutually exclusive and do not interfere with one another. They
> currently interfere which makes it impossible to build Atlas with
> -Pdist,berkeley-elasticsearch because the 'dist' profile will exclude jars
> that are needed by the berkeley-elasticsearch profile - which leads to class
> not found exceptions when the Atlas server is started. The solution to this
> could be very simple, or slightly more sophisticated, depending on how many
> of the combinations we choose to support.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (ATLAS-2270) Supported combinations of persistent store and index backend
[
https://issues.apache.org/jira/browse/ATLAS-2270?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16268883#comment-16268883
]
Pierre Padovani commented on ATLAS-2270:
[~davidrad] Looking at the Janus documentation it indicates full support for
Cassandra and Elasticsearch. It just requires particular setup steps.
In terms of Atlas, what I can tell the only issue seems to be, is a bug in the
Atlas support of Elasticsearch. It looks like the mapping for Elasticsearch is
created in one form, then an attempt to update it in a way that is incompatible
with the previous creation causes the bean creation lifecycle to fail.
I'll use the ticket I referenced above to put a full patch forward that:
* Adds Cassandra support
* Fixes Elasticsearch support
* Adds a new Dockerfile that creates a standalone Cassandra + Elasticsearch
configuration.
> Supported combinations of persistent store and index backend
>
>
> Key: ATLAS-2270
> URL: https://issues.apache.org/jira/browse/ATLAS-2270
> Project: Atlas
> Issue Type: Bug
>Reporter: Graham Wallis
>
> We need to discuss and decide which combinations of persistent store and
> indexing backend Atlas 1.0.0 (master) should support. This includes
> building/running Atlas as a standalone package and running UTs/ITs as part of
> the Atlas build.
> This JIRA focusses on titan0 and janusgraph 0.2.0, as they are the graph
> databases that will be supported in master/1.0.0. This JIRA deliberately
> ignores titan1 and janusgraph 0.1.1 as the former should be
> deprecated/removed and the other is a transient state as we get to janusgraph
> 0.2.0.
> With titan0 as the graph provider, Atlas has supported the following
> combinations of persistent store and indexer. It is suggested that this set
> is kept unchanged:
> {{
> titan0 solr es
>
> berkeley 0 1
> hbase 1 0
> cassandra0 0
> }}
> With janusgraph (0.2.0) as the graph provider, Atlas *could* support
> additional combinations. Cassandra is included in this discussion pending
> response to ATLAS-2259.
> {{
> janus 0.2.0 solr es
>
> berkeley ? 1
> hbase 1 ?
> cassandra? ?
> }}
> It is suggested that the combinations marked with '1' should be continued and
> the remaining 4 combinations, marked with '?', should be considered. There
> seems to be evidence of people using all 4 of these combinations, although
> not necessarily with Atlas.
> Depending on the decision made above, we need to ensure that it is possible
> to build Atlas as a standalone package with any of the combinations - i.e.
> that they are mutually exclusive and do not interfere with one another. They
> currently interfere which makes it impossible to build Atlas with
> -Pdist,berkeley-elasticsearch because the 'dist' profile will exclude jars
> that are needed by the berkeley-elasticsearch profile - which leads to class
> not found exceptions when the Atlas server is started. The solution to this
> could be very simple, or slightly more sophisticated, depending on how many
> of the combinations we choose to support.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Comment Edited] (ATLAS-2270) Supported combinations of persistent store and index backend
[
https://issues.apache.org/jira/browse/ATLAS-2270?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16267205#comment-16267205
]
Pierre Padovani edited comment on ATLAS-2270 at 11/27/17 9:15 PM:
--
Not everyone wants to be tied to using Hadoop/HDFS as part of their
infrastructure. :D
The issue I linked to contains the changes required to the pom.xml for Janus to
enable Cassandra support. The only other thing needed is documentation around
how to set it up. I've built up standalone docker images that contain:
- zookeeper
- cassandra
- Elasticsearch
- atlas (Janus 0.1.0 - working to build 0.2.0 now)
We currently deploy these images for development while we prepare to roll Atlas
in this same configuration.
I'm happy to contribute what else may be needed in this regard.
NOTE: That issue's pom.xml does not work with 0.2.0 of Janus, I'll update it
when I have it working locally. I have configured everything in what I believe
should work, but am getting an error:
{code:java}
Caused by: org.elasticsearch.client.ResponseException: PUT
http://127.0.0.1:9200/janusgraph_vertex_index/_mapping/vertex_index: HTTP/1.1
400 Bad Request
{"error":{"root_cause":[{"type":"illegal_argument_exception","reason":"Can't
merge a non object mapping [__type] with an object mapping
[__type]"}],"type":"illegal_argument_exception","reason":"Can't merge a non
object mapping [__type] with an object mapping [__type]"},"status":400}
at org.elasticsearch.client.RestClient$1.completed(RestClient.java:354)
~[rest-5.5.3.jar:5.5.3]
at org.elasticsearch.client.RestClient$1.completed(RestClient.java:343)
~[rest-5.5.3.jar:5.5.3]
at
org.apache.http.concurrent.BasicFuture.completed(BasicFuture.java:119)
~[httpcore-4.4.1.jar:4.4.1]
at
org.apache.http.impl.nio.client.DefaultClientExchangeHandlerImpl.responseCompleted(DefaultClientExchangeHandlerImpl.java:177)
~[httpasyncclient-4.1.2.jar:4.1.2]
at
org.apache.http.nio.protocol.HttpAsyncRequestExecutor.processResponse(HttpAsyncRequestExecutor.java:436)
~[httpcore-nio-4.4.5.jar:4.4.5]
at
org.apache.http.nio.protocol.HttpAsyncRequestExecutor.inputReady(HttpAsyncRequestExecutor.java:326)
~[httpcore-nio-4.4.5.jar:4.4.5]
at
org.apache.http.impl.nio.client.InternalRequestExecutor.inputReady(InternalRequestExecutor.java:83)
~[httpasyncclient-4.1.2.jar:4.1.2]
at
org.apache.http.impl.nio.DefaultNHttpClientConnection.consumeInput(DefaultNHttpClientConnection.java:265)
~[httpcore-nio-4.4.5.jar:4.4.5]
at
org.apache.http.impl.nio.client.InternalIODispatch.onInputReady(InternalIODispatch.java:81)
~[httpasyncclient-4.1.2.jar:4.1.2]
at
org.apache.http.impl.nio.client.InternalIODispatch.onInputReady(InternalIODispatch.java:39)
~[httpasyncclient-4.1.2.jar:4.1.2]
at
org.apache.http.impl.nio.reactor.AbstractIODispatch.inputReady(AbstractIODispatch.java:114)
~[httpcore-nio-4.4.5.jar:4.4.5]
at
org.apache.http.impl.nio.reactor.BaseIOReactor.readable(BaseIOReactor.java:162)
~[httpcore-nio-4.4.5.jar:4.4.5]
at
org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvent(AbstractIOReactor.java:337)
~[httpcore-nio-4.4.5.jar:4.4.5]
at
org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvents(AbstractIOReactor.java:315)
~[httpcore-nio-4.4.5.jar:4.4.5]
{code}
I'll dig in and see if I can figure out what is going on. This combination does
work with Janus 0.1.0.
was (Author: ppadovani):
Not everyone wants to be tied to using Hadoop/HDFS as part of their
infrastructure. :D
The issue I linked to contains the changes required to the pom.xml for Janus to
enable Cassandra support. The only other thing needed is documentation around
how to set it up. I've built up standalone docker images that contain:
- zookeeper
- cassandra
- Elasticsearch
- atlas (Janus 0.1.0 - working to build 0.2.0 now)
We currently deploy these images for development while we prepare to roll Atlas
in this same configuration.
I'm happy to contribute what else may be needed in this regard.
NOTE: That issue's pom.xml does not work with 0.2.0 of Janus, I'll update it
when I have it working locally.
> Supported combinations of persistent store and index backend
>
>
> Key: ATLAS-2270
> URL: https://issues.apache.org/jira/browse/ATLAS-2270
> Project: Atlas
> Issue Type: Bug
>Reporter: Graham Wallis
>
> We need to discuss and decide which combinations of persistent store and
> indexing backend Atlas 1.0.0 (master) should support. This includes
> building/running Atlas as a standalone package and running UTs/ITs as part of
> the Atlas build.
> This JIRA focusses on titan0 and janusgraph 0.2.0, as they are the graph
> databases that will be supported in master/1.0.0.
[jira] [Commented] (ATLAS-2259) Add Janus Graph Cassandra support to the default build
[
https://issues.apache.org/jira/browse/ATLAS-2259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16267284#comment-16267284
]
Pierre Padovani commented on ATLAS-2259:
The above is not currently working after the update to Janus 0.2.0. At the very
least it is missing:
org.janusgraph
janusgraph-cql
${janus.version}
I'll update this ticket once I have it fully working.
> Add Janus Graph Cassandra support to the default build
> --
>
> Key: ATLAS-2259
> URL: https://issues.apache.org/jira/browse/ATLAS-2259
> Project: Atlas
> Issue Type: Improvement
> Components: atlas-core
>Affects Versions: 1.0.0
>Reporter: Pierre Padovani
> Fix For: 1.0.0
>
>
> Atlas should have support for Cassandra as a backend for Janus available by
> default. If someone wants this type of configuration, they have to modify the
> pom.xml and rebuild Atlas. Here is the pom.xml modification required to
> enable this support:
> {code:java}
>
> org.janusgraph
> janusgraph-cassandra
> ${janus.version}
>
>
> org.codehaus.jettison
> jettison
>
>
> commons-lang
> commons-lang
>
>
>
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Comment Edited] (ATLAS-2270) Supported combinations of persistent store and index backend
[
https://issues.apache.org/jira/browse/ATLAS-2270?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16267205#comment-16267205
]
Pierre Padovani edited comment on ATLAS-2270 at 11/27/17 7:10 PM:
--
Not everyone wants to be tied to using Hadoop/HDFS as part of their
infrastructure. :D
The issue I linked to contains the changes required to the pom.xml for Janus to
enable Cassandra support. The only other thing needed is documentation around
how to set it up. I've built up standalone docker images that contain:
- zookeeper
- cassandra
- Elasticsearch
- atlas (Janus 0.1.0 - working to build 0.2.0 now)
We currently deploy these images for development while we prepare to roll Atlas
in this same configuration.
I'm happy to contribute what else may be needed in this regard.
NOTE: That issue's pom.xml does not work with 0.2.0 of Janus, I'll update it
when I have it working locally.
was (Author: ppadovani):
Not everyone wants to be tied to using Hadoop/HDFS as part of their
infrastructure. :D
The issue I linked to contains the changes required to the pom.xml for Janus to
enable Cassandra support. The only other thing needed is documentation around
how to set it up. I've built up standalone docker images that contain:
- zookeeper
- cassandra
- Elasticsearch
- atlas (Janus 0.1.0 - working to build 0.2.0 now)
We currently deploy these images for development while we prepare to roll Atlas
in this same configuration.
I'm happy to contribute what else may be needed in this regard.
> Supported combinations of persistent store and index backend
>
>
> Key: ATLAS-2270
> URL: https://issues.apache.org/jira/browse/ATLAS-2270
> Project: Atlas
> Issue Type: Bug
>Reporter: Graham Wallis
>
> We need to discuss and decide which combinations of persistent store and
> indexing backend Atlas 1.0.0 (master) should support. This includes
> building/running Atlas as a standalone package and running UTs/ITs as part of
> the Atlas build.
> This JIRA focusses on titan0 and janusgraph 0.2.0, as they are the graph
> databases that will be supported in master/1.0.0. This JIRA deliberately
> ignores titan1 and janusgraph 0.1.1 as the former should be
> deprecated/removed and the other is a transient state as we get to janusgraph
> 0.2.0.
> With titan0 as the graph provider, Atlas has supported the following
> combinations of persistent store and indexer. It is suggested that this set
> is kept unchanged:
> {{
> titan0 solr es
>
> berkeley 0 1
> hbase 1 0
> cassandra0 0
> }}
> With janusgraph (0.2.0) as the graph provider, Atlas *could* support
> additional combinations. Cassandra is included in this discussion pending
> response to ATLAS-2259.
> {{
> janus 0.2.0 solr es
>
> berkeley ? 1
> hbase 1 ?
> cassandra? ?
> }}
> It is suggested that the combinations marked with '1' should be continued and
> the remaining 4 combinations, marked with '?', should be considered. There
> seems to be evidence of people using all 4 of these combinations, although
> not necessarily with Atlas.
> Depending on the decision made above, we need to ensure that it is possible
> to build Atlas as a standalone package with any of the combinations - i.e.
> that they are mutually exclusive and do not interfere with one another. They
> currently interfere which makes it impossible to build Atlas with
> -Pdist,berkeley-elasticsearch because the 'dist' profile will exclude jars
> that are needed by the berkeley-elasticsearch profile - which leads to class
> not found exceptions when the Atlas server is started. The solution to this
> could be very simple, or slightly more sophisticated, depending on how many
> of the combinations we choose to support.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (ATLAS-2270) Supported combinations of persistent store and index backend
[
https://issues.apache.org/jira/browse/ATLAS-2270?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16267205#comment-16267205
]
Pierre Padovani commented on ATLAS-2270:
Not everyone wants to be tied to using Hadoop/HDFS as part of their
infrastructure. :D
The issue I linked to contains the changes required to the pom.xml for Janus to
enable Cassandra support. The only other thing needed is documentation around
how to set it up. I've built up standalone docker images that contain:
- zookeeper
- cassandra
- Elasticsearch
- atlas (Janus 0.1.0 - working to build 0.2.0 now)
We currently deploy these images for development while we prepare to roll Atlas
in this same configuration.
I'm happy to contribute what else may be needed in this regard.
> Supported combinations of persistent store and index backend
>
>
> Key: ATLAS-2270
> URL: https://issues.apache.org/jira/browse/ATLAS-2270
> Project: Atlas
> Issue Type: Bug
>Reporter: Graham Wallis
>
> We need to discuss and decide which combinations of persistent store and
> indexing backend Atlas 1.0.0 (master) should support. This includes
> building/running Atlas as a standalone package and running UTs/ITs as part of
> the Atlas build.
> This JIRA focusses on titan0 and janusgraph 0.2.0, as they are the graph
> databases that will be supported in master/1.0.0. This JIRA deliberately
> ignores titan1 and janusgraph 0.1.1 as the former should be
> deprecated/removed and the other is a transient state as we get to janusgraph
> 0.2.0.
> With titan0 as the graph provider, Atlas has supported the following
> combinations of persistent store and indexer. It is suggested that this set
> is kept unchanged:
> {{
> titan0 solr es
>
> berkeley 0 1
> hbase 1 0
> cassandra0 0
> }}
> With janusgraph (0.2.0) as the graph provider, Atlas *could* support
> additional combinations. Cassandra is included in this discussion pending
> response to ATLAS-2259.
> {{
> janus 0.2.0 solr es
>
> berkeley ? 1
> hbase 1 ?
> cassandra? ?
> }}
> It is suggested that the combinations marked with '1' should be continued and
> the remaining 4 combinations, marked with '?', should be considered. There
> seems to be evidence of people using all 4 of these combinations, although
> not necessarily with Atlas.
> Depending on the decision made above, we need to ensure that it is possible
> to build Atlas as a standalone package with any of the combinations - i.e.
> that they are mutually exclusive and do not interfere with one another. They
> currently interfere which makes it impossible to build Atlas with
> -Pdist,berkeley-elasticsearch because the 'dist' profile will exclude jars
> that are needed by the berkeley-elasticsearch profile - which leads to class
> not found exceptions when the Atlas server is started. The solution to this
> could be very simple, or slightly more sophisticated, depending on how many
> of the combinations we choose to support.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Comment Edited] (ATLAS-2270) Supported combinations of persistent store and index backend
[
https://issues.apache.org/jira/browse/ATLAS-2270?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16267074#comment-16267074
]
Pierre Padovani edited comment on ATLAS-2270 at 11/27/17 5:11 PM:
--
I would vote to add Cassandra as an option. Currently you *MUST* modify the
pom.xml to add Cassandra support and rebuild atlas. I raised this issue around
that: [ATLAS-2259|https://issues.apache.org/jira/browse/ATLAS-2259]
was (Author: ppadovani):
I would vote to add Cassandra as an option. Currently you *MUST* modify the
pom.xml to add Cassandra support and rebuild atlas. I raised this issue around
that: https://issues.apache.org/jira/browse/ATLAS-2259
> Supported combinations of persistent store and index backend
>
>
> Key: ATLAS-2270
> URL: https://issues.apache.org/jira/browse/ATLAS-2270
> Project: Atlas
> Issue Type: Bug
>Reporter: Graham Wallis
>
> We need to discuss and decide which combinations of persistent store and
> indexing backend Atlas 1.0.0 (master) should support. This includes
> building/running Atlas as a standalone package and running UTs/ITs as part of
> the Atlas build.
> This JIRA focusses on titan0 and janusgraph 0.2.0, as they are the graph
> databases that will be supported in master/1.0.0. This JIRA deliberately
> ignores titan1 and janusgraph 0.1.1 as the former should be
> deprecated/removed and the other is a transient state as we get to janusgraph
> 0.2.0.
> With titan0 as the graph provider, Atlas has supported the following
> combinations of persistent store and indexer. It is suggested that this set
> is kept unchanged:
> {{
> titan0 solr es
>
> berkeley 0 1
> hbase 1 0
> cassandra0 0
> }}
> With janusgraph (0.2.0) as the graph provider, Atlas *could* support
> additional combinations. Cassandra is included in this discussion pending
> response to ATLAS-2259.
> {{
> janus 0.2.0 solr es
>
> berkeley ? 1
> hbase 1 ?
> cassandra? ?
> }}
> It is suggested that the combinations marked with '1' should be continued and
> the remaining 4 combinations, marked with '?', should be considered. There
> seems to be evidence of people using all 4 of these combinations, although
> not necessarily with Atlas.
> Depending on the decision made above, we need to ensure that it is possible
> to build Atlas as a standalone package with any of the combinations - i.e.
> that they are mutually exclusive and do not interfere with one another. They
> currently interfere which makes it impossible to build Atlas with
> -Pdist,berkeley-elasticsearch because the 'dist' profile will exclude jars
> that are needed by the berkeley-elasticsearch profile - which leads to class
> not found exceptions when the Atlas server is started. The solution to this
> could be very simple, or slightly more sophisticated, depending on how many
> of the combinations we choose to support.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (ATLAS-2270) Supported combinations of persistent store and index backend
[
https://issues.apache.org/jira/browse/ATLAS-2270?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16267074#comment-16267074
]
Pierre Padovani commented on ATLAS-2270:
I would vote to add Cassandra as an option. Currently you *MUST* modify the
pom.xml to add Cassandra support and rebuild atlas. I raised this issue around
that: https://issues.apache.org/jira/browse/ATLAS-2259
> Supported combinations of persistent store and index backend
>
>
> Key: ATLAS-2270
> URL: https://issues.apache.org/jira/browse/ATLAS-2270
> Project: Atlas
> Issue Type: Bug
>Reporter: Graham Wallis
>
> We need to discuss and decide which combinations of persistent store and
> indexing backend Atlas 1.0.0 (master) should support. This includes
> building/running Atlas as a standalone package and running UTs/ITs as part of
> the Atlas build.
> This JIRA focusses on titan0 and janusgraph 0.2.0, as they are the graph
> databases that will be supported in master/1.0.0. This JIRA deliberately
> ignores titan1 and janusgraph 0.1.1 as the former should be
> deprecated/removed and the other is a transient state as we get to janusgraph
> 0.2.0.
> With titan0 as the graph provider, Atlas has supported the following
> combinations of persistent store and indexer. It is suggested that this set
> is kept unchanged:
> {{
> titan0 solr es
>
> berkeley 0 1
> hbase 1 0
> cassandra0 0
> }}
> With janusgraph (0.2.0) as the graph provider, Atlas *could* support
> additional combinations. Cassandra is included in this discussion pending
> response to ATLAS-2259.
> {{
> janus 0.2.0 solr es
>
> berkeley ? 1
> hbase 1 ?
> cassandra? ?
> }}
> It is suggested that the combinations marked with '1' should be continued and
> the remaining 4 combinations, marked with '?', should be considered. There
> seems to be evidence of people using all 4 of these combinations, although
> not necessarily with Atlas.
> Depending on the decision made above, we need to ensure that it is possible
> to build Atlas as a standalone package with any of the combinations - i.e.
> that they are mutually exclusive and do not interfere with one another. They
> currently interfere which makes it impossible to build Atlas with
> -Pdist,berkeley-elasticsearch because the 'dist' profile will exclude jars
> that are needed by the berkeley-elasticsearch profile - which leads to class
> not found exceptions when the Atlas server is started. The solution to this
> could be very simple, or slightly more sophisticated, depending on how many
> of the combinations we choose to support.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (ATLAS-2259) Add Janus Graph Cassandra support to the default build
[
https://issues.apache.org/jira/browse/ATLAS-2259?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pierre Padovani updated ATLAS-2259:
---
Issue Type: Improvement (was: Bug)
> Add Janus Graph Cassandra support to the default build
> --
>
> Key: ATLAS-2259
> URL: https://issues.apache.org/jira/browse/ATLAS-2259
> Project: Atlas
> Issue Type: Improvement
> Components: atlas-core
>Affects Versions: 1.0.0
>Reporter: Pierre Padovani
> Fix For: 1.0.0
>
>
> Atlas should have support for Cassandra as a backend for Janus available by
> default. If someone wants this type of configuration, they have to modify the
> pom.xml and rebuild Atlas. Here is the pom.xml modification required to
> enable this support:
> {code:java}
>
> org.janusgraph
> janusgraph-cassandra
> ${janus.version}
>
>
> org.codehaus.jettison
> jettison
>
>
> commons-lang
> commons-lang
>
>
>
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (ATLAS-2259) Add Janus Graph Cassandra support to the default build
Pierre Padovani created ATLAS-2259:
--
Summary: Add Janus Graph Cassandra support to the default build
Key: ATLAS-2259
URL: https://issues.apache.org/jira/browse/ATLAS-2259
Project: Atlas
Issue Type: Bug
Components: atlas-core
Affects Versions: 1.0.0
Reporter: Pierre Padovani
Fix For: 1.0.0
Atlas should have support for Cassandra as a backend for Janus available by
default. If someone wants this type of configuration, they have to modify the
pom.xml and rebuild Atlas. Here is the pom.xml modification required to enable
this support:
{code:java}
org.janusgraph
janusgraph-cassandra
${janus.version}
org.codehaus.jettison
jettison
commons-lang
commons-lang
{code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (ATLAS-2249) Upgrade Atlas Client to use Jersey 2
[ https://issues.apache.org/jira/browse/ATLAS-2249?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Padovani updated ATLAS-2249: --- Attachment: client-jersey2-V3.patch Updated the patch to suppress HTTP compliance validation. This was needed because Atlas does not conform to the standards around DELETE. This verb requires no payload, but Atlas can accept a set of entities to delete. > Upgrade Atlas Client to use Jersey 2 > > > Key: ATLAS-2249 > URL: https://issues.apache.org/jira/browse/ATLAS-2249 > Project: Atlas > Issue Type: Improvement > Components: atlas-intg >Affects Versions: 1.0.0 >Reporter: Pierre Padovani > Labels: patch-available > Fix For: 1.0.0 > > Attachments: client-jersey2-V2.patch, client-jersey2-V3.patch, > client-jersey2.patch > > > Using the existing Atlas client within a Jersey 2 service causes conflicts. I > recommend upgrading just the client code to Jersey 2. The enclosed patch > fully upgrades the V1 and V2 client to Jersey 2, and we have tested this > against our own installation of 1.0.0-SNAPSHOT. > NOTE: We have not tested SSL/Kerberos authentication/security -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (ATLAS-2249) Upgrade Atlas Client to use Jersey 2
[ https://issues.apache.org/jira/browse/ATLAS-2249?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Padovani updated ATLAS-2249: --- Attachment: client-jersey2-V2.patch Updated the patch to handle a case with a null value for an attribute being deserialized as a string "null" instead of null. > Upgrade Atlas Client to use Jersey 2 > > > Key: ATLAS-2249 > URL: https://issues.apache.org/jira/browse/ATLAS-2249 > Project: Atlas > Issue Type: Improvement > Components: atlas-intg >Affects Versions: 1.0.0 >Reporter: Pierre Padovani >Priority: Major > Labels: patch-available > Fix For: 1.0.0 > > Attachments: client-jersey2-V2.patch, client-jersey2.patch > > > Using the existing Atlas client within a Jersey 2 service causes conflicts. I > recommend upgrading just the client code to Jersey 2. The enclosed patch > fully upgrades the V1 and V2 client to Jersey 2, and we have tested this > against our own installation of 1.0.0-SNAPSHOT. > NOTE: We have not tested SSL/Kerberos authentication/security -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (ATLAS-2249) Upgrade Atlas Client to use Jersey 2
Pierre Padovani created ATLAS-2249: -- Summary: Upgrade Atlas Client to use Jersey 2 Key: ATLAS-2249 URL: https://issues.apache.org/jira/browse/ATLAS-2249 Project: Atlas Issue Type: Improvement Components: atlas-intg Affects Versions: 1.0.0 Reporter: Pierre Padovani Priority: Major Fix For: 1.0.0 Attachments: client-jersey2.patch Using the existing Atlas client within a Jersey 2 service causes conflicts. I recommend upgrading just the client code to Jersey 2. The enclosed patch fully upgrades the V1 and V2 client to Jersey 2, and we have tested this against our own installation of 1.0.0-SNAPSHOT. NOTE: We have not tested SSL/Kerberos authentication/security -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (ATLAS-2215) UI: Editing array of Structs behave as array of entity references
Pierre Padovani created ATLAS-2215: -- Summary: UI: Editing array of Structs behave as array of entity references Key: ATLAS-2215 URL: https://issues.apache.org/jira/browse/ATLAS-2215 Project: Atlas Issue Type: Bug Components: atlas-webui Affects Versions: 1.0.0 Reporter: Pierre Padovani Reproduction steps: * create a struct with a set of attributes * create an entity type with an attribute defined as an array of the previously defined struct * Attempt to create an instance of the entity and set the struct value. It will behave as if the type is an array of entities and not a free form field. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (ATLAS-2178) Sending RelationshipAttributes with entity update using legacy refs fails to create edge
[
https://issues.apache.org/jira/browse/ATLAS-2178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pierre Padovani updated ATLAS-2178:
---
Description:
*UPDATE*
I tracked down specifically what causes this and have updated the title and
this description to reflect that. I've left the old description as reference.
When using the AtlasClientV2 while performing complex add/modify operations, if
you do not NULL out the relationship attributes on an update, any references
that you may have added will not be set. Using the relationship REST API also
fails to set the legacy attributes on relationships, and fails to set them at
all in some cases.
This works:
{code:java}
{
"referredEntities": {},
"entity": {
"typeName": "Customer",
"attributes": {
"owner": null,
"extractProcesses": null,
"systems": [{"guid":"cd80c575-f092-4ea7-ad43-3ee0e4bbefd3",
"typeName":"SorSystem"}],
"qualifiedName": "clueacademy@dev-app-1",
"name": "clueAcademy",
"description": "Cluemasters Academy",
"collectedDataSets": null
},
"guid": "efb17484-15d7-4001-a0e5-e135d96eca9e",
"status": "ACTIVE",
"createdBy": "admin",
"updatedBy": "admin",
"createTime": 1506961203462,
"updateTime": 1506961207280,
"version": 0
}
}
{code}
This doesn't work:
{code:java}
{
"referredEntities": {},
"entity": {
"typeName": "Customer",
"attributes": {
"owner": null,
"extractProcesses": null,
"systems": [{"guid":"cd80c575-f092-4ea7-ad43-3ee0e4bbefd3",
"typeName":"SorSystem"}],
"qualifiedName": "clueacademy@dev-app-1",
"name": "clueAcademy",
"description": "Cluemasters Academy",
"collectedDataSets": null
},
"guid": "efb17484-15d7-4001-a0e5-e135d96eca9e",
"status": "ACTIVE",
"createdBy": "admin",
"updatedBy": "admin",
"createTime": 1506961203462,
"updateTime": 1506961207280,
"version": 0,
"relationshipAttributes": {
"extractProcesses": null,
"systems": null,
"collectedDataSets": null
},
"classifications": []
}
}
{code}
*Below is no longer valid*
If a type inherits from another that has an old style reference attribute, and
a new style RelationshipDef defined, creating the relationship instance fails
to set the legacy attributes and in some cases fails to create the relationship
itself.
Steps to reproduce:
* create a type, say my_table, that inherits from the hive_table
* Create an instance of the new type my_table
* Create an instance of a hive_column
* Create the containment relationship between the two models.
This behavior causes odd and inconsistent behavior and fully breaks lineage
reports for process types that inherit from the built in Process.
was:
*UPDATE*
I tracked down specifically what causes this and have updated the title and
this description to reflect that. I've left the old description as reference.
When using the AtlasClientV2 while performing complex add/modify operations, if
you do not NULL out the relationship attributes on an update, any references
that you may have added will not be set. Using the relationship REST API also
fails to set the legacy attributes on relationships, and fails to set them at
all in some cases.
This works:
{code:java}
{
"referredEntities": {},
"entity": {
"typeName": "Customer",
"attributes": {
"owner": null,
"extractProcesses": null,
"systems": [{"guid":"cd80c575-f092-4ea7-ad43-3ee0e4bbefd3",
"typeName":"SorSystem"}],
"qualifiedName": "clueacademy@dev-app-1",
"name": "clueAcademy",
"description": "Cluemasters Academy",
"collectedDataSets": null
},
"guid": "efb17484-15d7-4001-a0e5-e135d96eca9e"
}
}
{code}
This doesn't work:
{code:java}
{
"referredEntities": {},
"entity": {
"typeName": "Customer",
"attributes": {
"owner": null,
"extractProcesses": null,
"systems": [{"guid":"cd80c575-f092-4ea7-ad43-3ee0e4bbefd3",
"typeName":"SorSystem"}],
"qualifiedName": "clueacademy@dev-app-1",
"name": "clueAcademy",
"description": "Cluemasters Academy",
"collectedDataSets": null
},
"guid": "efb17484-15d7-4001-a0e5-e135d96eca9e",
"status": "ACTIVE",
"createdBy": "admin",
"updatedBy": "admin",
"createTime": 1506961203462,
"updateTime": 1506961207280,
"version": 0,
"relationshipAttributes": {
"extractProcesses": null,
"systems": null,
"collectedDataSets": null
},
"classifications": []
}
[jira] [Updated] (ATLAS-2178) Sending RelationshipAttributes with entity update using legacy refs fails to create edge
[
https://issues.apache.org/jira/browse/ATLAS-2178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pierre Padovani updated ATLAS-2178:
---
Description:
*UPDATE*
I tracked down specifically what causes this and have updated the title and
this description to reflect that. I've left the old description as reference.
When using the AtlasClientV2 while performing complex add/modify operations, if
you do not NULL out the relationship attributes on an update, any references
that you may have added will not be set. Using the relationship REST API also
fails to set the legacy attributes on relationships, and fails to set them at
all in some cases.
This works:
{code:java}
{
"referredEntities": {},
"entity": {
"typeName": "Customer",
"attributes": {
"owner": null,
"extractProcesses": null,
"systems": [{"guid":"cd80c575-f092-4ea7-ad43-3ee0e4bbefd3",
"typeName":"SorSystem"}],
"qualifiedName": "clueacademy@dev-app-1",
"name": "clueAcademy",
"description": "Cluemasters Academy",
"collectedDataSets": null
},
"guid": "efb17484-15d7-4001-a0e5-e135d96eca9e"
}
}
{code}
This doesn't work:
{code:java}
{
"referredEntities": {},
"entity": {
"typeName": "Customer",
"attributes": {
"owner": null,
"extractProcesses": null,
"systems": [{"guid":"cd80c575-f092-4ea7-ad43-3ee0e4bbefd3",
"typeName":"SorSystem"}],
"qualifiedName": "clueacademy@dev-app-1",
"name": "clueAcademy",
"description": "Cluemasters Academy",
"collectedDataSets": null
},
"guid": "efb17484-15d7-4001-a0e5-e135d96eca9e",
"status": "ACTIVE",
"createdBy": "admin",
"updatedBy": "admin",
"createTime": 1506961203462,
"updateTime": 1506961207280,
"version": 0,
"relationshipAttributes": {
"extractProcesses": null,
"systems": null,
"collectedDataSets": null
},
"classifications": []
}
}
{code}
*Below is no longer valid*
If a type inherits from another that has an old style reference attribute, and
a new style RelationshipDef defined, creating the relationship instance fails
to set the legacy attributes and in some cases fails to create the relationship
itself.
Steps to reproduce:
* create a type, say my_table, that inherits from the hive_table
* Create an instance of the new type my_table
* Create an instance of a hive_column
* Create the containment relationship between the two models.
This behavior causes odd and inconsistent behavior and fully breaks lineage
reports for process types that inherit from the built in Process.
was:
If a type inherits from another that has an old style reference attribute, and
a new style RelationshipDef defined, creating the relationship instance fails
to set the legacy attributes and in some cases fails to create the relationship
itself.
Steps to reproduce:
* create a type, say my_table, that inherits from the hive_table
* Create an instance of the new type my_table
* Create an instance of a hive_column
* Create the containment relationship between the two models.
This behavior causes odd and inconsistent behavior and fully breaks lineage
reports for process types that inherit from the built in Process.
> Sending RelationshipAttributes with entity update using legacy refs fails to
> create edge
>
>
> Key: ATLAS-2178
> URL: https://issues.apache.org/jira/browse/ATLAS-2178
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Affects Versions: 1.0.0
>Reporter: Pierre Padovani
> Attachments: entities.json, rel1.json, rel2.json, rel3.json,
> rel4.json, rel5.json, types.json
>
>
> *UPDATE*
> I tracked down specifically what causes this and have updated the title and
> this description to reflect that. I've left the old description as reference.
> When using the AtlasClientV2 while performing complex add/modify operations,
> if you do not NULL out the relationship attributes on an update, any
> references that you may have added will not be set. Using the relationship
> REST API also fails to set the legacy attributes on relationships, and fails
> to set them at all in some cases.
> This works:
> {code:java}
> {
> "referredEntities": {},
> "entity": {
> "typeName": "Customer",
> "attributes": {
> "owner": null,
> "extractProcesses": null,
> "systems": [{"guid":"cd80c575-f092-4ea7-ad43-3ee0e4bbefd3",
> "typeName":"SorSystem"}],
> "qualifiedName": "clueacademy@dev-app-1",
> "name": "clueAcademy",
> "description": "Cluemast
[jira] [Updated] (ATLAS-2178) Sending RelationshipAttributes with entity update using legacy refs fails to create edge
[ https://issues.apache.org/jira/browse/ATLAS-2178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Padovani updated ATLAS-2178: --- Summary: Sending RelationshipAttributes with entity update using legacy refs fails to create edge (was: Type Inheritance in conjunction with RelationshipDef fails to set legacy attributes) > Sending RelationshipAttributes with entity update using legacy refs fails to > create edge > > > Key: ATLAS-2178 > URL: https://issues.apache.org/jira/browse/ATLAS-2178 > Project: Atlas > Issue Type: Bug > Components: atlas-core >Affects Versions: 1.0.0 >Reporter: Pierre Padovani > Attachments: entities.json, rel1.json, rel2.json, rel3.json, > rel4.json, rel5.json, types.json > > > If a type inherits from another that has an old style reference attribute, > and a new style RelationshipDef defined, creating the relationship instance > fails to set the legacy attributes and in some cases fails to create the > relationship itself. > Steps to reproduce: > * create a type, say my_table, that inherits from the hive_table > * Create an instance of the new type my_table > * Create an instance of a hive_column > * Create the containment relationship between the two models. > This behavior causes odd and inconsistent behavior and fully breaks lineage > reports for process types that inherit from the built in Process. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (ATLAS-2178) Type Inheritance in conjunction with RelationshipDef fails to set legacy attributes
[ https://issues.apache.org/jira/browse/ATLAS-2178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Padovani updated ATLAS-2178: --- Attachment: (was: types.json) > Type Inheritance in conjunction with RelationshipDef fails to set legacy > attributes > --- > > Key: ATLAS-2178 > URL: https://issues.apache.org/jira/browse/ATLAS-2178 > Project: Atlas > Issue Type: Bug > Components: atlas-core >Affects Versions: 1.0.0 >Reporter: Pierre Padovani > Attachments: entities.json, rel1.json, rel2.json, rel3.json, > rel4.json, rel5.json, types.json > > > If a type inherits from another that has an old style reference attribute, > and a new style RelationshipDef defined, creating the relationship instance > fails to set the legacy attributes and in some cases fails to create the > relationship itself. > Steps to reproduce: > * create a type, say my_table, that inherits from the hive_table > * Create an instance of the new type my_table > * Create an instance of a hive_column > * Create the containment relationship between the two models. > This behavior causes odd and inconsistent behavior and fully breaks lineage > reports for process types that inherit from the built in Process. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (ATLAS-2178) Type Inheritance in conjunction with RelationshipDef fails to set legacy attributes
[ https://issues.apache.org/jira/browse/ATLAS-2178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Padovani updated ATLAS-2178: --- Attachment: types.json Updated types.json with fixes to inconsistent and missing RelationshipDefs. This does not change the overall behavior of relationships not being properly created. > Type Inheritance in conjunction with RelationshipDef fails to set legacy > attributes > --- > > Key: ATLAS-2178 > URL: https://issues.apache.org/jira/browse/ATLAS-2178 > Project: Atlas > Issue Type: Bug > Components: atlas-core >Affects Versions: 1.0.0 >Reporter: Pierre Padovani > Attachments: entities.json, rel1.json, rel2.json, rel3.json, > rel4.json, rel5.json, types.json > > > If a type inherits from another that has an old style reference attribute, > and a new style RelationshipDef defined, creating the relationship instance > fails to set the legacy attributes and in some cases fails to create the > relationship itself. > Steps to reproduce: > * create a type, say my_table, that inherits from the hive_table > * Create an instance of the new type my_table > * Create an instance of a hive_column > * Create the containment relationship between the two models. > This behavior causes odd and inconsistent behavior and fully breaks lineage > reports for process types that inherit from the built in Process. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Comment Edited] (ATLAS-2178) Type Inheritance in conjunction with RelationshipDef fails to set legacy attributes
[
https://issues.apache.org/jira/browse/ATLAS-2178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16184647#comment-16184647
]
Pierre Padovani edited comment on ATLAS-2178 at 9/29/17 9:00 PM:
-
Update: If I set the property 'atlas.ui.editable.entity.types=*' to allow for
custom types to be created from the UI, and create two entities (one on each
side of the CustomerOwned defined relationship type) I can properly define the
relationship on each side, and the data is set correctly.
I would assume that this is due to the UI using the V1 API rather than making a
call to the V2 Relationship API. That being said, my assumption was that if I
set the legacy flag on the RelationshipDef the attributes that correspond to
the field names in each end would be set correctly when the relationship
instance was created. Tracing through the code during a create relationship
call via the REST api, I see nothing that honors the legacy flag. (unless it is
happening under the covers in the titan code.)
was (Author: ppadovani):
Update: If I set the property 'atlas.ui.editable.entity.types=*' to allow for
custom types to be created from the UI, and create two entities (one on each
side of the CustomerOwned defined relationship type) I can properly define the
relationship on each side, and the data is set correctly.
{code:java}
{
"enumDefs": [],
"structDefs": [
{
"name": "FingerprintSpecification",
"typeVersion": "0.1",
"attributeDefs": [
{
"name": "tablesWithMetadataGlobs",
"typeName": "array",
"cardinality": "SINGLE",
"isIndexable": false,
"isOptional": true,
"isUnique": false
},
{
"name": "tablesListGlobs",
"typeName": "array",
"cardinality": "SINGLE",
"isIndexable": false,
"isOptional": true,
"isUnique": false
}
]
},
{
"name": "PfKeyDefinition",
"typeVersion": "0.1",
"attributeDefs": [
{
"name": "tableName",
"typeName": "string",
"cardinality": "SINGLE",
"isIndexable": false,
"isOptional": true,
"isUnique": false
},
{
"name": "columns",
"typeName": "array",
"cardinality": "SINGLE",
"isIndexable": false,
"isOptional": true,
"isUnique": false
}
]
},
{
"name": "ComparisonFilter",
"typeVersion": "0.1",
"attributeDefs": [
{
"name": "compareColumnNullability",
"typeName": "boolean",
"cardinality": "SINGLE",
"isIndexable": false,
"isOptional": false,
"isUnique": false
},
{
"name": "compareColumnRemarks",
"typeName": "boolean",
"cardinality": "SINGLE",
"isIndexable": false,
"isOptional": false,
"isUnique": false
},
{
"name": "compareColumnType",
"typeName": "boolean",
"cardinality": "SINGLE",
"isIndexable": false,
"isOptional": false,
"isUnique": false
},
{
"name": "compareColumnSize",
"typeName": "boolean",
"cardinality": "SINGLE",
"isIndexable": false,
"isOptional": false,
"isUnique": false
},
{
"name": "comparePrimaryKey",
"typeName": "boolean",
"cardinality": "SINGLE",
"isIndexable": false,
"isOptional": false,
"isUnique": false
},
{
"name": "compareTableRemarks",
"typeName": "boolean",
"cardinality": "SINGLE",
"isIndexable": false,
"isOptional": false,
"isUnique": false
},
{
"name": "compareRelationships",
"typeName": "boolean",
"cardinality": "SINGLE",
"isIndexable": false,
"isOptional": false,
"isUnique": false
}
]
},
{
"name": "ScoreSpecification",
"typeVersion": "0.1",
"attributeDefs": [
{
"name": "missingPrimaryKeys",
"typeName": "float",
"cardinality": "SINGLE",
"isIndexable": false,
"isOptional": false,
"isUnique": false
},
{
"name": "changedColumnTypes",
"typeName": "float",
"cardinality": "SINGLE",
"isIndexable": false,
"isOptional": false,
"isUnique": false
},
{
"name": "changedNullability",
"typeName": "float",
"cardinality": "SINGLE",
"isIndexable": false,
"isOptional": false,
"isUnique": false
},
{
"name": "
[jira] [Comment Edited] (ATLAS-2178) Type Inheritance in conjunction with RelationshipDef fails to set legacy attributes
[
https://issues.apache.org/jira/browse/ATLAS-2178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16184647#comment-16184647
]
Pierre Padovani edited comment on ATLAS-2178 at 9/29/17 8:58 PM:
-
Update: If I set the property 'atlas.ui.editable.entity.types=*' to allow for
custom types to be created from the UI, and create two entities (one on each
side of the CustomerOwned defined relationship type) I can properly define the
relationship on each side, and the data is set correctly.
{code:java}
{
"enumDefs": [],
"structDefs": [
{
"name": "FingerprintSpecification",
"typeVersion": "0.1",
"attributeDefs": [
{
"name": "tablesWithMetadataGlobs",
"typeName": "array",
"cardinality": "SINGLE",
"isIndexable": false,
"isOptional": true,
"isUnique": false
},
{
"name": "tablesListGlobs",
"typeName": "array",
"cardinality": "SINGLE",
"isIndexable": false,
"isOptional": true,
"isUnique": false
}
]
},
{
"name": "PfKeyDefinition",
"typeVersion": "0.1",
"attributeDefs": [
{
"name": "tableName",
"typeName": "string",
"cardinality": "SINGLE",
"isIndexable": false,
"isOptional": true,
"isUnique": false
},
{
"name": "columns",
"typeName": "array",
"cardinality": "SINGLE",
"isIndexable": false,
"isOptional": true,
"isUnique": false
}
]
},
{
"name": "ComparisonFilter",
"typeVersion": "0.1",
"attributeDefs": [
{
"name": "compareColumnNullability",
"typeName": "boolean",
"cardinality": "SINGLE",
"isIndexable": false,
"isOptional": false,
"isUnique": false
},
{
"name": "compareColumnRemarks",
"typeName": "boolean",
"cardinality": "SINGLE",
"isIndexable": false,
"isOptional": false,
"isUnique": false
},
{
"name": "compareColumnType",
"typeName": "boolean",
"cardinality": "SINGLE",
"isIndexable": false,
"isOptional": false,
"isUnique": false
},
{
"name": "compareColumnSize",
"typeName": "boolean",
"cardinality": "SINGLE",
"isIndexable": false,
"isOptional": false,
"isUnique": false
},
{
"name": "comparePrimaryKey",
"typeName": "boolean",
"cardinality": "SINGLE",
"isIndexable": false,
"isOptional": false,
"isUnique": false
},
{
"name": "compareTableRemarks",
"typeName": "boolean",
"cardinality": "SINGLE",
"isIndexable": false,
"isOptional": false,
"isUnique": false
},
{
"name": "compareRelationships",
"typeName": "boolean",
"cardinality": "SINGLE",
"isIndexable": false,
"isOptional": false,
"isUnique": false
}
]
},
{
"name": "ScoreSpecification",
"typeVersion": "0.1",
"attributeDefs": [
{
"name": "missingPrimaryKeys",
"typeName": "float",
"cardinality": "SINGLE",
"isIndexable": false,
"isOptional": false,
"isUnique": false
},
{
"name": "changedColumnTypes",
"typeName": "float",
"cardinality": "SINGLE",
"isIndexable": false,
"isOptional": false,
"isUnique": false
},
{
"name": "changedNullability",
"typeName": "float",
"cardinality": "SINGLE",
"isIndexable": false,
"isOptional": false,
"isUnique": false
},
{
"name": "changedColumnRemarks",
"typeName": "float",
"cardinality": "SINGLE",
"isIndexable": false,
"isOptional": false,
"isUnique": false
},
{
"name": "missingTables",
"typeName": "float",
"cardinality": "SINGLE",
"isIndexable": false,
"isOptional": false,
"isUnique": false
},
{
"name": "missingForeignKeys",
"typeName": "float",
"cardinality": "SINGLE",
"isIndexable": false,
"isOptional": false,
"isUnique": false
},
{
"name": "unexpectedTables",
"typeName": "float",
"cardinality": "SINGLE",
"isIndexable": false,
"isOptional": false,
"isUnique": false
},
[jira] [Commented] (ATLAS-2178) Type Inheritance in conjunction with RelationshipDef fails to set legacy attributes
[
https://issues.apache.org/jira/browse/ATLAS-2178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16184647#comment-16184647
]
Pierre Padovani commented on ATLAS-2178:
Update: If I set the property 'atlas.ui.editable.entity.types=*' to allow for
custom types to be created from the UI, and create two entities (one on each
side of the CustomerOwned defined relationship type) I can properly define the
relationship on each side, and the data is set correctly.
```{
"referredEntities": {
"1f85676c-b0a5-45b3-b528-8420e5cae066": {
"typeName": "SorSystem",
"attributes": {
"owner": null,
"tables": null,
"qualifiedName": "sor@cust1",
"name": "sor",
"description": null,
"customer": {
"guid": "bec94ddc-dec3-42cb-b1c3-b19dff655693",
"typeName": "Customer"
}
},
"guid": "1f85676c-b0a5-45b3-b528-8420e5cae066",
"status": "ACTIVE",
"createdBy": "admin",
"updatedBy": "admin",
"createTime": 1506624242409,
"updateTime": 1506624364249,
"version": 0,
"relationshipAttributes": {
"tables": [],
"sourceToProcesses": [],
"sinkFromProcesses": [],
"customer": {
"guid": "bec94ddc-dec3-42cb-b1c3-b19dff655693",
"typeName": "Customer",
"displayText": "cust1",
"relationshipGuid": "ad2bb1bb-9da1-4fc5-9ca6-25d09b583541",
"relationshipAttributes": {
"typeName": "CustomerOwnership"
}
}
},
"classifications": []
}
},
"entity": {
"typeName": "Customer",
"attributes": {
"owner": null,
"systems": [
{
"guid": "1f85676c-b0a5-45b3-b528-8420e5cae066",
"typeName": "SorSystem"
}
],
"qualifiedName": "cust1",
"name": "cust1",
"description": null
},
"guid": "bec94ddc-dec3-42cb-b1c3-b19dff655693",
"status": "ACTIVE",
"createdBy": "admin",
"updatedBy": "admin",
"createTime": 1506624164918,
"updateTime": 1506624332364,
"version": 0,
"relationshipAttributes": {
"systems": {
"guid": "1f85676c-b0a5-45b3-b528-8420e5cae066",
"typeName": "SorSystem",
"displayText": "sor",
"relationshipGuid": "e63c1d2b-08c4-483a-a345-c2d6d8cb7bb7",
"relationshipAttributes": {
"typeName": "CustomerOwnership"
}
}
},
"classifications": []
}
}```
I would assume that this is due to the UI using the V1 API rather than making a
call to the V2 Relationship API. That being said, my assumption was that if I
set the legacy flag on the RelationshipDef the attributes that correspond to
the field names in each end would be set correctly when the relationship
instance was created. Tracing through the code during a create relationship
call via the REST api, I see nothing that honors the legacy flag. (unless it is
happening under the covers in the titan code.)
> Type Inheritance in conjunction with RelationshipDef fails to set legacy
> attributes
> ---
>
> Key: ATLAS-2178
> URL: https://issues.apache.org/jira/browse/ATLAS-2178
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Affects Versions: 1.0.0
>Reporter: Pierre Padovani
> Attachments: entities.json, rel1.json, rel2.json, rel3.json,
> rel4.json, rel5.json, types.json
>
>
> If a type inherits from another that has an old style reference attribute,
> and a new style RelationshipDef defined, creating the relationship instance
> fails to set the legacy attributes and in some cases fails to create the
> relationship itself.
> Steps to reproduce:
> * create a type, say my_table, that inherits from the hive_table
> * Create an instance of the new type my_table
> * Create an instance of a hive_column
> * Create the containment relationship between the two models.
> This behavior causes odd and inconsistent behavior and fully breaks lineage
> reports for process types that inherit from the built in Process.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Comment Edited] (ATLAS-2178) Type Inheritance in conjunction with RelationshipDef fails to set legacy attributes
[
https://issues.apache.org/jira/browse/ATLAS-2178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16184647#comment-16184647
]
Pierre Padovani edited comment on ATLAS-2178 at 9/28/17 6:54 PM:
-
Update: If I set the property 'atlas.ui.editable.entity.types=*' to allow for
custom types to be created from the UI, and create two entities (one on each
side of the CustomerOwned defined relationship type) I can properly define the
relationship on each side, and the data is set correctly.
{code:java}
{
"referredEntities": {
"1f85676c-b0a5-45b3-b528-8420e5cae066": {
"typeName": "SorSystem",
"attributes": {
"owner": null,
"tables": null,
"qualifiedName": "sor@cust1",
"name": "sor",
"description": null,
"customer": {
"guid": "bec94ddc-dec3-42cb-b1c3-b19dff655693",
"typeName": "Customer"
}
},
"guid": "1f85676c-b0a5-45b3-b528-8420e5cae066",
"status": "ACTIVE",
"createdBy": "admin",
"updatedBy": "admin",
"createTime": 1506624242409,
"updateTime": 1506624364249,
"version": 0,
"relationshipAttributes": {
"tables": [],
"sourceToProcesses": [],
"sinkFromProcesses": [],
"customer": {
"guid": "bec94ddc-dec3-42cb-b1c3-b19dff655693",
"typeName": "Customer",
"displayText": "cust1",
"relationshipGuid": "ad2bb1bb-9da1-4fc5-9ca6-25d09b583541",
"relationshipAttributes": {
"typeName": "CustomerOwnership"
}
}
},
"classifications": []
}
},
"entity": {
"typeName": "Customer",
"attributes": {
"owner": null,
"systems": [
{
"guid": "1f85676c-b0a5-45b3-b528-8420e5cae066",
"typeName": "SorSystem"
}
],
"qualifiedName": "cust1",
"name": "cust1",
"description": null
},
"guid": "bec94ddc-dec3-42cb-b1c3-b19dff655693",
"status": "ACTIVE",
"createdBy": "admin",
"updatedBy": "admin",
"createTime": 1506624164918,
"updateTime": 1506624332364,
"version": 0,
"relationshipAttributes": {
"systems": {
"guid": "1f85676c-b0a5-45b3-b528-8420e5cae066",
"typeName": "SorSystem",
"displayText": "sor",
"relationshipGuid": "e63c1d2b-08c4-483a-a345-c2d6d8cb7bb7",
"relationshipAttributes": {
"typeName": "CustomerOwnership"
}
}
},
"classifications": []
}
}
{code}
I would assume that this is due to the UI using the V1 API rather than making a
call to the V2 Relationship API. That being said, my assumption was that if I
set the legacy flag on the RelationshipDef the attributes that correspond to
the field names in each end would be set correctly when the relationship
instance was created. Tracing through the code during a create relationship
call via the REST api, I see nothing that honors the legacy flag. (unless it is
happening under the covers in the titan code.)
was (Author: ppadovani):
Update: If I set the property 'atlas.ui.editable.entity.types=*' to allow for
custom types to be created from the UI, and create two entities (one on each
side of the CustomerOwned defined relationship type) I can properly define the
relationship on each side, and the data is set correctly.
```{
"referredEntities": {
"1f85676c-b0a5-45b3-b528-8420e5cae066": {
"typeName": "SorSystem",
"attributes": {
"owner": null,
"tables": null,
"qualifiedName": "sor@cust1",
"name": "sor",
"description": null,
"customer": {
"guid": "bec94ddc-dec3-42cb-b1c3-b19dff655693",
"typeName": "Customer"
}
},
"guid": "1f85676c-b0a5-45b3-b528-8420e5cae066",
"status": "ACTIVE",
"createdBy": "admin",
"updatedBy": "admin",
"createTime": 1506624242409,
"updateTime": 1506624364249,
"version": 0,
"relationshipAttributes": {
"tables": [],
"sourceToProcesses": [],
"sinkFromProcesses": [],
"customer": {
"guid": "bec94ddc-dec3-42cb-b1c3-b19dff655693",
"typeName": "Cu
[jira] [Comment Edited] (ATLAS-2178) Type Inheritance in conjunction with RelationshipDef fails to set legacy attributes
[ https://issues.apache.org/jira/browse/ATLAS-2178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16182661#comment-16182661 ] Pierre Padovani edited comment on ATLAS-2178 at 9/27/17 3:36 PM: - I've attached json files that can reproduce the issues I'm seeing. types.json is the type system, while entities.json are the new entities created. Each of the rel#.json files contains the payload to create a relationship. The guids in the rel#.json files have to be updated to match the created entities prior to submission. Results seen from these files: SorTable instance does not know about linked column, column doesn't know about table. SorExtraction (Process) only has outputs set, but no inputs. Many other inconsistencies in how relationships were set including the legacy attributes not being set correctly. was (Author: ppadovani): types.json is the type system, while entities.json are the new entities created. Each of the rel#.json files contains the payload to create a relationship. The guids in the relationship#.json files have to be updated to match the created entities prior to submission. Results seen from these files: SorTable instance does not know about linked column, column doesn't know about table. SorExtraction (Process) only has outputs set, but no inputs. Many other inconsistencies in how relationships were set including the legacy attributes not being set correctly. > Type Inheritance in conjunction with RelationshipDef fails to set legacy > attributes > --- > > Key: ATLAS-2178 > URL: https://issues.apache.org/jira/browse/ATLAS-2178 > Project: Atlas > Issue Type: Bug > Components: atlas-core >Affects Versions: 1.0.0 >Reporter: Pierre Padovani > Attachments: entities.json, rel1.json, rel2.json, rel3.json, > rel4.json, rel5.json, types.json > > > If a type inherits from another that has an old style reference attribute, > and a new style RelationshipDef defined, creating the relationship instance > fails to set the legacy attributes and in some cases fails to create the > relationship itself. > Steps to reproduce: > * create a type, say my_table, that inherits from the hive_table > * Create an instance of the new type my_table > * Create an instance of a hive_column > * Create the containment relationship between the two models. > This behavior causes odd and inconsistent behavior and fully breaks lineage > reports for process types that inherit from the built in Process. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (ATLAS-2178) Type Inheritance in conjunction with RelationshipDef fails to set legacy attributes
[ https://issues.apache.org/jira/browse/ATLAS-2178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Padovani updated ATLAS-2178: --- Attachment: entities.json rel1.json rel2.json rel3.json rel4.json rel5.json types.json types.json is the type system, while entities.json are the new entities created. Each of the rel#.json files contains the payload to create a relationship. The guids in the relationship#.json files have to be updated to match the created entities prior to submission. Results seen from these files: SorTable instance does not know about linked column, column doesn't know about table. SorExtraction (Process) only has outputs set, but no inputs. Many other inconsistencies in how relationships were set including the legacy attributes not being set correctly. > Type Inheritance in conjunction with RelationshipDef fails to set legacy > attributes > --- > > Key: ATLAS-2178 > URL: https://issues.apache.org/jira/browse/ATLAS-2178 > Project: Atlas > Issue Type: Bug > Components: atlas-core >Affects Versions: 1.0.0 >Reporter: Pierre Padovani > Attachments: entities.json, rel1.json, rel2.json, rel3.json, > rel4.json, rel5.json, types.json > > > If a type inherits from another that has an old style reference attribute, > and a new style RelationshipDef defined, creating the relationship instance > fails to set the legacy attributes and in some cases fails to create the > relationship itself. > Steps to reproduce: > * create a type, say my_table, that inherits from the hive_table > * Create an instance of the new type my_table > * Create an instance of a hive_column > * Create the containment relationship between the two models. > This behavior causes odd and inconsistent behavior and fully breaks lineage > reports for process types that inherit from the built in Process. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (ATLAS-1690) Introduce top level relationships
[ https://issues.apache.org/jira/browse/ATLAS-1690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16181262#comment-16181262 ] Pierre Padovani commented on ATLAS-1690: [~davidrad] I create this issue, as it seems to be the one common behavior that seems to be affecting this feature: https://issues.apache.org/jira/browse/ATLAS-2178 > Introduce top level relationships > - > > Key: ATLAS-1690 > URL: https://issues.apache.org/jira/browse/ATLAS-1690 > Project: Atlas > Issue Type: Improvement >Reporter: David Radley >Assignee: David Radley > Labels: VirtualDataConnector > Attachments: Atlas_RelationDef_Json_Structure_v1.pdf, Atlas > Relationships proposal v1.0.pdf, Atlas Relationships proposal v1.10.pdf, > Atlas Relationships proposal v1.1.pdf, Atlas Relationships proposal v1.2.pdf, > Atlas Relationships proposal v1.3.pdf, Atlas Relationships proposal v1.4.pdf, > Atlas Relationships proposal v1.5.pdf, Atlas Relationships proposal v1.6.pdf, > Atlas Relationships proposal v1.7.pdf, Atlas Relationships proposal v1.8.pdf, > Atlas Relationships proposal v1.9.pdf > > > Introduce top level relationships including support for > -many to many relationships > - relationship names including the name for both ends and the relationship. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (ATLAS-2178) Type Inheritance in conjunction with RelationshipDef fails to set legacy attributes
Pierre Padovani created ATLAS-2178: -- Summary: Type Inheritance in conjunction with RelationshipDef fails to set legacy attributes Key: ATLAS-2178 URL: https://issues.apache.org/jira/browse/ATLAS-2178 Project: Atlas Issue Type: Bug Components: atlas-core Affects Versions: 0.9-incubating Reporter: Pierre Padovani If a type inherits from another that has an old style reference attribute, and a new style RelationshipDef defined, creating the relationship instance fails to set the legacy attributes and in some cases fails to create the relationship itself. Steps to reproduce: * create a type, say my_table, that inherits from the hive_table * Create an instance of the new type my_table * Create an instance of a hive_column * Create the containment relationship between the two models. This behavior causes odd and inconsistent behavior and fully breaks lineage reports for process types that inherit from the built in Process. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (ATLAS-1690) Introduce top level relationships
[ https://issues.apache.org/jira/browse/ATLAS-1690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16180893#comment-16180893 ] Pierre Padovani commented on ATLAS-1690: [~davidrad] Thanks for the explanation. What is the status of this work? I've tried using 0.9 and have found the new relationship system to be a bit unstable in regards to the legacy compatibility. I've also noticed that the existing lineage API does not function correctly in conjunction with an implementation that uses relationship defs. The exact same model without defined relationship defs, functions correctly in 0.8.1. Thanks! Pierre > Introduce top level relationships > - > > Key: ATLAS-1690 > URL: https://issues.apache.org/jira/browse/ATLAS-1690 > Project: Atlas > Issue Type: Improvement >Reporter: David Radley >Assignee: David Radley > Labels: VirtualDataConnector > Attachments: Atlas_RelationDef_Json_Structure_v1.pdf, Atlas > Relationships proposal v1.0.pdf, Atlas Relationships proposal v1.10.pdf, > Atlas Relationships proposal v1.1.pdf, Atlas Relationships proposal v1.2.pdf, > Atlas Relationships proposal v1.3.pdf, Atlas Relationships proposal v1.4.pdf, > Atlas Relationships proposal v1.5.pdf, Atlas Relationships proposal v1.6.pdf, > Atlas Relationships proposal v1.7.pdf, Atlas Relationships proposal v1.8.pdf, > Atlas Relationships proposal v1.9.pdf > > > Introduce top level relationships including support for > -many to many relationships > - relationship names including the name for both ends and the relationship. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (ATLAS-1690) Introduce top level relationships
[ https://issues.apache.org/jira/browse/ATLAS-1690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16149536#comment-16149536 ] Pierre Padovani commented on ATLAS-1690: Two questions on this work: 1) Can a RelationshipType inherit from another RelationshipType? 2) If yes for #1, how does that affect tag propagation? My use case: Given a set of metadata that belongs to a particular customer, where we use a classification with an attribute that contains the customer id, we would want that classification to propagate to all related entities. If we had a relationship type that was generic in nature and specified that the classification for the customer propagated, it would be nice to be able to specify sub-types that provide additional behaviors. > Introduce top level relationships > - > > Key: ATLAS-1690 > URL: https://issues.apache.org/jira/browse/ATLAS-1690 > Project: Atlas > Issue Type: Improvement >Reporter: David Radley >Assignee: David Radley > Labels: VirtualDataConnector > Attachments: Atlas_RelationDef_Json_Structure_v1.pdf, Atlas > Relationships proposal v1.0.pdf, Atlas Relationships proposal v1.10.pdf, > Atlas Relationships proposal v1.1.pdf, Atlas Relationships proposal v1.2.pdf, > Atlas Relationships proposal v1.3.pdf, Atlas Relationships proposal v1.4.pdf, > Atlas Relationships proposal v1.5.pdf, Atlas Relationships proposal v1.6.pdf, > Atlas Relationships proposal v1.7.pdf, Atlas Relationships proposal v1.8.pdf, > Atlas Relationships proposal v1.9.pdf > > > Introduce top level relationships including support for > -many to many relationships > - relationship names including the name for both ends and the relationship. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (ATLAS-1764) Design and implement Atlas Collections
[ https://issues.apache.org/jira/browse/ATLAS-1764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16138884#comment-16138884 ] Pierre Padovani commented on ATLAS-1764: My use case: Consider a composite metadata entity that is made up of many (50-70+) Atlas entities. This composite entity is pulled in order to drive a portion of a data pipeline. The composite might contain things like: * tables * columns * config * target system information Individually, the small Atlas metadata entities are useless to the data pipeline, only if the runtime engine has all of the metadata required will it be able to function. Currently composing and decomposing an entity such as this requires specialized code that understands how to traverse and retrieve all of the pieces. Implementing a form of entity collection would allow for the creation of these types of composed metadata sets that could be retrieved etc. far easier. > Design and implement Atlas Collections > -- > > Key: ATLAS-1764 > URL: https://issues.apache.org/jira/browse/ATLAS-1764 > Project: Atlas > Issue Type: New Feature > Components: atlas-core >Affects Versions: 0.9-incubating >Reporter: Sarath Subramanian >Assignee: Sarath Subramanian > Labels: features > > Design and implement Atlas Collections - A first class element in Atlas to > group related entities together and perform operations on these collections - > associate tags, add/remove entities... > Operations: > --- > a. Create collection(s) with attributes, constraints > b. Update existing collection(s) > c. Delete collection(s) - soft delete, hard delete > d. Retrieve collections by id, name > e. Add entity(s) to collection(s) > f. Remove entity(s) from collection(s) > g. Associate classification(s) to collection(s) > h. Disassociate classification(s) from collection(s) -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (ATLAS-1863) Set default value for primitive types attributes in entity based on attributeDef in Typedef
[
https://issues.apache.org/jira/browse/ATLAS-1863?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16138871#comment-16138871
]
Pierre Padovani commented on ATLAS-1863:
IMHO - A default value implies that the field is 'not null' *NOT* optional.
This implies that the field is required to have a value when the entity is
created, just that user input of the field is optional. Having a default value
set for an optional field just doesn't make that much sense to me, however
having it set for a field that is required makes a little more sense but still
doesn't quite work either.
> Set default value for primitive types attributes in entity based on
> attributeDef in Typedef
> ---
>
> Key: ATLAS-1863
> URL: https://issues.apache.org/jira/browse/ATLAS-1863
> Project: Atlas
> Issue Type: Improvement
> Components: atlas-intg
>Reporter: Nixon Rodrigues
>Assignee: Ruchi Solani
>Priority: Critical
> Fix For: 0.9-incubating
>
> Attachments: ATLAS-1863.2.patch, ATLAS-1863.3.patch, ATLAS-1863.patch
>
>
> While creating entity if attribute value are not set explicitly for primitive
> type which are optional, then default value should be set from attributedef.
> eg of typedef attributeDef
> {code}
> "attributeDefs": [{
> "name": "sourceCode",
> "typeName": "string",
> "isOptional": true,
> "cardinality": "SINGLE",
> "valuesMinCount": 0,
> "valuesMaxCount": 1,
> "isUnique": false,
> "isIndexable": true,
> "defaultValue": "xyz"
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
