[
https://issues.apache.org/jira/browse/ATLAS-902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vimal Sharma updated ATLAS-902:
-------------------------------
Attachment: ATLAS-902.patch
> Atlas throws exception due to null definition in Hive create table statement
> ----------------------------------------------------------------------------
>
> Key: ATLAS-902
> URL: https://issues.apache.org/jira/browse/ATLAS-902
> Project: Atlas
> Issue Type: Bug
> Affects Versions: 0.5-incubating
> Environment: HDP 2.4 Sandbox
> Reporter: Ryan Templeton
> Assignee: Vimal Sharma
> Priority: Minor
> Attachments: ATLAS-902.patch
>
>
> I was using the following SQL statement to create a Hive table, but Atlas was
> throwing the following exception.
> create external table got_deaths(
> name varchar(64),
> allegiances varchar(32),
> death_year smallint,
> book_of_death tinyint,
> death_chapter tinyint,
> book_intro_chapter tinyint,
> gender tinyint,
> nobility tinyint,
> got tinyint,
> cok tinyint,
> sos tinyint,
> ffc tinyint,
> dwd tinyint
> )
> row format delimited fields terminated by ',' lines terminated by '\n' null
> defined as ''
> stored as TEXTFILE
> location '/user/root/simpleHive/'
> tblproperties ("skip.header.line.count"="1")
> After dropping the "null defined as '' " portion of the create statement,
> Atlas stopped throwing exceptions. I will point out that although Atlas was
> throwing an exception, Hive was able to create the table without issue. So I
> don't think this is semantically incorrect.
> FAILED: Hive Internal Error: org.apache.atlas.AtlasServiceException(Metadata
> service API CREATE_ENTITY failed with status 400(Bad Request) Response Body
> ({"error":"Cannot convert value 'org.apache.atlas.typesystem.Struct@621c7b32'
> to datatype
> hive_serde","stackTrace":"org.apache.atlas.typesystem.types.ValueConversionException:
> Cannot convert value 'org.apache.atlas.typesystem.Struct@621c7b32' to
> datatype hive_serde\n\tat
> org.apache.atlas.typesystem.types.TypedStructHandler.convert(TypedStructHandler.java:67)\n\tat
>
> org.apache.atlas.typesystem.types.StructType.convert(StructType.java:168)\n\tat
>
> org.apache.atlas.typesystem.types.StructType.convert(StructType.java:31)\n\tat
>
> org.apache.atlas.typesystem.persistence.StructInstance.set(StructInstance.java:118)\n\tat
>
> org.apache.atlas.typesystem.types.ClassType.convert(ClassType.java:141)\n\tat
> org.apache.atlas.services.DefaultMetadataService.deserializeClassInstance(DefaultMetadataService.java:252)\n\tat
>
> org.apache.atlas.services.DefaultMetadataService.createEntity(DefaultMetadataService.java:230)\n\tat
>
> org.apache.atlas.web.resources.EntityResource.submit(EntityResource.java:96)\n\tat
> sun.reflect.GeneratedMethodAccessor32.invoke(Unknown Source)\n\tat
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)\n\tat
> java.lang.reflect.Method.invoke(Method.java:606)\n\tat
> com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60)\n\tat
>
> com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$ResponseOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:205)\n\tat
>
> com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75)\n\tat
>
> com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:288)\n\tat
>
> com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:108)\n\tat
>
> com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)\n\tat
>
> com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:84)\n\tat
>
> com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1469)\n\tat
>
> com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1400)\n\tat
>
> com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1349)\n\tat
>
> com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1339)\n\tat
>
> com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:409)\n\tat
>
> com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:558)\n\tat
>
> com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:733)\n\tat
> javax.servlet.http.HttpServlet.service(HttpServlet.java:790)\n\tat
> com.google.inject.servlet.ServletDefinition.doServiceImpl(ServletDefinition.java:287)\n\tat
>
> com.google.inject.servlet.ServletDefinition.doService(ServletDefinition.java:277)\n\tat
>
> com.google.inject.servlet.ServletDefinition.service(ServletDefinition.java:182)\n\tat
>
> com.google.inject.servlet.ManagedServletPipeline.service(ManagedServletPipeline.java:91)\n\tat
>
> com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:85)\n\tat
> org.apache.atlas.web.filters.AuditFilter.doFilter(AuditFilter.java:67)\n\tat
> com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82)\n\tat
>
> com.google.inject.servlet.ManagedFilterPipeline.dispatch(ManagedFilterPipeline.java:119)\n\tat
> com.google.inject.servlet.GuiceFilter$1.call(GuiceFilter.java:133)\n\tat
> com.google.inject.servlet.GuiceFilter$1.call(GuiceFilter.java:130)\n\tat
> com.google.inject.servlet.GuiceFilter$Context.call(GuiceFilter.java:203)\n\tat
> com.google.inject.servlet.GuiceFilter.doFilter(GuiceFilter.java:130)\n\tat
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)\n\tat
>
> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)\n\tat
>
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)\n\tat
>
> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:577)\n\tat
>
> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:223)\n\tat
>
> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)\n\tat
>
> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)\n\tat
>
> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)\n\tat
>
> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)\n\tat
>
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)\n\tat
>
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)\n\tat
> org.eclipse.jetty.server.Server.handle(Server.java:499)\n\tat
> org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310)\n\tat
> org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)\n\tat
>
> org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540)\n\tat
>
> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)\n\tat
>
> org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)\n\tat
> java.lang.Thread.run(Thread.java:745)\nCaused by:
> org.apache.atlas.typesystem.types.ValueConversionException$NullConversionException:
> Null value not allowed for multiplicty Multiplicity{lower=1, upper=1,
> isUnique=false}\n\tat
> org.apache.atlas.typesystem.types.DataTypes$PrimitiveType.convertNull(DataTypes.java:93)\n\tat
>
> org.apache.atlas.typesystem.types.DataTypes$StringType.convert(DataTypes.java:469)\n\tat
>
> org.apache.atlas.typesystem.types.DataTypes$StringType.convert(DataTypes.java:452)\n\tat
>
> org.apache.atlas.typesystem.types.DataTypes$MapType.convert(DataTypes.java:606)\n\tat
>
> org.apache.atlas.typesystem.types.DataTypes$MapType.convert(DataTypes.java:562)\n\tat
>
> org.apache.atlas.typesystem.persistence.StructInstance.set(StructInstance.java:118)\n\tat
>
> org.apache.atlas.typesystem.types.TypedStructHandler.convert(TypedStructHandler.java:65)\n\t...
> 55 more\n"}))
> org.apache.atlas.AtlasServiceException: Metadata service API CREATE_ENTITY
> failed with status 400(Bad Request) Response Body ({"error":"Cannot convert
> value 'org.apache.atlas.typesystem.Struct@621c7b32' to datatype
> hive_serde","stackTrace":"org.apache.atlas.typesystem.types.ValueConversionException:
> Cannot convert value 'org.apache.atlas.typesystem.Struct@621c7b32' to
> datatype hive_serde\n\tat
> org.apache.atlas.typesystem.types.TypedStructHandler.convert(TypedStructHandler.java:67)\n\tat
>
> org.apache.atlas.typesystem.types.StructType.convert(StructType.java:168)\n\tat
>
> org.apache.atlas.typesystem.types.StructType.convert(StructType.java:31)\n\tat
>
> org.apache.atlas.typesystem.persistence.StructInstance.set(StructInstance.java:118)\n\tat
>
> org.apache.atlas.typesystem.types.ClassType.convert(ClassType.java:141)\n\tat
> org.apache.atlas.services.DefaultMetadataService.deserializeClassInstance(DefaultMetadataService.java:252)\n\tat
>
> org.apache.atlas.services.DefaultMetadataService.createEntity(DefaultMetadataService.java:230)\n\tat
>
> org.apache.atlas.web.resources.EntityResource.submit(EntityResource.java:96)\n\tat
> sun.reflect.GeneratedMethodAccessor32.invoke(Unknown Source)\n\tat
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)\n\tat
> java.lang.reflect.Method.invoke(Method.java:606)\n\tat
> com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60)\n\tat
>
> com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$ResponseOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:205)\n\tat
>
> com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75)\n\tat
>
> com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:288)\n\tat
>
> com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:108)\n\tat
>
> com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)\n\tat
>
> com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:84)\n\tat
>
> com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1469)\n\tat
>
> com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1400)\n\tat
>
> com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1349)\n\tat
>
> com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1339)\n\tat
>
> com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:409)\n\tat
>
> com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:558)\n\tat
>
> com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:733)\n\tat
> javax.servlet.http.HttpServlet.service(HttpServlet.java:790)\n\tat
> com.google.inject.servlet.ServletDefinition.doServiceImpl(ServletDefinition.java:287)\n\tat
>
> com.google.inject.servlet.ServletDefinition.doService(ServletDefinition.java:277)\n\tat
>
> com.google.inject.servlet.ServletDefinition.service(ServletDefinition.java:182)\n\tat
>
> com.google.inject.servlet.ManagedServletPipeline.service(ManagedServletPipeline.java:91)\n\tat
>
> com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:85)\n\tat
> org.apache.atlas.web.filters.AuditFilter.doFilter(AuditFilter.java:67)\n\tat
> com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82)\n\tat
>
> com.google.inject.servlet.ManagedFilterPipeline.dispatch(ManagedFilterPipeline.java:119)\n\tat
> com.google.inject.servlet.GuiceFilter$1.call(GuiceFilter.java:133)\n\tat
> com.google.inject.servlet.GuiceFilter$1.call(GuiceFilter.java:130)\n\tat
> com.google.inject.servlet.GuiceFilter$Context.call(GuiceFilter.java:203)\n\tat
> com.google.inject.servlet.GuiceFilter.doFilter(GuiceFilter.java:130)\n\tat
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)\n\tat
>
> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)\n\tat
>
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)\n\tat
>
> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:577)\n\tat
>
> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:223)\n\tat
>
> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)\n\tat
>
> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)\n\tat
>
> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)\n\tat
>
> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)\n\tat
>
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)\n\tat
>
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)\n\tat
> org.eclipse.jetty.server.Server.handle(Server.java:499)\n\tat
> org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310)\n\tat
> org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)\n\tat
>
> org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540)\n\tat
>
> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)\n\tat
>
> org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)\n\tat
> java.lang.Thread.run(Thread.java:745)\nCaused by:
> org.apache.atlas.typesystem.types.ValueConversionException$NullConversionException:
> Null value not allowed for multiplicty Multiplicity{lower=1, upper=1,
> isUnique=false}\n\tat
> org.apache.atlas.typesystem.types.DataTypes$PrimitiveType.convertNull(DataTypes.java:93)\n\tat
>
> org.apache.atlas.typesystem.types.DataTypes$StringType.convert(DataTypes.java:469)\n\tat
>
> org.apache.atlas.typesystem.types.DataTypes$StringType.convert(DataTypes.java:452)\n\tat
>
> org.apache.atlas.typesystem.types.DataTypes$MapType.convert(DataTypes.java:606)\n\tat
>
> org.apache.atlas.typesystem.types.DataTypes$MapType.convert(DataTypes.java:562)\n\tat
>
> org.apache.atlas.typesystem.persistence.StructInstance.set(StructInstance.java:118)\n\tat
>
> org.apache.atlas.typesystem.types.TypedStructHandler.convert(TypedStructHandler.java:65)\n\t...
> 55 more\n"})
> at org.apache.atlas.AtlasClient.callAPIWithResource(AtlasClient.java:365)
> at org.apache.atlas.AtlasClient.callAPI(AtlasClient.java:370)
> at org.apache.atlas.AtlasClient.createEntity(AtlasClient.java:210)
> at
> org.apache.atlas.hive.bridge.HiveMetaStoreBridge.createInstance(HiveMetaStoreBridge.java:132)
> at
> org.apache.atlas.hive.bridge.HiveMetaStoreBridge.fillStorageDescStruct(HiveMetaStoreBridge.java:474)
> at
> org.apache.atlas.hive.bridge.HiveMetaStoreBridge.registerTable(HiveMetaStoreBridge.java:302)
> at org.apache.atlas.hive.hook.HiveHook.handleCreateTable(HiveHook.java:271)
> at org.apache.atlas.hive.hook.HiveHook.fireAndForget(HiveHook.java:205)
> at org.apache.atlas.hive.hook.HiveHook.run(HiveHook.java:172)
> at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1585)
> at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1254)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1118)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1108)
> at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:216)
> at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:168)
> at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:379)
> at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:739)
> at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:684)
> at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:624)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
