Suma Shivaprasad created ATLAS-1050:
---------------------------------------
Summary: Sqoop Hook does not package HDFS model jars which is
required
Key: ATLAS-1050
URL: https://issues.apache.org/jira/browse/ATLAS-1050
Project: Atlas
Issue Type: Bug
Reporter: Suma Shivaprasad
Assignee: Suma Shivaprasad
Sqoop hook should package hdfs model jar and when sqoop commands to import are
run, see this error since hdfs model jar is missing resulting in errors while
loading data.
{noformat}
2016-07-22 17:06:36,181 DEBUG [main]: ql.Driver
(DriverContext.java:shutdown(132)) - Shutting down query
LOAD DATA INPATH 'hdfs://localhost.localdomain:8020/user/hive/x_service' INTO
TABLE `x_service`
2016-07-22 17:06:36,181 INFO [main]: CliDriver
(SessionState.java:printInfo(939)) - Time taken: 4.791 seconds
2016-07-22 17:06:36,181 INFO [main]: log.PerfLogger
(PerfLogger.java:PerfLogBegin(148)) - <PERFLOG method=releaseLocks
from=org.apache.hadoop.hive.ql.Driver>
2016-07-22 17:06:36,181 INFO [main]: log.PerfLogger
(PerfLogger.java:PerfLogEnd(176)) - </PERFLOG method=releaseLocks
start=1469232396181 end=1469232396181 duration=0
from=org.apache.hadoop.hive.ql.Driver>
2016-07-22 17:06:36,182 DEBUG [main]: session.SessionState
(SessionState.java:close(1452)) - Removing resource dir
/tmp/f60f6488-691f-440d-8fa0-01e26873f8a5_resources
2016-07-22 17:06:36,191 DEBUG [Atlas Logger 0]:
classloader.AtlasPluginClassLoader (AtlasPluginClassLoader.java:loadClass(115))
- ==> AtlasPluginClassLoader.loadClass(org.apache.hadoop.fs.Path)
2016-07-22 17:06:36,192 DEBUG [Atlas Logger 0]:
classloader.AtlasPluginClassLoader (AtlasPluginClassLoader.java:loadClass(123))
- AtlasPluginClassLoader.loadClass(org.apache.hadoop.fs.Path): calling
pluginClassLoader.loadClass()
2016-07-22 17:06:36,193 DEBUG [Atlas Logger 0]:
classloader.AtlasPluginClassLoader (AtlasPluginClassLoader.java:findClass(79))
- ==> AtlasPluginClassLoader.findClass(org.apache.hadoop.fs.Path)
2016-07-22 17:06:36,193 DEBUG [Atlas Logger 0]:
classloader.AtlasPluginClassLoader (AtlasPluginClassLoader.java:findClass(87))
- AtlasPluginClassLoader.findClass(org.apache.hadoop.fs.Path): calling
pluginClassLoader.findClass()
2016-07-22 17:06:36,193 DEBUG [Atlas Logger 0]:
classloader.AtlasPluginClassLoader (AtlasPluginClassLoader.java:findClass(97))
- AtlasPluginClassLoader.findClass(org.apache.hadoop.fs.Path): calling
componentClassLoader.findClass()
2016-07-22 17:06:36,193 DEBUG [Atlas Logger 0]:
classloader.AtlasPluginClassLoader (AtlasPluginClassLoader.java:loadClass(133))
- AtlasPluginClassLoader.loadClass(org.apache.hadoop.fs.Path): calling
componentClassLoader.loadClass()
2016-07-22 17:06:36,193 DEBUG [Atlas Logger 0]:
classloader.AtlasPluginClassLoader (AtlasPluginClassLoader.java:loadClass(142))
- <== AtlasPluginClassLoader.loadClass(org.apache.hadoop.fs.Path): class
org.apache.hadoop.fs.Path
2016-07-22 17:06:36,193 DEBUG [Atlas Logger 0]: hook.HiveHook
(HiveHook.java:processHiveEntity(632)) - Registering DFS Path
hdfs://localhost.localdomain:8020/user/hive/x_service
2016-07-22 17:06:36,193 DEBUG [Atlas Logger 0]:
classloader.AtlasPluginClassLoader (AtlasPluginClassLoader.java:loadClass(115))
- ==> AtlasPluginClassLoader.loadClass(org.apache.atlas.fs.model.FSDataTypes)
2016-07-22 17:06:36,193 DEBUG [Atlas Logger 0]:
classloader.AtlasPluginClassLoader (AtlasPluginClassLoader.java:loadClass(123))
- AtlasPluginClassLoader.loadClass(org.apache.atlas.fs.model.FSDataTypes):
calling pluginClassLoader.loadClass()
2016-07-22 17:06:36,193 DEBUG [Atlas Logger 0]:
classloader.AtlasPluginClassLoader (AtlasPluginClassLoader.java:findClass(79))
- ==> AtlasPluginClassLoader.findClass(org.apache.atlas.fs.model.FSDataTypes)
2016-07-22 17:06:36,194 DEBUG [Atlas Logger 0]:
classloader.AtlasPluginClassLoader (AtlasPluginClassLoader.java:findClass(87))
- AtlasPluginClassLoader.findClass(org.apache.atlas.fs.model.FSDataTypes):
calling pluginClassLoader.findClass()
2016-07-22 17:06:36,194 DEBUG [Atlas Logger 0]:
classloader.AtlasPluginClassLoader (AtlasPluginClassLoader.java:findClass(97))
- AtlasPluginClassLoader.findClass(org.apache.atlas.fs.model.FSDataTypes):
calling componentClassLoader.findClass()
2016-07-22 17:06:36,194 DEBUG [Atlas Logger 0]:
classloader.AtlasPluginClassLoader (AtlasPluginClassLoader.java:loadClass(133))
- AtlasPluginClassLoader.loadClass(org.apache.atlas.fs.model.FSDataTypes):
calling componentClassLoader.loadClass()
2016-07-22 17:06:36,197 ERROR [Atlas Logger 0]: hook.HiveHook
(HiveHook.java:run(188)) - Atlas hook failed due to error
java.lang.NoClassDefFoundError: org/apache/atlas/fs/model/FSDataTypes
at
org.apache.atlas.hive.bridge.HiveMetaStoreBridge.fillHDFSDataSet(HiveMetaStoreBridge.java:556)
at
org.apache.atlas.hive.hook.HiveHook.processHiveEntity(HiveHook.java:634)
at
org.apache.atlas.hive.hook.HiveHook.registerProcess(HiveHook.java:599)
at org.apache.atlas.hive.hook.HiveHook.fireAndForget(HiveHook.java:227)
at org.apache.atlas.hive.hook.HiveHook.access$200(HiveHook.java:82)
at org.apache.atlas.hive.hook.HiveHook$2.run(HiveHook.java:186)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.ClassNotFoundException:
org.apache.atlas.fs.model.FSDataTypes
at java.lang.ClassLoader.findClass(ClassLoader.java:530)
at
org.apache.atlas.plugin.classloader.AtlasPluginClassLoader$MyClassLoader.findClass(AtlasPluginClassLoader.java:333)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at
org.apache.atlas.plugin.classloader.AtlasPluginClassLoader.loadClass(AtlasPluginClassLoader.java:137)
... 11 more
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
