[
https://issues.apache.org/jira/browse/TEZ-982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13949764#comment-13949764
]
Arpit Gupta commented on TEZ-982:
---------------------------------
Here is a snippet from the am log.
{code}
2014-03-27 09:47:10,428 INFO [InputInitializer [Map 1] #0]
org.apache.hadoop.hive.ql.log.PerfLogger: </PERFLOG method=deserializePlan
start=1395913630091 end=1395913630428 duration=337
from=org.apache.hadoop.hive.ql.exec.Utilities>
2014-03-27 09:47:10,429 INFO [InputInitializer [Map 1] #0]
org.apache.hadoop.hive.ql.exec.Utilities: Processing alias store
2014-03-27 09:47:10,429 INFO [InputInitializer [Map 1] #0]
org.apache.hadoop.hive.ql.exec.Utilities: Adding input file
hdfs://ha-2-secure-2/user/hrt_qa/hive-ha/test_hive_job/store
2014-03-27 09:47:10,432 INFO [InputInitializer [Map 1] #0]
org.apache.hadoop.hive.ql.io.HiveInputFormat: Generating splits
2014-03-27 09:47:10,433 ERROR [InputInitializer [Map 5] #0]
com.hadoop.compression.lzo.GPLNativeCodeLoader: Could not load native gpl
library
java.lang.UnsatisfiedLinkError: no gplcompression in java.library.path
at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1738)
at java.lang.Runtime.loadLibrary0(Runtime.java:823)
at java.lang.System.loadLibrary(System.java:1028)
at
com.hadoop.compression.lzo.GPLNativeCodeLoader.<clinit>(GPLNativeCodeLoader.java:32)
at com.hadoop.compression.lzo.LzoCodec.<clinit>(LzoCodec.java:71)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:247)
at
org.apache.hadoop.conf.Configuration.getClassByNameOrNull(Configuration.java:1834)
at
org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:1799)
at
org.apache.hadoop.io.compress.CompressionCodecFactory.getCodecClasses(CompressionCodecFactory.java:128)
at
org.apache.hadoop.io.compress.CompressionCodecFactory.<init>(CompressionCodecFactory.java:175)
at
org.apache.hadoop.mapred.TextInputFormat.configure(TextInputFormat.java:45)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:106)
at
org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:75)
at
org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:133)
at
org.apache.hadoop.hive.ql.io.HiveInputFormat.getInputFormatFromCache(HiveInputFormat.java:197)
at
org.apache.hadoop.hive.ql.io.HiveInputFormat.getSplits(HiveInputFormat.java:372)
at
org.apache.hadoop.mapred.split.TezGroupedSplitsInputFormat.getSplits(TezGroupedSplitsInputFormat.java:67)
at
org.apache.tez.mapreduce.hadoop.MRHelpers.generateOldSplits(MRHelpers.java:261)
at
org.apache.tez.mapreduce.common.MRInputAMSplitGenerator.initialize(MRInputAMSplitGenerator.java:139)
at
org.apache.tez.dag.app.dag.RootInputInitializerRunner$InputInitializerCallable$1.run(RootInputInitializerRunner.java:145)
at
org.apache.tez.dag.app.dag.RootInputInitializerRunner$InputInitializerCallable$1.run(RootInputInitializerRunner.java:138)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1548)
at
org.apache.tez.dag.app.dag.RootInputInitializerRunner$InputInitializerCallable.call(RootInputInitializerRunner.java:138)
at
org.apache.tez.dag.app.dag.RootInputInitializerRunner$InputInitializerCallable.call(RootInputInitializerRunner.java:109)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
2014-03-27 09:47:10,437 ERROR [InputInitializer [Map 5] #0]
com.hadoop.compression.lzo.LzoCodec: Cannot load native-lzo without
native-hadoop
2014-03-27 09:47:10,441 WARN [InputInitializer [Map 9] #0]
org.apache.hadoop.hdfs.DFSClient: dfs.client.test.drop.namenode.response.number
is set to 1, this hacked client will proactively drop responses
2014-03-27 09:47:10,441 WARN [InputInitializer [Map 5] #0]
org.apache.hadoop.hdfs.DFSClient: dfs.client.test.drop.namenode.response.number
is set to 1, this hacked client will proactively drop responses
2014-03-27 09:47:10,441 WARN [InputInitializer [Map 1] #0]
org.apache.hadoop.hdfs.DFSClient: dfs.client.test.drop.namenode.response.number
is set to 1, this hacked client will proactively drop responses
2014-03-27 09:47:10,444 INFO [InputInitializer [Map 11] #0]
org.apache.hadoop.hive.ql.log.PerfLogger: </PERFLOG method=deserializePlan
start=1395913630091 end=1395913630444 duration=353
from=org.apache.hadoop.hive.ql.exec.Utilities>
2014-03-27 09:47:10,444 INFO [InputInitializer [Map 11] #0]
org.apache.hadoop.hive.ql.exec.Utilities: Processing alias date_dim
2014-03-27 09:47:10,444 INFO [InputInitializer [Map 11] #0]
org.apache.hadoop.hive.ql.exec.Utilities: Adding input file
hdfs://ha-2-secure-2/user/hrt_qa/hive-ha/test_hive_job/date_dim
2014-03-27 09:47:10,445 INFO [InputInitializer [Map 11] #0]
org.apache.hadoop.hive.ql.io.HiveInputFormat: Generating splits
2014-03-27 09:47:10,446 WARN [InputInitializer [Map 11] #0]
org.apache.hadoop.hdfs.DFSCli
{code}
> DAG succeeds even when there is an error in the input initialized
> -----------------------------------------------------------------
>
> Key: TEZ-982
> URL: https://issues.apache.org/jira/browse/TEZ-982
> Project: Apache Tez
> Issue Type: Bug
> Affects Versions: 0.4.0
> Reporter: Arpit Gupta
> Priority: Critical
>
> Noticed in one of the applications running we saw an exception while
> generating input splits.
> The application succeeded even with this exception.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
