My best guess would be that an older version of commons-codec is also on the classpath for the running task. If you have access to the local-dirs configured under YARN - you could find the application dir in the local-dirs and see what exists in the classpath for a container.
Alternately, set tez.generate.debug.artifacts to true. This should give you access to the dag plan in text form via the YARN UI - which will list out the resources being localized for tasks. On Tue, May 19, 2015 at 2:19 AM, patcharee <patcharee.thong...@uni.no> wrote: > Hi, > > I am using Pig version 0.14 with tez version 0.5.2. I have apache > commons-codec-1.4.jar on the machine. However, I got the common codec > exception when I executed a Pig job with tez. > > 2015-05-19 11:01:04,784 INFO [AsyncDispatcher event handler] > history.HistoryEventHandler: > [HISTORY][DAG:dag_1431972385685_0021_1][Event:TASK_ATTEMPT_FINISHED]: > vertexName=scope-384, > taskAttemptId=attempt_1431972385685_0021_1_00_000004_0, > startTime=1432025450752, finishTime=1432026064784, timeTaken=614032, > status=FAILED, diagnostics=Error: Fatal Error cause TezChild > exit.:java.lang.NoSuchMethodError: > org.apache.commons.codec.binary.Base64.decodeBase64(Ljava/lang/String;)[B > at > org.apache.hadoop.yarn.util.AuxiliaryServiceHelper.getServiceDataFromEnv(AuxiliaryServiceHelper.java:37) > at > org.apache.tez.runtime.api.impl.TezTaskContextImpl.getServiceProviderMetaData(TezTaskContextImpl.java:175) > at > org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput.generateEventsOnClose(OrderedPartitionedKVOutput.java:187) > at > org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput.close(OrderedPartitionedKVOutput.java:148) > at > org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.close(LogicalIOProcessorRuntimeTask.java:348) > at > org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:178) > at > org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:415) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628) > at > org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168) > at > org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163) > at java.util.concurrent.FutureTask.run(FutureTask.java:262) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at java.lang.Thread.run(Thread.java:744) > > What could be wrong? > > BR, > Patcharee > > > >
