[ https://issues.apache.org/jira/browse/TEZ-494?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14541312#comment-14541312 ]
Richard Huang commented on TEZ-494: ----------------------------------- [~hiteshahuja]I had this issue when we have a lot of files to process (let's say around 2400 files, each is about 700M). when I pick up less partitions to process, it goes fine. I use pig on tez, here is the configuration: MapR 2.4.1 hadoop pig 0.14.0 hive 0.13 tez 0.5.3 (build with the mapr lib) The job runs fine with the regular MR job though. here is the tez output: 2015-05-13 04:40:02,999 [PigTezLauncher-0] ERROR org.apache.pig.backend.hadoop.executionengine.tez.TezJob - Cannot submit DAG - Application id: application_1430152270521_27554 org.apache.tez.dag.api.TezException: com.google.protobuf.ServiceException: java.io.IOException: Failed on local exception: java.io.EOFException; Host Details : local host is: "xxxxxxxxxx"; destination host is: "xxxxxxxxx":51956; at org.apache.tez.client.TezClient.submitDAGSession(TezClient.java:415) at org.apache.tez.client.TezClient.submitDAG(TezClient.java:351) at org.apache.pig.backend.hadoop.executionengine.tez.TezJob.run(TezJob.java:162) at org.apache.pig.backend.hadoop.executionengine.tez.TezLauncher$1.run(TezLauncher.java:167) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) Caused by: com.google.protobuf.ServiceException: java.io.IOException: Failed on local exception: java.io.EOFException; Host Details : local host is: "xxxxxxxx"; destination host is: "xxxxxxxxx":51956; at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:216) at com.sun.proxy.$Proxy13.submitDAG(Unknown Source) at org.apache.tez.client.TezClient.submitDAGSession(TezClient.java:408) ... 8 more Caused by: java.io.IOException: Failed on local exception: java.io.EOFException; Host Details : local host is: "xxxxxx"; destination host is: "xxxxxxxx":51956; at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:764) at org.apache.hadoop.ipc.Client.call(Client.java:1416) at org.apache.hadoop.ipc.Client.call(Client.java:1365) at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:206) ... 10 more Caused by: java.io.EOFException at java.io.DataInputStream.readInt(DataInputStream.java:392) at org.apache.hadoop.ipc.Client$Connection.receiveRpcResponse(Client.java:1056) at org.apache.hadoop.ipc.Client$Connection.run(Client.java:951) 2015-05-13 04:40:03,528 [main] INFO org.apache.pig.tools.pigstats.tez.TezPigScriptStats - Script Statistics: HadoopVersion: 2.4.1-mapr-1408 PigVersion: 0.14.0-mapr-1502 TezVersion: 0.5.3 UserId: prod FileName: xxxx.pig StartedAt: 2015-05-13 04:35:25 FinishedAt: 2015-05-13 04:40:03 Features: GROUP_BY,FILTER,UNION > Investigate: Exception while submitting a dag to a TezSession > ------------------------------------------------------------- > > Key: TEZ-494 > URL: https://issues.apache.org/jira/browse/TEZ-494 > Project: Apache Tez > Issue Type: Bug > Reporter: Siddharth Seth > Assignee: Rekha Joshi > > {code} > destination host is: "local":54446; > at org.apache.tez.client.TezSession.submitDAG(TezSession.java:139) > at > org.apache.tez.mapreduce.examples.TestBroadcast.main(TestBroadcast.java:155) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > at java.lang.reflect.Method.invoke(Method.java:597) > at > org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:72) > at org.apache.hadoop.util.ProgramDriver.run(ProgramDriver.java:144) > at > org.apache.tez.mapreduce.examples.ExampleDriver.main(ExampleDriver.java:78) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > at java.lang.reflect.Method.invoke(Method.java:597) > at org.apache.hadoop.util.RunJar.main(RunJar.java:212) > Caused by: com.google.protobuf.ServiceException: java.io.IOException: Failed > on local exception: java.io.EOFException; Host Details : local host is: > "local/10.11.2.219"; destination host is: "local":54446; > at > org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:216) > at $Proxy12.submitDAG(Unknown Source) > at org.apache.tez.client.TezSession.submitDAG(TezSession.java:137) > ... 13 more > Caused by: java.io.IOException: Failed on local exception: > java.io.EOFException; Host Details : local host is: "local/10.11.2.219"; > destination host is: "local":54446; > at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:764) > at org.apache.hadoop.ipc.Client.call(Client.java:1408) > at org.apache.hadoop.ipc.Client.call(Client.java:1357) > at > org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:206) > ... 15 more > Caused by: java.io.EOFException > at java.io.DataInputStream.readInt(DataInputStream.java:375) > at > org.apache.hadoop.ipc.Client$Connection.receiveRpcResponse(Client.java:1049) > at org.apache.hadoop.ipc.Client$Connection.run(Client.java:944) > {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)