Hitesh Both of things
On Sat, Jul 30, 2016 at 10:20 PM, Hitesh Shah <[email protected]> wrote: > Hello Sandeep, > > 2 things to check: > - When compiling Tez, is the hadoop.version in the top-level pom ( and > addition of mapr’s maven repo ) being used to compile against MapR’s hadoop > distribution and not the std. apache release? The Tez AM cannot seem to do > a handshake with the YARN RM. If MapR changed anything in that code path, > this issue could crop up. > - For tez.lib.uris, are you using the minimal tez tarball and not the > full one? The full one has hadoop jars in it so it should not be used in > your deployment setup as you are using MapR specific jars. You may want to > check for jar clashes in terms of the content of the tarball vs the jars > from the cluster. The only ones we do bundle in the minimal one are some > mapreduce jars I believe. > > FWIW, I think MapR does provide a Tez bundle for their clusters. You may > wish to check on their user lists too to see if they made some changes. If > you do find out anything, please do let us know in case this requires some > fixes in Tez for better compatibility across all distros. > > thanks > — Hitesh > > > > > On Jul 30, 2016, at 8:55 AM, Sandeep Khurana <[email protected]> > wrote: > > > > Hello > > > > > > On mapR single node cluster, I compiled Tez as per the documentation on > apache tez site. > > > > The only thing I changed from documentation was in tez-site.xml, where I > gave the value of tez.use.cluster.hadoop-libs as true. I did it because > mapR has its own jars for hadoop components. And without these tez might > not work. > > > > When I run any MR job or hive query I get this below error in the job on > RM link. (DIGEST-MD5: digest response format violation. Mismatched URI: > default/; expecting: null/default) > > > > I am using mapr 4.1 and tez 0.7.0. mapR 4.1 uses hive 1.2.0. > > > > Any pointer or suggestion what might be causing this issue or where > should I look? > > > > 2016-07-30 09:58:38,737 INFO [main] ipc.Server: Stopping server on 45868 > > 2016-07-30 09:58:38,737 INFO [IPC Server Responder] ipc.Server: Stopping > IPC Server Responder > > 2016-07-30 09:58:38,739 INFO [IPC Server listener on 45868] ipc.Server: > Stopping IPC Server listener on 45868 > > 2016-07-30 09:58:38,739 INFO [IPC Server Responder] ipc.Server: Stopping > IPC Server Responder > > 2016-07-30 09:58:38,739 ERROR [main] app.DAGAppMaster: Error starting > DAGAppMaster > > org.apache.tez.dag.api.TezUncheckedException: > javax.security.sasl.SaslException: DIGEST-MD5: digest response format > violation. Mismatched URI: default/; expecting: null/default [Caused by > org.apache.hadoop.ipc.RemoteException(javax.security.sasl.SaslException): > DIGEST-MD5: digest response format violation. Mismatched URI: default/; > expecting: null/default] > > at > org.apache.tez.dag.app.rm.YarnTaskSchedulerService.serviceStart(YarnTaskSchedulerService.java:384) > > at > org.apache.hadoop.service.AbstractService.start(AbstractService.java:193) > > at > org.apache.tez.dag.app.rm.TaskSchedulerEventHandler.serviceStart(TaskSchedulerEventHandler.java:353) > > at > org.apache.hadoop.service.AbstractService.start(AbstractService.java:193) > > at > org.apache.tez.dag.app.DAGAppMaster$ServiceWithDependency.start(DAGAppMaster.java:1573) > > at > org.apache.tez.dag.app.DAGAppMaster$ServiceThread.run(DAGAppMaster.java:1591) > > Caused by: javax.security.sasl.SaslException: DIGEST-MD5: digest > response format violation. Mismatched URI: default/; expecting: > null/default [Caused by > org.apache.hadoop.ipc.RemoteException(javax.security.sasl.SaslException): > DIGEST-MD5: digest response format violation. Mismatched URI: default/; > expecting: null/default] > > at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native > Method) > > at > sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57) > > at > sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) > > at java.lang.reflect.Constructor.newInstance(Constructor.java:526) > > at > org.apache.hadoop.yarn.ipc.RPCUtil.instantiateException(RPCUtil.java:53) > > at > org.apache.hadoop.yarn.ipc.RPCUtil.unwrapAndThrowException(RPCUtil.java:104) > > at > org.apache.hadoop.yarn.api.impl.pb.client.ApplicationMasterProtocolPBClientImpl.registerApplicationMaster(ApplicationMasterProtocolPBClientImpl.java:109) > > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) > > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > > at java.lang.reflect.Method.invoke(Method.java:606) > > at > org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:187) > > at > org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:102) > > at com.sun.proxy.$Proxy24.registerApplicationMaster(Unknown Source) > > at > org.apache.hadoop.yarn.client.api.impl.AMRMClientImpl.registerApplicationMaster(AMRMClientImpl.java:222) > > at > org.apache.hadoop.yarn.client.api.impl.AMRMClientImpl.registerApplicationMaster(AMRMClientImpl.java:214) > > at > org.apache.hadoop.yarn.client.api.async.impl.AMRMClientAsyncImpl.registerApplicationMaster(AMRMClientAsyncImpl.java:138) > > at > org.apache.tez.dag.app.rm.YarnTaskSchedulerService.serviceStart(YarnTaskSchedulerService.java:368) > > ... 5 more > > Caused by: > org.apache.hadoop.ipc.RemoteException(javax.security.sasl.SaslException): > DIGEST-MD5: digest response format violation. Mismatched URI: default/; > expecting: null/default > > at org.apache.hadoop.ipc.Client.call(Client.java:1468) > > at org.apache.hadoop.ipc.Client.call(Client.java:1399) > > at > org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:232) > > at com.sun.proxy.$Proxy23.registerApplicationMaster(Unknown Source) > > at > org.apache.hadoop.yarn.api.impl.pb.client.ApplicationMasterProtocolPBClientImpl.registerApplicationMaster(ApplicationMasterProtocolPBClientImpl.java:106) > > ... 16 more > > 2016-07-30 09:58:38,741 INFO [Thread-2] app.DAGAppMaster: > DAGAppMasterShutdownHook invoked > > > > > > -- > > Thanks and regards > > Sandeep Khurana > > -- Thanks and regards Sandeep Khurana
