Sorry, by mistake sent half written message. I will try both now and update this thread.
About trying tez bundle, I tried with the steps at http://doc.mapr.com/display/MapR41/Installing+and+Configuring+Tez+0.5.3 .But hive job gave some NumberFormatError and found out by googling that there is version mismatch between tez and hadoop libs. On Sat, Jul 30, 2016 at 10:22 PM, Sandeep Khurana <[email protected]> wrote: > Hitesh > > Both of things > > On Sat, Jul 30, 2016 at 10:20 PM, Hitesh Shah <[email protected]> wrote: > >> Hello Sandeep, >> >> 2 things to check: >> - When compiling Tez, is the hadoop.version in the top-level pom ( and >> addition of mapr’s maven repo ) being used to compile against MapR’s hadoop >> distribution and not the std. apache release? The Tez AM cannot seem to do >> a handshake with the YARN RM. If MapR changed anything in that code path, >> this issue could crop up. >> - For tez.lib.uris, are you using the minimal tez tarball and not the >> full one? The full one has hadoop jars in it so it should not be used in >> your deployment setup as you are using MapR specific jars. You may want to >> check for jar clashes in terms of the content of the tarball vs the jars >> from the cluster. The only ones we do bundle in the minimal one are some >> mapreduce jars I believe. >> >> FWIW, I think MapR does provide a Tez bundle for their clusters. You may >> wish to check on their user lists too to see if they made some changes. If >> you do find out anything, please do let us know in case this requires some >> fixes in Tez for better compatibility across all distros. >> >> thanks >> — Hitesh >> >> >> >> > On Jul 30, 2016, at 8:55 AM, Sandeep Khurana <[email protected]> >> wrote: >> > >> > Hello >> > >> > >> > On mapR single node cluster, I compiled Tez as per the documentation on >> apache tez site. >> > >> > The only thing I changed from documentation was in tez-site.xml, where >> I gave the value of tez.use.cluster.hadoop-libs as true. I did it because >> mapR has its own jars for hadoop components. And without these tez might >> not work. >> > >> > When I run any MR job or hive query I get this below error in the job >> on RM link. (DIGEST-MD5: digest response format violation. Mismatched URI: >> default/; expecting: null/default) >> > >> > I am using mapr 4.1 and tez 0.7.0. mapR 4.1 uses hive 1.2.0. >> > >> > Any pointer or suggestion what might be causing this issue or where >> should I look? >> > >> > 2016-07-30 09:58:38,737 INFO [main] ipc.Server: Stopping server on 45868 >> > 2016-07-30 09:58:38,737 INFO [IPC Server Responder] ipc.Server: >> Stopping IPC Server Responder >> > 2016-07-30 09:58:38,739 INFO [IPC Server listener on 45868] ipc.Server: >> Stopping IPC Server listener on 45868 >> > 2016-07-30 09:58:38,739 INFO [IPC Server Responder] ipc.Server: >> Stopping IPC Server Responder >> > 2016-07-30 09:58:38,739 ERROR [main] app.DAGAppMaster: Error starting >> DAGAppMaster >> > org.apache.tez.dag.api.TezUncheckedException: >> javax.security.sasl.SaslException: DIGEST-MD5: digest response format >> violation. Mismatched URI: default/; expecting: null/default [Caused by >> org.apache.hadoop.ipc.RemoteException(javax.security.sasl.SaslException): >> DIGEST-MD5: digest response format violation. Mismatched URI: default/; >> expecting: null/default] >> > at >> org.apache.tez.dag.app.rm.YarnTaskSchedulerService.serviceStart(YarnTaskSchedulerService.java:384) >> > at >> org.apache.hadoop.service.AbstractService.start(AbstractService.java:193) >> > at >> org.apache.tez.dag.app.rm.TaskSchedulerEventHandler.serviceStart(TaskSchedulerEventHandler.java:353) >> > at >> org.apache.hadoop.service.AbstractService.start(AbstractService.java:193) >> > at >> org.apache.tez.dag.app.DAGAppMaster$ServiceWithDependency.start(DAGAppMaster.java:1573) >> > at >> org.apache.tez.dag.app.DAGAppMaster$ServiceThread.run(DAGAppMaster.java:1591) >> > Caused by: javax.security.sasl.SaslException: DIGEST-MD5: digest >> response format violation. Mismatched URI: default/; expecting: >> null/default [Caused by >> org.apache.hadoop.ipc.RemoteException(javax.security.sasl.SaslException): >> DIGEST-MD5: digest response format violation. Mismatched URI: default/; >> expecting: null/default] >> > at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native >> Method) >> > at >> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57) >> > at >> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) >> > at java.lang.reflect.Constructor.newInstance(Constructor.java:526) >> > at >> org.apache.hadoop.yarn.ipc.RPCUtil.instantiateException(RPCUtil.java:53) >> > at >> org.apache.hadoop.yarn.ipc.RPCUtil.unwrapAndThrowException(RPCUtil.java:104) >> > at >> org.apache.hadoop.yarn.api.impl.pb.client.ApplicationMasterProtocolPBClientImpl.registerApplicationMaster(ApplicationMasterProtocolPBClientImpl.java:109) >> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >> > at >> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) >> > at >> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) >> > at java.lang.reflect.Method.invoke(Method.java:606) >> > at >> org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:187) >> > at >> org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:102) >> > at com.sun.proxy.$Proxy24.registerApplicationMaster(Unknown >> Source) >> > at >> org.apache.hadoop.yarn.client.api.impl.AMRMClientImpl.registerApplicationMaster(AMRMClientImpl.java:222) >> > at >> org.apache.hadoop.yarn.client.api.impl.AMRMClientImpl.registerApplicationMaster(AMRMClientImpl.java:214) >> > at >> org.apache.hadoop.yarn.client.api.async.impl.AMRMClientAsyncImpl.registerApplicationMaster(AMRMClientAsyncImpl.java:138) >> > at >> org.apache.tez.dag.app.rm.YarnTaskSchedulerService.serviceStart(YarnTaskSchedulerService.java:368) >> > ... 5 more >> > Caused by: >> org.apache.hadoop.ipc.RemoteException(javax.security.sasl.SaslException): >> DIGEST-MD5: digest response format violation. Mismatched URI: default/; >> expecting: null/default >> > at org.apache.hadoop.ipc.Client.call(Client.java:1468) >> > at org.apache.hadoop.ipc.Client.call(Client.java:1399) >> > at >> org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:232) >> > at com.sun.proxy.$Proxy23.registerApplicationMaster(Unknown >> Source) >> > at >> org.apache.hadoop.yarn.api.impl.pb.client.ApplicationMasterProtocolPBClientImpl.registerApplicationMaster(ApplicationMasterProtocolPBClientImpl.java:106) >> > ... 16 more >> > 2016-07-30 09:58:38,741 INFO [Thread-2] app.DAGAppMaster: >> DAGAppMasterShutdownHook invoked >> > >> > >> > -- >> > Thanks and regards >> > Sandeep Khurana >> >> > > > -- > Thanks and regards > Sandeep Khurana > -- Thanks and regards Sandeep Khurana
