Hi, Nelson, We build and test Samza against YARN-2.5. There should not be an incompatibility issue here. From your logs, it seems that it is a security exception. Could you let us know your YARN site configuration? Is there any security mechanism configured in your YARN cluster that requires the AppMaster to register w/ some security credentials? We could try to re-produce the issue if we know your configuration better.
Thanks! -Yi On Wed, Jul 1, 2015 at 6:33 AM, Nelson VERDIER <nelson.verd...@ysance.com> wrote: > Hi, > > Being curious about samza i've decided to play with it a little. Having > successfully run hello samza plus some custom StreamTasks of my own on my > local computer i decided to reproduce the experience on a in-house cluster > we have. > > The cluster is a MapR 4.1.0 running yarn 2.5.1. The problem we face is when > yarn is trying to register samza app master. > > Below are container logs: > > Exception in thread "main" javax.security.sasl.SaslException: > DIGEST-MD5: digest response format violation. Mismatched URI: > default/; expecting: null/default [Caused by > org.apache.hadoop.ipc.RemoteException(javax.security.sasl.SaslException): > DIGEST-MD5: digest response format violation. Mismatched URI: > default/; expecting: null/default] > at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native > Method) > at > sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57) > at > sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) > at java.lang.reflect.Constructor.newInstance(Constructor.java:526) > at > org.apache.hadoop.yarn.ipc.RPCUtil.instantiateException(RPCUtil.java:53) > at > org.apache.hadoop.yarn.ipc.RPCUtil.unwrapAndThrowException(RPCUtil.java:104) > at > org.apache.hadoop.yarn.api.impl.pb.client.ApplicationMasterProtocolPBClientImpl.registerApplicationMaster(ApplicationMasterProtocolPBClientImpl.java:109) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:606) > at > org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:190) > at > org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:103) > at com.sun.proxy.$Proxy17.registerApplicationMaster(Unknown Source) > at > org.apache.hadoop.yarn.client.api.impl.AMRMClientImpl.registerApplicationMaster(AMRMClientImpl.java:196) > at > org.apache.hadoop.yarn.client.api.async.impl.AMRMClientAsyncImpl.registerApplicationMaster(AMRMClientAsyncImpl.java:138) > at > org.apache.samza.job.yarn.SamzaAppMasterLifecycle.onInit(SamzaAppMasterLifecycle.scala:39) > at > org.apache.samza.job.yarn.SamzaAppMaster$$anonfun$run$1.apply(SamzaAppMaster.scala:108) > at > org.apache.samza.job.yarn.SamzaAppMaster$$anonfun$run$1.apply(SamzaAppMaster.scala:108) > at scala.collection.immutable.List.foreach(List.scala:318) > at > org.apache.samza.job.yarn.SamzaAppMaster$.run(SamzaAppMaster.scala:108) > at > org.apache.samza.job.yarn.SamzaAppMaster$.main(SamzaAppMaster.scala:95) > at > org.apache.samza.job.yarn.SamzaAppMaster.main(SamzaAppMaster.scala) > > And RM logs: > > 2015-07-01 14:13:28,551 INFO > > org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptImpl: > appattempt_1435679592284_0004_000001 State change from ALLOCATED to > LAUNCHED > 2015-07-01 14:13:29,519 INFO > org.apache.hadoop.yarn.server.resourcemanager.rmcontainer.RMContainerImpl: > container_1435679592284_0004_01_000001 Container Transitioned from > ACQUIRED to RUNNING > 2015-07-01 14:13:34,899 WARN > SecurityLogger.org.apache.hadoop.ipc.Server: Auth failed for > 10.19.1.194:44153:null (DIGEST-MD5: digest response format violation. > Mismatched URI: default/; expecting: null/default) > 2015-07-01 14:13:34,900 INFO org.apache.hadoop.ipc.Server: Socket > Reader #1 for port 8030: readAndProcess from client 10.19.1.194 threw > exception [javax.security.sasl.SaslException: DIGEST-MD5: digest > response format violation. Mismatched URI: default/ > ; expecting: null/default] > 2015-07-01 14:13:35,763 WARN > SecurityLogger.org.apache.hadoop.ipc.Server: Auth failed for > 10.19.1.194:44154:null (DIGEST-MD5: digest response format violation. > Mismatched URI: default/; expecting: null/default) > 2015-07-01 14:13:35,764 INFO org.apache.hadoop.ipc.Server: Socket > Reader #1 for port 8030: readAndProcess from client 10.19.1.194 threw > exception [javax.security.sasl.SaslException: DIGEST-MD5: digest > response format violation. Mismatched URI: default/ > ; expecting: null/default] > > I am desperate to understand why the samzaAppMaster can't be launched > properly. Is Yarn 2.5 compatible with samza 0.9. Do you have any idea why > we have this digest-md5 issue? > > Regards, > > -- > *Nelson Verdier* >
