Hi,
Hadoop 2.3 (Cloudera)
JDK: Oracle 1.8.0_31
I try to execute a java 8 application with twill v0.4.0, v0.4.1, v0.5.0 &
v0.6.0 but I get the following exception:
Exception in thread " STARTING" java.lang.IllegalArgumentException:
INVOKESPECIAL/STATIC on interfaces require ASM 5
at org.objectweb.asm.MethodVisitor.visitMethodInsn(Unknown Source)
at org.objectweb.asm.ClassReader.a(Unknown Source)
at org.objectweb.asm.ClassReader.b(Unknown Source)
at org.objectweb.asm.ClassReader.accept(Unknown Source)
at org.objectweb.asm.ClassReader.accept(Unknown Source)
at
org.apache.twill.internal.utils.Dependencies.findClassDependencies(Dependencies.java:102)
at
org.apache.twill.internal.ApplicationBundler.findDependencies(ApplicationBundler.java:179)
at
org.apache.twill.internal.ApplicationBundler.createBundle(ApplicationBundler.java:136)
at
org.apache.twill.yarn.YarnTwillPreparer.createContainerJar(YarnTwillPreparer.java:390)
at
org.apache.twill.yarn.YarnTwillPreparer.access$300(YarnTwillPreparer.java:106)
at
org.apache.twill.yarn.YarnTwillPreparer$1.call(YarnTwillPreparer.java:264)
at
org.apache.twill.yarn.YarnTwillPreparer$1.call(YarnTwillPreparer.java:253)
at
org.apache.twill.yarn.YarnTwillController.doStartUp(YarnTwillController.java:97)
at
org.apache.twill.internal.AbstractZKServiceController.startUp(AbstractZKServiceController.java:75)
at
org.apache.twill.internal.AbstractExecutionServiceController$ServiceDelegate.startUp(AbstractExecutionServiceController.java:109)
at
com.google.common.util.concurrent.AbstractIdleService$1$1.run(AbstractIdleService.java:43)
at java.lang.Thread.run(Thread.java:745)
I added a fix to org.apache.twill.internal.utils.Dependencies-class which
replaces ASM4 with ASM5, but the application fails to start. This error is
related to ASM5, because java 7 applications fail here too. If I revert to
ASM4 and start a java 7 application, everything works fine.
... [lots of output before here]
06:05:40.825 [ STARTING] DEBUG o.a.hadoop.ipc.ProtobufRpcEngine - Call:
getApplicationReport took 5ms
06:05:40.825 [ STARTING] DEBUG o.a.twill.yarn.YarnTwillController - Yarn
application status for Vertx application_1424646112016_0006: ACCEPTED
06:05:41.826 [IPC Parameter Sending Thread #0] DEBUG
org.apache.hadoop.ipc.Client - IPC Client (307720848) connection to /
0.0.0.0:8032 from cloudera sending #9
06:05:41.829 [IPC Client (307720848) connection to /0.0.0.0:8032 from
cloudera] DEBUG org.apache.hadoop.ipc.Client - IPC Client (307720848)
connection to /0.0.0.0:8032 from cloudera got value #9
06:05:41.829 [ STARTING] DEBUG o.a.hadoop.ipc.ProtobufRpcEngine - Call:
getApplicationReport took 3ms
06:05:41.829 [ STARTING] DEBUG o.a.twill.yarn.YarnTwillController - Yarn
application status for Vertx application_1424646112016_0006: ACCEPTED
06:05:42.033 [Kafka-Consumer-log-0] DEBUG
o.a.t.i.k.client.SimpleKafkaConsumer - No leader for topic partition
TopicPartition{topic=log, partition=0}.
06:05:42.037 [Kafka-Consumer-log-0] DEBUG
o.a.t.i.k.client.SimpleKafkaConsumer - No leader for topic partition
TopicPartition{topic=log, partition=0}.
06:05:42.829 [IPC Parameter Sending Thread #0] DEBUG
org.apache.hadoop.ipc.Client - IPC Client (307720848) connection to /
0.0.0.0:8032 from cloudera sending #10
06:05:42.831 [IPC Client (307720848) connection to /0.0.0.0:8032 from
cloudera] DEBUG org.apache.hadoop.ipc.Client - IPC Client (307720848)
connection to /0.0.0.0:8032 from cloudera got value #10
06:05:42.833 [ STARTING] DEBUG o.a.hadoop.ipc.ProtobufRpcEngine - Call:
getApplicationReport took 4ms
*06:05:42.833 [ STARTING] DEBUG o.a.twill.yarn.YarnTwillController - Yarn
application status for Vertx application_1424646112016_0006: FAILED*
06:05:43.834 [ STARTING] INFO o.a.twill.yarn.YarnTwillController - Yarn
application Vertx application_1424646112016_0006 is in state FAILED
06:05:43.834 [ STARTING] INFO o.a.twill.yarn.YarnTwillController - Yarn
application Vertx application_1424646112016_0006 is not in running state.
Shutting down controller.
06:05:43.846 [YarnTwillRunnerService
STARTING-SendThread(quickstart.cloudera:2181)] DEBUG
org.apache.zookeeper.ClientCnxn - Reading reply
sessionid:0x14bb38598e10020, packet::
clientPath:/Vertx/instances/2f141ba6-5ea9-4db3-a92c-7e0413e973ad
serverPath:/Vertx/instances/2f141ba6-5ea9-4db3-a92c-7e0413e973ad
finished:false header:: 12,3 replyHeader:: 12,9238,-101 request::
'/Vertx/instances/2f141ba6-5ea9-4db3-a92c-7e0413e973ad,T response::
06:05:43.863 [IPC Parameter Sending Thread #0] DEBUG
org.apache.hadoop.ipc.Client - IPC Client (307720848) connection to /
0.0.0.0:8032 from cloudera sending #11
06:05:43.867 [IPC Client (307720848) connection to /0.0.0.0:8032 from
cloudera] DEBUG org.apache.hadoop.ipc.Client - IPC Client (307720848)
connection to /0.0.0.0:8032 from cloudera got value #11
06:05:43.869 [ STOPPING] DEBUG o.a.hadoop.ipc.ProtobufRpcEngine - Call:
getApplicationReport took 10ms
06:05:43.877 [ STOPPING] DEBUG o.a.twill.yarn.YarnTwillController - Yarn
application Vertx application_1424646112016_0006 completed with status
FAILED
06:05:43.884 [ STOPPING] INFO o.a.t.i.k.client.SimpleKafkaConsumer -
Requesting stop of all consumer threads.
06:05:43.884 [ STOPPING] INFO o.a.t.i.k.client.SimpleKafkaConsumer -
Terminate requested Kafka-Consumer-log-0
06:05:43.891 [ STOPPING] INFO o.a.t.i.k.client.SimpleKafkaConsumer - Wait
for all consumer threads to stop.
06:05:43.899 [ STOPPING] INFO o.a.t.i.k.client.SimpleKafkaConsumer - All
consumer threads stopped.
06:05:43.906 [ZKKafkaClientService STOPPING] INFO
o.a.t.i.k.c.ZKKafkaClientService - Stopping KafkaClientService
06:05:43.911 [ZKKafkaClientService STOPPING] INFO
o.a.t.i.k.client.SimpleKafkaConsumer - Stopping Kafka consumer
06:05:43.917 [ZKKafkaClientService STOPPING] INFO
o.a.t.i.k.client.SimpleKafkaConsumer - Requesting stop of all consumer
threads.
06:05:43.917 [ZKKafkaClientService STOPPING] INFO
o.a.t.i.k.client.SimpleKafkaConsumer - Terminate requested
Kafka-Consumer-log-0
06:05:43.917 [ZKKafkaClientService STOPPING] INFO
o.a.t.i.k.client.SimpleKafkaConsumer - Wait for all consumer threads to
stop.
06:05:43.920 [ZKKafkaClientService STOPPING] INFO
o.a.t.i.k.client.SimpleKafkaConsumer - All consumer threads stopped.
06:05:43.920 [ZKKafkaClientService STOPPING] INFO
o.a.t.i.k.client.SimpleKafkaConsumer - Kafka Consumer stopped
06:05:43.929 [ZKKafkaClientService STOPPING] INFO
o.a.t.i.k.c.ZKKafkaClientService - KafkaClientService stopped
Any ideas how to execute java 8 applications with twill?
best regards
Oliver
