I am running the hello world example and running into this error
/usr/java/jdk1.7.0_67-cloudera/bin/java -cp $CP
org.apache.twill.example.yarn.HelloWorld twill-demo-1.mycluster.com:2181
The YARN app runs fine but here is the error
23:11:53.715 [Kafka-Consumer-log-0] INFO
o.a.t.i.k.client.SimpleKafkaConsumer - Exception when fetching message on
TopicPartition{topic=log, partition=0}.
java.net.ConnectException: Connection refused
at sun.nio.ch.Net.connect0(Native Method) ~[na:1.7.0_67]
at sun.nio.ch.Net.connect(Net.java:465) ~[na:1.7.0_67]
at sun.nio.ch.Net.connect(Net.java:457) ~[na:1.7.0_67]
at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670)
~[na:1.7.0_67]
at kafka.network.BlockingChannel.connect(Unknown Source)
~[twill-examples-yarn-0.7.0-incubating-SNAPSHOT.jar:0.7.0-incubating-SNAPSHOT]
at kafka.consumer.SimpleConsumer.connect(Unknown Source)
~[twill-examples-yarn-0.7.0-incubating-SNAPSHOT.jar:0.7.0-incubating-SNAPSHOT]
at kafka.consumer.SimpleConsumer.reconnect(Unknown Source)
~[twill-examples-yarn-0.7.0-incubating-SNAPSHOT.jar:0.7.0-incubating-SNAPSHOT]
at kafka.consumer.SimpleConsumer.liftedTree1$1(Unknown Source)
~[twill-examples-yarn-0.7.0-incubating-SNAPSHOT.jar:0.7.0-incubating-SNAPSHOT]
at
kafka.consumer.SimpleConsumer.kafka$consumer$SimpleConsumer$$sendRequest(Unknown
Source)
~[twill-examples-yarn-0.7.0-incubating-SNAPSHOT.jar:0.7.0-incubating-SNAPSHOT]
at
kafka.consumer.SimpleConsumer$$anonfun$fetch$1$$anonfun$apply$mcV$sp$1.apply$mcV$sp(Unknown
Source)
~[twill-examples-yarn-0.7.0-incubating-SNAPSHOT.jar:0.7.0-incubating-SNAPSHOT]
at
kafka.consumer.SimpleConsumer$$anonfun$fetch$1$$anonfun$apply$mcV$sp$1.apply(Unknown
Source)
~[twill-examples-yarn-0.7.0-incubating-SNAPSHOT.jar:0.7.0-incubating-SNAPSHOT]
at
kafka.consumer.SimpleConsumer$$anonfun$fetch$1$$anonfun$apply$mcV$sp$1.apply(Unknown
Source)
~[twill-examples-yarn-0.7.0-incubating-SNAPSHOT.jar:0.7.0-incubating-SNAPSHOT]
at kafka.metrics.KafkaTimer.time(Unknown Source)
~[twill-examples-yarn-0.7.0-incubating-SNAPSHOT.jar:0.7.0-incubating-SNAPSHOT]
at kafka.consumer.SimpleConsumer$$anonfun$fetch$1.apply$mcV$sp(Unknown
Source)
~[twill-examples-yarn-0.7.0-incubating-SNAPSHOT.jar:0.7.0-incubating-SNAPSHOT]
at kafka.consumer.SimpleConsumer$$anonfun$fetch$1.apply(Unknown Source)
~[twill-examples-yarn-0.7.0-incubating-SNAPSHOT.jar:0.7.0-incubating-SNAPSHOT]
at kafka.consumer.SimpleConsumer$$anonfun$fetch$1.apply(Unknown Source)
~[twill-examples-yarn-0.7.0-incubating-SNAPSHOT.jar:0.7.0-incubating-SNAPSHOT]
at kafka.metrics.KafkaTimer.time(Unknown Source)
~[twill-examples-yarn-0.7.0-incubating-SNAPSHOT.jar:0.7.0-incubating-SNAPSHOT]
at kafka.consumer.SimpleConsumer.fetch(Unknown Source)
~[twill-examples-yarn-0.7.0-incubating-SNAPSHOT.jar:0.7.0-incubating-SNAPSHOT]
at kafka.javaapi.consumer.SimpleConsumer.fetch(Unknown Source)
~[twill-examples-yarn-0.7.0-incubating-SNAPSHOT.jar:0.7.0-incubating-SNAPSHOT]
at
org.apache.twill.internal.kafka.client.SimpleKafkaConsumer$ConsumerThread.fetchMessages(SimpleKafkaConsumer.java:419)
~[twill-examples-yarn-0.7.0-incubating-SNAPSHOT.jar:0.7.0-incubating-SNAPSHOT]
at
org.apache.twill.internal.kafka.client.SimpleKafkaConsumer$ConsumerThread.run(SimpleKafkaConsumer.java:355)
~[twill-examples-yarn-0.7.0-incubating-SNAPSHOT.jar:0.7.0-incubating-SNAPSHOT]
----------------------
To give more context, here is the meaningful part of the logs
2015-08-24T06:11:50,807Z INFO o.a.t.i.a.ApplicationMasterService [
twill-demo-2.mycluster.com] [ApplicationMasterService]
ApplicationMasterService:handleCompleted(ApplicationMasterService.java:449)
- Container container_1440313579459_0019_01_000002 completed with COMPLETE:.
2015-08-24T06:11:50,811Z INFO o.a.t.i.a.RunningContainers [
twill-demo-2.mycluster.com] [ApplicationMasterService]
RunningContainers:handleCompleted(RunningContainers.java:393) - Container
container_1440313579459_0019_01_000002 exited normally with state COMPLETE
2015-08-24T06:11:50,824Z INFO o.a.t.i.a.ApplicationMasterService [
twill-demo-2.mycluster.com] [ApplicationMasterService]
ApplicationMasterService:doRun(ApplicationMasterService.java:362) - All
containers completed. Shutting down application master.
2015-08-24T06:11:50,826Z INFO o.a.t.i.a.ApplicationMasterService [
twill-demo-2.mycluster.com] [ApplicationMasterService]
ApplicationMasterService:doStop(ApplicationMasterService.java:238) - Stop
application master with spec:
{"name":"HelloWorldRunnable","runnables":{"HelloWorldRunnable":{"name":"HelloWorldRunnable","runnable":{"classname":"org.apache.twill.example.yarn.HelloWorld$HelloWorldRunnable","name":"HelloWorldRunnable","arguments":{}},"resources":{"cores":1,"memorySize":512,"instances":1,"uplink":-1,"downlink":-1},"files":[]}},"orders":[{"names":["HelloWorldRunnable"],"type":"STARTED"}],"placementPolicies":[],"handler":{"classname":"org.apache.twill.internal.LogOnlyEventHandler","configs":{}}}
2015-08-24T06:11:50,829Z INFO o.a.t.i.a.RunningContainers [
twill-demo-2.mycluster.com] [ApplicationMasterService]
RunningContainers:stopAll(RunningContainers.java:332) - Stopping all
instances of HelloWorldRunnable
2015-08-24T06:11:50,829Z INFO o.a.t.i.a.RunningContainers [
twill-demo-2.mycluster.com] [ApplicationMasterService]
RunningContainers:stopAll(RunningContainers.java:342) - Terminated all
instances of HelloWorldRunnable
2015-08-24T06:11:50,848Z INFO o.a.t.i.a.ApplicationMasterService [
twill-demo-2.mycluster.com] [ApplicationMasterService]
ApplicationMasterService:cleanupDir(ApplicationMasterService.java:326) -
Application directory deleted: hdfs://
twill-demo-1.mycluster.com:8020/user/root/HelloWorldRunnable/526a193b-e9e0-40ce-99f9-a79b174d2871
2015-08-24T06:11:50,848Z INFO o.a.t.i.AbstractTwillService [
twill-demo-2.mycluster.com] [ApplicationMasterService]
AbstractTwillService:removeLiveNode(AbstractTwillService.java:209) - Remove
live node
twill-demo-1.mycluster.com:2181/HelloWorldRunnable/instances/526a193b-e9e0-40ce-99f9-a79b174d2871
2015-08-24T06:11:50,851Z INFO o.a.t.i.AbstractTwillService [
twill-demo-2.mycluster.com] [ApplicationMasterService]
AbstractTwillService:shutDown(AbstractTwillService.java:190) - Service
ApplicationMasterService with runId 526a193b-e9e0-40ce-99f9-a79b174d2871
shutdown completed
2015-08-24T06:11:50,852Z INFO o.a.t.i.ServiceMain [
twill-demo-2.mycluster.com] [main] ServiceMain:doMain(ServiceMain.java:100)
- Service ApplicationMasterService [TERMINATED] completed.
23:11:51.859 [IPC Parameter Sending Thread #0] DEBUG
org.apache.hadoop.ipc.Client - IPC Client (2014438912) connection to
twill-demo-1.mycluster.com/172.26.22.109:8032 from root sending #48
23:11:51.861 [IPC Client (2014438912) connection to
twill-demo-1.mycluster.com/172.26.22.109:8032from root] DEBUG
org.apache.hadoop.ipc.Client - IPC Client (2014438912) connection to
twill-demo-1.mycluster.com/172.26.22.109:8032 from root got value #48
23:11:51.861
[HelloWorldRunnable-application_1440313579459_0019-yarn-poller] DEBUG
o.a.hadoop.ipc.ProtobufRpcEngine - Call: getApplicationReport took 2ms
23:11:51.863 [ STARTING-SendThread(twill-demo-1.mycluster.com:2181)] DEBUG
org.apache.zookeeper.ClientCnxn - Reading reply
sessionid:0x14eb5f8db18012f, packet::
clientPath:/HelloWorldRunnable/instances/526a193b-e9e0-40ce-99f9-a79b174d2871
serverPath:/HelloWorldRunnable/instances/526a193b-e9e0-40ce-99f9-a79b174d2871
finished:false header:: 34,3 replyHeader:: 34,139514,-101 request::
'/HelloWorldRunnable/instances/526a193b-e9e0-40ce-99f9-a79b174d2871,F
response::
23:11:52.864 [IPC Parameter Sending Thread #0] DEBUG
org.apache.hadoop.ipc.Client - IPC Client (2014438912) connection to
twill-demo-1.mycluster.com/172.26.22.109:8032 from root sending #49
23:11:52.865 [IPC Client (2014438912) connection to
twill-demo-1.mycluster.com/172.26.22.109:8032from root] DEBUG
org.apache.hadoop.ipc.Client - IPC Client (2014438912) connection to
twill-demo-1.mycluster.com/172.26.22.109:8032 from root got value #49
23:11:52.866
[HelloWorldRunnable-application_1440313579459_0019-yarn-poller] DEBUG
o.a.hadoop.ipc.ProtobufRpcEngine - Call: getApplicationReport took 3ms
23:11:52.866 [ STARTING-SendThread(twill-demo-1.mycluster.com:2181)] DEBUG
org.apache.zookeeper.ClientCnxn - Got notification
sessionid:0x14eb5f8db18012f
23:11:52.866 [ STARTING-SendThread(twill-demo-1.mycluster.com:2181)] DEBUG
org.apache.zookeeper.ClientCnxn - Got WatchedEvent state:SyncConnected
type:NodeDeleted
path:/HelloWorldRunnable/526a193b-e9e0-40ce-99f9-a79b174d2871/kafka/brokers/ids/1
for sessionid 0x14eb5f8db18012f
23:11:52.868 [ STARTING-SendThread(twill-demo-1.mycluster.com:2181)] DEBUG
org.apache.zookeeper.ClientCnxn - Reading reply
sessionid:0x14eb5f8db18012f, packet::
clientPath:/HelloWorldRunnable/instances/526a193b-e9e0-40ce-99f9-a79b174d2871
serverPath:/HelloWorldRunnable/instances/526a193b-e9e0-40ce-99f9-a79b174d2871
finished:false header:: 35,3 replyHeader:: 35,139515,-101 request::
'/HelloWorldRunnable/instances/526a193b-e9e0-40ce-99f9-a79b174d2871,F
response::
23:11:53.180 [ STARTING-SendThread(twill-demo-1.mycluster.com:2181)] DEBUG
org.apache.zookeeper.ClientCnxn - Got notification
sessionid:0x14eb5f8db18012f
23:11:53.180 [ STARTING-SendThread(twill-demo-1.mycluster.com:2181)] DEBUG
org.apache.zookeeper.ClientCnxn - Got WatchedEvent state:SyncConnected
type:NodeDeleted
path:/HelloWorldRunnable/526a193b-e9e0-40ce-99f9-a79b174d2871/kafka/brokers/topics/log/partitions/0/state
for sessionid 0x14eb5f8db18012f
15/08/23 23:11:53 INFO consumer.SimpleConsumer: Reconnect due to socket
error: Connection reset by peer
23:11:53.715 [Kafka-Consumer-log-0] INFO
o.a.t.i.k.client.SimpleKafkaConsumer - Exception when fetching message on
TopicPartition{topic=log, partition=0}.
java.net.ConnectException: Connection refused
at sun.nio.ch.Net.connect0(Native Method) ~[na:1.7.0_67]
at sun.nio.ch.Net.connect(Net.java:465) ~[na:1.7.0_67]
at sun.nio.ch.Net.connect(Net.java:457) ~[na:1.7.0_67]
at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670)
~[na:1.7.0_67]
at kafka.network.BlockingChannel.connect(Unknown Source)
~[twill-examples-yarn-0.7.0-incubating-SNAPSHOT.jar:0.7.0-incubating-SNAPSHOT]
at kafka.consumer.SimpleConsumer.connect(Unknown Source)
~[twill-examples-yarn-0.7.0-incubating-SNAPSHOT.jar:0.7.0-incubating-SNAPSHOT]
at kafka.consumer.SimpleConsumer.reconnect(Unknown Source)
~[twill-examples-yarn-0.7.0-incubating-SNAPSHOT.jar:0.7.0-incubating-SNAPSHOT]
at kafka.consumer.SimpleConsumer.liftedTree1$1(Unknown Source)
~[twill-examples-yarn-0.7.0-incubating-SNAPSHOT.jar:0.7.0-incubating-SNAPSHOT]
at
kafka.consumer.SimpleConsumer.kafka$consumer$SimpleConsumer$$sendRequest(Unknown
Source)
~[twill-examples-yarn-0.7.0-incubating-SNAPSHOT.jar:0.7.0-incubating-SNAPSHOT]
at
kafka.consumer.SimpleConsumer$$anonfun$fetch$1$$anonfun$apply$mcV$sp$1.apply$mcV$sp(Unknown
Source)
~[twill-examples-yarn-0.7.0-incubating-SNAPSHOT.jar:0.7.0-incubating-SNAPSHOT]
at
kafka.consumer.SimpleConsumer$$anonfun$fetch$1$$anonfun$apply$mcV$sp$1.apply(Unknown
Source)
~[twill-examples-yarn-0.7.0-incubating-SNAPSHOT.jar:0.7.0-incubating-SNAPSHOT]
at
kafka.consumer.SimpleConsumer$$anonfun$fetch$1$$anonfun$apply$mcV$sp$1.apply(Unknown
Source)
~[twill-examples-yarn-0.7.0-incubating-SNAPSHOT.jar:0.7.0-incubating-SNAPSHOT]
at kafka.metrics.KafkaTimer.time(Unknown Source)
~[twill-examples-yarn-0.7.0-incubating-SNAPSHOT.jar:0.7.0-incubating-SNAPSHOT]
at kafka.consumer.SimpleConsumer$$anonfun$fetch$1.apply$mcV$sp(Unknown
Source)
~[twill-examples-yarn-0.7.0-incubating-SNAPSHOT.jar:0.7.0-incubating-SNAPSHOT]
at kafka.consumer.SimpleConsumer$$anonfun$fetch$1.apply(Unknown Source)
~[twill-examples-yarn-0.7.0-incubating-SNAPSHOT.jar:0.7.0-incubating-SNAPSHOT]
at kafka.consumer.SimpleConsumer$$anonfun$fetch$1.apply(Unknown Source)
~[twill-examples-yarn-0.7.0-incubating-SNAPSHOT.jar:0.7.0-incubating-SNAPSHOT]
at kafka.metrics.KafkaTimer.time(Unknown Source)
~[twill-examples-yarn-0.7.0-incubating-SNAPSHOT.jar:0.7.0-incubating-SNAPSHOT]
at kafka.consumer.SimpleConsumer.fetch(Unknown Source)
~[twill-examples-yarn-0.7.0-incubating-SNAPSHOT.jar:0.7.0-incubating-SNAPSHOT]
at kafka.javaapi.consumer.SimpleConsumer.fetch(Unknown Source)
~[twill-examples-yarn-0.7.0-incubating-SNAPSHOT.jar:0.7.0-incubating-SNAPSHOT]
at
org.apache.twill.internal.kafka.client.SimpleKafkaConsumer$ConsumerThread.fetchMessages(SimpleKafkaConsumer.java:419)
~[twill-examples-yarn-0.7.0-incubating-SNAPSHOT.jar:0.7.0-incubating-SNAPSHOT]
at
org.apache.twill.internal.kafka.client.SimpleKafkaConsumer$ConsumerThread.run(SimpleKafkaConsumer.java:355)
~[twill-examples-yarn-0.7.0-incubating-SNAPSHOT.jar:0.7.0-incubating-SNAPSHOT]
23:11:53.719 [ STARTING-SendThread(twill-demo-1.mycluster.com:2181)] DEBUG
org.apache.zookeeper.ClientCnxn - Reading reply
sessionid:0x14eb5f8db18012f, packet::
clientPath:/HelloWorldRunnable/526a193b-e9e0-40ce-99f9-a79b174d2871/kafka/brokers/ids/1
serverPath:/HelloWorldRunnable/526a193b-e9e0-40ce-99f9-a79b174d2871/kafka/brokers/ids/1
finished:false header:: 36,3 replyHeader:: 36,139540,-101 request::
'/HelloWorldRunnable/526a193b-e9e0-40ce-99f9-a79b174d2871/kafka/brokers/ids/1,T
response::
23:11:53.720 [Kafka-Consumer-log-0] DEBUG
o.a.t.i.k.client.SimpleKafkaConsumer - No leader for topic partition
TopicPartition{topic=log, partition=0}.
23:11:53.870 [IPC Parameter Sending Thread #0] DEBUG
org.apache.hadoop.ipc.Client - IPC Client (2014438912) connection to
twill-demo-1.mycluster.com/172.26.22.109:8032 from root sending #50
23:11:53.872 [IPC Client (2014438912) connection to
twill-demo-1.mycluster.com/172.26.22.109:8032from root] DEBUG
org.apache.hadoop.ipc.Client - IPC Client (2014438912) connection to
twill-demo-1.mycluster.com/172.26.22.109:8032 from root got value #50
23:11:53.872
[HelloWorldRunnable-application_1440313579459_0019-yarn-poller] DEBUG
o.a.hadoop.ipc.ProtobufRpcEngine - Call: getApplicationReport took 2ms
23:11:53.872
[HelloWorldRunnable-application_1440313579459_0019-yarn-poller] DEBUG
o.a.twill.yarn.YarnTwillController - Stop polling status from Yarn for
HelloWorldRunnable application_1440313579459_0019.
23:11:53.873
[HelloWorldRunnable-application_1440313579459_0019-yarn-poller] INFO
o.a.twill.yarn.YarnTwillController - Yarn application HelloWorldRunnable
application_1440313579459_0019 completed. Shutting down controller.
23:11:53.879 [IPC Parameter Sending Thread #0] DEBUG
org.apache.hadoop.ipc.Client - IPC Client (2014438912) connection to
twill-demo-1.mycluster.com/172.26.22.109:8032 from root sending #51
23:11:53.880 [IPC Client (2014438912) connection to
twill-demo-1.mycluster.com/172.26.22.109:8032from root] DEBUG
org.apache.hadoop.ipc.Client - IPC Client (2014438912) connection to
twill-demo-1.mycluster.com/172.26.22.109:8032 from root got value #51
23:11:53.880 [ STOPPING] DEBUG o.a.hadoop.ipc.ProtobufRpcEngine - Call:
getApplicationReport took 1ms
23:11:53.881 [ STOPPING] DEBUG o.a.twill.yarn.YarnTwillController - Yarn
application HelloWorldRunnable application_1440313579459_0019 completed
with status SUCCEEDED
23:11:53.881 [ STOPPING] INFO o.a.t.i.k.client.SimpleKafkaConsumer -
Requesting stop of all consumer threads.
23:11:53.881 [ STOPPING] INFO o.a.t.i.k.client.SimpleKafkaConsumer -
Terminate requested Kafka-Consumer-log-0
23:11:53.882 [ STOPPING] INFO o.a.t.i.k.client.SimpleKafkaConsumer - Wait
for all consumer threads to stop.
23:11:53.882 [ STOPPING] INFO o.a.t.i.k.client.SimpleKafkaConsumer - All
consumer threads stopped.
23:11:53.886 [ZKKafkaClientService STOPPING] INFO
o.a.t.i.k.c.ZKKafkaClientService - Stopping KafkaClientService
23:11:53.886 [ZKKafkaClientService STOPPING] INFO
o.a.t.i.k.client.SimpleKafkaConsumer - Stopping Kafka consumer
23:11:53.886 [ZKKafkaClientService STOPPING] INFO
o.a.t.i.k.client.SimpleKafkaConsumer - Kafka Consumer stopped
23:11:53.888 [ZKKafkaClientService STOPPING] INFO
o.a.t.i.k.c.ZKKafkaClientService - KafkaClientService stopped
23:11:53.960 [Thread-2] DEBUG org.apache.hadoop.ipc.Client - stopping
client from cache: org.apache.hadoop.ipc.Client@231556b
23:11:53.975 [zk-client-EventThread] DEBUG org.apache.zookeeper.ZooKeeper -
Closing session: 0x14eb5f8db18012f
23:11:53.975 [zk-client-EventThread] DEBUG org.apache.zookeeper.ClientCnxn
- Closing client for session: 0x14eb5f8db18012f
23:11:53.977 [ STARTING-SendThread(twill-demo-1.mycluster.com:2181)] DEBUG
org.apache.zookeeper.ClientCnxn - Reading reply
sessionid:0x14eb5f8db18012f, packet:: clientPath:null serverPath:null
finished:false header:: 37,-11 replyHeader:: 37,139541,0 request:: null
response:: null
23:11:53.977 [zk-client-EventThread] DEBUG org.apache.zookeeper.ClientCnxn
- Disconnecting client for session: 0x14eb5f8db18012f
23:11:53.977 [zk-client-EventThread] INFO org.apache.zookeeper.ZooKeeper -
Session: 0x14eb5f8db18012f closed
23:11:53.977 [ STARTING-SendThread(twill-demo-1.mycluster.com:2181)] DEBUG
org.apache.zookeeper.ClientCnxn - An exception was thrown while closing
send thread for session 0x14eb5f8db18012f : Unable to read additional data
from server sessionid 0x14eb5f8db18012f, likely server has closed socket
23:11:53.977 [ STARTING-EventThread] INFO org.apache.zookeeper.ClientCnxn
- EventThread shut down
23:11:53.978 [Hadoop21YarnAppClient STOPPING] DEBUG
o.a.hadoop.service.AbstractService - Service:
org.apache.hadoop.yarn.client.api.impl.YarnClientImpl entered state STOPPED
23:11:53.978 [Hadoop21YarnAppClient STOPPING] DEBUG
org.apache.hadoop.ipc.Client - stopping client from cache:
org.apache.hadoop.ipc.Client@231556b
23:11:53.978 [Hadoop21YarnAppClient STOPPING] DEBUG
org.apache.hadoop.ipc.Client - removing client from cache:
org.apache.hadoop.ipc.Client@231556b
23:11:53.978 [Hadoop21YarnAppClient STOPPING] DEBUG
org.apache.hadoop.ipc.Client - stopping actual client because no more
references remain: org.apache.hadoop.ipc.Client@231556b
23:11:53.978 [Hadoop21YarnAppClient STOPPING] DEBUG
org.apache.hadoop.ipc.Client - Stopping client
23:11:53.979 [IPC Client (2014438912) connection to
twill-demo-1.mycluster.com/172.26.22.109:8032from root] DEBUG
org.apache.hadoop.ipc.Client - IPC Client (2014438912) connection to
twill-demo-1.mycluster.com/172.26.22.109:8032 from root: closed
23:11:53.979 [IPC Client (2014438912) connection to
twill-demo-1.mycluster.com/172.26.22.109:8032from root] DEBUG
org.apache.hadoop.ipc.Client - IPC Client (2014438912) connection to
twill-demo-1.mycluster.com/172.26.22.109:8032 from root: stopped, remaining
connections 0
