On computer 1:
./s4 zkServer &
02:38:31.650 [main] INFO org.apache.s4.tools.ZKServer - Starting zookeeper
server on port [2181]
02:38:31.655 [main] INFO org.apache.s4.tools.ZKServer - cleaning existing data
in [/tmp/tmp/zookeeper/data] and [/tmp/tmp/zookeeper/log]
02:38:31.925 [main] INFO org.apache.s4.tools.ZKServer - Zookeeper server
started on port [2181]
./s4 newCluster -c=cluster1 -nbTasks=1 -flp=12000
02:38:45.357 [main] INFO org.apache.s4.tools.DefineCluster - preparing new
cluster [cluster1] with [1] node(s)
02:38:45.654 [main] INFO org.apache.s4.tools.DefineCluster - New cluster
configuration uploaded into zookeeper
On computer 2 I run the S4 node:
./s4 node -c=cluster1 -zk 172.16.0.1:2181
calling referenced s4 script : /root/s4-piper/incubator-s4/s4
16:25:46.986 [main] INFO org.apache.s4.core.Main - Initializing S4 node with :
- comm module class [org.apache.s4.comm.DefaultCommModule]
- comm configuration file [default.s4.comm.properties from classpath]
- core module class [org.apache.s4.core.DefaultCoreModule]
- core configuration file[default.s4.core.properties from classpath]
- extra modules: []
- inline parameters: []
16:25:47.014 [main] DEBUG org.apache.s4.core.Main - Adding named parameters for
injection : [s4.cluster.zk_address=172.16.0.1:2181]
16:25:47.678 [main] INFO org.apache.s4.core.Main - Starting S4 node. This node
will automatically download applications published for the cluster it belongs to
16:25:47.880 [main] INFO o.a.s.comm.topology.AssignmentFromZK - New
session:-2881053689315323; state is : SyncConnected
16:25:48.011 [main] INFO o.a.s.comm.topology.AssignmentFromZK - Successfully
acquired task:Task-0 by computer2
Then on computer 1 I deploy the application (the S4R archive is already
uploaded to computer 2):
./s4 deploy -s4r=/tmp/testApp.s4r -c=cluster1 -appName=testApp
And I see the following output on computer 2 that runs the node:
16:25:48.089 [ZkClient-EventThread-13-172.16.0.1:2181] INFO
org.apache.s4.core.Server - Loading application [testApp] from file
[/tmp/tmp1277962986968632504s4r]
16:25:48.090 [ZkClient-EventThread-13-172.16.0.1:2181] WARN
o.a.s4.base.util.S4RLoaderFactory - s4.tmp.dir not specified, using temporary
directory [/tmp/1344263148090-0] for unpacking S4R. You may want to specify a
parent non-temporary directory.
16:25:48.090 [ZkClient-EventThread-13-172.16.0.1:2181] INFO
o.a.s4.base.util.S4RLoaderFactory - Unzipping S4R archive in
[/tmp/1344263148090-0]
16:25:48.098 [main] INFO org.apache.s4.core.Server - Loading application
[testApp] from file [/tmp/tmp2955192375317617977s4r]
16:25:48.098 [main] WARN o.a.s4.base.util.S4RLoaderFactory - s4.tmp.dir not
specified, using temporary directory [/tmp/1344263148098-0] for unpacking S4R.
You may want to specify a parent non-temporary directory.
16:25:48.098 [main] INFO o.a.s4.base.util.S4RLoaderFactory - Unzipping S4R
archive in [/tmp/1344263148098-0]
16:25:48.403 [ZkClient-EventThread-13-172.16.0.1:2181] INFO
org.apache.s4.core.Server - App class name is: testpkg.TestApp
16:25:48.435 [main] INFO org.apache.s4.core.Server - App class name is:
testpkg.TestApp
16:25:48.519 [ZkClient-EventThread-13-172.16.0.1:2181] INFO
o.a.s4.comm.topology.ClusterFromZK - Changing cluster topology to {
nbNodes=1,name=cluster1,mode=unicast,type=,nodes=[{partition=0,port=12000,machineName=computer,taskId=Task-0}]}
from null
16:25:48.548 [main] INFO o.a.s4.comm.topology.ClusterFromZK - Changing cluster
topology to {
nbNodes=1,name=cluster1,mode=unicast,type=,nodes=[{partition=0,port=12000,machineName=computer2,taskId=Task-0}]}
from null
16:25:48.584 [ZkClient-EventThread-13-172.16.0.1:2181] INFO
o.a.s4.comm.topology.ClusterFromZK - Adding topology change
listener:org.apache.s4.comm.tcp.TCPEmitter@24e33e18
16:25:48.607 [main] INFO o.a.s4.comm.topology.ClusterFromZK - Adding topology
change listener:org.apache.s4.comm.tcp.TCPEmitter@3bc486f2
16:25:48.632 [ZkClient-EventThread-13-172.16.0.1:2181] INFO
o.a.s4.comm.topology.ClustersFromZK - New session:-2881053689315320
16:25:48.640 [ZkClient-EventThread-13-172.16.0.1:2181] INFO
o.a.s4.comm.topology.ClustersFromZK - New session:-2881053689315319
16:25:48.643 [main] INFO o.a.s4.comm.topology.ClustersFromZK - New
session:-2881053689315318
16:25:48.650 [ZkClient-EventThread-13-172.16.0.1:2181] INFO
o.a.s4.comm.topology.ClusterFromZK - Changing cluster topology to {
nbNodes=1,name=cluster1,mode=unicast,type=,nodes=[{partition=0,port=12000,machineName=computer2,taskId=Task-0}]}
from null
16:25:48.651 [ZkClient-EventThread-13-172.16.0.1:2181] INFO
org.apache.s4.core.Server - Loaded application from file
/tmp/tmp1277962986968632504s4r
16:25:48.651 [ZkClient-EventThread-13-172.16.0.1:2181] INFO
o.a.s.d.DistributedDeploymentManager - Successfully installed application
testApp
16:25:48.679 [ZkClient-EventThread-13-172.16.0.1:2181] DEBUG
o.a.s.c.g.OverloadDispatcherGenerator - Dumping generated overload dispatcher
class for PE of class [class testpkg.Block3]
16:25:48.709 [main] INFO o.a.s4.comm.topology.ClusterFromZK - Changing cluster
topology to {
nbNodes=1,name=cluster1,mode=unicast,type=,nodes=[{partition=0,port=12000,machineName=computer2,taskId=Task-0}]}
from null
16:25:48.709 [main] INFO org.apache.s4.core.Server - Loaded application from
file /tmp/tmp2955192375317617977s4r
16:25:48.709 [main] INFO o.a.s.d.DistributedDeploymentManager - Successfully
installed application testApp
16:25:48.722 [ZkClient-EventThread-13-172.16.0.1:2181] DEBUG
o.a.s.c.g.OverloadDispatcherGenerator - Dumping generated overload dispatcher
class for PE of class [class testpkg.Block2]
16:25:48.726 [ZkClient-EventThread-13-172.16.0.1:2181] DEBUG
o.a.s.c.g.OverloadDispatcherGenerator - Dumping generated overload dispatcher
class for PE of class [class testpkg.Block1]
16:25:48.728 [ZkClient-EventThread-13-172.16.0.1:2181] DEBUG
o.a.s4.comm.topology.ClustersFromZK - Adding input stream [RawData] for app
[-1] in cluster [cluster1]
16:25:48.732 [main] DEBUG o.a.s.c.g.OverloadDispatcherGenerator - Dumping
generated overload dispatcher class for PE of class [class testpkg.Block3]
16:25:48.737 [main] DEBUG o.a.s.c.g.OverloadDispatcherGenerator - Dumping
generated overload dispatcher class for PE of class [class testpkg.Block2]
16:25:48.741 [main] DEBUG o.a.s.c.g.OverloadDispatcherGenerator - Dumping
generated overload dispatcher class for PE of class [class testpkg.Block1]
16:25:48.838 [ZkClient-EventThread-13-172.16.0.1:2181] INFO
o.a.s4.comm.topology.ClustersFromZK - Detected new stream [RawData]
16:25:48.995 [main] DEBUG o.a.s4.comm.topology.ClustersFromZK - Adding input
stream [RawData] for app [-1] in cluster [cluster1]
16:25:48.996 [ZkClient-EventThread-13-172.16.0.1:2181] INFO
org.apache.s4.core.App - Init prototype [testpkg.Block3].
16:25:48.998 [ZkClient-EventThread-13-172.16.0.1:2181] INFO
org.apache.s4.core.App - Init prototype [testpkg.Block2].
16:25:48.998 [ZkClient-EventThread-13-172.16.0.1:2181] INFO
org.apache.s4.core.App - Init prototype [testpkg.Block1].
16:25:49.065 [main] INFO org.apache.s4.core.App - Init prototype
[testpkg.Block3].
16:25:49.065 [main] INFO org.apache.s4.core.App - Init prototype
[testpkg.Block2].
16:25:49.065 [main] INFO org.apache.s4.core.App - Init prototype
[testpkg.Block1].
16:31:40.058 [ZkClient-EventThread-13-172.16.0.1:2181] ERROR
o.a.s.d.DistributedDeploymentManager - Application undeployment is not
supported yet
16:31:40.508 [ZkClient-EventThread-22-172.16.0.1:2181] ERROR
o.a.s4.comm.topology.ClusterFromZK - Zookeeper session expired, possibly due to
a network partition for cluster [{
nbNodes=1,name=cluster1,mode=unicast,type=,nodes=[{partition=0,port=12000,machineName=computer2,taskId=Task-0}]}].
This node is considered as dead by Zookeeper. Proceeding to stop this node.
Can you help me? I can start the application and the adapater on a local
cluster (only one node).
Thank you
Regards
- Davide
