Thanks for the reply Edward J. Yoon. I made the changes in the configuration suggested that Edward Yoon continues with more problems. What could observe is that for smaller graphs, with 2000 vertices My application works perfectly. But when I put larger graphs got the following error:
hduser@mestre:/usr/local/hama$ bin/hama jar aplicmenosmsg.jar 8719 /hdfs/disco/grafonewc.txt /hdfs/disco/grafoout 36 12/12/28 10:00:54 INFO bsp.FileInputFormat: Total input paths to process : 1 12/12/28 10:00:54 INFO bsp.FileInputFormat: Total # of splits: 36 12/12/28 10:00:57 INFO bsp.BSPJobClient: Running job: job_201212280959_0001 12/12/28 10:01:00 INFO bsp.BSPJobClient: Current supersteps number: 0 12/12/28 10:01:30 INFO bsp.BSPJobClient: Current supersteps number: 2 12/12/28 10:01:33 INFO bsp.BSPJobClient: Current supersteps number: 4 12/12/28 10:01:39 INFO bsp.BSPJobClient: Current supersteps number: 5 12/12/28 10:02:03 INFO bsp.BSPJobClient: Current supersteps number: 6 12/12/28 10:02:09 INFO bsp.BSPJobClient: Current supersteps number: 0 attempt_201212280959_0001_000027_0: 12/12/28 10:01:21 INFO zookeeper.ZooKeeper: Client environment:zookeeper.version=3.3.3-1073969, built on 02/23/2011 22:27 GMT attempt_201212280959_0001_000027_0: 12/12/28 10:01:21 INFO zookeeper.ZooKeeper: Client environment:host.name=node01 attempt_201212280959_0001_000027_0: 12/12/28 10:01:21 INFO zookeeper.ZooKeeper: Client environment:java.version=1.6.0_24 attempt_201212280959_0001_000027_0: 12/12/28 10:01:21 INFO zookeeper.ZooKeeper: Client environment:java.vendor=Sun Microsystems Inc. attempt_201212280959_0001_000027_0: 12/12/28 10:01:21 INFO zookeeper.ZooKeeper: Client environment:java.home=/usr/lib/jvm/java-6-openjdk-i386/jre . . . attempt_201212280959_0001_000027_0: 12/12/28 10:01:50 INFO ipc.NettyTransceiver: [id: 0x0129c445, /10.16.1.201:46065 => node08/ 10.16.1.208:61001] INTEREST_CHANGED 12/12/28 10:02:12 INFO bsp.BSPJobClient: Job failed. Has anyone had this problem? Got Resolve? If I increase the number of cluster nodes is possible to solve this problem? Is this is a limitation of the Graph API Hama? or my application? 2012/12/27 Edward J. Yoon <[email protected]> > You should increase max heap size of child JVMs, instead of HAMA_HEAPSIZE. > > Add below property to hama-site.xml. > > <property> > <name>bsp.child.java.opts</name> > <value>-Xmx2048m</value> > <description>Java opts for the groom server child processes. > The following symbol, if present, will be interpolated: @taskid@ > is replaced > by current TaskID. Any other occurrences of '@' will go unchanged. > For example, to enable verbose gc logging to a file named for the > taskid in > /tmp and to set the heap maximum to be a gigabyte, pass a 'value' of: > -Xmx1024m -verbose:gc -Xloggc:/tmp/@[email protected] > The configuration variable bsp.child.ulimit can be used to control the > maximum virtual memory of the child processes. > </description> > </property> > > On Thu, Dec 27, 2012 at 5:11 AM, Francisco Sanches > <[email protected]> wrote: > > Thanks for the replies: > > > > Excuse me by the size of the log. But after I changed the parameter of > > hama-env.sh to "export HAMA_HEAPSIZE = 2000" My program stopped giving > the > > memory error and displays the error below. Remembering that I am running > > for a graph with just over 65 000 vertices. What is it like to understand > > is that if I increase the number of nodes in the cluster can run my > > application with this graph of 65 000 vertices? > > > > LOG: > > > > hduser@mestre:/usr/local/hama$ bin/hama jar aplicpronto.jar 65537 > > /hdfs/disco/grafogrande /hdfs/disco/grafoout 36 > > 12/12/26 17:00:19 INFO bsp.FileInputFormat: Total input paths to process > : 1 > > 12/12/26 17:00:19 INFO bsp.FileInputFormat: Total # of splits: 36 > > 12/12/26 17:00:23 INFO bsp.BSPJobClient: Running job: > job_201212261658_0001 > > 12/12/26 17:00:26 INFO bsp.BSPJobClient: Current supersteps number: 0 > > 12/12/26 17:00:56 INFO bsp.BSPJobClient: Current supersteps number: 1 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:31 INFO > > zookeeper.ZooKeeper: Client environment:zookeeper.version=3.3.3-1073969, > > built on 02/23/2011 22:27 GMT > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:31 INFO > > zookeeper.ZooKeeper: Client environment:host.name=mestre > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:31 INFO > > zookeeper.ZooKeeper: Client environment:java.version=1.6.0_24 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:31 INFO > > zookeeper.ZooKeeper: Client environment:java.vendor=Sun Microsystems Inc. > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:31 INFO > > zookeeper.ZooKeeper: Client > > environment:java.home=/usr/lib/jvm/java-6-openjdk-i386/jre > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:31 INFO > > zookeeper.ZooKeeper: Client > > > environment:java.class.path=/usr/local/hama/bin/../conf:/usr/lib/jvm/java-6-openjdk-i386/lib/tools.jar:/usr/local/hama/bin/../hama-core-0.5.0.jar:/usr/local/hama/bin/../hama-examples-0.5.0.jar:/usr/local/hama/bin/../hama-graph-0.5.0.jar:/usr/local/hama/bin/../lib/ant-1.7.1.jar:/usr/local/hama/bin/../lib/ant-launcher-1.7.1.jar:/usr/local/hama/bin/../lib/avro-1.6.0.jar:/usr/local/hama/bin/../lib/avro-ipc-1.6.0.jar:/usr/local/hama/bin/../lib/commons-cli-1.2.jar:/usr/local/hama/bin/../lib/commons-configuration-1.7.jar:/usr/local/hama/bin/../lib/commons-lang-2.6.jar:/usr/local/hama/bin/../lib/commons-logging-1.1.1.jar:/usr/local/hama/bin/../lib/commons-math3-3.0.jar:/usr/local/hama/bin/../lib/guava-10.0.1.jar:/usr/local/hama/bin/../lib/hadoop-core-1.0.0.jar:/usr/local/hama/bin/../lib/hadoop-test-1.0.0.jar:/usr/local/hama/bin/../lib/jackson-core-asl-1.9.2.jar:/usr/local/hama/bin/../lib/jackson-mapper-asl-1.9.2.jar:/usr/local/hama/bin/../lib/jetty-6.1.14.jar:/usr/local/hama/bin/../lib/jetty-annotations-6.1.14.jar:/usr/local/hama/bin/../lib/jetty-util-6.1.14.jar:/usr/local/hama/bin/../lib/jsp-2.1-6.1.14.jar:/usr/local/hama/bin/../lib/jsp-api-2.1-6.1.14.jar:/usr/local/hama/bin/../lib/junit-4.8.1.jar:/usr/local/hama/bin/../lib/log4j-1.2.16.jar:/usr/local/hama/bin/../lib/netty-3.2.6.Final.jar:/usr/local/hama/bin/../lib/servlet-api-6.0.32.jar:/usr/local/hama/bin/../lib/slf4j-api-1.5.8.jar:/usr/local/hama/bin/../lib/slf4j-log4j12-1.5.8.jar:/usr/local/hama/bin/../lib/snappy-java-1.0.4.1.jar:/usr/local/hama/bin/../lib/zookeeper-3.3.3.jar::/hdfs/disco/hama-hduser/bsp/local/groomServer/attempt_201212261658_0001_000000_0/work/classes:/hdfs/disco/hama-hduser/bsp/local/groomServer/attempt_201212261658_0001_000000_0/work > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:31 INFO > > zookeeper.ZooKeeper: Client > > > environment:java.library.path=/usr/lib/jvm/java-6-openjdk-i386/jre/lib/i386/server:/usr/lib/jvm/java-6-openjdk-i386/jre/lib/i386:/usr/lib/jvm/java-6-openjdk-i386/jre/../lib/i386:/usr/java/packages/lib/i386:/usr/lib/i386-linux-gnu/jni:/lib/i386-linux-gnu:/usr/lib/i386-linux-gnu:/usr/lib/jni:/lib:/usr/lib > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:31 INFO > > zookeeper.ZooKeeper: Client environment:java.io.tmpdir=/tmp > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:31 INFO > > zookeeper.ZooKeeper: Client environment:java.compiler=<NA> > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:31 INFO > > zookeeper.ZooKeeper: Client environment:os.name=Linux > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:31 INFO > > zookeeper.ZooKeeper: Client environment:os.arch=i386 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:31 INFO > > zookeeper.ZooKeeper: Client environment:os.version=3.2.0-32-generic-pae > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:31 INFO > > zookeeper.ZooKeeper: Client environment:user.name=hduser > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:31 INFO > > zookeeper.ZooKeeper: Client environment:user.home=/home/hduser > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:31 INFO > > zookeeper.ZooKeeper: Client > > > environment:user.dir=/hdfs/disco/hama-hduser/bsp/local/groomServer/attempt_201212261658_0001_000000_0/work > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:31 INFO > > zookeeper.ZooKeeper: Initiating client connection, > > connectString=node02:21810,node01:21810,mestre:21810 > sessionTimeout=1200000 > > watcher=org.apache.hama.bsp.sync.ZooKeeperSyncClientImpl@1deeb40 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:31 INFO > > zookeeper.ClientCnxn: Opening socket connection to server mestre/ > > 10.16.1.200:21810 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:31 INFO > > zookeeper.ClientCnxn: Socket connection established to mestre/ > > 10.16.1.200:21810, initiating session > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:31 INFO > > sync.ZooKeeperSyncClientImpl: Start connecting to Zookeeper! At mestre/ > > 10.16.1.200:61002 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:32 INFO > > zookeeper.ClientCnxn: Session establishment complete on server mestre/ > > 10.16.1.200:21810, sessionid = 0x3bd8cbb48a0006, negotiated timeout = > > 1200000 > > attempt_201212261658_0001_000000_0: SLF4J: Class path contains multiple > > SLF4J bindings. > > attempt_201212261658_0001_000000_0: SLF4J: Found binding in > > > [jar:file:/usr/local/hama/lib/slf4j-log4j12-1.5.8.jar!/org/slf4j/impl/StaticLoggerBinder.class] > > attempt_201212261658_0001_000000_0: SLF4J: Found binding in > > > [file:/hdfs/disco/hama-hduser/bsp/local/groomServer/attempt_201212261658_0001_000000_0/work/org/slf4j/impl/StaticLoggerBinder.class] > > attempt_201212261658_0001_000000_0: SLF4J: See > > http://www.slf4j.org/codes.html#multiple_bindings for an explanation. > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO > > ipc.NettyTransceiver: Connecting to node03/10.16.1.203:61003 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO > > ipc.NettyTransceiver: [id: 0x002f8116] OPEN > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO > > ipc.NettyTransceiver: [id: 0x002f8116, /10.16.1.200:47685 => node03/ > > 10.16.1.203:61003] BOUND: /10.16.1.200:47685 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO > > ipc.NettyTransceiver: [id: 0x002f8116, /10.16.1.200:47685 => node03/ > > 10.16.1.203:61003] CONNECTED: node03/10.16.1.203:61003 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO > > ipc.NettyTransceiver: Connecting to node05/10.16.1.205:61001 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO > > ipc.NettyTransceiver: [id: 0x00d8ca48] OPEN > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO > > ipc.NettyTransceiver: [id: 0x00d8ca48, /10.16.1.200:52329 => node05/ > > 10.16.1.205:61001] BOUND: /10.16.1.200:52329 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO > > ipc.NettyTransceiver: [id: 0x00d8ca48, /10.16.1.200:52329 => node05/ > > 10.16.1.205:61001] CONNECTED: node05/10.16.1.205:61001 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO > > ipc.NettyTransceiver: Connecting to node04/10.16.1.204:61002 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO > > ipc.NettyTransceiver: [id: 0x018adae2] OPEN > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO > > ipc.NettyTransceiver: [id: 0x018adae2, /10.16.1.200:59179 => node04/ > > 10.16.1.204:61002] BOUND: /10.16.1.200:59179 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO > > ipc.NettyTransceiver: [id: 0x018adae2, /10.16.1.200:59179 => node04/ > > 10.16.1.204:61002] CONNECTED: node04/10.16.1.204:61002 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO > > ipc.NettyTransceiver: Connecting to node06/10.16.1.206:61001 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO > > ipc.NettyTransceiver: [id: 0x003a835d] OPEN > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO > > ipc.NettyTransceiver: [id: 0x003a835d, /10.16.1.200:45011 => node06/ > > 10.16.1.206:61001] BOUND: /10.16.1.200:45011 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO > > ipc.NettyTransceiver: [id: 0x003a835d, /10.16.1.200:45011 => node06/ > > 10.16.1.206:61001] CONNECTED: node06/10.16.1.206:61001 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO > > ipc.NettyTransceiver: Connecting to node04/10.16.1.204:61003 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO > > ipc.NettyTransceiver: [id: 0x00061cd2] OPEN > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO > > ipc.NettyTransceiver: [id: 0x00061cd2, /10.16.1.200:54578 => node04/ > > 10.16.1.204:61003] BOUND: /10.16.1.200:54578 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO > > ipc.NettyTransceiver: [id: 0x00061cd2, /10.16.1.200:54578 => node04/ > > 10.16.1.204:61003] CONNECTED: node04/10.16.1.204:61003 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO > > ipc.NettyTransceiver: Connecting to node05/10.16.1.205:61002 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO > > ipc.NettyTransceiver: [id: 0x00fb6c5f] OPEN > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO > > ipc.NettyTransceiver: [id: 0x00fb6c5f, /10.16.1.200:52964 => node05/ > > 10.16.1.205:61002] BOUND: /10.16.1.200:52964 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO > > ipc.NettyTransceiver: [id: 0x00fb6c5f, /10.16.1.200:52964 => node05/ > > 10.16.1.205:61002] CONNECTED: node05/10.16.1.205:61002 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO > > ipc.NettyTransceiver: Connecting to node03/10.16.1.203:61001 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO > > ipc.NettyTransceiver: [id: 0x00cb42cf] OPEN > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO > > ipc.NettyTransceiver: [id: 0x00cb42cf, /10.16.1.200:36765 => node03/ > > 10.16.1.203:61001] BOUND: /10.16.1.200:36765 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO > > ipc.NettyTransceiver: [id: 0x00cb42cf, /10.16.1.200:36765 => node03/ > > 10.16.1.203:61001] CONNECTED: node03/10.16.1.203:61001 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO > > ipc.NettyTransceiver: Connecting to node01/10.16.1.201:61003 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO > > ipc.NettyTransceiver: [id: 0x018bbf55] OPEN > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO > > ipc.NettyTransceiver: [id: 0x018bbf55, /10.16.1.200:45734 => node01/ > > 10.16.1.201:61003] BOUND: /10.16.1.200:45734 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO > > ipc.NettyTransceiver: [id: 0x018bbf55, /10.16.1.200:45734 => node01/ > > 10.16.1.201:61003] CONNECTED: node01/10.16.1.201:61003 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO > > ipc.NettyTransceiver: Connecting to node02/10.16.1.202:61002 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO > > ipc.NettyTransceiver: [id: 0x00ed5d9d] OPEN > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO > > ipc.NettyTransceiver: [id: 0x00ed5d9d, /10.16.1.200:60724 => node02/ > > 10.16.1.202:61002] BOUND: /10.16.1.200:60724 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO > > ipc.NettyTransceiver: [id: 0x00ed5d9d, /10.16.1.200:60724 => node02/ > > 10.16.1.202:61002] CONNECTED: node02/10.16.1.202:61002 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO > > ipc.NettyTransceiver: Connecting to node03/10.16.1.203:61002 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO > > ipc.NettyTransceiver: [id: 0x014b84c7] OPEN > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO > > ipc.NettyTransceiver: [id: 0x014b84c7, /10.16.1.200:45839 => node03/ > > 10.16.1.203:61002] BOUND: /10.16.1.200:45839 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO > > ipc.NettyTransceiver: [id: 0x014b84c7, /10.16.1.200:45839 => node03/ > > 10.16.1.203:61002] CONNECTED: node03/10.16.1.203:61002 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO > > ipc.NettyTransceiver: Connecting to node02/10.16.1.202:61003 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO > > ipc.NettyTransceiver: [id: 0x0160c4b0] OPEN > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO > > ipc.NettyTransceiver: [id: 0x0160c4b0, /10.16.1.200:44757 => node02/ > > 10.16.1.202:61003] BOUND: /10.16.1.200:44757 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO > > ipc.NettyTransceiver: [id: 0x0160c4b0, /10.16.1.200:44757 => node02/ > > 10.16.1.202:61003] CONNECTED: node02/10.16.1.202:61003 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO > > ipc.NettyTransceiver: Connecting to node04/10.16.1.204:61001 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO > > ipc.NettyTransceiver: [id: 0x0029ea31] OPEN > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO > > ipc.NettyTransceiver: [id: 0x0029ea31, /10.16.1.200:40846 => node04/ > > 10.16.1.204:61001] BOUND: /10.16.1.200:40846 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO > > ipc.NettyTransceiver: [id: 0x0029ea31, /10.16.1.200:40846 => node04/ > > 10.16.1.204:61001] CONNECTED: node04/10.16.1.204:61001 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO > ipc.NettyServer: > > [id: 0x00158cc6, /10.16.1.210:36396 => /10.16.1.200:61002] OPEN > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO > ipc.NettyServer: > > [id: 0x00158cc6, /10.16.1.210:36396 => /10.16.1.200:61002] BOUND: / > > 10.16.1.200:61002 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO > ipc.NettyServer: > > [id: 0x00158cc6, /10.16.1.210:36396 => /10.16.1.200:61002] CONNECTED: / > > 10.16.1.210:36396 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO > ipc.NettyServer: > > [id: 0x016f5261, /10.16.1.200:41941 => /10.16.1.200:61002] OPEN > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO > ipc.NettyServer: > > [id: 0x016f5261, /10.16.1.200:41941 => /10.16.1.200:61002] BOUND: / > > 10.16.1.200:61002 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO > ipc.NettyServer: > > [id: 0x016f5261, /10.16.1.200:41941 => /10.16.1.200:61002] CONNECTED: / > > 10.16.1.200:41941 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO > ipc.NettyServer: > > [id: 0x008ddb93, /10.16.1.206:44493 => /10.16.1.200:61002] OPEN > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO > ipc.NettyServer: > > [id: 0x008ddb93, /10.16.1.206:44493 => /10.16.1.200:61002] BOUND: / > > 10.16.1.200:61002 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO > ipc.NettyServer: > > [id: 0x008ddb93, /10.16.1.206:44493 => /10.16.1.200:61002] CONNECTED: / > > 10.16.1.206:44493 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO > ipc.NettyServer: > > [id: 0x01a9f87c, /10.16.1.202:48243 => /10.16.1.200:61002] OPEN > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO > ipc.NettyServer: > > [id: 0x01a9f87c, /10.16.1.202:48243 => /10.16.1.200:61002] BOUND: / > > 10.16.1.200:61002 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO > ipc.NettyServer: > > [id: 0x01a9f87c, /10.16.1.202:48243 => /10.16.1.200:61002] CONNECTED: / > > 10.16.1.202:48243 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO > ipc.NettyServer: > > [id: 0x01f217ec, /10.16.1.200:41945 => /10.16.1.200:61002] OPEN > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO > ipc.NettyServer: > > [id: 0x01f217ec, /10.16.1.200:41945 => /10.16.1.200:61002] BOUND: / > > 10.16.1.200:61002 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO > ipc.NettyServer: > > [id: 0x01f217ec, /10.16.1.200:41945 => /10.16.1.200:61002] CONNECTED: / > > 10.16.1.200:41945 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO > > ipc.NettyTransceiver: Connecting to node01/10.16.1.201:61001 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO > > ipc.NettyTransceiver: [id: 0x00a86d12] OPEN > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO > > ipc.NettyTransceiver: [id: 0x00a86d12, /10.16.1.200:35763 => node01/ > > 10.16.1.201:61001] BOUND: /10.16.1.200:35763 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO > > ipc.NettyTransceiver: [id: 0x00a86d12, /10.16.1.200:35763 => node01/ > > 10.16.1.201:61001] CONNECTED: node01/10.16.1.201:61001 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO > > ipc.NettyTransceiver: Connecting to mestre/10.16.1.200:61002 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO > > ipc.NettyTransceiver: [id: 0x008932e8] OPEN > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO > ipc.NettyServer: > > [id: 0x016d3536, /10.16.1.200:41949 => /10.16.1.200:61002] OPEN > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO > ipc.NettyServer: > > [id: 0x016d3536, /10.16.1.200:41949 => /10.16.1.200:61002] BOUND: / > > 10.16.1.200:61002 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO > ipc.NettyServer: > > [id: 0x016d3536, /10.16.1.200:41949 => /10.16.1.200:61002] CONNECTED: / > > 10.16.1.200:41949 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO > > ipc.NettyTransceiver: [id: 0x008932e8, /10.16.1.200:41949 => mestre/ > > 10.16.1.200:61002] BOUND: /10.16.1.200:41949 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO > > ipc.NettyTransceiver: [id: 0x008932e8, /10.16.1.200:41949 => mestre/ > > 10.16.1.200:61002] CONNECTED: mestre/10.16.1.200:61002 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO > > ipc.NettyTransceiver: Connecting to node01/10.16.1.201:61002 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO > > ipc.NettyTransceiver: [id: 0x00f2164f] OPEN > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO > ipc.NettyServer: > > [id: 0x01aa0a15, /10.16.1.209:43193 => /10.16.1.200:61002] OPEN > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO > ipc.NettyServer: > > [id: 0x01aa0a15, /10.16.1.209:43193 => /10.16.1.200:61002] BOUND: / > > 10.16.1.200:61002 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO > ipc.NettyServer: > > [id: 0x01aa0a15, /10.16.1.209:43193 => /10.16.1.200:61002] CONNECTED: / > > 10.16.1.209:43193 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO > > ipc.NettyTransceiver: [id: 0x00f2164f, /10.16.1.200:44908 => node01/ > > 10.16.1.201:61002] BOUND: /10.16.1.200:44908 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO > > ipc.NettyTransceiver: [id: 0x00f2164f, /10.16.1.200:44908 => node01/ > > 10.16.1.201:61002] CONNECTED: node01/10.16.1.201:61002 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO > ipc.NettyServer: > > [id: 0x01cf662f, /10.16.1.207:57476 => /10.16.1.200:61002] OPEN > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO > ipc.NettyServer: > > [id: 0x01cf662f, /10.16.1.207:57476 => /10.16.1.200:61002] BOUND: / > > 10.16.1.200:61002 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO > ipc.NettyServer: > > [id: 0x003eec1a, /10.16.1.207:57478 => /10.16.1.200:61002] OPEN > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO > ipc.NettyServer: > > [id: 0x01cf662f, /10.16.1.207:57476 => /10.16.1.200:61002] CONNECTED: / > > 10.16.1.207:57476 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO > ipc.NettyServer: > > [id: 0x01115152, /10.16.1.201:37455 => /10.16.1.200:61002] OPEN > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO > ipc.NettyServer: > > [id: 0x003eec1a, /10.16.1.207:57478 => /10.16.1.200:61002] BOUND: / > > 10.16.1.200:61002 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO > ipc.NettyServer: > > [id: 0x003eec1a, /10.16.1.207:57478 => /10.16.1.200:61002] CONNECTED: / > > 10.16.1.207:57478 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO > ipc.NettyServer: > > [id: 0x01de6817, /10.16.1.210:36402 => /10.16.1.200:61002] OPEN > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO > ipc.NettyServer: > > [id: 0x01de6817, /10.16.1.210:36402 => /10.16.1.200:61002] BOUND: / > > 10.16.1.200:61002 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO > ipc.NettyServer: > > [id: 0x01de6817, /10.16.1.210:36402 => /10.16.1.200:61002] CONNECTED: / > > 10.16.1.210:36402 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO > ipc.NettyServer: > > [id: 0x01115152, /10.16.1.201:37455 => /10.16.1.200:61002] BOUND: / > > 10.16.1.200:61002 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO > ipc.NettyServer: > > [id: 0x01115152, /10.16.1.201:37455 => /10.16.1.200:61002] CONNECTED: / > > 10.16.1.201:37455 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO > ipc.NettyServer: > > [id: 0x00a56214, /10.16.1.207:57480 => /10.16.1.200:61002] OPEN > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO > ipc.NettyServer: > > [id: 0x00a56214, /10.16.1.207:57480 => /10.16.1.200:61002] BOUND: / > > 10.16.1.200:61002 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO > ipc.NettyServer: > > [id: 0x00a56214, /10.16.1.207:57480 => /10.16.1.200:61002] CONNECTED: / > > 10.16.1.207:57480 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO > ipc.NettyServer: > > [id: 0x004bb369, /10.16.1.205:56188 => /10.16.1.200:61002] OPEN > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO > ipc.NettyServer: > > [id: 0x004bb369, /10.16.1.205:56188 => /10.16.1.200:61002] BOUND: / > > 10.16.1.200:61002 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO > ipc.NettyServer: > > [id: 0x004bb369, /10.16.1.205:56188 => /10.16.1.200:61002] CONNECTED: / > > 10.16.1.205:56188 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO > ipc.NettyServer: > > [id: 0x01abdac9, /10.16.1.206:44506 => /10.16.1.200:61002] OPEN > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO > ipc.NettyServer: > > [id: 0x01abdac9, /10.16.1.206:44506 => /10.16.1.200:61002] BOUND: / > > 10.16.1.200:61002 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO > ipc.NettyServer: > > [id: 0x01abdac9, /10.16.1.206:44506 => /10.16.1.200:61002] CONNECTED: / > > 10.16.1.206:44506 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO > ipc.NettyServer: > > [id: 0x01f68272, /10.16.1.205:56189 => /10.16.1.200:61002] OPEN > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO > ipc.NettyServer: > > [id: 0x01f68272, /10.16.1.205:56189 => /10.16.1.200:61002] BOUND: / > > 10.16.1.200:61002 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO > ipc.NettyServer: > > [id: 0x01f68272, /10.16.1.205:56189 => /10.16.1.200:61002] CONNECTED: / > > 10.16.1.205:56189 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO > ipc.NettyServer: > > [id: 0x0103c29b, /10.16.1.202:48251 => /10.16.1.200:61002] OPEN > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO > ipc.NettyServer: > > [id: 0x0103c29b, /10.16.1.202:48251 => /10.16.1.200:61002] BOUND: / > > 10.16.1.200:61002 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO > ipc.NettyServer: > > [id: 0x0103c29b, /10.16.1.202:48251 => /10.16.1.200:61002] CONNECTED: / > > 10.16.1.202:48251 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO > ipc.NettyServer: > > [id: 0x01e8c585, /10.16.1.205:56191 => /10.16.1.200:61002] OPEN > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO > ipc.NettyServer: > > [id: 0x01e8c585, /10.16.1.205:56191 => /10.16.1.200:61002] BOUND: / > > 10.16.1.200:61002 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO > ipc.NettyServer: > > [id: 0x01e8c585, /10.16.1.205:56191 => /10.16.1.200:61002] CONNECTED: / > > 10.16.1.205:56191 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO > ipc.NettyServer: > > [id: 0x00ab835a, /10.16.1.208:50917 => /10.16.1.200:61002] OPEN > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO > ipc.NettyServer: > > [id: 0x003cbb4b, /10.16.1.208:50918 => /10.16.1.200:61002] OPEN > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO > ipc.NettyServer: > > [id: 0x003cbb4b, /10.16.1.208:50918 => /10.16.1.200:61002] BOUND: / > > 10.16.1.200:61002 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO > ipc.NettyServer: > > [id: 0x003cbb4b, /10.16.1.208:50918 => /10.16.1.200:61002] CONNECTED: / > > 10.16.1.208:50918 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO > ipc.NettyServer: > > [id: 0x00ab835a, /10.16.1.208:50917 => /10.16.1.200:61002] BOUND: / > > 10.16.1.200:61002 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO > ipc.NettyServer: > > [id: 0x00ab835a, /10.16.1.208:50917 => /10.16.1.200:61002] CONNECTED: / > > 10.16.1.208:50917 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO > ipc.NettyServer: > > [id: 0x01443800, /10.16.1.209:43200 => /10.16.1.200:61002] OPEN > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO > ipc.NettyServer: > > [id: 0x01443800, /10.16.1.209:43200 => /10.16.1.200:61002] BOUND: / > > 10.16.1.200:61002 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO > ipc.NettyServer: > > [id: 0x01443800, /10.16.1.209:43200 => /10.16.1.200:61002] CONNECTED: / > > 10.16.1.209:43200 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO > ipc.NettyServer: > > [id: 0x011ff451, /10.16.1.209:43201 => /10.16.1.200:61002] OPEN > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO > ipc.NettyServer: > > [id: 0x011ff451, /10.16.1.209:43201 => /10.16.1.200:61002] BOUND: / > > 10.16.1.200:61002 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO > ipc.NettyServer: > > [id: 0x011ff451, /10.16.1.209:43201 => /10.16.1.200:61002] CONNECTED: / > > 10.16.1.209:43201 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO > ipc.NettyServer: > > [id: 0x01a73d30, /10.16.1.208:50924 => /10.16.1.200:61002] OPEN > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO > ipc.NettyServer: > > [id: 0x01a73d30, /10.16.1.208:50924 => /10.16.1.200:61002] BOUND: / > > 10.16.1.200:61002 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO > ipc.NettyServer: > > [id: 0x01a73d30, /10.16.1.208:50924 => /10.16.1.200:61002] CONNECTED: / > > 10.16.1.208:50924 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO > ipc.NettyServer: > > [id: 0x01108727, /10.16.1.211:44099 => /10.16.1.200:61002] OPEN > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO > ipc.NettyServer: > > [id: 0x01108727, /10.16.1.211:44099 => /10.16.1.200:61002] BOUND: / > > 10.16.1.200:61002 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO > ipc.NettyServer: > > [id: 0x01108727, /10.16.1.211:44099 => /10.16.1.200:61002] CONNECTED: / > > 10.16.1.211:44099 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO > ipc.NettyServer: > > [id: 0x011415c8, /10.16.1.211:44100 => /10.16.1.200:61002] OPEN > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO > ipc.NettyServer: > > [id: 0x01a220e6, /10.16.1.206:44511 => /10.16.1.200:61002] OPEN > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO > ipc.NettyServer: > > [id: 0x01a220e6, /10.16.1.206:44511 => /10.16.1.200:61002] BOUND: / > > 10.16.1.200:61002 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO > ipc.NettyServer: > > [id: 0x01a220e6, /10.16.1.206:44511 => /10.16.1.200:61002] CONNECTED: / > > 10.16.1.206:44511 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO > ipc.NettyServer: > > [id: 0x011415c8, /10.16.1.211:44100 => /10.16.1.200:61002] BOUND: / > > 10.16.1.200:61002 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO > ipc.NettyServer: > > [id: 0x011415c8, /10.16.1.211:44100 => /10.16.1.200:61002] CONNECTED: / > > 10.16.1.211:44100 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO > ipc.NettyServer: > > [id: 0x012f41a5, /10.16.1.211:44101 => /10.16.1.200:61002] OPEN > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO > ipc.NettyServer: > > [id: 0x012f41a5, /10.16.1.211:44101 => /10.16.1.200:61002] BOUND: / > > 10.16.1.200:61002 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO > ipc.NettyServer: > > [id: 0x012f41a5, /10.16.1.211:44101 => /10.16.1.200:61002] CONNECTED: / > > 10.16.1.211:44101 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO > > ipc.NettyTransceiver: Connecting to node02/10.16.1.202:61001 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO > > ipc.NettyTransceiver: [id: 0x019c4091] OPEN > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO > ipc.NettyServer: > > [id: 0x000e7d53, /10.16.1.201:37460 => /10.16.1.200:61002] OPEN > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO > ipc.NettyServer: > > [id: 0x000e7d53, /10.16.1.201:37460 => /10.16.1.200:61002] BOUND: / > > 10.16.1.200:61002 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO > ipc.NettyServer: > > [id: 0x000e7d53, /10.16.1.201:37460 => /10.16.1.200:61002] CONNECTED: / > > 10.16.1.201:37460 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO > > ipc.NettyTransceiver: [id: 0x019c4091, /10.16.1.200:54333 => node02/ > > 10.16.1.202:61001] BOUND: /10.16.1.200:54333 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO > > ipc.NettyTransceiver: [id: 0x019c4091, /10.16.1.200:54333 => node02/ > > 10.16.1.202:61001] CONNECTED: node02/10.16.1.202:61001 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO > ipc.NettyServer: > > [id: 0x0053abbb, /10.16.1.203:46568 => /10.16.1.200:61002] OPEN > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO > ipc.NettyServer: > > [id: 0x0053abbb, /10.16.1.203:46568 => /10.16.1.200:61002] BOUND: / > > 10.16.1.200:61002 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO > ipc.NettyServer: > > [id: 0x0053abbb, /10.16.1.203:46568 => /10.16.1.200:61002] CONNECTED: / > > 10.16.1.203:46568 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO > ipc.NettyServer: > > [id: 0x009db992, /10.16.1.203:46569 => /10.16.1.200:61002] OPEN > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO > ipc.NettyServer: > > [id: 0x009db992, /10.16.1.203:46569 => /10.16.1.200:61002] BOUND: / > > 10.16.1.200:61002 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO > ipc.NettyServer: > > [id: 0x009db992, /10.16.1.203:46569 => /10.16.1.200:61002] CONNECTED: / > > 10.16.1.203:46569 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO > > ipc.NettyTransceiver: Connecting to mestre/10.16.1.200:61003 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO > > ipc.NettyTransceiver: [id: 0x01e80761] OPEN > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO > > ipc.NettyTransceiver: [id: 0x01e80761, /10.16.1.200:56327 => mestre/ > > 10.16.1.200:61003] BOUND: /10.16.1.200:56327 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO > > ipc.NettyTransceiver: [id: 0x01e80761, /10.16.1.200:56327 => mestre/ > > 10.16.1.200:61003] CONNECTED: mestre/10.16.1.200:61003 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO > > ipc.NettyTransceiver: Connecting to mestre/10.16.1.200:61001 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO > > ipc.NettyTransceiver: [id: 0x01dbb27d] OPEN > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO > ipc.NettyServer: > > [id: 0x0093c4f1, /10.16.1.203:46573 => /10.16.1.200:61002] OPEN > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO > ipc.NettyServer: > > [id: 0x0093c4f1, /10.16.1.203:46573 => /10.16.1.200:61002] BOUND: / > > 10.16.1.200:61002 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO > ipc.NettyServer: > > [id: 0x0093c4f1, /10.16.1.203:46573 => /10.16.1.200:61002] CONNECTED: / > > 10.16.1.203:46573 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO > > ipc.NettyTransceiver: [id: 0x01dbb27d, /10.16.1.200:51740 => mestre/ > > 10.16.1.200:61001] BOUND: /10.16.1.200:51740 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO > > ipc.NettyTransceiver: [id: 0x01dbb27d, /10.16.1.200:51740 => mestre/ > > 10.16.1.200:61001] CONNECTED: mestre/10.16.1.200:61001 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO > ipc.NettyServer: > > [id: 0x00b9b618, /10.16.1.204:35237 => /10.16.1.200:61002] OPEN > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO > ipc.NettyServer: > > [id: 0x00b9b618, /10.16.1.204:35237 => /10.16.1.200:61002] BOUND: / > > 10.16.1.200:61002 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO > ipc.NettyServer: > > [id: 0x00b9b618, /10.16.1.204:35237 => /10.16.1.200:61002] CONNECTED: / > > 10.16.1.204:35237 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO > ipc.NettyServer: > > [id: 0x00dfc362, /10.16.1.204:35239 => /10.16.1.200:61002] OPEN > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO > ipc.NettyServer: > > [id: 0x00dfc362, /10.16.1.204:35239 => /10.16.1.200:61002] BOUND: / > > 10.16.1.200:61002 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO > ipc.NettyServer: > > [id: 0x00dfc362, /10.16.1.204:35239 => /10.16.1.200:61002] CONNECTED: / > > 10.16.1.204:35239 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO > > ipc.NettyTransceiver: Connecting to node11/10.16.1.211:61003 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO > > ipc.NettyTransceiver: [id: 0x01971eb3] OPEN > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO > ipc.NettyServer: > > [id: 0x01408a75, /10.16.1.202:48260 => /10.16.1.200:61002] OPEN > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO > ipc.NettyServer: > > [id: 0x01408a75, /10.16.1.202:48260 => /10.16.1.200:61002] BOUND: / > > 10.16.1.200:61002 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO > ipc.NettyServer: > > [id: 0x01408a75, /10.16.1.202:48260 => /10.16.1.200:61002] CONNECTED: / > > 10.16.1.202:48260 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO > > ipc.NettyTransceiver: [id: 0x01971eb3, /10.16.1.200:57399 => node11/ > > 10.16.1.211:61003] BOUND: /10.16.1.200:57399 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO > > ipc.NettyTransceiver: [id: 0x01971eb3, /10.16.1.200:57399 => node11/ > > 10.16.1.211:61003] CONNECTED: node11/10.16.1.211:61003 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO > ipc.NettyServer: > > [id: 0x00c713d2, /10.16.1.201:37463 => /10.16.1.200:61002] OPEN > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO > ipc.NettyServer: > > [id: 0x00c713d2, /10.16.1.201:37463 => /10.16.1.200:61002] BOUND: / > > 10.16.1.200:61002 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO > ipc.NettyServer: > > [id: 0x00c713d2, /10.16.1.201:37463 => /10.16.1.200:61002] CONNECTED: / > > 10.16.1.201:37463 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO > > ipc.NettyTransceiver: Connecting to node11/10.16.1.211:61001 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO > > ipc.NettyTransceiver: [id: 0x005b78cf] OPEN > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO > > ipc.NettyTransceiver: [id: 0x005b78cf, /10.16.1.200:41092 => node11/ > > 10.16.1.211:61001] BOUND: /10.16.1.200:41092 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO > > ipc.NettyTransceiver: [id: 0x005b78cf, /10.16.1.200:41092 => node11/ > > 10.16.1.211:61001] CONNECTED: node11/10.16.1.211:61001 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO > ipc.NettyServer: > > [id: 0x00c28cb7, /10.16.1.204:35245 => /10.16.1.200:61002] OPEN > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO > ipc.NettyServer: > > [id: 0x00c28cb7, /10.16.1.204:35245 => /10.16.1.200:61002] BOUND: / > > 10.16.1.200:61002 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO > ipc.NettyServer: > > [id: 0x00c28cb7, /10.16.1.204:35245 => /10.16.1.200:61002] CONNECTED: / > > 10.16.1.204:35245 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO > > ipc.NettyTransceiver: Connecting to node10/10.16.1.210:61002 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO > > ipc.NettyTransceiver: [id: 0x01f51e5c] OPEN > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO > > ipc.NettyTransceiver: [id: 0x01f51e5c, /10.16.1.200:48018 => node10/ > > 10.16.1.210:61002] BOUND: /10.16.1.200:48018 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO > > ipc.NettyTransceiver: [id: 0x01f51e5c, /10.16.1.200:48018 => node10/ > > 10.16.1.210:61002] CONNECTED: node10/10.16.1.210:61002 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO > > ipc.NettyTransceiver: Connecting to node09/10.16.1.209:61003 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO > > ipc.NettyTransceiver: [id: 0x00d36ff3] OPEN > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO > > ipc.NettyTransceiver: [id: 0x00d36ff3, /10.16.1.200:34916 => node09/ > > 10.16.1.209:61003] BOUND: /10.16.1.200:34916 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO > > ipc.NettyTransceiver: [id: 0x00d36ff3, /10.16.1.200:34916 => node09/ > > 10.16.1.209:61003] CONNECTED: node09/10.16.1.209:61003 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO > > ipc.NettyTransceiver: Connecting to node11/10.16.1.211:61002 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO > > ipc.NettyTransceiver: [id: 0x018e899c] OPEN > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO > > ipc.NettyTransceiver: [id: 0x018e899c, /10.16.1.200:58261 => node11/ > > 10.16.1.211:61002] BOUND: /10.16.1.200:58261 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO > > ipc.NettyTransceiver: [id: 0x018e899c, /10.16.1.200:58261 => node11/ > > 10.16.1.211:61002] CONNECTED: node11/10.16.1.211:61002 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO > > ipc.NettyTransceiver: Connecting to node10/10.16.1.210:61003 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO > > ipc.NettyTransceiver: [id: 0x014bc4e6] OPEN > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO > > ipc.NettyTransceiver: [id: 0x014bc4e6, /10.16.1.200:39073 => node10/ > > 10.16.1.210:61003] BOUND: /10.16.1.200:39073 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO > > ipc.NettyTransceiver: [id: 0x014bc4e6, /10.16.1.200:39073 => node10/ > > 10.16.1.210:61003] CONNECTED: node10/10.16.1.210:61003 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO > > ipc.NettyTransceiver: Connecting to node08/10.16.1.208:61002 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO > > ipc.NettyTransceiver: [id: 0x01be1041] OPEN > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO > > ipc.NettyTransceiver: [id: 0x01be1041, /10.16.1.200:39758 => node08/ > > 10.16.1.208:61002] BOUND: /10.16.1.200:39758 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO > > ipc.NettyTransceiver: [id: 0x01be1041, /10.16.1.200:39758 => node08/ > > 10.16.1.208:61002] CONNECTED: node08/10.16.1.208:61002 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO > > ipc.NettyTransceiver: Connecting to node09/10.16.1.209:61001 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO > > ipc.NettyTransceiver: [id: 0x01e090ee] OPEN > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO > > ipc.NettyTransceiver: [id: 0x01e090ee, /10.16.1.200:56029 => node09/ > > 10.16.1.209:61001] BOUND: /10.16.1.200:56029 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO > > ipc.NettyTransceiver: [id: 0x01e090ee, /10.16.1.200:56029 => node09/ > > 10.16.1.209:61001] CONNECTED: node09/10.16.1.209:61001 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO > > ipc.NettyTransceiver: Connecting to node07/10.16.1.207:61003 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO > > ipc.NettyTransceiver: [id: 0x0187f9f1] OPEN > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO > > ipc.NettyTransceiver: [id: 0x0187f9f1, /10.16.1.200:35151 => node07/ > > 10.16.1.207:61003] BOUND: /10.16.1.200:35151 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO > > ipc.NettyTransceiver: [id: 0x0187f9f1, /10.16.1.200:35151 => node07/ > > 10.16.1.207:61003] CONNECTED: node07/10.16.1.207:61003 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO > > ipc.NettyTransceiver: Connecting to node08/10.16.1.208:61003 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO > > ipc.NettyTransceiver: [id: 0x00c3358b] OPEN > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO > > ipc.NettyTransceiver: [id: 0x00c3358b, /10.16.1.200:35301 => node08/ > > 10.16.1.208:61003] BOUND: /10.16.1.200:35301 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO > > ipc.NettyTransceiver: [id: 0x00c3358b, /10.16.1.200:35301 => node08/ > > 10.16.1.208:61003] CONNECTED: node08/10.16.1.208:61003 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO > > ipc.NettyTransceiver: Connecting to node10/10.16.1.210:61001 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO > > ipc.NettyTransceiver: [id: 0x01e82b72] OPEN > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO > > ipc.NettyTransceiver: [id: 0x01e82b72, /10.16.1.200:34819 => node10/ > > 10.16.1.210:61001] BOUND: /10.16.1.200:34819 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO > > ipc.NettyTransceiver: [id: 0x01e82b72, /10.16.1.200:34819 => node10/ > > 10.16.1.210:61001] CONNECTED: node10/10.16.1.210:61001 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO > > ipc.NettyTransceiver: Connecting to node09/10.16.1.209:61002 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO > > ipc.NettyTransceiver: [id: 0x00d0deb5] OPEN > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO > > ipc.NettyTransceiver: [id: 0x00d0deb5, /10.16.1.200:34684 => node09/ > > 10.16.1.209:61002] BOUND: /10.16.1.200:34684 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO > > ipc.NettyTransceiver: [id: 0x00d0deb5, /10.16.1.200:34684 => node09/ > > 10.16.1.209:61002] CONNECTED: node09/10.16.1.209:61002 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO > > ipc.NettyTransceiver: Connecting to node06/10.16.1.206:61002 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO > > ipc.NettyTransceiver: [id: 0x01505258] OPEN > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO > > ipc.NettyTransceiver: [id: 0x01505258, /10.16.1.200:35962 => node06/ > > 10.16.1.206:61002] BOUND: /10.16.1.200:35962 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO > > ipc.NettyTransceiver: [id: 0x01505258, /10.16.1.200:35962 => node06/ > > 10.16.1.206:61002] CONNECTED: node06/10.16.1.206:61002 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO > > ipc.NettyTransceiver: Connecting to node07/10.16.1.207:61001 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO > > ipc.NettyTransceiver: [id: 0x01d8417c] OPEN > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO > > ipc.NettyTransceiver: [id: 0x01d8417c, /10.16.1.200:41009 => node07/ > > 10.16.1.207:61001] BOUND: /10.16.1.200:41009 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO > > ipc.NettyTransceiver: [id: 0x01d8417c, /10.16.1.200:41009 => node07/ > > 10.16.1.207:61001] CONNECTED: node07/10.16.1.207:61001 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO > > ipc.NettyTransceiver: Connecting to node05/10.16.1.205:61003 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO > > ipc.NettyTransceiver: [id: 0x017c398e] OPEN > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO > > ipc.NettyTransceiver: [id: 0x017c398e, /10.16.1.200:53654 => node05/ > > 10.16.1.205:61003] BOUND: /10.16.1.200:53654 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO > > ipc.NettyTransceiver: [id: 0x017c398e, /10.16.1.200:53654 => node05/ > > 10.16.1.205:61003] CONNECTED: node05/10.16.1.205:61003 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO > > ipc.NettyTransceiver: Connecting to node08/10.16.1.208:61001 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO > > ipc.NettyTransceiver: [id: 0x00e13e7b] OPEN > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO > > ipc.NettyTransceiver: [id: 0x00e13e7b, /10.16.1.200:55138 => node08/ > > 10.16.1.208:61001] BOUND: /10.16.1.200:55138 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO > > ipc.NettyTransceiver: [id: 0x00e13e7b, /10.16.1.200:55138 => node08/ > > 10.16.1.208:61001] CONNECTED: node08/10.16.1.208:61001 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO > > ipc.NettyTransceiver: Connecting to node06/10.16.1.206:61003 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO > > ipc.NettyTransceiver: [id: 0x01503458] OPEN > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO > > ipc.NettyTransceiver: [id: 0x01503458, /10.16.1.200:40540 => node06/ > > 10.16.1.206:61003] BOUND: /10.16.1.200:40540 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO > > ipc.NettyTransceiver: [id: 0x01503458, /10.16.1.200:40540 => node06/ > > 10.16.1.206:61003] CONNECTED: node06/10.16.1.206:61003 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO > > ipc.NettyTransceiver: Connecting to node07/10.16.1.207:61002 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO > > ipc.NettyTransceiver: [id: 0x01b963c4] OPEN > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO > > ipc.NettyTransceiver: [id: 0x01b963c4, /10.16.1.200:42816 => node07/ > > 10.16.1.207:61002] BOUND: /10.16.1.200:42816 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO > > ipc.NettyTransceiver: [id: 0x01b963c4, /10.16.1.200:42816 => node07/ > > 10.16.1.207:61002] CONNECTED: node07/10.16.1.207:61002 > > attempt_201212261658_0001_000000_0: 12/12/26 17:00:53 INFO > > graph.GraphJobRunner: Loading finished at 1 steps. > > 12/12/26 17:03:17 INFO bsp.BSPJobClient: Job failed. > > > > > > > > 2012/12/26 Edward J. Yoon <[email protected]> > > > >> BTW, This log looks like you ran out of memory while job was running > >> as a local mode. > >> > >> You can check the cluster status via web UI. > >> > >> 1. http://wiki.apache.org/hama/GettingStarted#Hama_Web_Interfaces > >> > >> On Wed, Dec 26, 2012 at 10:00 PM, Francisco Sanches > >> <[email protected]> wrote: > >> > attempt_201212260904_0005_000031_0: Exception in thread "Thread-1" > >> > java.lang.OutOfMemoryError: GC overhead limit exceeded > >> > >> > >> > >> -- > >> Best Regards, Edward J. Yoon > >> @eddieyoon > >> > > > > > > > > -- > > Francisco Sanches > > > > -- > Best Regards, Edward J. Yoon > @eddieyoon > -- Francisco Sanches
