Thanks for your reply. I am running hadoop pseudo distributed mode. (apache hadoop 1.2.1)
When I run HAMA 0.6.4 release in pseudo distributed mode (using start-bspd.sh) and execute my dummy bsp job in hama commands (hama jar ...), it works well. The problem happens when I ran daemons and my job in eclipse. Below is my full log of BSPMasterRunner in eclipse console. 14/04/25 18:29:01 INFO hama.BSPMasterRunner: STARTUP_MSG: /************************************************************ STARTUP_MSG: Starting BSPMaster STARTUP_MSG: host = km-knuth.local/172.29.9.73 STARTUP_MSG: args = [] STARTUP_MSG: version = 1.2.0 STARTUP_MSG: build = https://svn.apache.org/repos/asf/hadoop/common/branches/branch-1.2 -r 1479473; compiled by 'hortonfo' on Mon May 6 18:29:07 UTC 2013 STARTUP_MSG: java = 1.6.0_65 ************************************************************/ 14/04/25 18:29:01 INFO bsp.BSPMaster: RPC BSPMaster: host localhost port 40000 2014-04-25 18:29:01.614 java[53927:1003] Unable to load realm info from SCDynamicStore 14/04/25 18:29:01 INFO ipc.Server: Starting Socket Reader #1 for port 40000 14/04/25 18:29:01 INFO mortbay.log: Logging to org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog 14/04/25 18:29:01 INFO http.HttpServer: Port returned by webServer.getConnectors()[0].getLocalPort() before open() is -1. Opening the listener on 40013 14/04/25 18:29:01 INFO http.HttpServer: listener.getLocalPort() returned 40013 webServer.getConnectors()[0].getLocalPort() returned 40013 14/04/25 18:29:01 INFO http.HttpServer: Jetty bound to port 40013 14/04/25 18:29:01 INFO mortbay.log: jetty-6.1.14 14/04/25 18:29:01 INFO mortbay.log: Started SelectChannelConnector@localhost :40013 14/04/25 18:29:02 INFO bsp.BSPMaster: Cleaning up the system directory 14/04/25 18:29:02 INFO bsp.BSPMaster: hdfs://localhost:9000/tmp/hadoop-juns/bsp/system 14/04/25 18:29:02 INFO zookeeper.ZooKeeper: Client environment:zookeeper.version=3.4.5-1392090, built on 09/30/2012 17:52 GMT 14/04/25 18:29:02 INFO zookeeper.ZooKeeper: Client environment:host.name =172.29.9.73 14/04/25 18:29:02 INFO zookeeper.ZooKeeper: Client environment:java.version=1.6.0_65 14/04/25 18:29:02 INFO zookeeper.ZooKeeper: Client environment:java.vendor=Apple Inc. 14/04/25 18:29:02 INFO zookeeper.ZooKeeper: Client environment:java.home=/System/Library/Java/JavaVirtualMachines/1.6.0.jdk/Contents/Home 14/04/25 18:29:02 INFO zookeeper.ZooKeeper: Client environment:java.class.path=/Users/juns/workspaces/private/hama/core/target/test-classes:/Users/juns/workspaces/private/hama/core/target/classes:/Users/juns/.m2/repository/javax/xml/bind/jaxb-api/2.2.2/jaxb-api-2.2.2.jar:/Users/juns/.m2/repository/javax/xml/stream/stax-api/1.0-2/stax-api-1.0-2.jar:/Users/juns/.m2/repository/javax/activation/activation/1.1/activation-1.1.jar:/Users/juns/workspaces/private/hama/commons/target/test-classes:/Users/juns/workspaces/private/hama/commons/target/classes:/Users/juns/.m2/repository/com/google/guava/guava/13.0.1/guava-13.0.1.jar:/Users/juns/.m2/repository/org/apache/hadoop/hadoop-core/1.2.0/hadoop-core-1.2.0.jar:/Users/juns/.m2/repository/commons-cli/commons-cli/1.2/commons-cli-1.2.jar:/Users/juns/.m2/repository/xmlenc/xmlenc/0.52/xmlenc-0.52.jar:/Users/juns/.m2/repository/com/sun/jersey/jersey-core/1.8/jersey-core-1.8.jar:/Users/juns/.m2/repository/com/sun/jersey/jersey-json/1.8/jersey-json-1.8.jar:/Users/juns/.m2/repository/org/codehaus/jettison/jettison/1.1/jettison-1.1.jar:/Users/juns/.m2/repository/stax/stax-api/1.0.1/stax-api-1.0.1.jar:/Users/juns/.m2/repository/com/sun/xml/bind/jaxb-impl/2.2.3-1/jaxb-impl-2.2.3-1.jar:/Users/juns/.m2/repository/org/codehaus/jackson/jackson-core-asl/1.9.2/jackson-core-asl-1.9.2.jar:/Users/juns/.m2/repository/org/codehaus/jackson/jackson-mapper-asl/1.9.2/jackson-mapper-asl-1.9.2.jar:/Users/juns/.m2/repository/org/codehaus/jackson/jackson-jaxrs/1.7.1/jackson-jaxrs-1.7.1.jar:/Users/juns/.m2/repository/org/codehaus/jackson/jackson-xc/1.7.1/jackson-xc-1.7.1.jar:/Users/juns/.m2/repository/com/sun/jersey/jersey-server/1.8/jersey-server-1.8.jar:/Users/juns/.m2/repository/asm/asm/3.1/asm-3.1.jar:/Users/juns/.m2/repository/commons-io/commons-io/2.1/commons-io-2.1.jar:/Users/juns/.m2/repository/commons-httpclient/commons-httpclient/3.0.1/commons-httpclient-3.0.1.jar:/Users/juns/.m2/repository/junit/junit/4.8.1/junit-4.8.1.jar:/Users/juns/.m2/repository/commons-logging/commons-logging/1.1.1/commons-logging-1.1.1.jar:/Users/juns/.m2/repository/commons-codec/commons-codec/1.4/commons-codec-1.4.jar:/Users/juns/.m2/repository/org/apache/commons/commons-math/2.1/commons-math-2.1.jar:/Users/juns/.m2/repository/commons-configuration/commons-configuration/1.7/commons-configuration-1.7.jar:/Users/juns/.m2/repository/commons-collections/commons-collections/3.2.1/commons-collections-3.2.1.jar:/Users/juns/.m2/repository/commons-lang/commons-lang/2.6/commons-lang-2.6.jar:/Users/juns/.m2/repository/commons-digester/commons-digester/1.8.1/commons-digester-1.8.1.jar:/Users/juns/.m2/repository/commons-beanutils/commons-beanutils/1.8.3/commons-beanutils-1.8.3.jar:/Users/juns/.m2/repository/commons-net/commons-net/1.4.1/commons-net-1.4.1.jar:/Users/juns/.m2/repository/oro/oro/2.0.8/oro-2.0.8.jar:/Users/juns/.m2/repository/org/mortbay/jetty/jetty/6.1.14/jetty-6.1.14.jar:/Users/juns/.m2/repository/org/mortbay/jetty/jetty-util/6.1.14/jetty-util-6.1.14.jar:/Users/juns/.m2/repository/org/mortbay/jetty/servlet-api-2.5/6.1.14/servlet-api-2.5-6.1.14.jar:/Users/juns/.m2/repository/tomcat/jasper-runtime/5.5.12/jasper-runtime-5.5.12.jar:/Users/juns/.m2/repository/tomcat/jasper-compiler/5.5.12/jasper-compiler-5.5.12.jar:/Users/juns/.m2/repository/org/mortbay/jetty/jsp-api-2.1/6.1.14/jsp-api-2.1-6.1.14.jar:/Users/juns/.m2/repository/org/mortbay/jetty/jsp-2.1/6.1.14/jsp-2.1-6.1.14.jar:/Users/juns/.m2/repository/org/eclipse/jdt/core/3.1.1/core-3.1.1.jar:/Users/juns/.m2/repository/ant/ant/1.6.5/ant-1.6.5.jar:/Users/juns/.m2/repository/commons-el/commons-el/1.0/commons-el-1.0.jar:/Users/juns/.m2/repository/net/java/dev/jets3t/jets3t/0.6.1/jets3t-0.6.1.jar:/Users/juns/.m2/repository/hsqldb/hsqldb/ 1.8.0.10/hsqldb-1.8.0.10.jar:/Users/juns/.m2/repository/org/apache/hadoop/hadoop-test/1.2.0/hadoop-test-1.2.0.jar:/Users/juns/.m2/repository/org/apache/ftpserver/ftplet-api/1.0.0/ftplet-api-1.0.0.jar:/Users/juns/.m2/repository/org/apache/mina/mina-core/2.0.0-M5/mina-core-2.0.0-M5.jar:/Users/juns/.m2/repository/org/slf4j/slf4j-api/1.5.8/slf4j-api-1.5.8.jar:/Users/juns/.m2/repository/org/apache/ftpserver/ftpserver-core/1.0.0/ftpserver-core-1.0.0.jar:/Users/juns/.m2/repository/org/apache/ftpserver/ftpserver-deprecated/1.0.0-M2/ftpserver-deprecated-1.0.0-M2.jar:/Users/juns/.m2/repository/commons-codec/commons-codec/1.2/commons-codec-1.2.jar:/Users/juns/.m2/repository/org/xerial/snappy/snappy-java/1.0.5/snappy-java-1.0.5.jar:/Users/juns/.m2/repository/org/apache/commons/commons-math3/3.0/commons-math3-3.0.jar:/Users/juns/.m2/repository/org/apache/ant/ant/1.7.1/ant-1.7.1.jar:/Users/juns/.m2/repository/org/apache/ant/ant-launcher/1.7.1/ant-launcher-1.7.1.jar:/Users/juns/.m2/repository/org/apache/tomcat/servlet-api/6.0.32/servlet-api-6.0.32.jar:/Users/juns/.m2/repository/org/mortbay/jetty/jetty-annotations/6.1.14/jetty-annotations-6.1.14.jar:/Users/juns/.m2/repository/org/mortbay/jetty/jetty-plus/6.1.14/jetty-plus-6.1.14.jar:/Users/juns/.m2/repository/geronimo-spec/geronimo-spec-jta/1.0.1B-rc4/geronimo-spec-jta-1.0.1B-rc4.jar:/Users/juns/.m2/repository/org/apache/geronimo/specs/geronimo-annotation_1.0_spec/1.0/geronimo-annotation_1.0_spec-1.0.jar:/Users/juns/.m2/repository/org/slf4j/slf4j-log4j12/1.5.8/slf4j-log4j12-1.5.8.jar:/Users/juns/.m2/repository/log4j/log4j/1.2.16/log4j-1.2.16.jar:/Users/juns/.m2/repository/org/apache/zookeeper/zookeeper/3.4.5/zookeeper-3.4.5.jar:/Users/juns/.m2/repository/jline/jline/0.9.94/jline-0.9.94.jar:/Users/juns/.m2/repository/org/jboss/netty/netty/3.2.2.Final/netty-3.2.2.Final.jar:/Users/juns/workspaces/private/hama/conf 14/04/25 18:29:02 INFO zookeeper.ZooKeeper: Client environment:java.library.path=.:/Library/Java/Extensions:/System/Library/Java/Extensions:/usr/lib/java 14/04/25 18:29:02 INFO zookeeper.ZooKeeper: Client environment:java.io.tmpdir=/var/folders/rv/42slk6553x7885qkq08kdt3r0000gn/T/ 14/04/25 18:29:02 INFO zookeeper.ZooKeeper: Client environment:java.compiler=<NA> 14/04/25 18:29:02 INFO zookeeper.ZooKeeper: Client environment:os.name=Mac OS X 14/04/25 18:29:02 INFO zookeeper.ZooKeeper: Client environment:os.arch=x86_64 14/04/25 18:29:02 INFO zookeeper.ZooKeeper: Client environment:os.version=10.9.2 14/04/25 18:29:02 INFO zookeeper.ZooKeeper: Client environment:user.name =juns 14/04/25 18:29:02 INFO zookeeper.ZooKeeper: Client environment:user.home=/Users/juns 14/04/25 18:29:02 INFO zookeeper.ZooKeeper: Client environment:user.dir=/Users/juns/workspaces/private/hama/core 14/04/25 18:29:02 INFO zookeeper.ZooKeeper: Initiating client connection, connectString=localhost:21810 sessionTimeout=1200000 watcher=org.apache.hama.bsp.sync.ZKSyncBSPMasterClient@76d78df0 14/04/25 18:29:02 INFO sync.ZKSyncBSPMasterClient: Initialized ZK false 14/04/25 18:29:02 INFO sync.ZKSyncClient: Initializing ZK Sync Client 14/04/25 18:29:02 INFO zookeeper.ClientCnxn: Opening socket connection to server localhost/0:0:0:0:0:0:0:1:21810. Will not attempt to authenticate using SASL (Unable to locate a login configuration) 14/04/25 18:29:02 INFO zookeeper.ClientCnxn: Socket connection established to localhost/0:0:0:0:0:0:0:1:21810, initiating session 14/04/25 18:29:02 INFO zookeeper.ClientCnxn: Session establishment complete on server localhost/0:0:0:0:0:0:0:1:21810, sessionid = 0x145983756eb0000, negotiated timeout = 1200000 14/04/25 18:29:02 INFO ipc.Server: IPC Server Responder: starting 14/04/25 18:29:02 INFO ipc.Server: IPC Server listener on 40000: starting 14/04/25 18:29:02 INFO ipc.Server: IPC Server handler 0 on 40000: starting 14/04/25 18:29:02 INFO bsp.BSPMaster: Starting RUNNING 14/04/25 18:29:05 INFO bsp.BSPMaster: groomd_172.29.9.73_50000 is added. Exception in thread "Socket Reader #1 for port 40000" java.lang.OutOfMemoryError: Java heap space at org.apache.hama.ipc.RPC$Invocation.readFields(RPC.java:111) at org.apache.hama.ipc.Server$Connection.processData(Server.java:1121) at org.apache.hama.ipc.Server$Connection.processOneRpc(Server.java:1099) at org.apache.hama.ipc.Server$Connection.readAndProcess(Server.java:1063) at org.apache.hama.ipc.Server$Listener.doRead(Server.java:576) at org.apache.hama.ipc.Server$Listener$Reader.run(Server.java:363) On Fri, Apr 25, 2014 at 4:57 PM, Edward J. Yoon <edwardy...@apache.org>wrote: > Hi, can you attach the full log of the BSPMaster? > > Just to be sure, to run Hama in pseudo distributed mode, you must run > HDFS first. > > On Fri, Apr 25, 2014 at 4:19 PM, InJun Song <ijs...@gmail.com> wrote: > > Hi, > > > > I have a question about how to run hama in eclipse environment. > > > > At first, I set up my develop environment according to this page. ( > > http://wiki.apache.org/hama/HowToContribute) > > > > Then, because I wanted to run pseudo distributed mode, I ran each daemon > > under eclipse. (zookeeper -> bspmaster -> groomserver). To run those, I > > executed three runner for them; ZookeeprRunner, BSPMasterRunner and > > GroomserverRunner. > > After all daemons launched, I ran dummy BSP program. > > > > At that time, BSPMaster throws OOM exception. Code throws an exception is > > in "RPC.Invocation.readFields" method. While it executed this line, it > > throws the exception. > > > > parameters = new Object[in.readInt()]; > > > > The number of object to create is 308766068. If sizeof(java.lang.Object) > is > > 8 byte, total size is over 2G. I think that it must be wrong. > > > > What did I mistake? What mistake make me go wrong way? > > I am debugging with eclipse, but I want to some advices from you. > > > > Thank you. > > > > -- > Best Regards, Edward J. Yoon > CEO at DataSayer Co., Ltd. >