Hej, The original idea was to run a "local-mode" giraph for programming and testing without the need of configuration and the description on the BigTop project sounded like that was possible. Since I could not get it to work I just run things on the production cluster even though its complicated debugging.
If anyone knows how to run Giraph on a single machine "cluster" without loads setup and config tweaking I'm still interested. (preferred for the 1.1.0 trunk version) cheers Martin On Thu, Jan 30, 2014 at 2:23 AM, Eli Reisman <apache.mail...@gmail.com>wrote: > There is a heap check in there when setting up to launch the AM in the > client module. It uses information coming back from the RM of your YARN > cluster. I'm guessing its cluster configs? > > > On Tue, Jan 14, 2014 at 5:24 PM, Roman Shaposhnik <r...@apache.org> wrote: > >> Hi Martin, >> >> sorry for the belated reply. I am wondering how >> did you configure your cluster? Did you use >> Bigtop's puppet recipies or did you do it by hand? >> >> It seems that Giraph is working fine on the toy >> cluster that I'm deploying with the Bigtop bits. >> But I'm using Puppet and the topology is really >> simple. >> >> Basically without seeing your hadoop and giraph >> config files its pretty tough to answer your >> question in any greater details. >> >> Thanks, >> Roman. >> >> On Tue, Jan 7, 2014 at 5:44 AM, Martin Neumann <mneum...@spotify.com> >> wrote: >> > Hej, >> > >> > I installed Giraph from the apache BigTop project and want to try to run >> > some Giraph jobs locally on my machine. If I understood the website >> > correctly it should be preconfigured to do so. >> > But when I run a Giraph Job I get the following exception: >> > >> > Exception in thread "main" java.lang.IllegalStateException: Giraph's >> > estimated cluster heap 2048MB ask is greater than the current available >> > cluster heap of 0MB. Aborting Job. >> > >> > >> > To me it sounds like a configuration problem, I'm not even sure if it >> comes >> > from Giraph or from Yarn. If it is a configuration issue is there help >> page >> > that tells you to configure the environment correctly? >> > Thanks for the help. >> > >> > >> > >> > >> > >> > here the log of the whole execution: >> > >> > spotify@spotify-ThinkPad-T430s:~$ giraph >> > /home/spotify/workspace/GiraphExe/SpotifyConComp.jar >> > spotifyConnectedComponent.ConComp -eif >> > spotifyConnectedComponent.ConCompInput -of >> > spotifyConnectedComponent.ConCompOutput -c >> > spotifyConnectedComponent.MinTextCombiner -eip >> > /home/spotify/workspace/GiraphExe/in/sample -op >> > /home/spotify/workspace/GiraphExe/out/spotifyConnectedComponent.ConComp >> -w 1 >> > HADOOP_CONF_DIR=/etc/hadoop/conf >> > SLF4J: Class path contains multiple SLF4J bindings. >> > SLF4J: Found binding in >> > >> [jar:file:/usr/lib/hadoop/lib/slf4j-log4j12-1.6.1.jar!/org/slf4j/impl/StaticLoggerBinder.class] >> > SLF4J: Found binding in >> > >> [jar:file:/usr/lib/giraph/lib/slf4j-log4j12-1.7.2.jar!/org/slf4j/impl/StaticLoggerBinder.class] >> > SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an >> > explanation. >> > 14/01/07 14:40:35 INFO utils.ConfigurationUtils: No vertex input format >> > specified. Ensure your InputFormat does not require one. >> > 14/01/07 14:40:35 INFO yarn.GiraphYarnClient: Final output path is: >> > >> hdfs://localhost:8020/home/spotify/workspace/GiraphExe/out/spotifyConnectedComponent.ConComp >> > 14/01/07 14:40:35 INFO service.AbstractService: >> > Service:org.apache.hadoop.yarn.client.YarnClientImpl is inited. >> > 14/01/07 14:40:35 INFO service.AbstractService: >> > Service:org.apache.hadoop.yarn.client.YarnClientImpl is started. >> > 14/01/07 14:40:35 INFO yarn.GiraphYarnClient: Defaulting per-task heap >> size >> > to 1024MB. >> > Exception in thread "main" java.lang.IllegalStateException: Giraph's >> > estimated cluster heap 2048MB ask is greater than the current available >> > cluster heap of 0MB. Aborting Job. >> > at >> > >> org.apache.giraph.yarn.GiraphYarnClient.checkPerNodeResourcesAvailable(GiraphYarnClient.java:204) >> > at >> > org.apache.giraph.yarn.GiraphYarnClient.run(GiraphYarnClient.java:114) >> > at org.apache.giraph.GiraphRunner.run(GiraphRunner.java:96) >> > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) >> > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84) >> > at org.apache.giraph.GiraphRunner.main(GiraphRunner.java:126) >> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >> > at >> > >> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) >> > at >> > >> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) >> > at java.lang.reflect.Method.invoke(Method.java:606) >> > at org.apache.hadoop.util.RunJar.main(RunJar.java:212) >> > >