Hadoop2: How to configure hadoop to use partial amount of resources

2017-02-08 Thread Xenia Demetriou
I use a server with 16 cores, 64 GB ram, 2.5 TB disk and I want to execute a Giraph program. I have installed hadoop-2.7.2 and I don't know how can configure hadoop to use only a partial amount of server resources because the server used by many users. *Requirements:* Hadoop must use max 12 cores (

Re: Out of core computation fails with KryoException: Buffer underflow

2016-11-08 Thread Xenia Demetriou
Hi Denis, For the "java.lang.OutOfMemoryError: GC overhead limit exceeded" error I hope that the conversation in below link can help you. www.mail-archive.com/user@giraph.apache.org/msg02938.html Regards, Xenia 2016-11-08 12:25 GMT+02:00 Denis Dudinski : > Hi Hassan, > > Thank you for really

Re: OutOfMemoryError: Java heap space during Large graph running

2016-11-07 Thread Xenia Demetriou
t; --Panagiotis > > > 2016-11-05 20:28 GMT+02:00 Xenia Demetriou : > > Hi all, > > > > I execute my project from inside eclipse. Maybe this is the problem and > this > > value in mapred-site.xml didn't recognised? > > > > Basically I have a file w

Re: OutOfMemoryError: Java heap space during Large graph running

2016-11-05 Thread Xenia Demetriou
red.map.child.java.opts > -Xmx10g > > > Regards, > Panagiotis > > 2016-11-04 23:11 GMT+02:00 Xenia Demetriou : > > Hi, > > I have the same problem and I add the following in > > mapred-site.xml and hadoop-env.sh but I still have the same problem. > > I try

Re: OutOfMemoryError: Java heap space during Large graph running

2016-11-04 Thread Xenia Demetriou
Hi, I have the same problem and I add the following in mapred-site.xml and hadoop-env.sh but I still have the same problem. I try various values below but nothhing increase the memory. mapred-site.xml: mapred.child.java.opts -Xms256m -Xmx4096m hadoop-env.sh: export HADOOP_HEAPSIZE

Re: How implement Bucketlist/HashMap in Giraph

2016-05-15 Thread Xenia Demetriou
Specifically in Superstep 3 if I choose to present the values of the RealBucket (System.out.println(RealBuckets.bucketList[realMaxPointer_get]); ), I get the error: Caused by: java.lang.NullPointerException. Why the RealBucket since in superstep 2 is full up in Superstep 3 is null? How can I hold

How implement Bucketlist/HashMap in Giraph

2016-05-15 Thread Xenia Demetriou
Hi All, I am MSc student and I try to write an algorithm written in Java using apache Giraph. But below I have a part of a java code using HashMap in a way that I can't understan't how can be implemented in Giraph. I try to implement it to Giraph and for better understanding what actually the qu

Re: java.net.ConnectException: Connection refused

2014-11-03 Thread Xenia Demetriou
Hi Puneet, I am not an expert but I had the same error and I solved it by changing the hostnames of the cluster-Pcs in lowercase e.g Make iHadoop3 -> ihadoop3 -- Xenia 2014-11-02 14:08 GMT+02:00 Puneet Agarwal : > I have setup a cluster of 4 computers for running my Pregel jobs. > > When runn

Re: Hadoop Multi Node Cluster Configuration

2014-08-09 Thread Xenia Demetriou
Hi Alexander, Thanks for your help. Also I am not an expert. In my cluster (4 machines) I define as following: In master file in all the machines I define two of the machines as Master and SecondaryNameNode And in slave file in all the machines, I define the other two machines as Datanode1 and

Re: giraph 1.1.0 Execution Error

2014-08-08 Thread Xenia Demetriou
Hi experts, I found the solution for the Zookeeper Exception. I have change the hostname of the machines to lowercase. Now any example in Giraph can executed. Thanks for your help. 2014-08-08 8:34 GMT+03:00 Sardeshmukh, Vivek : > Hi Xenia, > > I think there is some problem with Zookeeper. Can

Re: giraph 1.1.0 Execution Error

2014-08-08 Thread Xenia Demetriou
Also I try to install Zookeeper separately but I still have the same error. 2014-08-08 8:34 GMT+03:00 Sardeshmukh, Vivek : > Hi Xenia, > > I think there is some problem with Zookeeper. Can you make sure that > Zookeeper server is running. If it is running then is it on port 22181? > (because you

Re: giraph 1.1.0 Execution Error

2014-08-08 Thread Xenia Demetriou
Hi Vivek, How can I see in which port Zookeeper running in my cluster? Giraph start Zookeeper for me. Thanks 2014-08-08 8:34 GMT+03:00 Sardeshmukh, Vivek : > Hi Xenia, > > I think there is some problem with Zookeeper. Can you make sure that > Zookeeper server is running. If it is running then