Re: issues while running giraph on Hadoop 2.2

2014-06-01 Thread Kiru Pakkirisamy
Hi Eli,
Thanks for the response.
I get this error now -

(for this cmd - 
hadoop jar giraph-examples-1.1.0-SNAPSHOT.jar org.apache.giraph.GiraphRunner 
org.apache.giraph.examples.SimpleShortestPathsComputation -vif 
org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputFormat -vip 
/user/hdfs/input/tiny.txt -vof 
org.apache.giraph.io.formats.IdWithValueTextOutputFormat -op 
/user/hdfs/output/shortest -w 1 -ca giraph.SplitMasterWorker=false -ca 
giraph.zkList=localhost:2181 -yj 
giraph-examples/target/giraph-examples-1.1.0-SNAPSHOT-for-hadoop-2.2.0-jar-with-dependencies.jar
 
)

14/06/01 14:25:20 INFO utils.ConfigurationUtils: No edge input format 
specified. Ensure your InputFormat does not require one.
14/06/01 14:25:20 INFO utils.ConfigurationUtils: No edge output format 
specified. Ensure your OutputFormat does not require one.
14/06/01 14:25:20 INFO utils.ConfigurationUtils: Setting custom argument 
[giraph.SplitMasterWorker] to [false] in GiraphConfiguration
14/06/01 14:25:20 INFO utils.ConfigurationUtils: Setting custom argument 
[giraph.zkList] to [localhost:2181] in GiraphConfiguration
Exception in thread "main" java.lang.ClassNotFoundException: 
org.apache.giraph.examples.SimpleShortestPathsComputation
at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:190)
at 
org.apache.giraph.utils.ConfigurationUtils.handleComputationClass(ConfigurationUtils.java:482)
at 
org.apache.giraph.utils.ConfigurationUtils.populateGiraphConfiguration(ConfigurationUtils.java:465)
at 
org.apache.giraph.utils.ConfigurationUtils.parseArgs(ConfigurationUtils.java:207)
at org.apache.giraph.GiraphRunner.run(GiraphRunner.java:74)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
at org.apache.giraph.GiraphRunner.main(GiraphRunner.java:127)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.util.RunJar.main(RunJar.java:212)

 
Regards,
- kiru


Kiru Pakkirisamy | webcloudtech.wordpress.com



 From: Eli Reisman 
To: user@giraph.apache.org; Kiru Pakkirisamy  
Sent: Tuesday, May 27, 2014 2:27 PM
Subject: Re: issues while running giraph on Hadoop 2.2
 


Hi, see here (from your stack trace):


14/05/07 23:45:42 INFO yarn.GiraphYarnClient: Defaulting per-task heap size to 
1024MB.
Exception in thread "main" java.lang.IllegalStateException: Giraph's estimated 
cluster heap 2048MB ask is greater than the current available cluster heap of 
0MB. Aborting Job.

You'll need to adjust the min/max heap allocations allowed on your YARN cluster 
if possible. Alternately you can alter the defaults set up in the Giraph-Yarn 
client code and rebuild Giraph with those set differently.



On Thu, May 15, 2014 at 7:56 PM, Kiru Pakkirisamy  
wrote:

I have the same issue running giraph built with -Phadoop_yarn 
-Dhadoop.version=2.2.0 on Hortonworks.
>I get classnotfoundexceptions, looks like the GiraphRunner may not be able to 
>load the classes correctly while running on Yarn.
>So if anybody has had success please share your experience, wondering what I 
>have been doing wrong.
>
>
>Exception in thread "main" java.lang.ClassNotFoundException: 
>org.apache.giraph.examples.SimpleShortestPathsComputation
>
>
> 
>Regards,
>- kiru
>
>
>
> From: yeshwanth kumar 
>To: "user@giraph.apache.org"  
>Sent: Thursday, May 8, 2014 4:47 AM
>Subject: issues while running giraph on Hadoop 2.2
> 
>
>
>hi i am running Hortonworks hadoop 2.2
>trying to run giraph shortest path example.
>i am getting this exception
>
>
>
>
>
>
>$ hadoop jar 
>giraph-examples/target/giraph-examples-1.1.0-SNAPSHOT-for-hadoop-2.2.0-jar-with-dependencies.jar
> org.apache.giraph.GiraphRunner 
>org.apache.giraph.examples.SimpleShortestPathsComputation -vif 
>org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputFormat -vip 
>/user/hdfs/input/tiny.txt -vof 
>org.apache.giraph.io.formats.IdWithValueTextOutputFormat -op 
>/user/hdfs/output/shortestpaths -w 1
>14/05/07 23:45:42 INFO utils.ConfigurationUtils: No edge input format 
>specified. Ensure your InputFormat d

Re: issues while running giraph on Hadoop 2.2

2014-05-27 Thread Eli Reisman
Hi, see here (from your stack trace):

14/05/07 23:45:42 INFO yarn.GiraphYarnClient: Defaulting per-task heap size
to 1024MB.
Exception in thread "main" java.lang.IllegalStateException: Giraph's
estimated cluster heap 2048MB ask is greater than the current available
cluster heap of 0MB. Aborting Job.

You'll need to adjust the min/max heap allocations allowed on your YARN
cluster if possible. Alternately you can alter the defaults set up in the
Giraph-Yarn client code and rebuild Giraph with those set differently.


On Thu, May 15, 2014 at 7:56 PM, Kiru Pakkirisamy  wrote:

> I have the same issue running giraph built with -Phadoop_yarn
> -Dhadoop.version=2.2.0 on Hortonworks.
> I get classnotfoundexceptions, looks like the GiraphRunner may not be able
> to load the classes correctly while running on Yarn.
> So if anybody has had success please share your experience, wondering what
> I have been doing wrong.
>
> Exception in thread "main" java.lang.ClassNotFoundException:
> org.apache.giraph.examples.SimpleShortestPathsComputation
>
>
> Regards,
> - kiru
>
>   --
>  *From:* yeshwanth kumar 
> *To:* "user@giraph.apache.org" 
> *Sent:* Thursday, May 8, 2014 4:47 AM
> *Subject:* issues while running giraph on Hadoop 2.2
>
> hi i am running Hortonworks hadoop 2.2
> trying to run giraph shortest path example.
> i am getting this exception
>
>
>
> $ hadoop jar
> giraph-examples/target/giraph-examples-1.1.0-SNAPSHOT-for-hadoop-2.2.0-jar-with-dependencies.jar
> org.apache.giraph.GiraphRunner
> org.apache.giraph.examples.SimpleShortestPathsComputation -vif
> org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputFormat
> -vip /user/hdfs/input/tiny.txt -vof
> org.apache.giraph.io.formats.IdWithValueTextOutputFormat -op
> /user/hdfs/output/shortestpaths -w 1
> 14/05/07 23:45:42 INFO utils.ConfigurationUtils: No edge input format
> specified. Ensure your InputFormat does not require one.
> 14/05/07 23:45:42 INFO utils.ConfigurationUtils: No edge output format
> specified. Ensure your OutputFormat does not require one.
> 14/05/07 23:45:42 INFO yarn.GiraphYarnClient: Final output path is:
> hdfs://server02.infnet:8020/user/hdfs/output/shortestpaths
> 14/05/07 23:45:42 INFO yarn.GiraphYarnClient: Running Client
> 14/05/07 23:45:42 INFO client.RMProxy: Connecting to ResourceManager at
> server02.infnet/192.168.1.85:8050
> 14/05/07 23:45:42 INFO yarn.GiraphYarnClient: Defaulting per-task heap
> size to 1024MB.
> Exception in thread "main" java.lang.IllegalStateException: Giraph's
> estimated cluster heap 2048MB ask is greater than the current available
> cluster heap of 0MB. Aborting Job.
>  at
> org.apache.giraph.yarn.GiraphYarnClient.checkPerNodeResourcesAvailable(GiraphYarnClient.java:230)
> at org.apache.giraph.yarn.GiraphYarnClient.run(GiraphYarnClient.java:124)
>  at org.apache.giraph.GiraphRunner.run(GiraphRunner.java:96)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
>  at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
> at org.apache.giraph.GiraphRunner.main(GiraphRunner.java:126)
>  at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>  at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
>  at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
>
>
>
> i am missing some configuration, can someone point me in the right
> direction.
>
> thanks,
> Yeshwanth
>
>
>


Re: issues while running giraph on Hadoop 2.2

2014-05-16 Thread Kiru Pakkirisamy
I have the same issue running giraph built with -Phadoop_yarn 
-Dhadoop.version=2.2.0 on Hortonworks.
I get classnotfoundexceptions, looks like the GiraphRunner may not be able to 
load the classes correctly while running on Yarn.
So if anybody has had success please share your experience, wondering what I 
have been doing wrong.

Exception in thread "main" java.lang.ClassNotFoundException: 
org.apache.giraph.examples.SimpleShortestPathsComputation

 
Regards,
- kiru



 From: yeshwanth kumar 
To: "user@giraph.apache.org"  
Sent: Thursday, May 8, 2014 4:47 AM
Subject: issues while running giraph on Hadoop 2.2
 


hi i am running Hortonworks hadoop 2.2
trying to run giraph shortest path example.
i am getting this exception



$ hadoop jar 
giraph-examples/target/giraph-examples-1.1.0-SNAPSHOT-for-hadoop-2.2.0-jar-with-dependencies.jar
 org.apache.giraph.GiraphRunner 
org.apache.giraph.examples.SimpleShortestPathsComputation -vif 
org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputFormat -vip 
/user/hdfs/input/tiny.txt -vof 
org.apache.giraph.io.formats.IdWithValueTextOutputFormat -op 
/user/hdfs/output/shortestpaths -w 1
14/05/07 23:45:42 INFO utils.ConfigurationUtils: No edge input format 
specified. Ensure your InputFormat does not require one.
14/05/07 23:45:42 INFO utils.ConfigurationUtils: No edge output format 
specified. Ensure your OutputFormat does not require one.
14/05/07 23:45:42 INFO yarn.GiraphYarnClient: Final output path is: 
hdfs://server02.infnet:8020/user/hdfs/output/shortestpaths
14/05/07 23:45:42 INFO yarn.GiraphYarnClient: Running Client
14/05/07 23:45:42 INFO client.RMProxy: Connecting to ResourceManager at 
server02.infnet/192.168.1.85:8050
14/05/07 23:45:42 INFO yarn.GiraphYarnClient: Defaulting per-task heap size to 
1024MB.
Exception in thread "main" java.lang.IllegalStateException: Giraph's estimated 
cluster heap 2048MB ask is greater than the current available cluster heap of 
0MB. Aborting Job.
at 
org.apache.giraph.yarn.GiraphYarnClient.checkPerNodeResourcesAvailable(GiraphYarnClient.java:230)
at org.apache.giraph.yarn.GiraphYarnClient.run(GiraphYarnClient.java:124)
at org.apache.giraph.GiraphRunner.run(GiraphRunner.java:96)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
at org.apache.giraph.GiraphRunner.main(GiraphRunner.java:126)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.util.RunJar.main(RunJar.java:212)



i am missing some configuration, can someone point me in the right direction.

thanks,
Yeshwanth

issues while running giraph on Hadoop 2.2

2014-05-12 Thread yeshwanth kumar
hi i am running Hortonworks hadoop 2.2
trying to run giraph shortest path example.
i am getting this exception



$ hadoop jar
giraph-examples/target/giraph-examples-1.1.0-SNAPSHOT-for-hadoop-2.2.0-jar-with-dependencies.jar
org.apache.giraph.GiraphRunner
org.apache.giraph.examples.SimpleShortestPathsComputation -vif
org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputFormat
-vip /user/hdfs/input/tiny.txt -vof
org.apache.giraph.io.formats.IdWithValueTextOutputFormat -op
/user/hdfs/output/shortestpaths -w 1
14/05/07 23:45:42 INFO utils.ConfigurationUtils: No edge input format
specified. Ensure your InputFormat does not require one.
14/05/07 23:45:42 INFO utils.ConfigurationUtils: No edge output format
specified. Ensure your OutputFormat does not require one.
14/05/07 23:45:42 INFO yarn.GiraphYarnClient: Final output path is:
hdfs://server02.infnet:8020/user/hdfs/output/shortestpaths
14/05/07 23:45:42 INFO yarn.GiraphYarnClient: Running Client
14/05/07 23:45:42 INFO client.RMProxy: Connecting to ResourceManager at
server02.infnet/192.168.1.85:8050
14/05/07 23:45:42 INFO yarn.GiraphYarnClient: Defaulting per-task heap size
to 1024MB.
Exception in thread "main" java.lang.IllegalStateException: Giraph's
estimated cluster heap 2048MB ask is greater than the current available
cluster heap of 0MB. Aborting Job.
at
org.apache.giraph.yarn.GiraphYarnClient.checkPerNodeResourcesAvailable(GiraphYarnClient.java:230)
at org.apache.giraph.yarn.GiraphYarnClient.run(GiraphYarnClient.java:124)
at org.apache.giraph.GiraphRunner.run(GiraphRunner.java:96)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
at org.apache.giraph.GiraphRunner.main(GiraphRunner.java:126)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.util.RunJar.main(RunJar.java:212)



i am missing some configuration, can someone point me in the right
direction.

thanks,
Yeshwanth