Thanks Kaushik, it's worked me well.
Sent from my iPad
On Aug 1, 2012, at 11:20 PM, KAUSHIK SARKAR <countkaus...@gmail.com> wrote:
> Hi,
>
> I found an alternative programmatic solution to the problem.
> We need to modify the run() method in the following way -
>
> ...
> @Override
> public int run(String[] argArray) throws Exception {
> Preconditions.checkArgument(argArray.length == 4,
> "run: Must have 4 arguments <input path> <output path> " +
> "<source vertex id> <# of workers>");
>
> GiraphJob job = new GiraphJob(getConf(), getClass().getName());
> // This is the addition - it will make hadoop look for other classes
> in the same jar that contains this class
> job.getInternalJob().setJarByClass(getClass());
> job.setVertexClass(getClass());
> ...
> }
>
> After this hadoop will look for missing classes in the same jar that contains
> the class returned by getClass(), and we will not need to add the job jar
> name separately to the -libjars option (as described by Marcin's
> stackoverflow post).
>
> I think Giraph may add a constructor like -
>
> GiraphJob(Configuration conf, Class jobClass, String jobName) {
> this(conf, jobName);
> this.getInternalJob().setJarByClass(jobClass);
> }
>
> in tradition of the Hadoop constructor -
>
> public JobConf(Configuration conf, Class exampleClass) {
> this(conf);
> setJarByClass(exampleClass);
> }
>
> It will solve the rather annoying class-not-found problem.
>
> Regards,
> Kaushik
>
> On Tue, Jul 24, 2012 at 11:31 AM, KAUSHIK SARKAR <countkaus...@gmail.com>
> wrote:
> Hi,
>
> Marcin's solution worked for me. Thanks!
>
> I didn't try out addFileToClassPath solution, but I think it should work.
> Thanks for the suggestion!
>
> As Marcin has already suggested, it is not very much clear why we have to do
> it like this and why the normal call to hadoop jar doesn't work. I guess it
> has more to do with the way Java (or Hadoop) loads class files than Giraph.
> However, I am a total novice in Java and would like to know the reason.
>
> N.B. I tried with the latest development snapshot of the Giraph (giraph-0.2)
> , and normal call to hadoop jar didn't work with it as well (of course
> Marcin's solution worked).
>
> Regards,
> Kaushik
>
> On Tue, Jul 24, 2012 at 6:22 AM, Etienne Dumoulin
> <etienne.dumou...@idiro.com> wrote:
> Hi,
>
> I cannot explain it.
> But if it any use, for making it working without specifying the library in
> command line you can use the distributed cache:
> DistributedCache#addFileToClassPath
>
> Regards,
>
> Étienne
>
>
> On 24 July 2012 13:37, Marcin Biczak <marcinbic...@gmail.com> wrote:
> Hi
>
> I had similar problem, here is "my solution",
> http://stackoverflow.com/questions/10700853/giraph-shortest-paths-example-classnotfoundexception.
> But I still don't know why I have do it like this, maybe someone here can
> explain this?
>
> regards
> marcin biczak
>
>
> 2012/7/24 KAUSHIK SARKAR <countkaus...@gmail.com>
> Hi,
>
> I created a new test project and copied the code from
> SimpleShortestPathsVertex.java file (org.apache.giraph.examples) into it. But
> when I run it, it gives me java.lang.ClassNotFoundException. However, if I
> run the same program from the giraph-0.1-jar-with-dependencies.jar, it runs
> just fine. Following is the relevant information :
>
> Hadoop version: hadoop-1.0.3 (running in pseudo distributed mode)
> giraph version: giraph-0.1
> jar created with Eclipse 3.3.1
> Output:
>
> kaushik@kaushik-laptop:~/hadoop/hadoop-1.0.3$ bin/hadoop jar ../jars/test.jar
> /shortestPathsInputGraph /shortestPathsOutputGraph 0 3
> 12/07/23 18:29:10 INFO mapred.JobClient: Running job: job_201207231222_0021
> 12/07/23 18:29:11 INFO mapred.JobClient: map 0% reduce 0%
> 12/07/23 18:29:20 INFO mapred.JobClient: Task Id :
> attempt_201207231222_0021_m_000005_0, Status : FAILED
> java.lang.RuntimeException: java.lang.RuntimeException:
> java.lang.ClassNotFoundException:
> test.giraph.SimpleShortestPathsVertex$SimpleShortestPathsVertexOutputFormat
> at
> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:899)
> at
> org.apache.giraph.graph.BspUtils.getVertexOutputFormatClass(BspUtils.java:134)
> at
> org.apache.giraph.bsp.BspOutputFormat.getOutputCommitter(BspOutputFormat.java:56)
> at org.apache.hadoop.mapred.Task.initialize(Task.java:515)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:353)
> at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:396)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121)
> at org.apache.hadoop.mapred.Child.main(Child.java:249)
> Caused by: java.lang.RuntimeException: java.lang.ClassNotFoundException:
> test.giraph.SimpleShortestPathsVertex$SimpleShortestPathsVertexOutputFormat
> at
> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:867)
> at
> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:891)
> ... 9 more
> Caused by: java.lang.ClassNotFoundException:
> test.giraph.SimpleShortestPathsVertex$SimpleShortestPathsVertexOutputFormat
> at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:247)
> at
> org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:820)
> at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:865)
> ... 10 more
>
>
>
> I have checked inside the test/giraph folder of the test.jar file. It
> contains the class
> SimpleShortestPathsVertex$SimpleShortestPathsVertexOutputFormat.
> Any help will be appreciated.
>
> Regards,
> Kaushik
>
>
>
> --
> martin
>
>
>
