Thanks Kaushik, it's worked me well. Sent from my iPad
On Aug 1, 2012, at 11:20 PM, KAUSHIK SARKAR <countkaus...@gmail.com> wrote: > Hi, > > I found an alternative programmatic solution to the problem. > We need to modify the run() method in the following way - > > ... > @Override > public int run(String[] argArray) throws Exception { > Preconditions.checkArgument(argArray.length == 4, > "run: Must have 4 arguments <input path> <output path> " + > "<source vertex id> <# of workers>"); > > GiraphJob job = new GiraphJob(getConf(), getClass().getName()); > // This is the addition - it will make hadoop look for other classes > in the same jar that contains this class > job.getInternalJob().setJarByClass(getClass()); > job.setVertexClass(getClass()); > ... > } > > After this hadoop will look for missing classes in the same jar that contains > the class returned by getClass(), and we will not need to add the job jar > name separately to the -libjars option (as described by Marcin's > stackoverflow post). > > I think Giraph may add a constructor like - > > GiraphJob(Configuration conf, Class jobClass, String jobName) { > this(conf, jobName); > this.getInternalJob().setJarByClass(jobClass); > } > > in tradition of the Hadoop constructor - > > public JobConf(Configuration conf, Class exampleClass) { > this(conf); > setJarByClass(exampleClass); > } > > It will solve the rather annoying class-not-found problem. > > Regards, > Kaushik > > On Tue, Jul 24, 2012 at 11:31 AM, KAUSHIK SARKAR <countkaus...@gmail.com> > wrote: > Hi, > > Marcin's solution worked for me. Thanks! > > I didn't try out addFileToClassPath solution, but I think it should work. > Thanks for the suggestion! > > As Marcin has already suggested, it is not very much clear why we have to do > it like this and why the normal call to hadoop jar doesn't work. I guess it > has more to do with the way Java (or Hadoop) loads class files than Giraph. > However, I am a total novice in Java and would like to know the reason. > > N.B. I tried with the latest development snapshot of the Giraph (giraph-0.2) > , and normal call to hadoop jar didn't work with it as well (of course > Marcin's solution worked). > > Regards, > Kaushik > > On Tue, Jul 24, 2012 at 6:22 AM, Etienne Dumoulin > <etienne.dumou...@idiro.com> wrote: > Hi, > > I cannot explain it. > But if it any use, for making it working without specifying the library in > command line you can use the distributed cache: > DistributedCache#addFileToClassPath > > Regards, > > Étienne > > > On 24 July 2012 13:37, Marcin Biczak <marcinbic...@gmail.com> wrote: > Hi > > I had similar problem, here is "my solution", > http://stackoverflow.com/questions/10700853/giraph-shortest-paths-example-classnotfoundexception. > But I still don't know why I have do it like this, maybe someone here can > explain this? > > regards > marcin biczak > > > 2012/7/24 KAUSHIK SARKAR <countkaus...@gmail.com> > Hi, > > I created a new test project and copied the code from > SimpleShortestPathsVertex.java file (org.apache.giraph.examples) into it. But > when I run it, it gives me java.lang.ClassNotFoundException. However, if I > run the same program from the giraph-0.1-jar-with-dependencies.jar, it runs > just fine. Following is the relevant information : > > Hadoop version: hadoop-1.0.3 (running in pseudo distributed mode) > giraph version: giraph-0.1 > jar created with Eclipse 3.3.1 > Output: > > kaushik@kaushik-laptop:~/hadoop/hadoop-1.0.3$ bin/hadoop jar ../jars/test.jar > /shortestPathsInputGraph /shortestPathsOutputGraph 0 3 > 12/07/23 18:29:10 INFO mapred.JobClient: Running job: job_201207231222_0021 > 12/07/23 18:29:11 INFO mapred.JobClient: map 0% reduce 0% > 12/07/23 18:29:20 INFO mapred.JobClient: Task Id : > attempt_201207231222_0021_m_000005_0, Status : FAILED > java.lang.RuntimeException: java.lang.RuntimeException: > java.lang.ClassNotFoundException: > test.giraph.SimpleShortestPathsVertex$SimpleShortestPathsVertexOutputFormat > at > org.apache.hadoop.conf.Configuration.getClass(Configuration.java:899) > at > org.apache.giraph.graph.BspUtils.getVertexOutputFormatClass(BspUtils.java:134) > at > org.apache.giraph.bsp.BspOutputFormat.getOutputCommitter(BspOutputFormat.java:56) > at org.apache.hadoop.mapred.Task.initialize(Task.java:515) > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:353) > at org.apache.hadoop.mapred.Child$4.run(Child.java:255) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:396) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121) > at org.apache.hadoop.mapred.Child.main(Child.java:249) > Caused by: java.lang.RuntimeException: java.lang.ClassNotFoundException: > test.giraph.SimpleShortestPathsVertex$SimpleShortestPathsVertexOutputFormat > at > org.apache.hadoop.conf.Configuration.getClass(Configuration.java:867) > at > org.apache.hadoop.conf.Configuration.getClass(Configuration.java:891) > ... 9 more > Caused by: java.lang.ClassNotFoundException: > test.giraph.SimpleShortestPathsVertex$SimpleShortestPathsVertexOutputFormat > at java.net.URLClassLoader$1.run(URLClassLoader.java:202) > at java.security.AccessController.doPrivileged(Native Method) > at java.net.URLClassLoader.findClass(URLClassLoader.java:190) > at java.lang.ClassLoader.loadClass(ClassLoader.java:306) > at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301) > at java.lang.ClassLoader.loadClass(ClassLoader.java:247) > at java.lang.Class.forName0(Native Method) > at java.lang.Class.forName(Class.java:247) > at > org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:820) > at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:865) > ... 10 more > > > > I have checked inside the test/giraph folder of the test.jar file. It > contains the class > SimpleShortestPathsVertex$SimpleShortestPathsVertexOutputFormat. > Any help will be appreciated. > > Regards, > Kaushik > > > > -- > martin > > >