Finally I was able to locate the logs in syslogs , I see the error it is because of the following exception.
java.lang.NoSuchMethodError: org.apache.http.impl.conn.SchemeRegistryFactory.createSystemDefault()Lorg/apache/http/conn/scheme/SchemeRegistry; Apparently this happens when an older version of the class is present in the classpath. I have added a fat jar to my nutch build to access the database and insert data into it along with other functions , looks like one of the jars that is included in my fat jar is conflicting with the Hadoop jars. Once I figured out which jar it is , I will need to make sure that only the right jar is in the class path. Is there a way to configure class loading policy when using Hadoop ? On Tue, Dec 10, 2013 at 7:47 PM, S.L <simpleliving...@gmail.com> wrote: > Tejas, > > I have looked at the Hadoop UI and under tools there is a 'Local logs' > link and under it there is a 'user logs linksand under it there is a > container_1386136488805_0971_01_000003 directory( these looks like job > specific logs) under each of these directories there are three log files. > > stderr 783 bytes Dec 4, 2013 9:08:59 AM > stdout 0 bytes Dec 4, 2013 9:08:59 AM > syslog 12300 bytes Dec 4, 2013 9:09:03 AM > > > > I have checked the syslog file it only has log files from Hadoop and > nothing from Nutch. > > Is there is a log4j property that is missing or am I not looking at the > right location for the Nutch specific logs ? Please advise. > > > On Mon, Dec 9, 2013 at 6:22 PM, Tejas Patil <tejas.patil...@gmail.com>wrote: > >> >> I am not able to locate the logs either to confirm that. Can you please >> let me know how to retrieve logs from Nutch on Hadoop? >> You should have seen the jobs over Hadoop UI and for each mapper and >> reducer you can view the logs over the browser by clicking on the job. >> >> >> YARN supposed to be backward compatible? >> It is. Nutch has not migrated fully to support the new MapReduce API from >> which YARN was sprung. >> >> >> On Mon, Dec 9, 2013 at 3:11 PM, S.L <simpleliving...@gmail.com> wrote: >> >> > Isnt ,YARN supposed to be backward compatible? >> > >> > Sent from my HTC Inspire™ 4G on AT&T >> > >> > ----- Reply message ----- >> > From: "Julien Nioche" <lists.digitalpeb...@gmail.com> >> > To: "user@nutch.apache.org" <user@nutch.apache.org> >> > Cc: "d...@nutch.apache.org" <d...@nutch.apache.org> >> > Subject: Nutch with YARN (aka Hadoop 2.0) >> > Date: Mon, Dec 9, 2013 3:54 am >> > >> > >> > I don't think Nutch has been fully ported to the new mapreduce API >> which is >> > a prerequisite for running it on Hadoop 2. >> > I can't think of a reason why that the performance would be any >> different >> > with Yarn. >> > >> > Julien >> > >> > >> > On 9 December 2013 06:42, Tejas Patil <tejas.patil...@gmail.com> wrote: >> > >> > > Has anyone tried out running Nutch over YARN ? If so, were there were >> any >> > > performance gains with the same ? >> > > >> > > Thanks, >> > > Tejas >> > > >> > >> > >> > >> > -- >> > >> > Open Source Solutions for Text Engineering >> > >> > http://digitalpebble.blogspot.com/ >> > http://www.digitalpebble.com >> > http://twitter.com/digitalpebble >> > >> > >
