Nutch Newbie wrote: > Well, I would like to agree with Piotr here but current development > i.e. 0.8 > version and onwards single machine nutch install is not optimal there > are various > hadoop related issue example > > http://issues.apache.org/jira/browse/HADOOP-206
Is it really still a valid issue? I'm pretty sure this was already fixed, or perhaps it was a matter of putting hard limits in hadoop-site.xml (which overrides even job.xml values). > The problem regarding 0.8 being slow on single machine is nothing new > just search the > mailing list you will find many example for it. 0.8 was released > earlier this year and the > problem is still not solved so I am sorry to be negative but I am just > stating facts. What Nutch needs at this moment is more developers and contributors. This and similar issues might be solved by directly addressing each problem, if we had human resources to do so. As it is now, there are few active Nutch developers at the moment, and issues are being addressed slower than we would wish it. (BTW, Chris Mattmann will be joining the committers group, so you can expect some improvements in this regard). But what Piotr stated is that use cases such as yours _are_ important to us, and this problem will be fixed sooner or later, whenever we have free resources to do it. If you can help us with debugging and testing, and providing patches, this process will be much quicker. I suspect that we (Nutch community) are the only serious user of Hadoop in local mode - most development efforts in Hadoop project are geared towards supporting massive clusters and not single machines. So, I would say it's up to us - the Nutch community - to provide sufficient feedback to Hadoop to have such issues addressed. -- Best regards, Andrzej Bialecki <>< ___. ___ ___ ___ _ _ __________________________________ [__ || __|__/|__||\/| Information Retrieval, Semantic Web ___|||__|| \| || | Embedded Unix, System Integration http://www.sigram.com Contact: info at sigram dot com ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 _______________________________________________ Nutch-general mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/nutch-general
