qi wu wrote:
>
> I wonder why so much space are required by linkdb reduce job, can I
> config some nutch or hadoop setting to reduce the disk space usage
> for linkdb? Any hints for me to overcome the problem? //bow
* what is you limit of links per url (db.max.inlinks property)? You
could try lowering it.
* Please try the following modification: somewhere around
LinkDb.java:283 add the following line:
job.setCombinerClass(LinkDb.class);
Recompile and re-run.
* Also, as others suggested, you may want to turn on compression.
--
Best regards,
Andrzej Bialecki <><
___. ___ ___ ___ _ _ __________________________________
[__ || __|__/|__||\/| Information Retrieval, Semantic Web
___|||__|| \| || | Embedded Unix, System Integration
http://www.sigram.com Contact: info at sigram dot com
-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
_______________________________________________
Nutch-general mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/nutch-general
