Author: dogacan Date: Wed Jun 27 05:46:05 2007 New Revision: 551147 URL: http://svn.apache.org/viewvc?view=rev&rev=551147 Log: NUTCH-498 - Use Combiner in LinkDb to increase speed of linkdb generation. Contributed by Espen Amble Kolstad.
Modified: lucene/nutch/trunk/CHANGES.txt lucene/nutch/trunk/src/java/org/apache/nutch/crawl/LinkDb.java Modified: lucene/nutch/trunk/CHANGES.txt URL: http://svn.apache.org/viewvc/lucene/nutch/trunk/CHANGES.txt?view=diff&rev=551147&r1=551146&r2=551147 ============================================================================== --- lucene/nutch/trunk/CHANGES.txt (original) +++ lucene/nutch/trunk/CHANGES.txt Wed Jun 27 05:46:05 2007 @@ -72,6 +72,9 @@ 23. NUTCH-499 - Refactor LinkDb and LinkDbMerger to reuse code. (dogacan) +24. NUTCH-498 - Use Combiner in LinkDb to increase speed of linkdb generation. + (Espen Amble Kolstad via dogacan) + Release 0.9 - 2007-04-02 1. Changed log4j confiquration to log to stdout on commandline Modified: lucene/nutch/trunk/src/java/org/apache/nutch/crawl/LinkDb.java URL: http://svn.apache.org/viewvc/lucene/nutch/trunk/src/java/org/apache/nutch/crawl/LinkDb.java?view=diff&rev=551147&r1=551146&r2=551147 ============================================================================== --- lucene/nutch/trunk/src/java/org/apache/nutch/crawl/LinkDb.java (original) +++ lucene/nutch/trunk/src/java/org/apache/nutch/crawl/LinkDb.java Wed Jun 27 05:46:05 2007 @@ -215,6 +215,7 @@ job.setInputFormat(SequenceFileInputFormat.class); job.setMapperClass(LinkDb.class); + job.setCombinerClass(LinkDbMerger.class); // if we don't run the mergeJob, perform normalization/filtering now if (normalize || filter) { try { ------------------------------------------------------------------------- This SF.net email is sponsored by DB2 Express Download DB2 Express C - the FREE version of DB2 express and take control of your XML. No limits. Just data. Click to get it now. http://sourceforge.net/powerbar/db2/ _______________________________________________ Nutch-cvs mailing list Nutch-cvs@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/nutch-cvs