Nutch should delegate compression to Hadoop
-------------------------------------------
Key: NUTCH-506
URL: https://issues.apache.org/jira/browse/NUTCH-506
Project: Nutch
Issue Type: Improvement
Reporter: Doğacan Güney
Fix For: 1.0.0
Some data structures within nutch (such as Content, ParseText) handle their own
compression. We should delegate all compressions to Hadoop.
Also, nutch should respect io.seqfile.compression.type setting. Currently even
if io.seqfile.compression.type is BLOCK or RECORD, nutch overrides it for some
structures and sets it to NONE (However, IMO, ParseText should always be
compressed as RECORD because of performance reasons).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
-------------------------------------------------------------------------
This SF.net email is sponsored by DB2 Express
Download DB2 Express C - the FREE version of DB2 express and take
control of your XML. No limits. Just data. Click to get it now.
http://sourceforge.net/powerbar/db2/
_______________________________________________
Nutch-developers mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/nutch-developers
