The message which was appearing in the logs is pasted below.
Basically, in org.apache.nutch.crawl.MapWritable#getKeyValueEntry the
Writable is instantiated. It's class is determined by a two byte code
(which is written to crawldb I guess), if there is no entry for the
class it fails to create it, regardless of if it's a Writable. You're
right in that it can potentially handle any Writabl object, but only
if it has a maping for it's class.
Cheers
Rob
07/07/25 11:52:00 WARN crawl.MapWritable: Unable to load meta data
entry, ignoring.. : java.io.IOException: unable to load class for id:
36
On 7/25/07, Doğacan Güney (JIRA) <[EMAIL PROTECTED]> wrote:
[
https://issues.apache.org/jira/browse/NUTCH-527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12515283
]
Doğacan Güney commented on NUTCH-527:
-------------------------------------
What was the error you were having? MapWritable supports reading and writing *all*
writables. The ones defined at the top of the file are an optimization and shouldn't
affect correctness (basically mapwritable first writes a byte and the associated
classname, then writes that byte to indicate classname everywhere else. For commonly used
types we statically define the association so that "first write the byte then the
classname" phase is not necessary).
> MapWritable doesn't support all hadoops writable types
> ------------------------------------------------------
>
> Key: NUTCH-527
> URL: https://issues.apache.org/jira/browse/NUTCH-527
> Project: Nutch
> Issue Type: Bug
> Affects Versions: 0.9.0
> Environment: Tested on Solaris and Windows with Java 1.5
> Reporter: Rob Young
> Attachments: mapwritable.patch
>
>
> The map of classes which implement org.apache.hadoop.io.Writable is not
complete. It does not, for example, include org.apache.hadoop.io.BooleanWritable.
I would happily provide a patch if someone would explain what the Byte parameter
is.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
