[
http://issues.apache.org/jira/browse/NUTCH-406?page=comments#action_12452285 ]
Chris A. Mattmann commented on NUTCH-406:
-----------------------------------------
Hi Doğacan,
Loooking at your latest patch, I'm not sure that it completely does the right
behavior. For example, what happens if there are 3 met values for a key k, and
one of them is null, but the other 2 are not? Specifically, what if the first
value is null, but the other 2 are not. In that case, your patch would skip
over writing all of the keys. Wouldn't it just be easier to do something like
this?
Index: src/java/org/apache/nutch/metadata/Metadata.java
===================================================================
--- src/java/org/apache/nutch/metadata/Metadata.java (revision 478613)
+++ src/java/org/apache/nutch/metadata/Metadata.java (working copy)
@@ -211,7 +211,9 @@
values = getValues(names[i]);
out.writeInt(values.length);
for (int j = 0; j < values.length; j++) {
- Text.writeString(out, values[j]);
+ if(values[j] != null && !values[j].equals("")){
+ Text.writeString(out, values[j]);
+ }
}
}
}
> Metadata tries to write null values
> -----------------------------------
>
> Key: NUTCH-406
> URL: http://issues.apache.org/jira/browse/NUTCH-406
> Project: Nutch
> Issue Type: Bug
> Affects Versions: 0.9.0
> Reporter: Doğacan Güney
> Assigned To: Chris A. Mattmann
> Attachments: NUTCH-406.patch, NUTCH-406.patch
>
>
> During parsing, some urls (especially pdfs, it seems) may create <some_key,
> null> pairs in ParseData's parseMeta.
> When Metadata.write() tries to write such a pair, it causes an NPE.
> Stack trace will be something like this:
> at org.apache.hadoop.io.Text.encode(Text.java:373)
> at org.apache.hadoop.io.Text.encode(Text.java:354)
> at org.apache.hadoop.io.Text.writeString(Text.java:394)
> at org.apache.nutch.metadata.Metadata.write(Metadata.java:214)
> I can consistently reproduce this using the following url:
> http://www.efesbev.com/corporate_governance/pdf/MergerAgreement.pdf
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
_______________________________________________
Nutch-developers mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/nutch-developers
