[
https://issues.apache.org/jira/browse/NUTCH-1647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14030426#comment-14030426
]
Julien Nioche edited comment on NUTCH-1647 at 6/13/14 10:58 AM:
----------------------------------------------------------------
Thanks Markus. Looks like the same error is caused by various reasons.
In the case of this URL, we are getting this because the byte[] content is
empty when passed to processGzipEncoding. It is empty because the HTTP metadata
returned by the server specify a content length of 0, which makes sense as this
page is a redirection.
I'll attach a patch which should fix this issue.
was (Author: jnioche):
Thanks Markus. Looks like the same error is caused by various reasons. Could be
related to NUTCH-1270, I'll have a look.
> protocol-http throws unzipBestEffort returned null for some pages
> -----------------------------------------------------------------
>
> Key: NUTCH-1647
> URL: https://issues.apache.org/jira/browse/NUTCH-1647
> Project: Nutch
> Issue Type: Bug
> Components: protocol
> Affects Versions: 1.7
> Reporter: Markus Jelsma
> Fix For: 1.9
>
>
> bin/nutch indexchecker
> http://www.provinciegroningen.nl/actueel/dossiers/rwe-centrale
> Fetch failed with protocol status: exception(16), lastModified=0:
> java.io.IOException: unzipBestEffort returned null
> {code}
> 2013-10-01 13:44:55,612 INFO http.Http - http.proxy.host = null
> 2013-10-01 13:44:55,612 INFO http.Http - http.proxy.port = 8080
> 2013-10-01 13:44:55,612 INFO http.Http - http.timeout = 12000
> 2013-10-01 13:44:55,612 INFO http.Http - http.content.limit = 5242880
> 2013-10-01 13:44:55,612 INFO http.Http - http.agent = Mozilla/5.0
> (compatible; OpenindexSpider;
> +http://www.openindex.io/en/webmasters/spider.html)
> 2013-10-01 13:44:55,612 INFO http.Http - http.accept.language =
> en-us,en-gb,en;q=0.7,*;q=0.3
> 2013-10-01 13:44:55,613 INFO http.Http - http.accept =
> text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
> 2013-10-01 13:44:55,925 ERROR http.Http - Failed to get protocol output
> java.io.IOException: unzipBestEffort returned null
> at
> org.apache.nutch.protocol.http.api.HttpBase.processGzipEncoded(HttpBase.java:317)
> at
> org.apache.nutch.protocol.http.HttpResponse.<init>(HttpResponse.java:164)
> at org.apache.nutch.protocol.http.Http.getResponse(Http.java:64)
> at
> org.apache.nutch.protocol.http.api.HttpBase.getProtocolOutput(HttpBase.java:140)
> at
> org.apache.nutch.indexer.IndexingFiltersChecker.run(IndexingFiltersChecker.java:86)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
> at
> org.apache.nutch.indexer.IndexingFiltersChecker.main(IndexingFiltersChecker.java:150)
> {code}
> Haven't got a clue yet as to what the exact issue is.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
