[
https://issues.apache.org/jira/browse/NUTCH-1736?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13937426#comment-13937426
]
lufeng commented on NUTCH-1736:
-------------------------------
Hi ysc
you can check the content size to fix this issue like this.
{code:java}
if (http.getMaxContent() >= 0 && (contentBytesRead + chunkLen) >
http.getMaxContent() )
chunkLen= http.getMaxContent() - contentBytesRead;
{code}
> Can't fetch page if http response header contains Transfer-Encoding：chunked
> ---------------------------------------------------------------------------
>
> Key: NUTCH-1736
> URL: https://issues.apache.org/jira/browse/NUTCH-1736
> Project: Nutch
> Issue Type: Bug
> Components: protocol
> Affects Versions: 1.6, 2.1, 1.7, 2.2, 2.3, 1.8, 2.4, 1.9, 2.2.1
> Reporter: ysc
> Priority: Critical
> Fix For: 2.3, 1.9
>
> Attachments: nutch-2.2.1.patch, nutch1.7.patch
>
> Original Estimate: 24h
> Remaining Estimate: 24h
>
> fetching:
> http://szs.mof.gov.cn/zhengwuxinxi/zhengcefabu/201402/t20140224_1046354.html
> Fetch failed with protocol status: EXCEPTION: java.io.IOException:
> unzipBestEffort returned null
--
This message was sent by Atlassian JIRA
(v6.2#6252)
