[ https://issues.apache.org/jira/browse/NUTCH-1098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Radim Kolar updated NUTCH-1098: ------------------------------- Attachment: nutch.diff Updated patch. It also normalizes unprintable % sequences to upper case. Like %1f -> %1F . upper case seems to be way more common. > better url-normalizer basic > --------------------------- > > Key: NUTCH-1098 > URL: https://issues.apache.org/jira/browse/NUTCH-1098 > Project: Nutch > Issue Type: Improvement > Components: fetcher > Affects Versions: 1.3 > Environment: Any > Reporter: Radim Kolar > Labels: encoding, url > Fix For: 1.4 > > Attachments: nutch.diff > > Original Estimate: 4h > Remaining Estimate: 4h > > Basic URL normalizer lacks 2 important features > Encode space in URL into %20 to unbreak httpclient and possibly others who do > not expect space inside URL > Ability to decode %33 encoding in URL. This is important for avoiding > duplicates -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira