[
https://issues.apache.org/jira/browse/NUTCH-620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12579243#action_12579243
]
Andrzej Bialecki commented on NUTCH-620:
-----------------------------------------
Is this an often occurring problem? I'm wary of extending the rules in basic
normalizer, because they will affect a lot of users, and running each new rule
adds certain cost in terms of performance and possible lock-ups of the regex
engine. If this is a problem experienced by many users, we should add this rule
to the default rule set, but if it's specific just to our environment I think
we shouldn't add it.
> BasicURLNormalizer should collapse runs of slashes with a single slash
> ----------------------------------------------------------------------
>
> Key: NUTCH-620
> URL: https://issues.apache.org/jira/browse/NUTCH-620
> Project: Nutch
> Issue Type: Bug
> Components: fetcher
> Affects Versions: 0.9.0
> Environment: JDK 1.6 update 5, Tomcat 6, Windows Server 2003,
> Reporter: Mark DeSpain
> Priority: Minor
> Fix For: 1.0.0
>
> Original Estimate: 0.5h
> Remaining Estimate: 0.5h
>
> The BasicURLNormalizer should collapse runs of slash characters '/' with a
> single slash.
> For example, the following URLs should be normalized to
> http://lucene.apache.org/nutch/about.html
> * http://lucene.apache.org/nutch//about.html
> * http://lucene.apache.org//nutch/about.html
> * http://lucene.apache.org/////nutch////about.html (an exaggerated example)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
