: But we do not have an inbuilt TokenFilter which does that. Nor does : DIH support it now . I have opened an issue for DIH : (https://issues.apache.org/jira/browse/SOLR-980) : Is it desirable to have TokenFilter which offers similar functionality?
Probably not (you would have to have a way of configuring what kind of analysis would be done on the file) My point was specificly about the original posters use case: he said he already had a TokenFilter that parsed the URL target the way he wanted -- in which case it's easy for him to to keep using that TokenFilter by writing a factory for it. -Hoss
