[ https://issues.apache.org/jira/browse/SOLR-1404?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12753779#action_12753779 ]
Jason Rutherglen commented on SOLR-1404: ---------------------------------------- It will be good to get this fixed, I have experienced problems in analyzing because of the bug and reverted back to HTMLStripReader. > Random failures with highlighting > --------------------------------- > > Key: SOLR-1404 > URL: https://issues.apache.org/jira/browse/SOLR-1404 > Project: Solr > Issue Type: Bug > Components: Analysis, highlighter > Affects Versions: 1.4 > Reporter: Anders Melchiorsen > Fix For: 1.4 > > Attachments: SOLR-1404.patch > > > With a recent Solr nightly, we started getting errors when highlighting. > I have not been able to reduce our real setup to a minimal one that is > failing, but the same error seems to pop up with the configuration below. > Note that the QUERY will mostly fail, but it will work sometimes. Notably, > after running "java -jar start.jar", the QUERY will work the first time, but > then start failing for a while. Seems that something is not being reset > properly. > The example uses the deprecated HTMLStripWhitespaceTokenizerFactory but the > problem apparently also exists with other tokenizers; I was just unable to > create a minimal example with other configurations. > SCHEMA > <?xml version="1.0" encoding="UTF-8" ?> > <schema name="example" version="1.2"> > <types> > <fieldType name="string" class="solr.StrField" /> > <fieldtype name="testtype" class="solr.TextField"> > <analyzer> > <tokenizer class="solr.HTMLStripWhitespaceTokenizerFactory" /> > </analyzer> > </fieldtype> > </types> > <fields> > <field name="id" type="string" indexed="true" stored="false" /> > <field name="test" type="testtype" indexed="false" stored="true" /> > </fields> > <uniqueKey>id</uniqueKey> > </schema> > INDEX > URL=http://localhost:8983/solr/update > curl $URL --data-binary '<add><doc><field name="id">1</field><field > name="test">test</field></doc></add>' -H 'Content-type:text/xml; > charset=utf-8' > curl $URL --data-binary '<commit/>' -H 'Content-type:text/xml; charset=utf-8' > QUERY > curl 'http://localhost:8983/solr/select/?hl.fl=test&hl=true&q=id:1' > ERROR > org.apache.lucene.search.highlight.InvalidTokenOffsetsException: Token test > exceeds length of provided text sized 4 > org.apache.solr.common.SolrException: > org.apache.lucene.search.highlight.InvalidTokenOffsetsException: Token test > exceeds length of provided text sized 4 > at > org.apache.solr.highlight.DefaultSolrHighlighter.doHighlighting(DefaultSolrHighlighter.java:328) > at > org.apache.solr.handler.component.HighlightComponent.process(HighlightComponent.java:89) > at > org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:195) > at > org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:131) > at org.apache.solr.core.SolrCore.execute(SolrCore.java:1299) > at > org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:338) > at > org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:241) > at > org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1089) > at > org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:365) > at > org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216) > at > org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:181) > at > org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:712) > at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:405) > at > org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:211) > at > org.mortbay.jetty.handler.HandlerCollection.handle(HandlerCollection.java:114) > at > org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:139) > at org.mortbay.jetty.Server.handle(Server.java:285) > at > org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:502) > at > org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:821) > at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:513) > at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:208) > at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:378) > at > org.mortbay.jetty.bio.SocketConnector$Connection.run(SocketConnector.java:226) > at > org.mortbay.thread.BoundedThreadPool$PoolThread.run(BoundedThreadPool.java:442) > Caused by: org.apache.lucene.search.highlight.InvalidTokenOffsetsException: > Token test exceeds length of provided text sized 4 > at > org.apache.lucene.search.highlight.Highlighter.getBestTextFragments(Highlighter.java:254) > at > org.apache.solr.highlight.DefaultSolrHighlighter.doHighlighting(DefaultSolrHighlighter.java:321) > ... 23 more -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.