Re: Solr 5.0 -- IllegalStateException: unexpected docvalues type NONE on result grouping
On 3/12/2015 3:36 PM, Alexandre Rafalovitch wrote: Manual optimize is no longer needed for modern Solr. It does great optimization automatically. The only reason I recommended it here is to make sure that all segments are brought up to the latest version and the deleted documents are purged. That's something that also would happen automatically eventually, but eventually was not an option for you. I am glad this helped. I am not 100% sure if you have to do it on each shard in SolrCloud mode, but I suspect so. In SolrCloud, whenever you send an optimize command to any shard replica in a collection, the entire collection will be optimized. SolrCloud will do the optimization sequentially, not in parallel. There is currently no way to optimize only one shard replica, and as far as I know, there is no way to ask for a parallel optimization. Alexandre's comments about the necessity of optimization (whether it's SolrCloud or not) is spot on. The only time that optimization should be done on a modern Solr index is when you have a lot of deleted documents and want to clean those up, either to reclaim disk space or remove them from the relevancy calculation. Most people do see a performance boost on an optimized index compared to a non-optimized index, but with a modern Solr install, you might actually see better performance on a multi-segment index when the indexing rate is high, because Lucene is moving to a model where there are per-segment caches that are not invalidated at commit time, only at merge time. Thanks, Shawn
Solr 5.0 -- IllegalStateException: unexpected docvalues type NONE on result grouping
Hi, I've a field which is being used for result grouping. Here's the field definition. field name=ADDedup type=string indexed=true stored=true multiValued=false required=false omitNorms=true docValues=true/ This started once I did a rolling update from 4.7 to 5.0. I started getting the error on any group by query -- SolrDispatchFilter null:java.lang. IllegalStateException: unexpected docvalues type NONE for field 'ADSKDedup' (expected=SORTED). Use UninvertingReader or dex with docvalues. Does this mean that I need to re-index documents to get over this error ? Regards, Shamik
Re: Solr 5.0 -- IllegalStateException: unexpected docvalues type NONE on result grouping
On 3/11/2015 4:45 PM, shamik wrote: field name=DocumentType type=string indexed=true stored=true multiValued=false required=false omitNorms=true docValues=true / 3/11/2015, 2:14:30 PM ERROR SolrDispatchFilter null:java.lang.IllegalStateException: unexpected docvalues type NONE for field 'DocumentType' (expected=SORTED). Use UninvertingReader or index with docvalues. null:java.lang.IllegalStateException: unexpected docvalues type NONE for field 'DocumentType' (expected=SORTED). Use UninvertingReader or index with docvalues. I admit right up front that I know very little about what might be happening here ... but I did have one idea. It could be completely wrong. Is it possible that you have an index that fits the following description? The field originally did not have docValues. You enabled docValues on the field in the schema, but there are index segments still in the index directory from *before* you changed the schema. If that sounds at all possible, then if you did not fully reindex, there would be segments with valid documents that do not have docValues. You should fully reindex and then optimize before upgrading. If you did fully reindex, but did not optimize, then there might be segments with *deleted* documents that do not have docValues ... and maybe 4.7 was fine with that but 5.0 isn't. Whenever I upgrade Solr, I always reindex from scratch, and often I will completely delete all the data directories. It takes longer, but then I know the index is 100% correct for the version and config I'm running. I'll reiterate that this whole idea could be 100% wrong. Thanks, Shawn
Re: Solr 5.0 -- IllegalStateException: unexpected docvalues type NONE on result grouping
Manual optimize is no longer needed for modern Solr. It does great optimization automatically. The only reason I recommended it here is to make sure that all segments are brought up to the latest version and the deleted documents are purged. That's something that also would happen automatically eventually, but eventually was not an option for you. I am glad this helped. I am not 100% sure if you have to do it on each shard in SolrCloud mode, but I suspect so. Regards, Alex. Solr Analyzers, Tokenizers, Filters, URPs and even a newsletter: http://www.solr-start.com/ On 12 March 2015 at 17:24, shamik sham...@gmail.com wrote: Wow, optimize worked like a charm. This really addressed the docvalues issue. A follow-up question, is it recommended to run optimize in a Production Solr index ? Also, in a Sorl cloud mode, do we need to run optimize on each instance / each shard / any instance ? Appreciate your help Alex. -- View this message in context: http://lucene.472066.n3.nabble.com/Solr-5-0-migration-IllegalStateException-unexpected-docvalues-type-NONE-on-fields-using-docvalues-tp4192477p4192732.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Solr 5.0 -- IllegalStateException: unexpected docvalues type NONE on result grouping
Wow, optimize worked like a charm. This really addressed the docvalues issue. A follow-up question, is it recommended to run optimize in a Production Solr index ? Also, in a Sorl cloud mode, do we need to run optimize on each instance / each shard / any instance ? Appreciate your help Alex. -- View this message in context: http://lucene.472066.n3.nabble.com/Solr-5-0-migration-IllegalStateException-unexpected-docvalues-type-NONE-on-fields-using-docvalues-tp4192477p4192732.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Solr 5.0 -- IllegalStateException: unexpected docvalues type NONE on result grouping
Do you have any really old segments in that index? Could be worth trying to optimize them down to one in latest format first. Like Shawn, this is just a one more idea proposal. Regards, Alex. Solr Analyzers, Tokenizers, Filters, URPs and even a newsletter: http://www.solr-start.com/ On 12 March 2015 at 14:47, shamik sham...@gmail.com wrote: Well, I think I've narrowed down the issue. The error is happening when I'm trying to do a rolling update from Solr 4.7 (which is our current version) to 5.0 . I'm able to re-produce this couple of times. If I do a fresh index on a 5.0, it works. Not sure if there's any other way to mitigate it. I'll appreciate if someone can share their experience on the same. -- View this message in context: http://lucene.472066.n3.nabble.com/Solr-5-0-migration-IllegalStateException-unexpected-docvalues-type-NONE-on-fields-using-docvalues-tp4192477p4192706.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Solr 5.0 -- IllegalStateException: unexpected docvalues type NONE on result grouping
Well, I think I've narrowed down the issue. The error is happening when I'm trying to do a rolling update from Solr 4.7 (which is our current version) to 5.0 . I'm able to re-produce this couple of times. If I do a fresh index on a 5.0, it works. Not sure if there's any other way to mitigate it. I'll appreciate if someone can share their experience on the same. -- View this message in context: http://lucene.472066.n3.nabble.com/Solr-5-0-migration-IllegalStateException-unexpected-docvalues-type-NONE-on-fields-using-docvalues-tp4192477p4192706.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Solr 5.0 -- IllegalStateException: unexpected docvalues type NONE on result grouping
Looks like it's happening for any field which is using docvalues. java.lang.IllegalStateException: unexpected docvalues type NONE for field 'title_sort' (expected=SORTED). Use UninvertingReader or index with docvalues. Any idea ? -- View this message in context: http://lucene.472066.n3.nabble.com/Solr-5-0-IllegalStateException-unexpected-docvalues-type-NONE-on-result-grouping-tp4192477p4192529.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Solr 5.0 -- IllegalStateException: unexpected docvalues type NONE on result grouping
1) the error message you posted doesn't appear to have been copied verbatim (note: ...or dex...) ... please proivde the *exact* error you are getting -- ideally with full stack trace from the solr logs. 2) the field/ you posted doens't match the field name in your error message 3)providing a field/ w/o the fieldType/ makes it hard to guess exactly what settings you are using (ie: is docValuesFormat specified on the fieldType?) 4) You said you are using result grouping, but you haven't actually shown us an example of what your request (with all params) looks like ... there's nohting in your email to even indicate grouping is involved. Details matter... https://wiki.apache.org/solr/UsingMailingLists (PS: assuming no obvious mistakes, it would be helpful to know if you can reproduce this problem using your same exact same configs schema with a newly built 5.0 index, or if it *only* affects you when you upgrade and use an existing index from 4.7) : Date: Wed, 11 Mar 2015 11:02:43 -0700 : From: Shamik Bandopadhyay sham...@gmail.com : Reply-To: solr-user@lucene.apache.org : To: solr-user@lucene.apache.org : Subject: Solr 5.0 -- IllegalStateException: unexpected docvalues type NONE : on result grouping : : Hi, : :I've a field which is being used for result grouping. Here's the field : definition. : : field name=ADDedup type=string indexed=true stored=true : multiValued=false required=false omitNorms=true docValues=true/ : : This started once I did a rolling update from 4.7 to 5.0. I started getting : the error on any group by query -- SolrDispatchFilter null:java.lang. : IllegalStateException: unexpected docvalues type NONE for field 'ADSKDedup' : (expected=SORTED). Use UninvertingReader or dex with docvalues. : : Does this mean that I need to re-index documents to get over this error ? : : Regards, : Shamik : -Hoss http://www.lucidworks.com/
Re: Solr 5.0 -- IllegalStateException: unexpected docvalues type NONE on result grouping
Thanks for your reply. Initially, I was under the impression that the issue is related to grouping as group queries were failing. Later, when I looked further, I found that it's happening for any field for which the docvalue has turned on. The second example I took was from another field. Here's a full stack trace for another field using docvalues. Field definition : field name=DocumentType type=string indexed=true stored=true multiValued=false required=false omitNorms=true docValues=true / 3/11/2015, 2:14:30 PM ERROR SolrDispatchFilter null:java.lang.IllegalStateException: unexpected docvalues type NONE for field 'DocumentType' (expected=SORTED). Use UninvertingReader or index with docvalues. null:java.lang.IllegalStateException: unexpected docvalues type NONE for field 'DocumentType' (expected=SORTED). Use UninvertingReader or index with docvalues. at org.apache.lucene.index.DocValues.checkField(DocValues.java:208) at org.apache.lucene.index.DocValues.getSorted(DocValues.java:264) at org.apache.lucene.search.FieldComparator$TermOrdValComparator.getSortedDocValues(FieldComparator.java:757) at org.apache.lucene.search.FieldComparator$TermOrdValComparator.getLeafComparator(FieldComparator.java:762) at org.apache.lucene.search.FieldValueHitQueue.getComparators(FieldValueHitQueue.java:183) at org.apache.lucene.search.TopFieldCollector$NonScoringCollector.getLeafCollector(TopFieldCollector.java:141) at org.apache.lucene.search.MultiCollector.getLeafCollector(MultiCollector.java:99) at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:583) at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:284) at org.apache.solr.search.SolrIndexSearcher.buildAndRunCollectorChain(SolrIndexSearcher.java:231) at org.apache.solr.search.SolrIndexSearcher.getDocListAndSetNC(SolrIndexSearcher.java:1766) at org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:1502) at org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:586) at org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:511) at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:227) at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:144) at org.apache.solr.core.SolrCore.execute(SolrCore.java:2006) at org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:777) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:413) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:204) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1419) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:455) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:557) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1075) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:384) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1009) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:255) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:154) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116) at org.eclipse.jetty.server.Server.handle(Server.java:368) at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:489) at org.eclipse.jetty.server.BlockingHttpConnection.handleRequest(BlockingHttpConnection.java:53) at org.eclipse.jetty.server.AbstractHttpConnection.content(AbstractHttpConnection.java:953) at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.content(AbstractHttpConnection.java:1014) at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:861) at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:240) at org.eclipse.jetty.server.BlockingHttpConnection.handle(BlockingHttpConnection.java:72) at org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.run(SocketConnector.java:264) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608) at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
