Erick, I agree, it doesn't make sense. I manually merged the solrconfig.xml from the distribution example with my 3.6 solrconfig.xml, pulling out what I didn't need. There is the possibility I removed something I shouldn't have though I don't know what it would be. Minus removing the dynamic fields, a custom tokenizer class, and changing all my fields to be stored, the schema.xml file should be the same as well. I'm not currently in the position to do so, but I'll double check those two files. Finally, the data was re-indexed when I moved to 4.3.
My statement about field values wasn't stated very well. What I meant is that the 'text' field has more unique terms than some of my other fields. As for this being an edge case, I'm not sure why it would manifest itself in 4.3 but not in 3.6 (short of me having a screwy configuration setting). If I get a chance, I'll see if I can duplicate the behavior with a small document count in a sandboxed environment. Shane On Wed, Jun 12, 2013 at 5:14 PM, Erick Erickson <erickerick...@gmail.com>wrote: > This doesn't make much sense, particularly the fact > that you added first/new searchers. I'm assuming that > these are sorting on the same field as your slow query. > > But sorting on a text field for which > "Overall, the values of the field are unique" > is a red-flag. Solr doesn't sort on fields that have > more than one term, so you might as well use a > string field and be done with it, it's possible you're > hitting some edge case. > > Did you just copy your 3.6 schema and configs to > 4.3? Did you re-index? > > Best > Erick > > On Wed, Jun 12, 2013 at 5:11 PM, Shane Perry <thry...@gmail.com> wrote: > > Thanks for the responses. > > > > Setting first/newSearcher had no noticeable effect. I'm sorting on a > > stored/indexed field named 'text' who's fieldType is solr.TextField. > > Overall, the values of the field are unique. The JVM is only using about > > 2G of the available 12G, so no OOM/GC issue (at least on the surface). > The > > server is question is a slave with approximately 56 million documents. > > Additionally, sorting on a field of the same type but with significantly > > less uniqueness results quick response times. > > > > The following is a sample of *debugQuery=true* for a query which returns > 1 > > document: > > > > <lst name="process"> > > <double name="time">61458.0</double> > > <lst name="query"> > > <double name="time">61452.0</double> > > </lst> > > <lst name="facet"> > > <double name="time">0.0</double> > > </lst> > > <lst name="mlt"> > > <double name="time">0.0</double> > > </lst> > > <lst name="highlight"> > > <double name="time">0.0</double> > > </lst> > > <lst name="stats"> > > <double name="time">0.0</double> > > </lst> > > <lst name="debug"> > > <double name="time">6.0</double> > > </lst> > > </lst> > > > > > > -- Update -- > > > > Out of desperation, I turned off replication by commenting out the *<list > > name="slave">* element in the replication requestHandler block. After > > restarting tomcat I was surprised to find that the replication admin UI > > still reported the core as replicating. Search queries were still slow. > I > > then disabled replication via the UI and the display updated to report > the > > core was no longer replicating. Queries are now fast so it appears that > > the sorting may be a red-herring. > > > > It's may be of note to also mention that the slow queries don't appear to > > be getting cached. > > > > Thanks again for the feed back. > > > > On Wed, Jun 12, 2013 at 2:33 PM, Jack Krupansky <j...@basetechnology.com > >wrote: > > > >> Rerun the sorted query with &debugQuery=true and look at the module > >> timings. See what stands out > >> > >> Are you actually sorting on a "text" field, as opposed to a "string" > field? > >> > >> Of course, it's always possible that maybe you're hitting some odd > OOM/GC > >> condition as a result of Solr growing between releases. > >> > >> -- Jack Krupansky > >> > >> -----Original Message----- From: Shane Perry > >> Sent: Wednesday, June 12, 2013 3:00 PM > >> To: solr-user@lucene.apache.org > >> Subject: Sorting by field is slow > >> > >> > >> In upgrading from Solr 3.6.1 to 4.3.0, our query response time has > >> increased exponentially. After testing in 4.3.0 it appears the same > query > >> (with 1 matching document) returns after 100 ms without sorting but > takes 1 > >> minute when sorting by a text field. I've looked around but haven't yet > >> found a reason for the degradation. Can someone give me some insight or > >> point me in the right direction for resolving this? In most cases, I > can > >> change my code to do client-side sorting but I do have a couple of > >> situations where pagination prevents client-side sorting. Any help > would > >> be greatly appreciated. > >> > >> Thanks, > >> > >> Shane > >> >
