after inspecting solrconfig.xml I see that I already have enabled lazy field loading by: <enableLazyFieldLoading>true</enableLazyFieldLoading> (I guess it was enabled by default)
Since any query returns about 10 fields (which differ from query to query) , would this mean that only these 10 of about 2000-4000 fields are retrieved / loaded? Thanks, Geert-Jan Erick Erickson wrote: > > From a Lucene perspective, it's certainly possible to do lazy field > loading. That is, when loading a document you can determine at > run time what fields to load, even on a per-document basis. I'm > not entirely sure how to accomplish this in Solr, but I'd give > long odds that there's a way..... > > I did a writeup on this on the Wiki, see: > > http://wiki.apache.org/lucene-java/FieldSelectorPerformance?highlight=%28fieldselectorperformance%29 > > > The title is FieldSelectorPerformance if you need to search the Wiki... > > Best > Erick > > On Dec 27, 2007 10:28 AM, Britske <[EMAIL PROTECTED]> wrote: > >> >> >> >> Yonik Seeley wrote: >> > >> > On Dec 27, 2007 9:45 AM, Britske <[EMAIL PROTECTED]> wrote: >> >> I am using SolrJ to communicate with SOLR. My Solr-queries perform >> within >> >> range (between 50 ms and 300 ms) by looking at the solr log as >> ouputted >> >> on >> >> my (windows) commandline. >> >> >> >> However I discovered that the following command at all times takes >> >> significantly longer than the number outputted in the solr-log, >> >> (sometimes >> >> about 400% longer): >> > >> > It's probably due to stored field retrieval. >> > The time in the response includes everything except the time to write >> > the response (since it appears at the beginning). Writing the >> > response involves reading the stored fields of documents (this was >> > done to allow one to stream a large number of documents w/o having >> > them all in memory). >> > >> > SolrJ's parsing of the response should be a relatively small constant >> > cost. >> > >> > -Yonik >> > >> > >> >> Is it normal to see this much time taken in stored field retrieval? And >> where would I start to make sure that it is indeed caused by stored field >> retrieval? >> >> It seems quite much to me, although I have kind if an out of the ordinary >> setup with between 2000-4000 stored fields per document. By far the >> largest >> part is taken by various 'product-variants' and their respective prices >> (indexed field) and other characteristics (stored only). >> However only about 10 stored fields per document are returned for any >> possible query. >> >> Would the time taken still include iterating the non-returned fields (of >> which there are many in my case), or are only the returned fields >> retrieved >> in a map-like implementation? >> >> Thanks, >> Geert-Jan >> -- >> View this message in context: >> http://www.nabble.com/big-perf-difference-between-solr-server-vs.--SOlrJ-req.process%28solrserver%29-tp14513964p14514441.html >> Sent from the Solr - User mailing list archive at Nabble.com. >> >> > > -- View this message in context: http://www.nabble.com/big-perf-difference-between-solr-server-vs.--SOlrJ-req.process%28solrserver%29-tp14513964p14514852.html Sent from the Solr - User mailing list archive at Nabble.com.
