@Israel: yes I got that point which yonik mentioned .. but is qtime the total time taken by solr server for that request or is it part of time taken by the solr for that request ( is there any thing that a solr server does for that particulcar request which is not included in that qtime bracket ) ? I am sorry for dragging in to this qtime. I just want to be sure, as we observed many times there is huge mismatch between qtime and time measured at the client for the response ( does this imply it is due to internal network issue )
@Erick: yes, many times query is slow first time its executed is there any solution to improve upon this factor .. for querying we use DisMaxRequestHandler , queries are quite long with many faceting parameters . On Mon, Nov 2, 2009 at 10:46 PM, Israel Ekpo <israele...@gmail.com> wrote: > On Mon, Nov 2, 2009 at 9:52 AM, bharath venkatesh < > bharathv6.proj...@gmail.com> wrote: > > > Thanks for the quick response > > @yonik > > > > >How much of a latency compared to normal, and what version of Solr are > > you using? > > > > latency is usually around 2-4 secs (some times it goes more than that > > ) which happens to only 15-20% of the request other 80-85% of > > request are very fast it is in milli secs ( around 200,000 requests > > happens every day ) > > > > @Israel we are not using java client .. we r using python at the > > client with response formatted in json > > > > @yonikn @Israel does qtime measure the total time taken at the solr > > server ? I am already measuring the time to get the response at > > client end . I would want a means to know how much time the solr > > server is taking to respond (process ) once it gets the request . so > > that I could identify whether it is a solr server issue or internal > > network issue > > > > It is the time spent at the Solr server. > > I think Yonik already answered this part in his response to your thread : > > This is what he said : > > QTime is the time spent in generating the in-memory representation for > the response before the response writer starts streaming it back in > whatever format was requested. The stored fields of returned > documents are also loaded at this point (to enable handling of huge > response lists w/o storing all in memory). > > > > > > @Israel we are using rhel server 5 on both client and server .. we > > have 6 solr sever . one is acting as master . both client and solr > > sever are on the same network . those servers are dedicated solr > > server except 2 severs which have DB and memcahce running .. we have > > adjusted the load accordingly > > > > > > > > > > > > > > > > On 11/2/09, Israel Ekpo <israele...@gmail.com> wrote: > > > On Mon, Nov 2, 2009 at 8:41 AM, Yonik Seeley > > > <yo...@lucidimagination.com>wrote: > > > > > >> On Mon, Nov 2, 2009 at 8:13 AM, bharath venkatesh > > >> <bharathv6.proj...@gmail.com> wrote: > > >> > We are using solr for many of ur products it is doing quite well > > >> > . But since no of hits are becoming high we are experiencing > latency > > >> > in certain requests ,about 15% of our requests are suffering a > latency > > >> > > >> How much of a latency compared to normal, and what version of Solr are > > >> you using? > > >> > > >> > . We are trying to identify the problem . It may be due to > network > > >> > issue or solr server is taking time to process the request . > other > > >> > than qtime which is returned along with the response is there any > > >> > other way to track solr servers performance ? > > >> > how is qtime calculated > > >> > , is it the total time from when solr server got the request till it > > >> > gave the response ? > > >> > > >> QTime is the time spent in generating the in-memory representation for > > >> the response before the response writer starts streaming it back in > > >> whatever format was requested. The stored fields of returned > > >> documents are also loaded at this point (to enable handling of huge > > >> response lists w/o storing all in memory). > > >> > > >> There are normally servlet container logs that can be configured to > > >> spit out the real total request time. > > >> > > >> > can we do some extra logging to track solr servers > > >> > performance . ideally I would want to pass some log id along with > the > > >> > request (query ) to solr server and solr server must log the > > >> > response time along with that log id . > > >> > > >> Yep - Solr isn't bothered by params it doesn't know about, so just put > > >> logid=xxxxxxx and it should also be logged with the other request > > >> params. > > >> > > >> -Yonik > > >> http://www.lucidimagination.com > > >> > > > > > > > > > > > > If you are not using Java then you may have to track the elapsed time > > > manually. > > > > > > If you are using the SolrJ Java client you may have the following > > options: > > > > > > There is a method called getElapsedTime() in > > > org.apache.solr.client.solrj.response.SolrResponseBase which is > available > > to > > > all the subclasses > > > > > > I have not used it personally but I think this should return the time > > spent > > > on the client side for that request. > > > > > > The QTime is not the time on the client side but the time spent > > internally > > > at the Solr server to process the request. > > > > > > > > > http://lucene.apache.org/solr//api/solrj/org/apache/solr/client/solrj/response/SolrResponseBase.html > > > > > > > > > http://lucene.apache.org/solr//api/solrj/org/apache/solr/client/solrj/response/QueryResponse.html > > > > > > Most likely it could be as a result of an internal network issue > between > > the > > > two servers or the Solr server is competing with other applications for > > > resources. > > > > > > What operating system is the Solr server running on? Is you client > > > application connection to a Solr server on the same network or over the > > > internet? Are there other applications like database servers etc > running > > on > > > the same machine? If so, then the DB server (or any other application) > > and > > > the Solr server could be competing for resources like CPU, memory etc. > > > > > > If you are using Tomcat, you can take a look in > > > $CATALINA_HOME/logs/catalina.out, there are timestamps there that can > > also > > > guide you. > > > > > > -- > > > "Good Enough" is not good enough. > > > To give anything less than your best is to sacrifice the gift. > > > Quality First. Measure Twice. Cut Once. > > > > > > > > > -- > "Good Enough" is not good enough. > To give anything less than your best is to sacrifice the gift. > Quality First. Measure Twice. Cut Once. >
