We have two boxes, they are really nice servers, 32 core cpu, 192 G memory with both RAID arrays and fusion IOs. But each of them running two instances of Solr, one for indexing and the other for searching.Search index is on fusion IO card.
Each instance has 11 cores and a small core for making indexing almost realtime. We have around 300 Million documents and 250G on disk. They are all metadata . Search queries are very diverse and they do not repeat very frequently , 40 -60 qps. Before we had two cores each 125 G on disk and solr was taking long time to get results from those two cores. CPU use was 90%. We never had problem with indexing. 50% of all our docs gets updated every day, so very high indexing rate. On Thu, Feb 14, 2013 at 4:20 PM, alxsss [via Lucene] <ml-node+s472066n4040545...@n3.nabble.com> wrote: > Hi, > > It is curious to know how many linux boxes do you have and how many cores in > each of them. It was my understanding that solr puts in the memory all > documents found for a keyword, not the whole index. So, why it must be > faster with more cores, when number of selected documents from many separate > cores are the same as from one core? > > Thanks. > Alex. > > > > > > > > -----Original Message----- > From: Mou <[hidden email]> > To: solr-user <[hidden email]> > Sent: Thu, Feb 14, 2013 2:35 pm > Subject: Re: long QTime for big index > > > Just to close this discussion , we solved the problem by splitting the > index. > It turned out that distributed search with 12 cores are faster than > searching two cores. > > All queries ,tomcat configuration, jvm configuration remain same. Now > queries are served in milliseconds. > > > On Thu, Jan 31, 2013 at 9:34 PM, Mou [via Lucene] > <[hidden email]> wrote: > >> Thank you again. >> >> Unfortunately the index files will not fit in the RAM.I have to try using >> document cache. I am also moving my index to SSD again, we took our index >> off when fusion IO cards failed twice during indexing and index was >> corrupted.Now with the bios upgrade and new driver, it is supposed to be >> more reliable. >> >> Also I am going to look into the client app to verify that it is making >> proper query requests. >> >> Surprisingly when I used a much lower value than default for >> defaultconnectionperhost and maxconnectionperhost in solrmeter , it >> performs >> very well, the same queries return in less than one sec . I am not sure >> yet, >> need to run solrmeter with different heap size , with cache and without >> cache etc. >> >> ________________________________ >> If you reply to this email, your message will be added to the discussion >> below: >> >> http://lucene.472066.n3.nabble.com/long-QTime-for-big-index-tp4037635p4037870.html >> To unsubscribe from long QTime for big index, click here. >> NAML > > > > > -- > View this message in context: > http://lucene.472066.n3.nabble.com/long-QTime-for-big-index-tp4037635p4040535.html > > Sent from the Solr - User mailing list archive at Nabble.com. > > > > > ________________________________ > If you reply to this email, your message will be added to the discussion > below: > http://lucene.472066.n3.nabble.com/long-QTime-for-big-index-tp4037635p4040545.html > To unsubscribe from long QTime for big index, click here. > NAML -- View this message in context: http://lucene.472066.n3.nabble.com/long-QTime-for-big-index-tp4037635p4040549.html Sent from the Solr - User mailing list archive at Nabble.com.
