Manuel, Whether to have the front end solr as aggregator of shard results depends on your requirements. To repeat, we found merging from many shards very inefficient fo our use case. It can be the opposite for you (i.e. requires testing). There are some limitations with distributed search, see here: http://docs.lucidworks.com/display/solr/Distributed+Search+with+Index+Sharding
On Wed, Sep 11, 2013 at 3:35 PM, Manuel Le Normand < manuel.lenorm...@gmail.com> wrote: > Dmitry - currently we don't have such a front end, this sounds like a good > idea creating it. And yes, we do query all 36 shards every query. > > Mikhail - I do think 1 minute is enough data, as during this exact minute I > had a single query running (that took a qtime of 1 minute). I wanted to > isolate these hard queries. I repeated this profiling few times. > > I think I will take the termInterval from 128 to 32 and check the results. > I'm currently using NRTCachingDirectoryFactory > > > > > On Mon, Sep 9, 2013 at 11:29 PM, Dmitry Kan <solrexp...@gmail.com> wrote: > > > Hi Manuel, > > > > The frontend solr instance is the one that does not have its own index > and > > is doing merging of the results. Is this the case? If yes, are all 36 > > shards always queried? > > > > Dmitry > > > > > > On Mon, Sep 9, 2013 at 10:11 PM, Manuel Le Normand < > > manuel.lenorm...@gmail.com> wrote: > > > > > Hi Dmitry, > > > > > > I have solr 4.3 and every query is distributed and merged back for > > ranking > > > purpose. > > > > > > What do you mean by frontend solr? > > > > > > > > > On Mon, Sep 9, 2013 at 2:12 PM, Dmitry Kan <solrexp...@gmail.com> > wrote: > > > > > > > are you querying your shards via a frontend solr? We have noticed, > that > > > > querying becomes much faster if results merging can be avoided. > > > > > > > > Dmitry > > > > > > > > > > > > On Sun, Sep 8, 2013 at 6:56 PM, Manuel Le Normand < > > > > manuel.lenorm...@gmail.com> wrote: > > > > > > > > > Hello all > > > > > Looking on the 10% slowest queries, I get very bad performances > (~60 > > > sec > > > > > per query). > > > > > These queries have lots of conditions on my main field (more than a > > > > > hundred), including phrase queries and rows=1000. I do return only > > id's > > > > > though. > > > > > I can quite firmly say that this bad performance is due to slow > > storage > > > > > issue (that are beyond my control for now). Despite this I want to > > > > improve > > > > > my performances. > > > > > > > > > > As tought in school, I started profiling these queries and the data > > of > > > ~1 > > > > > minute profile is located here: > > > > > > http://picpaste.com/pics/IMG_20130908_132441-ZyrfXeTY.1378637843.jpg > > > > > > > > > > Main observation: most of the time I do wait for readVInt, who's > > > > stacktrace > > > > > (2 out of 2 thread dumps) is: > > > > > > > > > > catalina-exec-3870 - Thread t@6615 > > > > > java.lang.Thread.State: RUNNABLE > > > > > at org.apadhe.lucene.store.DataInput.readVInt(DataInput.java:108) > > > > > at > > > > > > > > > > > > > > > > > > > > org.apaChe.lucene.codeosAockTreeIermsReade$FieldReader$SegmentTermsEnumFrame.loadBlock(BlockTreeTermsReader.java: > > > > > 2357) > > > > > at > > > > > > > > > > > > > > > > > > > > ora.apache.lucene.codecs.BlockTreeTermsReader$FieldReader$SegmentTermsEnum.seekExact(BlockTreeTermsReader.java:1745) > > > > > at org.apadhe.lucene.index.TermContext.build(TermContext.java:95) > > > > > at > > > > > > > > > > > > > > > > > > > > org.apache.lucene.search.PhraseQuery$PhraseWeight.<init>(PhraseQuery.java:221) > > > > > at > > > > > org.apache.lucene.search.PhraseQuery.createWeight(PhraseQuery.java:326) > > > > > at > > > > > > > > > > > > > > > > > > > > org.apache.lucene.search.BooleanQuery$BooleanWeight.<init>(BooleanQuery.java:183) > > > > > at > > > > > > > > > org.apache.lucene.search.BooleanQuery.createWeight(BooleanQuery.java:384) > > > > > at > > > > > > > > > > > > > > > > > > > > org.apache.lucene.searth.BooleanQuery$BooleanWeight.<init>(BooleanQuery.java:183) > > > > > at > > > > > > > > > oro.apache.lucene.search.BooleanQuery.createWeight(BooleanQuery.java:384) > > > > > at > > > > > > > > > > > > > > > > > > > > org.apache.lucene.searth.BooleanQuery$BooleanWeight.<init>(BooleanQuery.java:183) > > > > > at > > > > > > > > > org.apache.lucene.search.BooleanQuery.createWeight(BooleanQuery.java:384) > > > > > at > > > > > > > > > > > > > > > > > > > > org.apache.lucene.search.IndexSearcher.createNormalizedWeight(IndexSearcher.java:675) > > > > > at > > > org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:297) > > > > > > > > > > > > > > > So I do actually wait for IO as expected, but I might be too many > > time > > > > page > > > > > faulting while looking for the TermBlocks (tim file), ie locating > the > > > > term. > > > > > As I reindex now, would it be useful lowering down the termInterval > > > > > (default to 128)? As the FST (tip files) are that small (few 10-100 > > MB) > > > > so > > > > > there are no memory contentions, could I lower down this param to 8 > > for > > > > > example? The benefit from lowering down the term interval would be > to > > > > > obligate the FST to get on memory (JVM - thanks to the > > > > NRTCachingDirectory) > > > > > as I do not control the term dictionary file (OS caching, loads an > > > > average > > > > > of 6% of it). > > > > > > > > > > > > > > > General configs: > > > > > solr 4.3 > > > > > 36 shards, each has few million docs > > > > > These 36 servers (each server has 2 replicas) are running virtual, > > 16GB > > > > > memory each (4GB for JVM, 12GB remain for the OS caching), > consuming > > > > 260GB > > > > > of disk mounted for the index files. > > > > > > > > > > > > > > >
