Anybody on the list have a feel for how many simultaneous queries Solr can handle in parallel? Will it be linear WRT the number of CPU cores? Or are their other bottlenecks or locks in Lucene or Solr such that even with more CPU cores the Solr server will be saturated with fewer queries than the number of CPU cores?
-- Jack Krupansky On Wed, Jan 7, 2015 at 9:14 PM, Nishanth S <nishanth.2...@gmail.com> wrote: > Thanks Shawn and Walter.Yes those are 12,000 writes/second.Reads for the > moment would be in the 1000 reads/second. Guess finding out the right > number of shards would be my starting point. > > Thanks, > Nishanth > > > On Wed, Jan 7, 2015 at 6:28 PM, Walter Underwood <wun...@wunderwood.org> > wrote: > > > This is described as “write heavy”, so I think that is 12,000 > > writes/second, not queries. > > > > Walter Underwood > > wun...@wunderwood.org > > http://observer.wunderwood.org/ > > > > > > On Jan 7, 2015, at 5:16 PM, Shawn Heisey <apa...@elyograg.org> wrote: > > > > > On 1/7/2015 3:29 PM, Nishanth S wrote: > > >> I am working on coming up with a solr architecture layout for my use > > >> case.We are a very write heavy application with no down time > tolerance > > and > > >> have low SLAs on reads when compared with writes.I am looking at > around > > >> 12K tps with average index size of solr document in the range of 6kB.I > > >> would like to go with 3 replicas for that extra fault tolerance and > > trying > > >> to identify the number of shards.The machines are monsterous and have > > >> around 100 GB of RAM and more than 24 cores on each.Is there a way to > > >> come at the number of desired shards in this case.Any pointers would > be > > >> helpful. > > > > > > This is one of those questions that's nearly impossible to answer > > > without field trials that have a production load on a production index. > > > Minor changes to either config or schema can have a major impact on the > > > query load Solr will support. > > > > > > > > > https://lucidworks.com/blog/sizing-hardware-in-the-abstract-why-we-dont-have-a-definitive-answer/ > > > > > > A query load of 12000 queries per second is VERY high. That is likely > > > to require a **LOT** of hardware, because you're going to need a lot of > > > replicas. Because each server will be handling quite a lot of > > > simultaneous queries, the best results will come from having only one > > > replica (solr core) per server. > > > > > > Generally you'll get better results for a high query load if you don't > > > shard your index, but depending on how many docs you have, you might > > > want to shard. You haven't said how many docs you have. > > > > > > The key to excellent performance with Solr is to make sure that the > > > system never hits the disk to read index data -- for 12000 queries per > > > second, the index must be fully cached in RAM. If Solr must go to the > > > actual disk, query performance will drop significantly. > > > > > > Thanks, > > > Shawn > > > > > > > >
