Wow! Thanks Shawn. That's great info and helped and thanks for the
SolrPerformance article link, great article, helped a lot :)
I can't use Cloud hosting now since they charge on basis of the memory used
and it will be too expensive and like you said RAM and SSD is what I need
for SOLR performance.
In my solrconfig.xml I've got these caching config by default which I don't
think I will need. Since my index is updated with new documents every 3
minutes caching anything would be pointless. Am I on the right ?
<!-- <filterCache class="solr.FastLRUCache" size="512"
initialSize="512" autowarmCount="0" />
<queryResultCache class="solr.LRUCache" size="512"
initialSize="512" autowarmCount="0" />
<documentCache class="solr.LRUCache" size="512" initialSize="512"
autowarmCount="0" /> -->
If yes, saying I have an index of 80GB and since I won't be using cache I
won't have to worry about having too much RAM ? May be just enough ram for
processing ? say 8GB ram and 240GB SSD ?
On Thu, Jul 18, 2013 at 12:00 AM, Shawn Heisey <s...@elyograg.org> wrote:
> On 7/17/2013 1:22 AM, Ayman Plaha wrote:
> > *will this effect the query performance of the client website if the
> > index grew to 10 million records ? I mean while the commit is
> happening
> > does that *effect the performance of queries* and how will this effect
> > the queries if the index grew to 10 million records ?
>
> Every time you commit and open a new searcher, any data in the caches
> that Solr itself creates is wiped. If you have configured autowarming,
> then it will use keys from the old cache to repopulate the new cache, by
> using those keys as queries on the index. If autowarmCount is high,
> those warming queries can take a long time and put quite a load on the
> index. While the warming is happening, the old searcher continues to
> process queries.
>
> > - What *hosting specs* should I get ? How much RAM ? Considering my
> > - client application is very simple that just register users to
> database
> > and queries SOLR and displays SOLR results.
>
> This is almost impossible to answer. Even if you can give us more
> statistics about your setup, the only way to REALLY know is to
> experiment. I can give you some basic guidelines:
>
> 1) Get as much processing power as you can reasonably afford, but
> understand that I/O and RAM are likely to play a bigger role in Solr
> performance than bleeding-edge CPU power.
>
> 2) Multi-disk RAID10 or SSD performs best for an I/O layer.
>
> 3) For RAM, if Solr is the only thing running on the machine, the ideal
> amount is the size of your index on disk, plus the Solr JVM size, plus a
> little bit (1GB or less) for the OS. This lets the OS cache the entire
> index in RAM. Because the OS disk cache is very smart, you may be able
> to run effectively with less RAM, especially if you use SSD. If the
> available OS disk cache is too small, performance will really suffer.
>
> If Solr is not the only thing running on the machine, then you need to
> add the RAM requirements of the other processes. Those RAM requirements
> may extend beyond the memory required for the processes themselves,
> because other programs usually benefit from OS disk caching as well.
>
> Running only Solr on the server is recommended. If you are running in
> SolrCloud mode, it's normal to also run one of the required zookeeper
> instances on the same hardware, because zookeeper requirements are very
> small.
>
> Some basic information about RAM sizing can be found on this wiki page:
>
> http://wiki.apache.org/solr/SolrPerformanceProblems
>
> Thanks,
> Shawn
>
>
