Hi again,
Since you have a custom high availability solution over your solr
instances, I can't help much I guess... :-)
I usually rely on master/slave replication to separate index build and
index search processes.
The fact is that resources consumption at build time and search time are
not necessarily the same, and therefor hardware dimensioning can be
adapted as required.
I like to have the service related processes isolated and easy to deploy
wherever needed. Just in case things go wrong, hardware failures occur.
Build services on the other hand don't have the same availability
constraints, and can be off for a while, it's no issue (unless near
realtime indexing comes into party, that's an other thing)
In a slave configuration, the index doesn't need to commit. It simply
replicates its data from its associated master whenever the master
changes and performs a reopen of the searcher. "Change" events can be
triggered at commit, startup and / or optimize. (see
http://wiki.apache.org/solr/SolrReplication , although you seemed to be
not interested by this feature :) )
Having search and build on the same host is no bad point.
It simply depends on available resources and build vs service load
requirements.
For example with a big core such as the one you have, segments merging
can occur from time to time, which is an operation that is IO bound
(i.e. time is dependant of disk performances). Under high IO load, a
server can become less responsive and therefor having the service
separated from the build could became handy at that time.
As you see, I can't tell you what makes sense and what doesn't.
It's all about what you're doing, at which frequency, etc. :-)
Regards,
Tanguy
Le 07/11/2011 12:12, stockii a écrit :
hi thanks for the big reply ;)
i had the idea with the several and small 5M shards too.
and i think thats the next step i have to go, because our biggest index
grows each day with avg. 50K documents.
but make it sense to keep searcher AND updater cores on one big server? i
dont want to use replication, because with our own high avalibility solution
is this not possible.
my system is split into searcher and updater cores, each with his own index.
some search requests are over all this 8 cores with distributed search.
-----
------------------------------- System ----------------------------------------
One Server, 12 GB RAM, 2 Solr Instances, 8 Cores,
1 Core with 45 Million Documents other Cores< 200.000
- Solr1 for Search-Requests - commit every Minute - 5GB Xmx
- Solr2 for Update-Request - delta every Minute - 4GB Xmx
--
View this message in context:
http://lucene.472066.n3.nabble.com/best-way-for-sum-of-fields-tp3477517p3486652.html
Sent from the Solr - User mailing list archive at Nabble.com.
