There seems to be some improvement. The writes speeds are faster. Server
restarts are lower.
We changed the configuration to:
<maxDocs>50</maxDocs>
<maxTime>10000</maxTime>
Before the Change:
- Server Restarts: 10 times in 12 hours
- CPU load: Average:50 and Peak:90
After the Change:
- Server Restarts: 4 times in 12 hours.
- CPU load: Average:30 and Peak:~70
Our every day writes are around 60k and reads are around 1 million.
We are now changing the MaxDocs to 300 and MaxTime will be 10000 ms and
hoping to some more improvements.
The system configuration is 4GB RAM and 4 core x 2 CPUs. We start the
solr (1.3) like this: java -Xms512M -Xmx1024M -jar start.jar
Is there any other way we can reduce the high CPU load in the system?
Do you guys think that upgrading to 1.4 and having the replication in
place with reads and writes split into separate solrs will help? How
efficient will the replication be with above mentioned scenarios? Is
there any place we can look at for info on the disadvantages of
replication...
Please help.
Kalidoss.m,
Tom Alt wrote:
Nice to learn a new word for the day!
But to answer your question, or at least part of it, I don't really think
you want a configuration like
<autoCommit>
<maxDocs>1</maxDocs>
<maxTime>10</maxTime>
</autoCommit>
Committing every doc, and every 10 milliseconds? That's just asking for
problems. How about starting with 1000 docs, and five minutes for maxTime
(5*60*1000) or about 3 laks of milliseconds.
That should help performance a lot. Try that, and see how it works.
Tom
On Mon, Nov 16, 2009 at 2:43 PM, Shashi Kant <sk...@sloan.mit.edu> wrote:
I think it would be useful for members of this list to realize that not
everyone uses the same metrology and terms.
It is very easy for "Americans" to use the imperial system and presume
everyone does the same; Europeans to use the metric system etc. Hopefully
members on this list would be persuaded to use or at least clarify their
terminology.
While the apocryphal saying goes " the great thing about standards is they
are so many choose from", we should all make an effort to communicate
across
cultures and nations.
On Mon, Nov 16, 2009 at 5:33 PM, Israel Ekpo <israele...@gmail.com> wrote:
On Mon, Nov 16, 2009 at 5:22 PM, Walter Underwood <wun...@wunderwood.org
wrote:
Probably "lakh": 100,000.
So, 900k qpd and 3M docs.
http://en.wikipedia.org/wiki/Lakh
wunder
On Nov 16, 2009, at 2:17 PM, Otis Gospodnetic wrote:
Hi,
Your autoCommit settings are very aggressive. I'm guessing that's
what's
causing the CPU load.
btw. what is "laks"?
Otis
--
Sematext is hiring -- http://sematext.com/about/jobs.html?mls
Lucene, Solr, Nutch, Katta, Hadoop, HBase, UIMA, NLP, NER, IR
----- Original Message ----
From: kalidoss <kalidoss.muthuramalin...@sifycorp.com>
To: solr-user@lucene.apache.org
Sent: Mon, November 16, 2009 9:11:21 AM
Subject: Solr - Load Increasing.
Hi All.
My server solr box cpu utilization increasing b/w 60 to 90% and
some
time
solr is getting down and we are restarting it manually.
No of documents in solr 30 laks.
No of add/update requrest solr 30 thousand / day. Avg of every 30
minutes
around 500 writes.
No of search request 9laks / day.
Size of the data directory: 4gb.
My system ram is 8gb.
System available space 12gb.
processor Family: Pentium Pro
Our solr data size can be increase in number like 90 laks. and
writes
per day
will be around 1laks. - Hope its possible by solr.
For write commit i have configured like
1
10
Is all above can be possible? 90laks datas and 1laks per day
writes
and
30laks per day read?? - if yes what type of system configuration
would
require.
Please suggest us.
thanks,
Kalidoss.m,
Get your world in your inbox!
Mail, widgets, documents, spreadsheets, organizer and much more with
your
Sifymail WIYI id!
Log on to http://www.sify.com
********** DISCLAIMER **********
Information contained and transmitted by this E-MAIL is proprietary
to
Sify
Limited and is intended for use only by the individual or entity to
which it is
addressed, and may contain information that is privileged,
confidential
or
exempt from disclosure under applicable law. If this is a forwarded
message, the
content of this E-MAIL may not have been sent with the authority of
the
Company.
If you are not the intended recipient, an agent of the intended
recipient or a
person responsible for delivering the information to the named
recipient, you
are notified that any use, distribution, transmission, printing,
copying
or
dissemination of this information in any way or in any manner is
strictly
prohibited. If you have received this communication in error, please
delete this
mail & notify us immediately at ad...@sifycorp.com
Thanks Walter for clarifying that.
I too was wondering what "laks" meant.
It was a bit distracting when I read the original post.
--
"Good Enough" is not good enough.
To give anything less than your best is to sacrifice the gift.
Quality First. Measure Twice. Cut Once.
