Perhaps the container logs explain what happened? How about just throttling to the point where the failure rate is 0%? Too slow?
Otis -- Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch ----- Original Message ---- > From: Paleo Tek <[EMAIL PROTECTED]> > To: solr-user@lucene.apache.org > Sent: Friday, September 12, 2008 11:19:52 AM > Subject: No server response code on insert: how do I avoid this at high > speed? > > I have a largish index with a lot of churn, and inserts that come in > large bursts. My server is a multiprocessor with plenty of memory, so I > can multi-thread and stuff in about 1.6 million records per hour, going > full speed. I use a dozen or so threads to post curl inserts, and > monitor the responses. > > Using jetty, there is ~10% failure rate with no server response code > received. Switching to tomcat reduces the error rate to around 2%. > (which makes me like tomcat a lot, even though I'm a dog person...). I > suspect I'm overrunning the capacity of the servlet container. Tweaking > parameters in Jetty improved performance, and I can tune Tomcat. But > then I'll just be overrunning a tuned system, at a slightly faster rate. > > My work around is to keep track of which inserts fail, but I suspect > there's a better approach. Any suggestions how I can balance maximum > insert speed with a low error rate? Thanks! > > -Jim
