Yonik, I have provided an image below gives details on what is causing the blocked http thread. Is there any way to resolve this issue.
Thanks, John -- John Williams System Administrator 37signals
<<inline: Screen shot 2010-03-09 at 11.22.20 AM.png>>
On Mar 9, 2010, at 10:41 AM, John Williams wrote: > Yonik, > > I got yourkit setup to profile the Tomcat instance and as you will see in the > graph below all of the http threads are blocked (red) until around 4:40. > This is the point where the instance becomes responsive and CPU usage drops. > I have also ruled out GC being the issue by using the GC monitoring in > yourkit. Let me know your thoughts and if you have any questions. > > Thanks for your assistance. > > Thanks, > John > > -- > John Williams > System Administrator > 37signals > > <Screen shot 2010-03-09 at 10.35.15 AM.png> > On Mar 8, 2010, at 5:28 PM, Yonik Seeley wrote: > >> On Mon, Mar 8, 2010 at 6:07 PM, John Williams <j...@37signals.com> wrote: >>> Yonik, >>> >>> In all cases our "autowarmCount" is set to 0. Also, here is a link to our >>> config. http://pastebin.com/iUgruqPd >> >> Weird... on a quick glance, I don't see anything in your config that >> would cause work to be done on a commit. >> I expected something like autowarming, or rebuilding a spellcheck >> index, etc. I assume this is happening even w/o any requests hitting >> the server? >> >> Could it be GC? You could use -verbose:gc or jconsole to check if >> this corresponds to a big GC (which could naturally hit on an index >> change). 5 minutes is really excessive though, and I wouldn't expect >> it on startup. >> >> If it's not GC, perhaps the next step is to get some stack traces >> during the spike (or use a profiler) to figure out where the time is >> being spent. And verify that the solrconfig.xml shown actually still >> matches the one you provided. >> >> -Yonik >> http://www.lucidimagination.com >> >> >> >>> Thanks, >>> John >>> >>> -- >>> John Williams >>> System Administrator >>> 37signals >>> >>> On Mar 8, 2010, at 4:44 PM, Yonik Seeley wrote: >>> >>>> Is this just autowarming? >>>> Check your autowarmCount parameters in solrconfig.xml >>>> >>>> -Yonik >>>> http://www.lucidimagination.com >>>> >>>> On Mon, Mar 8, 2010 at 5:37 PM, John Williams <j...@37signals.com> wrote: >>>>> Good afternoon. >>>>> >>>>> We have been experiencing an odd issue with one of our Solr nodes. Upon >>>>> startup or when bringing in a new index we get a CPU spike for 5 minutes >>>>> or so. I have attached a graph of this spike. During this time simple >>>>> queries return without a problem but more complex queries do not return. >>>>> Here are some more details about the instance: >>>>> >>>>> Index Size: ~16G >>>>> Max Heap: 6144M >>>>> GC Option: -XX:+UseConcMarkSweepGC >>>>> System Memory: 16G >>>>> >>>>> We have a very similar instance to this one but with a much larger index >>>>> that we are not seeing this sort of issue. >>>>> >>>>> Your help is greatly appreciated. Let me know if you need any additional >>>>> information. >>>>> >>>>> Thanks, >>>>> John >>>>> >>>>> -- >>>>> John Williams >>>>> System Administrator >>>>> 37signals >>>>> >>>>> >>>>> >>> >>> >
smime.p7s
Description: S/MIME cryptographic signature
