I've seen this before. When you have a ton of threads, any synchronization point can become a bottleneck, regardless of how small the synchronized code is. If it wasn't isDeleted, it would be something else. The better strategy from an app standpoint is to throttle the number of threads.
-Yonik On Jan 27, 2008 2:24 PM, Michael Stoppelman <[EMAIL PROTECTED]> wrote: > Otis, > > Thanks for the response. > > All the threads are blocked in the same place. Here's the typical > stacktrace, I've remove the stuff outside of the lucene > libraries. I have about 100 threads blocked in the same place. > > btpool0-5 [BLOCKED; waiting to lock > [EMAIL PROTECTED] > org.apache.lucene.index.SegmentReader.isDeleted(SegmentReader.java) > org.apache.lucene.index.MultiReader.isDeleted(MultiReader.java:114) > org.apache.lucene.search.MatchAllDocsQuery$MatchAllScorer.next( > MatchAllDocsQuery.java:67) > org.apache.lucene.search.ConjunctionScorer.next(ConjunctionScorer.java:56) > org.apache.lucene.search.ReqExclScorer.next(ReqExclScorer.java:55) > org.apache.lucene.search.BooleanScorer2.score(BooleanScorer2.java:327) > org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:146) > org.apache.lucene.search.Searcher.search(Searcher.java:118) > org.apache.lucene.search.Searcher.search(Searcher.java:97) > org.apache.lucene.search.QueryWrapperFilter.bits(QueryWrapperFilter.java:50) > org.apache.lucene.search.CachingWrapperFilter.bits(CachingWrapperFilter.java > :58) > org.apache.lucene.misc.ChainedFilter.doChain(ChainedFilter.java:249) > org.apache.lucene.misc.ChainedFilter.bits(ChainedFilter.java:187) > org.apache.lucene.misc.ChainedFilter.bits(ChainedFilter.java:152) > org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:133) > org.apache.lucene.search.Searcher.search(Searcher.java:118) > > -M > > On Jan 26, 2008 8:37 PM, Otis Gospodnetic <[EMAIL PROTECTED]> > wrote: > > > > Can you kill -QUIT <your java PID> right after you fire those 20-30 > > concurrent queries? This could tell you/us where those threads are > > blocking, if they are blocking, or what they are all doing. > > > > Thanks, > > Otis > > -- > > Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch > > > > ----- Original Message ---- > > From: Michael Stoppelman <[EMAIL PROTECTED]> > > To: java-user@lucene.apache.org > > Sent: Friday, January 25, 2008 8:08:32 PM > > Subject: Re: Threads blocking on isDeleted when swapping indices for a > > very long time... > > > > For > > anyone > > interested > > I > > did > > work > > around > > this. > > The > > problem > > was > > that > > I > > wasn't > > warming > > the > > index > > enough. > > When > > I > > ran > > full > > queries > > against > > the > > new > > index > > being > > switched > > to > > everything > > runs > > smoothly > > when > > the > > indices > > are > > swapped > > now. > > There > > must > > be > > some > > lower > > level > > caches > > that > > need > > to > > be > > filled > > that > > I'm > > not > > aware > > of. > > The > > thread > > that > > I > > pointed > > at > > in > > the > > first > > post > > seems > > to > > suggest > > it's > > the > > caches > > in > > the > > Sort > > object. > > > > One > > can > > recreate > > this > > by > > starting > > up > > a > > non-warmed > > lucene > > index > > and > > firing > > 20 > > - > > 30 > > concurrent > > queries > > at > > it. > > Mostly > > likely > > lucene > > will > > freeze > > on > > all > > the > > threads. > > Maybe > > all > > the > > threads > > at > > the > > start > > are > > building > > the > > same > > cache > > multiple > > times? > > > > -M > > > > On > > Jan > > 25, > > 2008 > > 2:01 > > AM, > > Michael > > Stoppelman > > <[EMAIL PROTECTED]> > > wrote: > > > > > > > BTW, > > I'm > > using > > Lucene > > 2.2.0. > > > > > > > > -M > > > > > > > > p.s. > > Congrats > > on > > the > > 2.3.0 > > release! > > > > > > > > > > > On > > Jan > > 24, > > 2008 > > 7:42 > > PM, > > Michael > > Stoppelman > > <[EMAIL PROTECTED] > > > > > wrote: > > > > > > > > > > > Hi > > all, > > > > > > > > > > > > > > I've > > been > > tracking > > down > > a > > problem > > happening > > in > > our > > production > > > > > > > > environment. > > When > > we > > switch > > an > > index > > after > > doing > > deletes > > & > > adds, > > running > > > > > > > > some > > searches, > > and > > finally > > changing > > the > > pointer > > > > > > > > from > > old > > index > > to > > new > > all > > the > > threads > > start > > stacking > > up > > all > > waiting > > on > > > > > > > > isDeleted(). > > The > > threads > > seem > > to > > finish, > > they > > just > > get > > really > > slow > > taking > > up > > > > > > > > to > > 30 > > - > > 60 > > seconds. > > > > > > > > > > > > > > The > > problem > > has > > been > > discussed > > here > > before > > in > > 2005: > > > > > > > > > > http://mail-archives.apache.org/mod_mbox/lucene-java-user/200510.mbox/[EMAIL > > PROTECTED] > > > > > > > > > > > > > > > > > > > > Does > > anyone > > have > > any > > suggestions > > on > > how > > to > > work > > around > > this? > > > > > > > > > > > > > > -M > > > > > > > > > > > > > > > > > > > > > > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: [EMAIL PROTECTED] > > For additional commands, e-mail: [EMAIL PROTECTED] > > > > > --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
