On 13/07/2005, at 1:34 AM, Chris Hostetter wrote:
: Since this isn't in production yet, I'd rather be proven wrong now
: rather than later! :)
it sounds like what you're doing makes a lot of sense given your
situation, and the nature of your data.
the one thing you might not have concidered
ent: Monday, July 11, 2005 5:43 PM
To: java-user@lucene.apache.org
Subject: RE: Search deadlocking under load
Thanks for the advice. That ought to reduce contention a bit in that
particular method.
I've been reviewing a large amount of thread dumps today and I was wondering
if it's co
: Since this isn't in production yet, I'd rather be proven wrong now
: rather than later! :)
it sounds like what you're doing makes a lot of sense given your
situation, and the nature of your data.
the one thing you might not have concidered yet, which doesn't have to
make a big difference in yo
Many thanks for confirming the principles should work fine. It is a
load off my mind! :)
On index update, a small Event is triggered into a Buffer, that is
periodically (every 30 seconds) processed to coalesce them, then
ensure that any open IndexSearcher in the cache is closed.
On 12/07
gt; search
> method (the one that makes the actual call to Lucene's search: [14]
> com.nettemps.search.backend.SingleIndexManager.search
> (SingleIndexManager.java:335)). This also caused the same results
> when hit
> with too many threads.
>
> We're really stuck at this p
If you want really real-time updates of search results, then yes.
However, maybe you can live with near-real-time results, in which cases
you can add some logic to your application to check for index version
only every N requests/minutes/hours.
Otis
--- Aalap Parikh <[EMAIL PROTECTED]> wrote:
>I don't really know a lot about what gets loaded into
memory when you
>make/use a new searcher, but the one thing i've
learned from experience
>is
>that the FieldCache (which gets used when you sort on
a field) contains
>every term in the field you are sorting on, and an
instance of
>FieldCache
Paul - I'm doing the same (smaller indices) for Simpy.com for similar
reasons (fast, independent and faster reindexing, etc.). Each index
has its own IndexSearcher, and they are kept in a LRU data structure.
Before each search the index version is checked, and new IndexSearcher
created in case th
one that makes the actual call to Lucene's search: [14]
> com.nettemps.search.backend.SingleIndexManager.search
> (SingleIndexManager.java:335)). This also caused the same results
> when hit
> with too many threads.
>
> We're really stuck at this point as to what to try. Any advice?
>
>
#x27;re really stuck at this point as to what to try. Any advice?
-Original Message-
From: Otis Gospodnetic [mailto:[EMAIL PROTECTED]
Sent: Friday, July 08, 2005 3:40 PM
To: java-user@lucene.apache.org
Subject: Re: Search deadlocking under load
Nathan,
3) is the recommended usage.
You
Hello,
We are already using this design in production for a email job application
system.
Each client (company) have an account and may have multiple users
When a new client is created, a new lucene index is automatically created when
new job-applications arrive for this account.
Job applicati
On 11/07/2005, at 10:43 AM, Chris Hostetter wrote:
: > Generally speaking, you only ever need one active Searcher, which
: > all of
: > your threads should be able to use. (Of course, Nathan says that
: > in his
: > code base, doing this causes his JVM to freeze up, but I've
never seen
: >
: > Generally speaking, you only ever need one active Searcher, which
: > all of
: > your threads should be able to use. (Of course, Nathan says that
: > in his
: > code base, doing this causes his JVM to freeze up, but I've never seen
: > this myself).
: >
: Thanks for your response Chris. Do y
On 11/07/2005, at 9:15 AM, Chris Hostetter wrote:
: Nathan's point about pooling Searchers is something that we also
: addressed by a LRU cache mechanism. In testing we also found that
Generally speaking, you only ever need one active Searcher, which
all of
your threads should be able to u
: Nathan's point about pooling Searchers is something that we also
: addressed by a LRU cache mechanism. In testing we also found that
Generally speaking, you only ever need one active Searcher, which all of
your threads should be able to use. (Of course, Nathan says that in his
code base, doin
Nathan, first apologies for somewhat hijacking your thread, but I
believe my question to be very related.
Nathan's Scenario 1 is the one we're effectively employing (or in the
process of setting up). Rather than 1 Index To Rule Them All, I have
decided to partition the index structure. Us
Nathan,
3) is the recommended usage.
Your index is on an NFS share, which means you are searching it over
the network. Make it local, and you should see performance
improvements. Local or remove, it makes sense that searches take
longer to execute, and the load goes up. Yes, it shouldn't deadlo
Hey all,
We're looking to use Lucene as the back end to our website and we're running
into an unusual deadlocking problem.
For testing purposes, we're just running one web server (threaded
environment) against an index mounted on an NFS share. This machine performs
searches only against this inde
18 matches
Mail list logo
