This is my situation. I have an index, which has a lot of search requests
coming into it. I use just a single instance of IndexSearcher to process
these requests. At the same time, this index is also getting updated by an
IndexWriter. And I want these new changes to be reflected _only_ at certa
A simple variant on Approach 1 would be to open your writer with
autoCommit=false.
This way no reader will ever see the changes until you successfully
close the writer. If the machine crashes the index is still in the
starting state as of when the writer was first opened.
Also, re-open
Sridhar Raman wrote:
This way no reader will ever see the changes until you successfully
close the writer. If the machine crashes the index is still in the
starting state as of when the writer was first opened.
Ok, I have a slight doubt in this. Say I have gone ahead with
Approach 1
If I ha
> This way no reader will ever see the changes until you successfully
> close the writer. If the machine crashes the index is still in the
> starting state as of when the writer was first opened.
Ok, I have a slight doubt in this. Say I have gone ahead with Approach 1
If I have opened the writer
On Thu, Mar 6, 2008 at 8:02 AM, Sridhar Raman <[EMAIL PROTECTED]> wrote:
> > This way no reader will ever see the changes until you successfully
> > close the writer. If the machine crashes the index is still in the
> > starting state as of when the writer was first opened.
> Ok, I have a sligh
> Since Lucene buffers in memory, you will always have the risk of
> losing recently added documents that haven't been flushed yet.
> Committing on every document would be too slow to be practical.
Well it is not sooo slw...
I have indexed 10.000 docs, resulting in 14 MB index. The index has
On Thu, Mar 6, 2008 at 12:22 PM, <[EMAIL PROTECTED]> wrote:
> > Since Lucene buffers in memory, you will always have the risk of
> > losing recently added documents that haven't been flushed yet.
> > Committing on every document would be too slow to be practical.
>
> Well it is not sooo slw
> > With a commit after every add: 30 min.
> > With a commit after 100 add: 23 min.
> > Only one commit: 20 min.
>
> All of these times look pretty slow... perhaps lucene is not the
> bottleneck here?
Therefore I wrote:
"(including time to get the document from the archive)"
Not the absolute
Sridhar,
We have been using approach 2 in our production system with good results. We
have separate processes for indexing and searching. The main issue that came
up was in deleting old indexes (see: *http://tinyurl.com/32q8c4*). Most of
our production problems occur during indexing, and we are ab
On Thu, 2008-03-06 at 18:40 +0100, [EMAIL PROTECTED] wrote:
> > > With a commit after every add: 30 min.
> > > With a commit after 100 add: 23 min.
> > > Only one commit: 20 min.
[...]
> I think it is a real world scenario because one has always the read the docs
> from somewhere and offen has
> With a commit after every add: (286 sec / 10,000 docs) 28.6 ms.
> With a commit after every 100 add: (12 sec / 10,000 docs) 1.2 ms.
> Only one commit: (8 sec / 10,000 docs) 0.8 ms.
Of couse. If you need so less time to create a document than a commit which
may take, lets say 10 - 500 ms, will s
On Fri, 2008-03-07 at 15:38 +0100, [EMAIL PROTECTED] wrote:
> > With a commit after every add: (286 sec / 10,000 docs) 28.6 ms.
> > With a commit after every 100 add: (12 sec / 10,000 docs) 1.2 ms.
> > Only one commit: (8 sec / 10,000 docs) 0.8 ms.
>
> Of couse. If you need so less time to create
What's the CPU usage and MEM usage when doing "With a commit after every 100
add" vs. "Only one commit" ?
2008/3/10, Toke Eskildsen <[EMAIL PROTECTED]>:
>
> On Fri, 2008-03-07 at 15:38 +0100, [EMAIL PROTECTED] wrote:
> > > With a commit after every add: (286 sec / 10,000 docs) 28.6 ms.
> > > With
One quick doubt regarding copying of indexes. Is the copy done on the
indexes in memory as well, or is it only done on the committed indexes?
On Fri, Mar 7, 2008 at 12:29 AM, Peter Keegan <[EMAIL PROTECTED]>
wrote:
> Sridhar,
>
> We have been using approach 2 in our production system with good r
14 matches
Mail list logo
