[
https://issues.apache.org/jira/browse/LUCENE-847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12483737
]
Doug Cutting commented on LUCENE-847:
-
How public should such an API be? Should the interface be public? Should
[
https://issues.apache.org/jira/browse/LUCENE-847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12483742
]
Steven Parkes commented on LUCENE-847:
--
Visibility is one of those things I haven't cleaned up yet.
Client code
[
https://issues.apache.org/jira/browse/LUCENE-847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12483929
]
Michael McCandless commented on LUCENE-847:
---
Steven, I looked through the patch quickly. It looks great!
[
https://issues.apache.org/jira/browse/LUCENE-847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12483930
]
Michael McCandless commented on LUCENE-847:
---
My first comment, which I fear will be the most controversial
[
https://issues.apache.org/jira/browse/LUCENE-847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12483931
]
Michael McCandless commented on LUCENE-847:
---
OK some specific comments, only on the refactoring (ie I haven
[
https://issues.apache.org/jira/browse/LUCENE-847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12490192
]
Steven Parkes commented on LUCENE-847:
--
Here are some numbers comparing the load performance for the factored vs
[
https://issues.apache.org/jira/browse/LUCENE-847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12518013
]
Steven Parkes commented on LUCENE-847:
--
For the time being, the patch also contains some of the code from LUCENE
[
https://issues.apache.org/jira/browse/LUCENE-847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12518162
]
Michael McCandless commented on LUCENE-847:
---
This looks great Steve!
More specific feeedback soon, but ...
[
https://issues.apache.org/jira/browse/LUCENE-847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12518165
]
Steven Parkes commented on LUCENE-847:
--
I
think we ideally would like concurrency to be fully in
[
https://issues.apache.org/jira/browse/LUCENE-847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12518184
]
Michael McCandless commented on LUCENE-847:
---
Some more feedback:
- Is the separate IndexMerger interfac
[
https://issues.apache.org/jira/browse/LUCENE-847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12518186
]
Michael McCandless commented on LUCENE-847:
---
> > I think we ideally would like concurrency to be fully inde
[
https://issues.apache.org/jira/browse/LUCENE-847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12518210
]
Steven Parkes commented on LUCENE-847:
--
Is the separate IndexMerger interface really necessary?
I wrestled
[
https://issues.apache.org/jira/browse/LUCENE-847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12518222
]
Steven Parkes commented on LUCENE-847:
--
On a related note, Mike, there a few FIXME's in IW related to useCompoun
[
https://issues.apache.org/jira/browse/LUCENE-847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12518237
]
Michael McCandless commented on LUCENE-847:
---
> Looking at IW, with the new DocsWriter stuff, it looks like
[
https://issues.apache.org/jira/browse/LUCENE-847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12518251
]
Steven Parkes commented on LUCENE-847:
--
Ah. I understand better now. I have to admit, I haven't kept up to date
[
https://issues.apache.org/jira/browse/LUCENE-847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12518263
]
Michael McCandless commented on LUCENE-847:
---
> I'm thinking maybe a MergePolicy#useCompoundDocStore( Segmen
[
https://issues.apache.org/jira/browse/LUCENE-847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12518435
]
Michael McCandless commented on LUCENE-847:
---
New feedback:
* Are you going to fix all unit tests that ca
[
https://issues.apache.org/jira/browse/LUCENE-847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12518453
]
Ning Li commented on LUCENE-847:
On 8/8/07, Michael McCandless (JIRA) <[EMAIL PROTECTED]> wrote:
> Actually I was tal
[
https://issues.apache.org/jira/browse/LUCENE-847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12518486
]
Ning Li commented on LUCENE-847:
The following comments are about the impact on merge if we add
"deleteDocument(int d
[
https://issues.apache.org/jira/browse/LUCENE-847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12518508
]
Michael McCandless commented on LUCENE-847:
---
> It just occurred to me that there is a neat way to handle de
[
https://issues.apache.org/jira/browse/LUCENE-847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12518520
]
Ning Li commented on LUCENE-847:
> Furthermore, I think this is all contained within IndexWriter, right?
> Ie when we
[
https://issues.apache.org/jira/browse/LUCENE-847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12519790
]
Steven Parkes commented on LUCENE-847:
--
Are you going to fix all unit tests that call the now-deprecated
[
https://issues.apache.org/jira/browse/LUCENE-847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12519793
]
Steven Parkes commented on LUCENE-847:
--
It just occurred to me that there is a neat way to handle delete
[
https://issues.apache.org/jira/browse/LUCENE-847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12520260
]
Michael McCandless commented on LUCENE-847:
---
One new small item: you've added a "public void merge()" to
In
[
https://issues.apache.org/jira/browse/LUCENE-847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12520293
]
Steven Parkes commented on LUCENE-847:
--
One new small item: you've added a "public void merge()" to
[
https://issues.apache.org/jira/browse/LUCENE-847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12520374
]
Michael McCandless commented on LUCENE-847:
---
> Note that merge() was added not for users (which I have no s
[
https://issues.apache.org/jira/browse/LUCENE-847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12520408
]
Steven Parkes commented on LUCENE-847:
--
I don't think so: I think if someone changes the merge policy to
[
https://issues.apache.org/jira/browse/LUCENE-847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12520874
]
Michael McCandless commented on LUCENE-847:
---
> > I don't think so: I think if someone changes the merge pol
[
https://issues.apache.org/jira/browse/LUCENE-847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12520881
]
Steven Parkes commented on LUCENE-847:
--
my feeling is we should not deprecate
setUseCompoundFile
[
https://issues.apache.org/jira/browse/LUCENE-847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12523621
]
Ning Li commented on LUCENE-847:
I include comments for both LUCENE-847 and LUCENE-870 here since they are
closely r
[
https://issues.apache.org/jira/browse/LUCENE-847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12523798
]
Michael McCandless commented on LUCENE-847:
---
Thanks for the detailed review Ning!
> 1 As you pointed out,
[
https://issues.apache.org/jira/browse/LUCENE-847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12523957
]
Ning Li commented on LUCENE-847:
> True, but I was thinking CMPW could be an exception to this rule. I
> guess I wou
[
https://issues.apache.org/jira/browse/LUCENE-847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12524039
]
Michael McCandless commented on LUCENE-847:
---
> > Good point... I think I could refactor this so that casc
[
https://issues.apache.org/jira/browse/LUCENE-847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12524084
]
Ning Li commented on LUCENE-847:
> Not quite following you here... not being eligible because the merge
> is in-progr
[
https://issues.apache.org/jira/browse/LUCENE-847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12524138
]
Michael McCandless commented on LUCENE-847:
---
> > Not quite following you here... not being eligible becaus
[
https://issues.apache.org/jira/browse/LUCENE-847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12525797
]
Doug Cutting commented on LUCENE-847:
-
Is there any reason not to make ConcurrentMergeScheduler the default too a
[
https://issues.apache.org/jira/browse/LUCENE-847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12525802
]
Michael McCandless commented on LUCENE-847:
---
> Is there any reason not to make ConcurrentMergeScheduler the
[
https://issues.apache.org/jira/browse/LUCENE-847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12525903
]
Michael McCandless commented on LUCENE-847:
---
OK, as a better test of ConcurrentMergeScheduler, and towards
[
https://issues.apache.org/jira/browse/LUCENE-847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12526029
]
Ning Li commented on LUCENE-847:
Comments on optimize():
- In the while loop of optimize(), LogMergePolicy.findMer
[
https://issues.apache.org/jira/browse/LUCENE-847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12526305
]
Michael McCandless commented on LUCENE-847:
---
> In the while loop of optimize(), LogMergePolicy.findMergesF
[
https://issues.apache.org/jira/browse/LUCENE-847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12526628
]
Ning Li commented on LUCENE-847:
> OK, another rev of the patch (take6). I think it's close!
Yes, it's close! :)
>
[
https://issues.apache.org/jira/browse/LUCENE-847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12526712
]
Michael McCandless commented on LUCENE-847:
---
> > Good idea! I took exactly this approach in patch I just at
[
https://issues.apache.org/jira/browse/LUCENE-847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12527224
]
Ning Li commented on LUCENE-847:
Access of mergeThreads in ConcurrentMergeScheduler.merge() should be
synchronized.
[
https://issues.apache.org/jira/browse/LUCENE-847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12527227
]
Michael McCandless commented on LUCENE-847:
---
Ahh, good catch. Will fix!
> Factor merge policy out of Inde
[
https://issues.apache.org/jira/browse/LUCENE-847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12527239
]
Ning Li commented on LUCENE-847:
Hmm, it's actually possible to have concurrent merges with SerialMergeScheduler.
Ma
[
https://issues.apache.org/jira/browse/LUCENE-847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12527258
]
Michael McCandless commented on LUCENE-847:
---
> Hmm, it's actually possible to have concurrent merges with
>
[
https://issues.apache.org/jira/browse/LUCENE-847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12527286
]
Ning Li commented on LUCENE-847:
> This was actually intentional: I thought it fine if the application is
> sending m
[
https://issues.apache.org/jira/browse/LUCENE-847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12527289
]
Mark Miller commented on LUCENE-847:
I have to triple check, but on first glance, my apps performance halfed usin
[
https://issues.apache.org/jira/browse/LUCENE-847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12527295
]
Michael McCandless commented on LUCENE-847:
---
> Today, applications use multiple threads on IndexWriter to
[
https://issues.apache.org/jira/browse/LUCENE-847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12527297
]
Michael McCandless commented on LUCENE-847:
---
> I have to triple check, but on first glance, my apps perform
[
https://issues.apache.org/jira/browse/LUCENE-847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12527300
]
Mark Miller commented on LUCENE-847:
Looks like some anomalous tests. Last night I checked twice, but today resul
[
https://issues.apache.org/jira/browse/LUCENE-847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12527304
]
Michael McCandless commented on LUCENE-847:
---
> Looks like some anomalous tests. Last night I checked twice,
Hi Steven,
I haven't read the details, but should maxBufferedDocs be exposed in
some subinterfaces instead of the MergePolicy interface? Since some
policies may use it and others may use byte size or something else.
It's great that you've started on concurrent merge as well! I haven't
got a chan
> I haven't read the details, but should maxBufferedDocs be exposed in
> some subinterfaces instead of the MergePolicy interface?
I've been wondering about this, too, but haven't come to any strong
opinions (yet). I figured maybe playing with a few merge policies might
make things clearer.
maxBuf
reflects the idea that to some extent, the merge
policy is factoring the number of file descriptors required into its
decision.
-Original Message-
From: Michael McCandless (JIRA) [mailto:[EMAIL PROTECTED]
Sent: Sunday, March 25, 2007 5:49 AM
To: java-dev@lucene.apache.org
Subject:
lucene.apache.org
Subject: [jira] Commented: (LUCENE-847) Factor merge policy out of
IndexWriter
[
https://issues.apache.org/jira/browse/LUCENE-847?page=com.atlassian.jira
.plugin.system.issuetabpanels:comment-tabpanel#action_12483930 ]
Michael McCandles
"Steven Parkes" <[EMAIL PROTECTED]> wrote:
> Yes, I'll separate out issues related to the basic refactor before
> submitting a candidate patch. I actually thought it might be helpful to
> keep it in the rough version to see context. But I can do that at any
> time ...
OK, that makes sense to leav
"Steven Parkes" <[EMAIL PROTECTED]> wrote:
> Well, with all due respect, I don't find whitespace malignant ...
Oh, sorry. I call it "cancerous" because it has a tendency to
spread uncontrollably throughout the source code :)
> That said, I don't get into this anymore. I make all the necessary
>
hael McCandless (JIRA) [mailto:[EMAIL PROTECTED]
Sent: Sunday, March 25, 2007 5:47 AM
To: java-dev@lucene.apache.org
Subject: [jira] Commented: (LUCENE-847) Factor merge policy out of
IndexWriter
[
https://issues.apache.org/jira/browse/LUCENE-847?page=com.atlassian.jira
.plugin.system.i
"Steven Parkes" <[EMAIL PROTECTED]> wrote:
> I've been wondering about taking minMergeDocs out of LMP
> (LogarithmicMergePolicy): if IW is doing maxBufferedDocs, can we get by
> with
> ceil(log(docs))
> rather than
> ceil(log(ceil(docs/minMergeDocs))
> (That's not exactly right, but it
On 3/23/07, Steven Parkes (JIRA) <[EMAIL PROTECTED]> wrote:
In fact, there a few things here that are fairly subtle/important. The relationship/protocol
between the writer and policy is pretty strong. This can be seen in the strawman concurrent
merge code where the merge policy holds state and
ansactions. I'm pretty wary of touching
that code. Is there a way around that?
-Original Message-
From: Ning Li [mailto:[EMAIL PROTECTED]
Sent: Wednesday, May 02, 2007 7:54 AM
To: java-dev@lucene.apache.org
Subject: Re: [jira] Commented: (LUCENE-847) Factor merge policy out of
Ind
Having the merge policy own segmentInfos sounds kind of hard to me.
Among other things, there's a lot of code in IndexWriter for managing
segmentInfos with regards to transactions. I'm pretty wary of touching
that code. Is there a way around that?
But conceptually, do you agree it's a good idea
On 8/7/07, Steven Parkes (JIRA) <[EMAIL PROTECTED]> wrote:
>
> [
> https://issues.apache.org/jira/browse/LUCENE-847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12518210
> ]
>
> Steven Parkes commented on LUCENE-847:
> --
>
>
64 matches
Mail list logo
