Hi Mike:
Thank you! It would be really nice to get the optimizations you have
done.
-John
2009/10/23 Michael McCandless <luc...@mikemccandless.com>
> Agreed: so far I'm seeing serious performance loss with MultiPQ,
> especially as topN gets larger, and for int sorting.
>
> For small queue, String sort, it sometimes wins.
>
> So if I were forced to decide now based on the current results, I
> think we should keep the single PQ API.
>
> But: I am right now optimizing John's patch to see how fast Multi PQ
> can get. I'll post it once I get it working, and post output from
> re-running on my opensolaris box.
>
> Mike
>
> 2009/10/23 Mark Miller <markrmil...@gmail.com>:
> >>>I still think we should if performance is no
> >>>better with the new one.
> >
> > Where is there any indication performance is not better with the new one?
> >
> > The benchmarks are clearly against switching back. At best they could
> argue for two API's - even then it depends - a loss of 10% on Java 1.5
> > with the most recent linux for a topn:10 ? I'm all for more results, but
> its not looking like a good switch to me. What API do I use? Well, it
> depends - how many docs will you ask for back, what OS are running, how hard
> is it for you to grok one API over the other?
> >
> > And then as we make changes in the future we have to manage both APIs.
> >
> > bq. digging in deep and running thorough perf tests makes sense
> >
> > Again - no one is arguing against - dig all year - I'll help - but I
> don't see the treasure yet, and the hole is starting to look deep.
> >
> > bq. removing that if from the Multi PQ patch makes sense
> >
> > I didn't have a problem with that either - or other code changes - but
> > jeeze, mention what you are seeing with the switch. I'll tell you what I
> > saw it - not that much - a bit of improvement, but take a look at the
> > Java 1.5 run - it ended up being a blade of grass holding up a boulder
> > on Linux.
> >
> >
> >
> > Michael McCandless wrote:
> >> Sheesh I go to bed and so much all of a sudden happens!!
> >>
> >> Sorry Mark; I should've called out "PATCH IS ON 2.9 BRANCH" more
> >> clearly ;)
> >>
> >> There's no question in my mind that the new comparator API is more
> >> complex than the old one, and I really don't like that. I had to
> >> rewrite the section of LIA that gives an example of a [simple] custom
> >> sort and it wasn't pleasant! Two compare methods (compare,
> >> compareBottom)? Two copy methods (copy, setBottom)? Sure, you can
> >> grok it and get through it if you have to, but it is more complex
> >> because it's conflated with the PQ API.
> >>
> >> Ease on consumption of our APIs is very important, so, only when
> >> performance clearly warrants it should we adopt a more complex API.
> >>
> >> Also, yeah, it would suck to have to switch back to the old API at
> >> this point, but net/net I still think we should if performance is no
> >> better with the new one.
> >>
> >> The old API also fits cleanly with per-segment searching (John's
> >> initial patch shows that -- it's simply another per-segment Colletor).
> >> The two APIs (collection, comparator) are well decoupled.
> >>
> >> So, digging in deep and running thorough perf tests makes sense; we
> >> need to understand the performance to make the API switch decision.
> >> And definitely we should tune both approaches as much as possible
> >> (removing that if from the Multi PQ patch makes sense).
> >>
> >> But... Multi PQ's performance isn't better in many cases... though,
> >> we're clearly still iterating. I'll run a 1.5 (32 & 64 bit) test,
> >> with the if statement removed.
> >>
> >> Mike
> >>
> >> On Fri, Oct 23, 2009 at 3:53 AM, Earwin Burrfoot <ear...@gmail.com>
> wrote:
> >>
> >>> I did.
> >>>
> >>> On Fri, Oct 23, 2009 at 09:05, Jake Mannix <jake.man...@gmail.com>
> wrote:
> >>>
> >>>> On Thu, Oct 22, 2009 at 9:58 PM, Mark Miller <markrmil...@gmail.com>
> wrote:
> >>>>
> >>>>> Yes - I've seen a handful of non core devs report back that they
> >>>>> upgraded with no complaints on the difficulty. Its in the mailing
> list
> >>>>> archives. The only core dev I've seen say its easy is Uwe. He's super
> >>>>> sharp though, so I wasn't banking my comment on him ;)
> >>>>>
> >>>> Upgrade custom sorting? Where has anyone talked about this?
> >>>>
> >>>> 2.9 is great, I like the new apis, they're great in general. It's
> just this
> >>>> multi-segment sorting we're talking about here.
> >>>>
> >>>> -jake
> >>>>
> >>>>
> >>>>
> >>>
> >>> --
> >>> Kirill Zakharenko/Кирилл Захаренко (ear...@gmail.com)
> >>> Home / Mobile: +7 (495) 683-567-4 / +7 (903) 5-888-423
> >>> ICQ: 104465785
> >>>
> >>> ---------------------------------------------------------------------
> >>> To unsubscribe, e-mail: java-dev-unsubscr...@lucene.apache.org
> >>> For additional commands, e-mail: java-dev-h...@lucene.apache.org
> >>>
> >>>
> >>>
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: java-dev-unsubscr...@lucene.apache.org
> >> For additional commands, e-mail: java-dev-h...@lucene.apache.org
> >>
> >>
> >
> >
> > --
> > - Mark
> >
> > http://www.lucidimagination.com
> >
> >
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: java-dev-unsubscr...@lucene.apache.org
> > For additional commands, e-mail: java-dev-h...@lucene.apache.org
> >
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscr...@lucene.apache.org
> For additional commands, e-mail: java-dev-h...@lucene.apache.org
>
>
