[
https://issues.apache.org/jira/browse/LUCENE-8267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16448748#comment-16448748
]
David Smiley commented on LUCENE-8267:
--------------------------------------
Ah; I incorrectly assumed the proposal included the FST postings formats but
apparently not. It's too bad FSTPulsingFormat is long gone though since in the
text-tagging use-case it'd effectively be a substitute for
MemoryPostingsFormat. The FSTTermsReader accepts a PostingsReaderBase; maybe
it's possible to write an in-memory version of PostingsReaderBase, at least for
the "pulsed" (single posting) case. Nonetheless lets see how the text tagger
performs with these codec options.
Thanks for the suggestion to use MMapDirectory.preload, I didn't know about it,
but that appears to only help warmup, not sustained performance; right? And I
believe even with FileSwitchDirectory, on shutdown files with certain
extensions would vanish; right?
{quote}So I perceive your veto as an aggressive step. To me it's a last resort
after we can't find a solution that is good for all of us. The conversation
already has a tone that is not appropriate and could have been prevented by
formulating objections as questions. like I am using this postings format in X
and it's serving well, what are the alternatives. - I am sure you would have
got an awesome answer.
{quote}
The "sorry" word immediately after my veto was intended to prevent
misperceptions about tone; I don't mean to be aggressive – sorry! I agree I
could have asked for alternatives up-front; I'll try and remember that next
time. I was thinking my early vote could prevent work that someone does in vein
to remove these pieces. In retrospect I didn't need to vote yet to accomplish
that (e.g. convey disagreement with others). In this way I was trying to offer
improved communication where from other's I've seen no veto but a confusing
cloud of doubt as to wether there would be a veto or not (which in my mind is
worse). I respect you may feel differently though; just please understand my
intended tone is not aggressive.
{quote}if you can't remove stuff without others jumping in vetoing the reaction
will be to prevent additions in the same way due to _fear_ created by the
veto. This is a terrible place to be in, we have seen this in the past we
should prevent it.
{quote}
Do you mean if we add some new thingamajig, we might feel that we *have* to
support it indefinitely? (I wouldn't use the word "fear" for this; maybe I've
got your intent wrong still) Hmmm; I think it's very situationally dependent.
For example with queryNorm & coords, LUCENE-7347, I had concerns but ultimately
understood that maintaining these things were making things awkward for us.
But the PostingsFormats seem different to me. They conform to our APIs; they
don't get in the way or tie our hands. Yes there is maintenance though. I
think what I objected to most in the description of this issue was the notion
that, because Lucene-core doesn't use something and because there is
maintenance to that something, then we should delete that something. I get the
maintenance aspect but we need community input on such decisions to ascertain
real-world use.
> Remove memory codecs from the codebase
> --------------------------------------
>
> Key: LUCENE-8267
> URL: https://issues.apache.org/jira/browse/LUCENE-8267
> Project: Lucene - Core
> Issue Type: Task
> Reporter: Dawid Weiss
> Priority: Major
>
> Memory codecs (MemoryPostings*, MemoryDocValues*) are part of random
> selection of codecs for tests and cause occasional OOMs when a test with huge
> data is selected. We don't use those memory codecs anywhere outside of tests,
> it has been suggested to just remove them to avoid maintenance costs and OOMs
> in tests. [1]
> [1] https://apache.markmail.org/thread/mj53os2ekyldsoy3
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
