We have never promised or delivered back compat on Java APIs beyond best effort. I agree we should stick to http APIs and solrj for stronger back compat, and simply do our best to be reasonable with the rest.
Mark On Tue, Jan 5, 2016 at 2:05 AM Shai Erera <ser...@gmail.com> wrote: > In Lucene there are three types of public APIs: > > "public stable": not annotated and for them we try to maintain backward > compatibility across minor releases > "public experimental": annotated with @lucene.experimental. If possible, > back-compat is maintained, but otherwise you can expect them to change > between minor versions > "public internal": annotated with @lucene.internal. Due to Java > limitations, these are public classes/methods, but we have no intention to > keep them backward compatible. > > Even the "public stable" APIs are further divided, though not through > annotations, and more a feel and consensus on a particular issue, into > expert and non-expert APIs. Take for example the overhaul that was done to > SpanQuery in LUCENE-6308. I believe it was decided that users who implement > their own SpanQuery, are probably expert users and can take the API changes. > > When it comes to Solr, I always viewed user-facing APIs as SolrJ and the > REST APIs and therefore referred to them as "public stable". When it comes > to plug-ins, I view them at most as "public expert". I don't think that > Solr users who write their own plug-ins should be viewed as simple users. > They are more than the average user IMO. Even if the plug-in that they > write is super simple, like adding a log message, how bad can an API break > be for them to change their code? > > I think that, even if not publicly announced, in Lucene we sort of dropped > the requirement of jar drop-in ability. I personally believe that there are > only few users who expect to upgrade a search server running Solr from > version X to Y, without this affecting their code. So for the majority of > users who use SolrJ and the REST APIs, this will be the case since we > guarantee back-compat. For those who write their own plug-ins, it may be a > matter of luck. Either they do or don't have to recompile their code. For > the super expert ones, e.g. who write a plug-in from scratch or do really > invasive things, I think it's fair to ask them to align with the new APIs. > > I implemented a bunch of SpanQuery who worked fine in 5.2.1. Last week > I've decided to upgrade to 5.4 and had to rewrite the queries. In the > process, I discovered that the new API helps me implement my queries > better, as well one of the queries was now available from Lucene. I didn't > complain about it, as painful as it was, since I trust the developers who > have made that decision to weigh the pros and cons of the API changes. > > Also, I find it odd that some of the arguments made here distinguish Solr > users from Lucene users. Ain't a Solr user who implements his own > QParsePlugin subject to the same API changes (e.g. SpanQuery) in Lucene, > that non-Solr users are? Why do we even attempt to make that distinction? > Solr is built on top of Lucene, so most likely the plugin that you write > will need to interact w/ Lucene API too. > > As to the particular issue in SOLR-8475, the API break will be resolved by > any modern IDE with a simple "Organize Imports". And I don't view > SolrIndexSearcher as anywhere near the top-level of the APIs that we should > maintain. Nor do I think that SolrCache is one such (as another example, > unrelated to that issue). The interfaces that define plugins, maybe. But > the internals are subject, IMO, to either be "solr expert" or "solr > internal". > > If we want to refactor those pieces in Solr (and I think that we should, > and I try to help with it), we must be more flexible around our API > guarantees. Also, trust the devs to not rewrite APIs just for the hack of > it, but if they do, it's for a good reason, for them (as code maintainers) > and our users. > > Shai > > On Tue, Jan 5, 2016 at 8:36 AM Noble Paul <noble.p...@gmail.com> wrote: > >> I would say, SolrJ and REST APIs MUST BE backward compatible between >> minor versions. >> >> The question is about the internal java APIs. It is impossible to get >> 100% right on these things. If we can start annotating classes/methods >> and let users suggest stuff then we should be in a reasonably good >> situation over an year. >> As a first step open a ticket and define a process to make a certain >> API backcompat . >> may be an annotation or whatever. >> >> On Tue, Jan 5, 2016 at 11:19 AM, Anshum Gupta <ans...@anshumgupta.net> >> wrote: >> > Thanks David, >> > >> > I agree with what you've suggested but the bigger question here again is >> > *which* files do we guarantee back-compat for. I suggest we guarantee >> > back-compat for SolrJ and REST APIs. For everything else i.e. Java >> APIs, we >> > should try and maintain back-compat but there shouldn't be a guarantee >> and >> > should be the developer's prerogative else defining what is to be >> annotated >> > as "Long term support" and what isn't becomes subject to usage and >> debate. >> > >> > On Tue, Jan 5, 2016 at 10:32 AM, david.w.smi...@gmail.com >> > <david.w.smi...@gmail.com> wrote: >> >> >> >> Great topic Anshum. >> >> >> >> I’ve been frustrated with the back-compat situation since Lucene/Solr >> 5 as >> >> a maintainer of my “SolrTextTagger”. One version of the plugin >> supports 4.3 >> >> thru the end of the 4x line, whereas on the 5x side I’ve needed 3 >> versions >> >> already (5.0-5,1, 5.2, 5.3+)! Sometimes on the API consumer side >> there is >> >> simply a new way compatible with the old way and those don’t bother me >> much; >> >> what’s annoying is just flat-out differences that can’t easily be coded >> >> around without reflection. But in my experience through the >> SolrTextTagger >> >> (YMMV), Solr hasn’t been the main offender of such changes — it’s >> Lucene >> >> changes (be it removing liveDocs bits to get postings, changing the >> >> BytesRefAttribute signature, or something I don’t remember now). At >> least >> >> twice it was avoidable IMO. Nevertheless, we Solr devs should come up >> with >> >> a back-compat policy, a simple document/paragraph perhaps, and save it >> >> somewhere so we can refer to it. Lets not have to dig through the >> mailing >> >> list to know our policy some day in the future when we want to explain >> it! >> >> >> >> I suggest that Solr's *default* policy for any source file (Java API) >> that >> >> doesn’t otherwise annotate a back-compat statement is to be permissive >> to >> >> changes — developer judgement on how much back-compat makes sense to >> them. >> >> I say this because the Solr code base is large and I think a relatively >> >> small portion of it should aim for stability. Lets take >> SearchComponent as >> >> an example. *That* needs to be stable. But does HighlightComponent? >> I >> >> really don’t think so; besides, it only has one overridable method >> defined >> >> by this class that isn’t inherited. Oddly (IMO) there is a separate >> >> abstraction SolrHighlighter and I can intuit that it’s this guy that >> was >> >> intended to be the abstraction of the Highlighter implementation, not >> the >> >> some-what generic HighlightComponent. So arguably SolrHighlighter >> should be >> >> stable. DefaultSolrHighlighter is debatable as being stable — it’s a >> >> specific highlighter but it has a bunch of methods designed to be >> overridden >> >> (and I have done so). So I think that’s a judgement call (developer >> >> prerogative). >> >> >> >> Should we apply a back-compat policy statement (either through a simple >> >> comment or better through a new annotation), I don’t think we should >> feel >> >> helpless to strictly abide by it for the entire major version range. >> We >> >> might decide that such changes are possible provided it gets at least >> one +1 >> >> and no -1 veto from another developer. >> >> >> >> Summary: >> >> * Publish a back-compat policy/approach where we can refer to it >> easily. >> >> * The default policy of source files without annotations is the >> >> developer’s prerogative — no back-compat. >> >> * Annotate the back-compat Java source files as-such and allow us to >> break >> >> back-compat only if voted. >> >> >> >> ~ David >> >> >> >> On Mon, Jan 4, 2016 at 11:28 AM Anshum Gupta <ans...@anshumgupta.net> >> >> wrote: >> >>> >> >>> Hi, >> >>> >> >>> I was looking at refactoring code in Solr and it gets really tricky >> and >> >>> confusing in terms of what level of back-compat needs to be >> maintained. >> >>> Ideally, we should only maintain back-compat at the REST API level. >> We may >> >>> annotate a few really important Java APIs where we're guarantee >> back-compat >> >>> across minor versions, but we shouldn't certainly be doing that >> across the >> >>> board. >> >>> >> >>> Thoughts? >> >>> >> >>> P.S: I hope this doesn't spin-off into something I fear :) >> >>> >> >>> -- >> >>> Anshum Gupta >> >> >> >> -- >> >> Lucene/Solr Search Committer, Consultant, Developer, Author, Speaker >> >> LinkedIn: http://linkedin.com/in/davidwsmiley | Book: >> >> http://www.solrenterprisesearchserver.com >> > >> > >> > >> > >> > -- >> > Anshum Gupta >> >> >> >> -- >> ----------------------------------------------------- >> Noble Paul >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org >> For additional commands, e-mail: dev-h...@lucene.apache.org >> >> -- - Mark about.me/markrmiller
