On Fri, Apr 24, 2015 at 9:21 AM, Josh Elser <josh.el...@gmail.com> wrote:
> Cool, it seems that we have consensus. Let me try to give a patch since > you all were nice enough to entertain me with a good discussion on the > matter! > > https://issues.apache.org/jira/browse/HBASE-13554 > > > Thanks for filing the issue. Do we want to say SemVer in 2.0+? St.Ack > Enis Söztutar wrote: > >> +1 to "working toward semver". Is there any documentation that we would >> like to clarify in the book given this enlightenment? >> >> I would be in favor of going 2.6.0 and jackson upgrade in 1.1. >> >> Enis >> >> On Thu, Apr 23, 2015 at 5:34 PM, Nick Dimiduk<ndimi...@gmail.com> wrote: >> >> On Thu, Apr 23, 2015 at 4:13 PM, Stack<st...@duboce.net> wrote: >>> >>> Does this 'admission' help with the which-hadoop thread too? >>>> >>>> Right -- "working toward semver". >>> >>> Are we now at liberty to bump to 2.6 or 2.7 even for branch-1.1? I still >>> have Karthik's offer to roll a 2.5.3 with the HDFS issue resolved. >>> >>> What about the jackson issue with 2.5 YARN runtime? >>> >>> Thanks Sean and Josh for being our community conscience on these issues. >>> >>> Just want to make sure we're all on the same page (or find out if not). >>>> >>>>> On Thu, Apr 23, 2015 at 2:59 PM, Enis Söztutar<e...@apache.org> >>>>> >>>> wrote: >>> >>>> Then let's get Andrew's proposal committed: >>>>>> >>>>>> + APIs available in a patch version will be available in all later >>>>>> + patch versions. However, new APIs may be added which will not be >>>>>> + available in earlier patch versions. >>>>>> >>>>>> I propose the following change to the "Client Binary compatibility" >>>>>> >>>>> section >>>>> >>>>>> of Section 11.1: >>>>>> >>>>>> - Old client code can run unchanged (no recompilation needed) against >>>>>> >>>>> new >>>> >>>>> jars. >>>>>> + Client code written to APIs available in a given patch release >>>>>> + can run unchanged (no recompilation needed) against the new >>>>>> + jars of later patch versions. >>>>>> >>>>>> We can even claim that if you are not using new APIs, we are binary >>>>>> >>>>> compat >>>>> >>>>>> for downgrade. But you have to make sure that your code compiles with >>>>>> >>>>> that >>>>> >>>>>> downgraded version. >>>>>> >>>>>> Enis >>>>>> >>>>>> >>>>>> >>>>>> On Thu, Apr 23, 2015 at 2:04 PM, Sean Busbey<bus...@cloudera.com> >>>>>> >>>>> wrote: >>>>> >>>>>> I'm fine with us adding methods to public APIs in patch releases so >>>>>>> >>>>>> long >>>>> >>>>>> as >>>>>> >>>>>>> we stop claiming to follow semver. We can say we take the >>>>>>> >>>>>> principles >>> >>>> of >>>> >>>>> semver as inspiration. This would reflect the current state of the >>>>>>> >>>>>> world >>>>> >>>>>> WRT 1.0.1 and would still give us a reason keep the narrower >>>>>>> >>>>>> definition >>>> >>>>> of >>>>>> >>>>>>> "new features" in minor versions. >>>>>>> >>>>>>> >>>>>>> No longer claiming semver would also have the added benefit of >>>>>>> >>>>>> making >>> >>>> it >>>>> >>>>>> for me to easier to explain our compatibility promises to people. >>>>>>> >>>>>> Right >>>> >>>>> now >>>>>> >>>>>>> I have to explain the difference between the things that get proper >>>>>>> >>>>>> semver >>>>>> >>>>>>> treatment (e.g. public api, wire compat) and which things are >>>>>>> >>>>>> downgraded >>>>> >>>>>> to >>>>>> >>>>>>> breaking-on-minor (e.g. LimitedPrivate, binary compatibility). >>>>>>> >>>>>> Starting >>>> >>>>> with the context of "we like semver" will be easier than "we use >>>>>>> >>>>>> semver". >>>>> >>>>>> >>>>>>> Like Josh, my main concern is that we accurately advertise what >>>>>>> >>>>>> we're >>> >>>> doing. There are few things I've found more frustrating than being >>>>>>> >>>>>> an >>> >>>> end >>>>> >>>>>> user of a project that claims to follow semver without >>>>>>> >>>>>> understanding >>> >>>> the >>>>> >>>>>> burden that carries (and subsequently not meeting it). >>>>>>> >>>>>>> On Thu, Apr 23, 2015 at 3:48 PM, Stack<st...@duboce.net> wrote: >>>>>>> >>>>>>> I agree w/ the Enis characterization (so we need the callout on >>>>>>>> >>>>>>> semvar) >>>>> >>>>>> but >>>>>>> >>>>>>>> think we should practice what Seans says (patch is bug fixes >>>>>>>> >>>>>>> only). >>> >>>> St.Ack >>>>>>>> >>>>>>>> On Thu, Apr 23, 2015 at 1:31 PM, Sean Busbey< >>>>>>>> >>>>>>> bus...@cloudera.com> >>> >>>> wrote: >>>>>>> >>>>>>>> Why don't we just focus development after a minor release on >>>>>>>>> >>>>>>>> the >>> >>>> next >>>>> >>>>>> minor >>>>>>>> >>>>>>>>> release instead of the next patch release? >>>>>>>>> >>>>>>>>> We could limit backports to the patch releases to critical >>>>>>>>> >>>>>>>> bugs, >>> >>>> which >>>>>> >>>>>>> would cut down on how often someone has to deal with the pain >>>>>>>>> >>>>>>>> of >>> >>>> making >>>>>> >>>>>>> sure we don't add to public APIs. It also reduces the risk >>>>>>>>> >>>>>>>> someone >>>> >>>>> going >>>>>>> >>>>>>>> through an upgrade has, since there are fewer changes. >>>>>>>>> >>>>>>>>> If someone fixes a bug and doesn't want to do the work of >>>>>>>>> >>>>>>>> making >>> >>>> sure >>>>> >>>>>> it >>>>>>> >>>>>>>> doesn't add methods in a patch release, they just don't >>>>>>>>> >>>>>>>> backport >>> >>>> to >>>> >>>>> that >>>>>>> >>>>>>>> version and make a follow on e.g. "backport to 1.0.z" ticket. >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> On Thu, Apr 23, 2015 at 1:50 PM, Enis Söztutar< >>>>>>>>> >>>>>>>> enis....@gmail.com >>>> >>>>> wrote: >>>>>>>> >>>>>>>>> +1 to the proposal. >>>>>>>>>> >>>>>>>>>> The problem is that we have a very big API surface especially >>>>>>>>>> >>>>>>>>> with >>>>> >>>>>> the >>>>>>> >>>>>>>> coprocessors included in the report. Even simple bug fixes >>>>>>>>>> >>>>>>>>> can >>> >>>> introduce >>>>>>>> >>>>>>>>> protected or public methods to base classes, which makes >>>>>>>>>> >>>>>>>>> patch >>> >>>> releases >>>>>>> >>>>>>>> very hard to maintain. I would not want to spend the effort >>>>>>>>>> >>>>>>>>> to >>> >>>> spend >>>>>> >>>>>>> tons >>>>>>>> >>>>>>>>> of time trying to make a patch not introduce new methods in >>>>>>>>>> >>>>>>>>> order >>>> >>>>> to >>>>>> >>>>>>> backport. That effort can be spent elsewhere IMO. >>>>>>>>>> >>>>>>>>>> Looking at the report >>>>>>>>>> >>>>>>>>>> >>>>> https://people.apache.org/~enis/1.0.0_1.0.1RC2_compat_report.html, >>>>> >>>>>> nothing >>>>>>>>> >>>>>>>>>> strikes me as "new functionality". Going from current 1.0.0 >>>>>>>>>> >>>>>>>>> to >>> >>>> 1.0.1RC2 >>>>>>> >>>>>>>> should actually be as you would expect from upgrading a patch >>>>>>>>>> >>>>>>>>> release. >>>>>>> >>>>>>>> Yes, adding new API in patch releases will make downgrading >>>>>>>>>> >>>>>>>>> harder, >>>>> >>>>>> but I >>>>>>>> >>>>>>>>> think that is an acceptable tradeoff. We can document that if >>>>>>>>>> >>>>>>>>> your >>>>> >>>>>> application compiles (meaning that you are not using new API) >>>>>>>>>> >>>>>>>>> with >>>>> >>>>>> 1.0.0, >>>>>>>> >>>>>>>>> then you can swap your jars in a binary compat manner. >>>>>>>>>> >>>>>>>>>> Enis >>>>>>>>>> >>>>>>>>>> On Thu, Apr 23, 2015 at 10:03 AM, Andrew Purtell< >>>>>>>>>> >>>>>>>>> apurt...@apache.org> >>>>>>> >>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>> Anyone disagree with the point of view put forward by Josh >>>>>>>>>>> >>>>>>>>>> and >>>> >>>>> Sean? >>>>>>> >>>>>>>> >>>>>>>>>>> On Wed, Apr 22, 2015 at 10:35 AM, Josh Elser< >>>>>>>>>>> >>>>>>>>>> josh.el...@gmail.com >>>>>> >>>>>>> wrote: >>>>>>>>>> >>>>>>>>>>> Andy -- I understood your intent, but thanks for >>>>>>>>>>>> >>>>>>>>>>> clarifying. >>>> >>>>> (as >>>>>> >>>>>>> well >>>>>>>> >>>>>>>>> as >>>>>>>>>> >>>>>>>>>>> taking the time to break this discussion out in the first >>>>>>>>>>>> >>>>>>>>>>> place). I >>>>>>> >>>>>>>> agree >>>>>>>>>> >>>>>>>>>>> with your assessment. >>>>>>>>>>>> >>>>>>>>>>>> re: Sean's comments, if it wasn't clear by me asking in >>>>>>>>>>>> >>>>>>>>>>> the >>> >>>> first >>>>>> >>>>>>> place, >>>>>>>>>> >>>>>>>>>>> I >>>>>>>>>>> >>>>>>>>>>>> also think sticking as close as possible to semver's >>>>>>>>>>>> >>>>>>>>>>> rules >>> >>>> is >>>> >>>>> the >>>>>> >>>>>>> best >>>>>>>>> >>>>>>>>>> approach, although I'm getting the impression that there >>>>>>>>>>>> >>>>>>>>>>> have >>>> >>>>> been >>>>>>> >>>>>>>> some >>>>>>>>> >>>>>>>>>> previous reservations to doing so (especially by your >>>>>>>>>>>> >>>>>>>>>>> comment >>>> >>>>> about >>>>>>> >>>>>>>> backporting features if there is demand is). >>>>>>>>>>>> >>>>>>>>>>>> I've found adhering to the bug-fix release restrictions >>>>>>>>>>>> >>>>>>>>>>> can >>> >>>> be >>>>> >>>>>> a >>>>>> >>>>>>> very >>>>>>>> >>>>>>>>> painful and time-consuming task, so this is something to >>>>>>>>>>>> >>>>>>>>>>> get >>>> >>>>> a >>>>> >>>>>> representative sampling of those who do the work to make >>>>>>>>>>>> >>>>>>>>>>> sure >>>> >>>>> everyone >>>>>>>>> >>>>>>>>>> is >>>>>>>>>> >>>>>>>>>>> on board. >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Sean Busbey wrote: >>>>>>>>>>>> >>>>>>>>>>>> I'd much rather we stick with the definitions used in >>>>>>>>>>>>> >>>>>>>>>>>> Semantic >>>>> >>>>>> Versioning. >>>>>>>>>>> >>>>>>>>>>>> Our use is already confusing enough given our matrix of >>>>>>>>>>>>> >>>>>>>>>>>> compatibilities >>>>>>>>>> >>>>>>>>>>> that don't get "major version for breaking" protections. >>>>>>>>>>>>> >>>>>>>>>>>>> We've previously discussed how we'll do additional minor >>>>>>>>>>>>> >>>>>>>>>>>> releases >>>>>>> >>>>>>>> when >>>>>>>>> >>>>>>>>>> there's sufficient interest in the new features present >>>>>>>>>>>>> >>>>>>>>>>>> there. >>>>> >>>>>> What's >>>>>>>>> >>>>>>>>>> building that demand if any backwards compatible change >>>>>>>>>>>>> >>>>>>>>>>>> can >>>> >>>>> go >>>>> >>>>>> back >>>>>>>> >>>>>>>>> into a >>>>>>>>>>> >>>>>>>>>>>> patch release? >>>>>>>>>>>>> >>>>>>>>>>>>> Would we have an easier time restraining ourselves if we >>>>>>>>>>>>> >>>>>>>>>>>> had a >>>>> >>>>>> regular >>>>>>>>> >>>>>>>>>> schedule planned around new minor versions? >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> On Wed, Apr 22, 2015 at 12:03 PM, Josh Elser< >>>>>>>>>>>>> >>>>>>>>>>>> josh.el...@gmail.com >>>>>>> >>>>>>>> wrote: >>>>>>>>>>>>> >>>>>>>>>>>>> While I can understand the desire to want to add >>>>>>>>>>>>> >>>>>>>>>>>> things, >>> >>>> I >>>> >>>>> do >>>>> >>>>>> think >>>>>>>> >>>>>>>>> it >>>>>>>>>> >>>>>>>>>>> makes things harder for users to reliably write code >>>>>>>>>>>>>> >>>>>>>>>>>>> against >>>>> >>>>>> versions >>>>>>>>> >>>>>>>>>> of >>>>>>>>>>> >>>>>>>>>>>> HBase which (by their view) should be completely >>>>>>>>>>>>>> >>>>>>>>>>>>> compatible >>>> >>>>> with >>>>>>> >>>>>>>> one >>>>>>>>> >>>>>>>>>> another. >>>>>>>>>>>>>> >>>>>>>>>>>>>> Take this extremely hypothetical situation: I'm new to >>>>>>>>>>>>>> >>>>>>>>>>>>> HBase >>>>> >>>>>> and >>>>>>> >>>>>>>> start >>>>>>>>>> >>>>>>>>>>> writing some code against HBase 1.0.1 which was just >>>>>>>>>>>>>> >>>>>>>>>>>>> deployed >>>>> >>>>>> at >>>>>>> >>>>>>>> my >>>>>>>> >>>>>>>>> $job. I >>>>>>>>>>>>>> don't _know_ what APIs are new, I just know what exists >>>>>>>>>>>>>> >>>>>>>>>>>>> and >>>> >>>>> treat >>>>>>> >>>>>>>> that >>>>>>>>>> >>>>>>>>>>> as >>>>>>>>>>> >>>>>>>>>>>> acceptable for me to be using. Meanwhile in production, >>>>>>>>>>>>>> >>>>>>>>>>>>> some >>>>> >>>>>> other >>>>>>>> >>>>>>>>> people >>>>>>>>>>> >>>>>>>>>>>> find a bug with HBase 1.0.1 and roll back to 1.0.0 >>>>>>>>>>>>>> >>>>>>>>>>>>> which >>> >>>> they >>>>> >>>>>> had >>>>>>> >>>>>>>> been >>>>>>>>>> >>>>>>>>>>> previously using. My reaction would be "of course my >>>>>>>>>>>>>> >>>>>>>>>>>>> code >>> >>>> should >>>>>>> >>>>>>>> work >>>>>>>>> >>>>>>>>>> with >>>>>>>>>>>>>> HBase 1.0.0, I only used the public API" when in fact >>>>>>>>>>>>>> >>>>>>>>>>>>> this >>>> >>>>> is >>>>> >>>>>> not >>>>>>> >>>>>>>> true. >>>>>>>>>> >>>>>>>>>>> Personally, I think it's a little bold to say semver is >>>>>>>>>>>>>> >>>>>>>>>>>>> even >>>>> >>>>>> in >>>>>> >>>>>>> use >>>>>>>> >>>>>>>>> if >>>>>>>>>> >>>>>>>>>>> this principal isn't being followed as it doesn't >>>>>>>>>>>>>> >>>>>>>>>>>>> follow >>> >>>> at >>>> >>>>> all >>>>>> >>>>>>> with >>>>>>>>> >>>>>>>>>> my >>>>>>>>>> >>>>>>>>>>> understanding on the guarantees defined by semver for >>>>>>>>>>>>>> >>>>>>>>>>>>> bug-fix >>>>> >>>>>> releases. >>>>>>>>>> >>>>>>>>>>> That being said, if the intent *is* to allow ourselves >>>>>>>>>>>>>> >>>>>>>>>>>>> to >>> >>>> make >>>>>> >>>>>>> these >>>>>>>>> >>>>>>>>>> sorts >>>>>>>>>>>>>> of changes, I just think some sort of disclaimer should >>>>>>>>>>>>>> >>>>>>>>>>>>> be >>>> >>>>> present: >>>>>>>> >>>>>>>>> - HBase uses Semantic Versioning for its release >>>>>>>>>>>>>> >>>>>>>>>>>>> versioning >>>> >>>>> + HBase uses Semantic Versioning for its release >>>>>>>>>>>>>> >>>>>>>>>>>>> versioning >>>> >>>>> with >>>>>>> >>>>>>>> a >>>>>>>> >>>>>>>>> caveat >>>>>>>>>>> >>>>>>>>>>>> that methods and members might be added in newer >>>>>>>>>>>>>> >>>>>>>>>>>>> bug-fix >>> >>>> releases >>>>>>> >>>>>>>> that >>>>>>>>>> >>>>>>>>>>> were >>>>>>>>>>>>>> not present in the previous bug-fix release. >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> Andrew Purtell wrote: >>>>>>>>>>>>>> >>>>>>>>>>>>>> [Subject changed] >>>>>>>>>>>>>> >>>>>>>>>>>>>>> On Tue, Apr 21, 2015 at 8:47 PM, Josh Elser< >>>>>>>>>>>>>>> >>>>>>>>>>>>>> josh.el...@gmail.com> >>>>>>>> >>>>>>>>> wrote: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> I was a little surprised when I noticed method >>>>>>>>>>>>>>> >>>>>>>>>>>>>> additions >>>> >>>>> to >>>>>> >>>>>>> InterfaceAudience.Public annotated classes. This >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> means >>> >>>> that a >>>>>> >>>>>>> user >>>>>>>>> >>>>>>>>>> could >>>>>>>>>>>>>>>> write code against 1.0.1 that would not work against >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> 1.0.0 >>>>> >>>>>> which >>>>>>>> >>>>>>>>> seems >>>>>>>>>>> >>>>>>>>>>>> undesirable for a bugfix release. I read over the >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> book >>> >>>> section >>>>>>> >>>>>>>> on >>>>>>>> >>>>>>>>> compatibility and didn't see this addressed, so I >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> thought >>>> >>>>> I'd >>>>>> >>>>>>> ask. >>>>>>>>> >>>>>>>>>> >>>>>>>>>>>>>>>> Let's clarify this. It's not the first time this >>>>>>>>>>>>>>> >>>>>>>>>>>>>> question >>>> >>>>> has >>>>>> >>>>>>> been >>>>>>>> >>>>>>>>> asked. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> To get things moving: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> I propose the following addition to the "Client API >>>>>>>>>>>>>>> >>>>>>>>>>>>>> compatibility" >>>>>>>> >>>>>>>>> section >>>>>>>>>>>>>>> of Section 11.1: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> + APIs available in a patch version will be available >>>>>>>>>>>>>>> >>>>>>>>>>>>>> in >>> >>>> all >>>>> >>>>>> later >>>>>>>> >>>>>>>>> + patch versions. However, new APIs may be added which >>>>>>>>>>>>>>> >>>>>>>>>>>>>> will >>>>> >>>>>> not >>>>>>> >>>>>>>> be >>>>>>>> >>>>>>>>> + available in earlier patch versions. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> I propose the following change to the "Client Binary >>>>>>>>>>>>>>> >>>>>>>>>>>>>> compatibility" >>>>>>>>> >>>>>>>>>> section >>>>>>>>>>>>>>> of Section 11.1: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> - Old client code can run unchanged (no recompilation >>>>>>>>>>>>>>> >>>>>>>>>>>>>> needed) >>>>>> >>>>>>> against >>>>>>>>>> >>>>>>>>>>> new >>>>>>>>>>>>>>> jars. >>>>>>>>>>>>>>> + Client code written to APIs available in a given >>>>>>>>>>>>>>> >>>>>>>>>>>>>> patch >>> >>>> release >>>>>>> >>>>>>>> + can run unchanged (no recompilation needed) against >>>>>>>>>>>>>>> >>>>>>>>>>>>>> the >>>> >>>>> new >>>>>> >>>>>>> + jars of later patch versions. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> What do you think? >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> If these changes are (mostly) ok, then this clarifies >>>>>>>>>>>>>>> >>>>>>>>>>>>>> in >>> >>>> one >>>>> >>>>>> direction. >>>>>>>>>>> >>>>>>>>>>>> If these changes are not acceptable, I will propose >>>>>>>>>>>>>>> >>>>>>>>>>>>>> edits >>>> >>>>> that >>>>>> >>>>>>> clarify >>>>>>>>>> >>>>>>>>>>> toward the opposite meaning. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>
