Hi Xintong,

Thanks for the explanation. Please see the replies inline below.

I agree. And from my understanding, demoting a Public API is also a kind of
> such change, just like removing one, which can only happen with major
> version bumps. I'm not proposing to allow demoting Public APIs anytime, but
> only in the case major version bumps happen before reaching the
> 2-minor-release migration period. Actually, demoting would be a weaker
> change compared to removing the API immediately upon major version bumps,
> in order to keep the commitment about the 2-minor-release migration period.
> If the concern is that `@Public` -> `@PublicEvolving` sounds against
> conventions, we may introduce a new annotation if necessary, e.g.,
> `@PublicRetiring`, to avoid confusions.

As an end user who only uses Public APIs, if I don't change my code at all,
my expectation is the following:
1. Upgrading from 1.x to 2.x may have issues.
2. If I can upgrade from 1.x to 2.x without an issue, I am fine with all
the 2.x versions.
Actually I think there are some dependency version resolution policies out
there which picks the highest minor version when the dependencies pull in
multiple minor versions of the same jar, which may be broken if we remove
the API in minor releases.

I'm not sure about this. Yes, it's completely "legal" that we bump up the
> major version whenever a breaking change is needed. However, this also
> weakens the value of the commitment that public APIs will stay stable
> within the major release series, as the series can end anytime. IMHO, short
> major release series are not something "make the end users happy", but
> backdoors that allow us as the developers to make frequent breaking
> changes. On the contrary, with the demoting approach, we can still have
> longer major release series, while only allowing Public APIs deprecated at
> the end of the previous major version to be removed in the next major
> version.

I totally agree that frequent major version bumps are not ideal, but here
we are comparing it with a minor version bump which removes a Public API.
So the context is that we have already decided to remove this Public API
while keeping everything else backwards compatible. I think a major version
bump is a commonly understood signal here, compared with a minor version
change. From end users' perspective, for those who are not impacted, in
this case upgrading a major version is not necessarily more involved than
upgrading a minor version - both should be as smooth as a dependency
version change. For those who are impacted, they will lose the Public API
anyways and a major version bump ensures there is no surprise.

Thanks,

Jiangjie (Becket) Qin

On Fri, Jun 16, 2023 at 10:13 AM Xintong Song <tonysong...@gmail.com> wrote:

> Public API is a well defined common concept, and one of its
>> convention is that it only changes with a major version change.
>>
>
> I agree. And from my understanding, demoting a Public API is also a kind
> of such change, just like removing one, which can only happen with major
> version bumps. I'm not proposing to allow demoting Public APIs anytime, but
> only in the case major version bumps happen before reaching the
> 2-minor-release migration period. Actually, demoting would be a weaker
> change compared to removing the API immediately upon major version bumps,
> in order to keep the commitment about the 2-minor-release migration period.
> If the concern is that `@Public` -> `@PublicEvolving` sounds against
> conventions, we may introduce a new annotation if necessary, e.g.,
> `@PublicRetiring`, to avoid confusions.
>
> But it should be
>> completely OK to bump up the major version if we really want to get rid of
>> a public API, right?
>>
>
> I'm not sure about this. Yes, it's completely "legal" that we bump up the
> major version whenever a breaking change is needed. However, this also
> weakens the value of the commitment that public APIs will stay stable
> within the major release series, as the series can end anytime. IMHO, short
> major release series are not something "make the end users happy", but
> backdoors that allow us as the developers to make frequent breaking
> changes. On the contrary, with the demoting approach, we can still have
> longer major release series, while only allowing Public APIs deprecated at
> the end of the previous major version to be removed in the next major
> version.
>
> Given our track record I would prefer a regular cycle (1-2 years) to
>> force us to think about this whole topic, and not put it again to the
>> wayside and giving us (and users) a clear expectation on when breaking
>> changes can be made.
>>
>
> +1. I personally think 2-3 years would be a good time for new major
> versions, or longer if there's no breaking changes needed. That makes 1-2
> year a perfect time to revisit the topic, while leaving us more time to
> prepare the major release if needed.
>
> Best,
>
> Xintong
>
>
>
> On Thu, Jun 15, 2023 at 10:09 PM Chesnay Schepler <ches...@apache.org>
> wrote:
>
>> On 13/06/2023 17:26, Becket Qin wrote:
>> > It would be valuable if we can avoid releasing minor versions for
>> previous
>> > major versions.
>>
>> On paper, /absolutely /agree, but I'm not sure how viable that is in
>> practice.
>>
>> On the current 2.0 agenda is potentially dropping support for Java 8/11,
>> which may very well be a problem for our current users.
>>
>>
>> On 13/06/2023 17:26, Becket Qin wrote:
>> > Thanks for the feedback and sorry for the confusion about Public API
>> > deprecation. I just noticed that there was a mistake in the NOTES part
>> for
>> > Public API due to a copy-paste error... I just fixed it.
>> I'm very relieved to hear that. Glad to hear that we are on the same
>> page on that note.
>>
>>
>> On 15/06/2023 15:20, Becket Qin wrote:
>> > But it should be
>> > completely OK to bump up the major version if we really want to get rid
>> of
>> > a public API, right?
>>
>> Technically yes, but look at how long it took to get us to 2.0. ;)
>>
>> There's a separate discussion to be had on the cadence of major releases
>> going forward, and there seem to be different opinions on that.
>>
>> If we take the Kafka example of 2 minor releases between major ones,
>> that for us means that users have to potentially deal with breaking
>> changes every 6 months, which seems like a lot.
>>
>> Given our track record I would prefer a regular cycle (1-2 years) to
>> force us to think about this whole topic, and not put it again to the
>> wayside and giving us (and users) a clear expectation on when breaking
>> changes can be made.
>>
>> But again, maybe this should be in a separate thread.
>>
>> On 14/06/2023 11:37, Becket Qin wrote:
>> > Do you have an example of behavioral change in mind? Not sure I fully
>> > understand the concern for behavioral change here.
>>
>> This could be a lot of things. It can be performance in certain
>> edge-cases, a bug fix that users (maybe unknowingly) relied upon
>> (https://xkcd.com/1172/), a semantic change to some API.
>>
>> For a concrete example, consider the job submission. A few releases back
>> we made changes such that the initialization of the job master happens
>> asynchronously.
>> This meant the job submission call returns sooner, and the job state
>> enum was extended to cover this state.
>> API-wise we consider this a compatible change, but the observed behavior
>> may be different.
>>
>> Metrics are another example; I believe over time we changed what some
>> metrics returned a few times.
>>
>

Reply via email to