Re: NEW: API Stability annotations!
Got it! I was under the impression that deprecated means "there is something better now and we'll remove this in few releases". But yeah, lets avoid marking it for now. On Thu, Aug 13, 2015 at 4:58 PM, Neha Narkhede wrote: > Thanks for starting the discussion around API annotations, Gwen! I think it > is better to add the deprecated annotation after the new consumer API is > successfully deployed at a couple places. > > On Thu, Aug 13, 2015 at 3:42 PM, Ismael Juma wrote: > > > On Thu, Aug 13, 2015 at 10:41 PM, Gwen Shapira > wrote: > > > > > IMO, both old producer and old consumer APIs should be marked as > > deprecated > > > for 0.8.3 (since the new code will be in and we want to encourage the > > > switch). > > > > > > > I can see the appeal of this, but it's also worth considering the > downsides > > for users too: > > > > * It will introduce a number of deprecation warnings to everyone that > > upgrades to 0.8.3 even though the old Consumer APIs still work fine (also > > worth keeping in mind that in many projects, warnings cause a build > > failure) > > * The new Consumer is still marked as `Unstable` so it seems a bit odd to > > deprecate the old one > > > > I think a more conservative option would be to update the documentation > to > > encourage users to move to the new consumer without adding deprecation > > annotations to the old consumer APIs. Some features that are only > available > > in the new consumer (e.g. SSL support) provide further incentive to move. > > As Ewen suggested, the old consumer would then be deprecated in the > > following release. And removed in the one after that. The main downside > > would be having to maintain the old consumer for a little while longer. > > > > Something to think about. :) > > > > Best, > > Ismael > > > > > > -- > Thanks, > Neha >
Re: NEW: API Stability annotations!
Thanks for starting the discussion around API annotations, Gwen! I think it is better to add the deprecated annotation after the new consumer API is successfully deployed at a couple places. On Thu, Aug 13, 2015 at 3:42 PM, Ismael Juma wrote: > On Thu, Aug 13, 2015 at 10:41 PM, Gwen Shapira wrote: > > > IMO, both old producer and old consumer APIs should be marked as > deprecated > > for 0.8.3 (since the new code will be in and we want to encourage the > > switch). > > > > I can see the appeal of this, but it's also worth considering the downsides > for users too: > > * It will introduce a number of deprecation warnings to everyone that > upgrades to 0.8.3 even though the old Consumer APIs still work fine (also > worth keeping in mind that in many projects, warnings cause a build > failure) > * The new Consumer is still marked as `Unstable` so it seems a bit odd to > deprecate the old one > > I think a more conservative option would be to update the documentation to > encourage users to move to the new consumer without adding deprecation > annotations to the old consumer APIs. Some features that are only available > in the new consumer (e.g. SSL support) provide further incentive to move. > As Ewen suggested, the old consumer would then be deprecated in the > following release. And removed in the one after that. The main downside > would be having to maintain the old consumer for a little while longer. > > Something to think about. :) > > Best, > Ismael > -- Thanks, Neha
Re: NEW: API Stability annotations!
On Thu, Aug 13, 2015 at 10:41 PM, Gwen Shapira wrote: > IMO, both old producer and old consumer APIs should be marked as deprecated > for 0.8.3 (since the new code will be in and we want to encourage the > switch). > I can see the appeal of this, but it's also worth considering the downsides for users too: * It will introduce a number of deprecation warnings to everyone that upgrades to 0.8.3 even though the old Consumer APIs still work fine (also worth keeping in mind that in many projects, warnings cause a build failure) * The new Consumer is still marked as `Unstable` so it seems a bit odd to deprecate the old one I think a more conservative option would be to update the documentation to encourage users to move to the new consumer without adding deprecation annotations to the old consumer APIs. Some features that are only available in the new consumer (e.g. SSL support) provide further incentive to move. As Ewen suggested, the old consumer would then be deprecated in the following release. And removed in the one after that. The main downside would be having to maintain the old consumer for a little while longer. Something to think about. :) Best, Ismael
Re: NEW: API Stability annotations!
IMO, both old producer and old consumer APIs should be marked as deprecated for 0.8.3 (since the new code will be in and we want to encourage the switch). I'd like to see 0.9 released with just the new java clients. On Thu, Aug 13, 2015 at 6:47 AM, Ismael Juma wrote: > On Thu, Aug 13, 2015 at 8:36 AM, Ewen Cheslack-Postava > wrote: > > > On deprecation, I think we should definitely use the standard annotation > to > > handle this. > > > Thanks Ewen. I agree that we should use @Deprecated for language-level > deprecations. My question was regarding the following: > > - unstable - can change at any time > > - evolving - can break compatibility at minor releases (i.e. 0.9 may be > > incompatible with 0.8) > > - stable - will only break compatibility at major releases (1.0, 2.0, > etc) > > > > > Let's pick `evolving`, as an example. Does it mean that we can break > compatibility at minor releases without a deprecation cycle? Or does it > mean that we would need to deprecate it for one minor release and then > remove in the next? The latter is nicer for users as they have one release > where both the deprecated and recommended still work, but it adds more > overhead to Kafka development. What is the current thinking around this? > > The annotation doesn't necessarily work for everything (e.g., > > deprecating older versions of the protocol doesn't work via Java > > deprecation annotations), but we should definitely use it for deprecated > > Java/Scala APIs. Perhaps we should start by marking all the old producer > > APIs deprecated > > > Seems sensible assuming that users have found it easy to migrate to the new > producer in the latest release. > > and, after the next release, the old consumer APIs too? :) > > > Similar to above, it would be good to do that once we have some reports > from users who moved from one API to the next. > > Best, > Ismael >
Re: NEW: API Stability annotations!
On Thu, Aug 13, 2015 at 8:36 AM, Ewen Cheslack-Postava wrote: > On deprecation, I think we should definitely use the standard annotation to > handle this. Thanks Ewen. I agree that we should use @Deprecated for language-level deprecations. My question was regarding the following: - unstable - can change at any time > - evolving - can break compatibility at minor releases (i.e. 0.9 may be > incompatible with 0.8) > - stable - will only break compatibility at major releases (1.0, 2.0, etc) > Let's pick `evolving`, as an example. Does it mean that we can break compatibility at minor releases without a deprecation cycle? Or does it mean that we would need to deprecate it for one minor release and then remove in the next? The latter is nicer for users as they have one release where both the deprecated and recommended still work, but it adds more overhead to Kafka development. What is the current thinking around this? The annotation doesn't necessarily work for everything (e.g., > deprecating older versions of the protocol doesn't work via Java > deprecation annotations), but we should definitely use it for deprecated > Java/Scala APIs. Perhaps we should start by marking all the old producer > APIs deprecated Seems sensible assuming that users have found it easy to migrate to the new producer in the latest release. and, after the next release, the old consumer APIs too? :) Similar to above, it would be good to do that once we have some reports from users who moved from one API to the next. Best, Ismael
Re: NEW: API Stability annotations!
We could potentially add more annotations to cover things like public/private APIs, deprecation, etc. I lifted this initial implementation from Hadoop (thanks for the pointer Gwen!). Hadoop also has a public/private annotation that we could lift, and they use this to filter javadocs. These are useful signals, but we also don't want to make adding new code too heavyweight, so I'd prefer keeping these annotations minimal. The immediate benefit of adding stable/unstable annotations was to make it clear a) that people should stop trying to use our in-progress APIs and b) so it's clear when we're committing in-progress code that may not be as good quality as we would normally require but is acceptable for new, in-development code that we plan to quickly iterate and improve upon before release. In the latter case, it's a useful signal internally for Kafka development that an API isn't yet fully baked even for other Kafka devs. On deprecation, I think we should definitely use the standard annotation to handle this. The annotation doesn't necessarily work for everything (e.g., deprecating older versions of the protocol doesn't work via Java deprecation annotations), but we should definitely use it for deprecated Java/Scala APIs. Perhaps we should start by marking all the old producer APIs deprecated and, after the next release, the old consumer APIs too? :) -Ewen On Wed, Aug 12, 2015 at 3:42 PM, Ismael Juma wrote: > Hi Gwen, > > Nice to see this. There is no deprecation cycle for breakages? > > Ismael > > On Wed, Aug 12, 2015 at 11:05 PM, Gwen Shapira wrote: > > > Hi Team Kafka, > > > > Ewen just added stability annotations to Apache Kafka (KAFKA-2429). > > > > In the same PR, we marked the new Consumer API as "unstable" since we are > > still actively iterating on them. The goal is to mark them as "evolving" > > before the next release and after one release to validate them, we will > > mark them as "stable". > > > > When adding new public APIs, we encourage you to think of their stage of > > development and annotate correctly. > > > > The usage is: > > > > - unstable - can change at any time > > - evolving - can break compatibility at minor releases (i.e. 0.9 may be > > incompatible with 0.8) > > - stable - will only break compatibility at major releases (1.0, 2.0, > etc) > > > > Enjoy! > > > > Gwen > > > -- Thanks, Ewen
Re: NEW: API Stability annotations!
Very cool! Congrats! :) My only comment is that, as Gwen pointed out in the first email, it is a set of API compatibility annotations, but the name is InterfaceStability. Wouldn't ApiStability be a better name? InterfaceStability looks a bit restrictive, imho. https://github.com/apache/kafka/pull/133/files Cheers, Edward Good annotations. I can see a few future usages :) Jiangjie (Becket) Qin On Wed, Aug 12, 2015 at 3:05 PM, Gwen Shapira wrote: > Hi Team Kafka, > > Ewen just added stability annotations to Apache Kafka (KAFKA-2429). > > In the same PR, we marked the new Consumer API as "unstable" since we are > still actively iterating on them. The goal is to mark them as "evolving" > before the next release and after one release to validate them, we will > mark them as "stable". > > When adding new public APIs, we encourage you to think of their stage of > development and annotate correctly. > > The usage is: > > - unstable - can change at any time > - evolving - can break compatibility at minor releases (i.e. 0.9 may be > incompatible with 0.8) > - stable - will only break compatibility at major releases (1.0, 2.0, etc) > > Enjoy! > > Gwen >
Re: NEW: API Stability annotations!
Hi Gwen, Nice to see this. There is no deprecation cycle for breakages? Ismael On Wed, Aug 12, 2015 at 11:05 PM, Gwen Shapira wrote: > Hi Team Kafka, > > Ewen just added stability annotations to Apache Kafka (KAFKA-2429). > > In the same PR, we marked the new Consumer API as "unstable" since we are > still actively iterating on them. The goal is to mark them as "evolving" > before the next release and after one release to validate them, we will > mark them as "stable". > > When adding new public APIs, we encourage you to think of their stage of > development and annotate correctly. > > The usage is: > > - unstable - can change at any time > - evolving - can break compatibility at minor releases (i.e. 0.9 may be > incompatible with 0.8) > - stable - will only break compatibility at major releases (1.0, 2.0, etc) > > Enjoy! > > Gwen >
Re: NEW: API Stability annotations!
Good annotations. I can see a few future usages :) Jiangjie (Becket) Qin On Wed, Aug 12, 2015 at 3:05 PM, Gwen Shapira wrote: > Hi Team Kafka, > > Ewen just added stability annotations to Apache Kafka (KAFKA-2429). > > In the same PR, we marked the new Consumer API as "unstable" since we are > still actively iterating on them. The goal is to mark them as "evolving" > before the next release and after one release to validate them, we will > mark them as "stable". > > When adding new public APIs, we encourage you to think of their stage of > development and annotate correctly. > > The usage is: > > - unstable - can change at any time > - evolving - can break compatibility at minor releases (i.e. 0.9 may be > incompatible with 0.8) > - stable - will only break compatibility at major releases (1.0, 2.0, etc) > > Enjoy! > > Gwen >
Re: NEW: API Stability annotations!
Yeah, I agree that your description matches reality better :)
By "document" do you have any suggestions where? The interface itself has
documentation. Were you thinking Wiki?
On Wed, Aug 12, 2015 at 3:48 PM, Grant Henke wrote:
> This is a great idea!
>
> I do question the classifications a bit. @stable seams unreasonable as
> Kafka has not has a "major" release yet. I have typically thought of Kafka
> releases as 0.{major}.{minor}.{maintenance}. For instance 0.8.3 coming up
> has a significant amount of change. Though Kafka's release cycle could be
> changing and I may not understand the cycle/classification correctly.
>
> I am thinking something along the lines of:
> - unstable - can change at any time (i.e maintenance releases)
> - evolving - can break compatibility at minor releases (i.e. 0.8.3 may
> be incompatible
> with 0.8.2)
> - stable - will only break compatibility at major releases (0.8, 0.9, etc)
>
> Regardless the choice of the "contract", I do suggest we document this.
>
>
> On Wed, Aug 12, 2015 at 5:05 PM, Gwen Shapira wrote:
>
> > Hi Team Kafka,
> >
> > Ewen just added stability annotations to Apache Kafka (KAFKA-2429).
> >
> > In the same PR, we marked the new Consumer API as "unstable" since we are
> > still actively iterating on them. The goal is to mark them as "evolving"
> > before the next release and after one release to validate them, we will
> > mark them as "stable".
> >
> > When adding new public APIs, we encourage you to think of their stage of
> > development and annotate correctly.
> >
> > The usage is:
> >
> > - unstable - can change at any time
> > - evolving - can break compatibility at minor releases (i.e. 0.9 may be
> > incompatible with 0.8)
> > - stable - will only break compatibility at major releases (1.0, 2.0,
> etc)
> >
> > Enjoy!
> >
> > Gwen
> >
>
>
>
> --
> Grant Henke
> Software Engineer | Cloudera
> gr...@cloudera.com | twitter.com/gchenke | linkedin.com/in/granthenke
>
Re: NEW: API Stability annotations!
This is a great idea!
I do question the classifications a bit. @stable seams unreasonable as
Kafka has not has a "major" release yet. I have typically thought of Kafka
releases as 0.{major}.{minor}.{maintenance}. For instance 0.8.3 coming up
has a significant amount of change. Though Kafka's release cycle could be
changing and I may not understand the cycle/classification correctly.
I am thinking something along the lines of:
- unstable - can change at any time (i.e maintenance releases)
- evolving - can break compatibility at minor releases (i.e. 0.8.3 may
be incompatible
with 0.8.2)
- stable - will only break compatibility at major releases (0.8, 0.9, etc)
Regardless the choice of the "contract", I do suggest we document this.
On Wed, Aug 12, 2015 at 5:05 PM, Gwen Shapira wrote:
> Hi Team Kafka,
>
> Ewen just added stability annotations to Apache Kafka (KAFKA-2429).
>
> In the same PR, we marked the new Consumer API as "unstable" since we are
> still actively iterating on them. The goal is to mark them as "evolving"
> before the next release and after one release to validate them, we will
> mark them as "stable".
>
> When adding new public APIs, we encourage you to think of their stage of
> development and annotate correctly.
>
> The usage is:
>
> - unstable - can change at any time
> - evolving - can break compatibility at minor releases (i.e. 0.9 may be
> incompatible with 0.8)
> - stable - will only break compatibility at major releases (1.0, 2.0, etc)
>
> Enjoy!
>
> Gwen
>
--
Grant Henke
Software Engineer | Cloudera
gr...@cloudera.com | twitter.com/gchenke | linkedin.com/in/granthenke
