I agree with Reuven that our experimental annotation is not useful any more. For example Datastore IO in python sdk is experimental for 2 years now. Even though it is marked as experimental an upgrade is carefully planned [1] as if it is not experimental. Given that I do not think we can remove features within a small number of minor releases. (Exception to this would be, if we have a clear knowledge of very low usage of a certain IO.)
I am worried that tagging experimental features with release versions will add toil to the release process as mentioned and will also add to the user confusion. What would be the signal to a user if they see an experimental feature target release bumped between releases? How about tagging experimental features with JIRAs (similar to TODOs) with an action to either promote them as supported features or remove them? These JIRAs could have fix version targets as any other release blocking JIRAs. It will also clarify who is responsible for a given experimental feature. [1] https://lists.apache.org/thread.html/5ec88967aa4a382db07a60e0101c4eb36165909076867155ab3546a6@%3Cdev.beam.apache.org%3E On Wed, Apr 3, 2019 at 5:24 PM Reuven Lax <re...@google.com> wrote: > Experiments are already tagged with a Kind enum > (e.g. @Experimental(Kind.Schemas)). > This not the case for python's annotations. It will be a good idea to add there as well. > > On Wed, Apr 3, 2019 at 4:56 PM Ankur Goenka <goe...@google.com> wrote: > >> I think a release version with Experimental flag makes sense. >> In addition, I think many of our user start to rely on experimental >> features because they are not even aware that these features are >> experimental and its really hard to find the experimental features used >> without giving a good look at the Beam code and having some knowledge about >> it. >> >> It will be good it we can have a step at the pipeline submission time >> which can print all the experiments used in verbose mode. This might also >> require to add a meaningful group name for the experiment example >> >> @Experimental("SDF", 2.15.0) >> >> This will of-course add additional effort and require additional context >> while tagging experiments. >> >> On Wed, Apr 3, 2019 at 4:43 PM Reuven Lax <re...@google.com> wrote: >> >>> Our Experimental annotation has become almost useless. Many core, >>> widely-used parts of the API (e.g. triggers) are still all marked as >>> experimental. So many users use these features that we couldn't really >>> change them (in a backwards-incompatible) without hurting many users, so >>> the fact they are marked Experimental has become a fiction. >>> >>> Could we add a deadline to the Experimental tag - a release version when >>> it will be removed? e.g. >>> >>> @Experimental(2.15.0) >>> >>> We can have a test that ensure that the tag is removed at this version. >>> Of course if we're not ready to remove experimental by that version, it's >>> fine - we can always bump the tagged version. However this forces us to >>> think about each one. >>> >>> Downside - it might add more toil to the existing release process. >>> >>> Reuven >>> >>> >>> On Wed, Apr 3, 2019 at 4:00 PM Kyle Weaver <kcwea...@google.com> wrote: >>> >>>> > We might also want to get in the habit of reviewing if something >>>> should no longer be experimental. >>>> >>>> +1 >>>> >>>> Kyle Weaver | Software Engineer | kcwea...@google.com | +16502035555 >>>> >>>> >>>> On Wed, Apr 3, 2019 at 3:53 PM Kenneth Knowles <k...@apache.org> wrote: >>>> >>>>> I think option 2 with n=1 minor version seems OK. So users get the >>>>> message for one release and it is gone the next. We should make sure the >>>>> deprecation warning says "this is an experimental feature, so it will be >>>>> removed after 1 minor version". And we need a process for doing it so it >>>>> doesn't sit around. I think we should also leave room for using our own >>>>> judgment about whether the user pain is very little and then it is not >>>>> needed to have a deprecation cycle. >>>>> >>>>> We might also want to get in the habit of reviewing if something >>>>> should no longer be experimental. >>>>> >>>>> Kenn >>>>> >>>>> On Wed, Apr 3, 2019 at 2:33 PM Ismaël Mejía <ieme...@gmail.com> wrote: >>>>> >>>>>> When we did the first stable release of Beam (2.0.0) we decided to >>>>>> annotate most of the Beam IOs as @Experimental because we were >>>>>> cautious about not getting the APIs right in the first try. This was a >>>>>> really good decision because we could do serious improvements and >>>>>> refactorings to them in the first releases without the hassle of >>>>>> keeping backwards compatibility. However after some more releases >>>>>> users started to rely on features and supported versions, so we ended >>>>>> up in a situation where we could not change them arbitrarily without >>>>>> consequences to the final users. >>>>>> >>>>>> So we started to deprecate some features and parts of the API without >>>>>> removing them, e.g. the introduction of HadoopFormatIO deprecated >>>>>> HadoopInputFormatIO, we deprecated methods of MongoDbIO and MqttIO to >>>>>> improve the APIs (in most cases with valid/improved replacements), and >>>>>> recently it was discussed to removal of support for older versions in >>>>>> KafkaIO. >>>>>> >>>>>> Keeping deprecated stuff in experimental APIs does not seem to make >>>>>> sense, but it is what he have started to do to be ‘user friendly’, but >>>>>> it is probably a good moment to define, what should be the clear path >>>>>> for removal and breaking changes of experimental features, some >>>>>> options: >>>>>> >>>>>> 1. Stay as we were, do not mark things as deprecated and remove them >>>>>> at will because this is the contract of @Experimental. >>>>>> 2. Deprecate stuff and remove it after n versions (where n could be 3 >>>>>> releases). >>>>>> 3. Deprecate stuff and remove it just after a new LTS is decided to >>>>>> ensure users who need these features may still have them for some >>>>>> time. >>>>>> >>>>>> I would like to know your opinions about this, or if you have other >>>>>> ideas. Notice that in discussion I refer only to @Experimental >>>>>> features. >>>>>> >>>>>
