I'm not sure the main issue in the case of MVs is testing. In this case it seems to be that there are some design issues and/or the design was only works in some overly restrictive use cases. That MVs were committed knowing these were issues seems to be the real problem. So in the case of MVs, sure I don't think they should have ever made it to an experimental stage.
Thinking of how an experimental flag fits in the with the project going forward though, I disagree that we should avoid adding experimental features. On the contrary, I think leaning towards classifying new features as experimental would be better for users. Especially larger features and changes. Even with well spec'd, well tested, and well designed features, there will always be edge cases that you didn't think of, or you'll have made assumptions about the other parts of C* it relies on that aren't 100% correct. Small problems here can often affect correctness, or result in data loss. So, I think it makes sense to avoid marking them as ready for regular use until they've had time to bake in clusters where there are some expert operators that are sophisticated enough to understand the implications of running them, detect issues, and report bugs. Regarding historical examples, in hindsight I think committing 8099, or at the very least, parts of it, behind an experimental flag would have been the right thing to do. It was a huge change that we're still finding issues with 2 years later. On October 1, 2017 at 6:08:50 AM, DuyHai Doan (doanduy...@gmail.com) wrote: How should we transition one feature from the "experimental" state to "production ready" state ? On which criteria ? On Sun, Oct 1, 2017 at 12:12 PM, Marcus Eriksson <krum...@gmail.com> wrote: > I was just thinking that we should try really hard to avoid adding > experimental features - they are experimental due to lack of testing right? > There should be a clear path to making the feature non-experimental (or get > it removed) and having that path discussed on dev@ might give more > visibility to it. > > I'm also struggling a bit to find good historic examples of "this would > have been better off as an experimental feature" - I used to think that it > would have been good to commit DTCS with some sort of experimental flag, > but that would not have made DTCS any better - it would have been better to > do more testing, realise that it does not work and then not commit it at > all of course. > > Does anyone have good examples of features where it would have made sense > to commit them behind an experimental flag? SASI might be a good example, > but for MVs - if we knew how painful they would be, they really would not > have gotten committed at all, right? > > /Marcus > > On Sat, Sep 30, 2017 at 7:42 AM, Jeff Jirsa <jji...@gmail.com> wrote: > > > Reviewers should be able to suggest when experimental is warranted, and > > conversation on dev+jira to justify when it’s transitioned from > > experimental to stable? > > > > We should remove the flag as soon as we’re (collectively) confident in a > > feature’s behavior - at least correctness, if not performance. > > > > > > > On Sep 29, 2017, at 10:31 PM, Marcus Eriksson <krum...@gmail.com> > wrote: > > > > > > +1 on marking MVs experimental, but should there be some point in the > > > future where we consider removing them from the code base unless they > > have > > > gotten significant improvement as well? > > > > > > We probably need to enforce some kind of process for adding new > > > experimental features in the future - perhaps a mail like this one to > > dev@ > > > motivating why it should be experimental? > > > > > > /Marcus > > > > > > On Sat, Sep 30, 2017 at 1:15 AM, Vinay Chella > > <vche...@netflix.com.invalid> > > > wrote: > > > > > >> We tried perf testing MVs internally here but did not see good results > > with > > >> it, hence paused its usage. +1 on tagging certain features which are > not > > >> PROD ready or not stable enough. > > >> > > >> Regards, > > >> Vinay Chella > > >> > > >>> On Fri, Sep 29, 2017 at 7:22 PM, Ben Bromhead <b...@instaclustr.com> > > wrote: > > >>> > > >>> I'm a fan of introducing experimental flags in general as well, +1 > > >>> > > >>> > > >>> > > >>>> On Fri, 29 Sep 2017 at 13:22 Jon Haddad <j...@jonhaddad.com> wrote: > > >>>> > > >>>> I’m very much +1 on this, and to new features in general. > > >>>> > > >>>> I think having a clear line in which we classify something as > > >> production > > >>>> ready would be nice. It would be great if committers were using the > > >>>> feature in prod and could vouch for it’s stability. > > >>>> > > >>>>> On Sep 29, 2017, at 1:09 PM, Blake Eggleston <beggles...@apple.com > > > > >>>> wrote: > > >>>>> > > >>>>> Hi dev@, > > >>>>> > > >>>>> I’d like to propose that we retroactively classify materialized > views > > >>> as > > >>>> an experimental feature, disable them by default, and require users > to > > >>>> enable them through a config setting before using. > > >>>>> > > >>>>> Materialized views have several issues that make them (effectively) > > >>>> unusable in production. Some of the issues aren’t just > implementation > > >>>> problems, but problems with the design that aren’t easily fixed. > It’s > > >>>> unfair of us to make features available to users in this state > without > > >>>> providing a clear warning that bad or unexpected things are likely > to > > >>>> happen if they use it. > > >>>>> > > >>>>> Obviously, this isn’t great news for users that have already > adopted > > >>>> MVs, and I don’t have a great answer for that. I think that’s sort > of > > a > > >>>> sunk cost at this point. If they have any MV related problems, > they’ll > > >>> have > > >>>> them whether they’re marked experimental or not. I would expect this > > to > > >>>> reduce the number of users adopting MVs in the future though, and if > > >> they > > >>>> do, it would be opt-in. > > >>>>> > > >>>>> Once MVs reach a point where they’re usable in production, we can > > >>> remove > > >>>> the flag. Specifics of how the experimental flag would work can be > > >>> hammered > > >>>> out in a forthcoming JIRA, but I’d imagine it would just prevent > users > > >>> from > > >>>> creating new MVs, and maybe log warnings on startup for existing MVs > > if > > >>> the > > >>>> flag isn’t enabled. > > >>>>> > > >>>>> Let me know what you think. > > >>>>> > > >>>>> Thanks, > > >>>>> > > >>>>> Blake > > >>>> > > >>>> > > >>>> ------------------------------------------------------------ > --------- > > >>>> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org > > >>>> For additional commands, e-mail: dev-h...@cassandra.apache.org > > >>>> > > >>>> -- > > >>> Ben Bromhead > > >>> CTO | Instaclustr <https://www.instaclustr.com/> > > >>> +1 650 284 9692 > > >>> Reliability at Scale > > >>> Cassandra, Spark, Elasticsearch on AWS, Azure, GCP and Softlayer > > >>> > > >> > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org > > For additional commands, e-mail: dev-h...@cassandra.apache.org > > > > >
