I suggest we don't go with hadoop annotations as that will add a direct dependency from operators to hadoop even though most of them only need to be dependent on apex. In future if we port apex to run on other systems this will pose a problem. We either implement our own annotations or pick some other independent standard.
On Mon, Dec 14, 2015 at 10:16 PM, Chandni Singh <[email protected]> wrote: > We need to identify the operators and components that are stable if we want > to go with semver check of only Stable classes. > I can create an initial list. > > Thanks, > Chandni > > On Mon, Dec 14, 2015 at 9:24 PM, Isha Arkatkar <[email protected]> > wrote: > > > Yep, That's what I am doing now :) > > > > Thanks, > > Isha > > > > On Mon, Dec 14, 2015 at 9:22 PM, Chandni Singh <[email protected]> > > wrote: > > > > > Isha, > > > > > > I think for now you can configure the japicmp plugin to exclude the > > package > > > as follows in the pom.xml. > > > > > > <excludes> > > > > > > <exclude>com.datatorrent.lib.parser.*</exclude> > > > > > > </excludes> > > > > > > This is an example where we can benefit from inclusion approach with > > > japicmp 0.7 version. > > > > > > > > > Thanks, > > > > > > Chandni > > > > > > > > > On Mon, Dec 14, 2015 at 8:34 PM, Isha Arkatkar <[email protected]> > > > wrote: > > > > > > > +1 > > > > When 0.7 version of japicmp is available, we can add exclusions for > > > > @Evolving or inclusions for @Stable, whichever way is finalized. > > > > > > > > But before that should we add package exclusions individually if all > > the > > > > operators inside the package are marked Evolving? > > > > I wanted to make changes to some of the parser operators in Malhar. > But > > > > changing those breaks sem version check. > > > > > > > > Please suggest. > > > > > > > > Thanks, > > > > Isha > > > > > > > > On Sun, Dec 13, 2015 at 10:46 PM, Tushar Gosavi < > > [email protected]> > > > > wrote: > > > > > > > > > +1 > > > > > The new operators added will most likely to undergo change > frequently > > > > until > > > > > they become stable. > > > > > > > > > > - Tushar. > > > > > > > > > > > > > > > On Mon, Dec 14, 2015 at 12:05 PM, Siyuan Hua < > [email protected] > > > > > > > > wrote: > > > > > > > > > > > +1 > > > > > > > > > > > > On Sun, Dec 13, 2015 at 10:01 PM, Yogi Devendra < > > > > [email protected] > > > > > > > > > > > > wrote: > > > > > > > > > > > > > I am not against the Idea of using @Stable instead of marking > > every > > > > new > > > > > > > @Evolving. I agree that, it would convenient to have @Stable. > > > > > > > > > > > > > > Just raised the point which needs further discussion, so that > we > > > get > > > > > > > suggestions from the mentors and community. > > > > > > > > > > > > > > ~ Yogi > > > > > > > > > > > > > > On 14 December 2015 at 11:23, Chandni Singh < > > > [email protected] > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > Can we define some criteria for deciding when to consider > > > operator > > > > as > > > > > > > > @Stable? > > > > > > > > Yes, we should follow Hadoop's example and formalize some > > > criteria. > > > > > > > > > > > > > > > > [It would be difficult for an open source project to track > > which > > > > user > > > > > > is > > > > > > > > using which operators. So, above strategy may not work. ] > > > > > > > > Hadoop is an open source project which actually created these > > > > > > annotations > > > > > > > > and it's widely used. I think any new development takes time > to > > > > > become > > > > > > > > stable. > > > > > > > > If the operators are NOT marked as @Stable, users will know > > that > > > > when > > > > > > > they > > > > > > > > upgrade backward compatibility may be broken. > > > > > > > > I think it has the same affect of marking every new > > > operator/class > > > > as > > > > > > > > @Evolving. The benefit of checking semantic versioning of > > > "Stable" > > > > > > > > operators is that they are currently fewer in number and IMO > > easy > > > > to > > > > > > > manage > > > > > > > > and new development will be implicitly "Evolving". > > > > > > > > > > > > > > > > Chandni > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Sun, Dec 13, 2015 at 9:11 PM, Yogi Devendra < > > > > > > [email protected]> > > > > > > > > wrote: > > > > > > > > > > > > > > > > > When to mark certain operator as @Stable is not clearly > > > defined. > > > > > > > > > > > > > > > > > > Can we define some criteria for deciding when to consider > > > > operator > > > > > as > > > > > > > > > @Stable? > > > > > > > > > > > > > > > > > > For example one criteria could be, if operator is running > for > > > >1 > > > > > year > > > > > > > in > > > > > > > > > production environment for some user. Can we come with some > > > > > strategy > > > > > > > like > > > > > > > > > this? > > > > > > > > > [It would be difficult for an open source project to track > > > which > > > > > user > > > > > > > is > > > > > > > > > using which operators. So, above strategy may not work. ] > > > > > > > > > > > > > > > > > > ~ Yogi > > > > > > > > > > > > > > > > > > On 14 December 2015 at 05:42, Timothy Farkas < > > > > [email protected]> > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > +1 > > > > > > > > > > On Dec 13, 2015 4:08 PM, "Chandni Singh" < > > > > > [email protected]> > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > Hi, > > > > > > > > > > > > > > > > > > > > > > In Malhar there are relatively smaller number of > > > operators > > > > > that > > > > > > > we > > > > > > > > > use > > > > > > > > > > in > > > > > > > > > > > our demo applications, customer applications, POCs etc > > that > > > > are > > > > > > > > mature. > > > > > > > > > > > > > > > > > > > > > > The library is cluttered with operators especially in > > > > lib/util, > > > > > > > > > lib/algo, > > > > > > > > > > > lib/math packages which can be cleaned up by either > > > removing > > > > > them > > > > > > > or > > > > > > > > > > > improving them but that breaks semantic versioning. > > > > > > > > > > > > > > > > > > > > > > When we add new operators/utilities it takes certain > time > > > for > > > > > > them > > > > > > > to > > > > > > > > > > > mature. Japicmp doesn't help because it doesn't honor > > > > @Evolving > > > > > > > > > @Unstable > > > > > > > > > > > annotations for now. > > > > > > > > > > > > > > > > > > > > > > I wanted to propose that we add an annotation, let's > say, > > > > > re-use > > > > > > > > > hadoop's > > > > > > > > > > > @Stable and mark the operators which are stable with it > > and > > > > > > perform > > > > > > > > > > semver > > > > > > > > > > > check on just these operators. > > > > > > > > > > > > > > > > > > > > > > The 0.7.0 version of japi cmp has the support for > > > inclusions > > > > > (as > > > > > > > well > > > > > > > > > as > > > > > > > > > > > exclusions) based on annotations. > > > > > > > > > > > > > > > > > > > > > > Here is the info: > > > > > > > > > > > https://github.com/siom79/japicmp/issues/88 > > > > > > > > > > > > > > > > > > > > > > The reason I am inclined to the inclusion approach is > > that > > > > > there > > > > > > > are > > > > > > > > > > > relatively smaller number of operators which IMO are > > > stable. > > > > A > > > > > > lot > > > > > > > of > > > > > > > > > > them > > > > > > > > > > > aren't. > > > > > > > > > > > So instead of going and marking so many as Evolving, we > > > will > > > > > mark > > > > > > > > > > > relatively few of them as stable. > > > > > > > > > > > > > > > > > > > > > > Also new development can be facilitated by this. We > > > wouldn't > > > > > have > > > > > > > to > > > > > > > > > add > > > > > > > > > > > @Evolving to everything which is new. Instead we will > > mark > > > it > > > > > > > @Stable > > > > > > > > > > when > > > > > > > > > > > it is. > > > > > > > > > > > > > > > > > > > > > > Please let me know what you think? > > > > > > > > > > > > > > > > > > > > > > Thanks, > > > > > > > > > > > Chandni > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >
