This following pull request should be taken up in 4.0.0. See my comments in https://github.com/apache/apex-malhar/pull/664 <https://www.google.com/url?q=https%3A%2F%2Fgithub.com%2Fapache%2Fapex-malhar%2Fpull%2F664&sa=D&ust=1503020269444000&usg=AFQjCNHDPZIg2e6I33Jb1XjB5Ir1FkdURQ> https://github.com/apache/apex-malhar/pull/662 <https://www.google.com/url?q=https%3A%2F%2Fgithub.com%2Fapache%2Fapex-malhar%2Fpull%2F662&sa=D&ust=1503020269444000&usg=AFQjCNF7Xde7X55M3qmc8z-D5y3ZwNg7Fg>
This merge should not be done without a consensus. This will require code changes to existing apps. Thks, Amol E:a...@datatorrent.com | M: 510-449-2606 | Twitter: @*amolhkekre* www.datatorrent.com On Mon, Aug 14, 2017 at 7:48 PM, Thomas Weise <t...@apache.org> wrote: > Hi, > > I opened following PRs for the package change: > > https://github.com/apache/apex-malhar/pull/662 > > Moves all classes with history retained (hence 2 commits). Also contains > checkstyle and other mechanical changes. > > https://github.com/apache/apex-malhar/pull/664 > > Adds backward compatibility jar. > > Once above PRs are merged the new artifact can be deployed and introduced > as dependency in malhar-library. > > Please review. > > Thanks, > Thomas > > > > On Sun, Jul 16, 2017 at 7:04 AM, Thomas Weise <t...@apache.org> wrote: > > > My original list of work items contained the b/w compatibility aspect, I > > don't think there should be any confusion of whether it will be covered > > here or not. > > > > The proposed shading will provide the old classes and they will be frozen > > as of release 3.7. That's the same as making a copy of the code and never > > again making changes to the original classes. This cannot be accomplished > > by using the older 3.7 release in your project because you cannot use 2 > > different versions of Malhar in parallel unless you apply shading. > > > > The shaded artifact will only expose the com.datatorrent classes, and > they > > will be self-contained as the rest of the classes that they may depend on > > are shaded. The shaded artifact does not evolve, there are not more > changes > > to com.datatorrent classes after they are relocated in master. > > > > Thanks, > > Thomas > > > > > > On Sun, Jul 16, 2017 at 2:00 AM, Pramod Immaneni <pra...@datatorrent.com > > > > wrote: > > > >> I don't think we can limit the topic strictly to relocation without > having > >> a good b/w compatibility story or at least one that goes far enough. > >> > >> The shading idea sounds interesting. Why not let the shaded version move > >> forward with each release till we hit a major release. If it is going to > >> remain pegged at 3.7.0, why shade in the first place as the regular > 3.7.0 > >> release would do the same job and it would be same as the loss of b/w > >> compatibility with newer releases. > >> > >> Thanks > >> > >> On Sat, Jul 15, 2017 at 7:57 AM, Thomas Weise <t...@apache.org> wrote: > >> > >> > Discussing what in the future might become stable needs to be a > separate > >> > thread, it will be a much bigger discussion. > >> > > >> > The topic here is to relocate the packages. With a few exceptions > >> > relocation won't affect the semantic versioning. Semantic versioning > is > >> > essentially not effective for Malhar because almost everything is > >> @Evolving > >> > (and there are reasons for that.. -> separate topic) > >> > > >> > I don't really like the idea of creating bw compatibility stubs for > the > >> > follow up PR. It creates even more clutter in the source tree than > there > >> > already is and so here is an alternative suggestion: > >> > > >> > https://github.com/tweise/apex-malhar/blob/malhar37- > >> > compat/shaded-malhar37/pom.xml > >> > > >> > Create a shaded artifact that provides the old com.datatorrent.* > >> classes as > >> > of release 3.7. Users can include that artifact if they don't want to > >> > change import statements. At the same time they have an incentive to > >> switch > >> > to the relocated classes to take advantage of bug fixes and new > >> > functionality. > >> > > >> > I will work on the first PR that does the relocate. In the meantime, > we > >> can > >> > finalize what backward compatibility support we want to provide and > how. > >> > > >> > Thanks, > >> > Thomas > >> > > >> > > >> > > >> > > >> > On Fri, Jul 14, 2017 at 11:33 AM, Pramod Immaneni < > >> pra...@datatorrent.com> > >> > wrote: > >> > > >> > > How about coming up with a list of @Evolving operators that we would > >> like > >> > > to see in the eventual stable list and move those along with the not > >> > > @Evolving ones in org.apache.apex with b/w stubs and leave the rest > as > >> > they > >> > > are. Then have a follow up JIRA for the rest to be moved over to > >> contrib > >> > > and be deprecated. > >> > > > >> > > Thanks > >> > > > >> > > On Fri, Jul 14, 2017 at 10:37 AM, Thomas Weise < > >> thomas.we...@gmail.com> > >> > > wrote: > >> > > > >> > > > We need to keep the discussion here on topic, if other things are > >> piled > >> > > on > >> > > > top then nothing gets done. > >> > > > > >> > > > Most operators today are not stable, they are @Evolving. So > perhaps > >> it > >> > is > >> > > > necessary to have a separate discussion about that, outside of > this > >> > > thread. > >> > > > My guess is that there are only a few operators that we could > >> declare > >> > > > stable. Specifically, under contrib the closest one might have > been > >> > > Kafka, > >> > > > but that is already superseded by the newer versions. > >> > > > > >> > > > Thomas > >> > > > > >> > > > > >> > > > On Fri, Jul 14, 2017 at 10:21 AM, Pramod Immaneni < > >> > > pra...@datatorrent.com> > >> > > > wrote: > >> > > > > >> > > > > We had a discussion a while back, agreed to relegate non-stable > >> and > >> > > > > experimental operators to contrib and also added this to > >> contribution > >> > > > > guidelines. We aexecuted on this and cleaned up the repo by > moving > >> > > > > operators deemed non-suitable for production use at that time to > >> > > contrib. > >> > > > > So I wouldn't say the operators that are at the top level today > or > >> > the > >> > > > ones > >> > > > > in library are 0.x.x quality. Granted, we may need to do one > more > >> > pass > >> > > as > >> > > > > some of the operators may no longer be considered the right > >> > > > implementations > >> > > > > with the advent of the windowed operator. > >> > > > > > >> > > > > Since contrib used to be the place that housed operators that > >> > required > >> > > > > third party libraries, there are some production quality > >> operators in > >> > > > there > >> > > > > that need to be pulled out into top level modules. > >> > > > > > >> > > > > I think we are in agreement that for operators that we consider > >> > stable, > >> > > > we > >> > > > > should provide a b/w stub. I would add that we strongly consider > >> > > creating > >> > > > > the org.apache.apex counterparts of any stable operators that > are > >> in > >> > > > > contrib out in top level modules and have the com.datatorrent > >> stubs > >> > in > >> > > > > contrib. > >> > > > > > >> > > > > For the operators not considered stable, I would prefer we > either > >> > leave > >> > > > the > >> > > > > current package path as is or if we change the package path then > >> > create > >> > > > the > >> > > > > b/w stub as I am not sure how widely they are in use (so, in > >> essence, > >> > > > > preserve semantic versioning). It would be good if there is a > >> > followup > >> > > > JIRA > >> > > > > that takes a look at what other operators can be moved to > contrib > >> > with > >> > > > the > >> > > > > advent of the newer frameworks and understanding. > >> > > > > > >> > > > > Thanks > >> > > > > > >> > > > > On Fri, Jul 14, 2017 at 9:44 AM, Thomas Weise <t...@apache.org> > >> > wrote: > >> > > > > > >> > > > > > Most of the operators evolve, as is quite clearly indicated by > >> > > > @Evolving > >> > > > > > annotations. So while perhaps 0.x.x would be a more > appropriate > >> > > version > >> > > > > > number, I don't think you can change that. > >> > > > > > > >> > > > > > Thomas > >> > > > > > > >> > > > > > On Fri, Jul 14, 2017 at 9:35 AM, Vlad Rozov < > >> > v.ro...@datatorrent.com > >> > > > > >> > > > > > wrote: > >> > > > > > > >> > > > > > > If entire library is not stable, its version should be 0.x.x > >> and > >> > > > there > >> > > > > > > should not be any semantic versioning enabled or implied. It > >> > > evolves. > >> > > > > If > >> > > > > > > some operators are stable as 3.8.x version suggests, the > >> library > >> > > > should > >> > > > > > > follow semantic versioning and it is not OK to make a major > >> > change > >> > > > that > >> > > > > > > breaks semantic versioning across the entire library. It is > >> not a > >> > > > > finding > >> > > > > > > an excuse not to make a change. For me semantic versioning > and > >> > > > backward > >> > > > > > > compatibility is more important than a proper package name. > >> > > > > > > > >> > > > > > > Thank you, > >> > > > > > > > >> > > > > > > Vlad > >> > > > > > > > >> > > > > > > > >> > > > > > > On 7/14/17 09:11, Thomas Weise wrote: > >> > > > > > > > >> > > > > > >> Semantic versioning makes sense when you have a stable > >> baseline. > >> > > As > >> > > > > long > >> > > > > > >> as > >> > > > > > >> frequent fixes need to be made to address basic issues, it > >> makes > >> > > no > >> > > > > > sense > >> > > > > > >> to declare operators stable, which is why they are marked > >> > > evolving. > >> > > > > > >> > >> > > > > > >> Instead of finding reasons to avoid changes that should > have > >> > been > >> > > > > made a > >> > > > > > >> long time ago, why not discuss what a "stable operator" is > >> and > >> > > which > >> > > > > > ones > >> > > > > > >> deserve to be in that category. Those are the ones that > will > >> get > >> > > the > >> > > > > > >> backward compatibility stub. > >> > > > > > >> > >> > > > > > >> Thanks, > >> > > > > > >> Thomas > >> > > > > > >> > >> > > > > > >> > >> > > > > > >> On Fri, Jul 14, 2017 at 8:34 AM, Vlad Rozov < > >> > > > v.ro...@datatorrent.com> > >> > > > > > >> wrote: > >> > > > > > >> > >> > > > > > >> My preference is to agree that the next Malhar release is > >> 4.0.0 > >> > > and > >> > > > > make > >> > > > > > >>> the proposed changes when 3.8.0 goes out. Otherwise why to > >> keep > >> > > > > > semantic > >> > > > > > >>> versioning checks on Malhar in case there is no version > >> > > > > compatibility. > >> > > > > > >>> > >> > > > > > >>> Thank you, > >> > > > > > >>> > >> > > > > > >>> Vlad > >> > > > > > >>> > >> > > > > > >>> > >> > > > > > >>> On 7/14/17 08:11, Thomas Weise wrote: > >> > > > > > >>> > >> > > > > > >>> next release is 3.8.0 > >> > > > > > >>>> > >> > > > > > >>>> On Fri, Jul 14, 2017 at 7:55 AM, Vlad Rozov < > >> > > > > v.ro...@datatorrent.com> > >> > > > > > >>>> wrote: > >> > > > > > >>>> > >> > > > > > >>>> next release - 3.9.0 or 4.0.0? > >> > > > > > >>>> > >> > > > > > >>>>> Thank you, > >> > > > > > >>>>> > >> > > > > > >>>>> Vlad > >> > > > > > >>>>> > >> > > > > > >>>>> On 7/13/17 22:25, Thomas Weise wrote: > >> > > > > > >>>>> > >> > > > > > >>>>> It is time to resurrect this thread and get going with > the > >> > > work. > >> > > > > > >>>>> > >> > > > > > >>>>>> For the next release, I will sign up to do the package > >> move > >> > in > >> > > > > > Malhar: > >> > > > > > >>>>>> > >> > > > > > >>>>>> https://issues.apache.org/jira/browse/APEXMALHAR-2517 > >> > > > > > >>>>>> > >> > > > > > >>>>>> In general this will be straightforward; most classes > in > >> > > Malhar > >> > > > > are > >> > > > > > >>>>>> marked > >> > > > > > >>>>>> evolving and it is trivial for users to change import > >> > > > statements. > >> > > > > > >>>>>> However, > >> > > > > > >>>>>> I would suggest to discuss if there are selected > >> operators > >> > > that > >> > > > > are > >> > > > > > >>>>>> worth > >> > > > > > >>>>>> keeping a backward compatibility stub in the old > >> location. > >> > > > > > >>>>>> > >> > > > > > >>>>>> Here is my plan: > >> > > > > > >>>>>> > >> > > > > > >>>>>> 1. Relocate all classes in *lib* and *contrib* within > one > >> > PR - > >> > > > > this > >> > > > > > is > >> > > > > > >>>>>> all > >> > > > > > >>>>>> > >> > > > > > >>>>>> IDE automated work > >> > > > > > >>>>>> 2. Add backward compatibility classes, if, any in > >> separate > >> > PR > >> > > > > > >>>>>> 3. Create PR for Megh library to reflect moved classes > >> > > > > > >>>>>> > >> > > > > > >>>>>> Thanks, > >> > > > > > >>>>>> Thomas > >> > > > > > >>>>>> > >> > > > > > >>>>>> > >> > > > > > >>>>>> > >> > > > > > >>>>>> On Wed, Mar 1, 2017 at 2:24 PM, Pramod Immaneni < > >> > > > > > >>>>>> pra...@datatorrent.com > >> > > > > > >>>>>> wrote: > >> > > > > > >>>>>> > >> > > > > > >>>>>> Inline > >> > > > > > >>>>>> > >> > > > > > >>>>>> On Mon, Feb 27, 2017 at 11:03 PM, Thomas Weise < > >> > > t...@apache.org> > >> > > > > > >>>>>>> wrote: > >> > > > > > >>>>>>> > >> > > > > > >>>>>>> --> > >> > > > > > >>>>>>> > >> > > > > > >>>>>>> On Mon, Feb 27, 2017 at 1:27 PM, Pramod Immaneni < > >> > > > > > >>>>>>>> pra...@datatorrent.com > >> > > > > > >>>>>>>> > >> > > > > > >>>>>>>> wrote: > >> > > > > > >>>>>>>> > >> > > > > > >>>>>>>> For malhar, for existing operators, I prefer we do > >> this as > >> > > > part > >> > > > > of > >> > > > > > >>>>>>>> the > >> > > > > > >>>>>>>> > >> > > > > > >>>>>>>> planned refactoring for breaking the monolith modules > >> into > >> > > > baby > >> > > > > > >>>>>>>>> > >> > > > > > >>>>>>>>> packages > >> > > > > > >>>>>>>>> > >> > > > > > >>>>>>>> and would also prefer deprecating the existing > >> operators > >> > in > >> > > > > place. > >> > > > > > >>>>>>>> Refactor into smaller modules was discussed for > >> > > malhar-contrib > >> > > > > and > >> > > > > > >>>>>>>> given > >> > > > > > >>>>>>>> the overall state of that module I think it is OK to > >> defer > >> > > > > package > >> > > > > > >>>>>>>> > >> > > > > > >>>>>>>> renaming > >> > > > > > >>>>>>>> > >> > > > > > >>>>>>> there. I do however prefer to see the package rename > >> > > addressed > >> > > > > for > >> > > > > > >>>>>>> > >> > > > > > >>>>>>>> other > >> > > > > > >>>>>>>> modules, especially for the main library module. > >> > > > > > >>>>>>>> > >> > > > > > >>>>>>>> Should we consider breaking the library into smaller > >> > modules > >> > > > as > >> > > > > > >>>>>>>> well, > >> > > > > > >>>>>>>> > >> > > > > > >>>>>>>> the > >> > > > > > >>>>>>> file/block operators for example probably can be in > >> their > >> > own > >> > > > > > module > >> > > > > > >>>>>>> from > >> > > > > > >>>>>>> just an organizational perspective. > >> > > > > > >>>>>>> > >> > > > > > >>>>>>> > >> > > > > > >>>>>>> > >> > > > > > >>>>>>> This > >> > > > > > >>>>>>> > >> > > > > > >>>>>>>> will help us achieve two things. First, the user will > >> see > >> > > all > >> > > > > the > >> > > > > > >>>>>>>>> new > >> > > > > > >>>>>>>>> changes at once as opposed to dealing with it twice > >> (with > >> > > > > package > >> > > > > > >>>>>>>>> > >> > > > > > >>>>>>>>> rename > >> > > > > > >>>>>>>>> > >> > > > > > >>>>>>>> and dependency changes) and second it will allow for > a > >> > > > smoother > >> > > > > > >>>>>>>> transition > >> > > > > > >>>>>>>> > >> > > > > > >>>>>>>> as the existing code will still work in a deprecated > >> > state. > >> > > It > >> > > > > > will > >> > > > > > >>>>>>>> > >> > > > > > >>>>>>>>> also > >> > > > > > >>>>>>>>> > >> > > > > > >>>>>>>> give a more consistent structure to malhar. For new > >> > > operators, > >> > > > > we > >> > > > > > >>>>>>>> can > >> > > > > > >>>>>>>> go > >> > > > > > >>>>>>>> with the new package path but we need to ensure they > >> will > >> > > get > >> > > > > > moved > >> > > > > > >>>>>>>> into > >> > > > > > >>>>>>>> the baby packages as well. > >> > > > > > >>>>>>>> > >> > > > > > >>>>>>>> I think existing operators should be renamed so that > >> git > >> > > > history > >> > > > > > >>>>>>>>> > >> > > > > > >>>>>>>>> remains. A > >> > > > > > >>>>>>>> > >> > > > > > >>>>>>> possible solution for backward compatibility could be > to > >> > > > > > subsequently > >> > > > > > >>>>>>> > >> > > > > > >>>>>>>> add > >> > > > > > >>>>>>>> empty subclasses in the previous location (for > existing > >> > > > concrete > >> > > > > > >>>>>>>> > >> > > > > > >>>>>>>> operators > >> > > > > > >>>>>>>> > >> > > > > > >>>>>>> that we know are actually in use) to simplify > migration > >> for > >> > > > > users. > >> > > > > > >>>>>>> > >> > > > > > >>>>>>>> > >> > > > > > >>>>>>>> Yes we can do that. > >> > > > > > >>>>>>>> > >> > > > > > >>>>>>>> For demos, we can modify the paths as the apps are > >> > typically > >> > > > > used > >> > > > > > >>>>>>> > >> > > > > > >>>>>>> wholesale > >> > > > > > >>>>>>>> > >> > > > > > >>>>>>>> and the interface is typically manual interaction. > >> > > > > > >>>>>>>> > >> > > > > > >>>>>>>>> For core, if we are adding new api subsystems, like > >> the > >> > > > > launcher > >> > > > > > >>>>>>>>> api > >> > > > > > >>>>>>>>> we > >> > > > > > >>>>>>>>> added recently for example, we can go with new > package > >> > path > >> > > > but > >> > > > > > if > >> > > > > > >>>>>>>>> we > >> > > > > > >>>>>>>>> > >> > > > > > >>>>>>>>> are > >> > > > > > >>>>>>>>> > >> > > > > > >>>>>>>> making incremental additions to existing > >> functionality, I > >> > > feel > >> > > > > it > >> > > > > > is > >> > > > > > >>>>>>>> better > >> > > > > > >>>>>>>> > >> > > > > > >>>>>>>> to keep it in the same package. I also prefer we keep > >> the > >> > > > > package > >> > > > > > of > >> > > > > > >>>>>>>> > >> > > > > > >>>>>>>>> the > >> > > > > > >>>>>>>>> > >> > > > > > >>>>>>>> implementation classes consistent with api, for > >> > > > > understandability > >> > > > > > >>>>>>>> and > >> > > > > > >>>>>>>> > >> > > > > > >>>>>>>> readability of the code. So, for example, we don't > >> change > >> > > > > package > >> > > > > > >>>>>>>>> path > >> > > > > > >>>>>>>>> > >> > > > > > >>>>>>>>> of > >> > > > > > >>>>>>>>> > >> > > > > > >>>>>>>> LogicalPlan as it is an implementation of DAG. It is > >> > > > subjective, > >> > > > > > but > >> > > > > > >>>>>>>> it > >> > > > > > >>>>>>>> > >> > > > > > >>>>>>>> will be good if we can also do the same with classes > >> > closely > >> > > > > > related > >> > > > > > >>>>>>>>> to > >> > > > > > >>>>>>>>> > >> > > > > > >>>>>>>>> the > >> > > > > > >>>>>>>>> > >> > > > > > >>>>>>>> implementation classes as well. Maybe we can moving > >> these > >> > > on a > >> > > > > > >>>>>>>> package > >> > > > > > >>>>>>>> > >> > > > > > >>>>>>>>> by > >> > > > > > >>>>>>>>> > >> > > > > > >>>>>>>> package basis, like everything in > >> > > com.datatorrent.stram.engine > >> > > > > > could > >> > > > > > >>>>>>>> be > >> > > > > > >>>>>>>> > >> > > > > > >>>>>>>> moved. For completely internal components like buffer > >> > > server, > >> > > > we > >> > > > > > can > >> > > > > > >>>>>>>>> > >> > > > > > >>>>>>>>> move > >> > > > > > >>>>>>>>> > >> > > > > > >>>>>>>> them wholesale. We can consider moving all api and > >> > classes, > >> > > > when > >> > > > > > we > >> > > > > > >>>>>>>> go > >> > > > > > >>>>>>>> to > >> > > > > > >>>>>>>> next major release but would like to see if we can > >> find a > >> > > way > >> > > > to > >> > > > > > >>>>>>>> support > >> > > > > > >>>>>>>> existing api for one more major release in deprecated > >> > mode. > >> > > > > > >>>>>>>> > >> > > > > > >>>>>>>> The point of the major release is to enable backward > >> > > > > incompatible > >> > > > > > >>>>>>>>> > >> > > > > > >>>>>>>>> changes > >> > > > > > >>>>>>>> and I don't think it is realistic to support the > >> existing > >> > > API > >> > > > > for > >> > > > > > >>>>>>>> another > >> > > > > > >>>>>>>> major release. IMO it is also not necessary as most > >> > existing > >> > > > > > >>>>>>>> application > >> > > > > > >>>>>>>> code refers to operators, attributes and the > >> application > >> > > > > > interface. > >> > > > > > >>>>>>>> > >> > > > > > >>>>>>>> Perhaps > >> > > > > > >>>>>>>> > >> > > > > > >>>>>>> it is possible to keep those around as interface > >> extensions > >> > > to > >> > > > > help > >> > > > > > >>>>>>> > >> > > > > > >>>>>>>> migration. Custom operators may need to be migrated > to > >> > > reflect > >> > > > > API > >> > > > > > >>>>>>>> > >> > > > > > >>>>>>>> changes, > >> > > > > > >>>>>>>> > >> > > > > > >>>>>>> and I would consider that a reasonable task for > operator > >> > > > > developers > >> > > > > > >>>>>>> as > >> > > > > > >>>>>>> > >> > > > > > >>>>>>>> part > >> > > > > > >>>>>>>> > >> > > > > > >>>>>>> of a major upgrade. > >> > > > > > >>>>>>> > >> > > > > > >>>>>>>> It would be good if we can keep them as deprecated > >> > interface > >> > > > > > >>>>>>>> extensions > >> > > > > > >>>>>>>> > >> > > > > > >>>>>>>> for > >> > > > > > >>>>>>> one release to provide a smoother transition. > >> > > > > > >>>>>>> > >> > > > > > >>>>>>> > >> > > > > > >>>>>>> API and implementation in engine are kept separate > >> > > > intentionally. > >> > > > > > >>>>>>> They > >> > > > > > >>>>>>> > >> > > > > > >>>>>>> reside in different packages today, so I don't see a > >> > problem > >> > > > > > renaming > >> > > > > > >>>>>>>> com.datatorrent.stram.engine as you say, even when > the > >> API > >> > > > > cannot > >> > > > > > be > >> > > > > > >>>>>>>> touched right away. > >> > > > > > >>>>>>>> > >> > > > > > >>>>>>>> They are different packages but sharing a common > prefix > >> > with > >> > > > api > >> > > > > > >>>>>>>> will > >> > > > > > >>>>>>>> be > >> > > > > > >>>>>>>> > >> > > > > > >>>>>>>> helpful to someone new to codebase in terms of > >> > readability. > >> > > > Not > >> > > > > a > >> > > > > > >>>>>>> big > >> > > > > > >>>>>>> deal > >> > > > > > >>>>>>> and can be changed. > >> > > > > > >>>>>>> > >> > > > > > >>>>>>> > >> > > > > > >>>>>>> Thanks > >> > > > > > >>>>>>> > >> > > > > > >>>>>>> On Mon, Feb 27, 2017 at 7:39 AM, Thomas Weise < > >> > > t...@apache.org> > >> > > > > > >>>>>>>> wrote: > >> > > > > > >>>>>>>> > >> > > > > > >>>>>>>>> Hi, > >> > > > > > >>>>>>>>> > >> > > > > > >>>>>>>>> This topic has come up on several PRs and I think it > >> > > > warrants a > >> > > > > > >>>>>>>>>> > >> > > > > > >>>>>>>>>> broader > >> > > > > > >>>>>>>>>> > >> > > > > > >>>>>>>>> discussion. > >> > > > > > >>>>>>>> > >> > > > > > >>>>>>>> At the time of incubation, the decision was to defer > >> > change > >> > > of > >> > > > > > Java > >> > > > > > >>>>>>>>> > >> > > > > > >>>>>>>>>> packages from com.datatorrent to org.apache.apex > till > >> > next > >> > > > > major > >> > > > > > >>>>>>>>>> > >> > > > > > >>>>>>>>>> release > >> > > > > > >>>>>>>>>> > >> > > > > > >>>>>>>>> to > >> > > > > > >>>>>>>>> > >> > > > > > >>>>>>>>> ensure backward compatibility for users. > >> > > > > > >>>>>>>>> > >> > > > > > >>>>>>>>>> Unfortunately that has lead to some confusion, as > >> > > > contributors > >> > > > > > >>>>>>>>>> > >> > > > > > >>>>>>>>>> continue > >> > > > > > >>>>>>>>>> > >> > > > > > >>>>>>>>> to > >> > > > > > >>>>>>>> > >> > > > > > >>>>>>>> add new code under legacy packages. > >> > > > > > >>>>>>>>> > >> > > > > > >>>>>>>>>> It is also a wider issue that examples for using > Apex > >> > > > continue > >> > > > > > to > >> > > > > > >>>>>>>>>> > >> > > > > > >>>>>>>>>> refer > >> > > > > > >>>>>>>>>> > >> > > > > > >>>>>>>>> to > >> > > > > > >>>>>>>> > >> > > > > > >>>>>>>> com.datatorrent packages, nearly one year after > >> > graduation. > >> > > > More > >> > > > > > and > >> > > > > > >>>>>>>>> > >> > > > > > >>>>>>>>>> more > >> > > > > > >>>>>>>>>> > >> > > > > > >>>>>>>>> user code is being built on top of something that > >> needs > >> > to > >> > > > > > change, > >> > > > > > >>>>>>>>> the > >> > > > > > >>>>>>>>> > >> > > > > > >>>>>>>>> can > >> > > > > > >>>>>>>> > >> > > > > > >>>>>>>> is being kicked down the road and users will face > more > >> > > changes > >> > > > > > >>>>>>>>> later. > >> > > > > > >>>>>>>>> > >> > > > > > >>>>>>>>>> I would like to propose the following: > >> > > > > > >>>>>>>>>> > >> > > > > > >>>>>>>>>> 1. All new code has to be submitted under > >> > org.apache.apex > >> > > > > > packages > >> > > > > > >>>>>>>>>> > >> > > > > > >>>>>>>>>> 2. Not all code is under backward compatibility > >> > > restriction > >> > > > > and > >> > > > > > in > >> > > > > > >>>>>>>>>> > >> > > > > > >>>>>>>>>> those > >> > > > > > >>>>>>>>>> > >> > > > > > >>>>>>>>> cases we can rename the packages right away. > Examples: > >> > > buffer > >> > > > > > >>>>>>>>> server, > >> > > > > > >>>>>>>>> > >> > > > > > >>>>>>>>> engine, demos/examples, benchmarks > >> > > > > > >>>>>>>>>> > >> > > > > > >>>>>>>>>> 3. Discuss when the core API and operators can be > >> > changed. > >> > > > For > >> > > > > > >>>>>>>>>> > >> > > > > > >>>>>>>>>> operators > >> > > > > > >>>>>>>>>> > >> > > > > > >>>>>>>>> we > >> > > > > > >>>>>>>>> > >> > > > > > >>>>>>>>> have a bit more freedom to do changes before a major > >> > > release > >> > > > as > >> > > > > > >>>>>>>>> most > >> > > > > > >>>>>>>>> > >> > > > > > >>>>>>>>>> of > >> > > > > > >>>>>>>>>> > >> > > > > > >>>>>>>>> them are marked @Evolving and users have the ability > >> to > >> > > > > continue > >> > > > > > >>>>>>>> > >> > > > > > >>>>>>>> using > >> > > > > > >>>>>>>>> > >> > > > > > >>>>>>>>> prior version of Malhar with newer engine due to > >> engine > >> > > > > backward > >> > > > > > >>>>>>>> > >> > > > > > >>>>>>>> compatibility guarantee. > >> > > > > > >>>>>>>>> > >> > > > > > >>>>>>>>>> Thanks, > >> > > > > > >>>>>>>>>> Thomas > >> > > > > > >>>>>>>>>> > >> > > > > > >>>>>>>>>> > >> > > > > > >>>>>>>>>> > >> > > > > > >>>>>>>>>> > >> > > > > > > > >> > > > > > > >> > > > > > >> > > > > >> > > > >> > > >> > > > > >
