Sorry I've mistaken it with the apache issues account, it's ok now, my id is: cody.
On Tue, Nov 24, 2015 at 11:54 PM, P. Taylor Goetz <ptgo...@gmail.com> wrote: > Hi Cody, > > I wasn’t able to find your username. Are you sure you have an account on > cwiki.apache.org? > > -Taylor > > > On Nov 22, 2015, at 8:46 AM, Cody Innowhere <e.neve...@gmail.com> wrote: > > > > Hi Taylor, > > I'd like to help too, could you add me in? my id is: Cody > > > > On Sat, Nov 21, 2015 at 11:51 AM, 刘键(Basti Liu) < > basti...@alibaba-inc.com> > > wrote: > > > >> Hi Taylor， > >> > >> Sorry for the late response. > >> I'd like to help on this. Could you please help to give me the > permission? > >> Thanks. > >> UserName: basti.lj > >> > >> Regards > >> Basti > >> > >> -----Original Message----- > >> From: P. Taylor Goetz [mailto:ptgo...@gmail.com] > >> Sent: Thursday, November 19, 2015 6:24 AM > >> To: dev@storm.apache.org; Bobby Evans > >> Subject: Re: [DISCUSS] Plan for Merging JStorm Code > >> > >> All I have at this point is a placeholder wiki entry [1], and a lot of > >> local notes that likely would only make sense to me. > >> > >> Let me know your wiki username and I’ll give you permissions. The same > >> goes for anyone else who wants to help. > >> > >> -Taylor > >> > >> [1] > >> > https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=61328109 > >> > >>> On Nov 18, 2015, at 2:08 PM, Bobby Evans <ev...@yahoo-inc.com.INVALID> > >> wrote: > >>> > >>> Taylor and others I was hoping to get started filing JIRA and planning > >>> on how we are going to do the java migration + JStorm merger. Is > >>> anyone else starting to do this? If not would anyone object to me > >>> starting on it? - Bobby > >>> > >>> > >>> On Thursday, November 12, 2015 12:04 PM, P. Taylor Goetz < > >> ptgo...@gmail.com> wrote: > >>> > >>> > >>> Thanks for putting this together Basti, that comparison helps a lot. > >>> > >>> And thanks Bobby for converting it into markdown. I was going to just > >> attach the spreadsheet to JIRA, but markdown is a much better solution. > >>> > >>> -Taylor > >>> > >>>> On Nov 12, 2015, at 12:03 PM, Bobby Evans <ev...@yahoo-inc.com.INVALID > > > >> wrote: > >>>> > >>>> I translated the excel spreadsheet into a markdown file and put up a > >> pull request for it. > >>>> https://github.com/apache/storm/pull/877 > >>>> I did a few edits to it to make it work with Markdown, and to add in a > >> few of my own comments. I also put in a field for JIRAs to be able to > >> track the migration. > >>>> Overall I think your evaluation was very good. We have a fair amount > >> of work ahead of us to decide what version of various features we want > to > >> go forward with. > >>>> - Bobby > >>>> > >>>> > >>>> On Thursday, November 12, 2015 9:37 AM, 刘键(Basti Liu) < > >> basti...@alibaba-inc.com> wrote: > >>>> > >>>> > >>>> Hi Bobby & Jungtaek, > >>>> > >>>> Thanks for your replay. > >>>> I totally agree that compatibility is the most important thing. > >> Actually, JStorm has been compatible with the user API of Storm. > >>>> As you mentioned below, we indeed still have some features different > >> between Storm and JStorm. I have tried to list them (minor update or > >> improvements are not included). > >>>> Please refer to attachment for details. If any missing, please help > >>>> to point out. (The current working features are probably missing > here.) > >> Just have a look at these differences. For the missing features in > JStorm, > >> I did not see any obstacle which will block the merge to JStorm. > >>>> For the features which has different solution between Storm and > JStorm, > >> we can evaluate the solution one by one to decision which one is > >> appropriate. > >>>> After the finalization of evaluation, I think JStorm team can take the > >> merging job and publish a stable release in 2 months. > >>>> But anyway, the detailed implementation for these features with > >> different solution is transparent to user. So, from user's point of > view, > >> there is not any compatibility problem. > >>>> > >>>> Besides compatibility, by our experience, stability is also important > >> and is not an easy job. 4 people in JStorm team took almost one year to > >> finish the porting from "clojure core" > >>>> to "java core", and to make it stable. Of course, we have many devs in > >> community to make the porting job faster. But it still needs a long > time to > >> run many online complex topologys to find bugs and fix them. So, that is > >> the reason why I proposed to do merging and build on a stable "java > core". > >>>> > >>>> -----Original Message----- > >>>> From: Bobby Evans [mailto:ev...@yahoo-inc.com.INVALID] > >>>> Sent: Wednesday, November 11, 2015 10:51 PM > >>>> To: dev@storm.apache.org > >>>> Subject: Re: [DISCUSS] Plan for Merging JStorm Code > >>>> > >>>> +1 for doing a 1.0 release based off of the clojure 0.11.x code. > >> Migrating the APIs to org.apache.storm is a big non-backwards compatible > >> move, and a major version bump to 2.x seems like a good move there. > >>>> +1 for the release plan > >>>> > >>>> I would like the move for user facing APIs to org.apache to be one of > >> the last things we do. Translating clojure code into java and moving > it to > >> org.apache I am not too concerned about. > >>>> > >>>> Basti, > >>>> We have two code bases that have diverged significantly from one > >> another in terms of functionality. The storm code now or soon will > have A > >> Heartbeat Server, Nimbus HA (Different Implementation), Resource Aware > >> Scheduling, a distributed cache like API, log searching, security, > massive > >> performance improvements, shaded almost all of our dependencies, a REST > API > >> for programtically accessing everything on the UI, and I am sure I am > >> missing a few other things. JStorm also has many changes including > cgroup > >> isolation, restructured zookeeper layout, classpath isolation, and more > too. > >>>> No matter what we do it will be a large effort to port changes from > >>>> one code base to another, and from clojure to java. I proposed this > >>>> initially because it can be broken up into incremental changes. It > >>>> may take a little longer, but we will always have a working codebase > >>>> that is testable and compatible with the current storm release, at > >>>> least until we move the user facing APIs to be under org.apache. > >>>> This lets the community continue to build and test the master branch > >>>> and report problems that they find, which is incredibly valuable. I > >>>> personally don't think it will be much easier, especially if we are > >>>> intent on always maintaining compatibility with storm. - Bobby > >>>> > >>>> > >>>> On Wednesday, November 11, 2015 5:42 AM, 刘键(Basti Liu) < > >> basti...@alibaba-inc.com> wrote: > >>>> > >>>> > >>>> Hi Taylor, > >>>> > >>>> > >>>> > >>>> Thanks for the merge plan. I have a question about “Phase 2.2”. > >>>> > >>>> Do you mean community plan to create a fresh new “java core” based on > >> current “clojure core” firstly, and then migrate the features from > JStorm? > >>>> > >>>> If so, it confused me. It is really a huge job which might require a > >> long developing time to make it stable, while JStorm is already a stable > >> version. > >>>> > >>>> The release planned to be release after Nov 11th has already run > online > >> stably several month in Alibaba. > >>>> > >>>> Besides this, there are many valuable internal requirements in > Alibaba, > >> the fast evolution of JStorm is forseeable in next few months. > >>>> > >>>> If the “java core” is totally fresh new, it might bring many problems > >> for the coming merge. > >>>> > >>>> So, from the point of this view, I think it is much better and easier > >> to migrate the features of “clojure core” basing on JStorm for the “java > >> core”. > >>>> > >>>> Please correct me, if any misunderstanding. > >>>> > >>>> > >>>> > >>>> Regards > >>>> > >>>> Basti > >>>> > >>>> > >>>> > >>>> 发件人: P. Taylor Goetz [mailto:ptgo...@gmail.com] > >>>> 发送时间: 2015年11月11日 5:32 > >>>> 收件人: dev@storm.apache.org > >>>> 主题: [DISCUSS] Plan for Merging JStorm Code > >>>> > >>>> > >>>> > >>>> Based on a number of discussions regarding merging the JStorm code, > >> I’ve tried to distill the ideas presented and inserted some of my own. > The > >> result is below. > >>>> > >>>> > >>>> > >>>> I’ve divided the plan into three phases, though they are not > >> necessarily sequential — obviously some tasks can take place in > parallel. > >>>> > >>>> > >>>> > >>>> None of this is set in stone, just presented for discussion. Any and > >> all comments are welcome. > >>>> > >>>> > >>>> > >>>> ------- > >>>> > >>>> > >>>> > >>>> Phase 1 - Plan for 0.11.x Release > >>>> > >>>> 1. Determine feature set for 0.11.x and publish to wiki [1]. > >>>> > >>>> 2. Announce feature-freeze for 0.11.x > >>>> > >>>> 3. Create 0.11.x branch from master (Phase 2.4 can begin.) > >>>> > >>>> 4. Release 0.11.0 (or whatever version # we want to use) > >>>> > >>>> 5. Bug fixes and subsequent releases from 0.11.x-branch > >>>> > >>>> > >>>> > >>>> > >>>> > >>>> Phase 2 - Prepare for Merge ("master" and "jstorm-import" branches) > >>>> > >>>> 1. Determine/document unique features in JStorm (e.g. classpath > >> isolation, cgroups, etc.) and create JIRA for migrating the feature. > >>>> > >>>> 2. Create JIRA for migrating each clojure component (or logical group > >> of components) to Java. Assumes tests will be ported as well. > >>>> > >>>> 3. Discuss/establish style guide for Java coding conventions. Consider > >> using Oracle’s or Google’s Java conventions as a base — they are both > >> pretty solid. > >>>> > >>>> 4. align package names (e.g backtype.storm --> org.apache.storm / > >>>> com.alibaba.jstorm --> org.apache.storm) (Phase 3 can begin) > >>>> > >>>> > >>>> > >>>> > >>>> > >>>> Phase 3 - Migrate Clojure --> Java > >>>> > >>>> 1. Port code/tests to Java, leveraging existing JStorm code wherever > >> possible (core functionality only, features distinct to JStorm migrated > >> separately). > >>>> > >>>> 2. Port JStorm-specific features. > >>>> > >>>> 3. Begin releasing preview/beta versions. > >>>> > >>>> 4. Code cleanup (across the board) and refactoring using established > >>>> coding conventions, and leveraging PMD/Checkstyle reports for > >>>> reference. (Note: good oportunity for new contributors.) > >>>> > >>>> 5. Release 0.12.0 (or whatever version # we want to use) and lift > >> feature freeze. > >>>> > >>>> > >>>> > >>>> > >>>> > >>>> Notes: > >>>> > >>>> We should consider bumping up to version 1.0 sometime soon and then > >> switching to semantic versioning [3] from then on. > >>>> > >>>> > >>>> > >>>> > >>>> > >>>> With the exception of package name alignment, the "jstorm-import" > >> branch will largely be read-only throughout the process. > >>>> > >>>> > >>>> > >>>> During migration, it's probably easiest to operate with two local > >> clones of the Apache Storm repo: one for working (i.e. checked out to > >> working branch) and one for reference/copying (i.e. checked out to > >> "jstorm-import"). > >>>> > >>>> > >>>> > >>>> Feature-freeze probably only needs to be enforced against core > >> functionality. Components under "external" can likely be exempt, but we > >> should figure out a process for accepting and releasing new features > during > >> the migration. > >>>> > >>>> > >>>> > >>>> Performance testing should be continuous throughout the process. Since > >> we don't really have ASF infrastructure for performance testing, we will > >> need a volunteer(s) to host and run the performance tests. Performance > test > >> results can be posted to the wiki [2]. It would probably be a good idea > to > >> establish a baseline with the 0.10.0 release. > >>>> > >>>> > >>>> > >>>> I’ve attached an analysis document Sean Zhong put together a while > back > >> to the JStorm merge JIRA [4]. The analysis was against the 0.9.3 release > >> but is still relevant and has a lot of good information. > >>>> > >>>> > >>>> > >>>> > >>>> > >>>> [1] > >>>> https://cwiki.apache.org/confluence/display/STORM/Release+0.11.0+Feat > >>>> ure+Set > >>>> > >>>> [2] https://cwiki.apache.org/confluence/display/STORM/Storm+Home > >>>> > >>>> [3] http://semver.org > >>>> > >>>> [4] https://issues.apache.org/jira/browse/STORM-717 > >>>> > >>>> > >>>> > >>>> > >>>> > >>>> -Taylor > >>>> > >>>> > >>>> > >>>> > >>>> > >>>> > >>> > >>> > >> > >> > >> > >
