Re: Planning Flink release 0.7-incubating

2014-10-08 Thread Fabian Hueske
Done with reverting and merging the individual commits. Sorry, didn't wanted to sound too negative. I just think merged commits are an important incentive for contributors. And thanks for merging of course ;-) 2014-10-08 12:25 GMT+02:00 Robert Metzger : > Oh, damn. You're right. I've been using

Re: Planning Flink release 0.7-incubating

2014-10-08 Thread Robert Metzger
Oh, damn. You're right. I've been using the "merge_pull_request.sh" script which is squashing together the commits automatically. In this particular case, its quite unfortunate because it contains part of the work of Artems GSoC summer project, and also another pull request from Timo. I agree if y

Re: Planning Flink release 0.7-incubating

2014-10-08 Thread Fabian Hueske
I checked the merged Hadoop function compatibility pull request. The PR consisted of four individual commits of three authors. All commits have been squashed into a single commit such that two authors basically lost their contributions. I think this is not correct and should never be done. It might

Re: Planning Flink release 0.7-incubating

2014-10-08 Thread Robert Metzger
I merged the Hadoop compat and POJOs, currently rebasing stephan's fault tolerance On Wed, Oct 8, 2014 at 10:50 AM, Fabian Hueske wrote: > I think, the question is which PRs to merge into master before forking of > the release candidate branch. > You did already merge #126 into the master b

Re: Planning Flink release 0.7-incubating

2014-10-08 Thread Fabian Hueske
I think, the question is which PRs to merge into master before forking of the release candidate branch. You did already merge #126 into the master branch, so it will be in ;-) 2014-10-08 10:37 GMT+02:00 Till Rohrmann : > We can also include the blob manager. The corresponding PR is #126. > > On W

Re: Planning Flink release 0.7-incubating

2014-10-08 Thread Till Rohrmann
We can also include the blob manager. The corresponding PR is #126. On Wed, Oct 8, 2014 at 1:51 AM, Márton Balassi wrote: > Pushed the resolution for FLINK-1103 and the streaming bugfix commit. > > We are good for now on our side for an rc. Maybe I should add the streaming > connectors dependenc

Re: Planning Flink release 0.7-incubating

2014-10-07 Thread Márton Balassi
Pushed the resolution for FLINK-1103 and the streaming bugfix commit. We are good for now on our side for an rc. Maybe I should add the streaming connectors dependency fix that Stephan suggested here an the mailing list soon. Marton On Tue, Oct 7, 2014 at 11:44 PM, Stephan Ewen wrote: > We can

Re: Planning Flink release 0.7-incubating

2014-10-07 Thread Stephan Ewen
We can add it and keep it initially undocumented (experimental) until further tests... On Tue, Oct 7, 2014 at 10:40 PM, Robert Metzger wrote: > I would add #142. > > On Tue, Oct 7, 2014 at 10:29 PM, Stephan Ewen wrote: > > > I agree, having a release candidate out would be nice. > > > > What is

Re: Planning Flink release 0.7-incubating

2014-10-07 Thread Robert Metzger
I would add #142. On Tue, Oct 7, 2014 at 10:29 PM, Stephan Ewen wrote: > I agree, having a release candidate out would be nice. > > What is your opinion on issue #142? Fault tolerance is inactive by default, > but can be activated through the configuration. > > On Tue, Oct 7, 2014 at 7:18 PM, Ro

Re: Planning Flink release 0.7-incubating

2014-10-07 Thread Stephan Ewen
I agree, having a release candidate out would be nice. What is your opinion on issue #142? Fault tolerance is inactive by default, but can be activated through the configuration. On Tue, Oct 7, 2014 at 7:18 PM, Robert Metzger wrote: > Oh. I didn't know this. The last mail from Gyula sounded lik

Re: Planning Flink release 0.7-incubating

2014-10-07 Thread Robert Metzger
Oh. I didn't know this. The last mail from Gyula sounded like the Streaming part is ready2release. My personal goal was to prepare a release candiate today so that we have a reference point against which we can test (which would also mean forking of a 0.7-release branch and basically doing a featu

Re: Planning Flink release 0.7-incubating

2014-10-07 Thread Márton Balassi
As for the streaming side we would like to push some bugfix commits and the resolution of the FLINK-1103 JIRA issue. These are more or less ready, hopefully will be available at the end of this week. On Tue, Oct 7, 2014 at 6:50 PM, Robert Metzger wrote: > I've merged the record api deprecation

Re: Planning Flink release 0.7-incubating

2014-10-07 Thread Robert Metzger
I've merged the record api deprecation already. I'll merge #141 once Aljoscha provided his Scala changes. We certainly should merge #136 and #143 as well. On Tue, Oct 7, 2014 at 5:20 PM, Fabian Hueske wrote: > So which PRs will be included in the candidate? > > #141 POJOs > #144 Deprecate Recor

Re: Planning Flink release 0.7-incubating

2014-10-07 Thread Fabian Hueske
So which PRs will be included in the candidate? #141 POJOs #144 Deprecate Record API #136 Fixed example quickstart #143(?) Hadoop Compat: Documentation + Hadoop function wrappers (includes PR #131) 2014-10-07 16:26 GMT+02:00 Stephan Ewen : > Great news, looking forward to seeing this in the mast

Re: Planning Flink release 0.7-incubating

2014-10-07 Thread Stephan Ewen
Great news, looking forward to seeing this in the master! On Tue, Oct 7, 2014 at 1:53 PM, Robert Metzger wrote: > As an update for everyone: My POJO feature is finished, including > documentation. > Aljoscha is currently adopting the Scala API to have support for (nested) > POJOs as well. > Once

Re: Planning Flink release 0.7-incubating

2014-10-07 Thread Robert Metzger
As an update for everyone: My POJO feature is finished, including documentation. Aljoscha is currently adopting the Scala API to have support for (nested) POJOs as well. Once that is done, I'll merge everything and create a first candidate that we can use for testing. On Tue, Sep 30, 2014 at 10:49

Re: Planning Flink release 0.7-incubating

2014-09-30 Thread Fabian Hueske
I just checked the "Run Example" quickstart and it needs a bit of work. 2014-09-30 22:41 GMT+02:00 Robert Metzger : > I'm working hard on getting the POJOs ready. > > We also should do a pass over our documentation, the quickstarts and the > website to see if everything is in a good state (for ex

Re: Planning Flink release 0.7-incubating

2014-09-30 Thread Robert Metzger
I'm working hard on getting the POJOs ready. We also should do a pass over our documentation, the quickstarts and the website to see if everything is in a good state (for example the collection-based execution needs documentation as well). We should also finally document the hadoop-input format wr

Re: Planning Flink release 0.7-incubating

2014-09-30 Thread Stephan Ewen
I think we are approaching ready state. Last issues are going in and we started working on dependencies and test platform diversity in order to make stabilizing phase. We should have an official feature freeze soon and fork the 0.7-release branch. I personally vote to include the POJO support (I t

Re: Planning Flink release 0.7-incubating

2014-09-30 Thread Gyula Fora
Hey, So what is the current decision regarding the time of the upcoming release? As for the streaming component, we included all the features we wanted, we will start to test everything tomorrow, making sure that all works as intended. We are also almost finished with cleaning up the connector

Re: Planning Flink release 0.7-incubating

2014-09-26 Thread Fabian Hueske
+1 to manage this on JIRA (if possible) 2014-09-26 10:40 GMT+02:00 Aljoscha Krettek : > Can we not manage this stuff on Jira? > > On Fri, Sep 26, 2014 at 10:16 AM, Stephan Ewen wrote: > > I personally like the idea of SOFT time-based feature freezes. Otherwise, > > releases will get delayed agai

Re: Planning Flink release 0.7-incubating

2014-09-26 Thread Aljoscha Krettek
Can we not manage this stuff on Jira? On Fri, Sep 26, 2014 at 10:16 AM, Stephan Ewen wrote: > I personally like the idea of SOFT time-based feature freezes. Otherwise, > releases will get delayed again and again, because of features that we try > to squeeze in. > > Having reached the freeze point

Re: Planning Flink release 0.7-incubating

2014-09-26 Thread Stephan Ewen
I personally like the idea of SOFT time-based feature freezes. Otherwise, releases will get delayed again and again, because of features that we try to squeeze in. Having reached the freeze point already, we could still include the features that are pending ready state in the next days (streaming,

Re: Planning Flink release 0.7-incubating

2014-09-26 Thread Daniel Warneke
Hi, I like Fabian's idea. Is there a wiki page (or something similar) where we can collect the proposed JIRAs? Best regards, Daniel Am 24.09.2014 23:03, schrieb Fabian Hueske: I agree, a hard feature stop deadline might not be the best practice. How about the following procedure: We de

Re: Planning Flink release 0.7-incubating

2014-09-24 Thread Fabian Hueske
I agree, a hard feature stop deadline might not be the best practice. How about the following procedure: We decide two (or three) weeks before a targeted release date about which JIRAs to include. JIRAs that are selected for a release should be completed or really close to completion (via progress

Re: Planning Flink release 0.7-incubating

2014-09-24 Thread Márton Balassi
As for the streaming team we're also getting ready for the release, but a couple of days will be needed to finish the features that we would like to include. - A little work is still needed for reduce operations and groups/connected streams (any comment on Gyula's recent e-mail is really

Re: Planning Flink release 0.7-incubating

2014-09-24 Thread Ufuk Celebi
On 24 Sep 2014, at 18:37, Robert Metzger wrote: > Hey guys, > > exactly 3 weeks ago, we discussed to do a feature freeze for the > 0.7-incubating release today. > > From our initial feature list: > - *Flink Streaming* "Beta Preview". I would suggest to ship the streaming, > but clearly mark it

Re: Planning Flink release 0.7-incubating

2014-09-24 Thread Robert Metzger
Hey guys, exactly 3 weeks ago, we discussed to do a feature freeze for the 0.7-incubating release today. >From our initial feature list: - *Flink Streaming* "Beta Preview". I would suggest to ship the streaming, but clearly mark it as a preview in the documentation. -* Java API Pojo improvement

Re: Planning Flink release 0.7-incubating

2014-09-04 Thread Kostas Tzoumas
+1 On Thu, Sep 4, 2014 at 12:00 PM, Gyula Fóra wrote: > +1 > 2014.09.04. 11:52 ezt írta ("Stephan Ewen" ): > > > +1 > > Am 04.09.2014 00:05 schrieb "Fabian Hueske" : > > > > > +1 > > > > > > > > > 2014-09-03 22:39 GMT+02:00 Kostas Tzoumas : > > > > > > > +1 > > > > > > > > > > > > On Wed, Sep 3

Re: Planning Flink release 0.7-incubating

2014-09-04 Thread Gyula Fóra
+1 2014.09.04. 11:52 ezt írta ("Stephan Ewen" ): > +1 > Am 04.09.2014 00:05 schrieb "Fabian Hueske" : > > > +1 > > > > > > 2014-09-03 22:39 GMT+02:00 Kostas Tzoumas : > > > > > +1 > > > > > > > > > On Wed, Sep 3, 2014 at 10:12 PM, Márton Balassi < > > balassi.mar...@gmail.com> > > > wrote: > > > >

Re: Planning Flink release 0.7-incubating

2014-09-04 Thread Stephan Ewen
+1 Am 04.09.2014 00:05 schrieb "Fabian Hueske" : > +1 > > > 2014-09-03 22:39 GMT+02:00 Kostas Tzoumas : > > > +1 > > > > > > On Wed, Sep 3, 2014 at 10:12 PM, Márton Balassi < > balassi.mar...@gmail.com> > > wrote: > > > > > +1 > > > > > > > > > On Wed, Sep 3, 2014 at 9:54 PM, Till Rohrmann > > >

Re: Planning Flink release 0.7-incubating

2014-09-03 Thread Fabian Hueske
+1 2014-09-03 22:39 GMT+02:00 Kostas Tzoumas : > +1 > > > On Wed, Sep 3, 2014 at 10:12 PM, Márton Balassi > wrote: > > > +1 > > > > > > On Wed, Sep 3, 2014 at 9:54 PM, Till Rohrmann > > wrote: > > > > > +1 Sounds good to me. > > > > > > > > > On Wed, Sep 3, 2014 at 1:58 PM, Robert Metzger > >

Re: Planning Flink release 0.7-incubating

2014-09-03 Thread Kostas Tzoumas
+1 On Wed, Sep 3, 2014 at 10:12 PM, Márton Balassi wrote: > +1 > > > On Wed, Sep 3, 2014 at 9:54 PM, Till Rohrmann > wrote: > > > +1 Sounds good to me. > > > > > > On Wed, Sep 3, 2014 at 1:58 PM, Robert Metzger > > wrote: > > > > > Hi, > > > > > > sorry, my previous message was confusing. I s

Re: Planning Flink release 0.7-incubating

2014-09-03 Thread Márton Balassi
+1 On Wed, Sep 3, 2014 at 9:54 PM, Till Rohrmann wrote: > +1 Sounds good to me. > > > On Wed, Sep 3, 2014 at 1:58 PM, Robert Metzger > wrote: > > > Hi, > > > > sorry, my previous message was confusing. I suggest to release a new > Flink > > version all 3 months. > > BUT, the 0.7-incubating rel

Re: Planning Flink release 0.7-incubating

2014-09-03 Thread Till Rohrmann
+1 Sounds good to me. On Wed, Sep 3, 2014 at 1:58 PM, Robert Metzger wrote: > Hi, > > sorry, my previous message was confusing. I suggest to release a new Flink > version all 3 months. > BUT, the 0.7-incubating release is going to be feature freeze in 3 weeks > because 0.6-incubating was more a

Re: Planning Flink release 0.7-incubating

2014-09-03 Thread Robert Metzger
Hi, sorry, my previous message was confusing. I suggest to release a new Flink version all 3 months. BUT, the 0.7-incubating release is going to be feature freeze in 3 weeks because 0.6-incubating was more about getting the release infra set up and the apache rename out. The last release that cont

Re: Planning Flink release 0.7-incubating

2014-09-03 Thread Ufuk Celebi
Hey Robert, +1 to frequent regular frequent major releases (I guess you meant 3 weeks and not 3 months, right?). Ufuk On Tue, Sep 2, 2014 at 9:29 PM, Robert Metzger wrote: > Hi, > I agree with Fabian that the list of features is a lot of work. I would > prefer to have frequent regular major r

Re: Planning Flink release 0.7-incubating

2014-09-02 Thread Robert Metzger
Hi, I agree with Fabian that the list of features is a lot of work. I would prefer to have frequent regular major releases (say 3 months schedule). This way, users can quickly access the latest features and we don't force them to use SNAPSHOT versions. Also, releases draw attention to our project.

Re: Planning Flink release 0.7-incubating

2014-08-27 Thread Fabian Hueske
I agree that these should be features to add soon, but I'm a bit doubtful that we will have the next release in 5 weeks if we want to include all of this. I think we should either have a feature- or time-oriented release plan. If we want to have a fixed release date, we could make a feature stop 1

Re: Planning Flink release 0.7-incubating

2014-08-27 Thread Stephan Ewen
This is a nice list. I would like to add: - Rework JobManager internals to support incremental program rollout & execution - First parts of dynamic memory assignments On Wed, Aug 27, 2014 at 7:21 PM, Robert Metzger wrote: > Hi, > > since we have our release infrastructure in place now, I

Planning Flink release 0.7-incubating

2014-08-27 Thread Robert Metzger
Hi, since we have our release infrastructure in place now, I would suggest to aim for a 0.7-incubating release in the near future (say 3-5 weeks). While 0.6-incubating was mainly about getting the release infra / legal stuff sorted out, I think it would be nice to have a "feature" release soon. T