Re: Sorting of fields

2015-02-05 Thread Stephan Ewen
Based on this, we should also be able to implement a global top-k, which has come up as a frequent requirement. On Wed, Feb 4, 2015 at 2:55 PM, Fabian Hueske fhue...@gmail.com wrote: I just merged support for local output sorting yesterday :-) This allows to sort the data before it is given to

[jira] [Created] (FLINK-1479) The spawned threads in the sorter have no context class loader

2015-02-05 Thread Stephan Ewen (JIRA)
Stephan Ewen created FLINK-1479: --- Summary: The spawned threads in the sorter have no context class loader Key: FLINK-1479 URL: https://issues.apache.org/jira/browse/FLINK-1479 Project: Flink

Planning Release 0.8.1

2015-02-05 Thread Robert Metzger
Hi guys, I would like to bundle a minor bugfix release for Flink soon. Some users were complaining about incomplete Kryo support, in particular for Avro. Also, we fixed some other issues which are easy to to port to 0.8.1 (some of them are already in the branch). I would like to start the vote

Re: Planning Release 0.8.1

2015-02-05 Thread Stephan Ewen
I think we need to make a pass through the recent 0.9 commits and cherry pick some more into 0.8.1. There were quite a few bug fixes. Also, this one is rather critical and pending: https://github.com/apache/flink/pull/318 On Thu, Feb 5, 2015 at 2:27 PM, Robert Metzger rmetz...@apache.org wrote:

Re: [DISCUSS] Be more patient with PR and patches in the review

2015-02-05 Thread Henry Saputra
Ah awesome, I do not about that, thanks for letting me know. Mea culpa from me. I think I saw only couple cases but thought I raise the discussions before I forgot =P Thanks for addressing this so quickly, Stephan. - Henry On Thu, Feb 5, 2015 at 8:09 AM, Stephan Ewen se...@apache.org wrote:

Re: [DISCUSS] Be more patient with PR and patches in the review

2015-02-05 Thread Max Michels
Hi Henry, I forgot to leave a message stating that I'm fine with Stephan's changes that would soon be merged into the master. Stephan did not push to the master immediately, so further comments could have been made to the pull request. It would have been more transparent if we had posted the

Re: suitable implementation tasks / student projects?

2015-02-05 Thread Stephan Ewen
Hi Adnan! If you are looking for a bigger involvement (and a project of your own), you can have a look here at the roadmap and figure out the direction that interests you most: https://cwiki.apache.org/confluence/display/FLINK/Flink+Roadmap I think that a cool project would be the static code

Re: Cluster execution - Jobmanager unreachable

2015-02-05 Thread Till Rohrmann
It looks to me that the TaskManager does not receive a ConsumerNotificationResult after having send the ScheduleOrUpdateConsumers message. This can either mean that something went wrong in ExecutionGraph.scheduleOrUpdateConsumers method or the connection was disassociated for some reasons. The

Re: Cluster execution - Jobmanager unreachable

2015-02-05 Thread Stephan Ewen
I suspect that this is one of the cases where an exception in an actor causes the actor to die (here the job manager) On Thu, Feb 5, 2015 at 10:40 AM, Till Rohrmann trohrm...@apache.org wrote: It looks to me that the TaskManager does not receive a ConsumerNotificationResult after having send

[jira] [Created] (FLINK-1478) Add strictly local input split assignment

2015-02-05 Thread Stephan Ewen (JIRA)
Stephan Ewen created FLINK-1478: --- Summary: Add strictly local input split assignment Key: FLINK-1478 URL: https://issues.apache.org/jira/browse/FLINK-1478 Project: Flink Issue Type: New

Fwd: Google Summer of Code 2015 is coming

2015-02-05 Thread Henry Saputra
I have seen some interests from students about Flink. Maybe should officially submit proposal to Google summer of code this year? -- Forwarded message -- From: Ulrich Stärk u...@apache.org Date: Mon, Feb 2, 2015 at 2:44 PM Subject: Google Summer of Code 2015 is coming To:

Re: Cluster execution - Jobmanager unreachable

2015-02-05 Thread Till Rohrmann
I checked and indeed the scheduleOrUpdateConsumers method can throw an IllegalStateException without properly handling such an exception on the JobManager level. It is a design decision of Scala not to complain about unhandled exceptions which are otherwise properly annotated in Java code. We