Re: KeySelector when joining with the solution set

2014-09-05 Thread Vasiliki Kalavri
Hi, I've created issues for (1) https://issues.apache.org/jira/browse/FLINK-1091 and (2) https://issues.apache.org/jira/browse/FLINK-1092. Regarding (3), I'm not quite sure how it should be done or whether it should be part of a more general iteration type, so I'd like to think about it a bit

Re: KeySelector when joining with the solution set

2014-09-04 Thread Fabian Hueske
Hi Vasia, thanks for the nice wrap-up :-) All of these would be very valuable enhancements of the system, IMO. I'd suggest you create JIRAs for the individual points. It would be very cool, if you'd pick up some of the tasks. :-) Cheers, Fabian 2014-09-04 20:43 GMT+02:00 Vasiliki Kalavri

Re: KeySelector when joining with the solution set

2014-08-31 Thread Aljoscha Krettek
HI Guys, sorry for being so late in this discussion but I was thinking about how we could make iterations more intuitive. My Idea would be to make access to the solution set explicit in Delta Iterations. It should be usable as input for any operation and updates to it would be done through a

Re: KeySelector when joining with the solution set

2014-08-30 Thread Stephan Ewen
Hey! Sorry for the late reply, but here are some thoughts in that direction: Stateful operations: In order to keep state (in the form of a hash map or so) around in iterations, you need not use the restricted means that the delta iterations give you. All function instances stay around for all

Re: KeySelector when joining with the solution set

2014-08-30 Thread Stephan Ewen
Addendum: The issue to enforce partitioning of the data set is tracked here: https://issues.apache.org/jira/browse/FLINK-1060 On Sat, Aug 30, 2014 at 6:01 PM, Stephan Ewen se...@apache.org wrote: Hey! Sorry for the late reply, but here are some thoughts in that direction: Stateful

KeySelector when joining with the solution set

2014-07-31 Thread Vasiliki Kalavri
Hello all, I am working on a program where the solution set type is Tuple2Long, Tuple2Long, Double. At some point, I need to join with the solution set on the first field of the nested tuple (the Double), but I am getting an error that the solution set may only be joined with using tuple field

Re: KeySelector when joining with the solution set

2014-07-31 Thread Stephan Ewen
Hi Vasia! There is no fundamental reason, we simply have not gotten around to implementing it, yet. Any help along these lines is highly welcome. One reason that held us back is that we need to make sure that the key of the solution set and the key of the join is the same. That is hard to verify

Re: KeySelector when joining with the solution set

2014-07-31 Thread Vasiliki Kalavri
Hey, thanks for replying so fast :) I saw the discussion in a previous thread concerning changing the API to offer more explicit join functions. I think providing these special functions is a good way to disable any other kind of interaction with the solution set. However, as a user, I would