Re: Off-by-one issues in the windowing code
Hey Marton, Thanks for spotting the two issues and fixing the first one. Those should definitely be fixed in 0.9.1. Cheers, Max On Mon, Jun 29, 2015 at 11:14 AM, Gyula Fóra wrote: > The second issue is related to parallel time based aggregations. I think we > should fix this for 0.9.1. > > Also since the fix as you said is rather straight-forward there is no harm > doing it. As I understand if we keep the functionality of having time based > global windows, the implementations for merging the partial aggregates will > be conceptually similar after Aljoscha's update as well. > > Márton Balassi ezt írta (időpont: 2015. jún. > 29., H, 11:02): > > > I have found two off-by-one issues in the windowing code. > > > > The first may result in duplicate data in the last window and is easy to > > fix. [1] > > > > The second may result data being swallowed in the last window, and is > also > > not difficult to fix. [2] > > > > I've talked to Aljoscha about fixing the second one, and he suggested not > > to fix it right away as that part should be rewritten soon anyways, maybe > > we remove that functionality as a whole. As this is also in the 0.9.0 > > release I would still opt for having it, at least for the sake of 0.9.1. > > > > What do you think? > > > > [1] https://issues.apache.org/jira/browse/FLINK-2285 > > [2] https://issues.apache.org/jira/browse/FLINK-2286 > > > > Best, > > > > Marton > > >
Re: Off-by-one issues in the windowing code
The second issue is related to parallel time based aggregations. I think we should fix this for 0.9.1. Also since the fix as you said is rather straight-forward there is no harm doing it. As I understand if we keep the functionality of having time based global windows, the implementations for merging the partial aggregates will be conceptually similar after Aljoscha's update as well. Márton Balassi ezt írta (időpont: 2015. jún. 29., H, 11:02): > I have found two off-by-one issues in the windowing code. > > The first may result in duplicate data in the last window and is easy to > fix. [1] > > The second may result data being swallowed in the last window, and is also > not difficult to fix. [2] > > I've talked to Aljoscha about fixing the second one, and he suggested not > to fix it right away as that part should be rewritten soon anyways, maybe > we remove that functionality as a whole. As this is also in the 0.9.0 > release I would still opt for having it, at least for the sake of 0.9.1. > > What do you think? > > [1] https://issues.apache.org/jira/browse/FLINK-2285 > [2] https://issues.apache.org/jira/browse/FLINK-2286 > > Best, > > Marton >
Off-by-one issues in the windowing code
I have found two off-by-one issues in the windowing code. The first may result in duplicate data in the last window and is easy to fix. [1] The second may result data being swallowed in the last window, and is also not difficult to fix. [2] I've talked to Aljoscha about fixing the second one, and he suggested not to fix it right away as that part should be rewritten soon anyways, maybe we remove that functionality as a whole. As this is also in the 0.9.0 release I would still opt for having it, at least for the sake of 0.9.1. What do you think? [1] https://issues.apache.org/jira/browse/FLINK-2285 [2] https://issues.apache.org/jira/browse/FLINK-2286 Best, Marton
