Re: Kafka partition alignment for event time

2016-02-25 Thread Erdem Agaoglu
which I'll post a > release candidate today) as well. > > On Thu, Feb 18, 2016 at 3:24 PM, Erdem Agaoglu > wrote: > >> Thanks Stephan >> >> On Thu, Feb 18, 2016 at 3:00 PM, Stephan Ewen wrote: >> >>> You are right, the checkpoints shou

Re: Kafka partition alignment for event time

2016-02-18 Thread Erdem Agaoglu
Thanks Stephan On Thu, Feb 18, 2016 at 3:00 PM, Stephan Ewen wrote: > You are right, the checkpoints should contain all offsets. > > I created a Ticket for this: > https://issues.apache.org/jira/browse/FLINK-3440 > > > > > On Thu, Feb 18, 2016 at 10:15 AM, agaoglu

Re: Kafka partition alignment for event time

2016-02-18 Thread agaoglu
Hi, On a related and a more exaggerated setup, our kafka-producer (flume) seems to send data to a single partition at a time and switches it every few minutes. So when i run my flink datastream program for the first time, it starts on the *largest* offsets and shows something like this: . Fetched