Hi Thomas! In case of tailing a Kafka partition, the restriction would be [start_offset, infinity), and it would keep being split by checkpointing into [start_offset, end_offset) and [end_offset, infinity)
On Thu, Mar 8, 2018 at 3:52 PM Thomas Weise <t...@apache.org> wrote: > Eugene, > > I actually had one question regarding the application of SDF for the Kafka > consumer. Reading through a topic partition can be parallel by splitting a > partition into multiple restrictions (for use cases where order does not > matter). But how would the tail read be managed? I assume there would not > be a new restriction whenever new records arrive (added latency)? The > examples on slide 40 show an end offset for Kafka, but for a continuous > read there wouldn't be an end offset? > > Thanks, > Thomas > > > On Thu, Mar 8, 2018 at 2:59 PM, Thomas Weise <t...@apache.org> wrote: > >> Great, thanks for sharing! >> >> >> On Thu, Mar 8, 2018 at 12:16 PM, Eugene Kirpichov <kirpic...@google.com> >> wrote: >> >>> Oops that's just the template I used. Thanks for noticing, will >>> regenerate the PDF and reupload when I get to it. >>> >>> >>> On Thu, Mar 8, 2018, 11:59 AM Dan Halperin <dhalp...@apache.org> wrote: >>> >>>> Looks like it was a good talk! Why is it Google Confidential & >>>> Proprietary, though? >>>> >>>> Dan >>>> >>>> On Thu, Mar 8, 2018 at 11:49 AM, Eugene Kirpichov <kirpic...@google.com >>>> > wrote: >>>> >>>>> Hey all, >>>>> >>>>> The slides for my yesterday's talk at Strata San Jose >>>>> https://conferences.oreilly.com/strata/strata-ca/public/schedule/detail/63696 >>>>> have >>>>> been posted on the talk page. They may be of interest both to users and IO >>>>> authors. >>>>> >>>>> Thanks. >>>>> >>>> >>>> >> >
