+1 On Tue, 1 Jan 2019 at 12:45, <jan.d...@gmail.com> wrote:
> +1 from my side too :-) > And ideally I would want to have some hooks to let me know the extra > partitions have been picked up (or a way to query it). > > Although if that can't be provided I can work around it myself by sending > some specific message to the partition that somewhere results in a visible > state change in the pipeline. > > Also, as a quick (semi related) heads up: I will very likely soon > contribute a change to the LogAppendTimePolicy so that the idle partition > behavior (automatic watermark generation) can be disabled. > > (of course all related to my streamy-db project) > > Kind regards, > Jan > > > On Tue, 1 Jan 2019 at 08:19, Ramesh Nethi <ramesh.ne...@gmail.com> wrote: > >> +1 for this capability. This would enable pipelines to continue to run >> when such changes need to be made. >> >> regards >> Ramesh >> >> On Fri, 23 Nov 2018 at 00:40 Raghu Angadi <rang...@google.com> wrote: >> >>> On Thu, Nov 22, 2018 at 10:10 AM Raghu Angadi <rang...@google.com> >>> wrote: >>> >>>> - New partitions will be ignored during runtime. >>>> - Update will not succeed either. Error message on the workers should >>>> explain the mismatch. >>>> >>> >>> This is the current state. Supporting changes to number of partition is >>> quite doable if there is enough user interested (even in the current >>> UnnoundedSource API framework). >>> >>>> >>>> On Thu, Nov 22, 2018 at 2:15 AM Jozef Vilcek <jozo.vil...@gmail.com> >>>> wrote: >>>> >>>>> Hello, >>>>> just wanted to check how does Beam KafkaIO behaves when partitions are >>>>> added to the topic. >>>>> Will they be picked up or ignored during the runtime? >>>>> Will they be picked up on restart with state restore? >>>>> >>>>> Thanks, >>>>> Jozef >>>>> >>>>
