rmetzger commented on PR #6: URL: https://github.com/apache/flink-connector-redis-streams/pull/6#issuecomment-4398129085
> Job parallelism is naturally bounded by the number of distinct stream keys. Is it really the job parallelism, or the source parallelism? > Deferred ACK with Per-Checkpoint Snapshots Why is ACK-ing needed in the first place? I have no clue about Redis Streams, but my AI tells me that ACKing is only needed when using Consumer groups. But why do we need to have consumer groups for Flink? Can't we do the assignment of Flink splits to source instances ourselves, persist the last consumed message ID in Flink state, and on a failure we recover from the last message ID per partition? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
