Hi Stanislav, you're right we envision the replicator use case to have a single producer with offsets per partition (I am reading your post thinking about partitions rather than topics).
If a regular producer was to send its own records at the same time, it's very likely that the one sending with an offset will fail because of invalid offsets. Same if two producers were sending with offsets, likely both would then fail. > Does it make sense to *lock* the topic from other producers while there is > one that uses offsets? You could do that with ACL permissions if you wanted, I don't think it needs to be mandated by changing the broker logic. > Since we are tying the produce-with-offset request to the ACL, do we need > the `use_offset` field in the produce request? Maybe we make it mandatory > for produce requests with that ACL to have offsets. The flag is needed to distinguish a batch with a desired base offset of 0, from a regular batch for which offsets need to be generated. I would not restrict a principal to only send-with-offsets (by making that mandatory via the ACL). Thanks Edo & Mickael -------------------------------------------------- Edoardo Comar IBM Event Streams IBM UK Ltd, Hursley Park, SO21 2JN Stanislav Kozlovski <stanis...@confluent.io> wrote on 22/11/2018 16:17:11: > From: Stanislav Kozlovski <stanis...@confluent.io> > To: dev@kafka.apache.org > Date: 22/11/2018 16:17 > Subject: Re: [DISCUSS] KIP-391: Allow Producing with Offsets for > Cluster Replication > > Hey Edurdo, thanks for the KIP! > > I have some questions, apologies if they are naive: > Is this intended to work for a single producer use case only? > How would it work if two producers were producing to the same topic with > offsets? > How would it work if two producers, one with offsets and one without were > producing to a topic? > Does it make sense to *lock* the topic from other producers while there is > one that uses offsets? > > Since we are tying the produce-with-offset request to the ACL, do we need > the `use_offset` field in the produce request? Maybe we make it mandatory > for produce requests with that ACL to have offsets. > > Best, > Stanislav > > On Wed, Nov 21, 2018 at 5:14 PM Edoardo Comar <eco...@uk.ibm.com> wrote: > > > Hi, > > we've opened a KIP to improve data replication between Kafka clusters : > > > > > > INVALID URI REMOVED > u=https-3A__cwiki.apache.org_confluence_display_KAFKA_KIP-2D391-253A-2BAllow-2BProducing-2Bwith-2BOffsets-2Bfor-2BCluster-2BReplication&d=DwIBaQ&c=jf_iaSHvJObTbx- > siA1ZOg&r=EzRhmSah4IHsUZVekRUIINhltZK7U0OaeRo7hgW4_tQ&m=uUj9C3BdbYz0dDNA- > E6iXreg1M5hWiWgG6ClS86VIPI&s=Vav8_-N7_OpfYEW33yGOf_or8ESMUJ4S45t2g-EUWKg&e= > > > > We'd like to start a discussion, please post your feedback in this thread. > > > > Thank you > > Edo and Mickael > > > > > > -------------------------------------------------- > > > > Edoardo Comar > > > > IBM Event Streams > > IBM UK Ltd, Hursley Park, SO21 2JN > > > > Unless stated otherwise above: > > IBM United Kingdom Limited - Registered in England and Wales with number > > 741598. > > Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU > > > > > -- > Best, > Stanislav Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
