Re: [DISCUSS] KIP-588: Allow producers to recover gracefully from transaction timeouts

2020-04-13 Thread Boyang Chen
Thanks Guozhang! Will wait and see if there are more comments. On Fri, Apr 10, 2020 at 5:17 PM Guozhang Wang wrote: > Thanks Boyang, the newly added example looks good to me. > > On Thu, Apr 9, 2020 at 2:47 PM Boyang Chen > wrote: > > > Hey Guozhang, > > > > I have added an example of the

Re: [DISCUSS] KIP-588: Allow producers to recover gracefully from transaction timeouts

2020-04-10 Thread Guozhang Wang
Thanks Boyang, the newly added example looks good to me. On Thu, Apr 9, 2020 at 2:47 PM Boyang Chen wrote: > Hey Guozhang, > > I have added an example of the producer API usage under new improvements. > Let me know if this looks good to you. > > Boyang > > On Wed, Apr 8, 2020 at 1:38 PM Boyang

Re: [DISCUSS] KIP-588: Allow producers to recover gracefully from transaction timeouts

2020-04-09 Thread Boyang Chen
Hey Guozhang, I have added an example of the producer API usage under new improvements. Let me know if this looks good to you. Boyang On Wed, Apr 8, 2020 at 1:38 PM Boyang Chen wrote: > That's a good suggestion Jason. Adding a dedicated PRODUCER_FENCED error > should help distinguish

Re: [DISCUSS] KIP-588: Allow producers to recover gracefully from transaction timeouts

2020-04-08 Thread Boyang Chen
That's a good suggestion Jason. Adding a dedicated PRODUCER_FENCED error should help distinguish exceptions and could safely mark INVALID_PRODUCER_EPOCH exception as non-fatal in the new code. Updated the KIP. Boyang On Wed, Apr 8, 2020 at 12:18 PM Jason Gustafson wrote: > Hey Boyang, > >

Re: [DISCUSS] KIP-588: Allow producers to recover gracefully from transaction timeouts

2020-04-08 Thread Jason Gustafson
Hey Boyang, Thanks for the KIP. I think the main problem we've identified here is that the current errors conflate transaction timeouts with producer fencing. The first of these ought to be recoverable, but we cannot distinguish it. The suggestion to add a new error code makes sense to me, but it

Re: [DISCUSS] KIP-588: Allow producers to recover gracefully from transaction timeouts

2020-04-06 Thread Boyang Chen
Yep, updated the KIP, thanks! On Mon, Apr 6, 2020 at 3:11 PM Guozhang Wang wrote: > Regarding 2), sounds good, I saw UNKNOWN_PRODUCER_ID is properly handled > today in produce / add-partitions-to-txn / add-offsets-to-txn / end-txn > responses, so that should be well covered. > > Could you

Re: [DISCUSS] KIP-588: Allow producers to recover gracefully from transaction timeouts

2020-04-06 Thread Guozhang Wang
Regarding 2), sounds good, I saw UNKNOWN_PRODUCER_ID is properly handled today in produce / add-partitions-to-txn / add-offsets-to-txn / end-txn responses, so that should be well covered. Could you reflect this in the wiki page that the broker should be responsible for using different error codes

Re: [DISCUSS] KIP-588: Allow producers to recover gracefully from transaction timeouts

2020-04-06 Thread Boyang Chen
Thanks Guozhang for the review! On Sun, Apr 5, 2020 at 5:47 PM Guozhang Wang wrote: > Hello Boyang, > > Thank you for the proposed KIP. Just some minor comments below: > > 1. Could you also describe which producer APIs could potentially throw the > new TransactionTimedOutException, and also how

Re: [DISCUSS] KIP-588: Allow producers to recover gracefully from transaction timeouts

2020-04-05 Thread Guozhang Wang
Hello Boyang, Thank you for the proposed KIP. Just some minor comments below: 1. Could you also describe which producer APIs could potentially throw the new TransactionTimedOutException, and also how should callers handle them differently (i.e. just to make your description more concrete as

[DISCUSS] KIP-588: Allow producers to recover gracefully from transaction timeouts

2020-04-02 Thread Boyang Chen
Hey there, I would like to start discussion for KIP-588: https://cwiki.apache.org/confluence/display/KAFKA/KIP-588%3A+Allow+producers+to+recover+gracefully+from+transaction+timeouts which aims to improve Producer resilience to transaction timeout due to transient system gaps. Best, Boyang