Re: Error in kafka streams: The producer attempted to use a producer id which is not currently assigned to its transactional id

[Matthias J. Sax]

If KafkaStreams goes into ERROR state, you cannot do anything but
restart it.

We constantly improve KafkaStreams to avoid getting into ERROR state but
it's not always possible to auto-recover.

The point being: You should try to figure out the root cause for the
issue (not easy to do unfortunately). Maybe you could change some
configs, but it could also be a bug in KafkaStreams or the broker.

There are more improvement in Apache Kafka 2.6.0 for EOS though. So
maybe it's already fixed there.


-Matthias

On 9/9/20 10:07 PM, Pushkar Deole wrote:
> Matthias,
> 
> Is there any work around after the stream goes into error because of above
> issue like attaching a StateListener on the StreamBuilder and restart the
> stream in case of ERROR state?
> Right now, we need to start the pod that hosts the application which won't
> be feasible when the application goes into production.
> 
> On Thu, Sep 10, 2020 at 2:20 AM Matthias J. Sax  wrote:
> 
>> Well, it's for sure EOS related, but it seems to be a different root cause.
>>
>> I am not aware of any related bug.
>>
>> -Matthias
>>
>>
>> On 9/9/20 4:29 AM, Pushkar Deole wrote:
>>> Hi Matthias,
>>>
>>> We are using confluent kafka and upgraded to confluent version 5.5.0
>> which
>>> I believe maps to 2.5.0 of apache kafka. We tested on few labs by keeping
>>> the solution idle for few days and didn't observe the issue.
>>>
>>> However on one of the labs we observed issue again recently, this is the
>>> exception: unfortunately, don't have complete stack trace.
>>> Anyway, do you think it is same exception as above or is it different?
>> and
>>> whether this is also a kafka server issue that is being reported already?
>>>
>>>
>> {"@timestamp":"2020-08-12T09:17:35.270+00:00","@version":"1","message":"Unexpected
>>> exception in stream
>>>
>> processing.","logger_name":"com.avaya.analytics.filter.ApplicationConfig","thread_name":"analytics-event-filter-Str
>>>
>> eamThread-1","level":"ERROR","level_value":4,"stack_trace":"org.apache.kafka.common.errors.InvalidPidMappingException:
>>> The producer attempted to use a producer id which is not currently
>> assigned
>>> to its transactiona l id.\n"}
>>>
>>> On Sat, May 9, 2020 at 12:49 AM Matthias J. Sax 
>> wrote:
>>>
>> So does this issue relate to transactions which are used only when
>> exactly_once guarantee is set?

 Correct.

 On 5/8/20 6:28 AM, Pushkar Deole wrote:
> Hello Matthias,
>
> By the way, this error seems to be occurring in only one of the
>> services.
> There is another service which is also using kafka streams to consumer
 from
> source, uses processors and then a sink to the output topic, however
>> that
> service is running fine. The difference is this other service is using
> at_least_once guarantee while the service in error is exactly once
> guarantee.
> So does this issue relate to transactions which are used only when
> exactly_once guarantee is set?
>
> On Mon, Apr 27, 2020 at 12:37 PM Pushkar Deole 
 wrote:
>
>> came across this: seems to be the one
>> https://issues.apache.org/jira/browse/KAFKA-8710
>>
>> On Mon, Apr 27, 2020 at 12:17 PM Pushkar Deole 
>> wrote:
>>
>>> Thanks... can you point to those improvements/bugs that are fixed in
 2.5?
>>>
>>> On Mon, Apr 27, 2020 at 1:03 AM Matthias J. Sax 
 wrote:
>>>
 Well, what you say is correct. However, it's a "bug" in the sense
>> that
 for some cases the producer does not need to fail, but can
 re-initialize
 itself automatically. Of course, you can also see this as an
 improvement
 and not a bug :)


 -Matthias

 On 4/25/20 7:48 AM, Pushkar Deole wrote:
> version used is 2.3
> however, not sure if this is a bug.. after doing some search, came
 across
> following for the reason of this:
>
> essentially, the transaction coordinator of streams is cleaning up
 the
> producer and transaction ids after a certain time interval
>> controller
 by
> transactional.id.expiration.ms
> <


>> https://docs.confluent.io/current/installation/configuration/broker-configs.html#transactional-id-expiration-ms
> ,
> if the coordinator doesn't receive any updates/writes from the
 producer for
> that much time. Default of this parameter is 7 days and our labs
>> have
 been
> idle for more than that.
>
> On Fri, Apr 24, 2020 at 10:46 PM Matthias J. Sax >>
 wrote:
>
>> This version are you using?
>>
>> Couple of broker and client side exactly-once related bugs got fix
 in
>> the latest release 2.5.0.
>>
>>
>> -Matthias
>>
>> On 4/23/20 11:59 PM, Pushkar Deole wrote:
>>> Hello All,
>>>

Re: Error in kafka streams: The producer attempted to use a producer id which is not currently assigned to its transactional id

[Pushkar Deole]

Matthias,

Is there any work around after the stream goes into error because of above
issue like attaching a StateListener on the StreamBuilder and restart the
stream in case of ERROR state?
Right now, we need to start the pod that hosts the application which won't
be feasible when the application goes into production.

On Thu, Sep 10, 2020 at 2:20 AM Matthias J. Sax  wrote:

> Well, it's for sure EOS related, but it seems to be a different root cause.
>
> I am not aware of any related bug.
>
> -Matthias
>
>
> On 9/9/20 4:29 AM, Pushkar Deole wrote:
> > Hi Matthias,
> >
> > We are using confluent kafka and upgraded to confluent version 5.5.0
> which
> > I believe maps to 2.5.0 of apache kafka. We tested on few labs by keeping
> > the solution idle for few days and didn't observe the issue.
> >
> > However on one of the labs we observed issue again recently, this is the
> > exception: unfortunately, don't have complete stack trace.
> > Anyway, do you think it is same exception as above or is it different?
> and
> > whether this is also a kafka server issue that is being reported already?
> >
> >
> {"@timestamp":"2020-08-12T09:17:35.270+00:00","@version":"1","message":"Unexpected
> > exception in stream
> >
> processing.","logger_name":"com.avaya.analytics.filter.ApplicationConfig","thread_name":"analytics-event-filter-Str
> >
> eamThread-1","level":"ERROR","level_value":4,"stack_trace":"org.apache.kafka.common.errors.InvalidPidMappingException:
> > The producer attempted to use a producer id which is not currently
> assigned
> > to its transactiona l id.\n"}
> >
> > On Sat, May 9, 2020 at 12:49 AM Matthias J. Sax 
> wrote:
> >
>  So does this issue relate to transactions which are used only when
>  exactly_once guarantee is set?
> >>
> >> Correct.
> >>
> >> On 5/8/20 6:28 AM, Pushkar Deole wrote:
> >>> Hello Matthias,
> >>>
> >>> By the way, this error seems to be occurring in only one of the
> services.
> >>> There is another service which is also using kafka streams to consumer
> >> from
> >>> source, uses processors and then a sink to the output topic, however
> that
> >>> service is running fine. The difference is this other service is using
> >>> at_least_once guarantee while the service in error is exactly once
> >>> guarantee.
> >>> So does this issue relate to transactions which are used only when
> >>> exactly_once guarantee is set?
> >>>
> >>> On Mon, Apr 27, 2020 at 12:37 PM Pushkar Deole 
> >> wrote:
> >>>
>  came across this: seems to be the one
>  https://issues.apache.org/jira/browse/KAFKA-8710
> 
>  On Mon, Apr 27, 2020 at 12:17 PM Pushkar Deole 
>  wrote:
> 
> > Thanks... can you point to those improvements/bugs that are fixed in
> >> 2.5?
> >
> > On Mon, Apr 27, 2020 at 1:03 AM Matthias J. Sax 
> >> wrote:
> >
> >> Well, what you say is correct. However, it's a "bug" in the sense
> that
> >> for some cases the producer does not need to fail, but can
> >> re-initialize
> >> itself automatically. Of course, you can also see this as an
> >> improvement
> >> and not a bug :)
> >>
> >>
> >> -Matthias
> >>
> >> On 4/25/20 7:48 AM, Pushkar Deole wrote:
> >>> version used is 2.3
> >>> however, not sure if this is a bug.. after doing some search, came
> >> across
> >>> following for the reason of this:
> >>>
> >>> essentially, the transaction coordinator of streams is cleaning up
> >> the
> >>> producer and transaction ids after a certain time interval
> controller
> >> by
> >>> transactional.id.expiration.ms
> >>> <
> >>
> >>
> https://docs.confluent.io/current/installation/configuration/broker-configs.html#transactional-id-expiration-ms
> >>> ,
> >>> if the coordinator doesn't receive any updates/writes from the
> >> producer for
> >>> that much time. Default of this parameter is 7 days and our labs
> have
> >> been
> >>> idle for more than that.
> >>>
> >>> On Fri, Apr 24, 2020 at 10:46 PM Matthias J. Sax  >
> >> wrote:
> >>>
>  This version are you using?
> 
>  Couple of broker and client side exactly-once related bugs got fix
> >> in
>  the latest release 2.5.0.
> 
> 
>  -Matthias
> 
>  On 4/23/20 11:59 PM, Pushkar Deole wrote:
> > Hello All,
> >
> > While using kafka streams application, we are intermittently
> >> getting
> > following exception and stream is closed. We need to restart the
> > application to get it working again and start processing. This
> >> exception
>  is
> > observed in some of the labs which are being idle for some time
> but
> >> it is
> > not observed always. Any inputs appreciated here.
> >
> >
> 
> >>
> >>
> {"@timestamp":"2020-04-15T13:53:52.698+00:00","@version":"1","message":"stream-thread
> > 

Re: Error in kafka streams: The producer attempted to use a producer id which is not currently assigned to its transactional id

[Matthias J. Sax]

Well, it's for sure EOS related, but it seems to be a different root cause.

I am not aware of any related bug.

-Matthias


On 9/9/20 4:29 AM, Pushkar Deole wrote:
> Hi Matthias,
> 
> We are using confluent kafka and upgraded to confluent version 5.5.0 which
> I believe maps to 2.5.0 of apache kafka. We tested on few labs by keeping
> the solution idle for few days and didn't observe the issue.
> 
> However on one of the labs we observed issue again recently, this is the
> exception: unfortunately, don't have complete stack trace.
> Anyway, do you think it is same exception as above or is it different? and
> whether this is also a kafka server issue that is being reported already?
> 
> {"@timestamp":"2020-08-12T09:17:35.270+00:00","@version":"1","message":"Unexpected
> exception in stream
> processing.","logger_name":"com.avaya.analytics.filter.ApplicationConfig","thread_name":"analytics-event-filter-Str
> eamThread-1","level":"ERROR","level_value":4,"stack_trace":"org.apache.kafka.common.errors.InvalidPidMappingException:
> The producer attempted to use a producer id which is not currently assigned
> to its transactiona l id.\n"}
> 
> On Sat, May 9, 2020 at 12:49 AM Matthias J. Sax  wrote:
> 
 So does this issue relate to transactions which are used only when
 exactly_once guarantee is set?
>>
>> Correct.
>>
>> On 5/8/20 6:28 AM, Pushkar Deole wrote:
>>> Hello Matthias,
>>>
>>> By the way, this error seems to be occurring in only one of the services.
>>> There is another service which is also using kafka streams to consumer
>> from
>>> source, uses processors and then a sink to the output topic, however that
>>> service is running fine. The difference is this other service is using
>>> at_least_once guarantee while the service in error is exactly once
>>> guarantee.
>>> So does this issue relate to transactions which are used only when
>>> exactly_once guarantee is set?
>>>
>>> On Mon, Apr 27, 2020 at 12:37 PM Pushkar Deole 
>> wrote:
>>>
 came across this: seems to be the one
 https://issues.apache.org/jira/browse/KAFKA-8710

 On Mon, Apr 27, 2020 at 12:17 PM Pushkar Deole 
 wrote:

> Thanks... can you point to those improvements/bugs that are fixed in
>> 2.5?
>
> On Mon, Apr 27, 2020 at 1:03 AM Matthias J. Sax 
>> wrote:
>
>> Well, what you say is correct. However, it's a "bug" in the sense that
>> for some cases the producer does not need to fail, but can
>> re-initialize
>> itself automatically. Of course, you can also see this as an
>> improvement
>> and not a bug :)
>>
>>
>> -Matthias
>>
>> On 4/25/20 7:48 AM, Pushkar Deole wrote:
>>> version used is 2.3
>>> however, not sure if this is a bug.. after doing some search, came
>> across
>>> following for the reason of this:
>>>
>>> essentially, the transaction coordinator of streams is cleaning up
>> the
>>> producer and transaction ids after a certain time interval controller
>> by
>>> transactional.id.expiration.ms
>>> <
>>
>> https://docs.confluent.io/current/installation/configuration/broker-configs.html#transactional-id-expiration-ms
>>> ,
>>> if the coordinator doesn't receive any updates/writes from the
>> producer for
>>> that much time. Default of this parameter is 7 days and our labs have
>> been
>>> idle for more than that.
>>>
>>> On Fri, Apr 24, 2020 at 10:46 PM Matthias J. Sax 
>> wrote:
>>>
 This version are you using?

 Couple of broker and client side exactly-once related bugs got fix
>> in
 the latest release 2.5.0.


 -Matthias

 On 4/23/20 11:59 PM, Pushkar Deole wrote:
> Hello All,
>
> While using kafka streams application, we are intermittently
>> getting
> following exception and stream is closed. We need to restart the
> application to get it working again and start processing. This
>> exception
 is
> observed in some of the labs which are being idle for some time but
>> it is
> not observed always. Any inputs appreciated here.
>
>

>>
>> {"@timestamp":"2020-04-15T13:53:52.698+00:00","@version":"1","message":"stream-thread
> [analytics-event-filter-StreamThread-1] Failed to commit stream
>> task
>> 2_14
> due to the following
>

>>
>> error:","logger_name":"org.apache.kafka.streams.processor.internals.AssignedStreamsTasks","thread_name":"analytics-event-filter-StreamThread-1","level":"ERROR","level_value":4,"stack_trace":"org.apache.kafka.common.KafkaException:
> Unexpected error in AddOffsetsToTxnResponse: The producer attempted
>> to
 use
> a producer id which is not currently assigned to its transactional
> id.\n\tat
>

>>
>> 

Re: Error in kafka streams: The producer attempted to use a producer id which is not currently assigned to its transactional id

[Pushkar Deole]

Hi Matthias,

We are using confluent kafka and upgraded to confluent version 5.5.0 which
I believe maps to 2.5.0 of apache kafka. We tested on few labs by keeping
the solution idle for few days and didn't observe the issue.

However on one of the labs we observed issue again recently, this is the
exception: unfortunately, don't have complete stack trace.
Anyway, do you think it is same exception as above or is it different? and
whether this is also a kafka server issue that is being reported already?

{"@timestamp":"2020-08-12T09:17:35.270+00:00","@version":"1","message":"Unexpected
exception in stream
processing.","logger_name":"com.avaya.analytics.filter.ApplicationConfig","thread_name":"analytics-event-filter-Str
eamThread-1","level":"ERROR","level_value":4,"stack_trace":"org.apache.kafka.common.errors.InvalidPidMappingException:
The producer attempted to use a producer id which is not currently assigned
to its transactiona l id.\n"}

On Sat, May 9, 2020 at 12:49 AM Matthias J. Sax  wrote:

> >> So does this issue relate to transactions which are used only when
> >> exactly_once guarantee is set?
>
> Correct.
>
> On 5/8/20 6:28 AM, Pushkar Deole wrote:
> > Hello Matthias,
> >
> > By the way, this error seems to be occurring in only one of the services.
> > There is another service which is also using kafka streams to consumer
> from
> > source, uses processors and then a sink to the output topic, however that
> > service is running fine. The difference is this other service is using
> > at_least_once guarantee while the service in error is exactly once
> > guarantee.
> > So does this issue relate to transactions which are used only when
> > exactly_once guarantee is set?
> >
> > On Mon, Apr 27, 2020 at 12:37 PM Pushkar Deole 
> wrote:
> >
> >> came across this: seems to be the one
> >> https://issues.apache.org/jira/browse/KAFKA-8710
> >>
> >> On Mon, Apr 27, 2020 at 12:17 PM Pushkar Deole 
> >> wrote:
> >>
> >>> Thanks... can you point to those improvements/bugs that are fixed in
> 2.5?
> >>>
> >>> On Mon, Apr 27, 2020 at 1:03 AM Matthias J. Sax 
> wrote:
> >>>
>  Well, what you say is correct. However, it's a "bug" in the sense that
>  for some cases the producer does not need to fail, but can
> re-initialize
>  itself automatically. Of course, you can also see this as an
> improvement
>  and not a bug :)
> 
> 
>  -Matthias
> 
>  On 4/25/20 7:48 AM, Pushkar Deole wrote:
> > version used is 2.3
> > however, not sure if this is a bug.. after doing some search, came
>  across
> > following for the reason of this:
> >
> > essentially, the transaction coordinator of streams is cleaning up
> the
> > producer and transaction ids after a certain time interval controller
>  by
> > transactional.id.expiration.ms
> > <
> 
> https://docs.confluent.io/current/installation/configuration/broker-configs.html#transactional-id-expiration-ms
> > ,
> > if the coordinator doesn't receive any updates/writes from the
>  producer for
> > that much time. Default of this parameter is 7 days and our labs have
>  been
> > idle for more than that.
> >
> > On Fri, Apr 24, 2020 at 10:46 PM Matthias J. Sax 
>  wrote:
> >
> >> This version are you using?
> >>
> >> Couple of broker and client side exactly-once related bugs got fix
> in
> >> the latest release 2.5.0.
> >>
> >>
> >> -Matthias
> >>
> >> On 4/23/20 11:59 PM, Pushkar Deole wrote:
> >>> Hello All,
> >>>
> >>> While using kafka streams application, we are intermittently
> getting
> >>> following exception and stream is closed. We need to restart the
> >>> application to get it working again and start processing. This
>  exception
> >> is
> >>> observed in some of the labs which are being idle for some time but
>  it is
> >>> not observed always. Any inputs appreciated here.
> >>>
> >>>
> >>
> 
> {"@timestamp":"2020-04-15T13:53:52.698+00:00","@version":"1","message":"stream-thread
> >>> [analytics-event-filter-StreamThread-1] Failed to commit stream
> task
>  2_14
> >>> due to the following
> >>>
> >>
> 
> error:","logger_name":"org.apache.kafka.streams.processor.internals.AssignedStreamsTasks","thread_name":"analytics-event-filter-StreamThread-1","level":"ERROR","level_value":4,"stack_trace":"org.apache.kafka.common.KafkaException:
> >>> Unexpected error in AddOffsetsToTxnResponse: The producer attempted
>  to
> >> use
> >>> a producer id which is not currently assigned to its transactional
> >>> id.\n\tat
> >>>
> >>
> 
> org.apache.kafka.clients.producer.internals.TransactionManager$AddOffsetsToTxnHandler.handleResponse(TransactionManager.java:1406)\n\tat
> >>>
> >>
> 
> 

Re: Error in kafka streams: The producer attempted to use a producer id which is not currently assigned to its transactional id

[Matthias J. Sax]

>> So does this issue relate to transactions which are used only when
>> exactly_once guarantee is set?

Correct.

On 5/8/20 6:28 AM, Pushkar Deole wrote:
> Hello Matthias,
> 
> By the way, this error seems to be occurring in only one of the services.
> There is another service which is also using kafka streams to consumer from
> source, uses processors and then a sink to the output topic, however that
> service is running fine. The difference is this other service is using
> at_least_once guarantee while the service in error is exactly once
> guarantee.
> So does this issue relate to transactions which are used only when
> exactly_once guarantee is set?
> 
> On Mon, Apr 27, 2020 at 12:37 PM Pushkar Deole  wrote:
> 
>> came across this: seems to be the one
>> https://issues.apache.org/jira/browse/KAFKA-8710
>>
>> On Mon, Apr 27, 2020 at 12:17 PM Pushkar Deole 
>> wrote:
>>
>>> Thanks... can you point to those improvements/bugs that are fixed in 2.5?
>>>
>>> On Mon, Apr 27, 2020 at 1:03 AM Matthias J. Sax  wrote:
>>>
 Well, what you say is correct. However, it's a "bug" in the sense that
 for some cases the producer does not need to fail, but can re-initialize
 itself automatically. Of course, you can also see this as an improvement
 and not a bug :)


 -Matthias

 On 4/25/20 7:48 AM, Pushkar Deole wrote:
> version used is 2.3
> however, not sure if this is a bug.. after doing some search, came
 across
> following for the reason of this:
>
> essentially, the transaction coordinator of streams is cleaning up the
> producer and transaction ids after a certain time interval controller
 by
> transactional.id.expiration.ms
> <
 https://docs.confluent.io/current/installation/configuration/broker-configs.html#transactional-id-expiration-ms
> ,
> if the coordinator doesn't receive any updates/writes from the
 producer for
> that much time. Default of this parameter is 7 days and our labs have
 been
> idle for more than that.
>
> On Fri, Apr 24, 2020 at 10:46 PM Matthias J. Sax 
 wrote:
>
>> This version are you using?
>>
>> Couple of broker and client side exactly-once related bugs got fix in
>> the latest release 2.5.0.
>>
>>
>> -Matthias
>>
>> On 4/23/20 11:59 PM, Pushkar Deole wrote:
>>> Hello All,
>>>
>>> While using kafka streams application, we are intermittently getting
>>> following exception and stream is closed. We need to restart the
>>> application to get it working again and start processing. This
 exception
>> is
>>> observed in some of the labs which are being idle for some time but
 it is
>>> not observed always. Any inputs appreciated here.
>>>
>>>
>>
 {"@timestamp":"2020-04-15T13:53:52.698+00:00","@version":"1","message":"stream-thread
>>> [analytics-event-filter-StreamThread-1] Failed to commit stream task
 2_14
>>> due to the following
>>>
>>
 error:","logger_name":"org.apache.kafka.streams.processor.internals.AssignedStreamsTasks","thread_name":"analytics-event-filter-StreamThread-1","level":"ERROR","level_value":4,"stack_trace":"org.apache.kafka.common.KafkaException:
>>> Unexpected error in AddOffsetsToTxnResponse: The producer attempted
 to
>> use
>>> a producer id which is not currently assigned to its transactional
>>> id.\n\tat
>>>
>>
 org.apache.kafka.clients.producer.internals.TransactionManager$AddOffsetsToTxnHandler.handleResponse(TransactionManager.java:1406)\n\tat
>>>
>>
 org.apache.kafka.clients.producer.internals.TransactionManager$TxnRequestHandler.onComplete(TransactionManager.java:1069)\n\tat
>>>
>>
 org.apache.kafka.clients.ClientResponse.onComplete(ClientResponse.java:109)\n\tat
>>>
>>
 org.apache.kafka.clients.NetworkClient.completeResponses(NetworkClient.java:561)\n\tat
>>>
 org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:553)\n\tat
>>>
>>
 org.apache.kafka.clients.producer.internals.Sender.maybeSendAndPollTransactionalRequest(Sender.java:425)\n\tat
>>>
>>
 org.apache.kafka.clients.producer.internals.Sender.runOnce(Sender.java:311)\n\tat
>>>
>>
 org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:244)\n\tat
>>> java.base/java.lang.Thread.run(Unknown Source)\n"}
>>>
>>
>>
>


> 



signature.asc
Description: OpenPGP digital signature

Re: Error in kafka streams: The producer attempted to use a producer id which is not currently assigned to its transactional id

[Pushkar Deole]

Hello Matthias,

By the way, this error seems to be occurring in only one of the services.
There is another service which is also using kafka streams to consumer from
source, uses processors and then a sink to the output topic, however that
service is running fine. The difference is this other service is using
at_least_once guarantee while the service in error is exactly once
guarantee.
So does this issue relate to transactions which are used only when
exactly_once guarantee is set?

On Mon, Apr 27, 2020 at 12:37 PM Pushkar Deole  wrote:

> came across this: seems to be the one
> https://issues.apache.org/jira/browse/KAFKA-8710
>
> On Mon, Apr 27, 2020 at 12:17 PM Pushkar Deole 
> wrote:
>
>> Thanks... can you point to those improvements/bugs that are fixed in 2.5?
>>
>> On Mon, Apr 27, 2020 at 1:03 AM Matthias J. Sax  wrote:
>>
>>> Well, what you say is correct. However, it's a "bug" in the sense that
>>> for some cases the producer does not need to fail, but can re-initialize
>>> itself automatically. Of course, you can also see this as an improvement
>>> and not a bug :)
>>>
>>>
>>> -Matthias
>>>
>>> On 4/25/20 7:48 AM, Pushkar Deole wrote:
>>> > version used is 2.3
>>> > however, not sure if this is a bug.. after doing some search, came
>>> across
>>> > following for the reason of this:
>>> >
>>> > essentially, the transaction coordinator of streams is cleaning up the
>>> > producer and transaction ids after a certain time interval controller
>>> by
>>> > transactional.id.expiration.ms
>>> > <
>>> https://docs.confluent.io/current/installation/configuration/broker-configs.html#transactional-id-expiration-ms
>>> >,
>>> > if the coordinator doesn't receive any updates/writes from the
>>> producer for
>>> > that much time. Default of this parameter is 7 days and our labs have
>>> been
>>> > idle for more than that.
>>> >
>>> > On Fri, Apr 24, 2020 at 10:46 PM Matthias J. Sax 
>>> wrote:
>>> >
>>> >> This version are you using?
>>> >>
>>> >> Couple of broker and client side exactly-once related bugs got fix in
>>> >> the latest release 2.5.0.
>>> >>
>>> >>
>>> >> -Matthias
>>> >>
>>> >> On 4/23/20 11:59 PM, Pushkar Deole wrote:
>>> >>> Hello All,
>>> >>>
>>> >>> While using kafka streams application, we are intermittently getting
>>> >>> following exception and stream is closed. We need to restart the
>>> >>> application to get it working again and start processing. This
>>> exception
>>> >> is
>>> >>> observed in some of the labs which are being idle for some time but
>>> it is
>>> >>> not observed always. Any inputs appreciated here.
>>> >>>
>>> >>>
>>> >>
>>> {"@timestamp":"2020-04-15T13:53:52.698+00:00","@version":"1","message":"stream-thread
>>> >>> [analytics-event-filter-StreamThread-1] Failed to commit stream task
>>> 2_14
>>> >>> due to the following
>>> >>>
>>> >>
>>> error:","logger_name":"org.apache.kafka.streams.processor.internals.AssignedStreamsTasks","thread_name":"analytics-event-filter-StreamThread-1","level":"ERROR","level_value":4,"stack_trace":"org.apache.kafka.common.KafkaException:
>>> >>> Unexpected error in AddOffsetsToTxnResponse: The producer attempted
>>> to
>>> >> use
>>> >>> a producer id which is not currently assigned to its transactional
>>> >>> id.\n\tat
>>> >>>
>>> >>
>>> org.apache.kafka.clients.producer.internals.TransactionManager$AddOffsetsToTxnHandler.handleResponse(TransactionManager.java:1406)\n\tat
>>> >>>
>>> >>
>>> org.apache.kafka.clients.producer.internals.TransactionManager$TxnRequestHandler.onComplete(TransactionManager.java:1069)\n\tat
>>> >>>
>>> >>
>>> org.apache.kafka.clients.ClientResponse.onComplete(ClientResponse.java:109)\n\tat
>>> >>>
>>> >>
>>> org.apache.kafka.clients.NetworkClient.completeResponses(NetworkClient.java:561)\n\tat
>>> >>>
>>> org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:553)\n\tat
>>> >>>
>>> >>
>>> org.apache.kafka.clients.producer.internals.Sender.maybeSendAndPollTransactionalRequest(Sender.java:425)\n\tat
>>> >>>
>>> >>
>>> org.apache.kafka.clients.producer.internals.Sender.runOnce(Sender.java:311)\n\tat
>>> >>>
>>> >>
>>> org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:244)\n\tat
>>> >>> java.base/java.lang.Thread.run(Unknown Source)\n"}
>>> >>>
>>> >>
>>> >>
>>> >
>>>
>>>

Re: Error in kafka streams: The producer attempted to use a producer id which is not currently assigned to its transactional id

[Pushkar Deole]

came across this: seems to be the one
https://issues.apache.org/jira/browse/KAFKA-8710

On Mon, Apr 27, 2020 at 12:17 PM Pushkar Deole  wrote:

> Thanks... can you point to those improvements/bugs that are fixed in 2.5?
>
> On Mon, Apr 27, 2020 at 1:03 AM Matthias J. Sax  wrote:
>
>> Well, what you say is correct. However, it's a "bug" in the sense that
>> for some cases the producer does not need to fail, but can re-initialize
>> itself automatically. Of course, you can also see this as an improvement
>> and not a bug :)
>>
>>
>> -Matthias
>>
>> On 4/25/20 7:48 AM, Pushkar Deole wrote:
>> > version used is 2.3
>> > however, not sure if this is a bug.. after doing some search, came
>> across
>> > following for the reason of this:
>> >
>> > essentially, the transaction coordinator of streams is cleaning up the
>> > producer and transaction ids after a certain time interval controller by
>> > transactional.id.expiration.ms
>> > <
>> https://docs.confluent.io/current/installation/configuration/broker-configs.html#transactional-id-expiration-ms
>> >,
>> > if the coordinator doesn't receive any updates/writes from the producer
>> for
>> > that much time. Default of this parameter is 7 days and our labs have
>> been
>> > idle for more than that.
>> >
>> > On Fri, Apr 24, 2020 at 10:46 PM Matthias J. Sax 
>> wrote:
>> >
>> >> This version are you using?
>> >>
>> >> Couple of broker and client side exactly-once related bugs got fix in
>> >> the latest release 2.5.0.
>> >>
>> >>
>> >> -Matthias
>> >>
>> >> On 4/23/20 11:59 PM, Pushkar Deole wrote:
>> >>> Hello All,
>> >>>
>> >>> While using kafka streams application, we are intermittently getting
>> >>> following exception and stream is closed. We need to restart the
>> >>> application to get it working again and start processing. This
>> exception
>> >> is
>> >>> observed in some of the labs which are being idle for some time but
>> it is
>> >>> not observed always. Any inputs appreciated here.
>> >>>
>> >>>
>> >>
>> {"@timestamp":"2020-04-15T13:53:52.698+00:00","@version":"1","message":"stream-thread
>> >>> [analytics-event-filter-StreamThread-1] Failed to commit stream task
>> 2_14
>> >>> due to the following
>> >>>
>> >>
>> error:","logger_name":"org.apache.kafka.streams.processor.internals.AssignedStreamsTasks","thread_name":"analytics-event-filter-StreamThread-1","level":"ERROR","level_value":4,"stack_trace":"org.apache.kafka.common.KafkaException:
>> >>> Unexpected error in AddOffsetsToTxnResponse: The producer attempted to
>> >> use
>> >>> a producer id which is not currently assigned to its transactional
>> >>> id.\n\tat
>> >>>
>> >>
>> org.apache.kafka.clients.producer.internals.TransactionManager$AddOffsetsToTxnHandler.handleResponse(TransactionManager.java:1406)\n\tat
>> >>>
>> >>
>> org.apache.kafka.clients.producer.internals.TransactionManager$TxnRequestHandler.onComplete(TransactionManager.java:1069)\n\tat
>> >>>
>> >>
>> org.apache.kafka.clients.ClientResponse.onComplete(ClientResponse.java:109)\n\tat
>> >>>
>> >>
>> org.apache.kafka.clients.NetworkClient.completeResponses(NetworkClient.java:561)\n\tat
>> >>>
>> org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:553)\n\tat
>> >>>
>> >>
>> org.apache.kafka.clients.producer.internals.Sender.maybeSendAndPollTransactionalRequest(Sender.java:425)\n\tat
>> >>>
>> >>
>> org.apache.kafka.clients.producer.internals.Sender.runOnce(Sender.java:311)\n\tat
>> >>>
>> >>
>> org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:244)\n\tat
>> >>> java.base/java.lang.Thread.run(Unknown Source)\n"}
>> >>>
>> >>
>> >>
>> >
>>
>>

Re: Error in kafka streams: The producer attempted to use a producer id which is not currently assigned to its transactional id

[Pushkar Deole]

Thanks... can you point to those improvements/bugs that are fixed in 2.5?

On Mon, Apr 27, 2020 at 1:03 AM Matthias J. Sax  wrote:

> Well, what you say is correct. However, it's a "bug" in the sense that
> for some cases the producer does not need to fail, but can re-initialize
> itself automatically. Of course, you can also see this as an improvement
> and not a bug :)
>
>
> -Matthias
>
> On 4/25/20 7:48 AM, Pushkar Deole wrote:
> > version used is 2.3
> > however, not sure if this is a bug.. after doing some search, came across
> > following for the reason of this:
> >
> > essentially, the transaction coordinator of streams is cleaning up the
> > producer and transaction ids after a certain time interval controller by
> > transactional.id.expiration.ms
> > <
> https://docs.confluent.io/current/installation/configuration/broker-configs.html#transactional-id-expiration-ms
> >,
> > if the coordinator doesn't receive any updates/writes from the producer
> for
> > that much time. Default of this parameter is 7 days and our labs have
> been
> > idle for more than that.
> >
> > On Fri, Apr 24, 2020 at 10:46 PM Matthias J. Sax 
> wrote:
> >
> >> This version are you using?
> >>
> >> Couple of broker and client side exactly-once related bugs got fix in
> >> the latest release 2.5.0.
> >>
> >>
> >> -Matthias
> >>
> >> On 4/23/20 11:59 PM, Pushkar Deole wrote:
> >>> Hello All,
> >>>
> >>> While using kafka streams application, we are intermittently getting
> >>> following exception and stream is closed. We need to restart the
> >>> application to get it working again and start processing. This
> exception
> >> is
> >>> observed in some of the labs which are being idle for some time but it
> is
> >>> not observed always. Any inputs appreciated here.
> >>>
> >>>
> >>
> {"@timestamp":"2020-04-15T13:53:52.698+00:00","@version":"1","message":"stream-thread
> >>> [analytics-event-filter-StreamThread-1] Failed to commit stream task
> 2_14
> >>> due to the following
> >>>
> >>
> error:","logger_name":"org.apache.kafka.streams.processor.internals.AssignedStreamsTasks","thread_name":"analytics-event-filter-StreamThread-1","level":"ERROR","level_value":4,"stack_trace":"org.apache.kafka.common.KafkaException:
> >>> Unexpected error in AddOffsetsToTxnResponse: The producer attempted to
> >> use
> >>> a producer id which is not currently assigned to its transactional
> >>> id.\n\tat
> >>>
> >>
> org.apache.kafka.clients.producer.internals.TransactionManager$AddOffsetsToTxnHandler.handleResponse(TransactionManager.java:1406)\n\tat
> >>>
> >>
> org.apache.kafka.clients.producer.internals.TransactionManager$TxnRequestHandler.onComplete(TransactionManager.java:1069)\n\tat
> >>>
> >>
> org.apache.kafka.clients.ClientResponse.onComplete(ClientResponse.java:109)\n\tat
> >>>
> >>
> org.apache.kafka.clients.NetworkClient.completeResponses(NetworkClient.java:561)\n\tat
> >>>
> org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:553)\n\tat
> >>>
> >>
> org.apache.kafka.clients.producer.internals.Sender.maybeSendAndPollTransactionalRequest(Sender.java:425)\n\tat
> >>>
> >>
> org.apache.kafka.clients.producer.internals.Sender.runOnce(Sender.java:311)\n\tat
> >>>
> >>
> org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:244)\n\tat
> >>> java.base/java.lang.Thread.run(Unknown Source)\n"}
> >>>
> >>
> >>
> >
>
>

Re: Error in kafka streams: The producer attempted to use a producer id which is not currently assigned to its transactional id

[Matthias J. Sax]

Well, what you say is correct. However, it's a "bug" in the sense that
for some cases the producer does not need to fail, but can re-initialize
itself automatically. Of course, you can also see this as an improvement
and not a bug :)


-Matthias

On 4/25/20 7:48 AM, Pushkar Deole wrote:
> version used is 2.3
> however, not sure if this is a bug.. after doing some search, came across
> following for the reason of this:
> 
> essentially, the transaction coordinator of streams is cleaning up the
> producer and transaction ids after a certain time interval controller by
> transactional.id.expiration.ms
> ,
> if the coordinator doesn't receive any updates/writes from the producer for
> that much time. Default of this parameter is 7 days and our labs have been
> idle for more than that.
> 
> On Fri, Apr 24, 2020 at 10:46 PM Matthias J. Sax  wrote:
> 
>> This version are you using?
>>
>> Couple of broker and client side exactly-once related bugs got fix in
>> the latest release 2.5.0.
>>
>>
>> -Matthias
>>
>> On 4/23/20 11:59 PM, Pushkar Deole wrote:
>>> Hello All,
>>>
>>> While using kafka streams application, we are intermittently getting
>>> following exception and stream is closed. We need to restart the
>>> application to get it working again and start processing. This exception
>> is
>>> observed in some of the labs which are being idle for some time but it is
>>> not observed always. Any inputs appreciated here.
>>>
>>>
>> {"@timestamp":"2020-04-15T13:53:52.698+00:00","@version":"1","message":"stream-thread
>>> [analytics-event-filter-StreamThread-1] Failed to commit stream task 2_14
>>> due to the following
>>>
>> error:","logger_name":"org.apache.kafka.streams.processor.internals.AssignedStreamsTasks","thread_name":"analytics-event-filter-StreamThread-1","level":"ERROR","level_value":4,"stack_trace":"org.apache.kafka.common.KafkaException:
>>> Unexpected error in AddOffsetsToTxnResponse: The producer attempted to
>> use
>>> a producer id which is not currently assigned to its transactional
>>> id.\n\tat
>>>
>> org.apache.kafka.clients.producer.internals.TransactionManager$AddOffsetsToTxnHandler.handleResponse(TransactionManager.java:1406)\n\tat
>>>
>> org.apache.kafka.clients.producer.internals.TransactionManager$TxnRequestHandler.onComplete(TransactionManager.java:1069)\n\tat
>>>
>> org.apache.kafka.clients.ClientResponse.onComplete(ClientResponse.java:109)\n\tat
>>>
>> org.apache.kafka.clients.NetworkClient.completeResponses(NetworkClient.java:561)\n\tat
>>> org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:553)\n\tat
>>>
>> org.apache.kafka.clients.producer.internals.Sender.maybeSendAndPollTransactionalRequest(Sender.java:425)\n\tat
>>>
>> org.apache.kafka.clients.producer.internals.Sender.runOnce(Sender.java:311)\n\tat
>>>
>> org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:244)\n\tat
>>> java.base/java.lang.Thread.run(Unknown Source)\n"}
>>>
>>
>>
> 



signature.asc
Description: OpenPGP digital signature

Re: Error in kafka streams: The producer attempted to use a producer id which is not currently assigned to its transactional id

[Pushkar Deole]

version used is 2.3
however, not sure if this is a bug.. after doing some search, came across
following for the reason of this:

essentially, the transaction coordinator of streams is cleaning up the
producer and transaction ids after a certain time interval controller by
transactional.id.expiration.ms
,
if the coordinator doesn't receive any updates/writes from the producer for
that much time. Default of this parameter is 7 days and our labs have been
idle for more than that.

On Fri, Apr 24, 2020 at 10:46 PM Matthias J. Sax  wrote:

> This version are you using?
>
> Couple of broker and client side exactly-once related bugs got fix in
> the latest release 2.5.0.
>
>
> -Matthias
>
> On 4/23/20 11:59 PM, Pushkar Deole wrote:
> > Hello All,
> >
> > While using kafka streams application, we are intermittently getting
> > following exception and stream is closed. We need to restart the
> > application to get it working again and start processing. This exception
> is
> > observed in some of the labs which are being idle for some time but it is
> > not observed always. Any inputs appreciated here.
> >
> >
> {"@timestamp":"2020-04-15T13:53:52.698+00:00","@version":"1","message":"stream-thread
> > [analytics-event-filter-StreamThread-1] Failed to commit stream task 2_14
> > due to the following
> >
> error:","logger_name":"org.apache.kafka.streams.processor.internals.AssignedStreamsTasks","thread_name":"analytics-event-filter-StreamThread-1","level":"ERROR","level_value":4,"stack_trace":"org.apache.kafka.common.KafkaException:
> > Unexpected error in AddOffsetsToTxnResponse: The producer attempted to
> use
> > a producer id which is not currently assigned to its transactional
> > id.\n\tat
> >
> org.apache.kafka.clients.producer.internals.TransactionManager$AddOffsetsToTxnHandler.handleResponse(TransactionManager.java:1406)\n\tat
> >
> org.apache.kafka.clients.producer.internals.TransactionManager$TxnRequestHandler.onComplete(TransactionManager.java:1069)\n\tat
> >
> org.apache.kafka.clients.ClientResponse.onComplete(ClientResponse.java:109)\n\tat
> >
> org.apache.kafka.clients.NetworkClient.completeResponses(NetworkClient.java:561)\n\tat
> > org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:553)\n\tat
> >
> org.apache.kafka.clients.producer.internals.Sender.maybeSendAndPollTransactionalRequest(Sender.java:425)\n\tat
> >
> org.apache.kafka.clients.producer.internals.Sender.runOnce(Sender.java:311)\n\tat
> >
> org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:244)\n\tat
> > java.base/java.lang.Thread.run(Unknown Source)\n"}
> >
>
>

Re: Error in kafka streams: The producer attempted to use a producer id which is not currently assigned to its transactional id

[Matthias J. Sax]

This version are you using?

Couple of broker and client side exactly-once related bugs got fix in
the latest release 2.5.0.


-Matthias

On 4/23/20 11:59 PM, Pushkar Deole wrote:
> Hello All,
> 
> While using kafka streams application, we are intermittently getting
> following exception and stream is closed. We need to restart the
> application to get it working again and start processing. This exception is
> observed in some of the labs which are being idle for some time but it is
> not observed always. Any inputs appreciated here.
> 
> {"@timestamp":"2020-04-15T13:53:52.698+00:00","@version":"1","message":"stream-thread
> [analytics-event-filter-StreamThread-1] Failed to commit stream task 2_14
> due to the following
> error:","logger_name":"org.apache.kafka.streams.processor.internals.AssignedStreamsTasks","thread_name":"analytics-event-filter-StreamThread-1","level":"ERROR","level_value":4,"stack_trace":"org.apache.kafka.common.KafkaException:
> Unexpected error in AddOffsetsToTxnResponse: The producer attempted to use
> a producer id which is not currently assigned to its transactional
> id.\n\tat
> org.apache.kafka.clients.producer.internals.TransactionManager$AddOffsetsToTxnHandler.handleResponse(TransactionManager.java:1406)\n\tat
> org.apache.kafka.clients.producer.internals.TransactionManager$TxnRequestHandler.onComplete(TransactionManager.java:1069)\n\tat
> org.apache.kafka.clients.ClientResponse.onComplete(ClientResponse.java:109)\n\tat
> org.apache.kafka.clients.NetworkClient.completeResponses(NetworkClient.java:561)\n\tat
> org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:553)\n\tat
> org.apache.kafka.clients.producer.internals.Sender.maybeSendAndPollTransactionalRequest(Sender.java:425)\n\tat
> org.apache.kafka.clients.producer.internals.Sender.runOnce(Sender.java:311)\n\tat
> org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:244)\n\tat
> java.base/java.lang.Thread.run(Unknown Source)\n"}
> 



signature.asc
Description: OpenPGP digital signature