[
https://issues.apache.org/jira/browse/KAFKA-6990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Eugen Feller updated KAFKA-6990:
--------------------------------
Description:
We are seeing a lot of CommitFailedExceptions on one of our Kafka stream apps.
Running Kafka Streams 0.10.2.1 and Kafka Broker 0.10.2.1. Full error message:
{code:java}
[visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] INFO
org.apache.kafka.streams.KafkaStreams - stream-client
[visits-bdb209f1-c9dd-404b-9233-064f2fb4227d] State transition from REBALANCING
to RUNNING.
[visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] WARN
org.apache.kafka.streams.processor.internals.StreamTask - task [0_0] Failed
offset commits {sightings_sighting_byclientmac_0-0=OffsetAndMetadata{offset=11,
metadata=''},
mactocontactmappings_contactmapping_byclientmac_0-0=OffsetAndMetadata{offset=26,
metadata=''}} due to CommitFailedException
[visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] WARN
org.apache.kafka.streams.processor.internals.AssignedTasks - stream-thread
[visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] Failed to commit
stream task 0_0 during commit state due to CommitFailedException; this task may
be no longer owned by the thread
[visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] WARN
org.apache.kafka.streams.processor.internals.AssignedTasks - stream-thread
[visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] Failed to commit
stream task 0_1 during commit state due to CommitFailedException; this task may
be no longer owned by the thread
[visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] WARN
org.apache.kafka.streams.processor.internals.StreamTask - task [0_1] Failed
offset commits
{mactocontactmappings_contactmapping_byclientmac_0-1=OffsetAndMetadata{offset=38,
metadata=''}, sightings_sighting_byclientmac_0-1=OffsetAndMetadata{offset=1,
metadata=''}} due to CommitFailedException
[visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] WARN
org.apache.kafka.streams.processor.internals.StreamTask - task [0_2] Failed
offset commits
{mactocontactmappings_contactmapping_byclientmac_0-2=OffsetAndMetadata{offset=8,
metadata=''}, sightings_sighting_byclientmac_0-2=OffsetAndMetadata{offset=24,
metadata=''}} due to CommitFailedException
[visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] WARN
org.apache.kafka.streams.processor.internals.AssignedTasks - stream-thread
[visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] Failed to commit
stream task 0_2 during commit state due to CommitFailedException; this task may
be no longer owned by the thread
[visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] WARN
org.apache.kafka.streams.processor.internals.StreamTask - task [0_3] Failed
offset commits {sightings_sighting_byclientmac_0-3=OffsetAndMetadata{offset=21,
metadata=''},
mactocontactmappings_contactmapping_byclientmac_0-3=OffsetAndMetadata{offset=102,
metadata=''}} due to CommitFailedException
[visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] WARN
org.apache.kafka.streams.processor.internals.AssignedTasks - stream-thread
[visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] Failed to commit
stream task 0_3 during commit state due to CommitFailedException; this task may
be no longer owned by the thread
[visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] WARN
org.apache.kafka.streams.processor.internals.StreamTask - task [0_4] Failed
offset commits {sightings_sighting_byclientmac_0-4=OffsetAndMetadata{offset=5,
metadata=''},
mactocontactmappings_contactmapping_byclientmac_0-4=OffsetAndMetadata{offset=20,
metadata=''}} due to CommitFailedException
[visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] WARN
org.apache.kafka.streams.processor.internals.AssignedTasks - stream-thread
[visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] Failed to commit
stream task 0_4 during commit state due to CommitFailedException; this task may
be no longer owned by the thread
[visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] WARN
org.apache.kafka.streams.processor.internals.StreamTask - task [0_5] Failed
offset commits
{mactocontactmappings_contactmapping_byclientmac_0-5=OffsetAndMetadata{offset=26,
metadata=''}, sightings_sighting_byclientmac_0-5=OffsetAndMetadata{offset=16,
metadata=''}} due to CommitFailedException
[visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] WARN
org.apache.kafka.streams.processor.internals.AssignedTasks - stream-thread
[visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] Failed to commit
stream task 0_5 during commit state due to CommitFailedException; this task may
be no longer owned by the thread
[visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] WARN
org.apache.kafka.streams.processor.internals.StreamTask - task [0_6] Failed
offset commits
{mactocontactmappings_contactmapping_byclientmac_0-6=OffsetAndMetadata{offset=22,
metadata=''}, sightings_sighting_byclientmac_0-6=OffsetAndMetadata{offset=3,
metadata=''}} due to CommitFailedException
[visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] WARN
org.apache.kafka.streams.processor.internals.AssignedTasks - stream-thread
[visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] Failed to commit
stream task 0_6 during commit state due to CommitFailedException; this task may
be no longer owned by the thread
[visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] WARN
org.apache.kafka.streams.processor.internals.StreamTask - task [0_7] Failed
offset commits {sightings_sighting_byclientmac_0-7=OffsetAndMetadata{offset=3,
metadata=''},
mactocontactmappings_contactmapping_byclientmac_0-7=OffsetAndMetadata{offset=5,
metadata=''}} due to CommitFailedException
[visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] WARN
org.apache.kafka.streams.processor.internals.AssignedTasks - stream-thread
[visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] Failed to commit
stream task 0_7 during commit state due to CommitFailedException; this task may
be no longer owned by the thread
[visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] WARN
org.apache.kafka.streams.processor.internals.AssignedTasks - stream-thread
[visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] Failed to commit
stream task 0_8 during commit state due to CommitFailedException; this task may
be no longer owned by the thread
[visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] WARN
org.apache.kafka.streams.processor.internals.StreamTask - task [0_8] Failed
offset commits {sightings_sighting_byclientmac_0-8=OffsetAndMetadata{offset=1,
metadata=''},
mactocontactmappings_contactmapping_byclientmac_0-8=OffsetAndMetadata{offset=3,
metadata=''}} due to CommitFailedException
[visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] WARN
org.apache.kafka.streams.processor.internals.StreamTask - task [0_9] Failed
offset commits
{mactocontactmappings_contactmapping_byclientmac_0-9=OffsetAndMetadata{offset=115,
metadata=''}, sightings_sighting_byclientmac_0-9=OffsetAndMetadata{offset=1,
metadata=''}} due to CommitFailedException
[visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] WARN
org.apache.kafka.streams.processor.internals.AssignedTasks - stream-thread
[visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] Failed to commit
stream task 0_9 during commit state due to CommitFailedException; this task may
be no longer owned by the thread
[visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] WARN
org.apache.kafka.streams.processor.internals.StreamTask - task [0_10] Failed
offset commits
{mactocontactmappings_contactmapping_byclientmac_0-10=OffsetAndMetadata{offset=22,
metadata=''}, sightings_sighting_byclientmac_0-10=OffsetAndMetadata{offset=2,
metadata=''}} due to CommitFailedException
[visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] WARN
org.apache.kafka.streams.processor.internals.AssignedTasks - stream-thread
[visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] Failed to commit
stream task 0_10 during commit state due to CommitFailedException; this task
may be no longer owned by the thread
[visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] WARN
org.apache.kafka.streams.processor.internals.AssignedTasks - stream-thread
[visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] Failed to commit
stream task 0_11 during commit state due to CommitFailedException; this task
may be no longer owned by the thread
[visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] WARN
org.apache.kafka.streams.processor.internals.StreamTask - task [0_11] Failed
offset commits
{mactocontactmappings_contactmapping_byclientmac_0-11=OffsetAndMetadata{offset=20,
metadata=''}} due to CommitFailedException
[visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] WARN
org.apache.kafka.streams.processor.internals.StreamTask - task [0_12] Failed
offset commits {sightings_sighting_byclientmac_0-12=OffsetAndMetadata{offset=1,
metadata=''},
mactocontactmappings_contactmapping_byclientmac_0-12=OffsetAndMetadata{offset=38,
metadata=''}} due to CommitFailedException
[visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] WARN
org.apache.kafka.streams.processor.internals.AssignedTasks - stream-thread
[visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] Failed to commit
stream task 0_12 during commit state due to CommitFailedException; this task
may be no longer owned by the thread
[visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] WARN
org.apache.kafka.streams.processor.internals.StreamTask - task [0_13] Failed
offset commits
{mactocontactmappings_contactmapping_byclientmac_0-13=OffsetAndMetadata{offset=17,
metadata=''}, sightings_sighting_byclientmac_0-13=OffsetAndMetadata{offset=13,
metadata=''}} due to CommitFailedException
[visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] WARN
org.apache.kafka.streams.processor.internals.AssignedTasks - stream-thread
[visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] Failed to commit
stream task 0_13 during commit state due to CommitFailedException; this task
may be no longer owned by the thread
[visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] WARN
org.apache.kafka.streams.processor.internals.StreamTask - task [0_14] Failed
offset commits
{mactocontactmappings_contactmapping_byclientmac_0-14=OffsetAndMetadata{offset=2,
metadata=''}, sightings_sighting_byclientmac_0-14=OffsetAndMetadata{offset=1,
metadata=''}} due to CommitFailedException
[visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] WARN
org.apache.kafka.streams.processor.internals.AssignedTasks - stream-thread
[visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] Failed to commit
stream task 0_14 during commit state due to CommitFailedException; this task
may be no longer owned by the thread
[visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] WARN
org.apache.kafka.streams.processor.internals.StreamTask - task [0_15] Failed
offset commits
{sightings_sighting_byclientmac_0-15=OffsetAndMetadata{offset=14, metadata=''},
mactocontactmappings_contactmapping_byclientmac_0-15=OffsetAndMetadata{offset=69,
metadata=''}} due to CommitFailedException
[visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] WARN
org.apache.kafka.streams.processor.internals.AssignedTasks - stream-thread
[visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] Failed to commit
stream task 0_15 during commit state due to CommitFailedException; this task
may be no longer owned by the thread
{code}
Same app also keeps rebalancing non stop:
{code:java}
[visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] INFO
org.apache.kafka.streams.processor.internals.StreamThread - stream-thread
[visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] partition
revocation took 2354 ms.
suspended active tasks: [0_0, 0_1, 1_0, 0_2, 1_1, 0_3, 1_2, 0_4, 1_3, 0_5, 1_4,
0_6, 1_5, 0_7, 1_6, 0_8, 1_7, 0_9, 1_8, 0_10, 1_9, 0_11, 1_10, 0_12, 1_11,
0_13, 1_12, 0_14, 1_13, 0_15, 1_14, 1_15]
suspended standby tasks: []
[visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] INFO
org.apache.kafka.clients.consumer.internals.AbstractCoordinator - (Re-)joining
group visits
[visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] INFO
org.apache.kafka.clients.consumer.internals.AbstractCoordinator - Successfully
joined group visits with generation 973
[visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] INFO
org.apache.kafka.clients.consumer.internals.ConsumerCoordinator - Setting newly
assigned partitions [sightings_sighting_byclientmac_0-16,
mactocontactmappings_contactmapping_byclientmac_0-25,
sightings_sighting_byclientmac_0-20,
mactocontactmappings_contactmapping_byclientmac_0-21,
mactocontactmappings_contactmapping_byclientmac_0-29,
visits_enrichedsighting_byclientmac_0-19,
visits_enrichedsighting_byclientmac_0-23, sightings_sighting_byclientmac_0-25,
mactocontactmappings_contactmapping_byclientmac_0-16,
visits_enrichedsighting_byclientmac_0-27, sightings_sighting_byclientmac_0-29,
visits_enrichedsighting_byclientmac_0-31,
mactocontactmappings_contactmapping_byclientmac_0-26,
sightings_sighting_byclientmac_0-19,
mactocontactmappings_contactmapping_byclientmac_0-22,
mactocontactmappings_contactmapping_byclientmac_0-30,
visits_enrichedsighting_byclientmac_0-16,
visits_enrichedsighting_byclientmac_0-20,
visits_enrichedsighting_byclientmac_0-24, sightings_sighting_byclientmac_0-24,
mactocontactmappings_contactmapping_byclientmac_0-17,
visits_enrichedsighting_byclientmac_0-28, sightings_sighting_byclientmac_0-28,
sightings_sighting_byclientmac_0-18,
mactocontactmappings_contactmapping_byclientmac_0-23,
sightings_sighting_byclientmac_0-22,
mactocontactmappings_contactmapping_byclientmac_0-19,
mactocontactmappings_contactmapping_byclientmac_0-31,
mactocontactmappings_contactmapping_byclientmac_0-27,
sightings_sighting_byclientmac_0-31, visits_enrichedsighting_byclientmac_0-17,
visits_enrichedsighting_byclientmac_0-21, sightings_sighting_byclientmac_0-23,
mactocontactmappings_contactmapping_byclientmac_0-18,
visits_enrichedsighting_byclientmac_0-25, sightings_sighting_byclientmac_0-27,
visits_enrichedsighting_byclientmac_0-29, sightings_sighting_byclientmac_0-17,
mactocontactmappings_contactmapping_byclientmac_0-24,
sightings_sighting_byclientmac_0-21,
mactocontactmappings_contactmapping_byclientmac_0-20,
mactocontactmappings_contactmapping_byclientmac_0-28,
visits_enrichedsighting_byclientmac_0-18,
visits_enrichedsighting_byclientmac_0-22,
visits_enrichedsighting_byclientmac_0-26, sightings_sighting_byclientmac_0-26,
visits_enrichedsighting_byclientmac_0-30, sightings_sighting_byclientmac_0-30]
for group visits
[visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] INFO
org.apache.kafka.streams.processor.internals.StreamThread - stream-thread
[visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] State transition
from PARTITIONS_REVOKED to PARTITIONS_ASSIGNED.
[visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] INFO
org.apache.kafka.streams.KafkaStreams - stream-client
[visits-bdb209f1-c9dd-404b-9233-064f2fb4227d] State transition from REBALANCING
to REBALANCING.
[visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] INFO
org.apache.kafka.streams.processor.internals.StreamThread - stream-thread
[visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] partition
assignment took 340 ms.
current active tasks: [0_16, 0_17, 1_16, 0_18, 1_17, 0_19, 1_18, 0_20, 1_19,
0_21, 1_20, 0_22, 1_21, 0_23, 1_22, 0_24, 1_23, 0_25, 1_24, 0_26, 1_25, 0_27,
1_26, 0_28, 1_27, 0_29, 1_28, 0_30, 1_29, 0_31, 1_30, 1_31]
current standby tasks: []
previous active tasks: [0_0, 0_1, 1_0, 0_2, 1_1, 0_3, 1_2, 0_4, 1_3, 0_5, 1_4,
0_6, 1_5, 0_7, 1_6, 0_8, 1_7, 0_9, 1_8, 0_10, 1_9, 0_11, 1_10, 0_12, 1_11,
0_13, 1_12, 0_14, 1_13, 0_15, 1_14, 1_15]
[visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] INFO
org.apache.kafka.streams.processor.internals.StreamThread - stream-thread
[visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] State transition
from PARTITIONS_ASSIGNED to RUNNING.
[visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] INFO
org.apache.kafka.streams.KafkaStreams - stream-client
[visits-bdb209f1-c9dd-404b-9233-064f2fb4227d] State transition from REBALANCING
to RUNNING.
[visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] INFO
org.apache.kafka.clients.consumer.internals.ConsumerCoordinator - Revoking
previously assigned partitions [sightings_sighting_byclientmac_0-16,
mactocontactmappings_contactmapping_byclientmac_0-25,
sightings_sighting_byclientmac_0-20,
mactocontactmappings_contactmapping_byclientmac_0-21,
mactocontactmappings_contactmapping_byclientmac_0-29,
visits_enrichedsighting_byclientmac_0-19,
visits_enrichedsighting_byclientmac_0-23, sightings_sighting_byclientmac_0-25,
mactocontactmappings_contactmapping_byclientmac_0-16,
visits_enrichedsighting_byclientmac_0-27, sightings_sighting_byclientmac_0-29,
visits_enrichedsighting_byclientmac_0-31,
mactocontactmappings_contactmapping_byclientmac_0-26,
sightings_sighting_byclientmac_0-19,
mactocontactmappings_contactmapping_byclientmac_0-22,
mactocontactmappings_contactmapping_byclientmac_0-30,
visits_enrichedsighting_byclientmac_0-16,
visits_enrichedsighting_byclientmac_0-20,
visits_enrichedsighting_byclientmac_0-24, sightings_sighting_byclientmac_0-24,
mactocontactmappings_contactmapping_byclientmac_0-17,
visits_enrichedsighting_byclientmac_0-28, sightings_sighting_byclientmac_0-28,
sightings_sighting_byclientmac_0-18,
mactocontactmappings_contactmapping_byclientmac_0-23,
sightings_sighting_byclientmac_0-22,
mactocontactmappings_contactmapping_byclientmac_0-19,
mactocontactmappings_contactmapping_byclientmac_0-31,
mactocontactmappings_contactmapping_byclientmac_0-27,
sightings_sighting_byclientmac_0-31, visits_enrichedsighting_byclientmac_0-17,
visits_enrichedsighting_byclientmac_0-21, sightings_sighting_byclientmac_0-23,
mactocontactmappings_contactmapping_byclientmac_0-18,
visits_enrichedsighting_byclientmac_0-25, sightings_sighting_byclientmac_0-27,
visits_enrichedsighting_byclientmac_0-29, sightings_sighting_byclientmac_0-17,
mactocontactmappings_contactmapping_byclientmac_0-24,
sightings_sighting_byclientmac_0-21,
mactocontactmappings_contactmapping_byclientmac_0-20,
mactocontactmappings_contactmapping_byclientmac_0-28,
visits_enrichedsighting_byclientmac_0-18,
visits_enrichedsighting_byclientmac_0-22,
visits_enrichedsighting_byclientmac_0-26, sightings_sighting_byclientmac_0-26,
visits_enrichedsighting_byclientmac_0-30, sightings_sighting_byclientmac_0-30]
for group visits
[visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] INFO
org.apache.kafka.streams.processor.internals.StreamThread - stream-thread
[visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] State transition
from RUNNING to PARTITIONS_REVOKED.
[visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] INFO
org.apache.kafka.streams.KafkaStreams - stream-client
[visits-bdb209f1-c9dd-404b-9233-064f2fb4227d] State transition from RUNNING to
REBALANCING.
[visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] INFO
org.apache.kafka.streams.processor.internals.StreamThread - stream-thread
[visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] partition
revocation took 951 ms.
suspended active tasks: [0_16, 1_16, 0_17, 1_17, 0_18, 1_18, 0_19, 1_19, 0_20,
1_20, 0_21, 1_21, 0_22, 1_22, 0_23, 1_23, 0_24, 1_24, 0_25, 1_25, 0_26, 1_26,
0_27, 1_27, 0_28, 1_28, 0_29, 1_29, 0_30, 1_30, 0_31, 1_31]
suspended standby tasks: []
[visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] INFO
org.apache.kafka.clients.consumer.internals.AbstractCoordinator - (Re-)joining
group visits
[visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] INFO
org.apache.kafka.clients.consumer.internals.AbstractCoordinator - Successfully
joined group visits with generation 974{code}
Any idea what could be going on? This service is only running with 2 instances
(each one stream thread).
was:
We are seeing a lot of CommitFailedExceptions on one of our Kafka stream apps.
Running Kafka Streams 0.10.2.1 and Kafka Broker 0.10.2.1. Full error message:
{code:java}
[visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] INFO
org.apache.kafka.streams.KafkaStreams - stream-client
[visits-bdb209f1-c9dd-404b-9233-064f2fb4227d] State transition from REBALANCING
to RUNNING.
[visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] WARN
org.apache.kafka.streams.processor.internals.StreamTask - task [0_0] Failed
offset commits {sightings_sighting_byclientmac_0-0=OffsetAndMetadata{offset=11,
metadata=''},
mactocontactmappings_contactmapping_byclientmac_0-0=OffsetAndMetadata{offset=26,
metadata=''}} due to CommitFailedException
[visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] WARN
org.apache.kafka.streams.processor.internals.AssignedTasks - stream-thread
[visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] Failed to commit
stream task 0_0 during commit state due to CommitFailedException; this task may
be no longer owned by the thread
[visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] WARN
org.apache.kafka.streams.processor.internals.AssignedTasks - stream-thread
[visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] Failed to commit
stream task 0_1 during commit state due to CommitFailedException; this task may
be no longer owned by the thread
[visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] WARN
org.apache.kafka.streams.processor.internals.StreamTask - task [0_1] Failed
offset commits
{mactocontactmappings_contactmapping_byclientmac_0-1=OffsetAndMetadata{offset=38,
metadata=''}, sightings_sighting_byclientmac_0-1=OffsetAndMetadata{offset=1,
metadata=''}} due to CommitFailedException
[visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] WARN
org.apache.kafka.streams.processor.internals.StreamTask - task [0_2] Failed
offset commits
{mactocontactmappings_contactmapping_byclientmac_0-2=OffsetAndMetadata{offset=8,
metadata=''}, sightings_sighting_byclientmac_0-2=OffsetAndMetadata{offset=24,
metadata=''}} due to CommitFailedException
[visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] WARN
org.apache.kafka.streams.processor.internals.AssignedTasks - stream-thread
[visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] Failed to commit
stream task 0_2 during commit state due to CommitFailedException; this task may
be no longer owned by the thread
[visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] WARN
org.apache.kafka.streams.processor.internals.StreamTask - task [0_3] Failed
offset commits {sightings_sighting_byclientmac_0-3=OffsetAndMetadata{offset=21,
metadata=''},
mactocontactmappings_contactmapping_byclientmac_0-3=OffsetAndMetadata{offset=102,
metadata=''}} due to CommitFailedException
[visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] WARN
org.apache.kafka.streams.processor.internals.AssignedTasks - stream-thread
[visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] Failed to commit
stream task 0_3 during commit state due to CommitFailedException; this task may
be no longer owned by the thread
[visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] WARN
org.apache.kafka.streams.processor.internals.StreamTask - task [0_4] Failed
offset commits {sightings_sighting_byclientmac_0-4=OffsetAndMetadata{offset=5,
metadata=''},
mactocontactmappings_contactmapping_byclientmac_0-4=OffsetAndMetadata{offset=20,
metadata=''}} due to CommitFailedException
[visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] WARN
org.apache.kafka.streams.processor.internals.AssignedTasks - stream-thread
[visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] Failed to commit
stream task 0_4 during commit state due to CommitFailedException; this task may
be no longer owned by the thread
[visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] WARN
org.apache.kafka.streams.processor.internals.StreamTask - task [0_5] Failed
offset commits
{mactocontactmappings_contactmapping_byclientmac_0-5=OffsetAndMetadata{offset=26,
metadata=''}, sightings_sighting_byclientmac_0-5=OffsetAndMetadata{offset=16,
metadata=''}} due to CommitFailedException
[visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] WARN
org.apache.kafka.streams.processor.internals.AssignedTasks - stream-thread
[visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] Failed to commit
stream task 0_5 during commit state due to CommitFailedException; this task may
be no longer owned by the thread
[visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] WARN
org.apache.kafka.streams.processor.internals.StreamTask - task [0_6] Failed
offset commits
{mactocontactmappings_contactmapping_byclientmac_0-6=OffsetAndMetadata{offset=22,
metadata=''}, sightings_sighting_byclientmac_0-6=OffsetAndMetadata{offset=3,
metadata=''}} due to CommitFailedException
[visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] WARN
org.apache.kafka.streams.processor.internals.AssignedTasks - stream-thread
[visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] Failed to commit
stream task 0_6 during commit state due to CommitFailedException; this task may
be no longer owned by the thread
[visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] WARN
org.apache.kafka.streams.processor.internals.StreamTask - task [0_7] Failed
offset commits {sightings_sighting_byclientmac_0-7=OffsetAndMetadata{offset=3,
metadata=''},
mactocontactmappings_contactmapping_byclientmac_0-7=OffsetAndMetadata{offset=5,
metadata=''}} due to CommitFailedException
[visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] WARN
org.apache.kafka.streams.processor.internals.AssignedTasks - stream-thread
[visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] Failed to commit
stream task 0_7 during commit state due to CommitFailedException; this task may
be no longer owned by the thread
[visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] WARN
org.apache.kafka.streams.processor.internals.AssignedTasks - stream-thread
[visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] Failed to commit
stream task 0_8 during commit state due to CommitFailedException; this task may
be no longer owned by the thread
[visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] WARN
org.apache.kafka.streams.processor.internals.StreamTask - task [0_8] Failed
offset commits {sightings_sighting_byclientmac_0-8=OffsetAndMetadata{offset=1,
metadata=''},
mactocontactmappings_contactmapping_byclientmac_0-8=OffsetAndMetadata{offset=3,
metadata=''}} due to CommitFailedException
[visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] WARN
org.apache.kafka.streams.processor.internals.StreamTask - task [0_9] Failed
offset commits
{mactocontactmappings_contactmapping_byclientmac_0-9=OffsetAndMetadata{offset=115,
metadata=''}, sightings_sighting_byclientmac_0-9=OffsetAndMetadata{offset=1,
metadata=''}} due to CommitFailedException
[visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] WARN
org.apache.kafka.streams.processor.internals.AssignedTasks - stream-thread
[visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] Failed to commit
stream task 0_9 during commit state due to CommitFailedException; this task may
be no longer owned by the thread
[visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] WARN
org.apache.kafka.streams.processor.internals.StreamTask - task [0_10] Failed
offset commits
{mactocontactmappings_contactmapping_byclientmac_0-10=OffsetAndMetadata{offset=22,
metadata=''}, sightings_sighting_byclientmac_0-10=OffsetAndMetadata{offset=2,
metadata=''}} due to CommitFailedException
[visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] WARN
org.apache.kafka.streams.processor.internals.AssignedTasks - stream-thread
[visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] Failed to commit
stream task 0_10 during commit state due to CommitFailedException; this task
may be no longer owned by the thread
[visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] WARN
org.apache.kafka.streams.processor.internals.AssignedTasks - stream-thread
[visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] Failed to commit
stream task 0_11 during commit state due to CommitFailedException; this task
may be no longer owned by the thread
[visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] WARN
org.apache.kafka.streams.processor.internals.StreamTask - task [0_11] Failed
offset commits
{mactocontactmappings_contactmapping_byclientmac_0-11=OffsetAndMetadata{offset=20,
metadata=''}} due to CommitFailedException
[visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] WARN
org.apache.kafka.streams.processor.internals.StreamTask - task [0_12] Failed
offset commits {sightings_sighting_byclientmac_0-12=OffsetAndMetadata{offset=1,
metadata=''},
mactocontactmappings_contactmapping_byclientmac_0-12=OffsetAndMetadata{offset=38,
metadata=''}} due to CommitFailedException
[visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] WARN
org.apache.kafka.streams.processor.internals.AssignedTasks - stream-thread
[visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] Failed to commit
stream task 0_12 during commit state due to CommitFailedException; this task
may be no longer owned by the thread
[visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] WARN
org.apache.kafka.streams.processor.internals.StreamTask - task [0_13] Failed
offset commits
{mactocontactmappings_contactmapping_byclientmac_0-13=OffsetAndMetadata{offset=17,
metadata=''}, sightings_sighting_byclientmac_0-13=OffsetAndMetadata{offset=13,
metadata=''}} due to CommitFailedException
[visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] WARN
org.apache.kafka.streams.processor.internals.AssignedTasks - stream-thread
[visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] Failed to commit
stream task 0_13 during commit state due to CommitFailedException; this task
may be no longer owned by the thread
[visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] WARN
org.apache.kafka.streams.processor.internals.StreamTask - task [0_14] Failed
offset commits
{mactocontactmappings_contactmapping_byclientmac_0-14=OffsetAndMetadata{offset=2,
metadata=''}, sightings_sighting_byclientmac_0-14=OffsetAndMetadata{offset=1,
metadata=''}} due to CommitFailedException
[visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] WARN
org.apache.kafka.streams.processor.internals.AssignedTasks - stream-thread
[visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] Failed to commit
stream task 0_14 during commit state due to CommitFailedException; this task
may be no longer owned by the thread
[visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] WARN
org.apache.kafka.streams.processor.internals.StreamTask - task [0_15] Failed
offset commits
{sightings_sighting_byclientmac_0-15=OffsetAndMetadata{offset=14, metadata=''},
mactocontactmappings_contactmapping_byclientmac_0-15=OffsetAndMetadata{offset=69,
metadata=''}} due to CommitFailedException
[visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] WARN
org.apache.kafka.streams.processor.internals.AssignedTasks - stream-thread
[visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] Failed to commit
stream task 0_15 during commit state due to CommitFailedException; this task
may be no longer owned by the thread
{code}
Any idea what could be going on? This service is only running with 2 instances
(each one stream thread).
> CommitFailedException; this task may be no longer owned by the thread
> ---------------------------------------------------------------------
>
> Key: KAFKA-6990
> URL: https://issues.apache.org/jira/browse/KAFKA-6990
> Project: Kafka
> Issue Type: Bug
> Components: streams
> Affects Versions: 0.10.2.1
> Reporter: Eugen Feller
> Priority: Blocker
>
> We are seeing a lot of CommitFailedExceptions on one of our Kafka stream
> apps. Running Kafka Streams 0.10.2.1 and Kafka Broker 0.10.2.1. Full error
> message:
> {code:java}
> [visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] INFO
> org.apache.kafka.streams.KafkaStreams - stream-client
> [visits-bdb209f1-c9dd-404b-9233-064f2fb4227d] State transition from
> REBALANCING to RUNNING.
> [visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] WARN
> org.apache.kafka.streams.processor.internals.StreamTask - task [0_0] Failed
> offset commits
> {sightings_sighting_byclientmac_0-0=OffsetAndMetadata{offset=11,
> metadata=''},
> mactocontactmappings_contactmapping_byclientmac_0-0=OffsetAndMetadata{offset=26,
> metadata=''}} due to CommitFailedException
> [visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] WARN
> org.apache.kafka.streams.processor.internals.AssignedTasks - stream-thread
> [visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] Failed to commit
> stream task 0_0 during commit state due to CommitFailedException; this task
> may be no longer owned by the thread
> [visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] WARN
> org.apache.kafka.streams.processor.internals.AssignedTasks - stream-thread
> [visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] Failed to commit
> stream task 0_1 during commit state due to CommitFailedException; this task
> may be no longer owned by the thread
> [visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] WARN
> org.apache.kafka.streams.processor.internals.StreamTask - task [0_1] Failed
> offset commits
> {mactocontactmappings_contactmapping_byclientmac_0-1=OffsetAndMetadata{offset=38,
> metadata=''}, sightings_sighting_byclientmac_0-1=OffsetAndMetadata{offset=1,
> metadata=''}} due to CommitFailedException
> [visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] WARN
> org.apache.kafka.streams.processor.internals.StreamTask - task [0_2] Failed
> offset commits
> {mactocontactmappings_contactmapping_byclientmac_0-2=OffsetAndMetadata{offset=8,
> metadata=''},
> sightings_sighting_byclientmac_0-2=OffsetAndMetadata{offset=24, metadata=''}}
> due to CommitFailedException
> [visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] WARN
> org.apache.kafka.streams.processor.internals.AssignedTasks - stream-thread
> [visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] Failed to commit
> stream task 0_2 during commit state due to CommitFailedException; this task
> may be no longer owned by the thread
> [visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] WARN
> org.apache.kafka.streams.processor.internals.StreamTask - task [0_3] Failed
> offset commits
> {sightings_sighting_byclientmac_0-3=OffsetAndMetadata{offset=21,
> metadata=''},
> mactocontactmappings_contactmapping_byclientmac_0-3=OffsetAndMetadata{offset=102,
> metadata=''}} due to CommitFailedException
> [visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] WARN
> org.apache.kafka.streams.processor.internals.AssignedTasks - stream-thread
> [visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] Failed to commit
> stream task 0_3 during commit state due to CommitFailedException; this task
> may be no longer owned by the thread
> [visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] WARN
> org.apache.kafka.streams.processor.internals.StreamTask - task [0_4] Failed
> offset commits
> {sightings_sighting_byclientmac_0-4=OffsetAndMetadata{offset=5, metadata=''},
> mactocontactmappings_contactmapping_byclientmac_0-4=OffsetAndMetadata{offset=20,
> metadata=''}} due to CommitFailedException
> [visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] WARN
> org.apache.kafka.streams.processor.internals.AssignedTasks - stream-thread
> [visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] Failed to commit
> stream task 0_4 during commit state due to CommitFailedException; this task
> may be no longer owned by the thread
> [visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] WARN
> org.apache.kafka.streams.processor.internals.StreamTask - task [0_5] Failed
> offset commits
> {mactocontactmappings_contactmapping_byclientmac_0-5=OffsetAndMetadata{offset=26,
> metadata=''},
> sightings_sighting_byclientmac_0-5=OffsetAndMetadata{offset=16, metadata=''}}
> due to CommitFailedException
> [visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] WARN
> org.apache.kafka.streams.processor.internals.AssignedTasks - stream-thread
> [visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] Failed to commit
> stream task 0_5 during commit state due to CommitFailedException; this task
> may be no longer owned by the thread
> [visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] WARN
> org.apache.kafka.streams.processor.internals.StreamTask - task [0_6] Failed
> offset commits
> {mactocontactmappings_contactmapping_byclientmac_0-6=OffsetAndMetadata{offset=22,
> metadata=''}, sightings_sighting_byclientmac_0-6=OffsetAndMetadata{offset=3,
> metadata=''}} due to CommitFailedException
> [visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] WARN
> org.apache.kafka.streams.processor.internals.AssignedTasks - stream-thread
> [visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] Failed to commit
> stream task 0_6 during commit state due to CommitFailedException; this task
> may be no longer owned by the thread
> [visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] WARN
> org.apache.kafka.streams.processor.internals.StreamTask - task [0_7] Failed
> offset commits
> {sightings_sighting_byclientmac_0-7=OffsetAndMetadata{offset=3, metadata=''},
> mactocontactmappings_contactmapping_byclientmac_0-7=OffsetAndMetadata{offset=5,
> metadata=''}} due to CommitFailedException
> [visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] WARN
> org.apache.kafka.streams.processor.internals.AssignedTasks - stream-thread
> [visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] Failed to commit
> stream task 0_7 during commit state due to CommitFailedException; this task
> may be no longer owned by the thread
> [visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] WARN
> org.apache.kafka.streams.processor.internals.AssignedTasks - stream-thread
> [visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] Failed to commit
> stream task 0_8 during commit state due to CommitFailedException; this task
> may be no longer owned by the thread
> [visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] WARN
> org.apache.kafka.streams.processor.internals.StreamTask - task [0_8] Failed
> offset commits
> {sightings_sighting_byclientmac_0-8=OffsetAndMetadata{offset=1, metadata=''},
> mactocontactmappings_contactmapping_byclientmac_0-8=OffsetAndMetadata{offset=3,
> metadata=''}} due to CommitFailedException
> [visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] WARN
> org.apache.kafka.streams.processor.internals.StreamTask - task [0_9] Failed
> offset commits
> {mactocontactmappings_contactmapping_byclientmac_0-9=OffsetAndMetadata{offset=115,
> metadata=''}, sightings_sighting_byclientmac_0-9=OffsetAndMetadata{offset=1,
> metadata=''}} due to CommitFailedException
> [visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] WARN
> org.apache.kafka.streams.processor.internals.AssignedTasks - stream-thread
> [visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] Failed to commit
> stream task 0_9 during commit state due to CommitFailedException; this task
> may be no longer owned by the thread
> [visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] WARN
> org.apache.kafka.streams.processor.internals.StreamTask - task [0_10] Failed
> offset commits
> {mactocontactmappings_contactmapping_byclientmac_0-10=OffsetAndMetadata{offset=22,
> metadata=''},
> sightings_sighting_byclientmac_0-10=OffsetAndMetadata{offset=2, metadata=''}}
> due to CommitFailedException
> [visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] WARN
> org.apache.kafka.streams.processor.internals.AssignedTasks - stream-thread
> [visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] Failed to commit
> stream task 0_10 during commit state due to CommitFailedException; this task
> may be no longer owned by the thread
> [visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] WARN
> org.apache.kafka.streams.processor.internals.AssignedTasks - stream-thread
> [visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] Failed to commit
> stream task 0_11 during commit state due to CommitFailedException; this task
> may be no longer owned by the thread
> [visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] WARN
> org.apache.kafka.streams.processor.internals.StreamTask - task [0_11] Failed
> offset commits
> {mactocontactmappings_contactmapping_byclientmac_0-11=OffsetAndMetadata{offset=20,
> metadata=''}} due to CommitFailedException
> [visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] WARN
> org.apache.kafka.streams.processor.internals.StreamTask - task [0_12] Failed
> offset commits
> {sightings_sighting_byclientmac_0-12=OffsetAndMetadata{offset=1,
> metadata=''},
> mactocontactmappings_contactmapping_byclientmac_0-12=OffsetAndMetadata{offset=38,
> metadata=''}} due to CommitFailedException
> [visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] WARN
> org.apache.kafka.streams.processor.internals.AssignedTasks - stream-thread
> [visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] Failed to commit
> stream task 0_12 during commit state due to CommitFailedException; this task
> may be no longer owned by the thread
> [visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] WARN
> org.apache.kafka.streams.processor.internals.StreamTask - task [0_13] Failed
> offset commits
> {mactocontactmappings_contactmapping_byclientmac_0-13=OffsetAndMetadata{offset=17,
> metadata=''},
> sightings_sighting_byclientmac_0-13=OffsetAndMetadata{offset=13,
> metadata=''}} due to CommitFailedException
> [visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] WARN
> org.apache.kafka.streams.processor.internals.AssignedTasks - stream-thread
> [visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] Failed to commit
> stream task 0_13 during commit state due to CommitFailedException; this task
> may be no longer owned by the thread
> [visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] WARN
> org.apache.kafka.streams.processor.internals.StreamTask - task [0_14] Failed
> offset commits
> {mactocontactmappings_contactmapping_byclientmac_0-14=OffsetAndMetadata{offset=2,
> metadata=''},
> sightings_sighting_byclientmac_0-14=OffsetAndMetadata{offset=1, metadata=''}}
> due to CommitFailedException
> [visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] WARN
> org.apache.kafka.streams.processor.internals.AssignedTasks - stream-thread
> [visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] Failed to commit
> stream task 0_14 during commit state due to CommitFailedException; this task
> may be no longer owned by the thread
> [visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] WARN
> org.apache.kafka.streams.processor.internals.StreamTask - task [0_15] Failed
> offset commits
> {sightings_sighting_byclientmac_0-15=OffsetAndMetadata{offset=14,
> metadata=''},
> mactocontactmappings_contactmapping_byclientmac_0-15=OffsetAndMetadata{offset=69,
> metadata=''}} due to CommitFailedException
> [visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] WARN
> org.apache.kafka.streams.processor.internals.AssignedTasks - stream-thread
> [visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] Failed to commit
> stream task 0_15 during commit state due to CommitFailedException; this task
> may be no longer owned by the thread
> {code}
>
> Same app also keeps rebalancing non stop:
> {code:java}
> [visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] INFO
> org.apache.kafka.streams.processor.internals.StreamThread - stream-thread
> [visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] partition
> revocation took 2354 ms.
> suspended active tasks: [0_0, 0_1, 1_0, 0_2, 1_1, 0_3, 1_2, 0_4, 1_3, 0_5,
> 1_4, 0_6, 1_5, 0_7, 1_6, 0_8, 1_7, 0_9, 1_8, 0_10, 1_9, 0_11, 1_10, 0_12,
> 1_11, 0_13, 1_12, 0_14, 1_13, 0_15, 1_14, 1_15]
> suspended standby tasks: []
> [visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] INFO
> org.apache.kafka.clients.consumer.internals.AbstractCoordinator -
> (Re-)joining group visits
> [visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] INFO
> org.apache.kafka.clients.consumer.internals.AbstractCoordinator -
> Successfully joined group visits with generation 973
> [visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] INFO
> org.apache.kafka.clients.consumer.internals.ConsumerCoordinator - Setting
> newly assigned partitions [sightings_sighting_byclientmac_0-16,
> mactocontactmappings_contactmapping_byclientmac_0-25,
> sightings_sighting_byclientmac_0-20,
> mactocontactmappings_contactmapping_byclientmac_0-21,
> mactocontactmappings_contactmapping_byclientmac_0-29,
> visits_enrichedsighting_byclientmac_0-19,
> visits_enrichedsighting_byclientmac_0-23,
> sightings_sighting_byclientmac_0-25,
> mactocontactmappings_contactmapping_byclientmac_0-16,
> visits_enrichedsighting_byclientmac_0-27,
> sightings_sighting_byclientmac_0-29,
> visits_enrichedsighting_byclientmac_0-31,
> mactocontactmappings_contactmapping_byclientmac_0-26,
> sightings_sighting_byclientmac_0-19,
> mactocontactmappings_contactmapping_byclientmac_0-22,
> mactocontactmappings_contactmapping_byclientmac_0-30,
> visits_enrichedsighting_byclientmac_0-16,
> visits_enrichedsighting_byclientmac_0-20,
> visits_enrichedsighting_byclientmac_0-24,
> sightings_sighting_byclientmac_0-24,
> mactocontactmappings_contactmapping_byclientmac_0-17,
> visits_enrichedsighting_byclientmac_0-28,
> sightings_sighting_byclientmac_0-28, sightings_sighting_byclientmac_0-18,
> mactocontactmappings_contactmapping_byclientmac_0-23,
> sightings_sighting_byclientmac_0-22,
> mactocontactmappings_contactmapping_byclientmac_0-19,
> mactocontactmappings_contactmapping_byclientmac_0-31,
> mactocontactmappings_contactmapping_byclientmac_0-27,
> sightings_sighting_byclientmac_0-31,
> visits_enrichedsighting_byclientmac_0-17,
> visits_enrichedsighting_byclientmac_0-21,
> sightings_sighting_byclientmac_0-23,
> mactocontactmappings_contactmapping_byclientmac_0-18,
> visits_enrichedsighting_byclientmac_0-25,
> sightings_sighting_byclientmac_0-27,
> visits_enrichedsighting_byclientmac_0-29,
> sightings_sighting_byclientmac_0-17,
> mactocontactmappings_contactmapping_byclientmac_0-24,
> sightings_sighting_byclientmac_0-21,
> mactocontactmappings_contactmapping_byclientmac_0-20,
> mactocontactmappings_contactmapping_byclientmac_0-28,
> visits_enrichedsighting_byclientmac_0-18,
> visits_enrichedsighting_byclientmac_0-22,
> visits_enrichedsighting_byclientmac_0-26,
> sightings_sighting_byclientmac_0-26,
> visits_enrichedsighting_byclientmac_0-30,
> sightings_sighting_byclientmac_0-30] for group visits
> [visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] INFO
> org.apache.kafka.streams.processor.internals.StreamThread - stream-thread
> [visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] State transition
> from PARTITIONS_REVOKED to PARTITIONS_ASSIGNED.
> [visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] INFO
> org.apache.kafka.streams.KafkaStreams - stream-client
> [visits-bdb209f1-c9dd-404b-9233-064f2fb4227d] State transition from
> REBALANCING to REBALANCING.
> [visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] INFO
> org.apache.kafka.streams.processor.internals.StreamThread - stream-thread
> [visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] partition
> assignment took 340 ms.
> current active tasks: [0_16, 0_17, 1_16, 0_18, 1_17, 0_19, 1_18, 0_20, 1_19,
> 0_21, 1_20, 0_22, 1_21, 0_23, 1_22, 0_24, 1_23, 0_25, 1_24, 0_26, 1_25, 0_27,
> 1_26, 0_28, 1_27, 0_29, 1_28, 0_30, 1_29, 0_31, 1_30, 1_31]
> current standby tasks: []
> previous active tasks: [0_0, 0_1, 1_0, 0_2, 1_1, 0_3, 1_2, 0_4, 1_3, 0_5,
> 1_4, 0_6, 1_5, 0_7, 1_6, 0_8, 1_7, 0_9, 1_8, 0_10, 1_9, 0_11, 1_10, 0_12,
> 1_11, 0_13, 1_12, 0_14, 1_13, 0_15, 1_14, 1_15]
> [visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] INFO
> org.apache.kafka.streams.processor.internals.StreamThread - stream-thread
> [visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] State transition
> from PARTITIONS_ASSIGNED to RUNNING.
> [visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] INFO
> org.apache.kafka.streams.KafkaStreams - stream-client
> [visits-bdb209f1-c9dd-404b-9233-064f2fb4227d] State transition from
> REBALANCING to RUNNING.
> [visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] INFO
> org.apache.kafka.clients.consumer.internals.ConsumerCoordinator - Revoking
> previously assigned partitions [sightings_sighting_byclientmac_0-16,
> mactocontactmappings_contactmapping_byclientmac_0-25,
> sightings_sighting_byclientmac_0-20,
> mactocontactmappings_contactmapping_byclientmac_0-21,
> mactocontactmappings_contactmapping_byclientmac_0-29,
> visits_enrichedsighting_byclientmac_0-19,
> visits_enrichedsighting_byclientmac_0-23,
> sightings_sighting_byclientmac_0-25,
> mactocontactmappings_contactmapping_byclientmac_0-16,
> visits_enrichedsighting_byclientmac_0-27,
> sightings_sighting_byclientmac_0-29,
> visits_enrichedsighting_byclientmac_0-31,
> mactocontactmappings_contactmapping_byclientmac_0-26,
> sightings_sighting_byclientmac_0-19,
> mactocontactmappings_contactmapping_byclientmac_0-22,
> mactocontactmappings_contactmapping_byclientmac_0-30,
> visits_enrichedsighting_byclientmac_0-16,
> visits_enrichedsighting_byclientmac_0-20,
> visits_enrichedsighting_byclientmac_0-24,
> sightings_sighting_byclientmac_0-24,
> mactocontactmappings_contactmapping_byclientmac_0-17,
> visits_enrichedsighting_byclientmac_0-28,
> sightings_sighting_byclientmac_0-28, sightings_sighting_byclientmac_0-18,
> mactocontactmappings_contactmapping_byclientmac_0-23,
> sightings_sighting_byclientmac_0-22,
> mactocontactmappings_contactmapping_byclientmac_0-19,
> mactocontactmappings_contactmapping_byclientmac_0-31,
> mactocontactmappings_contactmapping_byclientmac_0-27,
> sightings_sighting_byclientmac_0-31,
> visits_enrichedsighting_byclientmac_0-17,
> visits_enrichedsighting_byclientmac_0-21,
> sightings_sighting_byclientmac_0-23,
> mactocontactmappings_contactmapping_byclientmac_0-18,
> visits_enrichedsighting_byclientmac_0-25,
> sightings_sighting_byclientmac_0-27,
> visits_enrichedsighting_byclientmac_0-29,
> sightings_sighting_byclientmac_0-17,
> mactocontactmappings_contactmapping_byclientmac_0-24,
> sightings_sighting_byclientmac_0-21,
> mactocontactmappings_contactmapping_byclientmac_0-20,
> mactocontactmappings_contactmapping_byclientmac_0-28,
> visits_enrichedsighting_byclientmac_0-18,
> visits_enrichedsighting_byclientmac_0-22,
> visits_enrichedsighting_byclientmac_0-26,
> sightings_sighting_byclientmac_0-26,
> visits_enrichedsighting_byclientmac_0-30,
> sightings_sighting_byclientmac_0-30] for group visits
> [visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] INFO
> org.apache.kafka.streams.processor.internals.StreamThread - stream-thread
> [visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] State transition
> from RUNNING to PARTITIONS_REVOKED.
> [visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] INFO
> org.apache.kafka.streams.KafkaStreams - stream-client
> [visits-bdb209f1-c9dd-404b-9233-064f2fb4227d] State transition from RUNNING
> to REBALANCING.
> [visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] INFO
> org.apache.kafka.streams.processor.internals.StreamThread - stream-thread
> [visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] partition
> revocation took 951 ms.
> suspended active tasks: [0_16, 1_16, 0_17, 1_17, 0_18, 1_18, 0_19, 1_19,
> 0_20, 1_20, 0_21, 1_21, 0_22, 1_22, 0_23, 1_23, 0_24, 1_24, 0_25, 1_25, 0_26,
> 1_26, 0_27, 1_27, 0_28, 1_28, 0_29, 1_29, 0_30, 1_30, 0_31, 1_31]
> suspended standby tasks: []
> [visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] INFO
> org.apache.kafka.clients.consumer.internals.AbstractCoordinator -
> (Re-)joining group visits
> [visits-bdb209f1-c9dd-404b-9233-064f2fb4227d-StreamThread-1] INFO
> org.apache.kafka.clients.consumer.internals.AbstractCoordinator -
> Successfully joined group visits with generation 974{code}
> Any idea what could be going on? This service is only running with 2
> instances (each one stream thread).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
