[
https://issues.apache.org/jira/browse/APEXMALHAR-2518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ananth closed APEXMALHAR-2518.
------------------------------
> Kafka input operator stops reading tuples when there is a UNKNOWN_MEMBER_ID
> error during committed offset processing
> --------------------------------------------------------------------------------------------------------------------
>
> Key: APEXMALHAR-2518
> URL: https://issues.apache.org/jira/browse/APEXMALHAR-2518
> Project: Apache Apex Malhar
> Issue Type: Bug
> Reporter: Pramod Immaneni
> Assignee: Pramod Immaneni
> Fix For: 3.8.0
>
>
> Kafka 0.9 operator stores offsets that are completely processed and no longer
> needed (committed offsets) back in kafka. It does so by making a kafka API
> call. If the response from kafka server to this call comes back with an
> UNKNOWN_MEMBER_ID error, it results in the kafka consumer state changing to
> needing partition re-assignment and no further messages are returned by the
> consumer. There are a couple of other errors that result in the same state
> including when rebalance is in progress.
> What exactly caused this error is not known but the following is the likely
> reason due to the conditions surrounding the application. When the operator
> has temporarily stalled due to back-pressure exerted by the slow downstream,
> it will eventually stall the operator kafka consumer thread that is reading
> messages from kafka. This will result in the thread not making any kafka
> consumer API calls and it will result in no heartbeats being sent to kafka
> server. This can cause the server to evict the consumer after a timeout
> period. This could have been the cause for the UNKNOWN_MEMBER_ID error.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
