here is a scala test which produces UNKNOWN_MEMBER_ID
@Test def testJoinGroupUnknownConsumerNewGroup() { val joinGroupResult =
joinGroup(groupId, memberId, DefaultSessionTimeout, protocolType, protocols)
val joinGroupErrorCode = joinGroupResult.errorCode
assertEquals(Errors.UNKNOWN_MEMBER_ID.code, joinGroupErrorCode) }
an attribute of supplied group trait (trait is like java class) is either null
or invalid..log the groupId, memberId and verify e.g.
log.info("Logging GroupId %d MemberId %d".format(groupId, memberId))
Martin
> From: [email protected]
> To: [email protected]
> Subject: Consumer failure and producer topic deduplication
> Date: Sun, 21 Aug 2016 14:44:43 +0200
>
>
> Hi,
>
>
>
>
> I am using Kafka for a fairly specific use case with a need for a lot of
> control. I use the new KafkaConsumer to consume events from input topic,
> process the data and publish to multiple different topics. The use case
> requires fairly strict delivery and consistency guarantees. All messages
> must be published to the resulting topics "exactly once". I am using
> ExactlyOnceDeliveryConsumerRebalanceListener to handle cases where the
> message is published, but the consumer dies before committing the offset
> which would normally result in a at least once delivery guarantee and
> therefore possibly duplicated messages, because after a rebalance new
> consumer in the group would publish the same uncommited offset again.
>
>
>
>
> Therefore I am using an ExactlyOnceDeliveryConsumerRebalanceListener to
> perform correcting actions during group rebalance after a failure. I am
> using a SimpleConsumer (for granular control and ability to read/commit
> using the original consumer group) to find these issues and publish the
> partially processed records as necessary and most importantly commit the
> offset to avoid duplicates if required. The result of 'simpleConsumer.
> commitOffsets(new OffsetCommitRequest(group, topicPartition))' however is
> 'OffsetCommitResponse(Map([topic,0] -> 25),0)', i.e. UNKNOWN_MEMBER_ID
> error.
>
>
>
>
> Can anyone please explain why I might be getting this error and how to avoid
> it in my scenario?
>
>
>
>
> Thank you,
>
> Martin
