I agree that Camel 3.13 reconnects to Kafka as expected.
Logs are:
2021-12-09 11:50:36.267 WARN [fast-rms,,] 1 --- [t-kyc-response]]
org.apache.kafka.clients.NetworkClient : [Consumer
clientId=consumer-fast-rms-4, groupId=fast-rms] Connection to node 2
(dev-pf-kafka02.local.site.com/10.6.43.30:9093) could not be
established. Broker may not be available.
2021-12-09 11:50:36.267 INFO [fast-rms,,] 1 --- [t-kyc-response]]
o.a.kafka.clients.FetchSessionHandler : [Consumer
clientId=consumer-fast-rms-4, groupId=fast-rms] Error sending fetch
request (sessionId=1835519236, epoch=INITIAL) to node 2:
org.apache.kafka.common.errors.DisconnectException: null
2021-12-09 11:50:48.881 WARN [fast-rms,,] 1 --- [t-kyc-response]]
o.a.k.c.c.internals.ConsumerCoordinator : [Consumer
clientId=consumer-fast-rms-4, groupId=fast-rms] Offset commit failed on
partition wallet-kyc-response-0 at offset 14: The request timed out.
2021-12-09 11:50:48.881 INFO [fast-rms,,] 1 --- [t-kyc-response]]
o.a.k.c.c.internals.AbstractCoordinator : [Consumer
clientId=consumer-fast-rms-4, groupId=fast-rms] Group coordinator
dev-pf-kafka01.local.site.com:9093 (id: 2147483646 rack: null) is
unavailable or invalid due to cause: error response
REQUEST_TIMED_OUT.isDisconnected: false. Rediscovery will be attempted.
2021-12-09 11:50:48.999 INFO [fast-rms,,] 1 --- [t-kyc-response]]
o.a.k.c.c.internals.AbstractCoordinator : [Consumer
clientId=consumer-fast-rms-4, groupId=fast-rms] Discovered group
coordinator dev-pf-kafka01.local.site.com:9093 (id: 2147483646 rack: null)
On 11/18/21 14:10, Andrea Cosentino wrote:
For work around, I meant to say work in the area of health check, not
workaround.
So there are some improvements in 3.13.0, but not something specific
for your case.
Please keep the users ML through the addresses, it would be helpful
for the community.
Il giorno gio 18 nov 2021 alle ore 12:01 Vyacheslav Boyko
<mail4...@gmail.com> ha scritto:
Where to read about the work around?
On 11/18/21 12:09, Andrea Cosentino wrote:
There are some work around the health check and other stuff.
But first you need to provide what version of Camel you're using.
Il giorno gio 18 nov 2021 alle ore 10:07 Vyacheslav Boyko
<mail4...@gmail.com> ha scritto:
Hello!
I have just faced an issue.
My microservice (camel-kafka component) got disconnected from
kafka
(consumer and producer both) after Kafka restarted.
Last logs were:
2021-11-16 17:43:02.703 INFO 1 --- [-card-requests]]
o.a.k.c.c.internals.AbstractCoordinator : [Consumer
clientId=consumer-card-products-2, groupId=card-products] Group
coordinator dev-pf-kafka02.local.site.com:9093
<http://dev-pf-kafka02.local.site.com:9093> (id: 2147483645
rack:
null) is unavailable or invalid due to cause: coordinator
unavailable.isDisconnected: true. Rediscovery will be attempted.
2021-11-16 17:43:02.981 INFO 1 --- [-card-requests]]
o.a.k.c.c.internals.AbstractCoordinator : [Consumer
clientId=consumer-card-products-2, groupId=card-products]
Discovered
group coordinator dev-pf-kafka01.local.site.com:9093
<http://dev-pf-kafka01.local.site.com:9093> (id: 2147483646
rack: null)
2021-11-16 17:43:03.225 INFO 1 --- [-card-requests]]
o.a.kafka.clients.FetchSessionHandler : [Consumer
clientId=consumer-card-products-2, groupId=card-products]
Node 1 was
unable to process the fetch request with (sessionId=1076866435,
epoch=1028386): FETCH_SESSION_ID_NOT_FOUND.
2021-11-16 17:44:21.172 INFO 1 --- [-card-requests]]
o.a.kafka.clients.FetchSessionHandler : [Consumer
clientId=consumer-card-products-2, groupId=card-products]
Node 2 was
unable to process the fetch request with (sessionId=688855896,
epoch=1027353): FETCH_SESSION_ID_NOT_FOUND.
2021-11-16 17:44:23.373 INFO 1 --- [-card-requests]]
o.a.k.c.c.internals.AbstractCoordinator : [Consumer
clientId=consumer-card-products-2, groupId=card-products]
Attempt to
heartbeat failed since coordinator
dev-pf-kafka01.local.site.com:9093
<http://dev-pf-kafka01.local.site.com:9093>
(id: 2147483646 rack: null) is either not started or not valid
2021-11-16 17:44:23.373 INFO 1 --- [-card-requests]]
o.a.k.c.c.internals.AbstractCoordinator : [Consumer
clientId=consumer-card-products-2, groupId=card-products] Group
coordinator dev-pf-kafka01.local.site.com:9093
<http://dev-pf-kafka01.local.site.com:9093> (id: 2147483646
rack:
null) is unavailable or invalid due to cause: error response
NOT_COORDINATOR.isDisconnected: false. Rediscovery will be
attempted.
2021-11-16 17:44:23.774 INFO 1 --- [-card-requests]]
o.a.k.c.c.internals.AbstractCoordinator : [Consumer
clientId=consumer-card-products-2, groupId=card-products]
Discovered
group coordinator dev-pf-kafka02.local.site.com:9093
<http://dev-pf-kafka02.local.site.com:9093> (id: 2147483645
rack: null)
2021-11-16 17:44:23.800 INFO 1 --- [-card-requests]]
o.a.k.c.c.internals.AbstractCoordinator : [Consumer
clientId=consumer-card-products-2, groupId=card-products]
Discovered
group coordinator dev-pf-kafka02.local.site.com:9093
<http://dev-pf-kafka02.local.site.com:9093> (id: 2147483645
rack: null)
2021-11-16 17:49:34.004 INFO 1 --- [-card-requests]]
o.a.kafka.clients.FetchSessionHandler : [Consumer
clientId=consumer-card-products-2, groupId=card-products]
Node 3 was
unable to process the fetch request with (sessionId=25720628,
epoch=1027527): FETCH_SESSION_ID_NOT_FOUND
It was QA stage but... It was actually unexpected.
Is there any possibility to prevent such behavior in future?
How to
check liveness or how to reconnect to Kafka automatically?
My route may be simplified into:
from("kafka://start)....to("kafka://notification");
I'm confused about such reaction of kafka connector...
--
Vyacheslav Boyko
mailto:mail4...@gmail.com
--
Vyacheslav Boyko
mailto:mail4...@gmail.com <mailto:mail4...@gmail.com>
--
Vyacheslav Boyko
mailto:mail4...@gmail.com
