Hi John
Thanks for your message.
Indeed there's a potential feature here. Should I log it on Jira ?
Mathieu
Le ven. 20 nov. 2020 à 21:53, John Roesler a écrit :
> Hi Mathieu,
>
> Ah, that is unfortunate. I believe your analysis is correct. In general,
> we have no good solution to the problem
Hey all,
We have been running a kafka-stream based service in production for the
last couple of years (we have 4 brokers on this specific cluster).
Inside this service, which does a lot of stuff - there are 2 GlobalKTables
(which are based on 2 compacted topics).
When I restart my app and clean th
Hi There,
we are facing following issue in kafka. We have gone through some of the
references and found it as a bug from kafka side. Please let us know if there
are any solutions.
│ 47] [PID:1] Producer send message failed, err:kafka server: In the middle of
a leadership election, there is cur
Hello Kafka users, developers and client-developers,
This is the second candidate for release of Apache Kafka 2.6.1.
Since RC0, the following JIRAs have been fixed: KAFKA-10705,
KAFKA-10755 and KAFKA-10706
Release notes for the 2.6.1 release:
https://home.apache.org/~mimaison/kafka-2.6.1-rc1/REL
Thanks Liam!
We don't have a requirement to maintain order of processing for events even
within a partition. Essentially, these are events for various accounts
(customers) that we want to support and do necessary database provisioning
for those in our database. So they can be processed in parallel.
Hi Pushkar.
Just for your information, https://github.com/line/decaton is a Kafka
consumer framework that supports parallel processing per single partition.
It manages committable (i.e. the offset that all preceding offsets have
been processed) offset internally so that preserves at-least-once se
Thanks Haruki... right now the max of such types of events that we would
have is 100 since we would be supporting those many customers (accounts)
for now, for which we are considering a simple approach of a single
consumer and a thread pool with around 10 threads. So the question was
regarding how
Hey Ben,
I think the distinction between your and Matthias' points is analogous to,
borrowing from the OSI model, transport- vs presentation/application-level
concerns. When configured appropriately, Kafka durably provides transport
for messages from point A to point B, with such optional goodies
Hello Nitay,
Thanks for letting us know about your observations. When you restart the
application from empty local state Kafka Streams will try to restore all
the records up to the current log end for those global KTables, and during
this period of time there should be no processing.
Do you mind
Hello I try take metrics for bytes-consumed-rate . Bu t I have the following
problem.
I write the command
JMX_PORT= bin/kafka-run-class.sh kafka.tools.JmxTool --object-name
kafka.consumer:type=consumer-fetch-manager-metrics,client-id="([-.w]+)",name=bytes-consumed-rate
and I take the error
No
Hi Mohammad,
Leader elections are generally seldom and usually quick.
If a leader is unable to elected, this could point to a) replication being
set too low or b) inter-broker communication issues, plus a bunch of other
possible problems.
You'd have to look into your Kafka broker's logs to figur
Hello I try to take bytes-consumed-metrics metrics via JMX Mbeans but i have
the following issue.
I write the command
JMX_PORT= bin/kafka-run-class.sh kafka.tools.JmxTool --object-name
kafka.consumer:type=consumer-fetch-manager-metrics,client-id="([-.w]+)",name=bytes-consumed-rate
But I take t
I see.
Then I think the appropriate approach depends on your delivery latency
requirements.
Just retrying until success is simpler but it could block subsequent
messages to get processed. (also depends on thread pool size though)
Then another concern when using dead letter topic would be retrying
Hi Sunil,
Thanks for your reply.
all four words commands where giving connection refused.
I moved files (log.1,log.103,log.142,log.16a,log.17d - all where having oct
timestamp) present inside `/var/data/logs/zookeeper/version-2/` to some other
directory and started zookeeper and then kafka
I think we can handle the failures selectively e.g. if there are issues
with downstream database server then all the events will fail to process so
it will be worth to keep retrying. Else if there is issue only while
processing a particular event, then we can keep retry timeout and after
that timeo
15 matches
Mail list logo