Hi Monty,
I'm also planning to use a big amount of topics in Kafka, so recently I
made a test within a 3 nodes kafka cluster where I created 100k topics with
one partition. Sent 1M messages in total.
These are my conclusions:
- There is not any limitation on kafka regarding the number of topic
Guozhang,
I read the comments in KAFKA-6376 and realized that we do something similar
to handle exception in process().
We register a custom metric "process-skipped" and our alerting system looks
for both this and skipped-records.
Srikanth
On Fri, Jan 26, 2018 at 10:26 PM, Guozhang Wang wrote:
Guozhang,
While I understand that this metric is meaningless when handler is set to
FAIL, in my case I'm actually using LogAndContinueExceptionHandler.
In this case, app needs to report such occurrences. What I noticed is that
only skipped-records is set.
The granularity offered by skippedDueToDes
Changing the class operators are nested in can break compatibility with
existing savepoints. The following piece of code demonstrates this
https://gist.github.com/jelmerk/e55abeb0876539975cd32ad0ced8141a
If I change Operators in this file to Operators2 i will not be able to
recover from a savepo
Hello Srikanth,
Which version of Kafka are you using? I'd like to dig for that particular
branch again.
Guozhang
On Sun, Jan 28, 2018 at 8:54 AM, Srikanth wrote:
> Guozhang,
>
> While I understand that this metric is meaningless when handler is set to
> FAIL, in my case I'm actually using LogA
the command line tools should be using config/tools-log4j.properties which
should have log level set to WARN, not INFO
config/log4j.properties is used by the broker and by default is set to INFO
bin/kafka-run-class.sh will default to the first one if KAFKA_LOG4J_OPTS
isn't set.
# Log4j settings
i
Kafka-streams version "1.0.0".
Thanks,
Srikanth
On Mon, Jan 29, 2018 at 12:23 AM, Guozhang Wang wrote:
> Hello Srikanth,
>
> Which version of Kafka are you using? I'd like to dig for that particular
> branch again.
>
> Guozhang
>
> On Sun, Jan 28, 2018 at 8:54 AM, Srikanth wrote:
>
> > Guozhan
Matthias,
We moved to streams client 1.0.0 with 0.10.2.x broker now. Haven't see this
issue ever since.
Thanks!
Srikanth
On Thu, Dec 7, 2017 at 12:15 AM, Matthias J. Sax
wrote:
> Running Streams 1.0.0 should just work with 0.10.2 brokers. Of course,
> you can't use EOS feature.
>
> Not sure ho
