Hi,
We are using confluent 0.9.0.1 and we’ve noticed recently that we lost
__consumer_offsets and _schemas (used for schema registry) topic level config.
We checked zookeeper config/topics/ and found that there is no
__consumer_offsets and _schemas topic. Our server level config use cleanup
po
The problem is that Kafka Streams need to repartition the streams based on
the groupBy keys when doing aggregations. For your case, the original
stream may be read from a topic that is partitioned on "K", and you need to
first repartition on "category" on an intermediate topic before the
aggregatio
Jarko,
Do you have many topic partitions? Currently, if #partitions *
fetched_bytes in the response exceeds 2GB, we will get an integer overflow
and weird things can happen. We are trying to address this better in
KIP-74. If this is the issue, for now, you can try reducing the fetch size
or increa
Hi Guozhang,
Thanks for responding. Ah, I see what you're saying... in the case of
an update to the KTable, the aggregator's subtractor result would be
necessary if the group-by key changes in the update.
It makes sense, but unfortunately the behavior leaves me feeling a
little sketchy... when t
Hello Mathieu,
Note that semantics of KTable aggregations (i.e. "KTable.groupBy.aggregate"
as in 0.10.0) and KStream aggregations (i.e. "KStream.aggregateByKey" as in
0.10.0) are different, in the sense that when the table is updated (i.e. a
new record with the same key "K1" is received), the old
Yes, you can try setting it to -1 in 0.8.1, which is the equivalent of
"all" in 0.9 and above.
Thanks,
Jun
On Wed, Aug 17, 2016 at 8:32 AM, Mazhar Shaikh
wrote:
> Hi Jun,
>
> I'm using default configuration (ack=1),
> changing it t0 all or 2 will not help, as the producer queue will be
> exhau
Hi Jun,
I'm using default configuration (ack=1),
changing it t0 all or 2 will not help, as the producer queue will be
exhausted is any kafka broker goes down for long time.
Thanks.
Regards,
Mazhar Shaikh.
On Wed, Aug 17, 2016 at 8:11 PM, Jun Rao wrote:
> Are you using acks=1 or acks=all in
Hello again, kafka-users,
When I aggregate a KTable, a future input that updates a KTable's
value for a specific key causes the aggregate's subtractor to be
invoked, and then its adder. This part is great, completely
as-expected.
But what I didn't expect is that the intermediate result of the
su
Are you using acks=1 or acks=all in the producer? Only the latter
guarantees acked messages won't be lost after leader failure.
Thanks,
Jun
On Wed, Aug 10, 2016 at 11:41 PM, Mazhar Shaikh
wrote:
> Hi Kafka Team,
>
> I'm using kafka (kafka_2.11-0.9.0.1) with librdkafka (0.8.1) API for
> produce
Hi Tom,
Thank you for responding and sorry for delay.
I'm running with all the default configuration provided by kafka.
I don't have this config elements in my server.properties file.
However the default values specified in kafka documentation are as below (
http://kafka.apache.org/documentat
Hello,
I have a cluster of 3 nodes running kafka v.0.10.0.0. This cluster was starter
about a week ago with no data, no issues starting up.
Today we noticed 1 of the servers in the cluster did not work anymore, we
checked and indeed the server was not working anymore and all data was old.
We re
You can just consume the message once, and process it twice downstream:
> KStream stream = builder.stream("topic");
>
> stream.process(/* use P1 here */);
> stream.process(/* use P2 here */);
-Matthias
On 08/17/2016 07:51 AM, Tanay Soni wrote:
> Hi,
>
> I am wondering why not have two differen
12 matches
Mail list logo
