can one topic be registered in multiple brokers? if so, which component of
kafka decides which broker should get the message for that particular topic?
Thanks!
You can create multiple partitions of a topic and kafka will attempt to
distribute them evenly.
E.g if you have 3 brokers and you create 3 partitions of a topic, each
broker will be the leader of 1 of the 3 partitions.
P.S how did the benchmarking go?
On Sat, Sep 17, 2016 at 1:36 PM, kant kodali
so Zookeeper will select which broker it should direct the message to if I have
3 brokers and 3 partitions of a topic?
I only finished the benchmarks of Kafka and NSQ and still working NATS Streaming
Server (one more day I will finish it). But so far Kafka had a great throughput
with Java Client ab
why did Kafka choose pull instead of push for a consumer? push sounds like it
is more realtime to me than poll and also wouldn't poll just keeps polling even
when they are no messages in the broker causing more traffic? please enlighten
me
I'm only guessing here regarding if this is the reason:
Pull is much more sensible when a lot of data is pushed through. It allows
consumers consuming at their own pace, slow consumers do not slow the complete
system down.
--
Best regards,
Rad
On Sat, Sep 17, 2016 at 11:18 AM +0200, "kant k
Did you try props.put("group.id", "test");
On Thu, Sep 15, 2016 at 12:55 AM, Joyce Chen wrote:
> Hi,
>
> I created a few consumers that belong to the same group_id, but I noticed
> that each consumer get all messages instead of only some of the messages.
>
> As for the topic, I did create the t
Kafka uses murmur2 key hashing by default. You can also create your own custom
partitioner. The partitioner can be set on per producer / consumer basis.
--
Best regards,
Rad
On Sat, Sep 17, 2016 at 11:01 AM +0200, "kant kodali"
wrote:
so Zookeeper will select which broker it shoul
Hi Caleb,
I usually do './gradlew installAll' first and that places all the jars in my
local maven repo in ~/.m2/repository.
Eno
> On 17 Sep 2016, at 00:30, Caleb Welton wrote:
>
> Is there a specific way that I need to build kafka for that to work?
>
> bash$ export INCLUDE_TEST_JARS=true;
Hmm...Looks like Kafka is written in Scala. There is this thing called reactive
streams where a slow consumer can apply back pressure if they are consuming
slow. Even with Java this is possible with a Library called RxJava and these
ideas will be incorporated in Java 9 as well.
I still don't see wh
Kafka is not a queue. It’s a distributed commit log.
–
Best regards,
Radek Gruchalski
ra...@gruchalski.com
On September 17, 2016 at 9:23:09 PM, kant kodali (kanth...@gmail.com) wrote:
Hmm...Looks like Kafka is written in Scala. There is this thing called
reactive
streams where a slow consumer c
Still it should be possible to implement using reactive streams right.
Could you please enlighten me on what are the some major differences you see
between a commit log and a message queue? I see them being different only in the
implementation but not functionality wise so I would be glad to hear y
There are two distinct questions...
1. Regarding reactive streams, Akka has an implementation for Kafka:
https://github.com/akka/reactive-kafka
2. Kafka is not a queue. For example, it does not implement "dequeue"
operation.
All the message management / retention is not based on whether a message
w
Please read this article:
https://engineering.linkedin.com/distributed-systems/log-what-every-software-engineer-should-know-about-real-time-datas-unifying
–
Best regards,
Radek Gruchalski
ra...@gruchalski.com
On September 17, 2016 at 9:49:43 PM, kant kodali (kanth...@gmail.com) wrote:
Still it
+2 watching.
On Sat, Sep 17, 2016 at 2:45 AM, kant kodali wrote:
> https://issues.apache.org/jira/browse/KAFKA-1793
> It would be great to use Consul instead of Zookeeper for Kafka and I think
> it
> would benefit Kafka a lot from the exponentially growing consul community.
--
Jennifer Fo
14 matches
Mail list logo