Hi ,
I am using Kafka 2.10_8.1.1 in the local mode , multi broker scenario.
I wanted to establish how reliable is Kafka when it somes to guaranteed
messaging , so i came with following results. wanted to know if this
behaviour will be same distributed environment.
The results are as follows :
1. W
Hi,
> 4. but when i close my Ubuntu virtual machine and i start Kafka and
> Zookeeper again. I can see the topics present in the kafka cluster. However
> all are default to 0 offset and data seems to have been lost.
By default kafka stores the data in /tmp/kafka-logs directory.
When you restar
Thanks for the reply. If I use it only for activity streams like twitter:
I would want a topic for each #tag and a topic for each user and maybe
foreach city. Would that be too many topics or it doesn't matter since most
of them will be deleted in a specified interval.
Best Regards,
Aris Giachn
Thanks for the reply. If I use it only for activity streams like twitter:
I would want a topic for each #tag and a topic for each user and maybe
foreach city. Would that be too many topics or it doesn't matter since most
of them will be deleted in a specified interval.
Best Regards,
Aris Giachn
I'm not really sure about your exact use-case but I don't think having a
topic per user is very efficient. Deleting topics in kafka, at the moment,
isn't really straightforward. You should rethink your date pipeline a bit.
Also, just because kafka has the ability to store messages for a certain
ti
Yes, IMHO, that is going to be way too many topics. Use a smaller number of
topics, and embedded attributes like "tag" and "user" in the messages written
to Kafka.
Phiilp
-
http://www.philipotoole.com
On Friday, September 5, 2014 4:21 AM, Sharninder
Especially since the same code will need to process all these messages.
Topics are typically used to separate messages between apps, modules,
etc. If all your apps consume all the topics, there's something wrong
in the design.
Gwen
On Fri, Sep 5, 2014 at 9:31 AM, Philip O'Toole
wrote:
> Yes, IMH
+1
Topic Deletion with 0.8.1.1 is extremely problematic, and coupled with the fact
that rebalance/broker membership changes pay a cost per partition today,
whereby excessive partitions extend downtime in the case of a failure; this
means fewer topics (e.g. hundreds or thousands) is a best pract
The thought experiment I did ended up having a set of front end servers
corresponding to a given chunk of the user id space, each of which was a
separate subscriber to the same set of partitions. The you have one or
more partitions corresponding to that same chunk of users. You want the
chunk/set o
Dear cristian
You seem to have spent considerable time in thinking about this.
I thank you a lot and will study it to take ideas.
If you didn't then wow congrats you have a fast cpu there :)
Aris
On Sep 5, 2014 7:25 PM, "Christian Csar" wrote:
> The thought experiment I did ended up having a s
10 matches
Mail list logo
