Hi,
@Manoj/@Pete: Thanks for the inputs. I am already aware of the parallelism
provided by kafka. My use case needed single topic per user, but I came up
with a workaround for that and so the problem is solved.
@Guozhang: I agree, kafka stores data related to partitions and topics(
https://cwiki.
Siva,
For Kafka brokers as long as the total #.partitions is not too much I think
it should be fine (we have been hosting 600+ partitions on a single node).
You have to pay some attention on your ZK nodes though since with #.topics
increasing its metadata storage will take more and more space.
Gu
I meant you can read messages multiple times if you want to.
Yes, you would store offset and request reading from an offset with Simple
Consumer API to implement once and only once delivery.
regards
On Wed, Apr 15, 2015 at 10:55 AM, Pete Wright
wrote:
>
>
> On 04/15/15 09:31, Manoj Khangaonkar
On 04/15/15 09:31, Manoj Khangaonkar wrote:
> # I looked the documents of kafka and I see that there is no way a
>> consume instance can
>>read specific messages from partition.
>>
>
> With Kafka you read messages from the beginning multiple times. Since you
> say later that
> you do
# I looked the documents of kafka and I see that there is no way a
> consume instance can
>read specific messages from partition.
>
With Kafka you read messages from the beginning multiple times. Since you
say later that
you do not have many messages per topic, you can iterate over the
Hi,
# I looked the documents of kafka and I see that there is no way a
consume instance can
read specific messages from partition.
# I have an use case where I need to spawn a topic(single partition)
for each user,
so there would be 10k online users at a time, there would be
