Guozhang,
I set this in my producer.properties
topic.metadata.refresh.interval.ms=1000
Then I start the console producer as
./kafka-console-producer.sh --broker-list localhost:9092 --topic test2
I still dont see data being written to different partitions after every 1
second.
I wonder if the
In console producer you can specify the producer properties in command line
as metadata-expiry-ms.
You can type just ./kafka-console-producer.sh and it will show you all the
configs that you can specify.
Guozhang
On Wed, Jun 11, 2014 at 10:56 AM, Prakash Gowri Shankor
Can you please tell me how to set this property ?
topic.metadata.refresh.interval.ms
Is a value of 100 low enough to solve this issue ?
Im guessing I can set it to 100 and restart the command line producer and
the partitioning should work ? Please confirm.
Thanks
On Mon, Jun 9, 2014 at 5:09 PM,
Yes, reducing the refresh interval to 100ms will cause it to try to select
another partition every 100ms, not necessarily a different partition tough,
since it just gets a next random int % num.partitions.
Setting the key can also resolve this issue, as long as the key values are
evenly
Hi,
This is with 0.8.1.1 and I ran the command line console consumer.
I have one broker, one producer and several consumers. I have one topic,
many partitions m, many consumers n, m=n , one consumer group defined for
all the consumers
From using Kafka Monitor, I see that each partition is
Last time I've checked it, producer sticks to partition for 10 minutes.
On Mon, Jun 9, 2014 at 4:13 PM, Prakash Gowri Shankor
prakash.shan...@gmail.com wrote:
Hi,
This is with 0.8.1.1 and I ran the command line console consumer.
I have one broker, one producer and several consumers. I have
Is there a way to modify this duration ? This is not adhering to the
random behavior that the documentation talks about.
On Mon, Jun 9, 2014 at 4:41 PM, Kane Kane kane.ist...@gmail.com wrote:
Last time I've checked it, producer sticks to partition for 10 minutes.
On Mon, Jun 9, 2014 at 4:13
Kane is right, please see this FAQ for details:
https://cwiki.apache.org/confluence/display/KAFKA/FAQ#FAQ-Whyisdatanotevenlydistributedamongpartitionswhenapartitioningkeyisnotspecified
?
On Mon, Jun 9, 2014 at 4:41 PM, Kane Kane kane.ist...@gmail.com wrote:
Last time I've checked it, producer
I have seen that mail thread. Here is what i tried:
In my producer.properties I set topic.metadata.refresh.interval.ms=1000. I
guess this means that the a different partition will be selected every
second.
Then I restart my producer as : ./kafka-console-producer.sh --broker-list
localhost:9092
In the new producer we are changing the default behavior back to pure
random partitioning and let users to customize their own partitioning
schemes if they want. For now reducing topic.metadata.refresh.interval.ms
should help because the stickiness only persists until a metadata refresh.
Guozhang
Thank you Guozhang.
I've specified how i set and use the property in my previous mail. Can you
tell me if that is fine ?
I also noticed that the kafka-console-producer.sh takes a custom
property(key-value) on the command line. Would it help to set this property
directly on the command line of the
11 matches
Mail list logo
