Sounds good.
Yes, I'd want a guarantee that every future I get will always return the
recordmeta or an exception eventually.
Running into a similar issue with futures never returning with a pretty
straightforward case:
- Healthy producer/server setup
- Stop the server
- Send a message
- Call get
Did you see my response and have you checked the server logs especially the GC
logs? It still sounds like you are running out of memory on the broker. What is
your max heap memory and are you thrashing once you start writing to all those
partitions?
You have measured very thoroughly from an
I'll take a look at the GC profile of the brokers Right now I keep a tab on
the CPU, Messages in, Bytes in, Bytes out, free memory (on the machine not
JVM heap) free disk space on the broker. I'll need to take a look at the
JVM metrics too. What seemed strange is that going from 8 - 512 partitions
Ah I thought it was restarting the broker that made things better :)
Yeah I have no experience with the Java client so can't really help there.
Good luck!
-Original Message-
From: Rajiv Kurian [ra...@signalfuse.com]
Received: Sunday, 21 Dec 2014, 12:25PM
To: users@kafka.apache.org
I have a requirement to prove kafka producer can produce 1 million
events/second to Kafka cluster.
So far, best I could achieve is 200k events/sec on topic with 2 partitions.
The latency increases with adding more partitions so I want to test with 2
partitions for now.
Below are the details
*Kafka: *Apache Kafka 0.8.1.1
*SImplePartitioner.java*
public int partition(Object key, int a_numPartitions) {
int partition = Integer.parseInt((String)key);
LOG.debug(SimplePartitioner Partion: + partition);
return partition;
}
On Sun, Dec 21, 2014 at 10:54
FYI, I bumped server to 0.8.2-beta and I don't hit the basic failure I
mentioned above, which is great.
I haven't been able to find confirmation in the docs, but from past
conversation(
Hi,
I was trying out Kafka .8 and was using PHP kafka(
https://github.com/salebab/phpkafka ) extension to produce in PHP
Unfortunately it does not produce faster than a 3 messages per second
reliably.
Was wondering if anyone can help me speed this up.
Thanks,
Saurabh
