Re: Error for partition [__consumer_offsets,15] to broker

This is a known issue for us in 0.10 due to network related problems with ZK causing no leader exception and restarting quickly fixed it. You can increase time out to alleviate the problem a bit. On Dec 8, 2017 8:20 PM, "Abhit Kalsotra" wrote: > Guys can I get any reply of

Re: Error for partition [__consumer_offsets,15] to broker

Guys can I get any reply of help on the same.. this has been occuring very frequently in my production environment.. Please help.. Abhi On Dec 6, 2017 13:24, "Abhit Kalsotra" wrote: > Hello * > > I am running Kafka(*0.10.2.0*) on windows from the past one year ... > > But

Re: Error for partition [__consumer_offsets,15] to broker

And this is my typical broker config broker.id=0 port:9093 num.network.threads=3 num.io.threads=8 socket.send.buffer.bytes=102400 socket.receive.buffer.bytes=102400 socket.request.max.bytes=104857600 offsets.retention.minutes=360 advertised.host.name=1.1.1.2 advertised.port:9093 ctories under

Re: How can I repartition/rebalance topics processed by a Kafka Streams topology?

Hard to give a generic answer. 1. We recommend to over-partitions your input topics to start with (to avoid that you need to add new partitions later on); problem avoidance is the best strategy. There will be some overhead for this obviously on the broker side, but it's not too big. 2. Not sure

Re: Configuration: Retention and compaction

It does not. The current segment is open for writing and only closed (ie, rolled) segments are considered. There is a bunch of broker/topic configs that you can play with to influence log rolling and compaction. -Matthias On 12/8/17 10:53 AM, Dmitry Minkovsky wrote: > Matthias, you read my

How can I repartition/rebalance topics processed by a Kafka Streams topology?

I am about to put a topology into production and I am concerned that I don't know how to repartition/rebalance the topics in the event that I need to add more partitions. My inclination is that I should spin up a new cluster and run some kind of consumer/producer combination that takes data from

Re: Configuration: Retention and compaction

Matthias, you read my mind—having examined Kafka Streams intermediate topic configs and then Googled my way to KIP-71 , I was confused about this dual policy. Thank you. Still wondering

Re: Kafka Monitoring

Hi, We have no modifications in that file - what we do is having a "wrapper" that's just a Docker "entrypoint" (just a bash script) which contents is: export KAFKA_OPTS="$KAFKA_OPTS -javaagent:/jolokia-jvm-agent.jar=port=8074,host=0.0.0.0" exec ${KAFKA_DIR}/bin/kafka-server-start.sh

Re: Kafka Monitoring

Thanks Michal, can you kindly send me you kafka-run-class.sh and kafka-server-start.sh file, I have look what have you done. Because I have done same thing that you explained above but when i do this < http://localhost:/jolokia/list> i get only metrics for the zookeeper but not the above

Re: Running Kafka 1.0 binaries with inter.broker.protocol.version = 0.10

Can anyone let me know if I set both inter.broker.protocol.version & log.message.format.version to 0.10 with the updated 1.0 binaries ? How are the Kafka brokers supposed to behave? On Thu, Dec 7, 2017 at 5:10 PM, Debraj Manna wrote: > Hi > > Anyone any thoughts on my

Re: Kafka Monitoring

Hi Irtiza, I don't have any tutorial, but I can tell you what we do :-) First of all we have Jolokia agent jar included in our Kafka Docker image. Then we append this to KAFKA_OPTS -javaagent:/jolokia-jvm-agent.jar=port=8074,host=0.0.0.0 Relevant port is then "exposed" in Docker image and

Re: Kafka Monitoring

Hello Michal, Can you send me link to tutorial or provide some resources for the Jolokia configuration with kafka> Thank you Irtiza On Wed, Dec 6, 2017 at 8:00 PM, Michal Michalski < michal.michal...@zalando.ie> wrote: > Hi Irtiza, > > We're using Jolokia and we had no problems with it. > It

Re: Kafka Monitoring

thank you subhash. I will check it out On Wed, Dec 6, 2017 at 5:43 PM, Subhash Sriram wrote: > Hi Irtiza, > > Have you looked at jmxtrans? It has multiple output writers for the > metrics and one of them is the KeyOutWriter which just writes to disk. > >