Re: Supervisord for Kafka 0.8.1

2016-03-25 Thread Kashyap Mhaisekar
> export KAFKA_HEAP_OPTS="-Xms8g -Xmx8g" > > fi > > > > if [ "abc$KAFKA_JVM_PERFORMANCE_OPTS" == "abc" ]; then > > export KAFKA_JVM_PERFORMANCE_OPTS="-server -XX:+UseG1GC > > -XX:MaxGCPauseMillis=20 -XX:InitiatingHeapOccupancyPerce

Supervisord for Kafka 0.8.1

2016-03-25 Thread Kashyap Mhaisekar
Hi, Am having trouble configuring Kafka server starts with supervisord. Has anyone from this group succeeded in integrating Kafka server start and stop via supervisord? Can you please share the snippet of his out of configured? Thanks Kashyap

Re: Keeping Zookeeper and Kafka Server Up

2015-06-17 Thread Kashyap Mhaisekar
We use supervisord for this. It ensures that the processes are always up and running. Thanks Kashyap On Wednesday, June 17, 2015, Shayne S shaynest...@gmail.com wrote: kafka-server-start.sh has a -daemon option, but I don't think Zookeeper has it. On Tue, Jun 16, 2015 at 11:32 PM, Su She

Re: Security in 0.8.2 beta

2014-11-12 Thread Kashyap Mhaisekar
/ On Wed, Nov 12, 2014 at 2:41 AM, Mathias Herberts mathias.herbe...@gmail.com wrote: Simply encrypt your messages with a PSK between producers and consumers. On Nov 12, 2014 4:38 AM, Kashyap Mhaisekar kashya...@gmail.com wrote: Hi, Is there a way to secure

Security in 0.8.2 beta

2014-11-11 Thread Kashyap Mhaisekar
Hi, Is there a way to secure the topics created in Kafka 0.8.2 beta? The need is to ensure no one is asked to read data from the topic without authorization. Regards Kashyap

Partition reassign Kafka 0.8.1.1

2014-06-27 Thread Kashyap Mhaisekar
Hi, I was testing out Kafka 0.8.1.1 and found that i get the following exception during partition re-assignment : *./kafka-reassign-partitions.sh --path-to-json-file ritest.json --zookeeper localhost:2181* *Partitions reassignment failed due to Partition reassignment currently in progress for

Re: Loss of Leader in Kafka

2014-05-13 Thread Kashyap Mhaisekar
it in trunk. Could you give it a try and see if you see the same issue? Thanks, Jun On Mon, May 12, 2014 at 9:39 AM, Kashyap Mhaisekar kashya...@gmail.comjavascript:; wrote: Hi, I am hitting a strange exception while creating a topic in Kafka - Steps to generate this- 1. Created

Loss of Leader in Kafka

2014-05-12 Thread Kashyap Mhaisekar
Hi, I am hitting a strange exception while creating a topic in Kafka - Steps to generate this- 1. Created a topic multipartition_test with 2 partitions and 2 replicas 2. Added some data to this topics and verified data is coming up for both partitions 3. Deleted the topic. Checked only the

Re: Log Retention in Kafka

2014-05-05 Thread Kashyap Mhaisekar
#basic_ops_modify_topic Thanks, Jun On Sun, May 4, 2014 at 10:11 PM, Kashyap Mhaisekar kashya...@gmail.com wrote: Is there a way to do this at runtime using some available scripts in kafka/bin? If so, any pointers on which script? Regards, Kashyap On Tue, Apr 22, 2014 at 11:11 PM

Re: Log Retention in Kafka

2014-05-04 Thread Kashyap Mhaisekar
Is there a way to do this at runtime using some available scripts in kafka/bin? If so, any pointers on which script? Regards, Kashyap On Tue, Apr 22, 2014 at 11:11 PM, Kashyap Mhaisekar kashya...@gmail.comwrote: Thanks Joel. Am using version 2.8.0. Thanks, Kashyap On Tue, Apr 22, 2014

Log Retention in Kafka

2014-04-22 Thread Kashyap Mhaisekar
Hi, I wanted to set the message expiry for a message on a kafka topic. Is there anything like this in kafka? I came across a property - *log.retention.hours* and *topic.log.retention.hours* Had some queries around it.And it was mentioned that topic.log.retention.hours is per topic configuration.

Re: Log Retention in Kafka

2014-04-22 Thread Kashyap Mhaisekar
time-based retention using log.retention.minutes which will apply to all topics. You can override that on per-topic basis - see further down in the above page under topic-level configuration On Tue, Apr 22, 2014 at 02:34:24PM -0500, Kashyap Mhaisekar wrote: Hi, I wanted to set the message

Leader: -1 on Kafka

2014-04-21 Thread Kashyap Mhaisekar
Hi, At times, some of kafka topics end up showing the leader as -1. After this, the messages dont get added to the topic nor consumed. I tried digging into why the leader turns -1 *(leader: -1)* Is there a reason why this happens and how it can be resolved? Regards, kashyap

Re: Leader: -1 on Kafka

2014-04-21 Thread Kashyap Mhaisekar
-state-change.log for that topic/partition? On Mon, Apr 21, 2014 at 03:36:52PM -0500, Kashyap Mhaisekar wrote: Yes. I have 3 kafka brokers and I have created one topic with 1 partition and 1 replication (default options). All params are default. I was testing it with heavy load and had