Re: Log Retention in Kafka
See http://kafka.apache.org/documentation.html#basic_ops_modify_topic Thanks, Jun On Sun, May 4, 2014 at 10:11 PM, Kashyap Mhaisekar kashya...@gmail.comwrote: Is there a way to do this at runtime using some available scripts in kafka/bin? If so, any pointers on which script? Regards, Kashyap On Tue, Apr 22, 2014 at 11:11 PM, Kashyap Mhaisekar kashya...@gmail.com wrote: Thanks Joel. Am using version 2.8.0. Thanks, Kashyap On Tue, Apr 22, 2014 at 5:53 PM, Joel Koshy jjkosh...@gmail.com wrote: Which version of Kafka are you using? You can read up on the configuration options here: http://kafka.apache.org/documentation.html#configuration You can specify time-based retention using log.retention.minutes which will apply to all topics. You can override that on per-topic basis - see further down in the above page under topic-level configuration On Tue, Apr 22, 2014 at 02:34:24PM -0500, Kashyap Mhaisekar wrote: Hi, I wanted to set the message expiry for a message on a kafka topic. Is there anything like this in kafka? I came across a property - *log.retention.hours* and *topic.log.retention.hours* Had some queries around it.And it was mentioned that topic.log.retention.hours is per topic configuration. Had some queries around it - 1. Does it mean that I need to specific the topicname.log.retention.hours in the kafka config? 2. Can this property be overriden anywhere? 3. Is it possible for the producer to set a message expiry so that the message expires after a configurable period of time? Regards, Kashyap
Re: Log Retention in Kafka
Thanks Jun. I am using kafka_2.8.0-0.8.0-beta1.jar. I dont see this script kafka-topic.sh in the bin folder. Is there way to do this in the version mentioned? --Kashyap On Mon, May 5, 2014 at 11:18 AM, Jun Rao jun...@gmail.com wrote: See http://kafka.apache.org/documentation.html#basic_ops_modify_topic Thanks, Jun On Sun, May 4, 2014 at 10:11 PM, Kashyap Mhaisekar kashya...@gmail.com wrote: Is there a way to do this at runtime using some available scripts in kafka/bin? If so, any pointers on which script? Regards, Kashyap On Tue, Apr 22, 2014 at 11:11 PM, Kashyap Mhaisekar kashya...@gmail.com wrote: Thanks Joel. Am using version 2.8.0. Thanks, Kashyap On Tue, Apr 22, 2014 at 5:53 PM, Joel Koshy jjkosh...@gmail.com wrote: Which version of Kafka are you using? You can read up on the configuration options here: http://kafka.apache.org/documentation.html#configuration You can specify time-based retention using log.retention.minutes which will apply to all topics. You can override that on per-topic basis - see further down in the above page under topic-level configuration On Tue, Apr 22, 2014 at 02:34:24PM -0500, Kashyap Mhaisekar wrote: Hi, I wanted to set the message expiry for a message on a kafka topic. Is there anything like this in kafka? I came across a property - *log.retention.hours* and *topic.log.retention.hours* Had some queries around it.And it was mentioned that topic.log.retention.hours is per topic configuration. Had some queries around it - 1. Does it mean that I need to specific the topicname.log.retention.hours in the kafka config? 2. Can this property be overriden anywhere? 3. Is it possible for the producer to set a message expiry so that the message expires after a configurable period of time? Regards, Kashyap
Re: Log Retention in Kafka
Is there a way to do this at runtime using some available scripts in kafka/bin? If so, any pointers on which script? Regards, Kashyap On Tue, Apr 22, 2014 at 11:11 PM, Kashyap Mhaisekar kashya...@gmail.comwrote: Thanks Joel. Am using version 2.8.0. Thanks, Kashyap On Tue, Apr 22, 2014 at 5:53 PM, Joel Koshy jjkosh...@gmail.com wrote: Which version of Kafka are you using? You can read up on the configuration options here: http://kafka.apache.org/documentation.html#configuration You can specify time-based retention using log.retention.minutes which will apply to all topics. You can override that on per-topic basis - see further down in the above page under topic-level configuration On Tue, Apr 22, 2014 at 02:34:24PM -0500, Kashyap Mhaisekar wrote: Hi, I wanted to set the message expiry for a message on a kafka topic. Is there anything like this in kafka? I came across a property - *log.retention.hours* and *topic.log.retention.hours* Had some queries around it.And it was mentioned that topic.log.retention.hours is per topic configuration. Had some queries around it - 1. Does it mean that I need to specific the topicname.log.retention.hours in the kafka config? 2. Can this property be overriden anywhere? 3. Is it possible for the producer to set a message expiry so that the message expires after a configurable period of time? Regards, Kashyap
Log Retention in Kafka
Hi, I wanted to set the message expiry for a message on a kafka topic. Is there anything like this in kafka? I came across a property - *log.retention.hours* and *topic.log.retention.hours* Had some queries around it.And it was mentioned that topic.log.retention.hours is per topic configuration. Had some queries around it - 1. Does it mean that I need to specific the topicname.log.retention.hours in the kafka config? 2. Can this property be overriden anywhere? 3. Is it possible for the producer to set a message expiry so that the message expires after a configurable period of time? Regards, Kashyap
Re: Log Retention in Kafka
Which version of Kafka are you using? You can read up on the configuration options here: http://kafka.apache.org/documentation.html#configuration You can specify time-based retention using log.retention.minutes which will apply to all topics. You can override that on per-topic basis - see further down in the above page under topic-level configuration On Tue, Apr 22, 2014 at 02:34:24PM -0500, Kashyap Mhaisekar wrote: Hi, I wanted to set the message expiry for a message on a kafka topic. Is there anything like this in kafka? I came across a property - *log.retention.hours* and *topic.log.retention.hours* Had some queries around it.And it was mentioned that topic.log.retention.hours is per topic configuration. Had some queries around it - 1. Does it mean that I need to specific the topicname.log.retention.hours in the kafka config? 2. Can this property be overriden anywhere? 3. Is it possible for the producer to set a message expiry so that the message expires after a configurable period of time? Regards, Kashyap
Re: Log Retention in Kafka
Thanks Joel. Am using version 2.8.0. Thanks, Kashyap On Tue, Apr 22, 2014 at 5:53 PM, Joel Koshy jjkosh...@gmail.com wrote: Which version of Kafka are you using? You can read up on the configuration options here: http://kafka.apache.org/documentation.html#configuration You can specify time-based retention using log.retention.minutes which will apply to all topics. You can override that on per-topic basis - see further down in the above page under topic-level configuration On Tue, Apr 22, 2014 at 02:34:24PM -0500, Kashyap Mhaisekar wrote: Hi, I wanted to set the message expiry for a message on a kafka topic. Is there anything like this in kafka? I came across a property - *log.retention.hours* and *topic.log.retention.hours* Had some queries around it.And it was mentioned that topic.log.retention.hours is per topic configuration. Had some queries around it - 1. Does it mean that I need to specific the topicname.log.retention.hours in the kafka config? 2. Can this property be overriden anywhere? 3. Is it possible for the producer to set a message expiry so that the message expires after a configurable period of time? Regards, Kashyap
