I have 4 node cluster and node1 has zookeeper running. I added those parameters on each of my server.properties files in kafka nodes.
Eranga Heshan *Undergraduate* Computer Science & Engineering University of Moratuwa Mobile: +94 71 138 2686 <%2B94%2071%20552%202087> Email: [email protected] <[email protected]> <https://www.facebook.com/erangaheshan> <https://twitter.com/erangaheshan> <https://www.linkedin.com/in/erangaheshan> On Wed, Nov 23, 2016 at 9:36 AM, Sachin Mittal <[email protected]> wrote: > Kafka logs are not kept based on data consumption. They are deleted after > it reaches a particular size and retention time. > Even if they are consumed multiple times in that duration, they are still > kept. > > Where have you added those params and how are you starting your zookeeper > and server. > > > On Wed, Nov 23, 2016 at 8:00 AM, Eranga Heshan <[email protected]> > wrote: > > > Sorry for repeatedly asking the same question, but although I tried all > > those parameters; my log file still at large about 2.5GB. Is it due to > the > > data inside my topic is flushed to disk? I meant that there might be data > > inside topic which is not consumed yet or not been deleted even after > > consumed. > > > > Thanks, > > Regards, > > > > > > Eranga Heshan > > *Undergraduate* > > Computer Science & Engineering > > University of Moratuwa > > Mobile: +94 71 138 2686 <%2B94%2071%20552%202087> > > Email: [email protected] <[email protected]> > > <https://www.facebook.com/erangaheshan> <https://twitter.com/ > > erangaheshan> > > <https://www.linkedin.com/in/erangaheshan> > > > > On Tue, Nov 22, 2016 at 2:15 PM, Sachin Mittal <[email protected]> > wrote: > > > > > Set some value to log.retention.ms which is ms time to retain log and > > also > > > log.segment.bytes which is size of a single log file. > > > delete.topic.enable is needed if you want to delete the topic itself. > > > > > > Thanks > > > Sachin > > > > > > > > > On Tue, Nov 22, 2016 at 1:52 PM, Eranga Heshan <[email protected]> > > > wrote: > > > > > > > I tried log.retention.bytes=1049000 which is about 10MB. But it did > not > > > > work. Is it because I had not given any value to delete.topic.enable? > > > > > > > > > > > > Eranga Heshan > > > > *Undergraduate* > > > > Computer Science & Engineering > > > > University of Moratuwa > > > > Mobile: +94 71 138 2686 <%2B94%2071%20552%202087> > > > > Email: [email protected] <[email protected]> > > > > <https://www.facebook.com/erangaheshan> <https://twitter.com/ > > > > erangaheshan> > > > > <https://www.linkedin.com/in/erangaheshan> > > > > > > > > On Tue, Nov 22, 2016 at 1:47 PM, Sachin Mittal <[email protected]> > > > wrote: > > > > > > > > > Check > > > > > http://kafka.apache.org/documentation.html#brokerconfigs > > > > > log.retention.bytes > > > > > log.retention.ms > > > > > log.segment.bytes > > > > > delete.topic.enable > > > > > > > > > > > > > > > > > > > > > > > > > On Tue, Nov 22, 2016 at 1:36 PM, Eranga Heshan < > [email protected] > > > > > > > > wrote: > > > > > > > > > > > Hi all, > > > > > > > > > > > > I want to keep the size of kafka-logs reduced as much as > possible. > > To > > > > do > > > > > it > > > > > > I need to set it to delete its unwanted logs frequently. Is > there a > > > way > > > > > to > > > > > > do it? > > > > > > > > > > > > Thanks, > > > > > > Regards, > > > > > > > > > > > > Eranga Heshan > > > > > > *Undergraduate* > > > > > > Computer Science & Engineering > > > > > > University of Moratuwa > > > > > > Mobile: +94 71 138 2686 <%2B94%2071%20552%202087> > > > > > > Email: [email protected] <[email protected]> > > > > > > <https://www.facebook.com/erangaheshan> <https://twitter.com/ > > > > > > erangaheshan> > > > > > > <https://www.linkedin.com/in/erangaheshan> > > > > > > > > > > > > > > > > > > > > >
