Thank you! what performance impacts will it be if I change log.segment.bytes? Thanks.
On Mon, Jul 27, 2015 at 1:25 PM, Ewen Cheslack-Postava <e...@confluent.io> wrote: > I think log.cleanup.interval.mins was removed in the first 0.8 release. It > sounds like you're looking at outdated docs. Search for > log.retention.check.interval.ms here: > http://kafka.apache.org/documentation.html > > As for setting the values too low hurting performance, I'd guess it's > probably only an issue if you set them extremely small, such that file > creation and cleanup become a bottleneck. > > -Ewen > > On Mon, Jul 27, 2015 at 10:03 AM, Yuheng Du <yuheng.du.h...@gmail.com> > wrote: > > > If I want to get higher throughput, should I increase the > > log.segment.bytes? > > > > I don't see log.retention.check.interval.ms, but there is > > log.cleanup.interval.mins, is that what you mean? > > > > If I set log.roll.ms or log.cleanup.interval.mins too small, will it > hurt > > the throughput? Thanks. > > > > On Fri, Jul 24, 2015 at 11:03 PM, Ewen Cheslack-Postava < > e...@confluent.io > > > > > wrote: > > > > > You'll want to set the log retention policy via > > > log.retention.{ms,minutes,hours} or log.retention.bytes. If you want > > really > > > aggressive collection (e.g., on the order of seconds, as you > specified), > > > you might also need to adjust log.segment.bytes/log.roll.{ms,hours} and > > > log.retention.check.interval.ms. > > > > > > On Fri, Jul 24, 2015 at 12:49 PM, Yuheng Du <yuheng.du.h...@gmail.com> > > > wrote: > > > > > > > Hi, > > > > > > > > I am testing the kafka producer performance. So I created a queue and > > > > writes a large amount of data to that queue. > > > > > > > > Is there a way to delete the data automatically after some time, say > > > > whenever the data size reaches 50GB or the retention time exceeds 10 > > > > seconds, it will be deleted so my disk won't get filled and new data > > > can't > > > > be written in? > > > > > > > > Thanks.! > > > > > > > > > > > > > > > > -- > > > Thanks, > > > Ewen > > > > > > > > > -- > Thanks, > Ewen >
