Hi Jun,

In a previous email thread 
http://markmail.org/search/?q=kafka+log.retention.bytes#query:kafka%20log.retention.bytes+page:1+mid:qnt4pbq47goii2ui+state:results,
you said log.retention.bytes is for each partition. Could you clarify on that?

Say if I have a topic with three partitions. I want to limit the disk space to 
1Gb for each partition.
Then log.retention.bytes should be set to 1Gb (not 3Gb). Is that right?

If I want to use log.retention.bytes.per.topic to set the same limit, should it 
be set to 1G or 3G?

Libo


-----Original Message-----
From: Jun Rao [mailto:jun...@gmail.com] 
Sent: Saturday, August 17, 2013 12:40 AM
To: users@kafka.apache.org
Subject: Re: questtion about log.retention.bytes

log.retention.bytes is for all topics that are not included in 
log.retention.bytes.per.topic (which defines a map of topic -> size).

Currently, we don't have a total size limit across all topics.

Thanks,

Jun


On Fri, Aug 16, 2013 at 2:00 PM, Paul Christian
<pchrist...@salesforce.com>wrote:

> According to the Kafka 8 documentation under broker configuration. 
> There are these parameters and their definitions.
>
> log.retention.bytes -1 The maximum size of the log before deleting it 
> log.retention.bytes.per.topic "" The maximum size of the log for some 
> specific topic before deleting it
>
> I'm curious what the first value 'log.retention.bytes' is for if the 
> second one is for per topic logs, because aren't all logs generated 
> per topic? Is this an aggregate value across topics?
>
> Related question, is there a parameter for kafka where you can say 
> only hold this much TOTAL data across all topic ( logs/index together )? I.e.
> our  hosts have this much available space and so value 
> log.retention.whatever.aggregate == 75% total  disk space.
>

Reply via email to