log.retention.bytes is for all topics that are not included in
log.retention.bytes.per.topic
(which defines a map of topic -> size).
Currently, we don't have a total size limit across all topics.
Thanks,
Jun
On Fri, Aug 16, 2013 at 2:00 PM, Paul Christian
wrote:
> According to the Kafka 8 doc
Hi Jun,
Thank you for your reply. I'm still a little fuzzy on the concept.
Are you saying I can have topic A, B and C and with
log.retention.bytes.per.topic.A = 15MB
log.retention.bytes.per.topic.B = 20MB
log.retention.bytes = 30MB
And thus topic C will get the value 30MB? Since it's not define
For the first question, yes, topic C will get the value of 30MB.
For the second question, log.retention.bytes only controls the segment log
file size, not the index. Typically, index file size is much smaller than
the log file. The index file of the last (active) segment is presized to
the max ind
Paul,
I'm trying to understand the 2nd problem you reported. Are you saying that
you set the log.retention.bytes=11534336 (11MB) but nevertheless your log
grew to 114MB. Which means the config option didn't really work as expected?
Thanks,
Neha
On Mon, Aug 19, 2013 at 8:46 PM, Jun Rao wrote:
Jun,
For my first example is that syntax correct? I.e.
log.retention.bytes.per.topic.A = 15MB
log.retention.bytes.per.topic.B = 20MB
I totally guessed there and was wondering if I guessed right? Otherwise is
there a document with the proper formatting to full out this map?
Thank you,
Paul
Neha,
Correct, that is my question. We want to investigate capping our disk usage
so we don't fill up our hds. If you have any recommended configurations or
documents on these setting, please let us know.
Thank you,
Paul
On Tue, Aug 20, 2013 at 6:16 AM, Paul Christian
wrote:
> Jun,
>
> For m
All per topic configuration properties below have the format of csv (e.g.,
"topic1:value1,topic2:value2"). Updated our website to make it clear.
Thanks,
Jun
On Tue, Aug 20, 2013 at 6:16 AM, Paul Christian
wrote:
> Jun,
>
> For my first example is that syntax correct? I.e.
>
> log.retention.byt
...@gmail.com]
Sent: Saturday, August 17, 2013 12:40 AM
To: users@kafka.apache.org
Subject: Re: questtion about log.retention.bytes
log.retention.bytes is for all topics that are not included in
log.retention.bytes.per.topic (which defines a map of topic -> size).
Currently, we don't have a to
gt; should it be set to 1G or 3G?
>
> Libo
>
>
> -Original Message-
> From: Jun Rao [mailto:jun...@gmail.com]
> Sent: Saturday, August 17, 2013 12:40 AM
> To: users@kafka.apache.org
> Subject: Re: questtion about log.retention.bytes
>
> log.retention.bytes is
on.
> > Then log.retention.bytes should be set to 1Gb (not 3Gb). Is that right?
> >
> > If I want to use log.retention.bytes.per.topic to set the same limit,
> > should it be set to 1G or 3G?
> >
> > Libo
> >
> >
> > -O
n.
> > > Then log.retention.bytes should be set to 1Gb (not 3Gb). Is that right?
> > >
> > > If I want to use log.retention.bytes.per.topic to set the same limit,
> > > should it be set to 1G or 3G?
> > >
> > > Libo
> > >
> > >
> > >
t the
> > > disk space to 1Gb for each partition.
> > > Then log.retention.bytes should be set to 1Gb (not 3Gb). Is that right?
> > >
> > > If I want to use log.retention.bytes.per.topic to set the same
> > > limit, should it be set to 1G or 3G?
> >
12 matches
Mail list logo
