Re: log.retention.bytes.per.topic does not trigger deletion
Yes, once 0.8.1 is released we will add a separate quick-start page. On Sun, Jan 19, 2014 at 10:57 PM, Joe Stein wrote: > There are a lot of nuance changes to the QuickStart and other changes for > 0.8.1 that will need a separate page... Not just modifications. > > > /*** > Joe Stein > Founder, Principal Consultant > Big Data Open Source Security LLC > http://www.stealth.ly > Twitter: @allthingshadoop > / > > > On Jan 19, 2014, at 10:19 PM, Jason Rosenberg wrote: > > > Please be sure to update the online config docs with this change! The > > per topic options are still listed there > > > > Jason > > > > On Thu, Jan 16, 2014 at 9:57 PM, Ben Summer > wrote: > >> I see. I don't have version 0.8.1 yet. We just updated to 0.8.0 from > beta > >> after it became the "stable version". > >> Good to know there is a fix for this. I'll start trying it out in some > >> non-production environments. > >> > >> Thanks, > >> Ben > >> > >> > >> On Thu, Jan 16, 2014 at 7:42 PM, Guozhang Wang > wrote: > >> > >>> In the latest version, per-topic configs have been moved to Zookeeper > you > >>> and set them using admin tools instead of writing the config files. > Could > >>> you try trunk HEAD and see if this issue has already been resolved: > >>> > >>> https://issues.apache.org/jira/browse/KAFKA-554 > >>> > >>> Guozhang > >>> > >>> > >>> On Thu, Jan 16, 2014 at 6:27 PM, Ben Summer > >>> wrote: > >>> > v0.8.0, (non-beta) downloaded from the website. Let me know if there > is a > file I can check in the installation footprint that can give you an > >>> actual > change id. > > Is there a fix to this particular feature that you know was made in a > >>> later > build? > > Thanks for the quick response, > Ben > > > On Thu, Jan 16, 2014 at 6:22 PM, Guozhang Wang > >>> wrote: > > > Hello Ben, > > > > Which version are you using? > > > > Guozhang > > > > > > On Thu, Jan 16, 2014 at 3:15 PM, Ben Summer > > > wrote: > > > >> I tried using the following two retention properties > >> > >> log.retention.bytes=3221225472 > >> > >> > > > > >>> > log.retention.bytes.per.topic=first_topic:1099511627776,second_topic:1099511627776 > >> > >> which I interpret to mean "by default, keep 3GB per topic partition, > > except > >> for first_topic and second_topic, which should retain 1TB each > >>> (across > > all > >> their partitions). > >> > >> I've noticed that the data for first_topic and second_topic grow way > > beyond > >> 1TB and no deletions occur. Is the syntax correct? > >> > >> Thanks in advance, > >> Ben > >> > > > > > > > > -- > > -- Guozhang > > > > >>> > >>> > >>> > >>> -- > >>> -- Guozhang > >>> > -- -- Guozhang
Re: log.retention.bytes.per.topic does not trigger deletion
There are a lot of nuance changes to the QuickStart and other changes for 0.8.1 that will need a separate page... Not just modifications. /*** Joe Stein Founder, Principal Consultant Big Data Open Source Security LLC http://www.stealth.ly Twitter: @allthingshadoop / On Jan 19, 2014, at 10:19 PM, Jason Rosenberg wrote: > Please be sure to update the online config docs with this change! The > per topic options are still listed there > > Jason > > On Thu, Jan 16, 2014 at 9:57 PM, Ben Summer wrote: >> I see. I don't have version 0.8.1 yet. We just updated to 0.8.0 from beta >> after it became the "stable version". >> Good to know there is a fix for this. I'll start trying it out in some >> non-production environments. >> >> Thanks, >> Ben >> >> >> On Thu, Jan 16, 2014 at 7:42 PM, Guozhang Wang wrote: >> >>> In the latest version, per-topic configs have been moved to Zookeeper you >>> and set them using admin tools instead of writing the config files. Could >>> you try trunk HEAD and see if this issue has already been resolved: >>> >>> https://issues.apache.org/jira/browse/KAFKA-554 >>> >>> Guozhang >>> >>> >>> On Thu, Jan 16, 2014 at 6:27 PM, Ben Summer >>> wrote: >>> v0.8.0, (non-beta) downloaded from the website. Let me know if there is a file I can check in the installation footprint that can give you an >>> actual change id. Is there a fix to this particular feature that you know was made in a >>> later build? Thanks for the quick response, Ben On Thu, Jan 16, 2014 at 6:22 PM, Guozhang Wang >>> wrote: > Hello Ben, > > Which version are you using? > > Guozhang > > > On Thu, Jan 16, 2014 at 3:15 PM, Ben Summer > wrote: > >> I tried using the following two retention properties >> >> log.retention.bytes=3221225472 >> >> > >>> log.retention.bytes.per.topic=first_topic:1099511627776,second_topic:1099511627776 >> >> which I interpret to mean "by default, keep 3GB per topic partition, > except >> for first_topic and second_topic, which should retain 1TB each >>> (across > all >> their partitions). >> >> I've noticed that the data for first_topic and second_topic grow way > beyond >> 1TB and no deletions occur. Is the syntax correct? >> >> Thanks in advance, >> Ben >> > > > > -- > -- Guozhang > >>> >>> >>> >>> -- >>> -- Guozhang >>>
Re: log.retention.bytes.per.topic does not trigger deletion
Please be sure to update the online config docs with this change! The per topic options are still listed there Jason On Thu, Jan 16, 2014 at 9:57 PM, Ben Summer wrote: > I see. I don't have version 0.8.1 yet. We just updated to 0.8.0 from beta > after it became the "stable version". > Good to know there is a fix for this. I'll start trying it out in some > non-production environments. > > Thanks, > Ben > > > On Thu, Jan 16, 2014 at 7:42 PM, Guozhang Wang wrote: > >> In the latest version, per-topic configs have been moved to Zookeeper you >> and set them using admin tools instead of writing the config files. Could >> you try trunk HEAD and see if this issue has already been resolved: >> >> https://issues.apache.org/jira/browse/KAFKA-554 >> >> Guozhang >> >> >> On Thu, Jan 16, 2014 at 6:27 PM, Ben Summer >> wrote: >> >> > v0.8.0, (non-beta) downloaded from the website. Let me know if there is a >> > file I can check in the installation footprint that can give you an >> actual >> > change id. >> > >> > Is there a fix to this particular feature that you know was made in a >> later >> > build? >> > >> > Thanks for the quick response, >> > Ben >> > >> > >> > On Thu, Jan 16, 2014 at 6:22 PM, Guozhang Wang >> wrote: >> > >> > > Hello Ben, >> > > >> > > Which version are you using? >> > > >> > > Guozhang >> > > >> > > >> > > On Thu, Jan 16, 2014 at 3:15 PM, Ben Summer >> > > wrote: >> > > >> > > > I tried using the following two retention properties >> > > > >> > > > log.retention.bytes=3221225472 >> > > > >> > > > >> > > >> > >> log.retention.bytes.per.topic=first_topic:1099511627776,second_topic:1099511627776 >> > > > >> > > > which I interpret to mean "by default, keep 3GB per topic partition, >> > > except >> > > > for first_topic and second_topic, which should retain 1TB each >> (across >> > > all >> > > > their partitions). >> > > > >> > > > I've noticed that the data for first_topic and second_topic grow way >> > > beyond >> > > > 1TB and no deletions occur. Is the syntax correct? >> > > > >> > > > Thanks in advance, >> > > > Ben >> > > > >> > > >> > > >> > > >> > > -- >> > > -- Guozhang >> > > >> > >> >> >> >> -- >> -- Guozhang >>
Re: log.retention.bytes.per.topic does not trigger deletion
I see. I don't have version 0.8.1 yet. We just updated to 0.8.0 from beta after it became the "stable version". Good to know there is a fix for this. I'll start trying it out in some non-production environments. Thanks, Ben On Thu, Jan 16, 2014 at 7:42 PM, Guozhang Wang wrote: > In the latest version, per-topic configs have been moved to Zookeeper you > and set them using admin tools instead of writing the config files. Could > you try trunk HEAD and see if this issue has already been resolved: > > https://issues.apache.org/jira/browse/KAFKA-554 > > Guozhang > > > On Thu, Jan 16, 2014 at 6:27 PM, Ben Summer > wrote: > > > v0.8.0, (non-beta) downloaded from the website. Let me know if there is a > > file I can check in the installation footprint that can give you an > actual > > change id. > > > > Is there a fix to this particular feature that you know was made in a > later > > build? > > > > Thanks for the quick response, > > Ben > > > > > > On Thu, Jan 16, 2014 at 6:22 PM, Guozhang Wang > wrote: > > > > > Hello Ben, > > > > > > Which version are you using? > > > > > > Guozhang > > > > > > > > > On Thu, Jan 16, 2014 at 3:15 PM, Ben Summer > > > wrote: > > > > > > > I tried using the following two retention properties > > > > > > > > log.retention.bytes=3221225472 > > > > > > > > > > > > > > log.retention.bytes.per.topic=first_topic:1099511627776,second_topic:1099511627776 > > > > > > > > which I interpret to mean "by default, keep 3GB per topic partition, > > > except > > > > for first_topic and second_topic, which should retain 1TB each > (across > > > all > > > > their partitions). > > > > > > > > I've noticed that the data for first_topic and second_topic grow way > > > beyond > > > > 1TB and no deletions occur. Is the syntax correct? > > > > > > > > Thanks in advance, > > > > Ben > > > > > > > > > > > > > > > > -- > > > -- Guozhang > > > > > > > > > -- > -- Guozhang >
Re: log.retention.bytes.per.topic does not trigger deletion
In the latest version, per-topic configs have been moved to Zookeeper you and set them using admin tools instead of writing the config files. Could you try trunk HEAD and see if this issue has already been resolved: https://issues.apache.org/jira/browse/KAFKA-554 Guozhang On Thu, Jan 16, 2014 at 6:27 PM, Ben Summer wrote: > v0.8.0, (non-beta) downloaded from the website. Let me know if there is a > file I can check in the installation footprint that can give you an actual > change id. > > Is there a fix to this particular feature that you know was made in a later > build? > > Thanks for the quick response, > Ben > > > On Thu, Jan 16, 2014 at 6:22 PM, Guozhang Wang wrote: > > > Hello Ben, > > > > Which version are you using? > > > > Guozhang > > > > > > On Thu, Jan 16, 2014 at 3:15 PM, Ben Summer > > wrote: > > > > > I tried using the following two retention properties > > > > > > log.retention.bytes=3221225472 > > > > > > > > > log.retention.bytes.per.topic=first_topic:1099511627776,second_topic:1099511627776 > > > > > > which I interpret to mean "by default, keep 3GB per topic partition, > > except > > > for first_topic and second_topic, which should retain 1TB each (across > > all > > > their partitions). > > > > > > I've noticed that the data for first_topic and second_topic grow way > > beyond > > > 1TB and no deletions occur. Is the syntax correct? > > > > > > Thanks in advance, > > > Ben > > > > > > > > > > > -- > > -- Guozhang > > > -- -- Guozhang
Re: log.retention.bytes.per.topic does not trigger deletion
v0.8.0, (non-beta) downloaded from the website. Let me know if there is a file I can check in the installation footprint that can give you an actual change id. Is there a fix to this particular feature that you know was made in a later build? Thanks for the quick response, Ben On Thu, Jan 16, 2014 at 6:22 PM, Guozhang Wang wrote: > Hello Ben, > > Which version are you using? > > Guozhang > > > On Thu, Jan 16, 2014 at 3:15 PM, Ben Summer > wrote: > > > I tried using the following two retention properties > > > > log.retention.bytes=3221225472 > > > > > log.retention.bytes.per.topic=first_topic:1099511627776,second_topic:1099511627776 > > > > which I interpret to mean "by default, keep 3GB per topic partition, > except > > for first_topic and second_topic, which should retain 1TB each (across > all > > their partitions). > > > > I've noticed that the data for first_topic and second_topic grow way > beyond > > 1TB and no deletions occur. Is the syntax correct? > > > > Thanks in advance, > > Ben > > > > > > -- > -- Guozhang >
Re: log.retention.bytes.per.topic does not trigger deletion
Hello Ben, Which version are you using? Guozhang On Thu, Jan 16, 2014 at 3:15 PM, Ben Summer wrote: > I tried using the following two retention properties > > log.retention.bytes=3221225472 > > log.retention.bytes.per.topic=first_topic:1099511627776,second_topic:1099511627776 > > which I interpret to mean "by default, keep 3GB per topic partition, except > for first_topic and second_topic, which should retain 1TB each (across all > their partitions). > > I've noticed that the data for first_topic and second_topic grow way beyond > 1TB and no deletions occur. Is the syntax correct? > > Thanks in advance, > Ben > -- -- Guozhang
log.retention.bytes.per.topic does not trigger deletion
I tried using the following two retention properties log.retention.bytes=3221225472 log.retention.bytes.per.topic=first_topic:1099511627776,second_topic:1099511627776 which I interpret to mean "by default, keep 3GB per topic partition, except for first_topic and second_topic, which should retain 1TB each (across all their partitions). I've noticed that the data for first_topic and second_topic grow way beyond 1TB and no deletions occur. Is the syntax correct? Thanks in advance, Ben
