I guess that kind of makes sense. The following section in the config is what confused me: *"# The following configurations control the disposal of log segments. The policy can* *# be set to delete segments after a period of time, or after a given size has accumulated.* *# A segment will be deleted whenever *either* of these criteria are met. Deletion always happens* *# from the end of the log."*
That makes it sound like deletion will happen if either of the criteria is met. I thought the whole idea of those two settings (time and bytes) is telling the application when it will need to delete. On Mon, Sep 21, 2015 at 7:10 PM, noah <iamn...@gmail.com> wrote: > "minimum age of a log file to be eligible for deletion" Key word is > minimum. If you only have 1k logs, Kafka doesn't need to delete anything. > Try to push more data through and when it needs to, it will start deleting > old logs. > > On Mon, Sep 21, 2015 at 8:58 PM allen chan <allen.michael.c...@gmail.com> > wrote: > > > Hi, > > > > Just brought up new kafka cluster for testing. > > Was able to use the console producers to send 1k of logs and received it > on > > the console consumer side. > > > > The one issue that i have right now is that the retention period does not > > seem to be working. > > > > *# The minimum age of a log file to be eligible for deletion* > > *log.retention.hours=1* > > > > I have waited for almost 2 hours and the 1k of logs are still in kafka. > > > > I did see these messages pop up on the console > > *[2015-09-21 17:12:01,236] INFO Scheduling log segment 0 for log test-1 > for > > deletion. (kafka.log.Log)* > > *[2015-09-21 17:13:01,238] INFO Deleting segment 0 from log test-1. > > (kafka.log.Log)* > > *[2015-09-21 17:13:01,239] INFO Deleting index > > /var/log/kafka/test-1/00000000000000000000.index.deleted > > (kafka.log.OffsetIndex)* > > > > I know the logs are still in there because i am using > > the kafka-consumer-offset-checker.sh and it says how many messages the > > logSize is. > > > > What am i missing in my configuration? > > > > > > > > Thanks! > > > > -- > > Allen Michael Chan > > > -- Allen Michael Chan
