I see. The use case is indeed to be able to retain x number of months worth of logs, which isn't really dependent upon the index cycling method.
My assumption was that would've been more effective to accomplish it by rotating indexes - as it would require less I/O for deletion. The way I understood your message, I gather than the method you prefer would be something like: index1 - X messages, dated from A-B-C index2 - X messages, dated from D-E-F index3 - X messages, dated from G-H-I ... Which means that deleting messages older than E would mean delete index3, and messages older than E within index2. For a count deletion, you would say keep 2 indexes (thereby deleting index3). On my assumption this would have become: index1 - X messages, dated from A index2 - Y messages, dated from B index3 - Z messages, dated from C index4 - N messages, dated from D index5 - M messages, dated from E index6 - L messages, dated from F ... Which means that deleting messages older than E would mean delete index6 onwards. For a count deletion, if you wanted to keep C messages, you would delete any messages after sum(x,y,z...)>C (including whole indexes). They both achieve the same outcome - to be able to retain either based on date, or based on amount - so I'm happy either way. :-) I do wonder whether keeping each index as small as possible make any difference towards responsiveness and resource usage (upon deletion). Aristoteles Neto [email protected] On 4/03/2014, at 10:16 am, Lennart Koopmann <[email protected]> wrote: > > Hey Aristoteles, > > we are not planning to change the index cycling to something like > daily or hourly. This is because we want maximum flexibility by > defining index sizes with message count limits. What we are working on > is **retention cleaning** based on timeframes like days, weeks, months > or years. > > What is your use case that you need daily or hourly indices for? Happy > to help! :) > > Cheers, > Lennart > > On Mon, Mar 3, 2014 at 2:22 AM, Aristoteles Neto <[email protected]> wrote: >> Hi Guys, >> >> Love the work you guys are doing with Graylog2, and I'd love to replace >> Logstash with Graylog! >> >> Unfortunately theres a few things that are preventing me from doing so. >> >> I understand The Daily/Hourly index splitting and retention option was >> originally planned for v0.20. Now that v0.20.1 is out, was there a more >> accurate timeframe for having the feature available? Looking through the >> mailing list, it seems like a frequently touched on subject, so apologies if >> this has been asked previously. >> >> This is one of the things preventing the move from Logstash. >> >> Aristoteles Neto >> [email protected] >> >> -- >> You received this message because you are subscribed to the Google Groups >> "graylog2" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> For more options, visit https://groups.google.com/groups/opt_out. > > -- > You received this message because you are subscribed to the Google Groups > "graylog2" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > For more options, visit https://groups.google.com/groups/opt_out. > -- You received this message because you are subscribed to the Google Groups "graylog2" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/groups/opt_out.
