Re: [graylog2] Graylog not deleting old indexes upon restart
Thanks Edmundo, I will do as you suggest. Regards. On Wednesday, July 23, 2014 6:10:30 PM UTC+5:30, Edmundo Alvarez wrote: > > Hi Kaustubh, > > I think the new values are in place, but closed indices do not count for > Graylog2 index retention. You could confirm that by running Graylog2 server > in debug mode and checking if the line "Number of indices (X) lower than > limit (Y). Not performing any retention actions.” appear in your logs. To > start graylog2-server in debug mode, please pass the --debug option on the > command to start it. > > As the index retention does not affect re-opened indices neither, I would > delete closed indices by hand. The new index retention strategy should take > care of old indices in the future, once the number of opened indices > reaches the value set on “elasticsearch_max_number_of_indices”. > > I hope that helps! > > Regards, > > Edmundo Alvarez > Developer > > Tel.: +49 (0)40 609 452 077 > Fax.: +49 (0)40 609 452 078 > > TORCH GmbH > Steckelhörn 11 > 20457 Hamburg > Germany > https://www.torch.sh/ > > Commercial Reg. (Registergericht): Amtsgericht Hamburg, HRB 125175 > Geschäftsführer: Lennart Koopmann (CEO) > > On 23 Jul 2014, at 11:01, Kaustubh Padegaonkar > wrote: > > > Hi Edmundo, > > > > Thank you for replying. The current index number stands at 150. There > are 130 indices that are closed. > > > > Close was the indexing strategy before I changed it to delete. The > number was 20. It seems that ES/Graylog has not picked up the new values. > Seems far fetched, but I dont how else to rationalize it. > > > > I am using graylog2 v0.20.1, just in case you need to know. > > > > Thank you. > > > > > > On Tuesday, July 22, 2014 1:31:50 PM UTC+5:30, Edmundo Alvarez wrote: > > Hello, > > > > Could you please tell us how many indices called “graylog_” you > have in your elasticsearch cluster? Graylog2 will not start deleting old > indices until the value set on “elasticsearch_max_number_of_indices” is > reached, so you could use a lower value for that setting in order to use > less disk space. > > > > Regards, > > > > Edmundo Alvarez > > Developer > > > > Tel.: +49 (0)40 609 452 077 > > Fax.: +49 (0)40 609 452 078 > > > > TORCH GmbH > > Steckelhörn 11 > > 20457 Hamburg > > Germany > > https://www.torch.sh/ > > > > Commercial Reg. (Registergericht): Amtsgericht Hamburg, HRB 125175 > > Geschäftsführer: Lennart Koopmann (CEO) > > > > On 21 Jul 2014, at 10:29, Kaustubh Padegaonkar > wrote: > > > > > Hello, > > > > > > In my graylog conf file, I have set the retention_strategy to delete. > elasticsearch_max_number_of_indices is set to 60. > > > > > > elasticsearch_index_prefix is graylog and I can see that the indices > are named as graylog_ > > > > > > I restarted elasticsearch and graylog a few times, but the indices > dont seem to be deleted by graylog. The logs also dont show any goings on. > > > > > > What gives? I am running out of disk space. I can use the ES_head > plugin and delete the old ones one-by-one, but I need to know why this is > happening. > > > > > > Thank you. > > > > > > -- > > > You received this message because you are subscribed to the Google > Groups "graylog2" group. > > > To unsubscribe from this group and stop receiving emails from it, send > an email to graylog2+u...@googlegroups.com. > > > For more options, visit https://groups.google.com/d/optout. > > > > > > -- > > You received this message because you are subscribed to the Google > Groups "graylog2" group. > > To unsubscribe from this group and stop receiving emails from it, send > an email to graylog2+u...@googlegroups.com . > > For more options, visit https://groups.google.com/d/optout. > > -- You received this message because you are subscribed to the Google Groups "graylog2" group. To unsubscribe from this group and stop receiving emails from it, send an email to graylog2+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [graylog2] Graylog not deleting old indexes upon restart
Hi Kaustubh, I think the new values are in place, but closed indices do not count for Graylog2 index retention. You could confirm that by running Graylog2 server in debug mode and checking if the line "Number of indices (X) lower than limit (Y). Not performing any retention actions." appear in your logs. To start graylog2-server in debug mode, please pass the --debug option on the command to start it. As the index retention does not affect re-opened indices neither, I would delete closed indices by hand. The new index retention strategy should take care of old indices in the future, once the number of opened indices reaches the value set on "elasticsearch_max_number_of_indices". I hope that helps! Regards, Edmundo Alvarez Developer Tel.: +49 (0)40 609 452 077 Fax.: +49 (0)40 609 452 078 TORCH GmbH Steckelhörn 11 20457 Hamburg Germany https://www.torch.sh/ Commercial Reg. (Registergericht): Amtsgericht Hamburg, HRB 125175 Geschäftsführer: Lennart Koopmann (CEO) On 23 Jul 2014, at 11:01, Kaustubh Padegaonkar wrote: > Hi Edmundo, > > Thank you for replying. The current index number stands at 150. There are 130 > indices that are closed. > > Close was the indexing strategy before I changed it to delete. The number was > 20. It seems that ES/Graylog has not picked up the new values. Seems far > fetched, but I dont how else to rationalize it. > > I am using graylog2 v0.20.1, just in case you need to know. > > Thank you. > > > On Tuesday, July 22, 2014 1:31:50 PM UTC+5:30, Edmundo Alvarez wrote: > Hello, > > Could you please tell us how many indices called "graylog_" you have > in your elasticsearch cluster? Graylog2 will not start deleting old indices > until the value set on "elasticsearch_max_number_of_indices" is reached, so > you could use a lower value for that setting in order to use less disk space. > > Regards, > > Edmundo Alvarez > Developer > > Tel.: +49 (0)40 609 452 077 > Fax.: +49 (0)40 609 452 078 > > TORCH GmbH > Steckelhörn 11 > 20457 Hamburg > Germany > https://www.torch.sh/ > > Commercial Reg. (Registergericht): Amtsgericht Hamburg, HRB 125175 > Geschäftsführer: Lennart Koopmann (CEO) > > On 21 Jul 2014, at 10:29, Kaustubh Padegaonkar wrote: > > > Hello, > > > > In my graylog conf file, I have set the retention_strategy to delete. > > elasticsearch_max_number_of_indices is set to 60. > > > > elasticsearch_index_prefix is graylog and I can see that the indices are > > named as graylog_ > > > > I restarted elasticsearch and graylog a few times, but the indices dont > > seem to be deleted by graylog. The logs also dont show any goings on. > > > > What gives? I am running out of disk space. I can use the ES_head plugin > > and delete the old ones one-by-one, but I need to know why this is > > happening. > > > > Thank you. > > > > -- > > You received this message because you are subscribed to the Google Groups > > "graylog2" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to graylog2+u...@googlegroups.com. > > For more options, visit https://groups.google.com/d/optout. > > > -- > You received this message because you are subscribed to the Google Groups > "graylog2" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to graylog2+unsubscr...@googlegroups.com. > For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups "graylog2" group. To unsubscribe from this group and stop receiving emails from it, send an email to graylog2+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [graylog2] Graylog not deleting old indexes upon restart
Hi Edmundo, Thank you for replying. The current index number stands at 150. There are 130 indices that are closed. Close was the indexing strategy before I changed it to delete. The number was 20. It seems that ES/Graylog has not picked up the new values. Seems far fetched, but I dont how else to rationalize it. I am using graylog2 v0.20.1, just in case you need to know. Thank you. On Tuesday, July 22, 2014 1:31:50 PM UTC+5:30, Edmundo Alvarez wrote: > > Hello, > > Could you please tell us how many indices called “graylog_” you > have in your elasticsearch cluster? Graylog2 will not start deleting old > indices until the value set on “elasticsearch_max_number_of_indices” is > reached, so you could use a lower value for that setting in order to use > less disk space. > > Regards, > > Edmundo Alvarez > Developer > > Tel.: +49 (0)40 609 452 077 > Fax.: +49 (0)40 609 452 078 > > TORCH GmbH > Steckelhörn 11 > 20457 Hamburg > Germany > https://www.torch.sh/ > > Commercial Reg. (Registergericht): Amtsgericht Hamburg, HRB 125175 > Geschäftsführer: Lennart Koopmann (CEO) > > On 21 Jul 2014, at 10:29, Kaustubh Padegaonkar > wrote: > > > Hello, > > > > In my graylog conf file, I have set the retention_strategy to delete. > elasticsearch_max_number_of_indices is set to 60. > > > > elasticsearch_index_prefix is graylog and I can see that the indices are > named as graylog_ > > > > I restarted elasticsearch and graylog a few times, but the indices dont > seem to be deleted by graylog. The logs also dont show any goings on. > > > > What gives? I am running out of disk space. I can use the ES_head plugin > and delete the old ones one-by-one, but I need to know why this is > happening. > > > > Thank you. > > > > -- > > You received this message because you are subscribed to the Google > Groups "graylog2" group. > > To unsubscribe from this group and stop receiving emails from it, send > an email to graylog2+u...@googlegroups.com . > > For more options, visit https://groups.google.com/d/optout. > > -- You received this message because you are subscribed to the Google Groups "graylog2" group. To unsubscribe from this group and stop receiving emails from it, send an email to graylog2+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [graylog2] Graylog not deleting old indexes upon restart
Hello, Could you please tell us how many indices called "graylog_" you have in your elasticsearch cluster? Graylog2 will not start deleting old indices until the value set on "elasticsearch_max_number_of_indices" is reached, so you could use a lower value for that setting in order to use less disk space. Regards, Edmundo Alvarez Developer Tel.: +49 (0)40 609 452 077 Fax.: +49 (0)40 609 452 078 TORCH GmbH Steckelhörn 11 20457 Hamburg Germany https://www.torch.sh/ Commercial Reg. (Registergericht): Amtsgericht Hamburg, HRB 125175 Geschäftsführer: Lennart Koopmann (CEO) On 21 Jul 2014, at 10:29, Kaustubh Padegaonkar wrote: > Hello, > > In my graylog conf file, I have set the retention_strategy to delete. > elasticsearch_max_number_of_indices is set to 60. > > elasticsearch_index_prefix is graylog and I can see that the indices are > named as graylog_ > > I restarted elasticsearch and graylog a few times, but the indices dont seem > to be deleted by graylog. The logs also dont show any goings on. > > What gives? I am running out of disk space. I can use the ES_head plugin and > delete the old ones one-by-one, but I need to know why this is happening. > > Thank you. > > -- > You received this message because you are subscribed to the Google Groups > "graylog2" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to graylog2+unsubscr...@googlegroups.com. > For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups "graylog2" group. To unsubscribe from this group and stop receiving emails from it, send an email to graylog2+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[graylog2] Graylog not deleting old indexes upon restart
Hello, In my graylog conf file, I have set the retention_strategy to delete. elasticsearch_max_number_of_indices is set to 60. elasticsearch_index_prefix is graylog and I can see that the indices are named as graylog_ I restarted elasticsearch and graylog a few times, but the indices dont seem to be deleted by graylog. The logs also dont show any goings on. What gives? I am running out of disk space. I can use the ES_head plugin and delete the old ones one-by-one, but I need to know why this is happening. Thank you. -- You received this message because you are subscribed to the Google Groups "graylog2" group. To unsubscribe from this group and stop receiving emails from it, send an email to graylog2+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
