Hi, ingesting 2000000 messages per day (which comes down to ~25 messages per second) isn't that much and should work with pretty standard systems (4 CPUs, 8 GB of memory, disk space depending on average message size).
If the disk journal fills up, that usually means that the backend (i. e. Elasticsearch) is not fast enough. You should check the hardware specs for your Elasticsearch cluster. Cheers, Jochen On Monday, 11 April 2016 18:46:00 UTC+2, Graylog-WAF wrote: > > Thank you first for your reply. > > We are receiving about 2 millions messages and more per day. > > Please do you have any idea how to configure journal size or other > parameters in order to treat this big number. > > Journal is being full quickly so Graylog seems blocked and I can't find > anything in "Search" tab or in the customised dashboard. > > > > Le lundi 11 avril 2016 11:34:38 UTC+1, Jochen Schalanda a écrit : >> >> Hi, >> >> messages ingested by Graylog are first persisted to a disk journal. From >> there the messages are being read, processed (extractors, sorting into >> streams etc.), written to the outputs (by default Elasticsearch, other >> outputs depending on the configuration), and finally removed from the disk >> journal. >> >> On Saturday, 9 April 2016 22:36:38 UTC+2, Graylog-WAF wrote: >>> >>> I have used the OVA file in which there is only 4 Gb of RAM. >>> >>> Does this have effect on the capacity of storage? >>> >> >> It doesn't affect the storage capacity directly, but the maximum possible >> message fields being loaded into memory by Elasticsearch, e. g. for the >> quick values functionality or searches in general. >> >> >> Cheers, >> Jochen >> >> On Saturday, 9 April 2016 22:33:48 UTC+2, Graylog-WAF wrote: >>> >>> Hello everybody, >>> >>> We are implementing Graylog2 and which is integrated with WAF. >>> >>> It's receiving about 2 Millions events per day. >>> >>> I would like to know where logs are saved at the beginning (I mean are >>> they saved directly in disk or in DB and then in disk). >>> >>> Also, is it possible to know the exact percentage that's used until now. >>> >>> Thanks ! >>> >>> -- You received this message because you are subscribed to the Google Groups "Graylog Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to graylog2+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/graylog2/463cf6c1-5763-46d7-b4e7-db81a1b0e07e%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
