Initially, I used the OVA as well and I got horrendous performance. I pointed only two file servers and two domain controllers to that installation and the VM was so busy the web interface would hang constantly. I created another VM and installed Graylog using these instructions <http://www.itzgeek.com/how-tos/linux/ubuntu-how-tos/how-to-install-graylog2-on-ubuntu-14-04.html> and got great performance, even with the same memory and CPU core count. Once I got Graylog working on the VM, I used the same instructions to install it on a physical server. So far, I've collected almost 400 million log events in 11 days and the highest I've seen the CPU usage has been 14% with only 16GB of RAM. My suggestion is to ditch the OVA altogether.
On Monday, November 23, 2015 at 6:38:50 AM UTC-5, Matthew Simon wrote: > > Hi Jochen > > I installed Graylog2 via the OVA I downloaded off the Site. > > It can fluctuate you see there are specials that run on the website which > can cause massive amounts of traffic so it can be fine one week then within > the span of 2 days I can get anything from 1million messages to 4 million > messages. Common story is that it cant keep up with that influx of messages. > > Grok patterns that I use. Please see attachment Grokpatternsfile.txt > > Extractors please see attachment Extractors.txt > > I have not setup dashboards or streams as of yet. > > Purpose of Graylog is im trying to extract WAF logs (so if there is a > better way of doing this please help.) > > If there is more info you want please let me know and I do appreciate your > help in advance. > > Thanks > > > On Monday, November 23, 2015 at 12:58:45 PM UTC+2, Jochen Schalanda wrote: >> >> Hi Matthew, >> >> to not duplicate the information you already gave on Twitter ( >> https://twitter.com/Malfufi/status/668724729629556736): >> >>> heap 1.4G (changeability?) no local configuration has been made to >>> optimize,85mill and CPU 100% RAM 100% >> >> >> The heap size of Graylog can be changed quite easily. How did you install >> Graylog in the first place (OVA, DEB packages, RPM packages, tarball…)? >> >> "85 mill" what exactly? Messages per second/minute/hour/day/week/month? >> Are you running any extractors (e. g. RegEx or Grok extractors) or defined >> streams with relatively complicated regular expression rules? >> >> >> Cheers, >> Jochen >> >> On Monday, 23 November 2015 10:55:48 UTC+1, Matthew Simon wrote: >>> >>> Hi Guys >>> >>> >>> I have a problem! >>> >>> >>> I receive large amounts of logs to my Graylog2 server and i feel that >>> the server cant keep up with the incoming logs, Is there a way that I can >>> optimize my configuration to handle large amounts of LOGS. >>> >>> >>> Please see the image bellow. >>> >>> >>> Thanks in advance. >>> >>> >>> >>> >>> <https://lh3.googleusercontent.com/-n91LHSZ_HtQ/VlLhwvs-sKI/AAAAAAAAAqU/dXnmzSAQFCg/s1600/image1.png> >>> >>> >>> >>> -- You received this message because you are subscribed to the Google Groups "Graylog Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/graylog2/a2179e59-3d64-48f3-9a62-a395f7d14751%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
