[graylog2] Re: More Graylog/Elastic questions from the cheap seats
Hi Mark, the Elasticsearch node used internally by Graylog is just a client node by default, which means that it doesn't store any data at all. The message regarding -XX:PermSize and -XX:MaxPermSize=256m is probably caused by the switch from Java 7 to Java 8. With Java 8 the JVM doesn't have a permanent generation anymore but introduced a new memory region named Metaspace, see http://www.infoq.com/articles/Java-PERMGEN-Removed for details. Cheers, Jochen On Friday, 27 March 2015 05:24:45 UTC+1, Mark Moorcroft wrote: Next question... Why do all of the elastic stored records appear to reside in the default dynamic named node, but the apparently empty graylog2-server elastic node is the one gobbling up heap memory? According to my elastic node diags the empty graylog2-server node, that according to the graylog interface isn't used, the more memory I give it, the more it will use. Also, I switched from OpenJDK to Oracle today. It complains that -XX:PermSize=128m -XX:MaxPermSize=256m from /etc/sysconfig/graylog-server are no longer supported. On Wednesday, March 25, 2015 at 7:31:38 PM UTC-7, Mark Moorcroft wrote: In looking at trying to increase the heap size today after a general overhaul of our logging system I was reminded about a few things I never seemed to get answers to in the past. Some of these statements are in fact questions. Setting mlockall in elasticsearch apparently does NOT set it for graylog? I can't seem to find a way to increase the heap size for the graylog index beyond 972MB. From the beginning I have wondered why I need the default elastic index (node with the dynamic naming) that never seems to be used, as well as the graylog index(node). The default elastic index seems to have all of the recommended tweaks (like mlockall), but the graylog index doesn't. Where exactly am I supposed to be changing them? Many times today on both of my graylog systems clicking on System:Nodes produces This exception has been logged with id 6libgij97.. I don't see any other issues. If I run curl http://localhost:9200/_nodes/process?pretty; when I look at the nodes parameters the graylog node is version 1.3.7 but the default node is 1.3.4 with different build numbers. More dumb questions to follow if I can remember them ;-) -- You received this message because you are subscribed to the Google Groups graylog2 group. To unsubscribe from this group and stop receiving emails from it, send an email to graylog2+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[graylog2] Re: More Graylog/Elastic questions from the cheap seats
Next question... Why do all of the elastic stored records appear to reside in the default dynamic named node, but the apparently empty graylog2-server elastic node is the one gobbling up heap memory? According to my elastic node diags the empty graylog2-server node, that according to the graylog interface isn't used, the more memory I give it, the more it will use. Also, I switched from OpenJDK to Oracle today. It complains that -XX:PermSize=128m -XX:MaxPermSize=256m from /etc/sysconfig/graylog-server are no longer supported. On Wednesday, March 25, 2015 at 7:31:38 PM UTC-7, Mark Moorcroft wrote: In looking at trying to increase the heap size today after a general overhaul of our logging system I was reminded about a few things I never seemed to get answers to in the past. Some of these statements are in fact questions. Setting mlockall in elasticsearch apparently does NOT set it for graylog? I can't seem to find a way to increase the heap size for the graylog index beyond 972MB. From the beginning I have wondered why I need the default elastic index (node with the dynamic naming) that never seems to be used, as well as the graylog index(node). The default elastic index seems to have all of the recommended tweaks (like mlockall), but the graylog index doesn't. Where exactly am I supposed to be changing them? Many times today on both of my graylog systems clicking on System:Nodes produces This exception has been logged with id 6libgij97.. I don't see any other issues. If I run curl http://localhost:9200/_nodes/process?pretty; when I look at the nodes parameters the graylog node is version 1.3.7 but the default node is 1.3.4 with different build numbers. More dumb questions to follow if I can remember them ;-) -- You received this message because you are subscribed to the Google Groups graylog2 group. To unsubscribe from this group and stop receiving emails from it, send an email to graylog2+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[graylog2] Re: More Graylog/Elastic questions from the cheap seats
Still flailing without guidance I have some more questions. I changed elasticsearch_discovery_zen_ping_unicast_hosts = 127.0.0.1:9300, and with 9300 it appears that the default index (node) is being filled. Graylog creates a second node that I don't believe I need (port 9350 which I didn't set or chose), but if I look at the elasticsearch parameters after adjusting heap size in sysconfig, Graylog Nodes is showing me the heap size for the index that isn't being used (the one set in /etc/sysconfig/graylog-server). The memory usage fluctuates as though something is happening, but that index is totally empty. The default dynamically named index is filling, and I have increased the heap size there in /etc/sysconfig/elasticsearch. So the web interface is showing me status on the unused index (node). On Wednesday, March 25, 2015 at 7:31:38 PM UTC-7, Mark Moorcroft wrote: In looking at trying to increase the heap size today after a general overhaul of our logging system I was reminded about a few things I never seemed to get answers to in the past. Some of these statements are in fact questions. Setting mlockall in elasticsearch apparently does NOT set it for graylog? I can't seem to find a way to increase the heap size for the graylog index beyond 972MB. From the beginning I have wondered why I need the default elastic index (node with the dynamic naming) that never seems to be used, as well as the graylog index(node). The default elastic index seems to have all of the recommended tweaks (like mlockall), but the graylog index doesn't. Where exactly am I supposed to be changing them? -- You received this message because you are subscribed to the Google Groups graylog2 group. To unsubscribe from this group and stop receiving emails from it, send an email to graylog2+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[graylog2] Re: More Graylog/Elastic questions from the cheap seats
I'm not sure if it's considered a best practice to tweak the default /etc/sysconfig/graylog-server? GRAYLOG_SERVER_JAVA_OPTS=-Xms2g -Xmx2g -XX:NewRatio=1 -XX:PermSize=128m -XX:MaxPermSize=256m -server -XX:+ResizeTLAB -XX:+UseConcMarkSweepGC -XX:+CMSConcurrentMTEnabled -XX:+CMSClassUnloadingEnabled-XX:+UseParNewGC -XX:-OmitStackTraceInFastThrow But this at least seems to give you double the heap space. It's still not obvious how you should set mlockall. Or if I should even try. On Wednesday, March 25, 2015 at 7:31:38 PM UTC-7, Mark Moorcroft wrote: In looking at trying to increase the heap size today after a general overhaul of our logging system I was reminded about a few things I never seemed to get answers to in the past. Some of these statements are in fact questions. Setting mlockall in elasticsearch apparently does NOT set it for graylog? I can't seem to find a way to increase the heap size for the graylog index beyond 972MB. -- You received this message because you are subscribed to the Google Groups graylog2 group. To unsubscribe from this group and stop receiving emails from it, send an email to graylog2+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
