Removing the leading whitespaces didn't help. However in looking through the logs I found this in the primary node's graylog.log file:
ConnectTransportException[[ansted-search-01][x.x.x.149:9300] connect_timeout[30s]]; nested: ConnectException[Connection refused: /x.x.x.149:9300]; at org.elasticsearch.transport.netty.NettyTransport.connectToChannels(NettyTransport.java:987) at org.elasticsearch.transport.netty.NettyTransport.connectToNode(NettyTransport.java:920) at org.elasticsearch.transport.netty.NettyTransport.connectToNode(NettyTransport.java:893) at org.elasticsearch.transport.TransportService.connectToNode(TransportService.java:260) at org.elasticsearch.discovery.zen.ZenDiscovery.joinElectedMaster(ZenDiscovery.java:434) at org.elasticsearch.discovery.zen.ZenDiscovery.innerJoinCluster(ZenDiscovery.java:386) at org.elasticsearch.discovery.zen.ZenDiscovery.access$4800(ZenDiscovery.java:91) at org.elasticsearch.discovery.zen.ZenDiscovery$JoinThreadControl$1.run(ZenDiscovery.java:1237) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) It was repeated several times. That is it trying to connect to the second node on port 9300 and not being able to. I see in the documentation that 9300 is the default port and I have nothing in either of the ES YML files referencing that port number, so it seems to be all default. If I do a netstat on both hosts they are both listening on port 9200 and 9300. It would seem that it is listening, but only allowing connections to 9300 from localhost? What would I need to change to allow a connect from the other node? Nathan On Tuesday, August 2, 2016 at 10:22:44 AM UTC-4, Jochen Schalanda wrote: > > Hi Nathan, > > I'm not sure how Elasticsearch handles leading whitespace in their > configuration file. I'd recommend making sure that the configuration > settings really start at the beginning of a line. > > Additionally, please post the complete log files of your Elasticsearch and > Graylog nodes. > > Cheers, > Jochen > > On Tuesday, 2 August 2016 16:00:47 UTC+2, Nathan Mace wrote: >> >> Oh good grief! Clearly been staring at this problem to long, I >> completely missed those hash signs. >> >> OK, now ES is happily running on the proper IP addresses. I can access >> it via curl from other hosts. So that's a large improvement. However >> Graylog still only reports 1 node in the web interface. I've attached the >> current versions of the config files (vs copy/paste). Given my tunnel >> vision on the hash signs, this seems like it will be something obvious but >> I can't find it. >> >> Thank you so much for the help! >> >> Nathan >> >> On Tuesday, August 2, 2016 at 9:30:58 AM UTC-4, Jochen Schalanda wrote: >>> >>> Hi Nathan, >>> >>> leading hash signs (the # character) mean that the line is commented >>> out. >>> >>> For example the following line is completely ignored: >>> >>> # discovery.zen.ping.unicast.hosts: ["x.x.x.146", "x.x.x.149"] >>> >>> >>> While this line is "active" and will be obeyed: >>> >>> cluster.name: graylog >>> >>> >>> Maybe you've only copy & pasted your configuration files in a strange >>> way (which is why I would always recommend to send them as attachments), >>> but that's how it looks like. >>> >>> Cheers, >>> Jochen >>> >>> On Tuesday, 2 August 2016 15:23:22 UTC+2, Nathan Mace wrote: >>>> >>>> Thanks Jochen. I will make the changes. However I am very confused by >>>> your comment about the second node having the cluster.name setting >>>> unset. I'm showing that it is set to "graylog" just like the first node. >>>> I'm not sure at all what you mean. >>>> >>>> Nathan >>>> >>>> On Tuesday, August 2, 2016 at 6:38:45 AM UTC-4, Jochen Schalanda wrote: >>>>> >>>>> Hi Nathan, >>>>> >>>>> check the elasticsearch_network_host setting of your Graylog nodes. >>>>> It should be set to one (and only one!) public IP address of the Graylog >>>>> node which can be accessed by all other Elasticsearch nodes in the >>>>> cluster. elasticsearch_discovery_zen_ping_unicast_hosts should be a >>>>> comma-separated list of host/port pairs containing the addresses of the >>>>> Elasticsearch nodes, for example: >>>>> >>>>> elasticsearch_discovery_zen_ping_unicast_hosts = x.x.x.146:9300, >>>>> x.x.x.149 >>>>> >>>>> >>>>> See >>>>> http://docs.graylog.org/en/2.0/pages/configuration/elasticsearch.html#network-setup >>>>> >>>>> for details. >>>>> >>>>> Additionally, the cluster.name of your second Elasticsearch node is >>>>> unset, which makes it default to "elasticsearch". The logs of that >>>>> Elasticsearch node should show this pretty clearly. >>>>> >>>>> Also take a look at the network.host settings of both your >>>>> Elasticsearch nodes. This setting must be customized to your network >>>>> setup, >>>>> otherwise they'll only bind to the local network interface (i. e. >>>>> 127.0.0.1 or ::1). See >>>>> https://www.elastic.co/guide/en/elasticsearch/reference/2.3/modules-network.html#common-network-settings >>>>> >>>>> for details. >>>>> >>>>> Cheers, >>>>> Jochen >>>>> >>>>> On Monday, 1 August 2016 22:15:32 UTC+2, Nathan Mace wrote: >>>>>> >>>>>> Primary node (MonoDB, Graylog, and ES): IP Address: x.x.x.146 >>>>>> Secondary Node (ES Only): IP Address: x.x.x.149 >>>>>> >>>>>> Both on the same subnet. Can ping each other. >>>>>> […] >>>>>> >>>>> -- You received this message because you are subscribed to the Google Groups "Graylog Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to graylog2+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/graylog2/9995dc87-b7c3-44ac-9c4e-a10f21edc95e%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
