Hi Nathan,
it seems your Elasticsearch config is still wrong. Both nodes only bind to
localhost:
ES node 1:
> [2016-08-02 09:19:16,184][INFO ][transport ] [Betty Ross Banner]
> publish_address {127.0.0.1:9300}, bound_addresses {[::1]:9300},
> {127.0.0.1:9300}
>
ES node 2:
> [2016-08-02 09:19:16,064][INFO ][transport ] [Invisible Woman]
> publish_address {127.0.0.1:9300}, bound_addresses {[::1]:9300},
> {127.0.0.1:9300}
I suggest you double check the configuration files and do the changes I
suggested in the numerous mails before.
Cheers,
Jochen
On Tuesday, 2 August 2016 18:43:16 UTC+2, Nathan Mace wrote:
>
> Please see attached files. I got the elasticsearch.log file from
> /var/log/elasticsearch on both nodes. Additionally I got graylog.log from
> the same location on both nodes. Even though node 2 doesn't have graylog
> installed it had a log file for it. Not sure why that is.
>
> Thanks!
>
> Nathan
>
> On Tuesday, August 2, 2016 at 11:10:49 AM UTC-4, Jochen Schalanda wrote:
>>
>> Hi Nathan,
>>
>> please post the *complete* log files of your Elasticsearch and Graylog
>> nodes.
>>
>> Cheers,
>> Jochen
>>
>> On Tuesday, 2 August 2016 16:56:58 UTC+2, Nathan Mace wrote:
>>>
>>> Removing the leading whitespaces didn't help.
>>>
>>> However in looking through the logs I found this in the primary node's
>>> graylog.log file:
>>>
>>> ConnectTransportException[[ansted-search-01][x.x.x.149:9300]
>>> connect_timeout[30s]]; nested: ConnectException[Connection refused:
>>> /x.x.x.149:9300];
>>> at
>>> org.elasticsearch.transport.netty.NettyTransport.connectToChannels(NettyTransport.java:987)
>>> at
>>> org.elasticsearch.transport.netty.NettyTransport.connectToNode(NettyTransport.java:920)
>>> at
>>> org.elasticsearch.transport.netty.NettyTransport.connectToNode(NettyTransport.java:893)
>>> at
>>> org.elasticsearch.transport.TransportService.connectToNode(TransportService.java:260)
>>> at
>>> org.elasticsearch.discovery.zen.ZenDiscovery.joinElectedMaster(ZenDiscovery.java:434)
>>> at
>>> org.elasticsearch.discovery.zen.ZenDiscovery.innerJoinCluster(ZenDiscovery.java:386)
>>> at
>>> org.elasticsearch.discovery.zen.ZenDiscovery.access$4800(ZenDiscovery.java:91)
>>> at
>>> org.elasticsearch.discovery.zen.ZenDiscovery$JoinThreadControl$1.run(ZenDiscovery.java:1237)
>>> at
>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>>> at
>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>>> at java.lang.Thread.run(Thread.java:745)
>>>
>>> It was repeated several times. That is it trying to connect to the
>>> second node on port 9300 and not being able to. I see in the documentation
>>> that 9300 is the default port and I have nothing in either of the ES YML
>>> files referencing that port number, so it seems to be all default. If I do
>>> a netstat on both hosts they are both listening on port 9200 and 9300. It
>>> would seem that it is listening, but only allowing connections to 9300 from
>>> localhost? What would I need to change to allow a connect from the other
>>> node?
>>>
>>> Nathan
>>>
>>> On Tuesday, August 2, 2016 at 10:22:44 AM UTC-4, Jochen Schalanda wrote:
>>>>
>>>> Hi Nathan,
>>>>
>>>> I'm not sure how Elasticsearch handles leading whitespace in their
>>>> configuration file. I'd recommend making sure that the configuration
>>>> settings really start at the beginning of a line.
>>>>
>>>> Additionally, please post the complete log files of your Elasticsearch
>>>> and Graylog nodes.
>>>>
>>>> Cheers,
>>>> Jochen
>>>>
>>>> On Tuesday, 2 August 2016 16:00:47 UTC+2, Nathan Mace wrote:
>>>>>
>>>>> Oh good grief! Clearly been staring at this problem to long, I
>>>>> completely missed those hash signs.
>>>>>
>>>>> OK, now ES is happily running on the proper IP addresses. I can
>>>>> access it via curl from other hosts. So that's a large improvement.
>>>>> However Graylog still only reports 1 node in the web interface. I've
>>>>> attached the current versions of the config files (vs copy/paste). Given
>>>>> my tunnel vision on the hash signs, this seems like it will be something
>>>>> obvious but I can't find it.
>>>>>
>>>>> Thank you so much for the help!
>>>>>
>>>>> Nathan
>>>>>
>>>>> On Tuesday, August 2, 2016 at 9:30:58 AM UTC-4, Jochen Schalanda wrote:
>>>>>>
>>>>>> Hi Nathan,
>>>>>>
>>>>>> leading hash signs (the # character) mean that the line is commented
>>>>>> out.
>>>>>>
>>>>>> For example the following line is completely ignored:
>>>>>>
>>>>>> # discovery.zen.ping.unicast.hosts: ["x.x.x.146", "x.x.x.149"]
>>>>>>
>>>>>>
>>>>>> While this line is "active" and will be obeyed:
>>>>>>
>>>>>> cluster.name: graylog
>>>>>>
>>>>>>
>>>>>> Maybe you've only copy & pasted your configuration files in a strange
>>>>>> way (which is why I would always recommend to send them as attachments),
>>>>>> but that's how it looks like.
>>>>>>
>>>>>> Cheers,
>>>>>> Jochen
>>>>>>
>>>>>> On Tuesday, 2 August 2016 15:23:22 UTC+2, Nathan Mace wrote:
>>>>>>>
>>>>>>> Thanks Jochen. I will make the changes. However I am very confused
>>>>>>> by your comment about the second node having the cluster.name
>>>>>>> setting unset. I'm showing that it is set to "graylog" just like the
>>>>>>> first
>>>>>>> node. I'm not sure at all what you mean.
>>>>>>>
>>>>>>> Nathan
>>>>>>>
>>>>>>> On Tuesday, August 2, 2016 at 6:38:45 AM UTC-4, Jochen Schalanda
>>>>>>> wrote:
>>>>>>>>
>>>>>>>> Hi Nathan,
>>>>>>>>
>>>>>>>> check the elasticsearch_network_host setting of your Graylog
>>>>>>>> nodes. It should be set to one (and only one!) public IP address of
>>>>>>>> the
>>>>>>>> Graylog node which can be accessed by all other Elasticsearch nodes in
>>>>>>>> the
>>>>>>>> cluster. elasticsearch_discovery_zen_ping_unicast_hosts should be
>>>>>>>> a comma-separated list of host/port pairs containing the addresses of
>>>>>>>> the
>>>>>>>> Elasticsearch nodes, for example:
>>>>>>>>
>>>>>>>> elasticsearch_discovery_zen_ping_unicast_hosts = x.x.x.146:9300,
>>>>>>>> x.x.x.149
>>>>>>>>
>>>>>>>>
>>>>>>>> See
>>>>>>>> http://docs.graylog.org/en/2.0/pages/configuration/elasticsearch.html#network-setup
>>>>>>>>
>>>>>>>> for details.
>>>>>>>>
>>>>>>>> Additionally, the cluster.name of your second Elasticsearch node
>>>>>>>> is unset, which makes it default to "elasticsearch". The logs of that
>>>>>>>> Elasticsearch node should show this pretty clearly.
>>>>>>>>
>>>>>>>> Also take a look at the network.host settings of both your
>>>>>>>> Elasticsearch nodes. This setting must be customized to your network
>>>>>>>> setup,
>>>>>>>> otherwise they'll only bind to the local network interface (i. e.
>>>>>>>> 127.0.0.1 or ::1). See
>>>>>>>> https://www.elastic.co/guide/en/elasticsearch/reference/2.3/modules-network.html#common-network-settings
>>>>>>>>
>>>>>>>> for details.
>>>>>>>>
>>>>>>>> Cheers,
>>>>>>>> Jochen
>>>>>>>>
>>>>>>>> On Monday, 1 August 2016 22:15:32 UTC+2, Nathan Mace wrote:
>>>>>>>>>
>>>>>>>>> Primary node (MonoDB, Graylog, and ES): IP Address: x.x.x.146
>>>>>>>>> Secondary Node (ES Only): IP Address: x.x.x.149
>>>>>>>>>
>>>>>>>>> Both on the same subnet. Can ping each other.
>>>>>>>>> […]
>>>>>>>>>
>>>>>>>>
--
You received this message because you are subscribed to the Google Groups
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/graylog2/388534db-6490-4e70-90a6-4e7e76ad38fc%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
