Jochen,
I've looked over the config files and this thread (and then double
checked). I've cleaned up the two config files for ES (removed all the
comments and posted here just the uncommented lines). I also added options
that seemed like they might help. But the log files still show it trying
to bind port 9300 on 127.0.0.1. I've done everything I know to do to make
it NOT use the loopback interface. The config's as they exist now are:
cluster.name: graylog
node.name: node2
node.master: false
network.host: x.x.x.149
network.publish_host: x.x.x.149
transport.tcp.port: 9300
http.port: 9200
discovery.zen.ping.unicast.hosts: ["x.x.x.146", "x.x.x.149"]
discovery.zen.minimum_master_nodes: 1
cluster.name: graylog
node.name: node1
node.master: true
network.host: x.x.x.146
network.publish_host: x.x.x.146
transport.tcp.port: 9300
http.port: 9200
discovery.zen.ping.unicast.hosts: ["x.x.x.149", "x.x.x.146"]
discovery.zen.minimum_master_nodes: 1
I am completely out of ideas.
Nathan
On Tuesday, August 2, 2016 at 12:48:39 PM UTC-4, Jochen Schalanda wrote:
>
> Hi Nathan,
>
> it seems your Elasticsearch config is still wrong. Both nodes only bind to
> localhost:
>
> ES node 1:
>> [2016-08-02 09:19:16,184][INFO ][transport ] [Betty Ross Banner]
>> publish_address {127.0.0.1:9300}, bound_addresses {[::1]:9300}, {
>> 127.0.0.1:9300}
>>
>
>
> ES node 2:
>> [2016-08-02 09:19:16,064][INFO ][transport ] [Invisible Woman]
>> publish_address {127.0.0.1:9300}, bound_addresses {[::1]:9300}, {
>> 127.0.0.1:9300}
>
>
> I suggest you double check the configuration files and do the changes I
> suggested in the numerous mails before.
>
> Cheers,
> Jochen
>
>
> On Tuesday, 2 August 2016 18:43:16 UTC+2, Nathan Mace wrote:
>>
>> Please see attached files. I got the elasticsearch.log file from
>> /var/log/elasticsearch on both nodes. Additionally I got graylog.log from
>> the same location on both nodes. Even though node 2 doesn't have graylog
>> installed it had a log file for it. Not sure why that is.
>>
>> Thanks!
>>
>> Nathan
>>
>> On Tuesday, August 2, 2016 at 11:10:49 AM UTC-4, Jochen Schalanda wrote:
>>>
>>> Hi Nathan,
>>>
>>> please post the *complete* log files of your Elasticsearch and Graylog
>>> nodes.
>>>
>>> Cheers,
>>> Jochen
>>>
>>> On Tuesday, 2 August 2016 16:56:58 UTC+2, Nathan Mace wrote:
>>>>
>>>> Removing the leading whitespaces didn't help.
>>>>
>>>> However in looking through the logs I found this in the primary node's
>>>> graylog.log file:
>>>>
>>>> ConnectTransportException[[ansted-search-01][x.x.x.149:9300]
>>>> connect_timeout[30s]]; nested: ConnectException[Connection refused:
>>>> /x.x.x.149:9300];
>>>> at
>>>> org.elasticsearch.transport.netty.NettyTransport.connectToChannels(NettyTransport.java:987)
>>>> at
>>>> org.elasticsearch.transport.netty.NettyTransport.connectToNode(NettyTransport.java:920)
>>>> at
>>>> org.elasticsearch.transport.netty.NettyTransport.connectToNode(NettyTransport.java:893)
>>>> at
>>>> org.elasticsearch.transport.TransportService.connectToNode(TransportService.java:260)
>>>> at
>>>> org.elasticsearch.discovery.zen.ZenDiscovery.joinElectedMaster(ZenDiscovery.java:434)
>>>> at
>>>> org.elasticsearch.discovery.zen.ZenDiscovery.innerJoinCluster(ZenDiscovery.java:386)
>>>> at
>>>> org.elasticsearch.discovery.zen.ZenDiscovery.access$4800(ZenDiscovery.java:91)
>>>> at
>>>> org.elasticsearch.discovery.zen.ZenDiscovery$JoinThreadControl$1.run(ZenDiscovery.java:1237)
>>>> at
>>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>>>> at
>>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>>>> at java.lang.Thread.run(Thread.java:745)
>>>>
>>>> It was repeated several times. That is it trying to connect to the
>>>> second node on port 9300 and not being able to. I see in the
>>>> documentation
>>>> that 9300 is the default port and I have nothing in either of the ES YML
>>>> files referencing that port number, so it seems to be all default. If I
>>>> do
>>>> a netstat on both hosts they are both listening on port 9200 and 9300. It
>>>> would seem that it is listening, but only allowing connections to 9300
>>>> from
>>>> localhost? What would I need to change to allow a connect from the other
>>>> node?
>>>>
>>>> Nathan
>>>>
>>>> On Tuesday, August 2, 2016 at 10:22:44 AM UTC-4, Jochen Schalanda wrote:
>>>>>
>>>>> Hi Nathan,
>>>>>
>>>>> I'm not sure how Elasticsearch handles leading whitespace in their
>>>>> configuration file. I'd recommend making sure that the configuration
>>>>> settings really start at the beginning of a line.
>>>>>
>>>>> Additionally, please post the complete log files of your Elasticsearch
>>>>> and Graylog nodes.
>>>>>
>>>>> Cheers,
>>>>> Jochen
>>>>>
>>>>> On Tuesday, 2 August 2016 16:00:47 UTC+2, Nathan Mace wrote:
>>>>>>
>>>>>> Oh good grief! Clearly been staring at this problem to long, I
>>>>>> completely missed those hash signs.
>>>>>>
>>>>>> OK, now ES is happily running on the proper IP addresses. I can
>>>>>> access it via curl from other hosts. So that's a large improvement.
>>>>>> However Graylog still only reports 1 node in the web interface. I've
>>>>>> attached the current versions of the config files (vs copy/paste).
>>>>>> Given
>>>>>> my tunnel vision on the hash signs, this seems like it will be something
>>>>>> obvious but I can't find it.
>>>>>>
>>>>>> Thank you so much for the help!
>>>>>>
>>>>>> Nathan
>>>>>>
>>>>>> On Tuesday, August 2, 2016 at 9:30:58 AM UTC-4, Jochen Schalanda
>>>>>> wrote:
>>>>>>>
>>>>>>> Hi Nathan,
>>>>>>>
>>>>>>> leading hash signs (the # character) mean that the line is
>>>>>>> commented out.
>>>>>>>
>>>>>>> For example the following line is completely ignored:
>>>>>>>
>>>>>>> # discovery.zen.ping.unicast.hosts: ["x.x.x.146", "x.x.x.149"]
>>>>>>>
>>>>>>>
>>>>>>> While this line is "active" and will be obeyed:
>>>>>>>
>>>>>>> cluster.name: graylog
>>>>>>>
>>>>>>>
>>>>>>> Maybe you've only copy & pasted your configuration files in a
>>>>>>> strange way (which is why I would always recommend to send them as
>>>>>>> attachments), but that's how it looks like.
>>>>>>>
>>>>>>> Cheers,
>>>>>>> Jochen
>>>>>>>
>>>>>>> On Tuesday, 2 August 2016 15:23:22 UTC+2, Nathan Mace wrote:
>>>>>>>>
>>>>>>>> Thanks Jochen. I will make the changes. However I am very
>>>>>>>> confused by your comment about the second node having the
>>>>>>>> cluster.name setting unset. I'm showing that it is set to
>>>>>>>> "graylog" just like the first node. I'm not sure at all what you mean.
>>>>>>>>
>>>>>>>> Nathan
>>>>>>>>
>>>>>>>> On Tuesday, August 2, 2016 at 6:38:45 AM UTC-4, Jochen Schalanda
>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>> Hi Nathan,
>>>>>>>>>
>>>>>>>>> check the elasticsearch_network_host setting of your Graylog
>>>>>>>>> nodes. It should be set to one (and only one!) public IP address of
>>>>>>>>> the
>>>>>>>>> Graylog node which can be accessed by all other Elasticsearch nodes
>>>>>>>>> in the
>>>>>>>>> cluster. elasticsearch_discovery_zen_ping_unicast_hosts should
>>>>>>>>> be a comma-separated list of host/port pairs containing the addresses
>>>>>>>>> of
>>>>>>>>> the Elasticsearch nodes, for example:
>>>>>>>>>
>>>>>>>>> elasticsearch_discovery_zen_ping_unicast_hosts = x.x.x.146:9300,
>>>>>>>>> x.x.x.149
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> See
>>>>>>>>> http://docs.graylog.org/en/2.0/pages/configuration/elasticsearch.html#network-setup
>>>>>>>>>
>>>>>>>>> for details.
>>>>>>>>>
>>>>>>>>> Additionally, the cluster.name of your second Elasticsearch node
>>>>>>>>> is unset, which makes it default to "elasticsearch". The logs of that
>>>>>>>>> Elasticsearch node should show this pretty clearly.
>>>>>>>>>
>>>>>>>>> Also take a look at the network.host settings of both your
>>>>>>>>> Elasticsearch nodes. This setting must be customized to your network
>>>>>>>>> setup,
>>>>>>>>> otherwise they'll only bind to the local network interface (i. e.
>>>>>>>>> 127.0.0.1 or ::1). See
>>>>>>>>> https://www.elastic.co/guide/en/elasticsearch/reference/2.3/modules-network.html#common-network-settings
>>>>>>>>>
>>>>>>>>> for details.
>>>>>>>>>
>>>>>>>>> Cheers,
>>>>>>>>> Jochen
>>>>>>>>>
>>>>>>>>> On Monday, 1 August 2016 22:15:32 UTC+2, Nathan Mace wrote:
>>>>>>>>>>
>>>>>>>>>> Primary node (MonoDB, Graylog, and ES): IP Address: x.x.x.146
>>>>>>>>>> Secondary Node (ES Only): IP Address: x.x.x.149
>>>>>>>>>>
>>>>>>>>>> Both on the same subnet. Can ping each other.
>>>>>>>>>> […]
>>>>>>>>>>
>>>>>>>>>
--
You received this message because you are subscribed to the Google Groups
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/graylog2/fec94c8f-0519-47ff-a77d-45025a09762d%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
