[graylog2] Graylog 2 beta API port listening issue in the website

2016-04-21 Thread erict
Dear Graylog community support / users,

I have been using Graylog since 1.2 and its working great.

Just discover a change about a health check in Graylog's web just might 
cause problems.
It's known and normal that the Graylog's web service detects the server 
node(s) healthiness with API thru TCP 12900.

However I noticed an issue in Graylog 2.
When I am trying out Graylog 2 (Alpha and Beta), the web UI automatically 
calls TCP 12900 (API port) in the client side using the public address.
That is, from the developer mode of the browser, I can see URL call of 
http://:12900/system/cluster/node. This causes the following 
issues:

1) With the default configuration, such check listens to private IP of the 
server. So just when deploying the Graylog to internet, the check fails. 
(Unless we access the website through VPN IP or update *rest_transport_uri* 
in /opt/graylog/conf/graylog.conf)
2) Health check should probably be done in background in the server (i.e. 
like Graylog 1.2, 1.3...the checking will not be exposed to client side / 
browser)
3) We need to expose TCP 12900 of the web service to public, security 
concern arises as the API port would be facing the public internet as well

Thank you.
Eric

-- 
You received this message because you are subscribed to the Google Groups 
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/graylog2/a43a9ea9-2b6b-4d6a-8b91-1304b84dd008%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


[graylog2] Re: Reason: There was a problem with your search. We expected HTTP 200, but got a HTTP 500.

2016-04-21 Thread Shrawan Bhagwat
Hi Jochen,

I am not getting any error message in Elasticsearch logs on both the 
servers.
Server1:
[2016-04-20 18:34:36,386][INFO ][bootstrap] max_open_files 
[65536]
[2016-04-20 18:34:36,695][INFO ][node ] [Master 228] 
version[1.7.2], pid[2149], build[e43676b/2015-09-14T09:49:53Z]
[2016-04-20 18:34:36,695][INFO ][node ] [Master 228] 
initializing ...
[2016-04-20 18:34:37,100][INFO ][plugins  ] [Master 228] 
loaded [], sites [head, bigdesk]
[2016-04-20 18:34:37,213][INFO ][env  ] [Master 228] 
using [1] data paths, mounts [[/smapp (/dev/mapper/vg02-lvol01)]], net 
usable_space [23.5gb], net total_space [39.2gb], types [ext4]
[2016-04-20 18:34:46,644][INFO ][node ] [Master 228] 
initialized
[2016-04-20 18:34:46,645][INFO ][node ] [Master 228] 
starting ...
[2016-04-20 18:34:46,763][INFO ][transport] [Master 228] 
bound_address {inet[/192.168.178.228:9300]}, publish_address 
{inet[/192.168.178.228:9300]}
[2016-04-20 18:34:46,779][INFO ][discovery] [Master 228] 
UltimatixDev/OFXuALtwQOGN7GU4IdL95Q

server2:
[2016-04-20 18:15:18,642][INFO ][bootstrap] max_open_files 
[65536]
[2016-04-20 18:15:18,894][INFO ][node ] [Master 149] 
version[1.7.5], pid[12297], build[00f95f4/2016-02-02T09:55:30Z]
[2016-04-20 18:15:18,894][INFO ][node ] [Master 149] 
initializing ...
[2016-04-20 18:15:19,081][INFO ][plugins  ] [Master 149] 
loaded [], sites [bigdesk, head]
[2016-04-20 18:15:19,143][INFO ][env  ] [Master 149] 
using [1] data paths, mounts [[/smapp (/dev/mapper/vg02-lvol01)]], net 
usable_space [7gb], net total_space [39.2gb], types [ext4]
[2016-04-20 18:15:23,577][INFO ][node ] [Master 149] 
initialized
[2016-04-20 18:15:23,577][INFO ][node ] [Master 149] 
starting ...
[2016-04-20 18:15:23,658][INFO ][transport] [Master 149] 
bound_address {inet[/10.40.4.149:9300]}, publish_address 
{inet[/10.40.4.149:9300]}
[2016-04-20 18:15:23,672][INFO ][discovery] [Master 149] 
UltimatixDev/PziIysSaTy-lC57MwaJi5g
[2016-04-20 18:15:53,672][WARN ][discovery] [Master 149] 
waited for 30s and no initial state was set by the discovery
[2016-04-20 18:15:53,683][INFO ][http ] [Master 149] 
bound_address {inet[/10.40.4.149:9200]}, publish_address 
{inet[/10.40.4.149:9200]}
[2016-04-20 18:15:53,684][INFO ][node ] [Master 149] 
started


Please Guide.

Regards,
Shrawan

On Wednesday, 20 April 2016 19:59:41 UTC+5:30, Jochen Schalanda wrote:
>
> Hi Shrawan,
>
> looks like your Elasticsearch cluster or at least the communication within 
> the cluster is broken. Please check the logs of your Elasticsearch nodes 
> for error messages and make sure, that each node of the Elasticsearch 
> cluster is able to communicate with the rest of the cluster.
>
> Cheers,
> Jochen
>
> On Wednesday, 20 April 2016 15:17:24 UTC+2, Shrawan Bhagwat wrote:
>>
>> Hi All,
>>
>> I am using graylog-1.3.3 ,elasticsearch-1.7.5 and Logstash-2.2.2.
>>
>> I am getting Reason: There was a problem with your search. We expected 
>> HTTP 200, but got a HTTP 500. on the Graylog UI.
>>
>> I have found following errors in graylog log file:
>>
>> 2016-04-20 18:39:18,019 ERROR: org.graylog2.periodical.AlertScannerThread 
>> - Skipping alert check that threw an exception.
>> org.elasticsearch.discovery.MasterNotDiscoveredException: waited for [30s]
>> at 
>> org.elasticsearch.action.support.master.TransportMasterNodeOperationAction$4.onTimeout(TransportMasterNodeOperationAction.java:164)
>> at 
>> org.elasticsearch.cluster.ClusterStateObserver$ObserverClusterStateListener.onTimeout(ClusterStateObserver.java:231)
>> at 
>> org.elasticsearch.cluster.service.InternalClusterService$NotifyTimeout.run(InternalClusterService.java:560)
>> at 
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>> at 
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>> at java.lang.Thread.run(Thread.java:745)
>> 2016-04-20 18:39:23,467 WARN : 
>> org.elasticsearch.discovery.zen.ping.unicast - [Graylog2 Node 228] failed 
>> to send ping to [[#zen_unicast_2#][CDCUSMDECAPP01][inet[/10.40.4.149:9300]]]
>> org.elasticsearch.transport.SendRequestTransportException: 
>> [][inet[/10.40.4.149:9300]][internal:discovery/zen/unicast_gte_1_4]
>> at 
>> org.elasticsearch.transport.TransportService.sendRequest(TransportService.java:286)
>> at 
>> org.elasticsearch.discovery.zen.ping.unicast.UnicastZenPing.sendPingRequestTo14NodeWithFallback(UnicastZenPing.java:431)
>> at 
>> org.elasticsearch.discovery.zen.ping.unicast.UnicastZenPing.sendPings(UnicastZenPing.java:413)
>> at 
>> org.elasticsearch.discove

Re: [graylog2] Searching broke, no idea what to do (dashboard still works)

2016-04-21 Thread Edmundo Alvarez
Hi Ryan,

First of all, which Graylog version do you use?

I'm afraid that you were right in your first guess: the chart that you were 
trying to generate is still in your browser localStorage and it's breaking the 
search. Would you be so kind as to share with us the graphs that are in your 
browser for further debugging?

You can get the search graphs stored in your browser by executing 
`localStorage.getItem('pinned-field-charts')` and 
`localStorage.getItem('stacked-graphs')` in your browser's developer console.

To delete them, you can execute 
`localStorage.removeItem('pinned-field-charts')` and 
`localStorage.removeItem('stacked-graphs')`. After a page reload, the search 
should come back as usual.

Hope that helps.

Regards,
Edmundo

> On 21 Apr 2016, at 01:46, Ryan Anstey  wrote:
> 
> Appears to be a chrome issue. Incogneto tab does not have the same issue.
> 
> On Tuesday, April 19, 2016 at 3:50:40 PM UTC-7, Ryan Anstey wrote:
> Something went wrong with my search (screenshot). It's empty when I search, 
> but dashboards are still updating. It basically broke when I was searching 
> and creating graphs. I believe it spat something out at me about not being 
> able to do something because it wasn't a number field before basically all 
> the results went away.
> 
> I think it must have to do with how it saves recent graphs and stuff in your 
> search results. Is there any way to reset that info? I could be wrong though, 
> I tried using a different user and they can't search either
> 
> I have:
>   • Logged out, back in
>   • Tried another user
>   • Restarted graylog-server
>   • Restarted graylog-web
>   • Restarted elasticsearch
>   • Recalculated index ranges (Range re-calculated 22 minutes ago in 
> 1135ms. 29 segments, 0 open search contexts, 0 deleted messages)
>   • Checked the error log, but no logs are created when the search 
> fails it's just emptiness
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Graylog Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to graylog2+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/graylog2/8cca5b22-cb84-4aa7-a704-a32c3dc33da5%40googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/graylog2/D6047357-57E7-4C75-AAE8-798234D98698%40graylog.com.
For more options, visit https://groups.google.com/d/optout.


[graylog2] graylog 2.0 beta 3, nginx & https

2016-04-21 Thread Josep Maria Comas Serrano
We've configured nginx to reverse proxy port 80 to port 9000, and port 
12900 to 12900


To make it work, we've configured 


rest_transport_uri = http://public_ip:12900/


on graylog's server.conf

It works fine, 

Now we're trying to reverse proxy port 443 to port 9000, in order to have 
ssl enabled on the nginx front-end

After we enter user & password, nothing happens. It stays forever on the 
login dialog page.

It seems that it's not able to connect to port 12900, but we don't know 
why, any ideas about this behaviour? Anything missing on the configuration?

Best

JM


-- 
You received this message because you are subscribed to the Google Groups 
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/graylog2/decbb0f5-f8b4-4051-990c-1a15311766c7%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


[graylog2] Re: graylog 2.0 beta 3, nginx & https

2016-04-21 Thread Jochen Schalanda
Hi Josep,

if you're using HTTPS for the web interface, the Graylog REST API must also 
be accessed via HTTPS to prevent mixed content warnings of your web browser.

See 
https://developers.google.com/web/fundamentals/security/prevent-mixed-content/what-is-mixed-content
 
for details about mixed content.

Cheers,
Jochen

On Thursday, 21 April 2016 10:53:36 UTC+2, Josep Maria Comas Serrano wrote:
>
> We've configured nginx to reverse proxy port 80 to port 9000, and port 
> 12900 to 12900
>
>
> To make it work, we've configured 
>
>
> rest_transport_uri = http://public_ip:12900/
>
>
> on graylog's server.conf
>
> It works fine, 
>
> Now we're trying to reverse proxy port 443 to port 9000, in order to have 
> ssl enabled on the nginx front-end
>
> After we enter user & password, nothing happens. It stays forever on the 
> login dialog page.
>
> It seems that it's not able to connect to port 12900, but we don't know 
> why, any ideas about this behaviour? Anything missing on the configuration?
>
> Best
>
> JM
>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/graylog2/d20c7404-c7d6-4d35-8779-c9912650faa6%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


[graylog2] Re: Reason: There was a problem with your search. We expected HTTP 200, but got a HTTP 500.

2016-04-21 Thread Jochen Schalanda
Hi Shrawan,

as I said, make sure that all Elasticsearch nodes (including Graylog) in 
your cluster can communicate with each other. The IP addresses also look 
vastly different, so maybe you want to explicitly set those for each 
Elasticsearch node.

On a side note, you might want to use the same Elasticsearch version 
everywhere (i. e. ES 1.7.5).


Cheers,
Jochen

On Thursday, 21 April 2016 09:31:04 UTC+2, Shrawan Bhagwat wrote:
>
> Hi Jochen,
>
> I am not getting any error message in Elasticsearch logs on both the 
> servers.
> Server1:
> [2016-04-20 18:34:36,386][INFO ][bootstrap] max_open_files 
> [65536]
> [2016-04-20 18:34:36,695][INFO ][node ] [Master 228] 
> version[1.7.2], pid[2149], build[e43676b/2015-09-14T09:49:53Z]
> [2016-04-20 18:34:36,695][INFO ][node ] [Master 228] 
> initializing ...
> [2016-04-20 18:34:37,100][INFO ][plugins  ] [Master 228] 
> loaded [], sites [head, bigdesk]
> [2016-04-20 18:34:37,213][INFO ][env  ] [Master 228] 
> using [1] data paths, mounts [[/smapp (/dev/mapper/vg02-lvol01)]], net 
> usable_space [23.5gb], net total_space [39.2gb], types [ext4]
> [2016-04-20 18:34:46,644][INFO ][node ] [Master 228] 
> initialized
> [2016-04-20 18:34:46,645][INFO ][node ] [Master 228] 
> starting ...
> [2016-04-20 18:34:46,763][INFO ][transport] [Master 228] 
> bound_address {inet[/192.168.178.228:9300]}, publish_address {inet[/
> 192.168.178.228:9300]}
> [2016-04-20 18:34:46,779][INFO ][discovery] [Master 228] 
> UltimatixDev/OFXuALtwQOGN7GU4IdL95Q
>
> server2:
> [2016-04-20 18:15:18,642][INFO ][bootstrap] max_open_files 
> [65536]
> [2016-04-20 18:15:18,894][INFO ][node ] [Master 149] 
> version[1.7.5], pid[12297], build[00f95f4/2016-02-02T09:55:30Z]
> [2016-04-20 18:15:18,894][INFO ][node ] [Master 149] 
> initializing ...
> [2016-04-20 18:15:19,081][INFO ][plugins  ] [Master 149] 
> loaded [], sites [bigdesk, head]
> [2016-04-20 18:15:19,143][INFO ][env  ] [Master 149] 
> using [1] data paths, mounts [[/smapp (/dev/mapper/vg02-lvol01)]], net 
> usable_space [7gb], net total_space [39.2gb], types [ext4]
> [2016-04-20 18:15:23,577][INFO ][node ] [Master 149] 
> initialized
> [2016-04-20 18:15:23,577][INFO ][node ] [Master 149] 
> starting ...
> [2016-04-20 18:15:23,658][INFO ][transport] [Master 149] 
> bound_address {inet[/10.40.4.149:9300]}, publish_address {inet[/
> 10.40.4.149:9300]}
> [2016-04-20 18:15:23,672][INFO ][discovery] [Master 149] 
> UltimatixDev/PziIysSaTy-lC57MwaJi5g
> [2016-04-20 18:15:53,672][WARN ][discovery] [Master 149] 
> waited for 30s and no initial state was set by the discovery
> [2016-04-20 18:15:53,683][INFO ][http ] [Master 149] 
> bound_address {inet[/10.40.4.149:9200]}, publish_address {inet[/
> 10.40.4.149:9200]}
> [2016-04-20 18:15:53,684][INFO ][node ] [Master 149] 
> started
>
>
> Please Guide.
>
> Regards,
> Shrawan
>
> On Wednesday, 20 April 2016 19:59:41 UTC+5:30, Jochen Schalanda wrote:
>>
>> Hi Shrawan,
>>
>> looks like your Elasticsearch cluster or at least the communication 
>> within the cluster is broken. Please check the logs of your Elasticsearch 
>> nodes for error messages and make sure, that each node of the Elasticsearch 
>> cluster is able to communicate with the rest of the cluster.
>>
>> Cheers,
>> Jochen
>>
>> On Wednesday, 20 April 2016 15:17:24 UTC+2, Shrawan Bhagwat wrote:
>>>
>>> Hi All,
>>>
>>> I am using graylog-1.3.3 ,elasticsearch-1.7.5 and Logstash-2.2.2.
>>>
>>> I am getting Reason: There was a problem with your search. We expected 
>>> HTTP 200, but got a HTTP 500. on the Graylog UI.
>>>
>>> I have found following errors in graylog log file:
>>>
>>> 2016-04-20 18:39:18,019 ERROR: 
>>> org.graylog2.periodical.AlertScannerThread - Skipping alert check that 
>>> threw an exception.
>>> org.elasticsearch.discovery.MasterNotDiscoveredException: waited for 
>>> [30s]
>>> at 
>>> org.elasticsearch.action.support.master.TransportMasterNodeOperationAction$4.onTimeout(TransportMasterNodeOperationAction.java:164)
>>> at 
>>> org.elasticsearch.cluster.ClusterStateObserver$ObserverClusterStateListener.onTimeout(ClusterStateObserver.java:231)
>>> at 
>>> org.elasticsearch.cluster.service.InternalClusterService$NotifyTimeout.run(InternalClusterService.java:560)
>>> at 
>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>>> at 
>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>>> at java.lang.Thread.run(Thread.java:745)
>>> 2016-04-20 18:39:23,467 WARN : 
>>> org.elasticsearch.discovery.zen.ping.unicast - [Graylog2 Node 228] failed 
>>> to s

[graylog2] Re: Graylog 2 beta API port listening issue in the website

2016-04-21 Thread Jochen Schalanda
Hi Eric,

starting with Graylog 2.0.0 the old web interface has been merged into the 
server and is now served as a single-page application which communicates 
directly with the Graylog REST API.

The direct result of this is that each client using the Graylog web 
interface must also be able to communicate with the Graylog REST API.

Cheers,
Jochen

On Thursday, 21 April 2016 09:03:45 UTC+2, er...@muneris.io wrote:
>
> Dear Graylog community support / users,
>
> I have been using Graylog since 1.2 and its working great.
>
> Just discover a change about a health check in Graylog's web just might 
> cause problems.
> It's known and normal that the Graylog's web service detects the server 
> node(s) healthiness with API thru TCP 12900.
>
> However I noticed an issue in Graylog 2.
> When I am trying out Graylog 2 (Alpha and Beta), the web UI automatically 
> calls TCP 12900 (API port) in the client side using the public address.
> That is, from the developer mode of the browser, I can see URL call of 
> http:// web service hostname>:12900/system/cluster/node. This causes the 
> following issues:
>
> 1) With the default configuration, such check listens to private IP of the 
> server. So just when deploying the Graylog to internet, the check fails. 
> (Unless we access the website through VPN IP or update 
> *rest_transport_uri* in /opt/graylog/conf/graylog.conf)
> 2) Health check should probably be done in background in the server (i.e. 
> like Graylog 1.2, 1.3...the checking will not be exposed to client side / 
> browser)
> 3) We need to expose TCP 12900 of the web service to public, security 
> concern arises as the API port would be facing the public internet as well
>
> Thank you.
> Eric
>

-- 
You received this message because you are subscribed to the Google Groups 
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/graylog2/2fdf0e7b-eb7c-40e5-92e4-a47e278b96e6%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


[graylog2] Re: fielddata error with search

2016-04-21 Thread Jochen Schalanda
Hi Jason,

we'll gradually improve the error handling in Graylog if the need arises.

As for the underlying problem, I can only recommend the immensely unpopular 
"throw hardware at it or reduce your data size". You can add more nodes to 
your cluster or add more memory to the existing nodes (within reason and 
preferably less than 30.5 GB of heap memory).

Cheers,
Jochen

On Thursday, 21 April 2016 00:48:57 UTC+2, Jason Haar wrote:
>
> Hi there
>
> I tried to do what I thought was a simple search across a week's worth of 
> data on a single-box graylog server (ie it also has ES and mongodb on it)
>
> Basically I did a search for "fieldname:value1 OR fieldname:value2 OR 
> fieldname:value3" over 7days and graylog just sat there spinning it's 
> wheels (before hand I was happily doing searches that weren't causing any 
> grief at all)
>
> The CPU on the graylog server went through the roof, graylog error file 
> showed no problem, but ES logs showed a bunch of these
>
> indices.breaker.fielddata] [fielddata] New used memory 11155918063 
> [10.3gb] for data of [message] would be larger than configured breaker: 
> 10857952051 [10.1gb], breaking
>
> After five minutes of graylog just sitting there, I restarted ES, but 
> graylog was now borked. The input channels were still receiving data, but 
> nothing was flowing out. So I restarted graylog and all was good again
>
> Is this expected behaviour, and if so, what is needed to stop it? I've 
> seen other non-graylog related postings on the ES list about this happening 
> with large clusters, so it seems to be an error case for ES, but I'm more 
> concerned over how graylog reacted: ie why didn't it give up and give me an 
> error page for starters. It looks to me like graylog didn't expect that ES 
> search to error out and that caused it to block? (I'm assuming ES generated 
> an error - the logs shows that WARN - I dunno what happens next)
>
>
> -- 
> Cheers
>
> Jason Haar
> Information Security Manager, Trimble Navigation Ltd.
> Phone: +1 408 481 8171
> PGP Fingerprint: 7A2E 0407 C9A6 CAF6 2B9F 8422 C063 5EBB FE1D 66D1
>

-- 
You received this message because you are subscribed to the Google Groups 
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/graylog2/bc57f7c9-a5ce-4dea-a630-1b6cac866241%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


Re: [graylog2] Graylog 2 beta API port listening issue in the website

2016-04-21 Thread Dennis Oelkers
Hey Eric,

regarding point 3: what are your exact security concerns about exposing the 
REST API?

Kind regards,
D.

--
Tel.: +49 (0)40 609 452 077
Fax.: +49 (0)40 609 452 078

TORCH GmbH - A Graylog company
Steckelhörn 11
20457 Hamburg
Germany

Commercial Reg. (Registergericht): Amtsgericht Hamburg, HRB 125175
Geschäftsführer: Lennart Koopmann (CEO)

> On 21.04.2016, at 09:03, er...@muneris.io wrote:
> 
> Dear Graylog community support / users,
> 
> I have been using Graylog since 1.2 and its working great.
> 
> Just discover a change about a health check in Graylog's web just might cause 
> problems.
> It's known and normal that the Graylog's web service detects the server 
> node(s) healthiness with API thru TCP 12900.
> 
> However I noticed an issue in Graylog 2.
> When I am trying out Graylog 2 (Alpha and Beta), the web UI automatically 
> calls TCP 12900 (API port) in the client side using the public address.
> That is, from the developer mode of the browser, I can see URL call of 
> http://:12900/system/cluster/node. This causes 
> the following issues:
> 
> 1) With the default configuration, such check listens to private IP of the 
> server. So just when deploying the Graylog to internet, the check fails. 
> (Unless we access the website through VPN IP or update rest_transport_uri in 
> /opt/graylog/conf/graylog.conf)
> 2) Health check should probably be done in background in the server (i.e. 
> like Graylog 1.2, 1.3...the checking will not be exposed to client side / 
> browser)
> 3) We need to expose TCP 12900 of the web service to public, security concern 
> arises as the API port would be facing the public internet as well
> 
> Thank you.
> Eric
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Graylog Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to graylog2+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/graylog2/a43a9ea9-2b6b-4d6a-8b91-1304b84dd008%40googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/graylog2/7FE12566-B7BC-41BB-810F-BE3D31D632EF%40graylog.com.
For more options, visit https://groups.google.com/d/optout.


[graylog2] Re: fielddata error with search

2016-04-21 Thread Daniel Kamiński
you can change 'message' mapping template in ES via it's rest api, and add 
`"doc_values": true` to some less needed fields, more info or doc values 
here: 
https://www.elastic.co/guide/en/elasticsearch/reference/current/doc-values.html

W dniu czwartek, 21 kwietnia 2016 00:48:57 UTC+2 użytkownik Jason Haar 
napisał:
>
> Hi there
>
> I tried to do what I thought was a simple search across a week's worth of 
> data on a single-box graylog server (ie it also has ES and mongodb on it)
>
> Basically I did a search for "fieldname:value1 OR fieldname:value2 OR 
> fieldname:value3" over 7days and graylog just sat there spinning it's 
> wheels (before hand I was happily doing searches that weren't causing any 
> grief at all)
>
> The CPU on the graylog server went through the roof, graylog error file 
> showed no problem, but ES logs showed a bunch of these
>
> indices.breaker.fielddata] [fielddata] New used memory 11155918063 
> [10.3gb] for data of [message] would be larger than configured breaker: 
> 10857952051 [10.1gb], breaking
>
> After five minutes of graylog just sitting there, I restarted ES, but 
> graylog was now borked. The input channels were still receiving data, but 
> nothing was flowing out. So I restarted graylog and all was good again
>
> Is this expected behaviour, and if so, what is needed to stop it? I've 
> seen other non-graylog related postings on the ES list about this happening 
> with large clusters, so it seems to be an error case for ES, but I'm more 
> concerned over how graylog reacted: ie why didn't it give up and give me an 
> error page for starters. It looks to me like graylog didn't expect that ES 
> search to error out and that caused it to block? (I'm assuming ES generated 
> an error - the logs shows that WARN - I dunno what happens next)
>
>
> -- 
> Cheers
>
> Jason Haar
> Information Security Manager, Trimble Navigation Ltd.
> Phone: +1 408 481 8171
> PGP Fingerprint: 7A2E 0407 C9A6 CAF6 2B9F 8422 C063 5EBB FE1D 66D1
>

-- 
You received this message because you are subscribed to the Google Groups 
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/graylog2/56da2ae1-ed27-410d-b1dd-3a86e5716972%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


[graylog2] Re: Multiple retention times still not possible?

2016-04-21 Thread tokred
Hi Jochen,

thank you for your reply! From my perspective, your answer is kind of 
disturbing and irritating which is why I would be happy to discuss this 
further because I would like to understand the "why not". I am aware of 
Graylog's open source nature and the fact that developers work on this in 
their free time, so of course I have no right to claim certain features. No 
offence is intented in the following:

Some thoughts:

1. Maybe it is a feature that would require too many dev resources or it is 
an architectural issue that would require too many changes. While this 
would be sad, maybe only a small-scaled, basic solution could be realized 
(also see 2.)? Or maybe it's a candidate for a commercial feature?

2. Maybe it is a simple misunderstanding of the requirement. You talk about 
"concurrent and possibly incompatible" index schemes, however I am not sure 
how there could occur any incompatibility in retention times. 
My perspective/wish/understanding: I would simply like to be able to keep 
one large bunch of logs for X days and another, second bunch of 
distinctivly defined logs for Y (!=X) days, so no incompatibility (right?). 
I would be able to separate these two types e.g. based on inputs. From my 
understanding of the Graylog architecture, this would require two indices 
instead of one plus the required handling (deflectors etc.) with a 
retention strategy per index. Then, messages from one input could be routed 
into the first index and those from another input into the second index. I 
think many people could handle their environment with such a simple 
primary/secondary (master/slave, you-name-it,... simply 2 instead of 1) 
index scheme so there would be no need for a huge, work-intense solution to 
support a large arbitrary number of different indices. I think going from 1 
index to 2 indices is much easier than going from 1 to many (correct me if 
I'm wrong).

3. Maybe you (not necessarily you in person, but the dev team in general) 
do not consider the feature as relevant/essential/desirable and too 
specialized from your perspective. The Graylog idea portal suggests 
otherwise, because the idea with the most votes 
(https://graylog.ideas.aha.io/ideas/GL2E-I-356) is exactly the demand for 
multiple retention times and it has been opened almost 1 year ago. In 
January, Matt confirmed its consideration in v2.0 btw, could you please 
comment on that?

Thank you in advance for your respone.

Best regards,
tokred


On Monday, April 18, 2016 at 5:20:24 PM UTC+2, Jochen Schalanda wrote:
>
> Hi tokred,
>
> support for multiple (concurrent and possibly incompatible) index schemes 
> and retention times is not included in Graylog 2.0.0 and currently isn't on 
> the roadmap in the mid-term.
>
>
> Cheers,
> Jochen
>
> On Friday, 15 April 2016 14:49:59 UTC+2, tok...@gmx.net wrote:
>>
>> Hi all,
>>
>> I really appreciate the recent developments of Graylog towards 2.0 and 
>> big applause for the efforts of the developers!
>>
>> However, I am still desperately hoping/waiting for a features which has 
>> been number 1 on the idea portal for almost a year (!) - multiple indices 
>> with different retention times. From my understanding, the archiving 
>> feature does not cover this, right?
>>
>> Kindly ask for any plans or alternatives.
>>
>>
>> Best regards,
>> tokred
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/graylog2/dd610552-2301-454d-addf-525b8b9a1716%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


[graylog2] Re: fielddata error with search

2016-04-21 Thread Jochen Schalanda
Hi Daniel,

doc values don't work for analyzed string fields like "message": 

Doc values are supported on almost all field types, with the notable 
> exception of analyzed string fields.
>

Unfortunately that's exactly the field which trips the field data cache 
circuit breaker in Jason's case.


Cheers,
Jochen 

On Thursday, 21 April 2016 13:14:46 UTC+2, Daniel Kamiński wrote:
>
> you can change 'message' mapping template in ES via it's rest api, and add 
> `"doc_values": true` to some less needed fields, more info or doc values 
> here: 
> https://www.elastic.co/guide/en/elasticsearch/reference/current/doc-values.html
>
> W dniu czwartek, 21 kwietnia 2016 00:48:57 UTC+2 użytkownik Jason Haar 
> napisał:
>>
>> Hi there
>>
>> I tried to do what I thought was a simple search across a week's worth of 
>> data on a single-box graylog server (ie it also has ES and mongodb on it)
>>
>> Basically I did a search for "fieldname:value1 OR fieldname:value2 OR 
>> fieldname:value3" over 7days and graylog just sat there spinning it's 
>> wheels (before hand I was happily doing searches that weren't causing any 
>> grief at all)
>>
>> The CPU on the graylog server went through the roof, graylog error file 
>> showed no problem, but ES logs showed a bunch of these
>>
>> indices.breaker.fielddata] [fielddata] New used memory 11155918063 
>> [10.3gb] for data of [message] would be larger than configured breaker: 
>> 10857952051 [10.1gb], breaking
>>
>> After five minutes of graylog just sitting there, I restarted ES, but 
>> graylog was now borked. The input channels were still receiving data, but 
>> nothing was flowing out. So I restarted graylog and all was good again
>>
>> Is this expected behaviour, and if so, what is needed to stop it? I've 
>> seen other non-graylog related postings on the ES list about this happening 
>> with large clusters, so it seems to be an error case for ES, but I'm more 
>> concerned over how graylog reacted: ie why didn't it give up and give me an 
>> error page for starters. It looks to me like graylog didn't expect that ES 
>> search to error out and that caused it to block? (I'm assuming ES generated 
>> an error - the logs shows that WARN - I dunno what happens next)
>>
>>
>> -- 
>> Cheers
>>
>> Jason Haar
>> Information Security Manager, Trimble Navigation Ltd.
>> Phone: +1 408 481 8171
>> PGP Fingerprint: 7A2E 0407 C9A6 CAF6 2B9F 8422 C063 5EBB FE1D 66D1
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/graylog2/a2aee280-56e8-4705-8c7b-4edd39a0d5a8%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


Re: [graylog2] High-Availability Graylog deployment

2016-04-21 Thread John Buchanan
Out of curiosity, have you switched MongoDB to use authentication, to 
protect that configuration info contained therein?  I'm working on bringing 
our production setup online (3 centos 6.7 VM's each with graylog-server, 
graylog-web, and MongoDB, 2 HP DL380 Elasticsearch servers, Kemp HA 
loadmaster pair). I have everything up and running, the 3 mongo instances 
running in replica mode, but I'm not yet using any authentication to Mongo 
and want to get that online, but haven't used Mongo previous to this 
project and need to learn more about the command syntax and configuration 
methodology.

On Monday, April 11, 2016 at 2:54:44 AM UTC-5, ryangr...@outlook.com wrote:
>
> Hi Micheal,
>
> I don't think the first URI is necessarily used as primary.
>
> I rechecked our source code; setting multiple URIs in mongodb_uri gets 
> around this need. If any of the supplied URIs point to the mongoDb primary, 
> Graylog will work as expected. In our case, it made it irrelevant which 
> MongoDB 
> server was primary, since MongoDB will automatically elect one server in a 
> replica set to primary.
>
> Regards,
> Ryan
>
> On Thursday, April 7, 2016 at 2:21:41 PM UTC+1, Michael Laporte wrote:
>>
>> Hi Ryan
>>
>> Did you ever find out if the first Mongo_uri server is used as the 
>> primary?
>>
>>
>>
>> On Friday, 26 February 2016 08:45:33 UTC, ryangr...@outlook.com wrote:
>>>
>>> Hi,
>>>
>>> Thank you very much; your suggested changes to mongodb_uri worked 
>>> perfectly, and our several-node Graylog cluster is now working as expected.
>>>
>>> On Monday, February 22, 2016 at 5:33:52 PM UTC, Florent B wrote:

 Hi,

 You can set multiple MongoDB servers in mongodb_uri as specified in 
 example config file :

 mongodb_uri = 
 mongodb://grayloguser:secret@localhost:27017,localhost:27018,localhost:27019/graylog2

 On 02/22/2016 03:39 PM, ryangr...@outlook.com wrote:


 Hello,

 We're currently implementing a setup with three-graylog servers, and 
 we're having a lot of problems getting MongoDb to work correctly with 
 Graylog:

- In order to run a cluster of three Graylog (1.2.1) servers, you 
need to set up a MongoDB replica set.
- 'mongodb_uri' (set in the configuration file) needs to point to 
the MongoDB replica set *primary*; (in this case we try to set it 
to "vm-0").
- Graylog fails with the following error if the URI does not point 
to the replica set primary (at least on initial startup):


 ERROR [CmdLineTool] Guice error (more detail on log level debug): Error 
 injecting constructor, com.mongodb.CommandFailureException: { 
 "serverUsed" : "vm-0:27017" , "ok" : 0.0 , "errmsg" : "not master" , 
 "code" : 10107}



- There is no way to specify three mongodb_uri values, and let 
Graylog 'detect' which of these is primary.
- It is difficult to force MongoDB to set a particular VM to be 
primary reliably. 

 This means that it is difficult to guarantee that mongodb_uri is 
 pointing to the current MongoDB master. There's only two ways I know how 
 to 
 ensure Graylog' mongo_uri is pointing to the current master:

- Dynamically change the Graylog configuration file to point to the 
current primary each time Graylog starts up. 
- Write scripts to try force MongoDB's to place its primary on a 
particular server. 

 Neither of these are very good solutions, and manual setup is not an 
 option (the deployment is fully automated).

 Is there an easier way of configuring Graylog to always point towards 
 the current MongoDB replica set primary? Unfortunately, my options don't 
 include upgrading to 2.0.

 Thanks for any help you can offer; if any clarification is needed 
 please leave an email and I'll respond quickly.


 *Server Layout:*

 VM-0:

- Graylog-Server
- ElasticSearch
- MongoDb


 VM-1:

- Graylog-Server
- Graylog-Web
- ElasticSearch
- MongoDb



 VM-2:

- Graylog-Server
- ElasticSearch
- MongoDb


 *Versions:*


- Web1.3.3 
- Server 1.2.2 



 *Graylog's MongoDb-Related Configuration:*

 mongodb_useauth = false
 mongodb_uri = mongodb:
 //vm-0:27017/graylog2
 mongodb_max_connections = 100
 mongodb_threads_allowed_to_block_multiplier = 5
 mongodb_replica_set = vm-0:27017,vm-1:
 27017,vm-2:27017


 *MongoDB Configuration:*

 storage:
   dbPath: /var/lib/mongodb
   journal:
 enabled: true


 systemLo

[graylog2] [ANNOUNCE] Graylog v2.0.0-beta.3 has been released

2016-04-21 Thread Bernd Ahlers
Hey folks,

we just released Graylog v2.0.0-rc.1. Read more in the release announcement:

https://www.graylog.org/blog/54-announcing-graylog-v2-0-rc-1

Have fun!

Regards,
Bernd

-- 
You received this message because you are subscribed to the Google Groups 
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/graylog2/CAA%2BQeYUp3F5%3DfShyZ4as1kMaBigRGrkvErtBSK0_hVVpZCeRJw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.


[graylog2] Re: [ANNOUNCE] Graylog v2.0.0-beta.3 has been released

2016-04-21 Thread Bernd Ahlers
Oups, wrong subject. Sent another one. :-D

On 21 April 2016 at 18:11, Bernd Ahlers  wrote:
> Hey folks,
>
> we just released Graylog v2.0.0-rc.1. Read more in the release announcement:
>
> https://www.graylog.org/blog/54-announcing-graylog-v2-0-rc-1
>
> Have fun!
>
> Regards,
> Bernd



-- 
Developer

Tel.: +49 (0)40 609 452 077
Fax.: +49 (0)40 609 452 078

TORCH GmbH - A Graylog company
Steckelhörn 11
20457 Hamburg
Germany

Commercial Reg. (Registergericht): Amtsgericht Hamburg, HRB 125175
Geschäftsführer: Lennart Koopmann (CEO)

-- 
You received this message because you are subscribed to the Google Groups 
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/graylog2/CAA%2BQeYW9%2BGq6Z1ABdfVrgh%2B6oRWwa1WjnJaLU-xxoBDW_gsYDw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.


[graylog2] [ANNOUNCE] Graylog v2.0.0-rc.1 has been released

2016-04-21 Thread Bernd Ahlers
Hey folks,

we just released Graylog v2.0.0-rc.1. Read more in the release announcement:

https://www.graylog.org/blog/54-announcing-graylog-v2-0-rc-1

Have fun!

Regards,
Bernd

-- 
You received this message because you are subscribed to the Google Groups 
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/graylog2/CAA%2BQeYUuj3C9Xd60WEPJ-Zezv0Oy1s7X%3DoOL2bE19URueO76iA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.


Re: [graylog2] Searching broke, no idea what to do (dashboard still works)

2016-04-21 Thread Ryan Anstey
localStorage.getItem('pinned-field-charts')
"{"jr586onx5yk44om1d33ztwi8fxysddc0albt":{"chartid":"jr586onx5yk44om1d33ztwi8fxysddc0albt","field":"fxx_qty_pull","interval":"hour","streamid":"","interpolation":"basis","renderer":"area","valuetype":"total","query":"_exists_:fxx_qty_pull","rangetype":"relative","range":{"relative":604800},"createdAt":1461099659351},"x3j20k5nf04g8b0enthcblipwgche8zh8s5j":{"chartid":"x3j20k5nf04g8b0enthcblipwgche8zh8s5j","field":"fxx_qty_pull","interval":"day","streamid":"","interpolation":"basis","renderer":"area","valuetype":"total","query":"_exists_:fxx_qty_pull","rangetype":"relative","range":{"relative":2592000},"createdAt":1461100530497},"fbzbvwtn0e22i52him0rajiav18mbkpeg10j":{"chartid":"fbzbvwtn0e22i52him0rajiav18mbkpeg10j","field":"fx_qtyshp","interval":"hour","streamid":"","interpolation":"bundle","renderer":"area","valuetype":"total","query":"_exists_:fx_qtyshp","rangetype":"relative","range":{"relative":604800},"createdAt":1461099596810},"1xuq29v3j1gu208kdrrkbrm2pn5m2v2aqalj":{"chartid":"1xuq29v3j1gu208kdrrkbrm2pn5m2v2aqalj","field":"fx_qty_ship","interval":"day","streamid":"","interpolation":"basis","renderer":"area","valuetype":"total","query":"_exists_:fx_qty_ship","rangetype":"relative","range":{"relative":2592000},"createdAt":1461100592817}}"

localStorage.removeItem('stacked-graphs')
undefined

On Thursday, April 21, 2016 at 1:34:57 AM UTC-7, Edmundo Alvarez wrote:
>
> Hi Ryan, 
>
> First of all, which Graylog version do you use? 
>
> I'm afraid that you were right in your first guess: the chart that you 
> were trying to generate is still in your browser localStorage and it's 
> breaking the search. Would you be so kind as to share with us the graphs 
> that are in your browser for further debugging? 
>
> You can get the search graphs stored in your browser by executing 
> `localStorage.getItem('pinned-field-charts')` and 
> `localStorage.getItem('stacked-graphs')` in your browser's developer 
> console. 
>
> To delete them, you can execute 
> `localStorage.removeItem('pinned-field-charts')` and 
> `localStorage.removeItem('stacked-graphs')`. After a page reload, the 
> search should come back as usual. 
>
> Hope that helps. 
>
> Regards, 
> Edmundo 
>
> > On 21 Apr 2016, at 01:46, Ryan Anstey > 
> wrote: 
> > 
> > Appears to be a chrome issue. Incogneto tab does not have the same 
> issue. 
> > 
> > On Tuesday, April 19, 2016 at 3:50:40 PM UTC-7, Ryan Anstey wrote: 
> > Something went wrong with my search (screenshot). It's empty when I 
> search, but dashboards are still updating. It basically broke when I was 
> searching and creating graphs. I believe it spat something out at me about 
> not being able to do something because it wasn't a number field before 
> basically all the results went away. 
> > 
> > I think it must have to do with how it saves recent graphs and stuff in 
> your search results. Is there any way to reset that info? I could be wrong 
> though, I tried using a different user and they can't search either 
> > 
> > I have: 
> > • Logged out, back in 
> > • Tried another user 
> > • Restarted graylog-server 
> > • Restarted graylog-web 
> > • Restarted elasticsearch 
> > • Recalculated index ranges (Range re-calculated 22 minutes ago 
> in 1135ms. 29 segments, 0 open search contexts, 0 deleted messages) 
> > • Checked the error log, but no logs are created when the search 
> fails it's just emptiness 
> > 
> > -- 
> > You received this message because you are subscribed to the Google 
> Groups "Graylog Users" group. 
> > To unsubscribe from this group and stop receiving emails from it, send 
> an email to graylog2+u...@googlegroups.com . 
> > To view this discussion on the web visit 
> https://groups.google.com/d/msgid/graylog2/8cca5b22-cb84-4aa7-a704-a32c3dc33da5%40googlegroups.com.
>  
>
> > For more options, visit https://groups.google.com/d/optout. 
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/graylog2/15e87c50-3a46-4192-9bc3-0d005de7915a%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


Re: [graylog2] Re: Graylog-web time range problem

2016-04-21 Thread hasan akgöz
Hi Jochen,

I just saw your post. I solved the problem. I've arranged Elasticsearch In 
the time zone again. like this is -Duser.timezone=Europe/Istanbul

thank you for attention.

On Monday, April 18, 2016 at 6:22:42 PM UTC+3, Jochen Schalanda wrote:
>
> Hi Hasan,
>
> please try using a "relative" timezone for your user (e. g. UTC+3) to rule 
> out any problems with the used data/time library.
>
> FWIW, the Elasticsearch query is completely correct as all messages are 
> indexed with a timestamp in the timezone UTC, so it's most likely just a 
> display problem.
>
>
> Cheers,
> Jochen
>

-- 
You received this message because you are subscribed to the Google Groups 
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/graylog2/72373b7f-19fa-448b-89f5-77b719a152b7%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


[graylog2] Extractor reliability on Syslog-UDP input

2016-04-21 Thread John Buchanan
I've got my production Graylog-1.3.4 cluster online this week, primarily 
intaking logs via Syslog-TCP and Syslog-UDP Inputs. I'm running extractors 
for source / destination IP, and requested hostname thus far. Extraction 
works like a charm on my Syslog-TCP Input, but seems very hit or miss on my 
UDP input. Now, the message inflow rate on the UDP Input is significantly 
higher than TCP, 100 - 1,100 msgs/s. I have 3 CentOS 6.7 VM's (on beefy 
Vmware clustered hosts) running graylog-server/web and MongoDB, and a pair 
(3rd on order) of HP DL380 Gen9 Elasticsearch servers online, with 
everything flowing through a HA pair of Kemp Lm2600 balancers. 

I am using "replace with regular expression" for hostname extraction, which 
performs flawlessly in the Create/Edit Extractor screen, but seemingly only 
occasionally on the live flow of messages. I am using one of the GROK 
patterns to pull source and destination IP addresses. Again, my extraction 
success rate on the Syslog-TCP Input appears to be 100%, but hardly 10% on 
Syslog-UDP.

Log source(s) for TCP are 2 HA pair of F5 BIGIP ASM modules, for UDP a pair 
of Cisco SourceFire IPS boxes.

Thanks much

John

-- 
You received this message because you are subscribed to the Google Groups 
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/graylog2/b3fe22cc-3ae5-4247-87ee-68f4bf029e35%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


[graylog2] Permission to see the log

2016-04-21 Thread hasan akgöz
Hi Guys,

I want to  some logs such as apache log records to see or search only the 
developers and  only system administrator see logs of syslog . What is the 
best way to do this? I think only stream. What else can be ?

have a nice day

-- 
You received this message because you are subscribed to the Google Groups 
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/graylog2/25aca3bd-6b26-4215-ab0b-e3a85cae6780%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


[graylog2] Regex extractor for xml logs

2016-04-21 Thread Nik Horvat
I'm running the appliance of 2.0-beta3 at home just to test things out and 
I'm running into an issue parsing a log that comes in an XML format using 
regex.  They are single line messages and I'm passing them in using 
filebeat.I can't get a regex to match the tags in the message to allow 
me to parse out the information.  

For example (using the message below), I was using the regex  
(.*)<\/ID> to try to parse out the ID field and i get no matches. I'm 
not the greatest at regex, but every tester I've tried matches the data in 
the ID field when using that pattern.  I tried escaping all of the angle 
brackets and that didn't change the result.  Trying to match directly on 
any particular string fails too.  If I try using classes I get odd results 
like:

([[:graph:]]) or ([[:ascii:]]) i get a match on 'a'.
([[:alnum:]]) i get a match on 'n'.  



*Message:*
04f4f9f8-24db-4f30-bfa0-cf4197383ac12016-04-21T22:57:34.9232a4ff629-31bb-48a9-b9a0-79249142b5c12PimIndexMaintenanceSvc_50ccaC:\WINDOWS\system32\svchost.exe
 
-k 
UnistackSvcGroupC:\WINDOWS\system32\svchost.exe0439448497852ED44AFF902D502015792D315D4Google Updater UA - User - 04/21/16 - 3:59:02 
pm

I know its probably something small I'm missing in my regex, but I'm at my 
wit's end with this one.  Any suggestions?

-- 
You received this message because you are subscribed to the Google Groups 
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/graylog2/f967b16d-6ed3-4a18-b9d6-cf4921938335%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


Re: [graylog2] Graylog 2 beta API port listening issue in the website

2016-04-21 Thread Eric Tang
Dear Jochen,

Thank you for your answer.


Dear Dennis,

It's not really a big matter as the APIs require authentication. Though we
don't use the API from public network so it's good to hide it up and
prevent any DDoS in case :P .
Thank you.

Eric

On Thu, Apr 21, 2016 at 7:05 PM, Dennis Oelkers  wrote:

> Hey Eric,
>
> regarding point 3: what are your exact security concerns about exposing
> the REST API?
>
> Kind regards,
> D.
>
> --
> Tel.: +49 (0)40 609 452 077
> Fax.: +49 (0)40 609 452 078
>
> TORCH GmbH - A Graylog company
> Steckelhörn 11
> 20457 Hamburg
> Germany
>
> Commercial Reg. (Registergericht): Amtsgericht Hamburg, HRB 125175
> Geschäftsführer: Lennart Koopmann (CEO)
>
> > On 21.04.2016, at 09:03, er...@muneris.io wrote:
> >
> > Dear Graylog community support / users,
> >
> > I have been using Graylog since 1.2 and its working great.
> >
> > Just discover a change about a health check in Graylog's web just might
> cause problems.
> > It's known and normal that the Graylog's web service detects the server
> node(s) healthiness with API thru TCP 12900.
> >
> > However I noticed an issue in Graylog 2.
> > When I am trying out Graylog 2 (Alpha and Beta), the web UI
> automatically calls TCP 12900 (API port) in the client side using the
> public address.
> > That is, from the developer mode of the browser, I can see URL call of
> http://:12900/system/cluster/node. This
> causes the following issues:
> >
> > 1) With the default configuration, such check listens to private IP of
> the server. So just when deploying the Graylog to internet, the check
> fails. (Unless we access the website through VPN IP or update
> rest_transport_uri in /opt/graylog/conf/graylog.conf)
> > 2) Health check should probably be done in background in the server
> (i.e. like Graylog 1.2, 1.3...the checking will not be exposed to client
> side / browser)
> > 3) We need to expose TCP 12900 of the web service to public, security
> concern arises as the API port would be facing the public internet as well
> >
> > Thank you.
> > Eric
> >
> > --
> > You received this message because you are subscribed to the Google
> Groups "Graylog Users" group.
> > To unsubscribe from this group and stop receiving emails from it, send
> an email to graylog2+unsubscr...@googlegroups.com.
> > To view this discussion on the web visit
> https://groups.google.com/d/msgid/graylog2/a43a9ea9-2b6b-4d6a-8b91-1304b84dd008%40googlegroups.com
> .
> > For more options, visit https://groups.google.com/d/optout.
>
> --
> You received this message because you are subscribed to a topic in the
> Google Groups "Graylog Users" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/graylog2/FAovHmo0ctE/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> graylog2+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/graylog2/7FE12566-B7BC-41BB-810F-BE3D31D632EF%40graylog.com
> .
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/graylog2/CAF2Mi_xPK29GbZGvYgpRzoOXELawQOUsZU%2B-H1tT-A13JvscUA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.