[graylog2] can not search googlebot

[celtar]

Hi,

we use graylog 2.1.2 with the apache-gelf Module from the marketplace.

If we try to search "Googlebot" in this string (type agent:)

agent = (Original Message) :  "Mozilla/5.0 (compatible; Googlebot/2.1; 
+http://www.google.com/bot.html)"

1. Search = Input AND agent:*Googlebot* = result none found

2. Search = InputAND agent:*Googleb* = result none found

3. Search = Input AND agent:*Google*  = graylog result = ok but i think 
graylog only found the string google from www.google.com

I try different ways to search Google* or *Google  Google "Googlebot". 
Every time it is the same result. Google was found but not the String 
Googlbot

Is there any Syntax failure in searching or maybe it is not possible.

Do i have to use another Inputfilter  (like Logstash)?

The same problem have we with different search string (not only apache-gelf 
module).

Thanks for helping and have a great day

:) John Celtar




-- 
You received this message because you are subscribed to the Google Groups 
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/graylog2/ca9cf761-7ee3-457e-959b-40d4ec95968c%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[graylog2] Re: Logging of Graylog-Server to Syslog

[Jochen Schalanda]

Hi Frank,

thanks for the update!

Cheers,
Jochen 

-- 
You received this message because you are subscribed to the Google Groups 
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/graylog2/1c2d2400-9587-4b06-aa35-cba6e629332d%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[graylog2] Re: Incorrect Graylog Cluster details

[Jochen Schalanda]

Hi Paweł,

as a matter of fact, everything is correct, except for your configuration.

The transport_address attribute in the output of GET 
/api/system/cluster/nodes shows, that both nodes are using 
http://127.0.0.1:9000/api/ as their transport address.
This address can be configured with the rest_transport_uri setting 

 
and has to be the public URI of the Graylog REST API of each Graylog node. 
It's being used by every Graylog node to communicate with other Graylog 
nodes.

tl;dr: Set rest_transport_uri on each Graylog node to a URI which can be 
accessed by all the other Graylog nodes.

Cheers,
Jochen

On Monday, 13 February 2017 19:51:02 UTC+1, Paweł Karoluk wrote:
>
> Hi, I have two node Graylog Cluster and as you can see there is some wrong 
> with cluster config:
>
>
> *GET /api/system/cluster/nodes*
>
> {
> nodes: [
> {
> cluster_id: "6701202c-a9fe-42d2-8d5a-015acf66fbfa",
> node_id: "5f596ebf-a988-4c08-858e-67d38a3e483b",
> type: "server",
> transport_address: "http://127.0.0.1:9000/api/";,
> last_seen: "2017-02-10T00:45:30.000Z",
> short_node_id: "5f596ebf",
> hostname: "analog1.local",
> is_master: true
> },
> {
> cluster_id: "6701202c-a9fe-42d2-8d5a-015acf66fbfa",
> node_id: "8be9e293-f60b-40c6-a0e6-8af6d617eb1a",
> type: "server",
> transport_address: "http://127.0.0.1:9000/api/";,
> last_seen: "2017-02-10T00:45:30.000Z",
> short_node_id: "8be9e293",
> hostname: "analog2.local",
> is_master: false
> }
> ],
> total: 2
> }
>
>
> *GET /api/cluster*
>
> {
> 5f596ebf-a988-4c08-858e-67d38a3e483b: {
> facility: "graylog-server",
> codename: "Smuttynose",
> node_id: "5f596ebf-a988-4c08-858e-67d38a3e483b",
> cluster_id: "6701202c-a9fe-42d2-8d5a-015acf66fbfa",
> version: "2.1.3+040d371",
> started_at: "2017-02-10T00:27:13.101Z",
> hostname: "analog1.local",
> lifecycle: "running",
> lb_status: "alive",
> timezone: "Europe/Warsaw",
> operating_system: "Linux 2.6.32-642.13.1.el6.x86_64",
> is_processing: true
> },
> 8be9e293-f60b-40c6-a0e6-8af6d617eb1a: {
> facility: "graylog-server",
> codename: "Smuttynose",
> node_id: "5f596ebf-a988-4c08-858e-67d38a3e483b",
> cluster_id: "6701202c-a9fe-42d2-8d5a-015acf66fbfa",
> version: "2.1.3+040d371",
> started_at: "2017-02-10T00:27:13.101Z",
> hostname: "analog1.local",
> lifecycle: "running",
> lb_status: "alive",
> timezone: "Europe/Warsaw",
> operating_system: "Linux 2.6.32-642.13.1.el6.x86_64",
> is_processing: true
> }
> }
>
>
> In /api/cluster I supposed to get two different node_id and hostname, but 
> hostnames are the same. As the result when I want to check the 
> /system/nodes I got dubbed stats only of one host. The real HeapSize of 
> analog2 is only 2GB (img: analog2-system-nodes 
> ) not 4GB as analog1 - master node 
> (img: analog1-system-nodes ).
>
>
> MongoDB and ES Cluster are external and shared for both hosts.
>
>
> Thanks Guys
>
>
> Pawel
>

-- 
You received this message because you are subscribed to the Google Groups 
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/graylog2/6a85729a-f686-4b3c-b239-d43500d897d1%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[graylog2] Incorrect Graylog Cluster details

[Paweł Karoluk]

Hi, I have two node Graylog Cluster and as you can see there is some wrong 
with cluster config:


*GET /api/system/cluster/nodes*

{
nodes: [
{
cluster_id: "6701202c-a9fe-42d2-8d5a-015acf66fbfa",
node_id: "5f596ebf-a988-4c08-858e-67d38a3e483b",
type: "server",
transport_address: "http://127.0.0.1:9000/api/";,
last_seen: "2017-02-10T00:45:30.000Z",
short_node_id: "5f596ebf",
hostname: "analog1.local",
is_master: true
},
{
cluster_id: "6701202c-a9fe-42d2-8d5a-015acf66fbfa",
node_id: "8be9e293-f60b-40c6-a0e6-8af6d617eb1a",
type: "server",
transport_address: "http://127.0.0.1:9000/api/";,
last_seen: "2017-02-10T00:45:30.000Z",
short_node_id: "8be9e293",
hostname: "analog2.local",
is_master: false
}
],
total: 2
}


*GET /api/cluster*

{
5f596ebf-a988-4c08-858e-67d38a3e483b: {
facility: "graylog-server",
codename: "Smuttynose",
node_id: "5f596ebf-a988-4c08-858e-67d38a3e483b",
cluster_id: "6701202c-a9fe-42d2-8d5a-015acf66fbfa",
version: "2.1.3+040d371",
started_at: "2017-02-10T00:27:13.101Z",
hostname: "analog1.local",
lifecycle: "running",
lb_status: "alive",
timezone: "Europe/Warsaw",
operating_system: "Linux 2.6.32-642.13.1.el6.x86_64",
is_processing: true
},
8be9e293-f60b-40c6-a0e6-8af6d617eb1a: {
facility: "graylog-server",
codename: "Smuttynose",
node_id: "5f596ebf-a988-4c08-858e-67d38a3e483b",
cluster_id: "6701202c-a9fe-42d2-8d5a-015acf66fbfa",
version: "2.1.3+040d371",
started_at: "2017-02-10T00:27:13.101Z",
hostname: "analog1.local",
lifecycle: "running",
lb_status: "alive",
timezone: "Europe/Warsaw",
operating_system: "Linux 2.6.32-642.13.1.el6.x86_64",
is_processing: true
}
}


In /api/cluster I supposed to get two different node_id and hostname, but 
hostnames are the same. As the result when I want to check the 
/system/nodes I got dubbed stats only of one host. The real HeapSize of 
analog2 is only 2GB (img: analog2-system-nodes 
) not 4GB as analog1 - master node 
(img: analog1-system-nodes ).


MongoDB and ES Cluster are external and shared for both hosts.


Thanks Guys


Pawel

-- 
You received this message because you are subscribed to the Google Groups 
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/graylog2/0e955943-52a2-4a69-aba8-a7bac1b29238%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[graylog2] Re: Logging of Graylog-Server to Syslog

[Frank Engler]

Am Dienstag, 7. Februar 2017, 00:10:23 schrieb Jochen Schalanda:
> This shows that the appender mechanism itself is working but that either
> the Syslog appender doesn't work or that you have configured it wrong.
> 
> Try starting with the absolute minimum configuration for the Syslog
> appender...

I was abled to solve the problem. It was an error in the configuration though 
Log4j isn't very specific about what's right and what is the absolute minimum. 
Messages from Log4j are written 
to stderr instead of the log file, so I overlooked them at first:

...
Caused by: java.lang.$llegalArgumentException: No structured id name was 
supplied
...
main ERROR Null object returned for Syslog in Appenders.
main ERROR Unable to locate appender "RFC5424" for logger config "root"

Knowing that message it was an easy google to find the issue of Log4j having 
the mdcId parameter without a default 
(https://issues.apache.org/jira/browse/LOG4J2-922). A parameter nobody even 
cared to document 
(https://logging.apache.org/log4j/2.x/manual/appenders.html#SyslogAppender). 
If you really want to find out something about mdcId you have to go to 
https://logging.apache.org/log4j/2.0/log4j-core/apidocs/org/apache/logging/log4j/core/layout/Rfc5424Layout.html#createLayout%28org.apache.logging.log4j.core.net.Facility,
%20java.lang.String,%20int,%20boolean,%20java.lang.String,%20java.lang.String,
%20java.lang.String,%20boolean,%20java.lang.String,%20java.lang.String,
%20java.lang.String,%20java.lang.String,%20java.lang.String,
%20java.lang.String,%20java.lang.String,%20boolean,
%20org.apache.logging.log4j.core.layout.LoggerFields[],
%20org.apache.logging.log4j.core.config.Configuration%29 (yes, that's a link). 
Point is the mdcId parameter needs a value and doesn't have one. Log4j does 
not care what value it is, even an empty string is fine. So the minimum config 
for a RFC5424 syslog appender is:



Greetings
Frank

-- 
You received this message because you are subscribed to the Google Groups 
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/graylog2/10857472.ASFgAvzGMa%40studio.engler.invalid.
For more options, visit https://groups.google.com/d/optout.

[graylog2] Re: HELP-ME Duplications mensages

[Jochen Schalanda]

Hi Anderson,

On Monday, 13 February 2017 14:25:29 UTC+1, Anderson Gabriel wrote:
>
> Hello, the timestamp is the same. But the ID is different
>

This means that these identical messages have been sent to Graylog multiple 
times and that Graylog doesn't duplicate them.

Are you sure that Logstash is running only once on your system? Are you 
sure that your Logstash configuration doesn't duplicate the messages?

Cheers,
Jochen

-- 
You received this message because you are subscribed to the Google Groups 
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/graylog2/4572c89b-539c-4de4-81ec-47cf27130216%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[graylog2] Re: HELP-ME Duplications mensages

[Anderson Gabriel]

Hello, the timestamp is the same. But the ID is different

Em terça-feira, 6 de dezembro de 2016 14:31:37 UTC-2, Jochen Schalanda 
escreveu:
>
> Hi Anderson,
>
> do all "duplicated" messages have the same timestamp and the same message 
> ID or are they different?
>
> Cheers,
> Jochen
>
> On Wednesday, 23 November 2016 16:29:56 UTC+1, Anderson Gabriel wrote:
>>
>> Hello, I have a graylog with elasticsearch configured on only 1 server. I 
>> am capturing jboss logs. But my messages are replicating for more than 
>> 20x each.
>>
>> Configuration logstash:
>>
>> input {
>> filet {
>> Type => "aeq-pro"
>> Add_field => { "ugcs_server" => "172.29.1.114"}
>> Add_field => { "ugcs_application" => "aeq-pro"}
>> Add_field => { "ugcs_environment" => "production"}
>> Add_field => { "ugcs_type" => "jboss-pro"}
>> Path => "/nfs/prod/logs/jboss6/aeq/server.log"
>> Codec => multiline {
>> Pattern => "^% {TIMESTAMP_ISO8601}"
>> Negate => true
>> What => previous
>>  }
>> }
>> }
>>
>> filter {
>> mutate {
>> Type => "aeq-pro"
>> Add_tag => [ "aeq-pro"]
>> Remove_tag => [ "multiline"]
>> }
>> }
>>
>> # Output
>>
>> output {
>> gelf {
>> Host => "172.29.1.181"
>> Port = "" 12212 "
>> Full_message => ""
>> }
>> }
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/graylog2/e7fea286-7707-4d3d-9c06-0d6c3664de54%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[graylog2] Re: fresh install of graylog 2.1.2 -> can't get it running

[Denny Gebel]

Thanks Jochen!

I switched the puppet-config for 
"elasticsearch_discovery_zen_ping_unicast_hosts" from Array to String.

Denny

Am Montag, 13. Februar 2017 10:42:40 UTC+1 schrieb Jochen Schalanda:
>
> Hi Denny,
>
> it looks like the elasticsearch_discovery_zen_ping_unicast_hosts setting 
> is wrong. Please refer to 
> http://docs.graylog.org/en/2.2/pages/configuration/elasticsearch.html#network-setup
>  
> for details.
>
> Cheers,
> Jochen
>
> On Monday, 13 February 2017 10:23:18 UTC+1, Denny Gebel wrote:
>>
>> Hi Grayloggers,
>>
>> I have a working 1.3.4 multi-server setup which needs to be upgraded.
>>
>> I've installed a new test environment with graylog 2.1.2 on CentOS 7 (no 
>> firewall enabled, selinux off):
>>
>> 3 VMs with Graylog (2.1.2) and MongoDB (2.6.12) + 3 VMs ES (2.4.4).
>> ES-Cluster is running fine, as well as the MongoDB-Replicaset.
>>
>> Configuration of Graylog is done via Graylog-Puppet-Module.
>>
>> The generated config file looks like this:
>> # WARNING: Maintained by Puppet, manual changes will be lost!
>>
>> allow_highlighting = true
>> allow_leading_wildcard_searches = true
>> content_packs_dir = /usr/share/graylog-server/contentpacks
>> elasticsearch_cluster_name = graylogcluster
>> elasticsearch_discovery_zen_ping_unicast_hosts = 
>> ["graylog-elasticsearch01.my.domain:9300", 
>> "graylog-elasticsearch02.my.domain:9300", 
>> "graylog-elasticsearch03.my.domain:9300"]
>> elasticsearch_index_prefix = graylog
>> elasticsearch_max_number_of_indices = 30
>> elasticsearch_max_time_per_index = 1d
>> elasticsearch_replicas = 1
>> elasticsearch_shards = 4
>> is_master = true
>> message_journal_dir = /var/lib/graylog-server/journal
>> mongodb_uri = 
>> mongodb://graylog:secretp...@graylog01.my.domain:27017,graylog02.my.domain:27017,graylog03.my.domain:27017/graylog
>> node_id_file = /etc/graylog/server/node-id
>> password_secret = supersecretpass
>> plugin_dir = /usr/share/graylog-server/plugin
>> rest_listen_uri = http://172.16.0.93:9000/api/
>> rest_transport_uri = http://172.16.0.93:9000/api/
>> retention_strategy = delete
>> root_password_sha2 = supersecretrootpass
>> root_timezone = Europe/Berlin
>> root_username = admin
>> rotation_strategy = time
>> web_enable = true
>> web_listen_uri = http://172.16.0.93:9000/
>>
>> Problem is: There's no web interface listening on port 9000 - which I 
>> would expect.
>>
>> Is there anything I missed? The log file (attached) doesn't show any 
>> error or something :/
>>
>> Thanks for your help.
>>
>> Denny
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/graylog2/83004cc3-6cb3-4e05-9953-7ce29e1643b3%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[graylog2] Re: fresh install of graylog 2.1.2 -> can't get it running

[Jochen Schalanda]

Hi Denny,

it looks like the elasticsearch_discovery_zen_ping_unicast_hosts setting is 
wrong. Please refer 
to 
http://docs.graylog.org/en/2.2/pages/configuration/elasticsearch.html#network-setup
 
for details.

Cheers,
Jochen

On Monday, 13 February 2017 10:23:18 UTC+1, Denny Gebel wrote:
>
> Hi Grayloggers,
>
> I have a working 1.3.4 multi-server setup which needs to be upgraded.
>
> I've installed a new test environment with graylog 2.1.2 on CentOS 7 (no 
> firewall enabled, selinux off):
>
> 3 VMs with Graylog (2.1.2) and MongoDB (2.6.12) + 3 VMs ES (2.4.4).
> ES-Cluster is running fine, as well as the MongoDB-Replicaset.
>
> Configuration of Graylog is done via Graylog-Puppet-Module.
>
> The generated config file looks like this:
> # WARNING: Maintained by Puppet, manual changes will be lost!
>
> allow_highlighting = true
> allow_leading_wildcard_searches = true
> content_packs_dir = /usr/share/graylog-server/contentpacks
> elasticsearch_cluster_name = graylogcluster
> elasticsearch_discovery_zen_ping_unicast_hosts = 
> ["graylog-elasticsearch01.my.domain:9300", 
> "graylog-elasticsearch02.my.domain:9300", 
> "graylog-elasticsearch03.my.domain:9300"]
> elasticsearch_index_prefix = graylog
> elasticsearch_max_number_of_indices = 30
> elasticsearch_max_time_per_index = 1d
> elasticsearch_replicas = 1
> elasticsearch_shards = 4
> is_master = true
> message_journal_dir = /var/lib/graylog-server/journal
> mongodb_uri = 
> mongodb://graylog:secretp...@graylog01.my.domain:27017,graylog02.my.domain:27017,graylog03.my.domain:27017/graylog
> node_id_file = /etc/graylog/server/node-id
> password_secret = supersecretpass
> plugin_dir = /usr/share/graylog-server/plugin
> rest_listen_uri = http://172.16.0.93:9000/api/
> rest_transport_uri = http://172.16.0.93:9000/api/
> retention_strategy = delete
> root_password_sha2 = supersecretrootpass
> root_timezone = Europe/Berlin
> root_username = admin
> rotation_strategy = time
> web_enable = true
> web_listen_uri = http://172.16.0.93:9000/
>
> Problem is: There's no web interface listening on port 9000 - which I 
> would expect.
>
> Is there anything I missed? The log file (attached) doesn't show any error 
> or something :/
>
> Thanks for your help.
>
> Denny
>

-- 
You received this message because you are subscribed to the Google Groups 
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/graylog2/221dd34e-38f4-45c8-b519-5e11fbb26f3b%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[graylog2] fresh install of graylog 2.1.2 -> can't get it running

[Denny Gebel]

Hi Grayloggers,

I have a working 1.3.4 multi-server setup which needs to be upgraded.

I've installed a new test environment with graylog 2.1.2 on CentOS 7 (no 
firewall enabled, selinux off):

3 VMs with Graylog (2.1.2) and MongoDB (2.6.12) + 3 VMs ES (2.4.4).
ES-Cluster is running fine, as well as the MongoDB-Replicaset.

Configuration of Graylog is done via Graylog-Puppet-Module.

The generated config file looks like this:
# WARNING: Maintained by Puppet, manual changes will be lost!

allow_highlighting = true
allow_leading_wildcard_searches = true
content_packs_dir = /usr/share/graylog-server/contentpacks
elasticsearch_cluster_name = graylogcluster
elasticsearch_discovery_zen_ping_unicast_hosts = 
["graylog-elasticsearch01.my.domain:9300", 
"graylog-elasticsearch02.my.domain:9300", 
"graylog-elasticsearch03.my.domain:9300"]
elasticsearch_index_prefix = graylog
elasticsearch_max_number_of_indices = 30
elasticsearch_max_time_per_index = 1d
elasticsearch_replicas = 1
elasticsearch_shards = 4
is_master = true
message_journal_dir = /var/lib/graylog-server/journal
mongodb_uri = 
mongodb://graylog:secretp...@graylog01.my.domain:27017,graylog02.my.domain:27017,graylog03.my.domain:27017/graylog
node_id_file = /etc/graylog/server/node-id
password_secret = supersecretpass
plugin_dir = /usr/share/graylog-server/plugin
rest_listen_uri = http://172.16.0.93:9000/api/
rest_transport_uri = http://172.16.0.93:9000/api/
retention_strategy = delete
root_password_sha2 = supersecretrootpass
root_timezone = Europe/Berlin
root_username = admin
rotation_strategy = time
web_enable = true
web_listen_uri = http://172.16.0.93:9000/

Problem is: There's no web interface listening on port 9000 - which I would 
expect.

Is there anything I missed? The log file (attached) doesn't show any error 
or something :/

Thanks for your help.

Denny

-- 
You received this message because you are subscribed to the Google Groups 
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/graylog2/14f5f910-7d5e-4351-ab4e-a8abd7ab6862%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
2017-02-13T10:08:18.832+01:00 INFO  [CmdLineTool] Loaded plugin: Elastic Beats 
Input 1.1.5 [org.graylog.plugins.beats.BeatsInputPlugin]
2017-02-13T10:08:18.833+01:00 INFO  [CmdLineTool] Loaded plugin: Collector 
1.1.3 [org.graylog.plugins.collector.CollectorPlugin]
2017-02-13T10:08:18.834+01:00 INFO  [CmdLineTool] Loaded plugin: Enterprise 
Integration Plugin 1.1.3 
[org.graylog.plugins.enterprise_integration.EnterpriseIntegrationPlugin]
2017-02-13T10:08:18.834+01:00 INFO  [CmdLineTool] Loaded plugin: 
MapWidgetPlugin 1.1.3 [org.graylog.plugins.map.MapWidgetPlugin]
2017-02-13T10:08:18.834+01:00 INFO  [CmdLineTool] Loaded plugin: Pipeline 
Processor Plugin 1.1.3 [org.graylog.plugins.pipelineprocessor.ProcessorPlugin]
2017-02-13T10:08:18.926+01:00 INFO  [CmdLineTool] Running with JVM arguments: 
-Xms2g -Xmx2g -XX:NewRatio=1 -XX:+ResizeTLAB -XX:+UseConcMarkSweepGC 
-XX:+CMSConcurrentMTEnabled -XX:+CMSClassUnloadingEnabled -XX:+UseParNewGC 
-XX:-OmitStackTraceInFastThrow 
-Dlog4j.configurationFile=file:///etc/graylog/server/log4j2.xml 
-Djava.library.path=/usr/share/graylog-server/lib/sigar 
-Dgraylog2.installation_source=rpm
2017-02-13T10:08:19.121+01:00 INFO  [Version] HV01: Hibernate Validator 
5.2.4.Final
2017-02-13T10:08:20.999+01:00 INFO  [InputBufferImpl] Message journal is 
enabled.
2017-02-13T10:08:21.022+01:00 INFO  [NodeId] Node ID: 
a5dbb7c9-104a-48ae-bbb9-506b0d3d0b06
2017-02-13T10:08:21.220+01:00 INFO  [LogManager] Loading logs.
2017-02-13T10:08:21.256+01:00 WARN  [Log] Found a corrupted index file, 
/var/lib/graylog-server/journal/messagejournal-0/.index, 
deleting and rebuilding index...
2017-02-13T10:08:21.289+01:00 INFO  [LogManager] Logs loading complete.
2017-02-13T10:08:21.289+01:00 INFO  [KafkaJournal] Initialized Kafka based 
journal at /var/lib/graylog-server/journal
2017-02-13T10:08:21.307+01:00 INFO  [InputBufferImpl] Initialized 
InputBufferImpl with ring size <65536> and wait strategy 
, running 2 parallel message handlers.
2017-02-13T10:08:21.328+01:00 INFO  [cluster] Cluster created with settings 
{hosts=[graylog01.my.domain:27017, graylog02.my.domain:27017, 
graylog03.my.domain:27017], mode=MULTIPLE, requiredClusterType=UNKNOWN, 
serverSelectionTimeout='3 ms', maxWaitQueueSize=5000}
2017-02-13T10:08:21.328+01:00 INFO  [cluster] Adding discovered server 
graylog01.my.domain:27017 to client view of cluster
2017-02-13T10:08:21.343+01:00 INFO  [cluster] Adding discovered server 
graylog02.my.domain:27017 to client view of cluster
2017-02-13T10:08:21.343+01:00 INFO  [cluster] Adding discovered server 
graylog03.my.domain:27017 to client view of cluster
2017-02-13T10:08:21.375+01:00 INFO  [cluster] No server chosen by 
ReadPrefe

[graylog2] Re: Github page on giving Graylog read-access to non-admin users

[Jochen Schalanda]

Hi,

please upgrade to Graylog 2.2.0, which supports your use case via a default 
stream containing all messages.

Cheers,
Jochen

On Friday, 10 February 2017 17:51:05 UTC+1, dhe...@gmail.com wrote:
>
> I've added LDAP auth to graylog 2.1.0-SNAPSHOT and assigned "Allow 
> Reading" roles to all my streams. I want users in this role to be able to 
> query the "regular" search data so I added a "Default Search" stream with a 
> rule to match "^.*$" on the "message" field (for syslog).  I've added 
> "Allow Reading" access for my LDAP user. When logged in as Admin, I can see 
> messages going into this stream. When logged in as the LDAP user, nothing 
> appears to be going in (under Streams menu - all messages/second counts 
> stay at 0).  As of this sentence, I re-loaded the Streams page for my LDAP 
> user and it shows up empty now. There were a handful of streams there a 
> minute ago :/
>
> Is non-admin user stream sharing still in development, or has this issue 
> been solved elsewhere? 
>
> A google turned up this page about it: a 
> https://github.com/Graylog2/graylog2-web-interface/issues/620
> "There are several work-arounds for this issue dating back to 
> 2015.recommend posting it" to mailing list or IRC. So I am asking here. 
>
> Graylog is a really great project. I'm not complaining. Actually really 
> satisfied with what it can do. Fits my needs perfectly. Just looking for a 
> way to let others in my group use it without using a shared admin account.
>
> Thanks!
>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/graylog2/4bd179cb-5c25-4b74-958c-192f1c5e5fc8%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[graylog2] Re: Parse JSON containing timestamp field

[Jochen Schalanda]

Hi Rui,

the timestamp field has to contain a valid date value, not a string that 
looks like a date.

You can use the message processing pipeline or the date extractor for this:

http://docs.graylog.org/en/2.2/pages/extractors.html#normalization
http://docs.graylog.org/en/2.2/pages/pipelines.html

Cheers,
Jochen

On Friday, 10 February 2017 15:57:13 UTC+1, Rui Goncalves wrote:
>
> Hi all.
> I want to send JSON documents into graylog containing a field ("ts") that 
> contains the timestamp event. I'm unable to set the "ts" field value as 
> "timestamp" value. Graylog sets a timestamp field when the message is 
> received, and I'm unable to update that field to "ts" value! 
>
> Sample message: {"ts": "2017-02-10T12:13:42Z", msg="", service="yyy", 
> ... }
>
> 1. I've created a raw TCP 
> 2. Added a JSON extractor, so all JSON fields get extracted
> 3. Added an extractor to cut ts field and store on the timestamp field. 
>
> I was expecting to get the timespamp field with the ts value! :-/ I've 
> also tried to rename the "ts" field in the source document to "timestamp", 
> but it does not work either.
>
> Is that possible to update the timestamp field? 
>
> Thanks,
> Rui
>
>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/graylog2/418a86ef-7bfa-4aa1-b039-ef3eb554afe5%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[graylog2] Re: Does graylog automatically detect duplicate messages on ingest?

[Jochen Schalanda]

Hi Matthew,

On Friday, 10 February 2017 00:51:57 UTC+1, Matthew Shapiro wrote:
>
> Does Graylog have any detection of duplicate messages to overwrite, and if 
> not is there any way to force an id on a message via an extractor?
>

No, Graylog doesn't support de-duplication of messages and overwriting the 
internal _id field is forbidden, so I'm afraid you'll have to delete the 
old messages manually from Elasticsearch using some other attributes.

Cheers,
Jochen

-- 
You received this message because you are subscribed to the Google Groups 
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/graylog2/ce50e0c2-9acb-4217-8514-7c7ecff05ddf%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[graylog2] Re: Monitoring Windows DHCP Server Activity

[Jochen Schalanda]

Hi Rob,

the Graylog Collector Sidecar simply configures and starts the actual 
collectors (Filebeat or nxlog), so you'll have to check with their docs if 
that's possible:

https://nxlog.co/docs/nxlog-ce/nxlog-reference-manual.html
https://www.elastic.co/guide/en/beats/filebeat/current/index.html

Cheers,
Jochen

On Thursday, 9 February 2017 23:16:11 UTC+1, Rob Repp wrote:
>
> The files are definitely updating. One interesting thing, I tried do 
> establish this by just tailing the file with both Notepad++ and with a 
> freeware "tail" utility for Windows and it never updated. I had to manually 
> reload the file to see any changes. Further, I never saw any update in the 
> file Date Modified. Is there some way to force collector sidecar to poll 
> the files even if they don't show any obvious activity?
>
> On Tuesday, February 7, 2017 at 1:55:07 AM UTC-6, Jochen Schalanda wrote:
>>
>> Hi Rob,
>>
>> this sounds like either there is simply no new content in the files 
>> you've configured nxlog to watch, or that the file pattern is wrong. Try 
>> using another File pattern in the nxlog im_file input or switch to 
>> Filebeat.
>>
>> Cheers,
>> Jochen
>>
>> On Monday, 6 February 2017 23:22:59 UTC+1, Rob Repp wrote:
>>>
>>> Okay, I did a packet capture that's showing traffic between the two 
>>> boxes. There seems to be the Graylog host sending a json of the nxlog.conf 
>>> config data to the DHCP server once every four seconds or so, and the DHCP 
>>> server sending back HTTP requests on port 9000. None of the exchanges look 
>>> like they contain data from the DHCP logs.
>>>
>>> On Monday, February 6, 2017 at 10:37:44 AM UTC-6, Jochen Schalanda wrote:

 Hi Rob,

 since the configuration doesn't show any obvious errors, please use 
 Wireshark or a similar tool like tcpdump to check if the log messages from 
 nxlog are sent to the correct host and if the UDP packets actually arrive 
 at the Graylog GELF UDP input.

 Cheers,
 Jochen

 On Monday, 6 February 2017 17:08:21 UTC+1, Rob Repp wrote:
>
> The traffic is not being blocked. There's no firewall on either 
> machine, and the network path is unobstructed. Further, the Collector 
> status for that Collector is showing green, with Backend "Nxlog: 
> running." 
> It looks like it's connected and responsive. It's just that there never 
> seem to be any messages on the associated Input.
> Tks,
> R.
>
> On Saturday, February 4, 2017 at 3:30:18 AM UTC-6, Jochen Schalanda 
> wrote:
>>
>> Hi Rob,
>>
>> the configuration looks good so far. Make sure that the host 
>> "re.da.ct.ed" can be accessed by your Windows machine and that port 
>> 5441/udp is open and not blocked by a firewall.
>>
>> Cheers,
>> Jochen
>>
>> On Friday, 3 February 2017 23:10:50 UTC+1, Rob Repp wrote:
>>>
>>> Okay, in order:
>>>
>>> 1. I'm using the OVA VM image from Graylog, so most of the 
>>> configuration is already done. All I did was add a Connector with one 
>>> nxlog 
>>> input and one nxlog output, and then the GELF UDP input that the 
>>> WinDHCP 
>>> json created.
>>>
>>> The WinDHCP input is configured like this:
>>>
>>> WinDHCPLogs-gelf GELF UDP RUNNING
>>> On node 771f3128 / graylog 
>>> 
>>>
>>>- bind_address:
>>>0.0.0.0
>>>- decompress_size_limit:
>>>8388608
>>>- override_source:
>>>**
>>>- port:
>>>5441
>>>- recv_buffer_size:
>>>1048576
>>>
>>>
>>> 2. The nxlog.conf file is:
>>>
>>> define ROOT C:\Program Files (x86)\nxlog
>>>
>>> 
>>>   Module xm_gelf
>>> 
>>>
>>> Moduledir %ROOT%\modules
>>> CacheDir %ROOT%\data
>>> Pidfile %ROOT%\data\nxlog.pid
>>> SpoolDir %ROOT%\data
>>> LogFile %ROOT%\data\nxlog.log
>>> LogLevel INFO
>>>
>>> 
>>> Module  xm_fileop
>>> 
>>> When@daily
>>> Execfile_cycle('%ROOT%\data\nxlog.log', 7);
>>>  
>>> 
>>>
>>> 
>>> Module im_file
>>> File 'C:\Windows\System32\dhcp\DhcpSrvLog-*.log'
>>> PollInterval 1
>>> SavePos True
>>> ReadFromLast True
>>> Recursive False
>>> RenameCheck True
>>> Exec $FileName = file_name(); # Send file name with each message
>>> 
>>>
>>> 
>>> Module om_udp
>>> Host re.da.ct.ed
>>> Port 5441
>>> OutputType  GELF
>>> Exec $short_message = $raw_event; # Avoids truncation of the 
>>> short_message field.
>>> Exec $gl2_source_collector = '9960a8cd-7abe-4021-939f-89b22909aa32';
>>> Exec $Hostname = hostname_fqdn();
>>> 
>>>
>>> 
>>>   Path 588bc33f682c990374bab049 => 588bc2db682c990374baafe0
>>> 
>>>
>>> 3. collector_sidecar.yml is this:
>

[graylog2] Re: pfSense Extractor

[Benbrahim Anass]

Hi 
the best way is to parse messages one by one
Cheers

Anas

Le vendredi 6 février 2015 11:41:14 UTC+1, VANTIN Dao a écrit :
>
> Hello,
> I use Graylog2 with Rsyslog and when my pfSense send log to my Graylog2 i 
> can't read the log then i download your extractor for pfsense on your 
> website and no message passes through the extractors
>
> Can you help me please ?
>
> Thank you in advance
>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/graylog2/6dd66e40-916e-4d59-bc4d-5068ac461dd1%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[graylog2] Re: How to parse OpenVPN logs in Graylog?

[Benbrahim Anass]

Hi

i use GROK to parse everything, try this :

%{WORD:program}%{NOTSPACE}: %{IPV4:IPClient}:%{NOTSPACE:PORT} 
\[%{WORD:User}\]


i track daily connections as follow,




feel free to contact me on linkedIn 
 for more 
Cheers,

Anas

-- 
You received this message because you are subscribed to the Google Groups 
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/graylog2/99b29466-48e7-486b-ba81-5a881fe25497%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[graylog2] Re: How do you build from source code for version 2.1.2 ?

[Jochen Schalanda]

Hi,

please refer 
to 
http://docs.graylog.org/en/2.2/pages/installation/operating_system_packages.html#rpm-yum-dnf
 
for the relevant information.

Cheers,
Jochen

On Friday, 10 February 2017 17:24:55 UTC+1, bernadet...@wavestrike.com 
wrote:
>
> I need to create RPMs for CENTOS 6 (eventually CENTOS 7)
>

-- 
You received this message because you are subscribed to the Google Groups 
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/graylog2/3fd91ffc-3466-497b-b6e7-99adaf002083%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[graylog2] Re: How do you track unique users that have hit your site/which version do you need

[Jochen Schalanda]

Hi,

please elaborate on your use case.

In general, we always recommend running the latest stable version of 
Graylog (which is Graylog 2.2.0 at the time of writing).

Cheers,
Jochen

On Friday, 10 February 2017 17:24:17 UTC+1, bernadet...@wavestrike.com 
wrote:
>
> we are using older version of graylog. Which version let's you figure out 
> a list of unique users ?
>

-- 
You received this message because you are subscribed to the Google Groups 
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/graylog2/d86cdc14-e8b8-476e-8436-b5888070cfc9%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[graylog2] Re: missing alerts menu

[Jochen Schalanda]

Hi Wallace,

are there any error messages in the logs of your Graylog node or in the 
Developer console of your web browser?

Which web browser are you using?

Cheers,
Jochen

On Friday, 10 February 2017 04:17:25 UTC+1, Wallace Turner wrote:
>
> my (latest) graylog installation is missing the 'Alerts' menu item
>
>
> 
>
>
> I'm trying to add/view the alerts. the docs at this page indicate it 
> should  be present
>
> http://docs.graylog.org/en/2.2/pages/getting_started/stream_alerts.html
>

-- 
You received this message because you are subscribed to the Google Groups 
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/graylog2/59eca43d-7b0b-4675-8065-4521bcfff286%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.