[graylog2] Re: Several indices from 1 and 2 hours ago

2016-07-25 Thread Jochen Schalanda
Hi Roberto, which exact version of Graylog are you using? There were some versions of Graylog which would rotate the indices on startup if the time-based rotation strategy was being used, even if the shouldn't be rotated according to their age. Would it be feasible for you to upgrade to Graylo

Re: [graylog2] Re: Elasticsearch cluster unhealthy (RED)

2016-07-25 Thread Jochen Schalanda
d > -rw--- 1 graylog graylog 4707 Jul 10 16:59 _2pkd.fnm > -rw--- 1 graylog graylog 568 Jul 10 16:59 _2pkd.si > -rw--- 1 graylog graylog 230 Jul 14 03:18 segments_35 > > Thank for the tools link. Been check between 30 - 50 messages/sec still > consid

[graylog2] Re: Input shows running but no messages getting retrieved

2016-07-25 Thread Jochen Schalanda
Hi Thara, I think your rsyslog configuration is incorrect. "." will not match any messages, I think you mean "*.*" instead. Please refer to https://github.com/Graylog2/graylog-guide-syslog-linux/blob/master/README.md#rsyslog for instructions how to configure rsyslog. Cheers, Jochen On Frid

[graylog2] Re: Changing map theme for geolocation

2016-07-25 Thread Jochen Schalanda
Hi Aykisn, that's currently not possible but feel free to open a feature request for this at https://github.com/Graylog2/graylog-plugin-map-widget/issues. Cheers, Jochen On Monday, 25 July 2016 08:11:05 UTC+2, Aykisn wrote: > > Hello, > > I am using the free GeoLite2 database and I was wonderin

[graylog2] Re: Removing some help messages on the web interface

2016-07-25 Thread Jochen Schalanda
Hi Aykisn, those hints can currently not be removed without forking Graylog and modifying the web interface yourself. Cheers, Jochen On Monday, 25 July 2016 09:24:39 UTC+2, Aykisn wrote: > > Hello, > > I didn't find any info on this. I was wondering i there was any way to > remove some of the

[graylog2] Re: graylog Bigger production setup

2016-07-22 Thread Jochen Schalanda
Hi, make sure that your Elasticsearch nodes are able to connect back to your Graylog nodes on port 9350/tcp. Graylog is running an embedded Elasticsearch instance that will need to connect to the ES cluster and thus other ES nodes need to communicate with it directly. Cheers, Jochen On Friday

[graylog2] Re: Graylor-collector for Debian 7

2016-07-22 Thread Jochen Schalanda
Hi Tony, there currently aren't any DEB packages for the Graylog Collector working on Debian 7. You can still download and install the official binaries from https://github.com/Graylog2/collector#binary-download and make your init system start it on boot. This being said, a working SysV init

[graylog2] Re: Input shows running but no messages getting retrieved

2016-07-21 Thread Jochen Schalanda
Hi Thara, please describe in detail which type of input you have set up in Graylog, how you have configured it, and how you have configured your clients. Cheers, Jochen On Thursday, 21 July 2016 19:54:29 UTC+2, Thara Savio wrote: > > The input shows running but no messages getting retrieved. Ev

[graylog2] Re: Inputs not displaying under sources

2016-07-21 Thread Jochen Schalanda
Hi Thara, it is very well possible that the "source" field of the messages received from that server don't contain the public host name or IP address of that server. For example syslog protocol and GELF protocol allow setting the source address to arbitrary values (e. g. for forwarding messages

[graylog2] Re: IF ELSE replace for Extractors

2016-07-21 Thread Jochen Schalanda
Hi Julio, you'll have to create multiple rules for this at the moment. Cheers, Jochen On Thursday, 21 July 2016 18:39:25 UTC+2, juli...@gmail.com wrote: > > Did come out with this: > > rule "Add ID Meaning" >> when >> has_field("ID") && contains(to_string($message.ID), "11") >> then >> s

[graylog2] Re: Do we have use separate ports servers to send logs to graylog

2016-07-21 Thread Jochen Schalanda
Hi Thara, different inputs usually listen on different ports, so if you have multiple input formats like syslog, GELF, or any other, you'll most likely have to use different ports for those inputs. This being said, if you only have syslog messages you want to record, you can use a single Syslo

[graylog2] Re: IF ELSE replace for Extractors

2016-07-21 Thread Jochen Schalanda
Hi Julio, currently that's not easily possible but we plan to introduce functions for lookups in dictionaries or external sources in the message processing pipelines (http://docs.graylog.org/en/2.0/pages/pipelines.html) in a future version. Cheers, Jochen On Thursday, 21 July 2016 17:19:48 UT

[graylog2] Re: ERROR Appenders contains an invalid element or attribute "Memory"

2016-07-21 Thread Jochen Schalanda
rsday, 21 July 2016 18:35:53 UTC+5:30, Jochen Schalanda wrote: >> >> Hi Pisa, >> >> how exactly did you install Graylog (please describe it step by step)? >> >> The error messages from the logs look like an invalid Log4j 2 >> configuration or an invalid cla

[graylog2] Re: ERROR Appenders contains an invalid element or attribute "Memory"

2016-07-21 Thread Jochen Schalanda
Hi Pisa, how exactly did you install Graylog (please describe it step by step)? The error messages from the logs look like an invalid Log4j 2 configuration or an invalid class path. The other error message you've mentioned ({"type":"ApiError","message":"HTTP 404 Not Found"}) is the normal resp

[graylog2] Re: graylog-collector not working on ubuntu 14.04.4

2016-07-21 Thread Jochen Schalanda
Hi, the Graylog Collector itself won't listen on any network interface. The server-url configuration setting simply specifies the URI of the Graylog REST API that the Graylog Connector should register at. Additionally, the GELF output should probably not be configured with port 12900 (which is

[graylog2] Re: GrayLog2 on mac

2016-07-21 Thread Jochen Schalanda
Hi, you need to provide a path to the Graylog configuration file if you don't use the default. See the "-f" or "--configfile" command line parameters of the Graylog command. Cheers, Jochen On Thursday, 21 July 2016 03:39:41 UTC+2, er.jayp...@gmail.com wrote: > > Can anyone please help me out

[graylog2] Re: Inputs not displaying under sources

2016-07-21 Thread Jochen Schalanda
Hi Thara, inputs will not show on the "Sources" page. That will simply show some comprehensive statistics about the "source" field of all indexed messages. If you don't receive any messages, there won't be anything to display on the "Sources" page in the web interface. Cheers, Jochen On Wedne

[graylog2] Re: Graylog compilation guideline

2016-07-19 Thread Jochen Schalanda
Hi Anant, you basically just need Java 8 (we recommend using the latest Oracle JDK) and Maven 3 on your system. Everything else (e. g. Node.js) will be downloaded automatically. Please refer to the .travis.yml file which is

[graylog2] Re: Disk Journal / Kafka Input / Throttling

2016-07-19 Thread Jochen Schalanda
Hi Eli, On Tuesday, 19 July 2016 13:18:49 UTC+2, Eli Jordan wrote: > > My understanding is that the disk journal is just an internal Kafka topic. > Since we are already using Kafka to buffer messages, this seems redundant. > (Also, since we are running graylog in docker the journal is transient

[graylog2] Re: Configure Graylog WebInterface on a dedicated server

2016-07-19 Thread Jochen Schalanda
ch one will have its own web interface ?? it is > not pratical when searching for logs > > > > > Le mardi 19 juillet 2016 11:15:24 UTC+2, Jochen Schalanda a écrit : >> >> Hi, >> >> no, Graylog 2.x currently doesn't allow running only the web interfa

[graylog2] Re: Configure Graylog WebInterface on a dedicated server

2016-07-19 Thread Jochen Schalanda
Hi, no, Graylog 2.x currently doesn't allow running only the web interface. Cheers, Jochen On Tuesday, 19 July 2016 11:10:47 UTC+2, sangh wrote: > > Hi, > > I am using two graylog server with a load balancer. i want to install the > web interface along with the load balancer. With Graylog 2.0 i

[graylog2] Re: Unble to get graylog webinterface

2016-07-18 Thread Jochen Schalanda
am not getting the UI of the "Graylog" only the above > mentioned message. I am attaching the Logs file of Graylog may be you can > find something which I am certainly missing to locate. > > Thanking in Advance > > Anant. > > > > > On Friday, 15 J

[graylog2] Re: Unble to get graylog webinterface

2016-07-15 Thread Jochen Schalanda
. Is there anything we need to change in the > conf files. How do we overcome this?? I am attaching the conf files. > > Again Thanks in advance!! > > Anant > > On Friday, 15 July 2016 18:31:47 UTC+5:30, Jochen Schalanda wrote: >> >> Hi Anant, >> >> accor

[graylog2] Re: Trouble Receiving Syslog Messages

2016-07-15 Thread Jochen Schalanda
;Input > page only has the following to choose from in the new input type: > > GELF AMQP > GELF HTTP > GELF TCP > GELF UDP > GELF KAFKA > JSON > > No plain text option. What could cause that? Thanks! > > Nathan > > > On Friday, July 15, 2016 at 4:28:44

[graylog2] Re: Unble to get graylog webinterface

2016-07-15 Thread Jochen Schalanda
Hi Anant, according to your logs, the Graylog REST API and the Graylog web interface have been successfully started: 2016-07-15 16:38:00,442 INFO : > org.graylog2.initializers.WebInterfaceService - Started Web Interface at > > 2016-07-15 16:38:00,443 INFO : > org.gray

[graylog2] Re: problem with certificate for HTTPS on the webinterface

2016-07-15 Thread Jochen Schalanda
Hi Thomas, the virtual machine appliances rely on the graylog-ctl script which will regenerate the Graylog configuration from a template each time you run graylog-ctl reconfigure. Please take a look at http://docs.graylog.org/en/2.0/pages/configuration/graylog_ctl.html#install-custom-ssl-cer

[graylog2] Re: Single Server Setup vs Multi Server

2016-07-15 Thread Jochen Schalanda
Hi Nathan, please take a look at https://www.graylog.org/tools/sizing-estimator for an educated guess about the hardware requirements for your environment. Cheers, Jochen On Thursday, 14 July 2016 19:46:24 UTC+2, Nathan Mace wrote: > > What is the amount of data inputted per day that you should

Re: [graylog2] Re: Elasticsearch cluster unhealthy (RED)

2016-07-15 Thread Jochen Schalanda
Hi Arief, On Friday, 15 July 2016 09:04:21 UTC+2, Arief Hydayat wrote: > > Just wondering if I continue using these current OVA with default setting > in indices is 2000 Max doc per index and current disk 200GB, how many > target server we can add-in to send messages to the Graylog? > That'

[graylog2] Re: Trouble Receiving Syslog Messages

2016-07-15 Thread Jochen Schalanda
Hi Nathan, On Thursday, 14 July 2016 19:38:20 UTC+2, Nathan Mace wrote: > > That said, how do I add the Raw/Plaintext input? I understand how to add > an input generally, but not one that is specifically for plain text. > There are several types of inputs n the System / Inputs page in the Grayl

[graylog2] Re: Graylog indexes

2016-07-14 Thread Jochen Schalanda
Hi Henrique, that's not possible with Graylog. What you can do, though, is create a separate stream for each of your servers by filtering on the "source" field of the ingested messages. Please refer to http://docs.graylog.org/en/2.0/pages/streams.html for more information about streams. Cheer

Re: [graylog2] Re: Elasticsearch cluster unhealthy (RED)

2016-07-14 Thread Jochen Schalanda
Hi Arief, running graylog-ctl reconfigure will recreate the configuration file from our templates and reset your changes. Cheers, Jochen On Thursday, 14 July 2016 04:45:43 UTC+2, Arief Hydayat wrote: > > Hi Jochen, > > OK I give a try on that. > > > > *ubuntu@graylog:~$ cat /opt/graylog/conf/gr

Re: [graylog2] Re: Elasticsearch cluster unhealthy (RED)

2016-07-14 Thread Jochen Schalanda
Hi Arief, the OVA is suited for small production setups. For the "real deal", we recommend setting up the components yourself (to be able to tweak them according to your use cases) using the official OS packages (DEB, RPM)

[graylog2] Re: How to deal with Journal Utilization is too high?

2016-07-14 Thread Jochen Schalanda
Hi Arief, the output_batch_size and output_flush_interval settings can be configured in Graylog's configuration file, and

[graylog2] Re: How to take a backup grylog

2016-07-13 Thread Jochen Schalanda
Hi, Graylog stores its configuration in two places: the configuration file (e. g. /etc/graylog/server.conf) and MongoDB. You can use the normal MongoDB backup and restore procedures to make a backup of Graylog's configuration: https://docs.mongodb.com/manual/core/backups/ If you additionally

[graylog2] Re: How to deal with Journal Utilization is too high?

2016-07-13 Thread Jochen Schalanda
Hi Arief, messages piling up in the Graylog journal usually means that Elasticsearch cannot keep up with indexing all the messages thrown at it. Try providing more memory and CPU cores to the virtual machine. You can also try and tweak several Elasticsearch related settings like output_batch_s

[graylog2] Re: Graylog2 Numeric values fields questions (explanation inside!)

2016-07-13 Thread Jochen Schalanda
Hi, On Wednesday, 13 July 2016 10:23:26 UTC+2, Zoizo wrote: > > My question is : is there a way to have a chart that will show which > domains (or ips) use the most bytes ? > Or maybe a way to show total bytes used in one day for example by a > specific domain ? > That's currently not possible

Re: [graylog2] Re: Elasticsearch cluster unhealthy (RED)

2016-07-13 Thread Jochen Schalanda
Hi Arief, you can make the Elasticsearch cluster health state GREEN, if you configure the indices (and Graylog) to not use replication, see https://github.com/Graylog2/graylog2-server/blob/2.0.3/misc/graylog.conf#L191-L193 . Cheers, Jochen On Wednesday, 13 July 2016 10:52:13 UTC+2, Arief Hyday

Re: [graylog2] Re: Elasticsearch cluster unhealthy (RED)

2016-07-13 Thread Jochen Schalanda
Hi Arief, you can see that each index has a replication factor of 1, meaning that there are 4 primary shards and 4 replica shards for each index. Since you're running only 1 Elasticsearch node, those replica shards cannot be placed anywhere, which is why the Elasticsearch cluster health state i

[graylog2] Re: Backfilling graylog with past data

2016-07-13 Thread Jochen Schalanda
Hi Jeremy, you can use Logstash or Filebeat (or any other log shipper) to backfill data into Graylog, too. Simply point it to the file (or source) you want to use as an input and use a GELF output to send data into Graylog. Also make sure that the timestamp field is valid, because otherwise Gra

[graylog2] Re: Setting Java Heap and ES_Heap in Graylog2 - installed from packages

2016-07-13 Thread Jochen Schalanda
Hi Casey, check the /etc/sysconfig/graylog-server file for configuring the JVM settings of Graylog like initial and maximum heap size. For Elasticsearch, check the /etc/sysconfig/elasticsearch file (see https://www.elastic.co/guide/en/elasticsearch/reference/2.3/setup-service.html#_rpm_based_dis

[graylog2] Re: Trouble Receiving Syslog Messages

2016-07-13 Thread Jochen Schalanda
Hi Nathan, try using a Raw/Plaintext UDP input instead of a Syslog UDP input. Sometimes those network appliances send syslog messages which aren't quite compliant to RFC 3164 or 5424. You can still extract the information you want to record with extractors on that input. Cheers, Jochen On Tu

Re: [graylog2] Re: upgrading to graylog2 2.x

2016-07-12 Thread Jochen Schalanda
e...@tsg2.com > 314-266-4837 > > On Tue, Jul 12, 2016 at 11:01 AM, Jochen Schalanda wrote: > >> Hi Andrew, >> >> most of the Graylog configuration (inputs and extractors, streams, >> dashboards etc.) are stored in MongoDB and you can use the normal MongoDB >

[graylog2] Re: upgrading to graylog2 2.x

2016-07-12 Thread Jochen Schalanda
Hi Andrew, most of the Graylog configuration (inputs and extractors, streams, dashboards etc.) are stored in MongoDB and you can use the normal MongoDB backup and restore procedures (see https://docs.mongodb.com/manual/tutorial/backup-and-restore-tools/) to create a backup of it. Also make su

[graylog2] Re: Help for wildcards

2016-07-12 Thread Jochen Schalanda
Hi Bruno, there are several things that might make the result being different from what you expected. Graylog is using an index mapping which sets all fields except message, full_message, and source to not_analyzed. For wildcard searches, you'll need to analyze those fields, see http://docs.

Re: [graylog2] Re: Elasticsearch cluster unhealthy (RED)

2016-07-12 Thread Jochen Schalanda
Hi Arief, please post the output of the following command: curl http://localhost:9200/_cat/indices?v Also take into account, that if you're running the OVA with only 1 Elasticsearch node, the cluster health status will never get GREEN because it's configured to use 1 replica shard by default,

[graylog2] Re: rsylog to graylog over tls

2016-07-11 Thread Jochen Schalanda
Hi John, please refer to the rsyslog documentation for instructions about setting up TLS: http://www.rsyslog.com/doc/v8-stable/tutorials/tls_cert_client.html Cheers, Jochen On Monday, 11 July 2016 10:23:24 UTC+2, john wrote: > > Hi, > > I created a cerificate and configured a tcp syslog input w

[graylog2] Re: Where are my GELF messages going?

2016-07-09 Thread Jochen Schalanda
Hi Cody, please check the logs of your Graylog node(s) for errors and tell us, which specific version of Graylog you are using. Additionally, please run a search "in the future" (using an absolute time range) to rule out timezone issues, e. g. messages being indexed with a timestamp some hours

[graylog2] Re: Graylog is not processing Messages from one input anymore

2016-07-08 Thread Jochen Schalanda
Hi Keamas, please refer to https://www.elastic.co/guide/en/elasticsearch/reference/2.3/setup-configuration.html and https://www.elastic.co/guide/en/elasticsearch/guide/2.x/heap-sizing.html for information about sizing Elasticsearch and changing its memory configuration. Elasticsearch should

[graylog2] Re: Graylog search and sum fields

2016-07-08 Thread Jochen Schalanda
Hi Keamas, aggregating or summing up different fields is currently not possible with Graylog. Cheers, Jochen On Thursday, 7 July 2016 16:00:21 UTC+2, Keamas M wrote: > > Hey, > if I have multiple logs like this: > > type=FWD|proto=TCP|srcIF=port7.101|srcIP=10.244.130.102|srcPort=54610|srcMAC=0

[graylog2] Re: Has any one successfully set up SSL on Graylog 2.0?

2016-07-08 Thread Jochen Schalanda
; # The password to unlock the private key used for securing the web > interface. > web_tls_key_password ="PASSWORD" > > Thanks for the help. > --Dave C. > > On Thursday, July 7, 2016 at 3:13:12 AM UTC-5, Jochen Schalanda wrote: >> >> Hi Dave, >>

[graylog2] Re: Graylog is not processing Messages from one input anymore

2016-07-08 Thread Jochen Schalanda
Hi Keamas, which version of Graylog are you using? What are the hardware specs of the machine(s) you're running Graylog and Elasticsearch on? What kind of inputs are you talking about? For Elasticsearch, 1 GB of heap memory is quite little and you should give it more memory (depending on how mu

[graylog2] Re: When is Graylog 2.1 releasing?

2016-07-08 Thread Jochen Schalanda
Hi Paul, we're targeting August 2016 for releasing Graylog 2.1.0. You can help by testing the alpha and beta versions until then. Cheers, Jochen On Friday, 8 July 2016 03:07:53 UTC+2, Paul Mendoza wrote: > > When will I be able to use Graylog 2.1? > > I'm waiting for the TCP TLS Graylog Colle

[graylog2] Re: can not start service

2016-07-07 Thread Jochen Schalanda
Hi, is there any error message in the log file of the Collector Sidecar (check the log_path setting in the configuration file, http://docs.graylog.org/en/2.0/pages/collector_sidecar.html#configuration). Cheers, Jochen On Thursday, 7 July 2016 08:37:14 UTC+2, ชีระวิทย์ ภูริเดชชัยพัฒน์ wrote: >

[graylog2] Re: Graylog Error ( invalid distance too far back)

2016-07-07 Thread Jochen Schalanda
Hi Yiannis, those messages mean that there were some corrupt GELF messages received by your GELF UDP input. This can have many causes, like corrupt UDP packets on the network, sudden connection drops (which also lead to corrupt UDP packets), or simply a broken GELF client. Cheers, Jochen On W

[graylog2] Re: Where to configure the elasticsearch cluster, server.conf or elasticsearch.yml?

2016-07-07 Thread Jochen Schalanda
Hi Tom, you need to configure the Elasticsearch cluster name, the network host, and a list of Elasticsearch nodes in your Graylog configuration, see http://docs.graylog.org/en/2.0/pages/configuration/elasticsearch.html#configuration . Additionally, you need to configure (at least) the cluster n

[graylog2] Re: Has any one successfully set up SSL on Graylog 2.0?

2016-07-07 Thread Jochen Schalanda
Hi Dave, the error message looks like the private key is in an incompatible or invalid format which Graylog can't process. Could you please share your Graylog configuration (the rest_* and web_* settings should be sufficient) and the output of the following OpenSSL command: openssl rsa -noout

Re: [graylog2] Graylog goes enterprise, but not for elastic/shield?

2016-07-06 Thread Jochen Schalanda
Hi Rennie, Graylog currently hosts an embedded Elasticsearch instance which joins the Elasticsearch cluster as a client node (i. e. no data is stored and it's not master-eligible). Due to some kind of "sanity check" (the JarHell check), the embedded Elasticsearch node fails to load any plugins

[graylog2] Re: debugging pipelines is... difficult

2016-07-06 Thread Jochen Schalanda
Hi Jason, there's something coming up in Graylog 2.1.0 which will vastly simplify testing pipeline rules. Feel free to give the alpha and beta releases a try! Cheers, Jochen On Wednesday, 6 July 2016 05:42:43 UTC+2, Jason Haar wrote: > > Hi there > > First I want to say how wonderful the "extr

[graylog2] Re: Sizing of Graylog

2016-07-06 Thread Jochen Schalanda
Hi, I'd say 1366x667 pixels. Cheers, Jochen PS: On a more serious note, there's not much we can do for you without any information. Please also try using the sizing estimator on the Graylog website: https://www.graylog.org/tools/sizing-estimator On Wednesday, 6 July 2016 09:48:20 UTC+2, ชีระ

[graylog2] Re: Plugin Development: POM for org.graylog.plugins:usage-statistics is missing

2016-07-05 Thread Jochen Schalanda
Hi, compile and install (as in mvn install) the Anonymous Usage Statistics plugin once and you should be fine. I used to develop and build plugins before, however, this was before the > bootstrap scripts complicated things. You don't have to use graylog-project and its scripts to develop plug

[graylog2] Re: How to run background Graylog collector on windows

2016-07-05 Thread Jochen Schalanda
Hi, you have to install and run the Graylog Collector as a Windows service. Please refer to the documentation at http://docs.graylog.org/en/2.0/pages/collector.html#windows for further details. Please also take note that the Graylog Collector has been deprecated and won't be developed any fur

[graylog2] Re: Plugin Development: POM for org.graylog.plugins:usage-statistics is missing

2016-07-05 Thread Jochen Schalanda
Hi, the Anonymous Usage Statistics plugin is being pulled in by the graylog-project meta project: https://github.com/Graylog2/graylog-project/blob/129db698ff8de4327c1f4ab23c9253e9afb56998/pom.xml#L40-L44 Cheers, Jochen On Tuesday, 5 July 2016 11:08:47 UTC+2, cazy wrote: > > Hi, > > I get the

[graylog2] Re: Elasticsearch Cluster Unavailable

2016-07-05 Thread Jochen Schalanda
Hi Sahil, please check the corresponding logs of your Graylog node(s) in /var/log/graylog/ or /var/log/graylog-server/. Cheers, Jochen On Friday, 1 July 2016 17:51:47 UTC+2, sahil narula wrote: > > Graylog2.0.3 shows this error: "could not load field information" when i > go to search tab in g

[graylog2] Re: Graylog alerts - X-Forwarded-For showing as 'null'

2016-07-01 Thread Jochen Schalanda
Hi George, what kind of alarm callback are you using? If it's supporting templates, which ones are you using? Cheers, Jochen On Thursday, 30 June 2016 18:40:57 UTC+2, George Nussbaum wrote: > > Hello, > > I have set up alerting on one of my streams. The alerts come through > fine. However, t

[graylog2] Re: Graylog 2.0 compilation error : Cannot run program "git

2016-06-30 Thread Jochen Schalanda
Hi Anant, you have to make sure that the git binary is on your %PATH% environment (see http://blog.countableset.ch/2012/06/07/adding-git-to-windows-7-path/) and executable. Please take note that we cannot give extensive support for setting up your development environment. You're basically on y

[graylog2] Re: email callback and message.source..

2016-06-30 Thread Jochen Schalanda
Hi Stefan, please read the previous posts I wrote in this thread and the documentation section I've linked to. There is no single message object in the email body but always a collection of messages in the backlog variable which you have to iterate over with foreach. Cheers, Jochen On Thursd

[graylog2] Re: email callback and message.source..

2016-06-30 Thread Jochen Schalanda
Hi Stefan, you can access any message field inside the email *body* template using the variables described in http://docs.graylog.org/en/2.0/pages/streams.html#email-alert-callback. Cheers, Jochen On Thursday, 30 June 2016 10:08:03 UTC+2, Stefan Krüger wrote: > > Hi Jochen, > > sorry for my ba

[graylog2] Re: email callback and message.source..

2016-06-29 Thread Jochen Schalanda
Hi Stefan, could you please elaborate on your use case or the problem? I didn't understand your question. Cheers, Jochen On Wednesday, 29 June 2016 11:49:10 UTC+2, Stefan Krüger wrote: > > Hi Jochen, > > ok if I understand it correct, it is not possible to alert me if root as > been logged in,

[graylog2] Re: Alerting for flopping ports

2016-06-29 Thread Jochen Schalanda
Hi Emil, that's currently not possible with Graylog out of the box. Cheers, Jochen On Monday, 27 June 2016 22:34:58 UTC+2, Emil Grama wrote: > > > I'm new with graylog and maybe one of you guys can help: > I have in graylog lots of events of the type application > online/application offline g

[graylog2] Re: Backup of indices in Graylog 1.3

2016-06-29 Thread Jochen Schalanda
Hi Roberto, you can simply follow the standard Elasticsearch backup/restore procedures, see https://www.elastic.co/guide/en/elasticsearch/reference/1.7/modules-snapshots.html and https://www.elastic.co/guide/en/elasticsearch/guide/1.x/backing-up-your-cluster.html . If you upgraded to Graylog

[graylog2] Re: Syslog messages look different between Splunk and Graylog

2016-06-29 Thread Jochen Schalanda
Hi Keamas, the search query languages of Splunk and Graylog ( http://docs.graylog.org/en/2.0/pages/queries.html#search-query-language) aren't similar at all. You'll probably have to rewrite all of your queries. Regarding the extraction of structured information from the syslog messages, you'll

[graylog2] Re: email callback and message.source..

2016-06-29 Thread Jochen Schalanda
Hi Stefan, alerts always contain a collection of messages (in the backlog variable) which you have to iterate over and never just a single message. Additionally, the templating can't be used in the email subject. Please also refer to http://docs.graylog.org/en/2.0/pages/streams.html#email-aler

[graylog2] Re: disk space

2016-06-22 Thread Jochen Schalanda
Hi Hakan, please refer to the instructions in the Graylog documentation: http://docs.graylog.org/en/2.0/pages/configuration/graylog_ctl.html#extend-disk-space Cheers, Jochen On Wednesday, 22 June 2016 09:12:55 UTC+2, Hakan ÜRKMEZ wrote: > > How to extend volume disk space for virtual appliance

[graylog2] Re: Graylog 2.0.2 fails to connect to Elasticsearch 2.3.3

2016-06-21 Thread Jochen Schalanda
Hi, try setting elasticsearch_network_host to an IP address (or host name) that is reachable for the other Elasticsearch nodes in the ES cluster ( https://github.com/Graylog2/graylog2-server/blob/2.0.3/misc/graylog.conf#L245-L250 ). Cheers, Jochen On Tuesday, 21 June 2016 16:02:37 UTC+2, ironma

[graylog2] Re: Elasticsearch cluster unhealthy (RED)

2016-06-21 Thread Jochen Schalanda
LOW] (reason: [shards started [[graylog_0][1], > [graylog_0][3], [graylog_0][2], [graylog_0][3], [graylog_0][1], > [graylog_0][2]] ...]). > > Is that OK? What can I do from here if this is not OK? > > On Monday, June 20, 2016 at 6:25:04 PM UTC+8, Jochen Schalanda wrote: >>

Re: [graylog2] Re: Stream Stopped - Audit Stream Activity

2016-06-21 Thread Jochen Schalanda
Hi Michael, streams that are taking too long to compute automatically paused by Graylog, see https://github.com/Graylog2/graylog2-server/blob/2.0.3/misc/graylog.conf#L350-L358 for relevant configuration settings. Cheers, Jochen On Tuesday, 21 June 2016 13:34:22 UTC+2, Michael Brosnan wrote:

[graylog2] Re: parsing syslog messages on many services

2016-06-21 Thread Jochen Schalanda
Hi, On Tuesday, 21 June 2016 12:58:13 UTC+2, Андрей Грошев wrote: > > In the case of "pipelines" each string will processed two times, > This may have an effect under heavy loads. > Right? > Yes, correct. Cheers, Jochen -- You received this message because you are subscribed to the Google Gr

[graylog2] Re: graylog server warning every 5-30 minutes

2016-06-21 Thread Jochen Schalanda
Hi Ariel, just for reference, I'll paraphrase the explanation from IRC: Each Graylog node "registers" itself (node id, URI to the Graylog REST API, > timestamp of the last heartbeat) in MongoDB (see the nodes collection). > The timeout/cleanup interval is quite aggressive (2s, see > stale_mast

[graylog2] Re: parsing syslog messages on many services

2016-06-21 Thread Jochen Schalanda
Hi, I'd recommend using different inputs for each type of device/service you have in your ecosystem. Using the new processing pipelines in Graylog 2.x (see http://docs.graylog.org/en/2.0/pages/pipelines.html for details), you could also use 1 input and run different rules for each source devi

[graylog2] Re: Widget not update when use keyword "TODAY"

2016-06-20 Thread Jochen Schalanda
Hi Ivo, this is a bug and will be fixed in Graylog 2.0.3 (see https://github.com/Graylog2/graylog2-server/pull/2335 for details). Cheers, Jochen On Monday, 20 June 2016 14:19:33 UTC+2, Ivo Sestren Junior wrote: > > Hi, > > I using keyword "TODAY" in filter for show a widget. > In this case, wid

[graylog2] Re: Elasticsearch cluster unhealthy (RED)

2016-06-20 Thread Jochen Schalanda
Hi Arief, please check the logs of the Elasticsearch node(s) for errors. You can find the log files in the /var/log/graylog/elasticsearch (or /var/log/elasticsearch) directory. Cheers, Jochen On Monday, 20 June 2016 09:32:28 UTC+2, Arief Hydayat wrote: > > Dear Graylog users and Guru, > > Need

[graylog2] Re: ApiError "HTTP 404 Not Found" when curling graylog-server:12900

2016-06-17 Thread Jochen Schalanda
Hi Sebastien, I logged on the container, locally curling localhost:12900 then got > "{"type":"ApiError","message":"HTTP 404 Not Found"}". That's correct. There is simply no handler defined for the "root" resource in Graylog. Is there anything not working for you? Cheers, Jochen On Friday,

[graylog2] Re: Input Failed to Start

2016-06-17 Thread Jochen Schalanda
Hi Justin, this sounds like you tried to bind an input to a privileged port (1-1024), which only the superuser (root) is allowed to. You can either change the port of the input to a high port (anything above 1024, e. g. 5514 instead of 514) or use a tool like authbind to allow the JVM to bind

[graylog2] Re: timezone weirdness

2016-06-16 Thread Jochen Schalanda
Hi John, do you receive any messages at all from the syslog server after changing its configuration? Also take a look at https://github.com/Graylog2/graylog-guide-syslog-linux/blob/master/README.md for instructions how to configure rsyslog to work with Graylog. Cheers, Jochen On Thursday, 16

[graylog2] Re: Cardinality field of Statistics

2016-06-16 Thread Jochen Schalanda
Hi Steve, it's just a SWAG , but field statistics only work for numeric fields, so maybe there are non-numeric values for that field in one or more of the indices covering the 8 hours time span but not in the indices covering the 1-2 hour

[graylog2] Re: Application Security Questions

2016-06-15 Thread Jochen Schalanda
Hi Beth, […] if there was any white box/black box or static code/web application > scanning that is done before a release is made. Is this up to the > individual contributor? That's currently up to the individual contributor and the reviewer of the change set. My second question is if there

[graylog2] Re: Can Graylog receive binary log???

2016-06-15 Thread Jochen Schalanda
Hi Scarlet, Graylog supports a variety of input formats out of the box, but the proprietary Juniper binary log is not one of them. You can, however, write a custom Graylog input plugin which will parse the incoming data and make sense of it. See http://docs.graylog.org/en/2.0/pages/plugins.htm

Re: [graylog2] Can't get Graylog Appliance 2.0.2 to work with SSL and external IP address.

2016-06-13 Thread Jochen Schalanda
Hi, On Monday, 13 June 2016 02:54:55 UTC+2, 123Dev wrote: > > *graylog-ctl enforce-ssl *is not setting the REST transport on HTTPS > > In our case API browser is on: http://10.20.1.229:12900/api-browser and > is accessible > If I try to check if it is also accessible on SSL, > https://10.20.1.22

[graylog2] Re: not able to send stream alert

2016-06-13 Thread Jochen Schalanda
Hi Nitiya, please don't spam the mailing list if you don't receive an answer within 5 minutes. https://groups.google.com/d/msg/graylog2/1PqMA5-43Js/MyxRMDvWBgAJ Cheers, Jochen On Monday, 13 June 2016 07:11:23 UTC+2, rvb n wrote: > > > > Hi Friends > > # Email transport > transport_email_enabl

[graylog2] Re: unable to send email alert

2016-06-13 Thread Jochen Schalanda
transport_email_auth_password = XXX > transport_email_subject_prefix = [graylog] > transport_email_from_email = r...@graylog.com > > > On Monday, 30 May 2016 13:56:40 UTC+5:30, Jochen Schalanda wrote: >> >> Hi, >> >> what's your current configuration for the

[graylog2] Re: https error

2016-06-13 Thread Jochen Schalanda
Hi Marcio, is Graylog using HTTPS on the backend (i. e. the web interface listener on port 9000 and the REST API listener on port 12900)? FWIW, it usually doesn't make too much sense to use HTTPS in the backend (Graylog) and in the frontend (Apache httpd as reverse proxy). Cheers, Jochen On S

[graylog2] Re: Best way to get tomcat logs including exception and stacktraces into graylog2? using tomcat and log4j 1.2.17

2016-06-11 Thread Jochen Schalanda
Hi Terry, I'd recommend using proper log4j (or j.u.l) GELF appender instead of reading the log files with another process like Filebeat, mostly because of the handling of multi-line log messages. Personally, I like logstash-gelf , simply because it covers basically

[graylog2] Re: Search with wildcard in other fields

2016-06-10 Thread Jochen Schalanda
Hi Carlos, the way Elasticsearch (or rather Lucene) is retrieving documents by query is very much dependent on the analyzer that has been used at index time. Please take a look at the following pages to gain deeper understanding about analyzers in Elasticsearch: - https://www.elastic.c

[graylog2] Re: I want a dashboard widget to display the most recent value of a field from the last message in a stream

2016-06-10 Thread Jochen Schalanda
Hi Al, that's currently not possible with Graylog out of the box but you should be able to write a simple plugin providing a dashboard widget for that. Cheers, Jochen On Thursday, 9 June 2016 18:51:37 UTC+2, Al J wrote: > > I want a dashboard widget to display the most recent value of a field f

[graylog2] Re: NTP

2016-06-09 Thread Jochen Schalanda
Hi, what exactly did you try to do and how did it fail? Installing NTP is basically just running apt-get install ntp and sudo service ntp start. Also check https://help.ubuntu.com/lts/serverguide/NTP.html for more details. Cheers, Jochen On Wednesday, 8 June 2016 16:39:38 UTC+2, zbwsys...@gma

[graylog2] Re: Lost config after upgrade

2016-06-09 Thread Jochen Schalanda
Hi Phil, a little too late, but the upgrade notes for Graylog 1.x to Graylog 2.0.x are listed in the documentation at http://docs.graylog.org/en/2.0/pages/upgrade.html, e. g. http://docs.graylog.org/en/2.0/pages/upgrade.html#overview for a list of removed/changed configuration settings. Cheer

[graylog2] Re: Search with wildcard in other fields

2016-06-09 Thread Jochen Schalanda
Hi Ivo, in order to do wildcard searches, the respective fields have to be analyzed while indexing. You can add a custom index mapping in Elasticsearch to define a schema for your data: - https://www.elastic.co/guide/en/elasticsearch/reference/2.3/mapping.html - https://www.elas

[graylog2] Re: how to export logs as csv

2016-06-09 Thread Jochen Schalanda
Hi, please check the logs of Graylog (should be "server"), as nginx is only a reverse proxy and won't contain any useful information in this case. Cheers, Jochen On Wednesday, 8 June 2016 19:31:26 UTC+2, Mohammad Amin Khodamoradi wrote: > > I went to this directory that include server , mongoDB

[graylog2] Re: how to export logs as csv

2016-06-07 Thread Jochen Schalanda
Hi, are there any error messages in the logs of your Graylog node? You can check this by logging into the virtual machine and checking the /var/log/graylog directory. Cheers, Jochen On Tuesday, 7 June 2016 17:50:57 UTC+2, Mohammad Amin Khodamoradi wrote: > > Hi > I want to export logs as csv

Re: [graylog2] Re: drop logs based on rules

2016-06-07 Thread Jochen Schalanda
f I want to drop a msg is this how we do it? > > rule "function howto" > when > has_field("transaction_date") > thendrop_message() > > > On Tue, Jun 7, 2016 at 1:35 PM, Jochen Schalanda wrote: > >> Hi Rajeev, >> >> you can use the mes

<    5   6   7   8   9   10   11   12   13   14   >