[graylog2] Re: graylog-web for 2.0

[sam]

HI Walderba,


Yes I had the below config changes made to server.conf file, 

I see the below error : 
 Server currently unavailable

We are experiencing problems connecting to the Graylog server running on 
*http://127.0.0.1:12900/*. Please verify that the server is healthy and 
working correctly.

You will be automatically redirected to the previous page once we can 
connect to the server.



On Wednesday, August 3, 2016 at 7:55:03 PM UTC-7, walderba...@gmail.com 
wrote:
>
> You must edit your server.conf and:
>
> 1. Ensure the web_listen_uri is an IP that is reachable by outside 
> servers, unless you only want to access it locally.
> 2. Same for rest_listen_uri.
> 3. Ensure web_listen variable is True.
> 4. Ensure you have a password_secret is set.
> 5. Ensure root_password_sha2 is set.
> 6. Access the WebUI at http://IPAddress:9000 
>
> On Wednesday, August 3, 2016 at 5:47:00 PM UTC-6, sam wrote:
>>
>> Hi All,
>>
>> I installed the graylog 2.0 version in my box. I don't see any graylog 
>> web for this version? DO we have one graylog -web for this version. If not 
>> can anyone of you tell me how to access the web interface. 
>>
>>
>>
>>
>>
>>
>>
>> Thank you 
>> sam
>>
>>

-- 
You received this message because you are subscribed to the Google Groups 
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/graylog2/d8604e4b-d470-47e6-b026-9c28f09fc059%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[graylog2] Re: graylog-web for 2.0

[Jochen Schalanda]

Hi Sam,

please 
read http://docs.graylog.org/en/2.0/pages/configuration/web_interface.html 
and follow that documentation.

Your Graylog REST API seems to only list on the loopback interface.

Cheers,
Jochen

On Thursday, 4 August 2016 09:43:24 UTC+2, sam wrote:
>
> HI Walderba,
>
>
> Yes I had the below config changes made to server.conf file, 
>
> I see the below error : 
>  Server currently unavailable
>
> We are experiencing problems connecting to the Graylog server running on 
> *http://127.0.0.1:12900/ 
> *. Please verify that the server is healthy and 
> working correctly.
>
> You will be automatically redirected to the previous page once we can 
> connect to the server.
>
>
>
> On Wednesday, August 3, 2016 at 7:55:03 PM UTC-7, walderba...@gmail.com 
> wrote:
>>
>> You must edit your server.conf and:
>>
>> 1. Ensure the web_listen_uri is an IP that is reachable by outside 
>> servers, unless you only want to access it locally.
>> 2. Same for rest_listen_uri.
>> 3. Ensure web_listen variable is True.
>> 4. Ensure you have a password_secret is set.
>> 5. Ensure root_password_sha2 is set.
>> 6. Access the WebUI at http://IPAddress:9000 
>>
>> On Wednesday, August 3, 2016 at 5:47:00 PM UTC-6, sam wrote:
>>>
>>> Hi All,
>>>
>>> I installed the graylog 2.0 version in my box. I don't see any graylog 
>>> web for this version? DO we have one graylog -web for this version. If not 
>>> can anyone of you tell me how to access the web interface. 
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>> Thank you 
>>> sam
>>>
>>>

-- 
You received this message because you are subscribed to the Google Groups 
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/graylog2/850357e9-14f0-4e49-9387-ff2d95cb7a79%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[graylog2] Re: Overall plan for use - comments?

[luke . vidler]

On Thursday, 4 August 2016 05:41:59 UTC+8, Linwood Ferguson wrote:
>
> I'm struggling a bit to avoid the "just throw logs in and figure out later 
> what to do with them" inclination, and trying to plan how the different 
> pieces might best be used.
>
> I'd appreciate any comments as to whether this is a good approach.  I even 
> have a picture.
>
> My thinking goes like this: 
>
> 1) Bring data in and use extractors (mostly grok) to normalize to some set 
> of standardized fields, somewhat based on what I can get free from Gelf.  I 
> expect this kind of normalization will be a work in progress forever.  Grok 
> especially but extractors in general seem easier to use than pipelines for 
> normalization.
>
> 2) Let everything just stay in the default stream at that point, and feed 
> into a set of pipeline rules.
>
> 3) Pipelines decide how to map the log messages from the physical origins 
> into logical groupings, for example actual device (e.g. hardware or 
> similar) events, infrastructure logins to network gear, VPN and similar 
> access, web logs (probably different types)., etc.
>
> 3A) Garbage messages no one really cares about get dropped here.
>
> 3B) Some messages might end up in two places, e.g. we might have certain 
> data access streams which are also web or FTP logs.
>
> 4) Streams control the alarms.
>
> All wet, or going in the right direction? 
>
>

Hi Linwood, 

Thanks for sharing this, been working with the boss on a Graylog project 
and we have discussed this a few times as there are various places where 
you can filter, so what is the optimal approach?

So far we have a good setup for Linux based server hosts (primarily only 
interested in '/var/log/security' naughties and SELinux 'avc = denied' 
which generally breaks something). I tried a number of 'shippers' but the 
one we decided on was elastic 'filebeat'. 

It had the following advantages:

- Written in Go so single binary (i.e. doesn't require a JVM)
- Easy config  (yaml based do config management friendly) 
- Allows for simple filtering before the data leaves the server (instead of 
pumping all your logs and then filtering with Graylog so your only 
searching against the data you care about)
- Graylog has an input Plugin that supports the beats shipper. 

For Linux based hosts IMO this is a lot more fun than syslog manipulation 
(which you will most likely be forced to do for your other devices). So 
outside of recommending the shipper (which was recommended to me) I think 
that filtering at the source is an effective strategy.

HTH Cheers Luke.
 

-- 
You received this message because you are subscribed to the Google Groups 
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/graylog2/1dbef3f3-76c1-4eda-86e4-c1b1ca0d8a16%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[graylog2] IIS logs working but how to search for response code like 404

[Guillaume Migaszewski]

Dear Graylog group. 

I am new to Graylog, sorry if my questions may look very basic . With your 
help (thanks a lot) , I have managed to setup Graylog and collect IIS logs .

IIS logs are now index inmy Graylog and of course  I would like to collect 
some info. 

As a sysadmin my dream is to have a nice dashboard per web server which 
will output info like  top client ip address  , http response code etc  
.

As advised by Graylog contextual help I try to search with command 
source:my web AND http_responde_code:400 .

I have no real search results. I have many fieds and the best way I have 
found to retrieve data logs is based on cs_referer  . source:mywebserver 
cs-Referer: 200 .


Questions :

 1. Is my way of indexing IIS log with NXlog efficient. ? Maybe there is an 
issue with my filed mapping ? .

 2 .Which field do you use to gather from the log  http response code ? How 
do you proceed within Graylog ?. 


Short sample of my IIS log Fields nxlog.conf

Fields $date, $time, $s-ip, $cs-method, $cs-uri-stem, $cs-uri-query, 
$s-port, $cs-username, $c-ip, $csUser-Agent, $cs-Referer, $sc-status, 
$sc-substatus, $sc-win32-status, $time-taken
FieldTypes string, string, string, string, string, string, integer, 
string, string, string, string, integer, integer, integer, integer


3. Is there a template/plugin for montoring IIS log ? 

Thanks a lot for your time.

Guillaume.



-- 
You received this message because you are subscribed to the Google Groups 
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/graylog2/ed076c29-71af-4686-9b89-226b8221d000%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [graylog2] Re: stuck to install graylog to our VPS Linux CentOS 6

[Lam Do]

Jochen,

I need your help again (I apologize for my limitation of knowledge in this
area, I'm just a newbie :( ). I will explain my steps bellow:

1. I'm working on the installations steps from the official document and
using 'service' command instead of 'systemd' to install and enable to run
the service of Java , MongoDB, ElasticSearch and Graylog server without
restarting the server (Do we really need to restart the server?). I'm
checking by 'top' command and see all these 3 services are running already.

2. Next step is configuring the SELinux but my VPS doesn't run SELinux so I
guess I don't need to follow the steps on the document for SE Linux.

3. Next, Do I need to enable Graylog if I don't use SELinux. I tried this
command but it doesn't work to me. It return sudo : semanage command not
found. I guess because SElinux is disable already so it won't run this
command.

   - Graylog REST API: sudo semanage port -a -t http_port_t -p tcp 12900


I'm kind of stuck here and don't know what to do next.

Beside, I'm thinking of working on graylog configuration on
/etc/graylog/server/server.conf and I also have some question:

- If I install Graylog to my VPS server then should I
configure web_listen_uri = http://my_VPS_IP:9000 and rest_transport_uri |
rest_listen_uri = http://my_VPS_IP:12900/ in order to use graylog-web
interface from end browser with the URL http://my_VPS_IP:9000 ?

Thanks again for your suggestion and your help. Please correct me if I'm
doing anything wrong.

Regards,
Luke

On Wed, Aug 3, 2016 at 11:18 PM, Lam Do  wrote:

> Jochen,
>
> It sounds good to me. I will start to work on it tomorrow. Thanks a lot
> for your help, I really appreciate it.
>
> Best Regards,
> Lam Do
>
> Sent from mobile device.
>
> On Aug 3, 2016 10:25 PM, "Jochen Schalanda"  wrote:
>
>> Hi Luke,
>>
>> I'd recommend following the official documentation, which is always
>> up-to-date, instead of some 3rd party blog posts:
>> http://docs.graylog.org/en/2.0/pages/installation/os/centos.html
>>
>> The steps to install Graylog on CentOS 6 are fairly similar. You'll have
>> to use SysV init scripts instead of systemd but except for that, it should
>> be the same.
>>
>> Cheers,
>> Jochen
>>
>> On Wednesday, 3 August 2016 16:23:49 UTC+2, Lam Do wrote:
>>>
>>> Hi Jochen,
>>>
>>> Now it makes sense to me because I installed the latest repository
>>> Graylog 2.0. And the documentation explains the steps for Centos7 while my
>>> VPS is Centos6 so I googled some other articles and those  mention about
>>> graylog-web but properly for Graylog 1.x. Others  is for Graylog2 with
>>> Centos6 and it mentions about graylog-web too, for example
>>> http://www.richardyau.com/?p=377
>>> 
>>> Is it good article to follow?
>>>
>>> Based on your suggestions,  it sounds to me to follow again the steps
>>> from Graylog 2.0 documentation. Does Centos6 have the same steps with
>>> Centos7?
>>>
>>> Thanks for your help,
>>>
>>> Best Regards,
>>> Lam Do
>>>
>> --
>> You received this message because you are subscribed to a topic in the
>> Google Groups "Graylog Users" group.
>> To unsubscribe from this topic, visit
>> https://groups.google.com/d/topic/graylog2/XwLVRe5geAg/unsubscribe.
>> To unsubscribe from this group and all its topics, send an email to
>> graylog2+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/graylog2/8b92d654-8ce0-464d-8382-4be501b8bd26%40googlegroups.com
>> 
>> .
>> For more options, visit https://groups.google.com/d/optout.
>>
>


-- 

*Do Nhu Lam (Luke)*

*Technical Director*



*SEADEV | International Marketing Agency*

*​**(VN)Office: *(84)0511.393.8185

*​**(US)Office:* (1)202.888.4943

*(VN)Mobile:* (84) 979.716.185

*Email:* lam...@seadev.com.vn

*Website: *www.seadev.com.vn

-- 
You received this message because you are subscribed to the Google Groups 
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/graylog2/CAKc-uyXcth0qdvfC9gmzTGtxi4icvPx%2B00Dbq3b2R9ybwugvgA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

[graylog2] Re: Graylog not writing to elasticsearch after out of disk space, ES green but...

[julioqc47]

Ok so what are those commands?? Can you provide a link or something? 
Googling what you mention isn't very explicit and results aren't helpful. 
Thx

On Tuesday, 5 April 2016 04:02:08 UTC-4, Jochen Schalanda wrote:
>
> Hi,
>
> it looks like your journal is corrupted. You can either try to recover and 
> repair it (see the journal-related commands in Graylog) or simply remove 
> the journal files from disk.
>
> Cheers,
> Jochen
>
> On Monday, 4 April 2016 22:56:47 UTC+2, kluch wrote:
>>
>> After "out of disk space" I removed journals, old indices, cycled 
>> deflector and then even deleted all indices from ES but it not helped at 
>> all. All graylog nodes restarted and still nothing.
>> I dont know from where graylog reads/displays negative (with minus) value 
>> of unprocessed messages.  On web it looks like this:
>> Processing *1* incoming and *0* outgoing msg/s. *-1,195,856,763 
>> unprocessed messages* are currently in the journal, in 1 segments. *1 
>> messages* have been appended to, and *0 messages* have been read from 
>> the journal in the last second.
>> Moreover graylog does not write to elasticsearch so only unprocessed 
>> messages counter is changing. Time is in sync with ntp. Version 1.3.4 
>> from OVA.
>> Any help?
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/graylog2/bd15bc59-bacb-419f-95fa-62314492cb86%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[graylog2] Re: Overall plan for use - comments?

[Linwood Ferguson]

Thanks.  So far I've largely ignored unix logs and even web logs, as the 
client I'm first considering this for is mostly windows and lots of network 
devices at lots of remote sites, so I've been concentrating there, but will 
experiment with filebeat when I get there.

The main place I was looking to do some normalization is the myriad of 
network devices (and about 12 years worth of firmware versions, slowly 
getting modernized). 


On Thursday, August 4, 2016 at 5:23:34 AM UTC-4, luke@e2x.co.uk wrote:
>
>
>
> On Thursday, 4 August 2016 05:41:59 UTC+8, Linwood Ferguson wrote:
>>
>> I'm struggling a bit to avoid the "just throw logs in and figure out 
>> later what to do with them" inclination, and trying to plan how the 
>> different pieces might best be used.
>>
>> I'd appreciate any comments as to whether this is a good approach.  I 
>> even have a picture.
>>
>> My thinking goes like this: 
>>
>> 1) Bring data in and use extractors (mostly grok) to normalize to some 
>> set of standardized fields, somewhat based on what I can get free from 
>> Gelf.  I expect this kind of normalization will be a work in progress 
>> forever.  Grok especially but extractors in general seem easier to use than 
>> pipelines for normalization.
>>
>> 2) Let everything just stay in the default stream at that point, and feed 
>> into a set of pipeline rules.
>>
>> 3) Pipelines decide how to map the log messages from the physical origins 
>> into logical groupings, for example actual device (e.g. hardware or 
>> similar) events, infrastructure logins to network gear, VPN and similar 
>> access, web logs (probably different types)., etc.
>>
>> 3A) Garbage messages no one really cares about get dropped here.
>>
>> 3B) Some messages might end up in two places, e.g. we might have certain 
>> data access streams which are also web or FTP logs.
>>
>> 4) Streams control the alarms.
>>
>> All wet, or going in the right direction? 
>>
>>
>
> Hi Linwood, 
>
> Thanks for sharing this, been working with the boss on a Graylog project 
> and we have discussed this a few times as there are various places where 
> you can filter, so what is the optimal approach?
>
> So far we have a good setup for Linux based server hosts (primarily only 
> interested in '/var/log/security' naughties and SELinux 'avc = denied' 
> which generally breaks something). I tried a number of 'shippers' but the 
> one we decided on was elastic 'filebeat'. 
>
> It had the following advantages:
>
> - Written in Go so single binary (i.e. doesn't require a JVM)
> - Easy config  (yaml based do config management friendly) 
> - Allows for simple filtering before the data leaves the server (instead 
> of pumping all your logs and then filtering with Graylog so your only 
> searching against the data you care about)
> - Graylog has an input Plugin that supports the beats shipper. 
>
> For Linux based hosts IMO this is a lot more fun than syslog manipulation 
> (which you will most likely be forced to do for your other devices). So 
> outside of recommending the shipper (which was recommended to me) I think 
> that filtering at the source is an effective strategy.
>
> HTH Cheers Luke.
>  
>

-- 
You received this message because you are subscribed to the Google Groups 
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/graylog2/e6bfaba7-f129-482b-9382-e021df9193bc%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[graylog2] Separate VMs Running Graylog & Elastic Search Not Connecting

[Nathan Mace]

Per my other thread, I decided to delete the VMs and start over from 
scratch.  Instead of running ES on two nodes and Graylog on one of those 
nodes, I've got two VM.  Running ES only and the other running Graylog and 
MongoDB only.

ansted -> x.x.x.146 Running Graylog and MongoDB
ansted-search-01 -> x.x.x.149 Running ElasticSearch

I've attached the two config files (one for Graylog and one for ES).  I 
removed all of the commented out lines from the text files.  I've also 
attached the log file that Graylog is generating.  The repeated error is:

2016-08-04T14:14:09.089-04:00 WARN  [unicast] 
[graylog-f51e5052-55d0-4c4a-92ca-89a28282b3f4] failed to send ping to 
[{#zen_unicast_1#}{x.x.x.149}{x.x.x.149:9200}]
org.elasticsearch.transport.ReceiveTimeoutTransportException: 
[][x.x.x.149:9200][internal:discovery/zen/unicast] request_id [18] timed 
out after [3750ms]
at 
org.elasticsearch.transport.TransportService$TimeoutHandler.run(TransportService.java:679)
 
[graylog.jar:?]
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) 
[?:1.8.0_101]
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) 
[?:1.8.0_101]
at java.lang.Thread.run(Thread.java:745) [?:1.8.0_101]

It is trying to connect to the ES server's IP, but I am out of ideas as for 
what is wrong.  I know I have ES set to use port 9300 for transport and 
Graylog is (currently) set to port 9200, but changing Graylog's config file 
doesn't seem to change the error in the Graylog log file.

-- 
You received this message because you are subscribed to the Google Groups 
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/graylog2/27bac447-822a-4a22-be1b-2453d44b2df1%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
2016-08-04T14:13:30.660-04:00 INFO  [CmdLineTool] Loaded plugin: Collector 
1.0.3 [org.graylog.plugins.collector.CollectorPlugin]
2016-08-04T14:13:30.661-04:00 INFO  [CmdLineTool] Loaded plugin: Enterprise 
Integration Plugin 1.0.3 
[org.graylog.plugins.enterprise_integration.EnterpriseIntegrationPlugin]
2016-08-04T14:13:30.661-04:00 INFO  [CmdLineTool] Loaded plugin: 
MapWidgetPlugin 1.0.3 [org.graylog.plugins.map.MapWidgetPlugin]
2016-08-04T14:13:30.661-04:00 INFO  [CmdLineTool] Loaded plugin: Pipeline 
Processor Plugin 1.0.0-beta.5 
[org.graylog.plugins.pipelineprocessor.ProcessorPlugin]
2016-08-04T14:13:30.661-04:00 INFO  [CmdLineTool] Loaded plugin: Anonymous 
Usage Statistics 2.0.3 [org.graylog.plugins.usagestatistics.UsageStatsPlugin]
2016-08-04T14:13:30.734-04:00 INFO  [CmdLineTool] Running with JVM arguments: 
-Xms1g -Xmx1g -XX:NewRatio=1 -XX:+ResizeTLAB -XX:+UseConcMarkSweepGC 
-XX:+CMSConcurrentMTEnabled -XX:+CMSClassUnloadingEnabled -XX:+UseParNewGC 
-XX:-OmitStackTraceInFastThrow 
-Dlog4j.configurationFile=file:///etc/graylog/server/log4j2.xml 
-Djava.library.path=/usr/share/graylog-server/lib/sigar 
-Dgraylog2.installation_source=rpm
2016-08-04T14:13:32.757-04:00 INFO  [InputBufferImpl] Message journal is 
enabled.
2016-08-04T14:13:32.926-04:00 INFO  [LogManager] Loading logs.
2016-08-04T14:13:32.990-04:00 INFO  [LogManager] Logs loading complete.
2016-08-04T14:13:32.991-04:00 INFO  [KafkaJournal] Initialized Kafka based 
journal at /var/lib/graylog-server/journal
2016-08-04T14:13:33.003-04:00 INFO  [InputBufferImpl] Initialized 
InputBufferImpl with ring size <65536> and wait strategy 
, running 2 parallel message handlers.
2016-08-04T14:13:33.034-04:00 INFO  [cluster] Cluster created with settings 
{hosts=[localhost:27017], mode=SINGLE, requiredClusterType=UNKNOWN, 
serverSelectionTimeout='3 ms', maxWaitQueueSize=5000}
2016-08-04T14:13:33.067-04:00 INFO  [cluster] No server chosen by 
ReadPreferenceServerSelector{readPreference=primary} from cluster description 
ClusterDescription{type=UNKNOWN, connectionMode=SINGLE, 
all=[ServerDescription{address=localhost:27017, type=UNKNOWN, 
state=CONNECTING}]}. Waiting for 3 ms before timing out
2016-08-04T14:13:33.099-04:00 INFO  [connection] Opened connection 
[connectionId{localValue:1, serverValue:84}] to localhost:27017
2016-08-04T14:13:33.100-04:00 INFO  [cluster] Monitor thread successfully 
connected to server with description ServerDescription{address=localhost:27017, 
type=STANDALONE, state=CONNECTED, ok=true, 
version=ServerVersion{versionList=[3, 2, 8]}, minWireVersion=0, 
maxWireVersion=4, maxDocumentSize=16777216, roundTripTimeNanos=568164}
2016-08-04T14:13:33.106-04:00 INFO  [connection] Opened connection 
[connectionId{localValue:2, serverValue:85}] to localhost:27017
2016-08-04T14:13:33.288-04:00 INFO  [NodeId] Node ID: 
f51e5052-55d0-4c4a-92ca-89a28282b3f4
2016-08-04T14:13:33.361-04:00 INFO  [node] 
[graylog-f51e5052-55d0-4c4a-92ca-89a28282b3f4] version[2.3.2], pid[24882], 
build[b9e4a6a/2016-04-21T16:03:47

[graylog2] Re: Overall plan for use - comments?

[walderbachjoshua]

I have graylog sidecar/nxlog on every windows machine, feed into a single 
input on my GL server.  I then use streams to filter out by sourcename and 
sourcemodulename that I defined in each nxlog.conf, to see for example just 
logs from a particular thing like my platform's logs.  I have a separate 
stream for Windows event logs.  If I ever wanna see everything coming in 
from a server, I look under collectors and select the machine.  Each 
collector shares the same name as the server in my situation.

-Joshua


On Wednesday, August 3, 2016 at 3:41:59 PM UTC-6, Linwood Ferguson wrote:
>
> I'm struggling a bit to avoid the "just throw logs in and figure out later 
> what to do with them" inclination, and trying to plan how the different 
> pieces might best be used.
>
> I'd appreciate any comments as to whether this is a good approach.  I even 
> have a picture.
>
> My thinking goes like this: 
>
> 1) Bring data in and use extractors (mostly grok) to normalize to some set 
> of standardized fields, somewhat based on what I can get free from Gelf.  I 
> expect this kind of normalization will be a work in progress forever.  Grok 
> especially but extractors in general seem easier to use than pipelines for 
> normalization.
>
> 2) Let everything just stay in the default stream at that point, and feed 
> into a set of pipeline rules.
>
> 3) Pipelines decide how to map the log messages from the physical origins 
> into logical groupings, for example actual device (e.g. hardware or 
> similar) events, infrastructure logins to network gear, VPN and similar 
> access, web logs (probably different types)., etc.
>
> 3A) Garbage messages no one really cares about get dropped here.
>
> 3B) Some messages might end up in two places, e.g. we might have certain 
> data access streams which are also web or FTP logs.
>
> 4) Streams control the alarms.
>
> All wet, or going in the right direction? 
>
>
>
>
>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/graylog2/ecc7d1b2-8015-40f6-8512-f51e74bae43e%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[graylog2] [ANNOUNCE] Graylog v2.1-beta.2 has been released

[Lennart Koopmann]

Hi everyone,

we just released Graylog v2.1-beta.2. Important changes and full
release announcement can be found here:

  * https://www.graylog.org/blog/63-announcing-graylog-v2-1-0-beta-2

Thanks,
Lennart

-- 
You received this message because you are subscribed to the Google Groups 
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/graylog2/CADRA1nmqoBanmsLn8FUpf1yX62CT%2B0WgQoc7Q1rNMmek0Sr43g%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

[graylog2] Re: Separate VMs Running Graylog & Elastic Search Not Connecting

[Jochen Schalanda]

Hi Nathan,

try removing the elasticsearch_transport_tcp_port setting from your Graylog 
configuration and add the port (9300) to the IP address in 
elasticsearch_discovery_zen_ping_unicast_hosts (so that it's bein set to 
x.x.x.149:9300).

Additionally, your web_endpoint_uri is wrong and should be removed 
completely (or at least point to the public address of the Graylog REST 
API).

Cheers,
Jochen

On Thursday, 4 August 2016 21:14:37 UTC+2, Nathan Mace wrote:
>
> Per my other thread, I decided to delete the VMs and start over from 
> scratch.  Instead of running ES on two nodes and Graylog on one of those 
> nodes, I've got two VM.  Running ES only and the other running Graylog and 
> MongoDB only.
>
> ansted -> x.x.x.146 Running Graylog and MongoDB
> ansted-search-01 -> x.x.x.149 Running ElasticSearch
>
> I've attached the two config files (one for Graylog and one for ES).  I 
> removed all of the commented out lines from the text files.  I've also 
> attached the log file that Graylog is generating.  The repeated error is:
>
> 2016-08-04T14:14:09.089-04:00 WARN  [unicast] 
> [graylog-f51e5052-55d0-4c4a-92ca-89a28282b3f4] failed to send ping to 
> [{#zen_unicast_1#}{x.x.x.149}{x.x.x.149:9200}]
> org.elasticsearch.transport.ReceiveTimeoutTransportException: 
> [][x.x.x.149:9200][internal:discovery/zen/unicast] request_id [18] timed 
> out after [3750ms]
> at 
> org.elasticsearch.transport.TransportService$TimeoutHandler.run(TransportService.java:679)
>  
> [graylog.jar:?]
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>  
> [?:1.8.0_101]
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>  
> [?:1.8.0_101]
> at java.lang.Thread.run(Thread.java:745) [?:1.8.0_101]
>
> It is trying to connect to the ES server's IP, but I am out of ideas as 
> for what is wrong.  I know I have ES set to use port 9300 for transport and 
> Graylog is (currently) set to port 9200, but changing Graylog's config file 
> doesn't seem to change the error in the Graylog log file.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/graylog2/89575922-a7f4-4cf4-ad78-ae8587f0b4a0%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [graylog2] Re: stuck to install graylog to our VPS Linux CentOS 6

[Lam Do]

Jochen, 

I need your help again (I apologize for my limitation of knowledge in this 
area, I'm just a newbie :( ). I will explain my steps bellow:

1. I'm working on the installations steps from the official document and 
using 'service' command instead of 'systemd' to install and enable to run 
the service of Java , MongoDB, ElasticSearch and Graylog server without 
restarting the server (Do we really need to restart the server?). I'm 
checking by 'top' command and see all these 3 services are running already. 

 

2. Next step is configuring the SELinux but my VPS doesn't run SELinux so I 
guess I don't need to follow the steps on the document for SE Linux. 

 

3. Next, Do I need to enable Graylog if I don't use SELinux. I tried this 
command but it doesn't work to me. It return sudo : semanage command not 
found. I guess because SElinux is disable already so it won't run this 
command. 

o   Graylog REST API: sudo semanage port -a -t http_port_t -p tcp 12900

 
I'm kind of stuck here and don't know what to do next. 

 

Beside, I'm thinking of working on graylog configuration on 
/etc/graylog/server/server.conf and I also have some question: 


- If I install Graylog to my VPS server then should I 
configure web_listen_uri = http://my_VPS_IP:9000  and 
rest_transport_uri 
| rest_listen_uri = http://my_VPS_IP:12900/  in 
order to use graylog-web interface from end browser with the URL 
http://my_VPS_IP:9000  ? 

 

Thanks again for your suggestion and your help. Please correct me if I'm 
doing anything wrong. 

Regards, 
Luke

On Wednesday, August 3, 2016 at 10:25:41 PM UTC+7, Jochen Schalanda wrote:
>
> Hi Luke,
>
> I'd recommend following the official documentation, which is always 
> up-to-date, instead of some 3rd party blog posts: 
> http://docs.graylog.org/en/2.0/pages/installation/os/centos.html
>
> The steps to install Graylog on CentOS 6 are fairly similar. You'll have 
> to use SysV init scripts instead of systemd but except for that, it should 
> be the same.
>
> Cheers,
> Jochen
>
> On Wednesday, 3 August 2016 16:23:49 UTC+2, Lam Do wrote:
>>
>> Hi Jochen,
>>
>> Now it makes sense to me because I installed the latest repository 
>> Graylog 2.0. And the documentation explains the steps for Centos7 while my 
>> VPS is Centos6 so I googled some other articles and those  mention about 
>> graylog-web but properly for Graylog 1.x. Others  is for Graylog2 with 
>> Centos6 and it mentions about graylog-web too, for example
>> http://www.richardyau.com/?p=377 
>> 
>>  
>> Is it good article to follow?
>>
>> Based on your suggestions,  it sounds to me to follow again the steps 
>> from Graylog 2.0 documentation. Does Centos6 have the same steps with 
>> Centos7?
>>
>> Thanks for your help,
>>
>> Best Regards,
>> Lam Do
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/graylog2/421d1715-0a81-4e95-acb9-1ebe4e86d445%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.