[graylog2] Copy input extractor failure

[Rayees Namathponnan]

Hi All,

I created “Copy Input” extractor to get key value pair, here is my message, 
trying to extract level and status from all the messages 



2016-09-28 19:21:52,466 level=INFO tag="run_workflow.py" msg="Run complete for 
appname=cils, job_date=20160912, status=Passed starttime=Wed Sep 28 19:15:25 
2016, endtime=Wed Sep 28 19:21:47 2016, duration=0:6:21, inputs=[{"path": 
"/esss/srg/20160912_1473688239855_4f29bbb6efdb3c39", "tag": "cilPurge", 
"stats": {"size": "13.47MB"}}, {"path": "/compressed/cil/20160912", "tag": 
"cil", "stats": {"size": "580.16MB"}}], outputs=[{"path": 
"/processed/test/parse//cil", "tag": "cil.output.folder", "stats": {"diffSize": 
"645.78MB", "newFiles": [], "endSize": "18.93GB", "startSize": "18.30GB"}}]”



Its failing with below error 

0]: index [graylog_8], type [message], id 
[c0e7ea80-f572-11e6-b21e-5254007b267d], message 
[java.lang.IllegalArgumentException: Document contains at least one immense 
term in field="level" (whose UTF8 encoding is longer than the max length 
32766), all of which were skipped.  Please correct the analyzer to not produce 
such terms.  The prefix of the first immense term is: '[70, 101, 98, 32, 49, 
55, 32, 49, 57, 58, 51, 57, 58, 51, 53, 32, 115, 106, 101, 108, 107, 51, 51, 
32, 115, 121, 115, 116, 101, 109]...', original message: bytes can be at most 
32766 in length; got 34944]
2017-02-17T19:39:56.795-05:00 ERROR [Extractor] Could not apply converter 
[tokenizer] of extractor [77e451d0-f3b9-11e6-b21e-5254007b267d].
java.lang.IllegalArgumentException: Multiple entries with same key: id=4038, 
and id=4038,
at 
com.google.common.collect.ImmutableMap.checkNoConflict(ImmutableMap.java:136) 
~[graylog.jar:?]
at 
com.google.common.collect.RegularImmutableMap.checkNoConflictInKeyBucket(RegularImmutableMap.java:98)
 ~[graylog.jar:?]
at 
com.google.common.collect.RegularImmutableMap.fromEntryArray(RegularImmutableMap.java:84)
 ~[graylog.jar:?]
at 
com.google.common.collect.ImmutableMap$Builder.build(ImmutableMap.java:295) 
~[graylog.jar:?]
at 
org.graylog2.inputs.converters.TokenizerConverter.convert(TokenizerConverter.java:55)
 ~[graylog.jar:?]
at 
org.graylog2.plugin.inputs.Extractor.runConverters(Extractor.java:242) 
[graylog.jar:?]
at 
org.graylog2.plugin.inputs.Extractor.runExtractor(Extractor.java:228) 
[graylog.jar:?]
at org.graylog2.filters.ExtractorFilter.filter(ExtractorFilter.java:73) 
[graylog.jar:?]
at 
org.graylog2.messageprocessors.MessageFilterChainProcessor.process(MessageFilterChainProcessor.java:100)
 [graylog.jar:?]
at 
org.graylog2.buffers.processors.ServerProcessBufferProcessor.handleMessage(ServerProcessBufferProcessor.java:56)
 [graylog.jar:?]
at 
org.graylog2.shared.buffers.processors.ProcessBufferProcessor.dispatchMessage(ProcessBufferProcessor.java:82)
 [graylog.jar:?]
at 
org.graylog2.shared.buffers.processors.ProcessBufferProcessor.onEvent(ProcessBufferProcessor.java:61)
 [graylog.jar:?]
at 
org.graylog2.shared.buffers.processors.ProcessBufferProcessor.onEvent(ProcessBufferProcessor.java:35)
 [graylog.jar:?]
at com.lmax.disruptor.WorkProcessor.run(WorkProcessor.java:143) 
[graylog.jar:?]
at 
com.codahale.metrics.InstrumentedThreadFactory$InstrumentedRunnable.run(InstrumentedThreadFactory.java:66)
 [graylog.jar:?]
at java.lang.Thread.run(Thread.java:745) [?:1.8.0_111]
2017-02-17T19:39:57.677-05:00 ERROR [Messages] Failed to index [1] messages. 
Please check the index error log in your web interface for the reason. Error: 
failure in bulk execution:
[0]: index [graylog_8], type [message], id 
[c560a160-f572-11e6-b21e-5254007b267d], message 
[java.lang.IllegalArgumentException: Document contains at least one immense 
term in field="level" (whose UTF8 encoding is longer than the max length 
32766), all of which were skipped.  Please correct the analyzer to not produce 
such terms.  The prefix of the first immense term is: '[70, 101, 98, 32, 49, 
55, 32, 49, 57, 58, 51, 57, 58, 52, 53, 32, 115, 106, 101, 108, 107, 51, 51, 
32, 115, 121, 115, 116, 101, 109]...', original message: bytes can be at most 
32766 in length; got 34944]
^Z

-- 
You received this message because you are subscribed to the Google Groups 
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/graylog2/3475055D-8B53-48B8-8A48-61887A5A1EC1%40gmail.com.
For more options, visit https://groups.google.com/d/optout.

[graylog2] graylog not working after upgrading to v2.2.0 from 2.1.2

[Marsel Qako]

Hi,

I have two graylog servers clustered. One is configured as the master with 
full configuration the other as bakend-server. I upgraded both virtual 
appliances from 2.1.2 to 2.2.0. Before the upgrade everything was working 
fine. Now i have multiple errors and no logs show when searching. 

Every 20 seconds the page will reload and for a second and "server 
currently unavailable" page will show. 

The pages are blank under /system/indices, or streams, or alerts. But some 
like dashboards, or sources, or input work fine




I get the following errors in the logs. I'm not sure what changed with the 
new version, but it used to parse this logs with no problem. 

2017-02-17_19:58:39.81255 [3053]: index [graylog_447], type [message], id 
[fa52e365-f54a-11e6-8af1-005056a7396f], message 
[MapperParsingException[failed to parse [EventDate]]; nested: 
IllegalArgumentException[Invalid format: "2017/02/17" is malformed at 
"/02/17"];]

payloadSize=156, timestamp=2017-02-17T20:08:41.486Z, 
remoteAddress=/1.1.1.1:1030} on input <57239495e765a00aa151081e>.
2017-02-17_20:31:14.33021 2017-02-17 12:31:14,329 ERROR: 
org.graylog2.shared.buffers.processors.DecodingProcessor - Error processing 
message RawMessage{id=e08a52e1-f54c-11e6-9231-005056a7396f, 
journalOffset=9857804159, codec=syslog, payloadSize=156, 
timestamp=2017-02-17T20:08:41.486Z, remoteAddress=/10.4.1.110:1030}
2017-02-17_20:31:14.33105 java.lang.IllegalArgumentException: Invalid 
format: "19293274:" is malformed at ":"
2017-02-17_20:31:14.33584   at 
org.joda.time.format.DateTimeFormatter.parseDateTime(DateTimeFormatter.java:945)
 
~[graylog.jar:?]
2017-02-17_20:31:14.33727   at 
org.joda.time.DateTime.parse(DateTime.java:160) ~[graylog.jar:?]
2017-02-17_20:31:14.33762   at 
org.joda.time.DateTime.parse(DateTime.java:149) ~[graylog.jar:?]
2017-02-17_20:31:14.33811   at 
org.graylog2.syslog4j.server.impl.event.SyslogServerEvent.parseDate(SyslogServerEvent.java:108)
 
~[graylog.jar:?]
2017-02-17_20:31:14.33955   at 
org.graylog2.syslog4j.server.impl.event.SyslogServerEvent.parsePriority(SyslogServerEvent.java:136)
 
~[graylog.jar:?]
2017-02-17_20:31:14.34209   at 
org.graylog2.syslog4j.server.impl.event.SyslogServerEvent.parse(SyslogServerEvent.java:152)
 
~[graylog.jar:?]
2017-02-17_20:31:14.34211   at 
org.graylog2.syslog4j.server.impl.event.SyslogServerEvent.(SyslogServerEvent.java:50)
 
~[graylog.jar:?]
2017-02-17_20:31:14.34212   at 
org.graylog2.inputs.codecs.SyslogCodec.parse(SyslogCodec.java:123) 
~[graylog.jar:?]
2017-02-17_20:31:14.34398   at 
org.graylog2.inputs.codecs.SyslogCodec.decode(SyslogCodec.java:91) 
~[graylog.jar:?]
2017-02-17_20:31:14.34595   at 
org.graylog2.shared.buffers.processors.DecodingProcessor.processMessage(DecodingProcessor.java:146)
 
~[graylog.jar:?]
2017-02-17_20:31:14.34625   at 
org.graylog2.shared.buffers.processors.DecodingProcessor.onEvent(DecodingProcessor.java:87)
 
[graylog.jar:?]
2017-02-17_20:31:14.34929   at 
org.graylog2.shared.buffers.processors.ProcessBufferProcessor.onEvent(ProcessBufferProcessor.java:79)
 
[graylog.jar:?]
2017-02-17_20:31:14.34963   at 
org.graylog2.shared.buffers.processors.ProcessBufferProcessor.onEvent(ProcessBufferProcessor.java:45)
 
[graylog.jar:?]
2017-02-17_20:31:14.35012   at 
com.lmax.disruptor.WorkProcessor.run(WorkProcessor.java:143) [graylog.jar:?]
2017-02-17_20:31:14.35134   at 
com.codahale.metrics.InstrumentedThreadFactory$InstrumentedRunnable.run(InstrumentedThreadFactory.java:66)
 
[graylog.jar:?]
2017-02-17_20:31:14.35179   at java.lang.Thread.run(Thread.java:745) 
[?:1.8.0_101]

/elasticsearch/current

17_20:33:58.10920 [2017-02-17 12:33:57,437][DEBUG][action.bulk 
 ] [Morg] [graylog_447][2] failed to execute bulk item (index) index 
{[graylog_deflector][message][79384092-f54f-11e6-969d-005056a71aa5], 
source[{"RepeatCount":"1","EventDate":"2017/02/17","gl2_remote_ip":"2.3.60.12","gl2_remote_port":43149,"IngressInterface":"ethernet1/3","source":"source","gl2_source_input":"57eafbb1e765a0322da6254e","DestinationPort":"161","Bytes":187,"SessionEndReason":"aged-out","SourceZone":"untrust","PktsSent":1,"YEAR":"2017","gl2_source_node":"33a8a3ac-4bd2-4295-889b-eea9ced9c321","MINUTE":"55","NATSourceIP":"0.0.0.0","DestinationLocation":"10.0.0.0-10.255.255.255","NATDestinationPort":"0","PktsReceived":1,"RuleName":"GL-VW-Rule-Inbound","MONTHNUM":"02","level":6,"ConfigVersion":"1","IPV4":"0.0.0.0","streams":["0001"],"Sequence":"6216921628","LogForwardingProfile":"Log
 
Profile","SerialNumber":"001801032530","EventTime":"11:55:25","LoggedTime":"11:55:25","BytesSent":93,"ActionFlags":"0x0","DestinationZone":"trust","Domain":"1","Application":"snmpv2","SessionID":"177745","Subtype":"end","MONTHDAY":"17","NATSourcePort":"0","SourceLocation":"so

Re: [graylog2] Re: Multiline message problems

[Andrew Badera]

Multiline regex does not need to match entire message. We have Elastic
support, so I turned to them for Filebeat help. The only thing I was
missing was checking the negate checkbox (still not entirely sure what it
does). Now it works perfectly fine with all of our messages.

--ab


On Fri, Feb 17, 2017 at 5:44 AM, Jan Doberstein  wrote:

> Hej Andy,
>
> maybe you should separate the multiple messages you have by type into
> different log files to be able to have one pattern for every logfile.
>
> I didn’t dig into NXLog that deep but again - someone in the NXLog
> community might help with that.
>
> /jd
>
> From: Andrew Badera  
> Reply: graylog2@googlegroups.com 
> 
> Date: 17. Februar 2017 at 11:58:37
> To: graylog2@googlegroups.com 
> 
> Subject:  Re: [graylog2] Re: Multiline message problems
>
> Hi Jan,
>
> Thanks for the reply.
>
> Before I share our million different log messages, can we discuss on the
> basis that a single regex won't capture our messages? We have multiline
> exceptions, multiline SQL statements, multiline various other types of
> messages. If NXLog multiline handling is stronger, is there anything I may
> have missed in terms of NXLog setup? Are there other alternatives (other
> than decorating our messages) I haven't considered, or obviously missed?
>
> Thanks-
> --ab
>
>
> On Fri, Feb 17, 2017 at 2:49 AM, Jan Doberstein  wrote:
>
>> Hej Andy,
>>
>> if you want help with the multiline detection of filebeat, we would need
>> to have some information about your logfile. examples welcome.
>>
>> with your question about nxlog the limit for one message is reached - you
>> would need to configure this limit. But for this the NXLog Community might
>> be the best place to ask.
>>
>> regards
>> Jan
>>
>> On Thursday, February 16, 2017 at 11:16:55 PM UTC+1, Andy Badera wrote:
>>>
>>> Hello all-
>>>
>>> Windows app server into Graylog 2.1.0.
>>>
>>> Like many, we have multiline log messages. There is presently no clearly
>>> defined syntax around these messages, no end delimiter.
>>>
>>> I'm able to flow messages in using filebeat, but I can't capture
>>> multiline messages properly. I believe per a Graylog blog entry, I need a
>>> regex that matches the entire message. I don't think this is feasible with
>>> our widely-varied messages. We do have a well-defined phrase that starts
>>> every message, but I'm not sure how I would define the end of and capture
>>> the varied messages.
>>>
>>> I've tried NXLog outputting to the system input of GELF TCP. I suspect
>>> NXLog has better multiline handling, but I can't flow messages reliably
>>> using NXLog - I get shut down repeatedly by the string size limit error in
>>> nxlog.log:
>>>
>>> 2017-02-16 17:13:06 INFO connecting to 10.100.15.196:12201
>>> 2017-02-16 17:13:06 INFO reconnecting in 1 seconds
>>> 2017-02-16 17:13:06 ERROR oversized string, limit is 1048576 bytes
>>>
>>> Is there any way for me to correct this string size limit issue using
>>> NXLog CE?
>>>
>>> Any other alternatives I'm not considering? Anything I'm doing obviously
>>> wrong, or missed?
>>>
>>> Thanks in advance!
>>> --ab
>>>
>>> --
>> You received this message because you are subscribed to a topic in the
>> Google Groups "Graylog Users" group.
>> To unsubscribe from this topic, visit https://groups.google.com/d/to
>> pic/graylog2/hhVs0N5d9tQ/unsubscribe.
>> To unsubscribe from this group and all its topics, send an email to
>> graylog2+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit https://groups.google.com/d/ms
>> gid/graylog2/84085e67-c94c-4a41-a045-164452b77be7%40googlegroups.com
>> .
>>
>>
>> For more options, visit https://groups.google.com/d/optout.
>>
>
> --
> You received this message because you are subscribed to a topic in the
> Google Groups "Graylog Users" group.
> To unsubscribe from this topic, visit https://groups.google.com/d/
> topic/graylog2/hhVs0N5d9tQ/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> graylog2+unsubscr...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/
> msgid/graylog2/CAAD%3DdiqqeCrJhmuDkEcNXOjwsNUeYOWs
> 7OVzE3hagLLxH8MCLA%40mail.gmail.com
> 
> .
> For more options, visit https://groups.google.com/d/optout.
>
> —
> Jan Doberstein
> Support Engineer
>
> Phone:  +49 40 609452029
> Fax:  +49 40 609452030 <+49%2040%20609452030>
>
> TORCH GmbH - A Graylog company 
> Poolstraße 21
> 20355  Hamburg, Germany
>
> Commercial Reg. (Registergericht): Amtsgericht Hamburg, HRB 125175
> Geschäftsführer: Lennart Koopmann (CEO)
>
> --
> You received this message because you are subscribed to a topic in the
> Google Groups "Graylog Users" group

Re: [graylog2] Re: Multiline message problems

[Jan Doberstein]

Hej Andy,

maybe you should separate the multiple messages you have by type into different 
log files to be able to have one pattern for every logfile.

I didn’t dig into NXLog that deep but again - someone in the NXLog community 
might help with that. 

/jd

From: Andrew Badera 
Reply: graylog2@googlegroups.com 
Date: 17. Februar 2017 at 11:58:37
To: graylog2@googlegroups.com 
Subject:  Re: [graylog2] Re: Multiline message problems  

Hi Jan,

Thanks for the reply.

Before I share our million different log messages, can we discuss on the basis 
that a single regex won't capture our messages? We have multiline exceptions, 
multiline SQL statements, multiline various other types of messages. If NXLog 
multiline handling is stronger, is there anything I may have missed in terms of 
NXLog setup? Are there other alternatives (other than decorating our messages) 
I haven't considered, or obviously missed?

Thanks-
--ab


On Fri, Feb 17, 2017 at 2:49 AM, Jan Doberstein  wrote:
Hej Andy,

if you want help with the multiline detection of filebeat, we would need to 
have some information about your logfile. examples welcome.

with your question about nxlog the limit for one message is reached - you would 
need to configure this limit. But for this the NXLog Community might be the 
best place to ask.

regards
Jan

On Thursday, February 16, 2017 at 11:16:55 PM UTC+1, Andy Badera wrote:
Hello all-

Windows app server into Graylog 2.1.0.

Like many, we have multiline log messages. There is presently no clearly 
defined syntax around these messages, no end delimiter.

I'm able to flow messages in using filebeat, but I can't capture multiline 
messages properly. I believe per a Graylog blog entry, I need a regex that 
matches the entire message. I don't think this is feasible with our 
widely-varied messages. We do have a well-defined phrase that starts every 
message, but I'm not sure how I would define the end of and capture the varied 
messages.

I've tried NXLog outputting to the system input of GELF TCP. I suspect NXLog 
has better multiline handling, but I can't flow messages reliably using NXLog - 
I get shut down repeatedly by the string size limit error in nxlog.log:

2017-02-16 17:13:06 INFO connecting to 10.100.15.196:12201
2017-02-16 17:13:06 INFO reconnecting in 1 seconds
2017-02-16 17:13:06 ERROR oversized string, limit is 1048576 bytes

Is there any way for me to correct this string size limit issue using NXLog CE?

Any other alternatives I'm not considering? Anything I'm doing obviously wrong, 
or missed?

Thanks in advance!
--ab

--
You received this message because you are subscribed to a topic in the Google 
Groups "Graylog Users" group.
To unsubscribe from this topic, visit 
https://groups.google.com/d/topic/graylog2/hhVs0N5d9tQ/unsubscribe.
To unsubscribe from this group and all its topics, send an email to 
graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/graylog2/84085e67-c94c-4a41-a045-164452b77be7%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to a topic in the Google 
Groups "Graylog Users" group.
To unsubscribe from this topic, visit 
https://groups.google.com/d/topic/graylog2/hhVs0N5d9tQ/unsubscribe.
To unsubscribe from this group and all its topics, send an email to 
graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/graylog2/CAAD%3DdiqqeCrJhmuDkEcNXOjwsNUeYOWs7OVzE3hagLLxH8MCLA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
— 
Jan Doberstein
Support Engineer

Phone:  +49 40 609452029
Fax:  +49 40 609452030

TORCH GmbH - A Graylog company 
Poolstraße 21
20355  Hamburg, Germany 

Commercial Reg. (Registergericht): Amtsgericht Hamburg, HRB 125175
Geschäftsführer: Lennart Koopmann (CEO)

-- 
You received this message because you are subscribed to the Google Groups 
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/graylog2/etPan.58a6e241.3de586f9.ad4%40graylog.com.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: Message signed with OpenPGP using AMPGpg

Re: [graylog2] Re: Multiline message problems

[Andrew Badera]

Hi Jan,

Thanks for the reply.

Before I share our million different log messages, can we discuss on the
basis that a single regex won't capture our messages? We have multiline
exceptions, multiline SQL statements, multiline various other types of
messages. If NXLog multiline handling is stronger, is there anything I may
have missed in terms of NXLog setup? Are there other alternatives (other
than decorating our messages) I haven't considered, or obviously missed?

Thanks-
--ab


On Fri, Feb 17, 2017 at 2:49 AM, Jan Doberstein  wrote:

> Hej Andy,
>
> if you want help with the multiline detection of filebeat, we would need
> to have some information about your logfile. examples welcome.
>
> with your question about nxlog the limit for one message is reached - you
> would need to configure this limit. But for this the NXLog Community might
> be the best place to ask.
>
> regards
> Jan
>
> On Thursday, February 16, 2017 at 11:16:55 PM UTC+1, Andy Badera wrote:
>>
>> Hello all-
>>
>> Windows app server into Graylog 2.1.0.
>>
>> Like many, we have multiline log messages. There is presently no clearly
>> defined syntax around these messages, no end delimiter.
>>
>> I'm able to flow messages in using filebeat, but I can't capture
>> multiline messages properly. I believe per a Graylog blog entry, I need a
>> regex that matches the entire message. I don't think this is feasible with
>> our widely-varied messages. We do have a well-defined phrase that starts
>> every message, but I'm not sure how I would define the end of and capture
>> the varied messages.
>>
>> I've tried NXLog outputting to the system input of GELF TCP. I suspect
>> NXLog has better multiline handling, but I can't flow messages reliably
>> using NXLog - I get shut down repeatedly by the string size limit error in
>> nxlog.log:
>>
>> 2017-02-16 17:13:06 INFO connecting to 10.100.15.196:12201
>> 2017-02-16 17:13:06 INFO reconnecting in 1 seconds
>> 2017-02-16 17:13:06 ERROR oversized string, limit is 1048576 bytes
>>
>> Is there any way for me to correct this string size limit issue using
>> NXLog CE?
>>
>> Any other alternatives I'm not considering? Anything I'm doing obviously
>> wrong, or missed?
>>
>> Thanks in advance!
>> --ab
>>
>> --
> You received this message because you are subscribed to a topic in the
> Google Groups "Graylog Users" group.
> To unsubscribe from this topic, visit https://groups.google.com/d/
> topic/graylog2/hhVs0N5d9tQ/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> graylog2+unsubscr...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/
> msgid/graylog2/84085e67-c94c-4a41-a045-164452b77be7%40googlegroups.com
> 
> .
>
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/graylog2/CAAD%3DdiqqeCrJhmuDkEcNXOjwsNUeYOWs7OVzE3hagLLxH8MCLA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

[graylog2] Re: Troubleshooting logs

[Jochen Schalanda]

Hi Tom,

On Friday, 17 February 2017 00:41:03 UTC+1, Tom Powers wrote:
>
> I've found this article on the right place to put the certs...but not sure 
> what format or how to get them out of the master server
>
>
> http://docs.graylog.org/en/2.0/pages/faq.html#i-have-configured-an-smtp-server-or-an-output-with-tls-connection-and-receive-handshake-errors-what-should-i-do
>

See 
http://docs.graylog.org/en/2.2/pages/configuration/https.html#adding-a-self-signed-certificate-to-the-jvm-trust-store
 
for instructions for how to add certificates to the JVM trust store.

Cheers,
Jochen 

-- 
You received this message because you are subscribed to the Google Groups 
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/graylog2/2134480e-bd5c-46b2-8521-3c93d1b3ac90%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[graylog2] Re: How to upgrade Graylog 2.1 > 2.2 ?

[Henri Volotinen]

Hey!

Okay, thank you! This is extremely valuable information.

Br,
Henri

On Friday, 17 February 2017 10:52:05 UTC+2, Jan Doberstein wrote:
>
> Hej Henri,
>
> this will be the best method for upgrading your environment.
>
> with kind regards
> Jan
>
> On Thursday, February 16, 2017 at 7:18:24 PM UTC+1, Henri Volotinen wrote:
>>
>> Hi,
>>
>> So rolling upgrade is not supported? Good to know, because I was going to 
>> upgrade our production setup (with 3 graylog-server nodes version 2.1.3) 
>> using the rolling upgrade method.
>>
>> So basically the upgrade steps are in my scenario are:
>> 1) Shutdown all (three) graylog-server nodes
>> 2) Upgrade all (three) graylog-server nodes to version 2.2.0
>> 3) Start the master node and wait for it to do some indexing magic to the 
>> Elasticsearch cluster until it fully starts up
>> 4) Start the other two non-master nodes
>>
>> Is this the correct way to do the upgrade?
>>
>> Thanks!
>>
>> Br,
>> Henri
>>
>> On Thursday, 16 February 2017 12:08:36 UTC+2, Jochen Schalanda wrote:
>>>
>>> Hi,
>>>
>>> On Thursday, 16 February 2017 10:34:07 UTC+1, jtkarvo wrote:

 Is is possible to do a rolling upgrade to a graylog cluster (from 2.1 
 to 2.2)?  If so, should I upgrade master first or non-master nodes first?

>>>
>>> Due to some changes in the index management it's not possible to do a 
>>> rolling upgrade from Graylog 2.x to Graylog 2.2.0.
>>>
>>> You should upgrade and start the master node first, then the upgrade and 
>>> start the secondary nodes.
>>>
>>> Cheers,
>>> Jochen
>>>
>>

-- 
You received this message because you are subscribed to the Google Groups 
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/graylog2/6681fa6c-3665-409e-850e-adc1d8402ccd%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[graylog2] Re: How to upgrade Graylog 2.1 > 2.2 ?

[Jan Doberstein]

Hej Henri,

this will be the best method for upgrading your environment.

with kind regards
Jan

On Thursday, February 16, 2017 at 7:18:24 PM UTC+1, Henri Volotinen wrote:
>
> Hi,
>
> So rolling upgrade is not supported? Good to know, because I was going to 
> upgrade our production setup (with 3 graylog-server nodes version 2.1.3) 
> using the rolling upgrade method.
>
> So basically the upgrade steps are in my scenario are:
> 1) Shutdown all (three) graylog-server nodes
> 2) Upgrade all (three) graylog-server nodes to version 2.2.0
> 3) Start the master node and wait for it to do some indexing magic to the 
> Elasticsearch cluster until it fully starts up
> 4) Start the other two non-master nodes
>
> Is this the correct way to do the upgrade?
>
> Thanks!
>
> Br,
> Henri
>
> On Thursday, 16 February 2017 12:08:36 UTC+2, Jochen Schalanda wrote:
>>
>> Hi,
>>
>> On Thursday, 16 February 2017 10:34:07 UTC+1, jtkarvo wrote:
>>>
>>> Is is possible to do a rolling upgrade to a graylog cluster (from 2.1 to 
>>> 2.2)?  If so, should I upgrade master first or non-master nodes first?
>>>
>>
>> Due to some changes in the index management it's not possible to do a 
>> rolling upgrade from Graylog 2.x to Graylog 2.2.0.
>>
>> You should upgrade and start the master node first, then the upgrade and 
>> start the secondary nodes.
>>
>> Cheers,
>> Jochen
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/graylog2/e72f951e-48ff-41e4-ac20-0f1049cf10fb%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[graylog2] Re: Multiline message problems

[Jan Doberstein]

Hej Andy,

if you want help with the multiline detection of filebeat, we would need to 
have some information about your logfile. examples welcome.

with your question about nxlog the limit for one message is reached - you 
would need to configure this limit. But for this the NXLog Community might 
be the best place to ask.

regards
Jan

On Thursday, February 16, 2017 at 11:16:55 PM UTC+1, Andy Badera wrote:
>
> Hello all-
>
> Windows app server into Graylog 2.1.0.
>
> Like many, we have multiline log messages. There is presently no clearly 
> defined syntax around these messages, no end delimiter.
>
> I'm able to flow messages in using filebeat, but I can't capture multiline 
> messages properly. I believe per a Graylog blog entry, I need a regex that 
> matches the entire message. I don't think this is feasible with our 
> widely-varied messages. We do have a well-defined phrase that starts every 
> message, but I'm not sure how I would define the end of and capture the 
> varied messages.
>
> I've tried NXLog outputting to the system input of GELF TCP. I suspect 
> NXLog has better multiline handling, but I can't flow messages reliably 
> using NXLog - I get shut down repeatedly by the string size limit error in 
> nxlog.log:
>
> 2017-02-16 17:13:06 INFO connecting to 10.100.15.196:12201
> 2017-02-16 17:13:06 INFO reconnecting in 1 seconds
> 2017-02-16 17:13:06 ERROR oversized string, limit is 1048576 bytes
>
> Is there any way for me to correct this string size limit issue using 
> NXLog CE?
>
> Any other alternatives I'm not considering? Anything I'm doing obviously 
> wrong, or missed?
>
> Thanks in advance!
> --ab
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/graylog2/84085e67-c94c-4a41-a045-164452b77be7%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.