Hi Al,
you might want to try to use your pattern with lower case 'y' for the year
component of the date pattern.
Cheers,
Jochen
On Wednesday, 8 February 2017 21:09:19 UTC+1, Al Reynolds wrote:
>
> I've noticed another error. The timestamp field is being replaced
> correctly, but the "gl2_processing_error" field is showing the following
> error (on all messages):
> For rule 'WO-CS-RAS': In call to function 'parse_date' at 8:15 an
> exception was thrown: Invalid format: "2017-02-08 15:05:59,170" is
> malformed at "17-02-08 15:05:59,170"
>
>
> It doesn't seem to have any adverse effects, but I'm curious as to what
> might be causing it?
>
> On Wednesday, February 8, 2017 at 1:56:17 PM UTC-5, Al Reynolds wrote:
>>
>> Figured it out--parse_date needed the timestamp . New rule looks like
>> this:
>> rule "WO-CS-RAS"
>> when
>>
>> contains(to_string($message.file),"centralserver\\ras-server\\log\\ras_cs_")
>> then
>> set_field("WO_Log_Source","RAS-CS");
>> let matches = grok(pattern: "%{WO_CS_RAS_CS_MESSAGE}", value:
>> to_string($message.message));
>> set_fields(matches);
>> let date = parse_date(to_string($message.WO_Timestamp), "YYYY-MM-dd
>> HH:mm:ss,SSS", "EST");
>> set_field("timestamp", date);
>> route_to_stream("WideOrbit Logs");
>> end
>>
>> I was under the impression that the timezone was optional?
>>
>> Thanks for all your help with this Jochen--it's greatly appreciated!
>>
>> Cheers,
>> Al
>>
>> On Wednesday, February 8, 2017 at 11:05:22 AM UTC-5, Al Reynolds wrote:
>>>
>>> That's what I get for typing it out...thank you for catching that!
>>> Unfortunately, even after correcting for the incorrect milliseconds value,
>>> it's still not replacing timestamp value. I sent the parsed date to a new
>>> field (in this case, "log_timestamp") to verify that the output data was in
>>> the correct format, which it is now, but it still won't replace the
>>> timestamp field.
>>>
>>> Message sample with "log_timestamp" field:
>>> WO_CS_RAS_CS_MESSAGE
>>> 2017-02-08 11:00:34,980 WARN [Task 'ATLANTA-FS' FS timer.1]
>>> FriendshipTasksServiceImpl = Could not obtain task info for:
>>> 2c95ac8e-57e3-91b2-0158-495b880b24e8REQUEST FAILED ==> STATUS CODE: 404,
>>> RESPONSE BODY:
>>> WO_LogLevel
>>> WARN
>>> WO_Log_Source
>>> RAS-CS
>>> WO_Message
>>> Could not obtain task info for: 2c95ac8e-57e3-91b2-0158-
>>> 495b880b24e8REQUEST FAILED ==> STATUS CODE: 404, RESPONSE BODY:
>>> WO_Process
>>> Task 'ATLANTA-FS' FS timer.1
>>> WO_SubProcess
>>> FriendshipTasksServiceImpl
>>> WO_Timestamp
>>> 2017-02-08 11:00:34,980
>>> facility
>>> filebeat
>>> file
>>> d:\centralserver\ras-server\log\ras_cs_WO-ATL-CS.log
>>> input_type
>>> log
>>> log_timestamp
>>> 2017-02-08T11:00:34.980Z
>>> message
>>> 2017-02-08 11:00:34,980 WARN [Task 'ATLANTA-FS' FS timer.1]
>>> FriendshipTasksServiceImpl = Could not obtain task info for:
>>> 2c95ac8e-57e3-91b2-0158-495b880b24e8REQUEST FAILED ==> STATUS CODE: 404,
>>> RESPONSE BODY:
>>> name
>>> WO-ATL-CS
>>> offset
>>> 2372156
>>> source
>>> WO-ATL-CS
>>> timestamp
>>> 2017-02-08T16:00:35.864Z
>>> type
>>> log
>>>
>>> Corrected rule:
>>> rule "WO-CS-RAS"
>>> when
>>>
>>> contains(to_string($message.file),"centralserver\\ras-server\\log\\ras_cs_")
>>> then
>>> set_field("WO_Log_Source","RAS-CS");
>>> let matches = grok(pattern: "%{WO_CS_RAS_CS_MESSAGE}", value:
>>> to_string($message.message));
>>> set_fields(matches);
>>> let date = parse_date(to_string($message.WO_Timestamp), "YYYY-MM-dd
>>> HH:mm:ss,SSS");
>>> set_field("timestamp", date);
>>> route_to_stream("WideOrbit Logs");
>>> end
>>>
>>> Thanks!
>>>
>>> Cheers,
>>> Al
>>>
>>> On Wednesday, February 8, 2017 at 10:55:03 AM UTC-5, Jochen Schalanda
>>> wrote:
>>>>
>>>> Hi Al,
>>>>
>>>> On Wednesday, 8 February 2017 15:46:07 UTC+1, Al Reynolds wrote:
>>>>>
>>>>> WO_Timestamp
>>>>> 2017-02-08 09:42:30,056
>>>>>
>>>>> Those messages are with the date parsing disabled. I'm attempting to
>>>>> replace "timestamp" with the "WO_Timestamp" field.
>>>>>
>>>>
>>>> The string in WO_Timestamp doesn't match the pattern "YYYY-MM-dd
>>>> HH:mm:ss,sss" used in parse_date(). See
>>>> http://www.joda.org/joda-time/apidocs/org/joda/time/format/DateTimeFormat.html
>>>>
>>>> for details.
>>>>
>>>> Hint: 's' and 'S' are not the same thing.
>>>>
>>>>
>>>> Side note: The full_message field is empty on my filebeat inputs--is
>>>>> that expected behavior?
>>>>>
>>>>
>>>> Yes, that's expected.
>>>>
>>>> What would you expect to find in the (optional) full_message field?
>>>>
>>>> Cheers,
>>>> Jochen
>>>>
>>>
--
You received this message because you are subscribed to the Google Groups
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/graylog2/561bf48b-bfdc-4ee6-b0a0-f7e706dc1e5c%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
