Hello Jochen,
I have nospecial extractors running. Maybe I can send you the real (not
anonymized) XML on a private way? To me it seems like it is a special
sequence in that XML that is causing this. I will try to reproduce it first
by myself.
The message that shows the shortened XML is still there.
I found this out because I am now tracking sizes of all messages sent to
Graylog to see big sized messages that should be shortened before sending
them. I investigated further and this one was a very large message but in
my framework internally shortened to 64kb (the full_message field) and ".."
appended. So the XML got cut. Nevertheless the message in graylog is
showing closing tags that were never sent and a full_message field of
~200byte. Very weired because I do no XML processing in the logging
framework as well just plain strings.
Thanks
Robby
Am Mittwoch, 19. Oktober 2016 12:41:16 UTC+2 schrieb Jochen Schalanda:
>
> Hi Robby,
>
> I cannot reproduce your problem.
>
> I've created a GELF UDP input on an installation of Graylog 2.1.1 and sent
> a GELF message to it with the XML example from your first email in the
> full_text message field.
>
> Everything has been ingested and is being shown as intended (i. e.
> unparsed and the verbatim XML). Maybe some extractor is interfering with
> your payload or even the client already sends the modified GELF message.
>
> Graylog and Elasticsearch itself do not parse XML from message fields by
> default.
>
> For reproduction, here's what I did:
>
> Create GELF UDP input with the following configuration and no extractors:
>
> bind_address: 127.0.0.1
> decompress_size_limit: 8388608
> override_source:
> port: 12201
> recv_buffer_size: 16384
>
>
> Send message to Graylog:
>
>
> $ echo '{"version":"1.1", "message":"XML TEST", "full_message":" version=\"1.0\" encoding=\"utf-8\"?>\n \n
> Success\n DoSomething\n
> 8\n \n 0\n \n
>
>
> Message in Graylog:
>
> $ curl -i -u admin:admin
> http://127.0.0.1:9000/api/messages/graylog2_27/43c3aca0-95e7-11e6-8fc7-6c4008b8fc28?pretty=true
> HTTP/1.1 200 OK
> X-Graylog-Node-ID: cd03ee44-b2a7-4824-be16-bb7456149dbd
> X-Runtime-Microseconds: 1887
> Content-Type: application/json
> Date: Wed, 19 Oct 2016 10:38:40 GMT
> Content-Length: 1975
>
> {
> "message" : {
> "fields" : {
> "full_message" : "\n
> \n Success\n DoSomething\n
>8\n \n 0\n \n
> "streams" : [ "552b92b2e4b0c055e41ffb8e", "0001"
> ],
> "gl2_source_node" : "cd03ee44-b2a7-4824-be16-bb7456149dbd",
> "_id" : "43c3aca0-95e7-11e6-8fc7-6c4008b8fc28",
> "source" : "unknown",
> "message" : "XML TEST",
> "gl2_source_input" : "5592913ec026f261b8d5ce9a",
> "timestamp" : "2016-10-19T10:31:58.824Z"
> },
> "streams" : [ ],
> "source_input_id" : null,
> "filter_out" : false,
> "journal_offset" : -9223372036854775808,
> "message" : "XML TEST",
> "id" : "43c3aca0-95e7-11e6-8fc7-6c4008b8fc28",
> "source" : "unknown",
> "field_names" : [ "full_message", "streams", "gl2_source_node", "_id",
> "source", "message", "gl2_source_input", "timestamp" ],
> "complete" : true,
> "timestamp" : "2016-10-19T10:31:58.824Z",
> "validation_errors" : "",
> "fields_entries" : [ {
> "full_message" : "\n
> \n Success\n DoSomething\n
>8\n \n 0\n \n
>}, {
> "streams" : [ "552b92b2e4b0c055e41ffb8e", "0001"
> ]
> }, {
> "gl2_source_node" : "cd03ee44-b2a7-4824-be16-bb7456149dbd"
> }, {
> "_id" : "43c3aca0-95e7-11e6-8fc7-6c4008b8fc28"
> }, {
> "source" : "unknown"
> }, {
> "message" : "XML TEST"
> }, {
> "gl2_source_input" : "5592913ec026f261b8d5ce9a"
> }, {
> "timestamp" : "2016-10-19T10:31:58.824Z"
> } ],
> "field_count" : 8,
> "stream_ids" : [ "552b92b2e4b0c055e41f