Re: Issue when trying to load JSON

2019-04-25 Thread Otto Fowler
On April 25, 2019 at 12:05:45, Nick Allen (n...@nickallen.org) wrote: > Otto: I’m not sure this is an enveloped issue, or a new feature for the json map parser This is not an issue with JSONMapParser. This is an issue with the "enveloping" mechanism, prior to when the JSONMapParser gets the

Re: Issue when trying to load JSON

2019-04-25 Thread Nick Allen
> Stephane: How can I debug this? We created the PARSER* functions to help debug issues like this. Unfortunately, it does not work in this case because the bug is in the Enveloping, not in the Parser itself. This is gap in the PARSER_* functionality that I think we should address to make

RE: Issue when trying to load JSON

2019-04-25 Thread stephane.davy
y understanding is that using “ENVELOPE”, the parser expects to have some high level JSON, and a CSV in payload, this is why I wanted to do the same with JSON. But as far as I understand, it doesn’t seem to work, does it? Stéphane From: Otto Fowler [mailto:ottobackwa...@gmail.com] Sent: Thursday,

Re: Issue when trying to load JSON

2019-04-25 Thread Nick Allen
> Otto: I’m not sure this is an enveloped issue, or a new feature for the json map parser This is not an issue with JSONMapParser. This is an issue with the "enveloping" mechanism, prior to when the JSONMapParser gets the message. The entire message has been parsed as a JSON object including

Re: Issue when trying to load JSON

2019-04-25 Thread Otto Fowler
Also, our support for nested, unflattened json isn’t great to begin with. Stephane, can you state your use case? Do you want to get _source only to transform it? or do you want to use source as the message and discard the top level fields? other? On April 25, 2019 at 11:31:36, Otto Fowler

Re: Issue when trying to load JSON

2019-04-25 Thread Otto Fowler
I’m not sure about the name, I’m more thinking about the case. I’m not sure this is an enveloped issue, or a new feature for the json map parser ( or if you could do it with the jsonMap parser and JSONPath ) On April 25, 2019 at 11:23:25, Simon Elliston Ball ( si...@simonellistonball.com)

Re: Issue when trying to load JSON

2019-04-25 Thread Simon Elliston Ball
Seems like this would a good additional strategy, something like ENVELOPE_PARSED? Any thoughts on a good name? On Thu, 25 Apr 2019 at 16:20, Otto Fowler wrote: > So, the enveloped message doesn’t support getting an already parsed json > object from the enveloped json, we would have to do some

Re: Issue when trying to load JSON

2019-04-25 Thread Otto Fowler
Raw message in this case assumes that the raw message is a String embedded in the json field that you supply, not a nested json object, so it is looking for “_source” : “some other embedded string of some format like syslog in json” There are other message strategies, but I’m not sure they

Issue when trying to load JSON

2019-04-25 Thread stephane.davy
Hello, I'm trying to load some JSON data which has the following structure (this is a sample): { "_index": "indexing", "_type": "Event", "_id": "AWAkTAefYn0uCUpkHmCy", "_score": 1, "_source": { "dst": "127.0.0.1", "devTimeEpoch": "151243734", "dstPort": "0",