Hi Muhammad,
>From what I understand, rsyslog parsed all the JSON, so you can pass on all
this JSON with a template that looks like this:
template(name="allTheJson" type="list") {
property(name="$!all-json")
}
--
Performance Monitoring * Log Analytics * Search Analytics
Solr & Elasticsearch
2015-05-25 0:47 GMT+02:00 Thomas D. :
> Hi,
>
> I started looking into this but it seems like you already fixed it?
sorry, yes I don't know why, but this apparently fixed it:
https://github.com/rsyslog/rsyslog/commit/4725aa27e70ca6d8b09827aa747f9c61ee668363
>
> The branch is now merged and it is
Ah, I just realize the cee cookie is missing -- mmjsonparse actually
parses cee, not generic json. This can be done with the new version of
liblognorm (git master branch). Or you change the file to
@cee: {json here}
Rainer
2015-05-25 9:55 GMT+02:00 Radu Gheorghe :
> Hi Muhammad,
>
> From what I
Hi all.
Can we remove the blank space inside the json strings?
I like to use mmjsonparse and omelasticsearch plugins. But the json
strings contain too many blank space, so we need more net flow, and
elasticsearch would store the blank space bit in it's _source JSON which
means more disk siz
sample please. But I need to say that $!all-json is pretty fixed...
2015-05-25 11:31 GMT+02:00 chenlin rao :
> Hi all.
> Can we remove the blank space inside the json strings?
> I like to use mmjsonparse and omelasticsearch plugins. But the json
> strings contain too many blank space, so w
yeah, not only $!all-json.
For example. I have a logline:
```
{"content":"Traceroute Result of 'api.weibo.cn':\nDNS is
'202.103.224.68'\nIP is
'180.149.153.216'\n1|192:168:1:1|34.482ms\n2|219:159:136:1|27.005ms\n3|218:65:201:21|206.549ms\n4|202:103:236:53|116.733ms\n5|*\n6|*\n7|*\n8|180:149:128:54
On Mon, 25 May 2015, chenlin rao wrote:
yeah, not only $!all-json.
For example. I have a logline:
```
{"content":"Traceroute Result of 'api.weibo.cn':\nDNS is
'202.103.224.68'\nIP is
'180.149.153.216'\n1|192:168:1:1|34.482ms\n2|219:159:136:1|27.005ms\n3|218:65:201:21|206.549ms\n4|202:103:236:53
On Mon, 25 May 2015, Rainer Gerhards wrote:
Ah, I just realize the cee cookie is missing -- mmjsonparse actually
parses cee, not generic json. This can be done with the new version of
liblognorm (git master branch). Or you change the file to
@cee: {json here}
no space between the : and the {
No, ES store the raw JSON in _source field. We can set `"_size": {
"enabled": true }` to check the record size.
$ curl
10.19.0.97:9200/testindex/testtype/AU2OSfj0ZRvQT5qcC_l3?fields=_size,_source
{"_index":"testindex","_type":"testtype","_id":"AU2OSfj0ZRvQT5qcC_l3","_version":1,"found":true,"_sour
@cee:{can i give whole file name here to parse whole file.}
Here is what is sent in ES.
[image: Inline image 1]
I want separate all field. It is storing whole message in msg field.
On Tue, May 26, 2015 at 4:51 AM, David Lang wrote:
> On Mon, 25 May 2015, Rainer Gerhards wrote:
>
> Ah, I just
Ok, so what you are asking for is a JSON minify option to the property replacer.
Is there an option in ES to have it not store teh _source field? that would save
FAR more space.
although, when you are putting things in ES, you are not going for the most
space efficient storage in the first pl
the @cee: is needed at teh beginning of each line in the file. It has nothing to
do with the filename.
David Lang
___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rs
in other words
sed -i -e s/^/'@cee:'/ filename
and then have rsyslog try to read it.
David Lang
On Mon, 25 May 2015, David Lang wrote:
Date: Mon, 25 May 2015 23:00:16 -0700 (PDT)
From: David Lang
Reply-To: rsyslog-users
To: rsyslog-users
Subject: Re: [rsyslog] parse logs to elasticserach
t
have a look here: https://github.com/rsyslog/rsyslog/pull/306
2015-05-26 7:57 GMT+02:00 David Lang :
> Ok, so what you are asking for is a JSON minify option to the property
> replacer.
>
> Is there an option in ES to have it not store teh _source field? that would
> save FAR more space.
>
> altho
14 matches
Mail list logo
