Hello all,
I have some exceptions from elastic search regarding messages sent by
rsyslog.
Template in rsyslog used for elasticsearch:
template(name="es_template"
type="list"
option.json="on")
{
constant(value="{")
constant(value="\"@timestamp\":\"")
property(name="timereported" dateFormat="rfc3339")
constant(value="\",\"timereported\":\"")
property(name="timereported" dateFormat="rfc3339")
constant(value="\",\"timegenerated\":\"")
property(name="timegenerated" dateFormat="rfc3339")
constant(value="\",\"message\":\"") property(name="msg")
constant(value="\",\"host\":\"")
property(name="hostname")
constant(value="\",\"severity\":\"")
property(name="syslogseverity-text")
constant(value="\",\"priority\":\"")
property(name="syslogpriority-text")
constant(value="\",\"facility\":\"")
property(name="syslogfacility-text")
constant(value="\",\"tag\":\"")
property(name="syslogtag")
constant(value="\",\"program_name\":\"")
property(name="programname")
constant(value="\"}")
}
Json sent by rsyslog:
{
"@timestamp": "2014-07-23T15:08:37.262843+03:00",
"timereported": "2014-07-23T15:08:37.262843+03:00",
"timegenerated": "2014-07-23T15:08:37.262843+03:00",
"message": " File
\"/opt/optymyze/collectd/scripts/python/web_plugin.py\", line 63, in
read#012 metrics.type_instance = re.sub(\"\ \",\"-\",transactions[j]) +
\"_response_time\"",
"host": "is-iasi-vm.synygy.net",
"severity": "err",
"priority": "err",
"facility": "daemon",
"tag": "collectd[29370]:",
"program_name": "collectd"
}
Elastic search complains about space character being escaped. This happens
here:
re.sub(\"\ \",\"-\"
Exception log:
[2014-07-23 15:08:37,273][DEBUG][action.bulk ]
[v-so-repo-02-es-01] [default-index][1] failed to execute bulk item (index)
index {[default-index][collectd][knQxDu2MR3eYgSiZc6TmLQ],
source[{"@timestamp":"2014-07-23T15:08:37.262843+03:00","timereported":"2014-07-23T15:08:37.262843+03:00","timegenerated":"2014-07-23T15:08:37.262843+03:00","message":"
File \"/opt/optymyze/collectd/scripts/python/web_plugin.py\", line 63, in
read#012 metrics.type_instance = re.sub(\"\ \",\"-\",transactions[j]) +
\"_response_time\"","host":"is-iasi-vm.synygy.net
","severity":"err","priority":"err","facility":"daemon","tag":"collectd[29370]:","program_name":"collectd"}]}
org.elasticsearch.index.mapper.MapperParsingException: failed to parse
[message]
at
org.elasticsearch.index.mapper.core.AbstractFieldMapper.parse(AbstractFieldMapper.java:417)
at
org.elasticsearch.index.mapper.object.ObjectMapper.serializeValue(ObjectMapper.java:637)
at
org.elasticsearch.index.mapper.object.ObjectMapper.parse(ObjectMapper.java:490)
at
org.elasticsearch.index.mapper.DocumentMapper.parse(DocumentMapper.java:515)
at
org.elasticsearch.index.mapper.DocumentMapper.parse(DocumentMapper.java:462)
at
org.elasticsearch.index.shard.service.InternalIndexShard.prepareCreate(InternalIndexShard.java:373)
at
org.elasticsearch.action.bulk.TransportShardBulkAction.shardIndexOperation(TransportShardBulkAction.java:425)
at
org.elasticsearch.action.bulk.TransportShardBulkAction.shardOperationOnPrimary(TransportShardBulkAction.java:158)
at
org.elasticsearch.action.support.replication.TransportShardReplicationOperationAction$AsyncShardOperationAction.performOnPrimary(TransportShardReplicationOperationAction.java:534)
at
org.elasticsearch.action.support.replication.TransportShardReplicationOperationAction$AsyncShardOperationAction$1.run(TransportShardReplicationOperationAction.java:433)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)
Caused by: org.elasticsearch.common.jackson.core.JsonParseException:
Unrecognized character escape ' ' (code 32)
at [Source: [B@5f263984; line: 1, column: 286]
at
org.elasticsearch.common.jackson.core.JsonParser._constructError(JsonParser.java:1524)
at
org.elasticsearch.common.jackson.core.base.ParserMinimalBase._reportError(ParserMinimalBase.java:557)
at
org.elasticsearch.common.jackson.core.base.ParserMinimalBase._handleUnrecognizedCharacterEscape(ParserMinimalBase.java:532)
at
org.elasticsearch.common.jackson.core.json.UTF8StreamJsonParser._decodeEscaped(UTF8StreamJsonParser.java:2817)
at
org.elasticsearch.common.jackson.core.json.UTF8StreamJsonParser._finishString2(UTF8StreamJsonParser.java:2193)
at
org.elasticsearch.common.jackson.core.json.UTF8StreamJsonParser._finishString(UTF8StreamJsonParser.java:2149)
at
org.elasticsearch.common.jackson.core.json.UTF8StreamJsonParser.getText(UTF8StreamJsonParser.java:281)
at
org.elasticsearch.common.xcontent.json.JsonXContentParser.text(JsonXContentParser.java:85)
at
org.elasticsearch.common.xcontent.support.AbstractXContentParser.textOrNull(AbstractXContentParser.java:194)
at
org.elasticsearch.index.mapper.core.StringFieldMapper.parseCreateFieldForString(StringFieldMapper.java:338)
at
org.elasticsearch.index.mapper.core.StringFieldMapper.parseCreateField(StringFieldMapper.java:278)
at
org.elasticsearch.index.mapper.core.AbstractFieldMapper.parse(AbstractFieldMapper.java:407)
... 12 more
Should I open a bug for this, or is something wrong on my side?
Best regards,
Cristian Falcas
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
