I am having an intermittent problem with indexing some logstash syslog entries. It complains about an invalid date format. Oddly it has stopped working and then seemingly started working again on its own. I obviously may have inadvertently changed something but I have not been able to pin down what it is. The remote host is an rsyslog ubuntu machine, I get the errors on both stock application syslog entries and entries from our app.
Below is a sample of the error I am getting. [2014-03-13 12:15:55,641][DEBUG][action.bulk ] [Power Princess] [logstash-2014.03.13][3] failed to execute bulk item (index) index {[logstash-2014.03.13][syslog][fwm304NgSEu8FRkTsh2EwQ], source[{"message":"Lab Manager Error: Error in undeploying. Contact the Administrator if the problem persists. on ba-labmanager02.efi.internal when undeploying ~calculus3~617436.i18~sutirtha~watkins","@version":"1","@timestamp":"2014-03-13T19:15:39.000Z","type":"syslog","host":"calculus-daemons-2.efi.internal","priority":156,"timestamp":"Mar 13 12:15:39","logsource":"calculus-daemons-2","program":"vfi","pid":"15207","severity":4,"facility":19,"facility_label":"local3","severity_label":"Warning","tags":["Calculus","IDC"],"location":"IDC","automationserver":"labmanagerBA"}]} org.elasticsearch.index.mapper.MapperParsingException: failed to parse [timestamp] at org.elasticsearch.index.mapper.core.AbstractFieldMapper.parse(AbstractFieldMapper.java:418) at org.elasticsearch.index.mapper.object.ObjectMapper.serializeValue(ObjectMapper.java:616) at org.elasticsearch.index.mapper.object.ObjectMapper.parse(ObjectMapper.java:469) at org.elasticsearch.index.mapper.DocumentMapper.parse(DocumentMapper.java:515) at org.elasticsearch.index.mapper.DocumentMapper.parse(DocumentMapper.java:462) at org.elasticsearch.index.shard.service.InternalIndexShard.prepareCreate(InternalIndexShard.java:371) at org.elasticsearch.action.bulk.TransportShardBulkAction.shardIndexOperation(TransportShardBulkAction.java:400) at org.elasticsearch.action.bulk.TransportShardBulkAction.shardOperationOnPrimary(TransportShardBulkAction.java:153) at org.elasticsearch.action.support.replication.TransportShardReplicationOperationAction$AsyncShardOperationAction.performOnPrimary(TransportShardReplicationOperationAction.java:556) at org.elasticsearch.action.support.replication.TransportShardReplicationOperationAction$AsyncShardOperationAction$1.run(TransportShardReplicationOperationAction.java:426) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:679) *Caused by: org.elasticsearch.index.mapper.MapperParsingException: failed to parse date field [Mar 13 12:15:39], tried both date format [dateOptionalTime], and timestamp number with locale []* at org.elasticsearch.index.mapper.core.DateFieldMapper.parseStringValue(DateFieldMapper.java:582) at org.elasticsearch.index.mapper.core.DateFieldMapper.innerParseCreateField(DateFieldMapper.java:510) at org.elasticsearch.index.mapper.core.NumberFieldMapper.parseCreateField(NumberFieldMapper.java:215) at org.elasticsearch.index.mapper.core.AbstractFieldMapper.parse(AbstractFieldMapper.java:408) ... 12 more Caused by: java.lang.IllegalArgumentException: Invalid format: "Mar 13 12:15:39" at org.elasticsearch.common.joda.time.format.DateTimeFormatter.parseMillis(DateTimeFormatter.java:754) at org.elasticsearch.index.mapper.core.DateFieldMapper.parseStringValue(DateFieldMapper.java:576) ... 15 more [2014-03-13 12:15:56,115][DEBUG][action.bulk ] [Power Princess] [logstash-2014.03.13][2] failed to execute bulk item (index) index {[logstash-2014.03.13][syslog][Pyb-uSDER0Cj7nuBQ7101Q], source[{"message":"Deleting ~calculus3~617436.i18~sutirtha~watkins on ba-labmanager02.efi.internal","@version":"1","@timestamp":"2014-03-13T19:15:39.000Z","type":"syslog","host":"calculus-daemons-2.efi.internal","priority":156,"timestamp":"Mar 13 12:15:39","logsource":"calculus-daemons-2","program":"vfi","pid":"15207","severity":4,"facility":19,"facility_label":"local3","severity_label":"Warning","tags":["Calculus","IDC"],"location":"IDC","automationserver":"labmanagerBA"}]} org.elasticsearch.index.mapper.MapperParsingException: failed to parse [timestamp] at org.elasticsearch.index.mapper.core.AbstractFieldMapper.parse(AbstractFieldMapper.java:418) at org.elasticsearch.index.mapper.object.ObjectMapper.serializeValue(ObjectMapper.java:616) at org.elasticsearch.index.mapper.object.ObjectMapper.parse(ObjectMapper.java:469) at org.elasticsearch.index.mapper.DocumentMapper.parse(DocumentMapper.java:515) at org.elasticsearch.index.mapper.DocumentMapper.parse(DocumentMapper.java:462) at org.elasticsearch.index.shard.service.InternalIndexShard.prepareCreate(InternalIndexShard.java:371) at org.elasticsearch.action.bulk.TransportShardBulkAction.shardIndexOperation(TransportShardBulkAction.java:400) at org.elasticsearch.action.bulk.TransportShardBulkAction.shardOperationOnPrimary(TransportShardBulkAction.java:153) at org.elasticsearch.action.support.replication.TransportShardReplicationOperationAction$AsyncShardOperationAction.performOnPrimary(TransportShardReplicationOperationAction.java:556) at org.elasticsearch.action.support.replication.TransportShardReplicationOperationAction$AsyncShardOperationAction$1.run(TransportShardReplicationOperationAction.java:426) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:679) Caused by: org.elasticsearch.index.mapper.MapperParsingException: failed to parse date field [Mar 13 12:15:39], tried both date format [dateOptionalTime], and timestamp number with locale [] at org.elasticsearch.index.mapper.core.DateFieldMapper.parseStringValue(DateFieldMapper.java:582) at org.elasticsearch.index.mapper.core.DateFieldMapper.innerParseCreateField(DateFieldMapper.java:510) at org.elasticsearch.index.mapper.core.NumberFieldMapper.parseCreateField(NumberFieldMapper.java:215) at org.elasticsearch.index.mapper.core.AbstractFieldMapper.parse(AbstractFieldMapper.java:408) ... 12 more Caused by: java.lang.IllegalArgumentException: Invalid format: "Mar 13 12:15:39" at org.elasticsearch.common.joda.time.format.DateTimeFormatter.parseMillis(DateTimeFormatter.java:754) at org.elasticsearch.index.mapper.core.DateFieldMapper.parseStringValue(DateFieldMapper.java:576) ... 15 more [2014-03-13 12:15:57,313][DEBUG][action.bulk ] [Power Princess] [logstash-2014.03.13][2] failed to execute bulk item (index) index {[logstash-2014.03.13][syslog][rhBT8L_dSsmmvFNTdlv4Bg], source[{"message":"Lab Manager Error: Cannot delete this configuration because one or more virtual machines inside it are currently deployed. on ba-labmanager02.efi.internal when deleting ~calculus3~617436.i18~sutirtha~watkins","@version":"1","@timestamp":"2014-03-13T19:15:41.000Z","type":"syslog","host":"calculus-daemons-2.efi.internal","priority":156,"timestamp":"Mar 13 12:15:41","logsource":"calculus-daemons-2","program":"vfi","pid":"15207","severity":4,"facility":19,"facility_label":"local3","severity_label":"Warning","tags":["Calculus","IDC"],"location":"IDC","automationserver":"labmanagerBA"}]} org.elasticsearch.index.mapper.MapperParsingException: failed to parse [timestamp] at org.elasticsearch.index.mapper.core.AbstractFieldMapper.parse(AbstractFieldMapper.java:418) at org.elasticsearch.index.mapper.object.ObjectMapper.serializeValue(ObjectMapper.java:616) at org.elasticsearch.index.mapper.object.ObjectMapper.parse(ObjectMapper.java:469) at org.elasticsearch.index.mapper.DocumentMapper.parse(DocumentMapper.java:515) at org.elasticsearch.index.mapper.DocumentMapper.parse(DocumentMapper.java:462) at org.elasticsearch.index.shard.service.InternalIndexShard.prepareCreate(InternalIndexShard.java:371) at org.elasticsearch.action.bulk.TransportShardBulkAction.shardIndexOperation(TransportShardBulkAction.java:400) at org.elasticsearch.action.bulk.TransportShardBulkAction.shardOperationOnPrimary(TransportShardBulkAction.java:153) at org.elasticsearch.action.support.replication.TransportShardReplicationOperationAction$AsyncShardOperationAction.performOnPrimary(TransportShardReplicationOperationAction.java:556) at org.elasticsearch.action.support.replication.TransportShardReplicationOperationAction$AsyncShardOperationAction$1.run(TransportShardReplicationOperationAction.java:426) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:679) Caused by: org.elasticsearch.index.mapper.MapperParsingException: failed to parse date field [Mar 13 12:15:41], tried both date format [dateOptionalTime], and timestamp number with locale [] at org.elasticsearch.index.mapper.core.DateFieldMapper.parseStringValue(DateFieldMapper.java:582) at org.elasticsearch.index.mapper.core.DateFieldMapper.innerParseCreateField(DateFieldMapper.java:510) at org.elasticsearch.index.mapper.core.NumberFieldMapper.parseCreateField(NumberFieldMapper.java:215) at org.elasticsearch.index.mapper.core.AbstractFieldMapper.parse(AbstractFieldMapper.java:408) ... 12 more Caused by: java.lang.IllegalArgumentException: Invalid format: "Mar 13 12:15:41" at org.elasticsearch.common.joda.time.format.DateTimeFormatter.parseMillis(DateTimeFormatter.java:754) at org.elasticsearch.index.mapper.core.DateFieldMapper.parseStringValue(DateFieldMapper.java:576) ... 15 more -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/f23cb220-1a7a-4533-b59f-d69221e16312%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.