Hi, Using ELK ES 1.5.0 LS 1.5.0rc2 Kibana 4.0.1
I have read about similiar issues but wasn't really sure what the proper way to fix this is. I basically have a bunch of log files written out in nested JSON. The logs can be a combination of various keys:values, sometimes with the value being null (literally) as you can see in my logs excerpt below. 2 thing of note: if i wipe out all my indicies, and restart the ELK stack, everything works again. This only seems to happen after about 2 to 3 weeks of being online. Second point, this wasn't an issue with logstash 1.4.2 and the corresponding ES version. Any assistance would be awesome. Thanks! Tony [2015-04-21 21:19:09,286][DEBUG][action.bulk ] [elasticsearch03] [logstash-2015.04.21][19] failed to execute bulk item (index) index {[logstash-2015.04.21][requestAds][AUzd2VGHPBf0vCHmQV4j], source[{"country":"BR","region":"18","city":"Curitiba","latitude":null,"longitude":null,"device_language":"pt","browser_user_agent":"VungleDroid/3.3.0","is_sd_card_available":1,"device_make":"motorola","device_model":"XT1033","device_height":1184,"device_width":720,"os_version":"5.0.2","platform":"android","sound_enabled":false,"volume":0,"device_id":"5ec495a3-80cb-4682-a0a9-c66c2ca122ea","ifa":"5ec495a3-80cb-4682-a0a9-c66c2ca122ea","isu":"5ec495a3-80cb-4682-a0a9-c66c2ca122ea","user_age":null,"user_gender":null,"ip_address":"189.123.219.242","connection":"wifi","network_operator":"TIM","pub_app_id":"507686ae771615941001aca5","pub_app_bundle_id":"com.kiloo.subwaysurf","ad_app_id":null,"campaign_id":null,"creative_id":null,"event_id":null,"sleep":-1,"strategy":null,"expiry":null,"post_bundle":null,"video_url":null,"show_close":null,"show_close_incentivized":null,"video_height":null,"video_width":null,"call_to_action_url":null,"call_to_action_destination":null,"countdown":null,"delay":null,"error":"Cached ad is better","shouldStream":false,"message":"","is_test":false,"host":"ip-10-155-170-179","level":"info","timestamp":"2015-04-21 21:18:21.280","@version":"1","@timestamp":"2015-04-21T21:19:07.996Z","type":"requestAds"}]} org.elasticsearch.index.mapper.MapperParsingException: failed to parse [error] at org.elasticsearch.index.mapper.core.AbstractFieldMapper.parse(AbstractFieldMapper.java:410) at org.elasticsearch.index.mapper.object.ObjectMapper.serializeValue(ObjectMapper.java:706) at org.elasticsearch.index.mapper.object.ObjectMapper.parse(ObjectMapper.java:497) at org.elasticsearch.index.mapper.DocumentMapper.parse(DocumentMapper.java:544) at org.elasticsearch.index.mapper.DocumentMapper.parse(DocumentMapper.java:493) at org.elasticsearch.index.shard.IndexShard.prepareCreate(IndexShard.java:438) at org.elasticsearch.action.bulk.TransportShardBulkAction.shardIndexOperation(TransportShardBulkAction.java:432) at org.elasticsearch.action.bulk.TransportShardBulkAction.shardOperationOnPrimary(TransportShardBulkAction.java:149) at org.elasticsearch.action.support.replication.TransportShardReplicationOperationAction$AsyncShardOperationAction.performOnPrimary(TransportShardReplicationOperationAction.java:515) at org.elasticsearch.action.support.replication.TransportShardReplicationOperationAction$AsyncShardOperationAction$1.run(TransportShardReplicationOperationAction.java:422) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) Caused by: java.lang.NumberFormatException: For input string: "Cached ad is better" at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65) at java.lang.Long.parseLong(Long.java:441) at java.lang.Long.parseLong(Long.java:483) at org.elasticsearch.common.xcontent.support.AbstractXContentParser.longValue(AbstractXContentParser.java:145) at org.elasticsearch.index.mapper.core.LongFieldMapper.innerParseCreateField(LongFieldMapper.java:300) at org.elasticsearch.index.mapper.core.NumberFieldMapper.parseCreateField(NumberFieldMapper.java:236) at org.elasticsearch.index.mapper.core.AbstractFieldMapper.parse(AbstractFieldMapper.java:400) ... 12 more [2015-04-21 21:19:09,286][DEBUG][action.bulk ] [elasticsearch03] [logstash-2015.04.21][8] failed to execute bulk item (index) index {[logstash-2015.04.21][albatross][AUzd2VGOtXgWXEfZvOn3], source[{"log_type":"albatross_vast_error","error":"Cannot have empty VAST","uri":"http://api.adfalcon.com/AdRequest/GetAd/?R_SID=a0cdeb3e8f6f4c4aa25b04146e7eb25c&R_F=vast2&R_ADTYPE=v&R_V=api-all-2.2.0&R_SSID=BrainWars%3A%20Competitive%20brain%20training%20game%20Brain%20Wars&R_SSName=BrainWars%3A%20Competitive%20brain%20training%20game%20Brain%20Wars&R_SSUrl=https%3A%2F%2Fitunes.apple.com%2Fus%2Fapp%2Fbrainwars-competitive-brain%2Fid845044428%3Fmt%3D8%26uo%3D4&R_SSMID=jp.co.translimit.brainwars&R_CT=Games&D_UID_IDFA=13f2156e-e269-4d2b-8228-52bda8f0d426&D_UID_DNT=false&R_IP=62.61.173.102&D_UA=Mozilla%2F5.0%20(iPhone%3B%20CPU%20iPhone%20OS%208_3%20like%20Mac%20OS%20X)%20AppleWebKit%2F600.1.4%20(KHTML%2C%20like%20Gecko)%20Mobile%2F12F70&D_CON_TYPE=wifi&D_DO=portrait&D_SW=${screen_width}&D_SH=1136&D_DMDL=iPhone6%2C2&D_OS=iOS&D_OSV=8.3&R_VMinBR=250&R_VMinD=1&R_VMaxD=15&TS=2015-04-21T17%3A12%3A24%2B00%3A00","vast_json":{},"pub_app_id":"845044428","level":"error","message":"","timestamp":"2015-04-21T17:12:24.762Z","@version":"1","@timestamp":"2015-04-21T21:19:08.726Z","type":"albatross"}]} org.elasticsearch.index.mapper.MapperParsingException: object mapping for [albatross] tried to parse field [error] as object, but got EOF, has a concrete value been provided to it? at org.elasticsearch.index.mapper.object.ObjectMapper.parse(ObjectMapper.java:495) at org.elasticsearch.index.mapper.DocumentMapper.parse(DocumentMapper.java:544) at org.elasticsearch.index.mapper.DocumentMapper.parse(DocumentMapper.java:493) at org.elasticsearch.index.shard.IndexShard.prepareCreate(IndexShard.java:438) at org.elasticsearch.action.bulk.TransportShardBulkAction.shardIndexOperation(TransportShardBulkAction.java:432) at org.elasticsearch.action.bulk.TransportShardBulkAction.shardOperationOnPrimary(TransportShardBulkAction.java:149) at org.elasticsearch.action.support.replication.TransportShardReplicationOperationAction$AsyncShardOperationAction.performOnPrimary(TransportShardReplicationOperationAction.java:515) at org.elasticsearch.action.support.replication.TransportShardReplicationOperationAction$AsyncShardOperationAction$1.run(TransportShardReplicationOperationAction.java:422) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/20e9e82f-c269-41f7-995d-52aaf39a7171%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.