Re: ES crashes when parsing fails due to mapping failure
It's not crashing, it is just a log that says the document insert was rejected -- Itamar Syn-Hershko http://code972.com | @synhershko https://twitter.com/synhershko Freelance Developer Consultant Lucene.NET committer and PMC member On Thu, Feb 5, 2015 at 2:56 PM, as...@singular.net wrote: We have automatic mapping turned for our logstash indexes. Every now and then our system logs a record that has a wrong (out of the ordinary) field data type. For example, a field that's been automatically mapped to be a number occasionally is logged as a string. This causes ES to crash with the following stack trace: at org.elasticsearch.search.SearchService.parseSource(SearchService.java:681) at org.elasticsearch.search.SearchService.createContext(SearchService.java:537) at org.elasticsearch.search.SearchService.createAndPutContext(SearchService.java:509) at org.elasticsearch.search.SearchService.executeQueryPhase(SearchService.java:264) at org.elasticsearch.search.action.SearchServiceTransportAction$5.call(SearchServiceTransportAction.java:231) at org.elasticsearch.search.action.SearchServiceTransportAction$5.call(SearchServiceTransportAction.java:228) at org.elasticsearch.search.action.SearchServiceTransportAction$23.run(SearchServiceTransportAction.java:559) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) Caused by: java.lang.NumberFormatException: For input string: 1.0 at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65) at java.lang.Long.parseLong(Long.java:441) at java.lang.Long.parseLong(Long.java:483) at org.elasticsearch.index.mapper.core.NumberFieldMapper.parseLongValue(NumberFieldMapper.java:357) at org.elasticsearch.index.mapper.core.LongFieldMapper.termQuery(LongFieldMapper.java:185) at org.apache.lucene.queryparser.classic.MapperQueryParser.getFieldQuerySingle(MapperQueryParser.java:257) at org.apache.lucene.queryparser.classic.MapperQueryParser.getFieldQuery(MapperQueryParser.java:168) at org.apache.lucene.queryparser.classic.QueryParserBase.getFieldQuery(QueryParserBase.java:487) at org.apache.lucene.queryparser.classic.MapperQueryParser.getFieldQuery(MapperQueryParser.java:287) at org.apache.lucene.queryparser.classic.QueryParserBase.handleQuotedTerm(QueryParserBase.java:875) at org.apache.lucene.queryparser.classic.QueryParser.Term(QueryParser.java:464) at org.apache.lucene.queryparser.classic.QueryParser.Clause(QueryParser.java:259) at org.apache.lucene.queryparser.classic.QueryParser.Query(QueryParser.java:183) at org.apache.lucene.queryparser.classic.QueryParser.Clause(QueryParser.java:263) at org.apache.lucene.queryparser.classic.QueryParser.Query(QueryParser.java:183) at org.apache.lucene.queryparser.classic.QueryParser.TopLevelQuery(QueryParser.java:172) at org.apache.lucene.queryparser.classic.QueryParserBase.parse(QueryParserBase.java:123) at org.apache.lucene.queryparser.classic.MapperQueryParser.parse(MapperQueryParser.java:882) at org.elasticsearch.index.query.QueryStringQueryParser.parse(QueryStringQueryParser.java:223) at org.elasticsearch.index.query.QueryParseContext.parseInnerQuery(QueryParseContext.java:277) at org.elasticsearch.index.query.FQueryFilterParser.parse(FQueryFilterParser.java:66) at org.elasticsearch.index.query.QueryParseContext.executeFilterParser(QueryParseContext.java:343) at org.elasticsearch.index.query.QueryParseContext.parseInnerFilter(QueryParseContext.java:324) at org.elasticsearch.index.query.BoolFilterParser.parse(BoolFilterParser.java:92) at org.elasticsearch.index.query.QueryParseContext.executeFilterParser(QueryParseContext.java:343) at org.elasticsearch.index.query.QueryParseContext.parseInnerFilter(QueryParseContext.java:324) at org.elasticsearch.index.query.FilteredQueryParser.parse(FilteredQueryParser.java:74) at org.elasticsearch.index.query.QueryParseContext.parseInnerQuery(QueryParseContext.java:277) at org.elasticsearch.index.query.IndexQueryParserService.innerParse(IndexQueryParserService.java:382) at org.elasticsearch.index.query.IndexQueryParserService.parse(IndexQueryParserService.java:281) at org.elasticsearch.index.query.IndexQueryParserService.parse(IndexQueryParserService.java:276) at org.elasticsearch.search.query.QueryParseElement.parse(QueryParseElement.java:33) at org.elasticsearch.search.SearchService.parseSource(SearchService.java:665) Is there a way to tell ES not to crash when failing to parse a field? I realize I can override the mapping, and so on, but regardless I'm also interested in getting ES to run reliably without crashing on rare inputs. Thanks, Assaf -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to
ES crashes when parsing fails due to mapping failure
We have automatic mapping turned for our logstash indexes. Every now and then our system logs a record that has a wrong (out of the ordinary) field data type. For example, a field that's been automatically mapped to be a number occasionally is logged as a string. This causes ES to crash with the following stack trace: at org.elasticsearch.search.SearchService.parseSource(SearchService.java:681) at org.elasticsearch.search.SearchService.createContext(SearchService.java:537) at org.elasticsearch.search.SearchService.createAndPutContext(SearchService.java:509) at org.elasticsearch.search.SearchService.executeQueryPhase(SearchService.java:264) at org.elasticsearch.search.action.SearchServiceTransportAction$5.call(SearchServiceTransportAction.java:231) at org.elasticsearch.search.action.SearchServiceTransportAction$5.call(SearchServiceTransportAction.java:228) at org.elasticsearch.search.action.SearchServiceTransportAction$23.run(SearchServiceTransportAction.java:559) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) Caused by: java.lang.NumberFormatException: For input string: 1.0 at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65) at java.lang.Long.parseLong(Long.java:441) at java.lang.Long.parseLong(Long.java:483) at org.elasticsearch.index.mapper.core.NumberFieldMapper.parseLongValue(NumberFieldMapper.java:357) at org.elasticsearch.index.mapper.core.LongFieldMapper.termQuery(LongFieldMapper.java:185) at org.apache.lucene.queryparser.classic.MapperQueryParser.getFieldQuerySingle(MapperQueryParser.java:257) at org.apache.lucene.queryparser.classic.MapperQueryParser.getFieldQuery(MapperQueryParser.java:168) at org.apache.lucene.queryparser.classic.QueryParserBase.getFieldQuery(QueryParserBase.java:487) at org.apache.lucene.queryparser.classic.MapperQueryParser.getFieldQuery(MapperQueryParser.java:287) at org.apache.lucene.queryparser.classic.QueryParserBase.handleQuotedTerm(QueryParserBase.java:875) at org.apache.lucene.queryparser.classic.QueryParser.Term(QueryParser.java:464) at org.apache.lucene.queryparser.classic.QueryParser.Clause(QueryParser.java:259) at org.apache.lucene.queryparser.classic.QueryParser.Query(QueryParser.java:183) at org.apache.lucene.queryparser.classic.QueryParser.Clause(QueryParser.java:263) at org.apache.lucene.queryparser.classic.QueryParser.Query(QueryParser.java:183) at org.apache.lucene.queryparser.classic.QueryParser.TopLevelQuery(QueryParser.java:172) at org.apache.lucene.queryparser.classic.QueryParserBase.parse(QueryParserBase.java:123) at org.apache.lucene.queryparser.classic.MapperQueryParser.parse(MapperQueryParser.java:882) at org.elasticsearch.index.query.QueryStringQueryParser.parse(QueryStringQueryParser.java:223) at org.elasticsearch.index.query.QueryParseContext.parseInnerQuery(QueryParseContext.java:277) at org.elasticsearch.index.query.FQueryFilterParser.parse(FQueryFilterParser.java:66) at org.elasticsearch.index.query.QueryParseContext.executeFilterParser(QueryParseContext.java:343) at org.elasticsearch.index.query.QueryParseContext.parseInnerFilter(QueryParseContext.java:324) at org.elasticsearch.index.query.BoolFilterParser.parse(BoolFilterParser.java:92) at org.elasticsearch.index.query.QueryParseContext.executeFilterParser(QueryParseContext.java:343) at org.elasticsearch.index.query.QueryParseContext.parseInnerFilter(QueryParseContext.java:324) at org.elasticsearch.index.query.FilteredQueryParser.parse(FilteredQueryParser.java:74) at org.elasticsearch.index.query.QueryParseContext.parseInnerQuery(QueryParseContext.java:277) at org.elasticsearch.index.query.IndexQueryParserService.innerParse(IndexQueryParserService.java:382) at org.elasticsearch.index.query.IndexQueryParserService.parse(IndexQueryParserService.java:281) at org.elasticsearch.index.query.IndexQueryParserService.parse(IndexQueryParserService.java:276) at org.elasticsearch.search.query.QueryParseElement.parse(QueryParseElement.java:33) at org.elasticsearch.search.SearchService.parseSource(SearchService.java:665) Is there a way to tell ES not to crash when failing to parse a field? I realize I can override the mapping, and so on, but regardless I'm also interested in getting ES to run reliably without crashing on rare inputs. Thanks, Assaf -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/9b6febf3-5800-40a6-8cf4-fcb24ac8abed%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
