Hey,
is there a complete stack trace in the elasticsearch log files available
you could post here?
Also, your query is only spanning one index and one type? If not can you
provide other mappings? (I dont think this is an issue here, when you say
that optimizing it down to one segment makes it work again, just want to
exclude things). Is it possible that there is one type in your index, that
has a different id mapping? And that type has been deleted again? This
would explain why it works after an optimize...
--Alex
On Thu, Feb 13, 2014 at 1:03 AM, Benji Smith <be...@benjismith.net> wrote:
> Hello hello! I have a bizarre error I've been trying to debug for a few
> weeks with no luck, and I'm finally left to conclude that it may be a bug
> in ElasticSearch.
>
> Once every few days, I start seeing shard failures in my query results,
> like this:
>
> {
> "index": "my_index",
> "shard": 3,
> "status": 500,
> "reason":
> "RemoteTransportException[[HOSTNAME][inet[/10.0.123.123:9300]][search/phase/query]];
> nested: QueryPhaseExecutionException[[my_index][3]:
> query[ConstantScore(cache(_type:my_type))],from[0],size[10],sort[<custom:\"id\":
>
> org.elasticsearch.index.fielddata.fieldcomparator.LongValuesComparatorSource@97c2b4f>]:
> Query Failed [Failed to execute main query]]; nested:
> ElasticSearchException[java.lang.NumberFormatException: Invalid shift value
> (64) in prefixCoded bytes (is encoded value really an INT?)]; nested:
> UncheckedExecutionException[java.lang.NumberFormatException: Invalid shift
> value (64) in prefixCoded bytes (is encoded value really an INT?)]; nested:
> NumberFormatException[Invalid shift value (64) in prefixCoded bytes (is
> encoded value really an INT?)]; "
> }
>
>
> This query is operating against an index with about 100 different fields
> (including several different nested types), but the relevant portion of the
> mapping looks like this:
>
> {
> "my_type" : {
> "_id" : { "type" : "long", "path" : "id" },
> "properties" : {
> "id" : { "type" : "long" },
> /* ... LOTS OF OTHER FIELDS, INCLUDING MANY NESTED TYPES */
> }}
> }
>
> I've been able to isolate the shard failures to a minimal query of this
> form:
>
> {
> "query" : { "match_all" : { } },
> "sort" : [{
> "id" : { "order" : "asc" }
> }]
> }
>
> Basically, sorting by (numeric) ID causes shard failures when the shards
> sometimes mistakenly think that there are non-numeric values in the "id"
> field. I've audited the data, and it conforms with the schema. The id
> fields always contain valid LONG values.
>
> Whenever the shard failures occur, I can silence them for a few days by
> optimizing the index, like this:
>
> curl -XPOST '
> http://HOSTNAME:9200/my_index/_optimize?max_num_segments=1'
>
> And the shard failures will stop for a day or two, but inevitably, within
> a few days the failures will return and I'll have to optimize the index
> again. The weird thing is that the status URL always reports GREEN status
> and all shards healthy, even when these queries are failing on every
> request.
>
> I experienced these failures originally on 0.90.5, but I continued seeing
> the same problems after recently upgrading to 0.90.10. I even deleted the
> index and rebuilt from scratch under 0.90.10, but I've kept seeing the same
> failures.
>
> Any idea what might be going on?
>
> Thanks!
>
> benji smith
>
> --
> You received this message because you are subscribed to the Google Groups
> "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to elasticsearch+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/elasticsearch/3d8da85f-ed52-4127-b283-94998e851713%40googlegroups.com
> .
> For more options, visit https://groups.google.com/groups/opt_out.
>
--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAGCwEM_bSg1WNQWwcHeTfXgUpE0TczNv%3DSByfBriNCf-SVLWEg%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.
