On 3 January 2017 at 16:26, Daniel Gruno <[email protected]> wrote: > On 01/03/2017 05:23 PM, sebb wrote: >> On 3 January 2017 at 14:47, Daniel Gruno <[email protected]> wrote: >>> On 01/03/2017 03:44 PM, sebb wrote: >>>> On 3 January 2017 at 12:33, Daniel Gruno <[email protected]> wrote: >>>>> On 01/03/2017 02:12 AM, sebb wrote: >>>>>> The test VM is now running ES 5.x with a basic database. >>>>>> The Apache config appears OK, Javascript is able to invoke lua scripts >>>>>> OK. >>>>>> >>>>>> However a couple of the queries fail: >>>>>> >>>>>> api/pminfo.lua:169: Backend Database returned code 400! >>>>>> >>>>>> {"error":{"root_cause":[{"type":"illegal_argument_exception","reason":"No >>>>>> search type for >>>>>> [scan]"}],"type":"illegal_argument_exception","reason":"No search type >>>>>> for [scan]"},"status":400} >>>>> >>>>> search_type changed in 5.x to mean something else, and should be omitted >>>>> in scan function in elastic.lua, AIUI. >>>>> >>>>>> >>>>>> api/stats.lua:410: Backend Database returned code 400! >>>>>> >>>>>> {"error":{"root_cause":[{"type":"parsing_exception","reason":"no >>>>>> [query] registered for >>>>>> [limit]","line":1,"col":405}],"type":"parsing_exception","reason":"no >>>>>> [query] registered for [limit]","line":1,"col":405},"status":400} >>>>> >>>>> limit is a thing of the past, and the size parameter has changed as well >>>>> (it not denotes the total docs in the next scroll, not per shard/node). >>>>> It might be easier to adjust this for ES 2.3/5.1 and not support earlier >>>>> versions of ElasticSearch going forward. >>>> >>>> +1 >>>> >>>>> I seem to also recall some harmonization needed between hard configured >>>>> limits to results and how the scroll API works. I'll look into that ASAP. >>>> >>>> So what does lists.apache.org use for the queries? >>>> Or does it not use ES 5.x? >>> >>> There are minor proprietary edits, but most importantly, it doesn't use >>> the word cloud feature, so the issue is not present there. >> >> Disabling word-cloud avoids the crash in stats.lua, but pminfo.lua still >> fails. >> Not surprising as that does not depend on the word cloud setting. >> I have tried with both the current and 0.9 code. >> >> The 0.9 JS ignores the error so it is not so obvious. >> >> However lists.a.o does not show the problem. >> So there must be some other code difference in lists.a.o to allow it >> to work with ES 5.x > > That bit is just removing 'search_type=scan' in the scan function in > elastic.lua
AFAIK that's not the only change needed; the response changes as well [2] This says: "The response of the initial search request will not contain any results in the hits array. The first results will be returned by the first scroll request. " > as that changed with 5.x (or maybe 2.x, I can't remember > and the docs don't say). It was deprecated in 2.1 and dropped in 5.0 [1] The doc says ES is just as efficient, but that only applies if one sorts by the "_doc" field which mostly we don't want to do. > Before we do that, we should agree on only supporting 2.x an onwards. I agree that the version support strategy needs to be decided soon. However AFAICT it's not a blocker for fixing the code to support 5.x. [1] https://www.elastic.co/guide/en/elasticsearch/reference/5.1/breaking_50_search_changes.html#_literal_search_type_scan_literal_removed [2] https://www.elastic.co/guide/en/elasticsearch/reference/2.0/search-request-scroll.html#scroll-scan > With regards, > Daniel. > >> >>>> >>>>>> >>>>>> I have tried the stats.lua query directly in curl and it fails with >>>>>> the same error, so this is not a Lua issue. >>>>>> >>>>>> It looks like the syntax for these particular queries has changed in 5.x. >>>>>> >>>>>> However I was told that lists.a.o was running on 5.x. >>>>>> If so, then I assume it must have had some fixes to the syntax which >>>>>> have not been back-ported to the mainline code? >>>>>> >>>>>> Anyone got any idea what needs fixing? >>>>>> >>>>> >>> >
