pgj opened a new pull request, #4458:
URL: https://github.com/apache/couchdb/pull/4458
Upon adding a text index, Mango queries may not return documents that should
match a selector that contains the `$regex` operator. Steps to reproduce
(thanks to @willholley), that are also captured in the integration tests:
1. Create an empty database.
2. Add a document:
```json
{
"_id": "foo",
"currency": "HUF",
"location": "EUROPE"
}
```
3. Run a Mango query without indexes -- this should return the document
above:
```json
{
"selector": {
"currency": {
"$regex": "HUF"
}
}
}
```
4. Add a text index for `location`:
```json
{
"index": {
"fields": [
{
"name": "location",
"type": "string"
}
]
},
"name": "location-text",
"type": "text"
}
```
5. Run the query once more -- this time, no documents are returned.
This is because the text index is chosen to answer the query even if it does
not contain the field on which the `$regex` match should be run. This is due
to the permissive logic of the index selection where the fields with `$regex`
in the selector are not cross-checked against the fields in the index.
The PR implements the necessary changes to ensure that text indexes are
chosen more carefully and covers them with tests. Note that this also means an
increase in the unit test coverage of the related functions to be less
dependent on a working Clouseau instance. As an extra, it aims to improve the
readability by introducing extra commentary (thanks to @mikerhodes) and type
specifications.
Random stats on coverage, before:
```console
$ make eunit apps=mango
==> mango (compile)
==> rel (compile)
==> couchdb (compile)
WARN: Missing plugins: [covertool]
WARN: Missing plugins: [pc]
==> couchdb (setup_eunit)
==> mango (eunit)
======================== EUnit ========================
[..]
=======================================================
All 55 tests passed.
[..]
Code Coverage:
Code Coverage:
mango_app : 0%
mango_crud : 0%
mango_cursor : 0%
mango_cursor_special : 0%
mango_cursor_text : 0%
mango_cursor_view : 33%
mango_doc : 3%
mango_epi : 100%
mango_error : 0%
mango_execution_stats : 0%
mango_fields : 50%
mango_httpd : 0%
mango_httpd_handlers : 0%
mango_idx : 18%
mango_idx_special : 0%
mango_idx_text : 29%
mango_idx_view : 1%
mango_json : 10%
mango_json_bookmark : 0%
mango_native_proc : 5%
mango_opts : 25%
mango_selector : 52%
mango_selector_text : 0%
mango_sort : 0%
mango_sup : 0%
mango_util : 23%
Total : 17%
==> rel (eunit)
==> couchdb (eunit)
```
after:
```console
$ make eunit apps=mango
==> mango (compile)
==> rel (compile)
==> couchdb (compile)
WARN: Missing plugins: [covertool]
WARN: Missing plugins: [pc]
==> couchdb (setup_eunit)
==> mango (eunit)
======================== EUnit ========================
[..]
module 'mango_idx_text'
[..]
mango_idx_text: indexable_fields_test...[0.106 s] ok
mango_idx_text: is_usable_test...ok
[done in 5.703 s]
[..]
module 'mango_selector_text'
mango_selector_text: convert_fields_test...ok
mango_selector_text: convert_default_test...ok
mango_selector_text: convert_lt_test...ok
mango_selector_text: convert_lte_test...ok
mango_selector_text: convert_eq_test...ok
mango_selector_text: convert_ne_test...ok
mango_selector_text: convert_gte_test...ok
mango_selector_text: convert_gt_test...ok
mango_selector_text: convert_all_test...ok
mango_selector_text: convert_elemMatch_test...ok
mango_selector_text: convert_allMatch_test...ok
mango_selector_text: convert_keyMapMatch_test...ok
mango_selector_text: convert_in_test...ok
mango_selector_text: convert_nin_test...ok
mango_selector_text: convert_exists_test...ok
mango_selector_text: convert_type_test...ok
mango_selector_text: convert_mod_test...ok
mango_selector_text: convert_regex_test...ok
mango_selector_text: convert_size_test...ok
mango_selector_text: convert_not_test...ok
mango_selector_text: convert_and_test...ok
mango_selector_text: convert_or_test...ok
mango_selector_text: convert_nor_test...ok
mango_selector_text: to_query_test...ok
[done in 0.074 s]
[..]
=======================================================
All 81 tests passed.
[..]
Code Coverage:
mango_app : 0%
mango_crud : 0%
mango_cursor : 0%
mango_cursor_special : 0%
mango_cursor_text : 0%
mango_cursor_view : 33%
mango_doc : 3%
mango_epi : 100%
mango_error : 0%
mango_execution_stats : 0%
mango_fields : 50%
mango_httpd : 0%
mango_httpd_handlers : 0%
mango_idx : 18%
mango_idx_special : 0%
mango_idx_text : 61%
mango_idx_view : 1%
mango_json : 10%
mango_json_bookmark : 0%
mango_native_proc : 5%
mango_opts : 25%
mango_selector : 62%
mango_selector_text : 86%
mango_sort : 0%
mango_sup : 0%
mango_util : 38%
Total : 31%
==> rel (eunit)
==> couchdb (eunit)
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
