mikerhodes commented on code in PR #4410: URL: https://github.com/apache/couchdb/pull/4410#discussion_r1094386528
########## src/docs/rfcs/018-mango-covering-json-index.md: ########## @@ -0,0 +1,397 @@ +--- +name: Formal RFC +about: Submit a formal Request For Comments for consideration by the team. +title: 'Support covering indexes when using Mango JSON (view) indexes' +labels: rfc, discussion +assignees: '' + +--- + +[NOTE]: # ( ^^ Provide a general summary of the RFC in the title above. ^^ ) + +# Introduction + +## Abstract + +[NOTE]: # ( Provide a 1-to-3 paragraph overview of the requested change. ) +[NOTE]: # ( Describe what problem you are solving, and the general approach. ) + +Covering indexes are used to reduce the time the database takes to respond to +queries. An index "covers" a query when the query only requires fields that are +in the index (in this way, "covering" is a property of index and query +combined). When this is the case, the database doesn't need to consult primary +data and can return results for the query from only the index. In more familiar +CouchDB terminology, this is equivalent to querying a view with +`include_docs=false`. + +When evaluating a query, Mango currently doesn't use the concept of covering +indexes; even if a query could be answered without reading each result's full +JSON document, Mango will still read it. This makes it impossible for Mango to +return data as quickly as the underlying view. + +My benchmarking shows that Mango can answer at the same rate as the underlying +view index. It currently runs at the same pace as calling the view with +`include_docs=true`. Preliminary modifications to Mango showed that, with +covering index support and a query that can use it, Mango can stream results +as quickly as the underlying view. Adding covering indexes therefore increases +the production use-cases Mango can support substantially. + +There are likely two phases to this: + +- Enable covering indexing processing for current indexes (ie, over view keys). +- Allow Mango view indexes to include extra data from documents, storing it in + the `value` of the view. Support use of this extra data within the covering + indexes feature. + +### Out of scope + +This proposal only covers adding covering indexes to JSON indexes and not text +indexes. The aim is to reduce the need for CouchDB users to run separate +processes, such as Lucene, to get improved querying performance and capability. + +We do not aim to replicate `reduce` functionality from views, only to bring +parity to non-reduced view execution speed (ie, when views are used to search +the document space) to Mango. + +## Requirements Language + +[NOTE]: # ( Do not alter the section below. Follow its instructions. ) + +The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", +"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this +document are to be interpreted as described in +[RFC 2119](https://www.rfc-editor.org/rfc/rfc2119.txt). + +## Terminology + +[TIP]: # ( Provide a list of any unique terms or acronyms, and their definitions here.) + +- Mango: CouchDB's Mongo inspired querying system. +- View / JSON index: Mango index that uses the same index as Cloudant views. +- Coordinator: the Erlang process that handles doing a distributed query across + a CouchDB cluster. Review Comment: I tend towards four spaces as, at least a few years ago, that used to be safest across Markdown renderers. For consistency, I've found the one item that _wasn't_ indented in that style and corrected it. Given CommonMark states that you'd use two spaces in this context, in future I'll probably drop this habit. But for now I've kept it. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
