Hi, An enhancement over the first idea (where we flatten JSON documents into keys and values where the key is the full path to every terminal value, regardless of depth in the JSON) is to allow users to register schemas.
For documents that match a registered schema (suggestion, a top level field called "_schema" is required to mark the documents) we can convert to key/value pairs much more compactly. The key name, barring whatever prefix identifies the database itself, is just the name of the schema and the ordinal of the schema item relative to the declaration. These schema documents (living under /dbname/_schema/$scheme_name akin to design documents) would list all required and optional fields, their types and perhaps other constraints (like valid ranges or relationships with other fields). We could get arbitrarily complex in schema definitions over time. Effectively, these are validate_doc_update functions without the Javascript evaluation pain. We don't necessarily need this in the first version, but it feels like a better response to the worries over the restrictions that the flattening idea is causing than switching to an opaque series of 100k chunks. thoughts? B -- Robert Newson b...@rsn.io On Thu, 31 Jan 2019, at 16:26, Adam Kocoloski wrote: > > > On Jan 31, 2019, at 1:47 AM, ermouth <ermo...@gmail.com> wrote: > > > >> As I don't see the 10k limitation as having significant merit > > > > Not sure it’s relevant here, but Mango indexes put selected doc values into > > keys. > > > > ermouth > > Totally relevant. Not just because of the possibility of putting a large > scalar value into the index, but because someone could create an index > on a field that is itself a container, and Mango could just dump the > entire container into the index as the key. > > Presumably there’s a followup discussion dedicated to indexing where we > can suss out what to do in that scenario. > > Adam
