Re: [DISCUSS] Mango indexes on FDB

2020-03-26 Thread Paul Davis
On Thu, Mar 26, 2020 at 5:33 AM Will Holley wrote: > > Ah - in that case I think we should remove step 3, as it leads to a > confusing mental model. It's much simpler to explain that Mango will only > use fresh indexes and any new indexes will build in the background. > Simpler in some respect. T

Re: [DISCUSS] Mango indexes on FDB

2020-03-26 Thread Paul Davis
The quoting here is weird. Are you saying to skip _all_docs in your proposal, Glynn? On Thu, Mar 26, 2020 at 5:46 AM Glynn Bird wrote: > > +1 on removing step 3 - my reservation on falling back on all_docs is that > users have no insight into how expensive a query is, other than measuring > laten

Re: _all_docs collation

2020-03-26 Thread Garren Smith
Honestly, I'm not sure. I'm fine with keeping raw collation. But it was definitely confusing when thinking about _all_docs. On Thu, Mar 26, 2020 at 2:43 PM Jonathan Hall wrote: > Would changing _all_docs, but keeping raw collation as an option, really > simplify anything anyway? > > Or are you

Re: _all_docs collation

2020-03-26 Thread Jonathan Hall
Would changing _all_docs, but keeping raw collation as an option, really simplify anything anyway? Or are you proposing removing the raw collation option entirely? Or maybe you're proposing this more as a UX change, than a technical simplification? Jonathan On 3/26/20 11:18 AM, Garren Smit

Re: [DISCUSS] Mango indexes on FDB

2020-03-26 Thread Garren Smith
On Thu, Mar 26, 2020 at 12:46 PM Glynn Bird wrote: > +1 on removing step 3 - my reservation on falling back on all_docs is that > users have no insight into how expensive a query is, other than measuring > latencies (which might depend on other factors). I would hope that folks > would use optio

Re: [DISCUSS] Mango indexes on FDB

2020-03-26 Thread Glynn Bird
+1 on removing step 3 - my reservation on falling back on all_docs is that users have no insight into how expensive a query is, other than measuring latencies (which might depend on other factors). I would hope that folks would use option 1 anyway. So Paul's list becomes: 1. If user specifies an

Re: [DISCUSS] Mango indexes on FDB

2020-03-26 Thread Will Holley
Ah - in that case I think we should remove step 3, as it leads to a confusing mental model. It's much simpler to explain that Mango will only use fresh indexes and any new indexes will build in the background. On Thu, 26 Mar 2020 at 10:15, Garren Smith wrote: > On Thu, Mar 26, 2020 at 11:04 AM W

Re: _all_docs collation

2020-03-26 Thread Garren Smith
Oh interesting, reading the documentation more carefully I see we have raw collation https://docs.couchdb.org/en/stable/ddocs/views/collation.html#raw-collation So _all_docs is using that and that explains why an object comes before a string. So do we want to keep raw collation for _all_docs? On T

Re: [DISCUSS] Mango indexes on FDB

2020-03-26 Thread Garren Smith
On Thu, Mar 26, 2020 at 11:04 AM Will Holley wrote: > Broadly, I think it's a big step forward if we can prevent Mango from > automatically selecting extremely stale indexes. > > I've been going back and forth on whether step 3 could lead to some > difficult-to-predict behaviour. If we assume tha

Re: _all_docs collation

2020-03-26 Thread Glynn Bird
It's not something I was aware of, but it's certainly a known "feature", documented here: https://docs.couchdb.org/en/stable/ddocs/views/collation.html#all-docs (probably because all keys are strings in all_docs, whereas they can be all sorts of mixed types with a view, and ascii collation would b

Re: [DISCUSS] Mango indexes on FDB

2020-03-26 Thread Glynn Bird
Agree with Will that falling back to _all_docs-powered queries is usually undesirable in all but the smallest data sets. More folks than you'd think end up going into production without the right index because the _all_docs-powered query in development (with a small data set) seemed to be fast enou

Re: [DISCUSS] Mango indexes on FDB

2020-03-26 Thread Will Holley
Broadly, I think it's a big step forward if we can prevent Mango from automatically selecting extremely stale indexes. I've been going back and forth on whether step 3 could lead to some difficult-to-predict behaviour. If we assume that requests have a short timeout - e.g. we can't return any resu

_all_docs collation

2020-03-26 Thread Garren Smith
Hi Everyone, While working on the Mango implementation for FDB, I've noticed that _all_docs has some weird ordering collation. If you do something like GET /db/_all_docs?startkey={} it will return all the documents even though in view collation an object is ordered after strings. The reason I've n