Re: Partition query endpoints in CouchDB 4.0

2020-05-13 Thread Adam Kocoloski
Great feedback. Answering your tangential question: document data is not colocated with the built-in _all_docs or _changes indexes in FDB. A request with include_docs=true will cause the CouchDB layer to submit additional range requests to retrieve the document data for each row in the feed. T

Re: Partition query endpoints in CouchDB 4.0

2020-05-13 Thread San Sato
I understand that range-based access to fdb-based views gets efficiently dispatched to node(s) having the key-range, causing zero load on nodes that aren't having that key-range. (tangential question: are the data expected to be co-located with the index/vice versa? or would access patterns tend t

Re: Partition query endpoints in CouchDB 4.0

2020-05-11 Thread Glynn Bird
I've worked with folks using partitioned database so I thought I'd drop my experience of that here: - partitioned databases can definitely give a performance boost (in CouchDB < 4 scenarios) to use-cases where the main "read" use-case can be directed to a single partition. In such cases, only a fr

Re: Partition query endpoints in CouchDB 4.0

2020-05-11 Thread Joshua Mintz
As a data point - during my work with clients, I have seen partition queries by successful in guiding more thoughtful data modeling design. Also in my experience, the guiding principles of correct partitioning are more approachable for people moving from other database systems like SQL Server, Post

Re: Partition query endpoints in CouchDB 4.0

2020-05-11 Thread Garren Smith
Coming back to this. I still think we should support it fully in 4.x so that anyone using it in 3.x will not experience any api changes when moving to 4.x. Once we have had more people use it in 3.x we can make a call on deprecating it for 5.x or look at adding more features to it. On Tue, Apr 21,

Re: Partition query endpoints in CouchDB 4.0

2020-04-21 Thread Robert Samuel Newson
On Adam's point that the partitioned query api encourages good choices ("discourages hot spots"), that's only true for folks that read the documentation, which in my experience is a low percentage of folks. I've encountered a heavy user of partitioned dbs that had precisely four partitions in m

Re: Partition query endpoints in CouchDB 4.0

2020-04-21 Thread Robert Samuel Newson
Hi, Good points on both sides of this. One thing we can hopefully get agreement on is the ?partitioned=true flag on creation and, deeper, the lack of distinction between the two "types" of database going forward? B. > On 21 Apr 2020, at 18:51, Garren Smith wrote: > > I'm on the fence when it

Re: Partition query endpoints in CouchDB 4.0

2020-04-21 Thread Garren Smith
I'm on the fence when it comes to removing it. In terms of the original plan of making querying faster by querying fewer shards that obviously isn't needed. But I think it does create a nice mental model/design pattern when building an application in CouchDB. Splitting your data into partitions th

Re: Partition query endpoints in CouchDB 4.0

2020-04-21 Thread Adam Kocoloski
I think it’s difficult to make a call when 3.0 is still so new. The case for deprecation here is basically less code to maintain, right? It’s not like a user of partitioned databases is causing pain for an FDB-based CouchDB; if anything, there’s a second-order benefit because the partitioning d

Re: Partition query endpoints in CouchDB 4.0

2020-04-20 Thread Kyle Snavely
My two cents is the same. Let's allow 3.* users migrate to 4.* without needing to e.g. change the PQ part of their application and remove the PQ endpoints in 5.0. Best, Kyle On Mon, Apr 20, 2020, 4:16 PM Ilya Khlopotov wrote: > Given that it unlikely that there are too many people using it and

Re: Partition query endpoints in CouchDB 4.0

2020-04-20 Thread Ilya Khlopotov
Given that it unlikely that there are too many people using it and it is being noop in FDB world. I think we should deprecate and remove _partition endpoint. On 2020/04/20 21:04:58, Robert Samuel Newson wrote: > Hi All, > > I'd like to get views on whether we should preserve the _partition en

Re: Partition query endpoints in CouchDB 4.0

2020-04-20 Thread Joan Touzet
On 2020-04-20 5:04 p.m., Robert Samuel Newson wrote: Hi All, I'd like to get views on whether we should preserve the _partition endpoints in CouchDB 4.0 or remove them. In CouchDB 4.0 all _view and _find queries will automatically benefit from the same performance boost that the "partitioned d

Partition query endpoints in CouchDB 4.0

2020-04-20 Thread Robert Samuel Newson
Hi All, I'd like to get views on whether we should preserve the _partition endpoints in CouchDB 4.0 or remove them. In CouchDB 4.0 all _view and _find queries will automatically benefit from the same performance boost that the "partitioned database" feature brings, by virtue of FoundationDB. I