View signatures and design documents have been around a *long* time, as has the deployment pattern around building the new version of the index in the background to minimize production downtime. I don’t think anything substantial has changed on this front in several years.
I think it’s likely that more sophisticated users specifically rely on the behavior that all of the views in a view group are updated together, so that stale=ok requests to different views in a group will all show the state of a database at the same point in time. But I’m also quite eager to hear feedback. I get the desire to avoid hitting the 10MB limit and maximizing the ability to work with all existing design documents; on the other hand, I think we’d consider any view group emitting more than 10MB of index data *for a single document* to be quite a bad design pattern. All in all - a tricky topic. Thanks for bringing it up. Adam > On May 28, 2019, at 5:26 AM, Garren Smith <gar...@apache.org> wrote: > > Hi Everyone, > > Traditionally every design doc has a view signature (a md5 checksum), which > is made up of the views defined in it along with the options and query > language. > > As Paul mentions on the map index RFC [1], the reason that indexes were > grouped together was to save space on the id tree index and to guarantee > that specific indexes are always up to date at the same time. Currently it > is also more efficient to service multiple map/reduce functions at once > when sending docs to the javascript query engine. > > With the move to FoundationDB, we don’t get the added space saving of > grouping view indexes together for the id tree index. And to guarantee that > all the views in a design doc are updated to the same point, we would have > to update multiple views in the transaction which will lead to larger > transactions and the possibility of exceeding the 10 MB transaction limit. > > Because of that, I would like to separate each index in a design document > so that it has its own unique signature and that each index will be built > on its own. That way only a specific index’s key/values will be updated in > a transaction. It also would mean that when a design doc is updated with > new views or some of the views are changed, the unchanged views won’t need > to be rebuilt as their view signatures would remain the same. > > We could look at adding a view group, which contains a collection of views > - most likely the views in one design document would be a view group. So > instead of updating one view in the background job queue, all the indexes > in a view group would be built by that job. But they would not be updated > in the same transaction, however the job would only be considered complete > once all the views in a view group were updated. > > Where I’m not 100% if this would work is around the javascript query > engine. This change would mean that only one function would be processed > at a time, so it could be a lot more inefficient. Ideally it would be > great to update how the javscript query engine works so it can handle this > case better. > > The usage of view signatures is pretty new to me, so I would appreciate > feedback from people that know more about the history of view signatures > and design docs, along with people that use views and rely on indexes in a > design doc all being updated at once. > > Cheers > Garren > > [1] > https://github.com/apache/couchdb-documentation/pull/410#pullrequestreview-236827606
