> On 16. May 2019, at 23:03, Robert Samuel Newson <rnew...@apache.org> wrote:
>
> I suggest an alternative; the new design document could include the _id of
> design document it’s replacing (“_replaces”:”_design/foo”). On completion of
> the view build of the new design document, CouchDB itself updates the named
> _id to the same content as the new design document (strictly, only the parts
> needed to make the view sig match) (perhaps it also deletes the new document).
>
> The advantage to this is that queries to the original design document
> continue to work throughout and at no point is there a discrepancy between
> the design documents contents (the map and reduce functions, etc) and the
> results you get from it.
As Stefan points out, this only works if the consuming code knows how to handle
both index result formats. If not, we need a way for either CouchDB to signal
the build being ready or a way to confirm the swap to coincide with a code
deploy.
>
> B.
>
>> On 16 May 2019, at 14:55, Jan Lehnardt <j...@apache.org> wrote:
>>
>> +1 on solving this for all users, and same caveats as Stefan raises :)
>>
>>> On 16. May 2019, at 09:38, Stefan du Fresne <ste...@medicmobile.org> wrote:
>>>
>>> Hey Garren,
>>>
>>> Having this a native part of CouchDB seems like a really cool idea: we have
>>> automated the manual dance you're talking about with our deployment
>>> tooling, but it would be really nice not to have to!
>>>
>>> I'm not clear how it would work though, at least in terms of coherent
>>> deployments. View changes are, like SQL migrations, an often non-backwards
>>> compatible change that has to occur as your new code deploys.
>>>
>>> Currently the naive approach is you deploy your new code alongside design
>>> doc changes, which then block view queries on first request until they're
>>> ready to go.
>>>
>>> The better approach is what you describe, which is what we do now, where we
>>> extract our design documents out of our deployment bundle and place them in
>>> a "staging" location to allow them to warm, then rename them and do the
>>> actual code deployment once that's complete (managed by an external
>>> deployment service we built). This importantly lets us split the "warming"
>>> bit from the deployment bit: we only deploy new code once the design
>>> documents that are shipped with that code is ready to go.
>>>
>>> How would you foresee this kind of flow happening here? Would there be a
>>> way to query the design doc to know if it had flipped to the new version
>>> yet? Would you be able to control when this flip occurs? Or would the
>>> expectation be that your code handles both versions gracefully?
>>>
>>> As an example to mull over, let's say you have design doc v1, which has
>>> view a. You push design doc v2, which has added view b, but has also
>>> changed view a in some backwards incompatible way. While v2 is still
>>> building and is not yet the active doc:
>>> - If you queried view a you'd get the v1 version, that's clear
>>> - If you queried view b you'd get... a 404? Some other custom code?
>>> - If you GET the design document what doc would you see? Presumably v2?
>>> - Could you query something to determine which version is currently active?
>>> Or perhaps just whether there is a background version building at all?
>>>
>>> Cheers,
>>> Stefan
>>>
>>>> On 16 May 2019, at 07:51, Garren Smith <gar...@apache.org> wrote:
>>>>
>>>> Hi Everyone,
>>>>
>>>> A common pattern we see for updating large indexes that can take a few days
>>>> to build, is create a new design docs with the new updated views. Then once
>>>> the new design doc is built, a user changes the new design doc’s id to the
>>>> old design doc. That way the CouchDB url for the views remain the same and
>>>> any requests to the design doc url automatically get the latest views only
>>>> once they built.
>>>>
>>>> This is an effective way of managing building large indexes, but the
>>>> process is quite complicated and often users get it wrong. I would like to
>>>> propose that we move this process into CouchDB and let CouchDB handle the
>>>> actual process. From a users perspective, they would add a field to the
>>>> options of a design document that lets CouchDB know, that this build needs
>>>> to be built in the background and only replace the current index once its
>>>> built:
>>>>
>>>> ```
>>>> {
>>>> "_id": "_design/design-doc-id",
>>>> "_rev": "2-8d361a23b4cb8e213f0868ea3d2742c2",
>>>> "views": {
>>>> "map-view": {
>>>> "map": "function (doc) {\n emit(doc._id, 1);\n}"
>>>> }
>>>> },
>>>> "language": "javascript",
>>>> "options": {
>>>> "build_and_replace": true
>>>> }
>>>> }
>>>> ```
>>>>
>>>> I think this is something we could build quite effectively once we have
>>>> CouchDB running on top of FoundationDB. I don’t want to implement it for
>>>> version 1 of CouchDB on FDB, but it would be nice to keep this in mind as
>>>> we build out the map/reduce indexes.
>>>>
>>>> What do you think? Any issues we might have by doing this internally?
>>>>
>>>> Cheers
>>>> Garren
>>>
>>
>> --
>> Professional Support for Apache CouchDB:
>> https://neighbourhood.ie/couchdb-support/
>>
>
--
Professional Support for Apache CouchDB:
https://neighbourhood.ie/couchdb-support/
