[
https://issues.apache.org/jira/browse/COUCHDB-1797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13817387#comment-13817387
]
Adam Kocoloski commented on COUCHDB-1797:
-----------------------------------------
I think that patch has too much of a performance impact to merge (it's a
gen_server:call per view row), but I do see a race condition here that could
lead to these {not_found, missing} / null results.
If you look at the code where we obtain the view group state:
https://github.com/apache/couchdb/blob/1.5.0/src/couch_mrview/src/couch_mrview.erl#L75
you can see that we already have the Db open. couch_mrivew ensures that the
index state we return is at a sequence >= the sequence of the Db:
https://github.com/apache/couchdb/blob/1.5.0/src/couch_mrview/src/couch_mrview_util.erl#L41-L50
It's quite possible that couch_index had to fire off an updater to satisfy that
condition, and when the updater starts it gets the latest Db:
https://github.com/apache/couchdb/blob/1.5.0/src/couch_index/src/couch_index_updater.erl#L129
If updates landed in between opening the Db used by query_view and opening the
Db used by couch_index_updater then we can end up with updates in the view
index that are not yet accessible by the Db held by query_view. Attempts to
open these latest docs will then come back with {not_found, missing}.
Re-opening the Db in query_view after we receive the index state but before we
start the fold alleviates the performance impact, but it's still a race
condition. The {not_found, missing} errors will disappear, but in their place
we might end up with {not_found, deleted} errors or documents that are at a
newer revision than the revision that was included in the index.
A really strictly consistent solution would have to find a way to communicate
the exact #db used by the indexer back to the query handler. That's
non-trivial to say the least.
I'm not opposed to a single reopen in the query handler after we receive the
index state (maybe we can make it conditional on the sequence of the secondary
index being greater than the sequence of the original DB), but we should be
clear that it really just trades one inconsistency for another.
> Query against a view w/ include_docs=true yields doc:null
> ---------------------------------------------------------
>
> Key: COUCHDB-1797
> URL: https://issues.apache.org/jira/browse/COUCHDB-1797
> Project: CouchDB
> Issue Type: Bug
> Components: Database Core
> Affects Versions: 1.3
> Reporter: Marten Feldtmann
> Priority: Critical
> Attachments:
> 0001-fix-null-doc-or-doc-not-found-when-including-it-in-a.patch
>
>
> I have a test system (CouchDB 1.3.0 under windows 7/64 Bit), where I insert
> lots of documents(100000) to get a regular flow of new incoming documents.
> The database has also some views (25). Another task is reading documents from
> a view, change some attributes and then the document should disappear from
> the view (and appear in another view). Something like a state machine.
> I do the following query against a view:
> http://localhost:5984/test5/_design/ticketorggraphics/_view/waiting?limit=5&include_docs=true&include_end=true
> and get the following string:
> {"total_rows":9224,"offset":0,"rows":[
> {"id":"92224c2862ef49c15f3d51e3e7283cde","key":"ni13_1_2013101_20130518155518065","value":"ni13_2013101_VsPercentResult_83519204509647","doc":{"_id":"92224c2862ef49c15f3d51e3e7283cde","_rev":"1-6e8b57744c2b88f3d2dcad602e2e3353","distribution":"global","type":"ESTicketCreateOriginalGraphic","createdTS":"20130518155518065","priority":1,"eDayID":"ni13","electionID":2013101,"layout":"ard2013","baseDocKey":"ni13_2013101_VsPercentResult_83519204509647","_attachments":{"bclitemsdata":{"content_type":"application/zip","revpos":1,"digest":"md5-sdff2jkxp4w6xsSlx5Yzxw==","length":4533,"stub":true}}}},
> {"id":"92224c2862ef49c15f3d51e3e7285d38","key":"ni13_1_2013101_20130518155518135","value":"ni13_2013101_VsPercentResult_16771681663808","doc":{"_id":"92224c2862ef49c15f3d51e3e7285d38","_rev":"1-05b92282ec5f2d1b6d56912760792702","distribution":"global","type":"ESTicketCreateOriginalGraphic","createdTS":"20130518155518135","priority":1,"eDayID":"ni13","electionID":2013101,"layout":"ard2013","baseDocKey":"ni13_2013101_VsPercentResult_16771681663808","_attachments":{"bclitemsdata":{"content_type":"application/zip","revpos":1,"digest":"md5-sdff2jkxp4w6xsSlx5Yzxw==","length":4533,"stub":true}}}},
> {"id":"92224c2862ef49c15f3d51e3e728c5e2","key":"ni13_1_2013101_20130518155518413","value":"ni13_2013101_VsPercentResult_10619936276742","doc":null},
> {"id":"b02d128d88188d78ef7f3478586c349b","key":"ni13_2_2013101_20130518155203924","value":"ni13_2013101_VsPercentResult_46202583943140","doc":{"_id":"b02d128d88188d78ef7f3478586c349b","_rev":"1-bd2a2bfb1f7461e6b3ec6d4d106e08e4","distribution":"global","type":"ESTicketCreateOriginalGraphic","createdTS":"20130518155203924","priority":2,"eDayID":"ni13","electionID":2013101,"layout":"ard2013","baseDocKey":"ni13_2013101_VsPercentResult_46202583943140","_attachments":{"bclitemsdata":{"content_type":"application/zip","revpos":1,"digest":"md5-ErSTxuEaSHZrIJ276ANdOQ==","length":4533,"stub":true}}}},
> {"id":"b02d128d88188d78ef7f3478586c72a1","key":"ni13_2_2013101_20130518155204124","value":"ni13_2013101_VsPercentResult_91455901404013","doc":{"_id":"b02d128d88188d78ef7f3478586c72a1","_rev":"1-6f702cff1d12f992d745fc19be84b846","distribution":"global","type":"ESTicketCreateOriginalGraphic","createdTS":"20130518155204124","priority":2,"eDayID":"ni13","electionID":2013101,"layout":"ard2013","baseDocKey":"ni13_2013101_VsPercentResult_91455901404013","_attachments":{"bclitemsdata":{"content_type":"application/zip","revpos":1,"digest":"md5-L9UXpNNRUyk/LSlzAyEsXQ==","length":4533,"stub":true}}}}
> ]}
> Please notice, that the doc attribute of the third item in the rows attribute
> has the value "null". That means, that the view entry has no belonging
> document ????????
> This seems to indicate an inconsistency within the database.
> I can not say, when this error happens, but I know, that it will happen in my
> test run.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
