pgj commented on PR #4958: URL: https://github.com/apache/couchdb/pull/4958#issuecomment-1919029569
Thanks for sharing your thoughts @rnewson. Yes, the execution stats are currently passed around in a dedicated `execution_stats` message but only at the end of the transmission. The other, separate generic `complete` message is the cue for submitting `execution_stats` which may go missing if `complete` is dropped on early termination. The problem is that this can happen even on a single-node cluster. Although I can imagine the scenario you described, I am still inclined to believe that the use of `stop` is such a hard stop that it even shuts down the actual worker who actively contributed to the response. Using `stop` in the view callback is on the coordinator — that is, it already has all the data it needs in order to complete serving the request. But the worker does not have the chance to say "okay, please find the execution stats in the next, final message" because it is immediately stopped right after `stop` is received. A remedy to this could be to let the worker hang around for a while to finalize the transmission that way, but it would also sacrifice some of the performance gain we are currently having with `stop`. This technically leaves us with streaming. Your suggestion looks similar to the one which is contained in this PR. However, I notice that you recommended to extend the `row` message (for `mango_cursor_view:handle_message/2`) directly and not to emit fine-grained, per-row `execution_stats` messages. I can see a case that it works better performance-wise, and I am happy to do a benchmark with that approach as well. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: notifications-unsubscr...@couchdb.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
