nickva commented on PR #4958: URL: https://github.com/apache/couchdb/pull/4958#issuecomment-1906853368
I was mainly suggesting a way to have a bound on the uncertainty in the stats, if it the increase in messages is a problem. It could be a time-bound (milliseconds) or doc examined count (if N > 100 emit ...) bound. It's a half-way hack, it's better than getting no stats at all, but maybe less expensive than emitting stats for every row. But, I don't know if it's a performance issue, I think you and @chewbranca did some measurements? > In such cases, zero counts are observed, therefore the _find results are invisible. From what I remember in mango the issue was happening when we have a `limit` specified? With a `limit` we spawn all the workers with the same limit value but we will stop sooner based on when we hit the limit on the coordinator side. So then, some workers won't get a chance to emit a `completed` message and will keep emitting rows, others will emit a `completed` message but the coordinator by that time is already stopping and won't care. When the coordinator stops it will immediately issue a `kill_all` rexi call to terminate all the workers. Is that roughly what was happing? > Is not there an optimization somewhere down the stack that enables [coalesencing](https://en.wikipedia.org/wiki/Coalescing_(computer_science)) of messages? I don't know if rexi has message batching. There is buffering if the send channel is blocked, and there is an un-acked message limit. But you're also right that the lower level dist and tcp channel will probably do the batching depending on how the environment is set up. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: notifications-unsubscr...@couchdb.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
