Re: [DISCUSS] Improve load shedding by enforcing timeouts throughout stack

2019-04-26 Thread Adam Kocoloski
Hi Joan, great topic. We don’t have enough realistic benchmarking data to be really specific yet, but my expectation is that the aggregate size of the underlying KV pairs is at least as important as number of documents in the batch. I have no doubt we’ll be able to ingest 1,000 1KB documents

Re: [DISCUSS] Improve load shedding by enforcing timeouts throughout stack

2019-04-26 Thread Joan Touzet
Hi Adam, I'll bring up a concern from a recent client with whom I engaged. They're on 1.x. On 1.x they have been doing 50k bulk update operations in a single request. 1.x doesn't time out. The updates are such that they guarantee that none will result in a conflict or be rejected, so all 50k

Re: [DISCUSS] Improve load shedding by enforcing timeouts throughout stack

2019-04-26 Thread Adam Kocoloski
Hi all, The point I’m on is that we should take advantage of this extra bit of information that we acquire out-of-band (e.g. we just decide as a project that all operations take less than 5 seconds) and come up with smarter / cheaper / faster ways of doing load shedding based on that