Hey Adam, Are there any plans to open source any of the benchmarking code Cloudant has done in the past for performance testing? If so I could give the new Windows build a benchmark against Linux on the same hardware.
If not I could try throwing something together in tsung, but it'd be pretty low priority work for me. -Joan ----- Original Message ----- > From: "Adam Kocoloski" <[email protected]> > To: [email protected] > Sent: Tuesday, November 1, 2016 12:08:13 PM > Subject: Re: Indexes are inaccessible while indexers are running > > Hi Graham, the indexes don’t update automatically, but it is possible > to prime the indexers by issuing a query to the view at any point > during the import. One interesting option for you is the > "?stale=update_after" flag, which will respond with the current > state of the view index and trigger a background update of the > indexes after the fact: > > GET /<db>/_design/<ddoc>/_view/<view>?stale=update_after > > You could also add a &limit=0 if you’re only interested in priming > the indexers. > > As far as indexing performance is concerned … ~4500 docs/second isn’t > awesome, but the devil is in the details: how many times is each > document indexed? Does the server have adequate CPU and IO? Are you > running 2.0 or one of the 1.x versions? > > I can dig up some benchmarks but I’m certain I’ve seen (Linux) > systems index several times faster than that. I haven’t seen a lot > of extensive performance testing on Windows though. Cheers, > > Adam > > > On Oct 31, 2016, at 7:41 AM, Graham Bull > > <[email protected]> wrote: > > > > Hello, > > > > I'm currently evaluating CouchDB (and other NoSQL databases). > > > > I have a number of databases of various sizes. After restarting the > > CouchDB > > service (I'm on Windows) eight "indexer" tasks started running on > > the > > largest database (40 million documents), which was recently > > imported. > > > > After 30 minutes the progress on all tasks is 20%. In the meantime > > I can't > > run any queries using the database's indexes. At this rate, it'll > > take > > around 2.5 hours to index the entire database. > > > > Presumably, when indexes are created, they're initially empty? And > > the > > indexer tasks are required to do the actual indexing? If so, then > > the > > performance is pretty bad. It took nearly 2 hours to import the 40 > > million > > records. Add on index creation, and you're looking at 4.5 hours. > > Without > > mentioning other relational and NoSQL databases by name, or giving > > any > > stats, CouchDB's import and indexing performance is pretty bad in > > comparison. > > > > Is there a way to force the indexers to run immediately after > > importing the > > data, and to query the indexing status so that my app can wait > > until it's > > completed? > > > > Thanks in advance, > > > > Graham > >
