> On Sep 2, 2015, at 3:50 PM, David Quon <dav...@amcoonline.net> wrote: > > Is there anything to be done to determine the actual total `changesCount` for > pushing and pulling so the `progress` bar becomes useful again?
I feel your pain :( Unfortunately there’s a trade-off between accurate progress reporting, vs. performance and memory footprint. Specifically, if we determined the total number of docs (changesCount) ahead of time, it would slow down replication and/or consume more memory. > Is this a CBL 1.1.0 bug or just the expected behavior that just happened to > work previously in CBL 1.0.3.1? I know of some specific changes that reduce memory usage during a first-time pull; they were made quite a while ago, but it’s likely they were after 1.0.3. The fix I’m thinking of was to address a problem where one developer’s app was literally crashing with an out-of-memory error when pulling 60,000+ documents. To give you an idea of what’s going on: the puller has a couple of parallel tasks. One of them is reading the “_changes feed” from the server, which is a stream of document/revision IDs that are newer than what the client already has. Another task is sending HTTP requests to download those docs, and a third task is inserting the downloaded docs into the local db. The puller doesn’t know how many docs it’s going to pull until it reaches EOF on the changes feed. On the other hand, if it reads the feed as fast as possible, it has to buffer all the doc/revision IDs in memory because the actual download process can’t keep up. That turned out to cause problems with large downloads. So now the feed is throttled so it doesn’t get too far ahead of the downloading … but that means the puller doesn’t know how many documents there are until fairly late in the process. Another possibility would be to ask the server how many documents are in the feed before reading the feed. Unfortunately that would require Sync Gateway to do a lot of redundant work, and the task of generating that feed is already pretty expensive (and complex) on the server side, so I don’t think that’s feasible. —Jens -- You received this message because you are subscribed to the Google Groups "Couchbase Mobile" group. To unsubscribe from this group and stop receiving emails from it, send an email to mobile-couchbase+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/mobile-couchbase/6D296E90-47A0-49A9-99FA-440325EA514B%40couchbase.com. For more options, visit https://groups.google.com/d/optout.
