That problem is likely due to the fact the user HTTP request is timing
out while waiting for the replication to complete, that in turn kills
the underlying replication process. Restarting the replication will
usually help as CouchDB avoids sending the same document twice, but if
the replication is exceptionally long it might not get past the point
where it it finishing examining the documents.
The problem is its only saves off the replication record once it
completes successfully, so until it completes it always examine the
same number of documents to see if they exist on the target replica.
The fix I need to implement is to have it save off the replication
record every x seconds during replication, then if it dies
unexpectedly it will pick back up from the last replication record,
reducing the number of documents needing to be reexamined.
Then we need to solve is the current problem of synchronous HTTP
request to perform the replication. In Futon, the browser doesn't do
the replication, it just sends a single replication request to the
CouchDB server. A CouchDB Erlang process then performs the
replication, accessing database either locally or via HTTP on other
Erlang servers. Right now, the browser can timeout the HTTP request
during a long replication, that in turn kills the replication process.
There are two potential solutions here, the first is to send a browser
ping to keep the connection alive. Easy do do with HTTP 1.1 I think,
just send an empty HTTP chunk. The second is to make it impossible for
the broken HTTP request to kill the replication request. They aren't
mutually exclusive, but the more I think about it, the more I dislike
the second solution.
-Damien
On Jul 16, 2008, at 11:13 AM, Chris Anderson wrote:
On Wed, Jul 16, 2008 at 2:18 AM, Jan Lehnardt <[EMAIL PROTECTED]> wrote:
I'm surprised that his wasn't reported earlier. CouchDB replication
is supposed to be reliable (when we got all the bugs out), so an
external replication thing should not be necessary. I would have
guessed that reporting this is easier than writing code to circumvent
the problem. This should be fixed in CouchDB and not worked
around.
My experience with replication has been that it works flawlessly for
smaller datasets, and as the dataset grows, it either starts to take
so long it may as well be broken (but shows no errors in the log) or
occasionally does the =ERROR REPORT==== thing in the log. The later is
a new symptom in my experience.
I haven't had a chance to bring my install up to latest trunk, so I
hesitated to report it. Today's my only sane day for a couple of weeks
on each side, so I'll see what progress I can make.
Chris
--
Chris Anderson
http://jchris.mfdz.com
