Re: Replication of attachment is extremely slow.. LOGGED INFORMATION

2014-01-24 Thread Scott Weber
Awesome Dave/Nick. We are using the binaries, when can we expect that they will be available? Although I can build from the source if I need to.  But I would need to install GIT, and all other dependencies that are needed. -Scott - Original Message - From: Paul Davis To: "user@couc

Re: Replication of attachment is extremely slow.. LOGGED INFORMATION

2014-01-24 Thread Paul Davis
Yeah. I happened to be staring at that function thinking "This looks really slow..." and remembered the recent PR. Turns out Nick had already done all the work fixing it. For reference I'm seeing a roughly 20x speedup on the HTTP based attachment replication. On Fri, Jan 24, 2014 at 4:37 PM, Adam

Re: Replication of attachment is extremely slow.. LOGGED INFORMATION

2014-01-24 Thread Adam Kocoloski
Well how about that. Go Nick! Adam > On Jan 24, 2014, at 7:31 PM, Paul Davis wrote: > > Interesting note, this is fixed by applying the patch to COUCHDB-1953. > >> On Fri, Jan 24, 2014 at 3:03 PM, Rian R. Maloney >> wrote: >> Will do Dave. Thanks again for the help >> >> Im away from desk a

Re: Replication of attachment is extremely slow.. LOGGED INFORMATION

2014-01-24 Thread Paul Davis
Interesting note, this is fixed by applying the patch to COUCHDB-1953. On Fri, Jan 24, 2014 at 3:03 PM, Rian R. Maloney wrote: > Will do Dave. Thanks again for the help > > Im away from desk at the moment but - Ive recreated this on 2 windows 7 pcs > and a MAC OS X mini. > > Im putting together

Re: Replication of attachment is extremely slow.. LOGGED INFORMATION

2014-01-24 Thread Rian R. Maloney
Will do Dave. Thanks again for the help Im away from desk at the moment but - Ive recreated this on 2 windows 7 pcs and a MAC OS X mini. Im putting together a cleansed file - this format is used for check image exchange between banks so I need to remove personal data. Thanks Rian On Friday,

Re: Replication of attachment is extremely slow.. LOGGED INFORMATION

2014-01-24 Thread Paul Davis
Duplicated locally. Poking around at debugging what's going on. On Fri, Jan 24, 2014 at 1:08 PM, Scott Weber wrote: > Thanks. I'll remember that for next time. > > Hopefully, there won't be a next time for a while :-) > > > > > - Original Message - > From: Jens Alfke > To: "user@couchdb.

Re: Replication of attachment is extremely slow.. LOGGED INFORMATION

2014-01-24 Thread Scott Weber
Thanks. I'll remember that for next time. Hopefully, there won't be a next time for a while :-) - Original Message - From: Jens Alfke To: "user@couchdb.apache.org" ; Scott Weber Cc: "replicat...@couchdb.apache.org" Sent: Friday, January 24, 2014 2:07 PM Subject: Re: Replication of

Re: Replication of attachment is extremely slow.. LOGGED INFORMATION

2014-01-24 Thread Rian R. Maloney
Thanks EVERYONE for the assistance on this one. Much appreciated! Great user group The following curl batch script consistently recreates the problem.  The difference between a fast and slow replication is whether or not the full URL is used for the "from" and "to" databases cls echo off echo

Re: Replication of attachment is extremely slow.. LOGGED INFORMATION

2014-01-24 Thread Scott Weber
Ah.  Now we have something. It was really easy to change the PHP script and re-run the batch file.  Over and over.  So I have a bunch of tests. When the script was modified to use only the names "to" and "from"  It worked fast... Really fast.  But any combination of the "localhost" was very lon

Re: Replication of attachment is extremely slow.. LOGGED INFORMATION

2014-01-24 Thread Paul Davis
The important thing to note here is that test 1 is using URLs for databases where tests 2 and 3 are using internal replication. Reading the logs it shows that the request to write the attachment took about 72 seconds. Out of curiosity could you rerun test 3 twice more with the following JSON bodies

Re: Replication of attachment is extremely slow.. LOGGED INFORMATION

2014-01-24 Thread Jens Alfke
On Jan 24, 2014, at 11:41 AM, Scott Weber wrote: > I do not see the log attached. It must have been stripped by the list server. > I will copy/paste it to the bottom of this email. It is 1024 lines. Speaking of netiquette :) please don't paste a thousand lines of logs into a message. It messe

Re: Top posting in threads on the list's

2014-01-24 Thread Mark Hahn
Go to the python forum and top-post. You will not be able to sleep at night after the beating you will receive. I personally top-post everywhere and have since the 80's. I have no problem reading it now because gmail hides the old stuff and I can usually remember the conversation. When I forget

Re: External access to couchdb

2014-01-24 Thread Mike Marino
Can you see the couchdb server from your PC at all, i.e. using ping/nmap/telnet? nmap [couchdb_server_address] telnet [couchdb_server_address] 5984 On Thu, Jan 23, 2014 at 11:22 AM, Jordi Cabré wrote: > Hi all! > >I'm trying to get access from an external PC to my couchdb. >I've in

External access to couchdb

2014-01-24 Thread Jordi Cabré
Hi all! I'm trying to get access from an external PC to my couchdb. I've installed (replicated) acralyzer and acra-distributuin on my coucdb. Locally, all works, but externally not (from another pc into the same net, from a mobile...) My bind_address on default.ini is 0.0.0.0. And it i

Re: Replication of attachment is extremely slow.. LOGGED INFORMATION

2014-01-24 Thread Scott Weber
We have a duplication of the problem from a cleaned installation.  And there are some interesting things in the log, but I don't know what they mean, since I am not familiar with the internals of CouchDB. I have attached the couch log. I can send the actual file being replicated, but it is abou

Re: Replication of attachment is extremely slow

2014-01-24 Thread Paul Davis
If you can duplicate this the first thing I'd look at during a slow replication is "sudo netstat -tanp tcp" to see if you're maybe bumping up against open socket limits. On Fri, Jan 24, 2014 at 7:40 AM, Scott Weber wrote: > I appreciate the digging, but in the case of the test file we were using,

Re: Replication of attachment is extremely slow

2014-01-24 Thread Scott Weber
I appreciate the digging, but in the case of the test file we were using, it is some text that doesn't have dashes or newlines, mixed with image data which are big binary blobs. So strings that look like mime boundaries aren't likely to be present. -Scott - Original Message - From:

Re: Top posting in threads on the list's

2014-01-24 Thread Tim Black
On 01/24/2014 02:53 AM, Benoit Chesneau wrote: > I don't think we should enforce any guideline here. Let's post arbitrary fields in JSON, and let the readers determine which fields they want to read! ; ) Tim

Re: Replication of attachment is extremely slow

2014-01-24 Thread Scott Weber
Well, as much as I hate to be a victim of "operator error", it appears that is the case.  There is still an issue, but it seems to be dependent on an external process. What I did: Stopped the service, and removed the logs and 'to'/'from' databases.  Started the service, and uploaded a 4 meg fil

Re: Replication of attachment is extremely slow

2014-01-24 Thread Nick North
On 24 January 2014 15:01, Jens Alfke wrote: > > On Jan 24, 2014, at 5:06 AM, Nick North wrote: > > > I'm not really expecting this problem to be the cause of the slowdown: > > the attachment needs to contain a lot of initial prefixes of the MIME > > boundary string for things to be really bad. >

Re: Replication of attachment is extremely slow

2014-01-24 Thread Jens Alfke
On Jan 24, 2014, at 5:06 AM, Nick North wrote: > I'm not really expecting this problem to be the cause of the slowdown: > the attachment needs to contain a lot of initial prefixes of the MIME > boundary string for things to be really bad. This is on the reading side, where the MIME parser is l

Re: Replication of attachment is extremely slow

2014-01-24 Thread Nick North
On 24 January 2014 12:44, Dave Cottlehuber wrote: > On 24. Jänner 2014 at 08:23:16, Nick North (nort...@gmail.com) wrote: > > From: Nick North > > > There is a known inefficiency > > in parsing of attachments containing certain character strings. > > This doesn't obviously sound like an instance

Re: Replication of attachment is extremely slow

2014-01-24 Thread Dave Cottlehuber
On 24. Jänner 2014 at 08:23:16, Nick North (nort...@gmail.com) wrote: > From: Nick North > There is a known inefficiency  > in parsing of attachments containing certain character strings.  > This doesn't obviously sound like an instance of it, but you can  > eliminate the possibility very quickly

Re: Top posting in threads on the list's

2014-01-24 Thread Benoit Chesneau
On Fri, Jan 24, 2014 at 9:26 AM, Andy Wenk wrote: > Garren, > > On 24 January 2014 08:45, Garren Smith wrote: > > > It is good to have these discussions, my only concern is that we scare > off > > people that are worried about sending a message to the mailing lists and > > getting it wrong. Ther

Re: Top posting in threads on the list's

2014-01-24 Thread Garren Smith
Hi Andy, Thats a great idea, look forward to the guidelines because I could use the help. Cheers Garren On 24 Jan 2014, at 10:26 AM, Andy Wenk wrote: > Garren, > > On 24 January 2014 08:45, Garren Smith wrote: > It is good to have these discussions, my only concern is that we scare off > pe

Re: Top posting in threads on the list's

2014-01-24 Thread Andy Wenk
Garren, On 24 January 2014 08:45, Garren Smith wrote: > It is good to have these discussions, my only concern is that we scare off > people that are worried about sending a message to the mailing lists and > getting it wrong. There was an email earlier today where that happened to > some degree.