[jira] Commented: (COUCHDB-240) Replication breaks with large Attachments.
[
https://issues.apache.org/jira/browse/COUCHDB-240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12699207#action_12699207
]
Jeff Hinrichs commented on COUCHDB-240:
---
Adam, Have you had any luck with this issue? With 0.9 rev760917
I am seeing
** reason for termination ==
** attachment_request_failed
In my log file when replicating via pull from one machine to another.
Replicating on the same machine runs without issue.
Replication breaks with large Attachments.
--
Key: COUCHDB-240
URL: https://issues.apache.org/jira/browse/COUCHDB-240
Project: CouchDB
Issue Type: Bug
Components: Database Core
Affects Versions: 0.9
Environment: r 741265. Debian Linux unknown revision, FreeBSD 7.0.
GBit Network connection between the hosts.
Reporter: Maximillian Dornseif
Assignee: Adam Kocoloski
Fix For: 0.10
I use the code in http://code.google.com/p/couchdb-python/issues/detail?id=54
to do replication between two machines.
I'm running 741265 on both machines. I have a Database with big attachments
(high-res images, 31.1 GB,34026 Docs). Pull replication breaks with
following message sent via http:
couchdb.client.ServerError: (500, ('function_clause',
[{lists,map,[#Funcouch_rep.10.28922857,ok]},\n
{couch_rep,open_doc_revs,4},\n
{couch_rep,'-enum_docs_parallel/3-fun-1-',3},\n
{couch_rep,'-spawn_worker/3-fun-0-',3}]))
With push replication the server just drops the connection
(httplib2/__init__.py, line 715, in connect
socket.error: (61, 'Connection refused') - why refused instead of
closed?). I have only been able to replicate the first 100 documents.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (COUCHDB-178) $PREFIX/etc/default/couchdb not installed
[ https://issues.apache.org/jira/browse/COUCHDB-178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeff Hinrichs updated COUCHDB-178: -- Attachment: output.txt as of couchdb - Apache CouchDB 0.9.0a752084 I am no longer experiencing this situation on Ubuntu 8.04 # etc/default/couchdb.tpl. Generated from couchdb.tpl.in by configure. # Sourced by init script for configuration. COUCHDB_USER=couchdb COUCHDB_INI_FILE=/usr/local/etc/couchdb/couch.ini COUCHDB_PID_FILE=/usr/local/var/run/couchdb.pid COUCHDB_STDOUT_FILE=/dev/null COUCHDB_STDERR_FILE=/dev/null COUCHDB_RESPAWN_TIMEOUT=5 However, on Ubuntu 8.10 /usr/local/etc/default/couchdb does not exist and couchdb.stdout / couchdb.stderr are created in what ever directory you issue the start/restart command from. I am including the output of sudo make install $PREFIX/etc/default/couchdb not installed - Key: COUCHDB-178 URL: https://issues.apache.org/jira/browse/COUCHDB-178 Project: CouchDB Issue Type: Bug Components: Build System Affects Versions: 0.9 Environment: Ubuntu 8.10 Reporter: Nolan Darilek Assignee: Noah Slater Priority: Blocker Fix For: 0.9 Attachments: output.txt /usr/local/etc/default/couchdb isn't installed on a fresh checkout of trunk under Ubuntu 8.10. As a result, the default init script creates couchdb.stdout and couchdb.stderr files in the current directory when run. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (COUCHDB-270) Replication w/ Large Attachments Fails
[
https://issues.apache.org/jira/browse/COUCHDB-270?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12682077#action_12682077
]
Jeff Hinrichs commented on COUCHDB-270:
---
forgot to mention on the last report: Apache CouchDB 0.9.0a752084
Replication w/ Large Attachments Fails
--
Key: COUCHDB-270
URL: https://issues.apache.org/jira/browse/COUCHDB-270
Project: CouchDB
Issue Type: Bug
Components: Database Core
Affects Versions: 0.9
Environment: Apache CouchDB 0.9.0a748379
Reporter: Jeff Hinrichs
Attachments: couchdb270_Test.py, couchdb270_Test.py, quick_fix.diff
Attempting to replicate a database with largish attachments (= ~18MB of
attachments in a doc, less thatn 200 docs) from one machine to another fails
consistently and at the same point.
Scenario:
Both servers are running from HEAD and I've been tracking for some time.
This problem has been around as long as I've been using couch.
Machine A holds the original database, Machine B is the server that is doing
a PULL replication
During the replication, Machine A starts showing the following sporadically
in the log:
[Fri, 27 Feb 2009 14:02:48 GMT] [debug] [0.5902.3] 'GET'
/delasco-invoices/INV00652429?revs=trueattachments=truelatest=trueopen_revs=[425644723]
{1,
1}
Headers: [{'Host',192.168.2.52:5984}]
[Fri, 27 Feb 2009 14:02:48 GMT] [error] [0.5901.3] Uncaught error in
HTTP request: {exit,normal}
[Fri, 27 Feb 2009 14:02:48 GMT] [debug] [0.5901.3] Stacktrace:
[{mochiweb_request,send,2},
{couch_httpd,send_chunk,2},
{couch_httpd_db,db_doc_req,3},
{couch_httpd_db,do_db_req,2},
{couch_httpd,handle_request,3},
{mochiweb_http,headers,5},
{proc_lib,init_p,5}]
[Fri, 27 Feb 2009 14:02:48 GMT] [debug] [0.5901.3] HTTPd 500 error response:
{error:error,reason:normal}
As the replication continues, the frequency of these error Uncaught error in
HTTP request: {exit,normal} increase. Until the error is being constantly
repeated. Then Machine B stops sending requests, no more log output, no
errors, the last thing in Machine B's log file is:
[Fri, 27 Feb 2009 14:03:24 GMT] [info] [0.20893.1] retrying
couch_rep HTTP get request due to {error, req_timedout}: [104,116,
116,112,58,
47,47,49,
57,50,46,
49,54,56,
46,50,46,
53,50,58,
53,57,56,
52,47,100,
101,108,97,
115,99,111,
45,105,110,
118,111,
105,99,101,
115,47,73,
78,86,48,
48,54,53,
50,49,51,
56,63,114,
101,118,
115,61,116,
114,117,
101,38,97,
116,116,97,
99,104,109,
101,110,
116,115,61,
116,114,
117,101,38,
108,97,116,
101,115,
116,61,116,
114,117,
[jira] Commented: (COUCHDB-270) Replication w/ Large Attachments Fails
[
https://issues.apache.org/jira/browse/COUCHDB-270?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12682076#action_12682076
]
Jeff Hinrichs commented on COUCHDB-270:
---
All of the large document tests are passing. All of the proper attachment pull
tests, 10,12,14,16,18 work just fine. However the proper attachment push tests
(13,15,17) still fail if the attachments are of any size. (#11 passes -
200x256K payload + 256K Attachment) I start getting connection refused errors
with those.
Sorry for the tardiness of my reply. Was out of town in SF, I ran the tests as
soon as I got back, it took a while to figure out what was going on since the
pushes left couch in a bad state and then the rest of the tests would fail.
Replication w/ Large Attachments Fails
--
Key: COUCHDB-270
URL: https://issues.apache.org/jira/browse/COUCHDB-270
Project: CouchDB
Issue Type: Bug
Components: Database Core
Affects Versions: 0.9
Environment: Apache CouchDB 0.9.0a748379
Reporter: Jeff Hinrichs
Attachments: couchdb270_Test.py, couchdb270_Test.py, quick_fix.diff
Attempting to replicate a database with largish attachments (= ~18MB of
attachments in a doc, less thatn 200 docs) from one machine to another fails
consistently and at the same point.
Scenario:
Both servers are running from HEAD and I've been tracking for some time.
This problem has been around as long as I've been using couch.
Machine A holds the original database, Machine B is the server that is doing
a PULL replication
During the replication, Machine A starts showing the following sporadically
in the log:
[Fri, 27 Feb 2009 14:02:48 GMT] [debug] [0.5902.3] 'GET'
/delasco-invoices/INV00652429?revs=trueattachments=truelatest=trueopen_revs=[425644723]
{1,
1}
Headers: [{'Host',192.168.2.52:5984}]
[Fri, 27 Feb 2009 14:02:48 GMT] [error] [0.5901.3] Uncaught error in
HTTP request: {exit,normal}
[Fri, 27 Feb 2009 14:02:48 GMT] [debug] [0.5901.3] Stacktrace:
[{mochiweb_request,send,2},
{couch_httpd,send_chunk,2},
{couch_httpd_db,db_doc_req,3},
{couch_httpd_db,do_db_req,2},
{couch_httpd,handle_request,3},
{mochiweb_http,headers,5},
{proc_lib,init_p,5}]
[Fri, 27 Feb 2009 14:02:48 GMT] [debug] [0.5901.3] HTTPd 500 error response:
{error:error,reason:normal}
As the replication continues, the frequency of these error Uncaught error in
HTTP request: {exit,normal} increase. Until the error is being constantly
repeated. Then Machine B stops sending requests, no more log output, no
errors, the last thing in Machine B's log file is:
[Fri, 27 Feb 2009 14:03:24 GMT] [info] [0.20893.1] retrying
couch_rep HTTP get request due to {error, req_timedout}: [104,116,
116,112,58,
47,47,49,
57,50,46,
49,54,56,
46,50,46,
53,50,58,
53,57,56,
52,47,100,
101,108,97,
115,99,111,
45,105,110,
118,111,
105,99,101,
115,47,73,
78,86,48,
48,54,53,
50,49,51,
56,63,114,
101,118,
115,61,116,
114,117,
101,38,97,
116,116,97,
99,104,109,
101,110,
[jira] Updated: (COUCHDB-270) Replication w/ Large Attachments Fails
[
https://issues.apache.org/jira/browse/COUCHDB-270?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jeff Hinrichs updated COUCHDB-270:
--
Attachment: couchdb270_Test.py
test script has been enhanced by:
adding additional tests for proper attachment(s) of varying sized
error handling for couchdb-272 has been added to avoid errors irrelevant to
couchdb-270 during the test setup phase
Replication w/ Large Attachments Fails
--
Key: COUCHDB-270
URL: https://issues.apache.org/jira/browse/COUCHDB-270
Project: CouchDB
Issue Type: Bug
Components: Database Core
Affects Versions: 0.9
Environment: Apache CouchDB 0.9.0a748379
Reporter: Jeff Hinrichs
Attachments: couchdb270_Test.py, couchdb270_Test.py, quick_fix.diff
Attempting to replicate a database with largish attachments (= ~18MB of
attachments in a doc, less thatn 200 docs) from one machine to another fails
consistently and at the same point.
Scenario:
Both servers are running from HEAD and I've been tracking for some time.
This problem has been around as long as I've been using couch.
Machine A holds the original database, Machine B is the server that is doing
a PULL replication
During the replication, Machine A starts showing the following sporadically
in the log:
[Fri, 27 Feb 2009 14:02:48 GMT] [debug] [0.5902.3] 'GET'
/delasco-invoices/INV00652429?revs=trueattachments=truelatest=trueopen_revs=[425644723]
{1,
1}
Headers: [{'Host',192.168.2.52:5984}]
[Fri, 27 Feb 2009 14:02:48 GMT] [error] [0.5901.3] Uncaught error in
HTTP request: {exit,normal}
[Fri, 27 Feb 2009 14:02:48 GMT] [debug] [0.5901.3] Stacktrace:
[{mochiweb_request,send,2},
{couch_httpd,send_chunk,2},
{couch_httpd_db,db_doc_req,3},
{couch_httpd_db,do_db_req,2},
{couch_httpd,handle_request,3},
{mochiweb_http,headers,5},
{proc_lib,init_p,5}]
[Fri, 27 Feb 2009 14:02:48 GMT] [debug] [0.5901.3] HTTPd 500 error response:
{error:error,reason:normal}
As the replication continues, the frequency of these error Uncaught error in
HTTP request: {exit,normal} increase. Until the error is being constantly
repeated. Then Machine B stops sending requests, no more log output, no
errors, the last thing in Machine B's log file is:
[Fri, 27 Feb 2009 14:03:24 GMT] [info] [0.20893.1] retrying
couch_rep HTTP get request due to {error, req_timedout}: [104,116,
116,112,58,
47,47,49,
57,50,46,
49,54,56,
46,50,46,
53,50,58,
53,57,56,
52,47,100,
101,108,97,
115,99,111,
45,105,110,
118,111,
105,99,101,
115,47,73,
78,86,48,
48,54,53,
50,49,51,
56,63,114,
101,118,
115,61,116,
114,117,
101,38,97,
116,116,97,
99,104,109,
101,110,
116,115,61,
116,114,
117,101,38,
108,97,116,
101,115,
[jira] Commented: (COUCHDB-270) Replication w/ Large Attachments Fails
[
https://issues.apache.org/jira/browse/COUCHDB-270?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12678323#action_12678323
]
Jeff Hinrichs commented on COUCHDB-270:
---
Adam, I am reproducing the similar results in my environment (laptop).
Although, with less memory??, neither 20M is passing for me.
200x20m push FAIL (connection refused)
200x20m pull FAIL
Thank you for your work on this.
Now that you are getting closer, should I update the tests to use proper
attachments in addition to extremely large documents? If so, I was thinking
tests for documents with a 512K body size +:
* one massive attachment (20MB)
* three attachments (10M/6M/4M) 20M overall in attachments
* a dozen attachments (5M/5M/10x1M) 20M overall in attachments
If you would like different/additional tests/parameters let me know and I'll
update the script to include them.
Now that I am on my second cup of joe, I should probably do a test that focuses
on a large number of revisions. Currently the tests create databases that are
relatively pristine with N revisions, where N is 0 or very small. If you would
like this in the tests what N or Ns should be used?
Jeff
Replication w/ Large Attachments Fails
--
Key: COUCHDB-270
URL: https://issues.apache.org/jira/browse/COUCHDB-270
Project: CouchDB
Issue Type: Bug
Components: Database Core
Affects Versions: 0.9
Environment: Apache CouchDB 0.9.0a748379
Reporter: Jeff Hinrichs
Attachments: couchdb270_Test.py, quick_fix.diff
Attempting to replicate a database with largish attachments (= ~18MB of
attachments in a doc, less thatn 200 docs) from one machine to another fails
consistently and at the same point.
Scenario:
Both servers are running from HEAD and I've been tracking for some time.
This problem has been around as long as I've been using couch.
Machine A holds the original database, Machine B is the server that is doing
a PULL replication
During the replication, Machine A starts showing the following sporadically
in the log:
[Fri, 27 Feb 2009 14:02:48 GMT] [debug] [0.5902.3] 'GET'
/delasco-invoices/INV00652429?revs=trueattachments=truelatest=trueopen_revs=[425644723]
{1,
1}
Headers: [{'Host',192.168.2.52:5984}]
[Fri, 27 Feb 2009 14:02:48 GMT] [error] [0.5901.3] Uncaught error in
HTTP request: {exit,normal}
[Fri, 27 Feb 2009 14:02:48 GMT] [debug] [0.5901.3] Stacktrace:
[{mochiweb_request,send,2},
{couch_httpd,send_chunk,2},
{couch_httpd_db,db_doc_req,3},
{couch_httpd_db,do_db_req,2},
{couch_httpd,handle_request,3},
{mochiweb_http,headers,5},
{proc_lib,init_p,5}]
[Fri, 27 Feb 2009 14:02:48 GMT] [debug] [0.5901.3] HTTPd 500 error response:
{error:error,reason:normal}
As the replication continues, the frequency of these error Uncaught error in
HTTP request: {exit,normal} increase. Until the error is being constantly
repeated. Then Machine B stops sending requests, no more log output, no
errors, the last thing in Machine B's log file is:
[Fri, 27 Feb 2009 14:03:24 GMT] [info] [0.20893.1] retrying
couch_rep HTTP get request due to {error, req_timedout}: [104,116,
116,112,58,
47,47,49,
57,50,46,
49,54,56,
46,50,46,
53,50,58,
53,57,56,
52,47,100,
101,108,97,
115,99,111,
45,105,110,
118,111,
105,99,101,
115,47,73,
78,86,48,
48,54,53,
50,49,51,
56,63,114,
101,118,
115,61,116,
[jira] Updated: (COUCHDB-270) Replication w/ Large Attachments Fails
[
https://issues.apache.org/jira/browse/COUCHDB-270?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jeff Hinrichs updated COUCHDB-270:
--
Attachment: couchdb270_Test.py
The attached script can produce a number of errors in replication. Hopefully a
python based script is helpful for you. The script requires couchdb-python 0.5
and nose (to run the tests)
One thing I have discovered is that the replication issues are not limited to
attachments but are related to overall document size. I am creating these
replication issues by only using docs of large size.
You will need to modify the top of the script
srvAuri = 'http://192.168.2.52:5984/'
srvBuri = 'http://192.168.2.194:5984/'
to make sense for your environment
The bigger the document size the harder/faster couchdb dies. 1MB documents are
enough to make it groan before going away, 10MB will occasionally garner a
wimpering before death -- though not always, while 20MB documents are akin to a
head shot.
Replication w/ Large Attachments Fails
--
Key: COUCHDB-270
URL: https://issues.apache.org/jira/browse/COUCHDB-270
Project: CouchDB
Issue Type: Bug
Components: Database Core
Affects Versions: 0.9
Environment: Apache CouchDB 0.9.0a748379
Reporter: Jeff Hinrichs
Attachments: couchdb270_Test.py
Attempting to replicate a database with largish attachments (= ~18MB of
attachments in a doc, less thatn 200 docs) from one machine to another fails
consistently and at the same point.
Scenario:
Both servers are running from HEAD and I've been tracking for some time.
This problem has been around as long as I've been using couch.
Machine A holds the original database, Machine B is the server that is doing
a PULL replication
During the replication, Machine A starts showing the following sporadically
in the log:
[Fri, 27 Feb 2009 14:02:48 GMT] [debug] [0.5902.3] 'GET'
/delasco-invoices/INV00652429?revs=trueattachments=truelatest=trueopen_revs=[425644723]
{1,
1}
Headers: [{'Host',192.168.2.52:5984}]
[Fri, 27 Feb 2009 14:02:48 GMT] [error] [0.5901.3] Uncaught error in
HTTP request: {exit,normal}
[Fri, 27 Feb 2009 14:02:48 GMT] [debug] [0.5901.3] Stacktrace:
[{mochiweb_request,send,2},
{couch_httpd,send_chunk,2},
{couch_httpd_db,db_doc_req,3},
{couch_httpd_db,do_db_req,2},
{couch_httpd,handle_request,3},
{mochiweb_http,headers,5},
{proc_lib,init_p,5}]
[Fri, 27 Feb 2009 14:02:48 GMT] [debug] [0.5901.3] HTTPd 500 error response:
{error:error,reason:normal}
As the replication continues, the frequency of these error Uncaught error in
HTTP request: {exit,normal} increase. Until the error is being constantly
repeated. Then Machine B stops sending requests, no more log output, no
errors, the last thing in Machine B's log file is:
[Fri, 27 Feb 2009 14:03:24 GMT] [info] [0.20893.1] retrying
couch_rep HTTP get request due to {error, req_timedout}: [104,116,
116,112,58,
47,47,49,
57,50,46,
49,54,56,
46,50,46,
53,50,58,
53,57,56,
52,47,100,
101,108,97,
115,99,111,
45,105,110,
118,111,
105,99,101,
115,47,73,
78,86,48,
48,54,53,
50,49,51,
56,63,114,
101,118,
115,61,116,
114,117,
101,38,97,
116,116,97,
[jira] Created: (COUCHDB-266) PUTting json docs 1MB causes Uncaught error in HTTP request: {exit,{body_too_large,content_length}}
PUTting json docs 1MB causes Uncaught error in HTTP request:
{exit,{body_too_large,content_length}}
--
Key: COUCHDB-266
URL: https://issues.apache.org/jira/browse/COUCHDB-266
Project: CouchDB
Issue Type: Bug
Components: HTTP Interface
Affects Versions: 0.9
Environment: Apache CouchDB 0.9.0a747258
Reporter: Jeff Hinrichs
error displays itself when trying to PUT a json document that is 1MB. First
noticed in the python interface, confirmed with curl
[Tue, 24 Feb 2009 13:30:00 GMT] [error] [0.1113.0] Uncaught error in HTTP
request: {exit,{body_too_large,content_length}}
2
3 [Tue, 24 Feb 2009 13:30:00 GMT] [debug] [0.1113.0] Stacktrace:
[{mochiweb_request,stream_body,5},
4 {mochiweb_request,recv_body,2},
5 {couch_httpd,json_body,1},
6 {couch_httpd_db,db_doc_req,3},
7 {couch_httpd_db,do_db_req,2},
8 {couch_httpd,handle_request,3},
9 {mochiweb_http,headers,4},
10 {proc_lib,init_p,5}]
modifying src/mochiweb/mochiweb_request.erl -define(MAX_RECV_BODY,
(1024*1024))
to something bigger, say -define(MAX_RECV_BODY, (1024*1024*16))
alleviates the problem temporarily.
issue confirmed by cmlenz on irc, he believed it to be a regression
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Created: (COUCHDB-209) Refine httpd_db_handlers API to map the externals from paths directly to scripts
Refine httpd_db_handlers API to map the externals from paths directly to scripts
Key: COUCHDB-209
URL: https://issues.apache.org/jira/browse/COUCHDB-209
Project: CouchDB
Issue Type: Improvement
Components: HTTP Interface
Affects Versions: 0.9
Environment: all
Reporter: Jeff Hinrichs
We could change the API to map the externals from paths directly to
scripts, like
[httpd_db_handlers]
_mypath = {couch_httpd_external, handle_external_req, /path/to/my/script}
which would be fine by me.
The current code is like it is because the original implementation was
designed to have multiple scripts mounted at the /db/_external path.
Do you mind opening a ticket about this? - I'm happy to write the code
but I'm supposed to be working on the book right now, so it'll have to
wait.
link to mail list thread:
http://mail-archives.apache.org/mod_mbox/couchdb-user/200901.mbox/%3c5aaed53f0901120631v112916eewcc50e96c44728...@mail.gmail.com%3e
It would appear to be a good solution to allow the flexibility desired while
narrowing the number of local.ini edits to accomplish.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
