[jira] [Created] (COUCHDB-2634) View server errors are incomplete
Eli Stevens created COUCHDB-2634: Summary: View server errors are incomplete Key: COUCHDB-2634 URL: https://issues.apache.org/jira/browse/COUCHDB-2634 Project: CouchDB Issue Type: Bug Security Level: public (Regular issues) Components: JavaScript View Server Reporter: Eli Stevens Current 1.6 view server error messages look like: [Mon, 09 Mar 2015 02:29:04 GMT] [info] [0.25727.141] OS Process #Port0.3191211 Log :: function raised exception (new TypeError(doc.attr is undefined, undefined, 3)) with doc._id some_doc_id This doesn't say which DB, design doc, or view is causing the issue, which makes it very difficult to debug. Please add those three items to the log message. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (COUCHDB-2304) changes_reader_died when trying to replicate 25mb DB locally
[
https://issues.apache.org/jira/browse/COUCHDB-2304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14111082#comment-14111082
]
Eli Stevens commented on COUCHDB-2304:
--
There's also a 46MB document, and deleting it makes the views, etc. work again.
And yes, when we've had large documents in the past, the error message
manifested differently. It's not clear to me why this one presented as it did
(though we have changed our replication strategy since we last encountered this
class of issues, so that might be it).
changes_reader_died when trying to replicate 25mb DB locally
Key: COUCHDB-2304
URL: https://issues.apache.org/jira/browse/COUCHDB-2304
Project: CouchDB
Issue Type: Bug
Security Level: public(Regular issues)
Components: Replication
Reporter: Eli Stevens
Attachments: couch.error.log
I get the following every time I try to replicate this DB under 1.5.0
(installed from the DCH Ubuntu PPA):
{code}
elis@build6:~$ time curl -X POST -H Content-Type: application/json -d
'{source:elisdev_tanker__linac_tps_2014_08_25_12_38_37_775791_5c6c80e1b17771208e9402b49beb6241,target:elisdev_tmp,create_target:true,filter:static/content}'
http://localhost:5984/_replicate
{error:changes_reader_died}
real0m4.278s
user0m0.008s
sys 0m0.004s
{code}
In addition, trying to access the
_design/ui/_view/requestCid-attachmentFilename-contentType--to--null view in
Futon results in this log file entry:
{code}
[Mon, 25 Aug 2014 22:56:12 GMT] [info] [0.8054.2] Opening index for db:
elisdev_tanker__linac_tps_2014_08_25_12_38_37_775791_5c6c80e1b17771208e9402b49beb6241
idx: _design/ui sig: 78a645c43cae59e1ea40a75e34b65629
[Mon, 25 Aug 2014 22:56:12 GMT] [info] [0.8059.2] Starting index update for
db:
elisdev_tanker__linac_tps_2014_08_25_12_38_37_775791_5c6c80e1b17771208e9402b49beb6241
idx: _design/ui
[Mon, 25 Aug 2014 22:56:12 GMT] [info] [0.7589.2] 172.22.22.5 - - GET
/_config/native_query_servers/ 200
[Mon, 25 Aug 2014 22:56:15 GMT] [info] [0.7589.2] 172.22.22.5 - - GET
/_utils/image/grippie.gif 200
[Mon, 25 Aug 2014 22:56:16 GMT] [error] [0.8064.2] OS Process Error
0.8067.2 :: {os_process_error,
{exit_status,1}}
[Mon, 25 Aug 2014 22:56:16 GMT] [error] [emulator] Error in process
0.8064.2 with exit value:
{{nocatch,{os_process_error,{exit_status,1}}},[{couch_os_process,prompt,2},{couch_query_servers,map_doc_raw,2},{couch_mrview_updater,'-map_docs/2-fun-0-',3},{lists,foldl,3},{couch_mrview_updater,map_docs,2}]}
[Mon, 25 Aug 2014 22:56:16 GMT] [info] [0.7264.2] 172.22.22.5 - - GET
/elisdev_tanker__linac_tps_2014_08_25_12_38_37_775791_5c6c80e1b17771208e9402b49beb6241/_design/ui/_view/requestCid-attachmentFilename-contentType--to--null?limit=101
500
[Mon, 25 Aug 2014 22:56:16 GMT] [error] [0.7264.2] httpd 500 error response:
{error:os_process_error,reason:{exit_status,1}}
{code}
Creating this DB is non-trivial, but we have the capability to do so
repeatedly by running our product on a certain dataset.
The DB and couch.log excerpt will be attached shortly.
Edit: .couch file is too large to attach:
https://www.dropbox.com/s/o492tun8vn01xjx/elisdev_tanker__linac_tps_2014_08_25_12_38_37_775791_5c6c80e1b17771208e9402b49beb6241.couch?dl=0
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Created] (COUCHDB-2304) changes_reader_died when trying to replicate 25mb DB locally
Eli Stevens created COUCHDB-2304:
Summary: changes_reader_died when trying to replicate 25mb DB
locally
Key: COUCHDB-2304
URL: https://issues.apache.org/jira/browse/COUCHDB-2304
Project: CouchDB
Issue Type: Bug
Security Level: public (Regular issues)
Components: Replication
Reporter: Eli Stevens
I get the following every time I try to replicate this DB under 1.5.0
(installed from the DCH Ubuntu PPA):
{code}
elis@build6:~$ time curl -X POST -H Content-Type: application/json -d
'{source:elisdev_tanker__linac_tps_2014_08_25_12_38_37_775791_5c6c80e1b17771208e9402b49beb6241,target:elisdev_tmp,create_target:true,filter:static/content}'
http://localhost:5984/_replicate
{error:changes_reader_died}
real0m4.278s
user0m0.008s
sys 0m0.004s
{code}
In addition, trying to access the
_design/ui/_view/requestCid-attachmentFilename-contentType--to--null view in
Futon results in this log file entry:
{code}
[Mon, 25 Aug 2014 22:56:12 GMT] [info] [0.8054.2] Opening index for db:
elisdev_tanker__linac_tps_2014_08_25_12_38_37_775791_5c6c80e1b17771208e9402b49beb6241
idx: _design/ui sig: 78a645c43cae59e1ea40a75e34b65629
[Mon, 25 Aug 2014 22:56:12 GMT] [info] [0.8059.2] Starting index update for
db:
elisdev_tanker__linac_tps_2014_08_25_12_38_37_775791_5c6c80e1b17771208e9402b49beb6241
idx: _design/ui
[Mon, 25 Aug 2014 22:56:12 GMT] [info] [0.7589.2] 172.22.22.5 - - GET
/_config/native_query_servers/ 200
[Mon, 25 Aug 2014 22:56:15 GMT] [info] [0.7589.2] 172.22.22.5 - - GET
/_utils/image/grippie.gif 200
[Mon, 25 Aug 2014 22:56:16 GMT] [error] [0.8064.2] OS Process Error
0.8067.2 :: {os_process_error,
{exit_status,1}}
[Mon, 25 Aug 2014 22:56:16 GMT] [error] [emulator] Error in process 0.8064.2
with exit value:
{{nocatch,{os_process_error,{exit_status,1}}},[{couch_os_process,prompt,2},{couch_query_servers,map_doc_raw,2},{couch_mrview_updater,'-map_docs/2-fun-0-',3},{lists,foldl,3},{couch_mrview_updater,map_docs,2}]}
[Mon, 25 Aug 2014 22:56:16 GMT] [info] [0.7264.2] 172.22.22.5 - - GET
/elisdev_tanker__linac_tps_2014_08_25_12_38_37_775791_5c6c80e1b17771208e9402b49beb6241/_design/ui/_view/requestCid-attachmentFilename-contentType--to--null?limit=101
500
[Mon, 25 Aug 2014 22:56:16 GMT] [error] [0.7264.2] httpd 500 error response:
{error:os_process_error,reason:{exit_status,1}}
{code}
Creating this DB is non-trivial, but we have the capability to do so repeatedly
by running our product on a certain dataset.
The DB and couch.log excerpt will be attached shortly.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (COUCHDB-2304) changes_reader_died when trying to replicate 25mb DB locally
[
https://issues.apache.org/jira/browse/COUCHDB-2304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Eli Stevens updated COUCHDB-2304:
-
Attachment: couch.error.log
changes_reader_died when trying to replicate 25mb DB locally
Key: COUCHDB-2304
URL: https://issues.apache.org/jira/browse/COUCHDB-2304
Project: CouchDB
Issue Type: Bug
Security Level: public(Regular issues)
Components: Replication
Reporter: Eli Stevens
Attachments: couch.error.log
I get the following every time I try to replicate this DB under 1.5.0
(installed from the DCH Ubuntu PPA):
{code}
elis@build6:~$ time curl -X POST -H Content-Type: application/json -d
'{source:elisdev_tanker__linac_tps_2014_08_25_12_38_37_775791_5c6c80e1b17771208e9402b49beb6241,target:elisdev_tmp,create_target:true,filter:static/content}'
http://localhost:5984/_replicate
{error:changes_reader_died}
real0m4.278s
user0m0.008s
sys 0m0.004s
{code}
In addition, trying to access the
_design/ui/_view/requestCid-attachmentFilename-contentType--to--null view in
Futon results in this log file entry:
{code}
[Mon, 25 Aug 2014 22:56:12 GMT] [info] [0.8054.2] Opening index for db:
elisdev_tanker__linac_tps_2014_08_25_12_38_37_775791_5c6c80e1b17771208e9402b49beb6241
idx: _design/ui sig: 78a645c43cae59e1ea40a75e34b65629
[Mon, 25 Aug 2014 22:56:12 GMT] [info] [0.8059.2] Starting index update for
db:
elisdev_tanker__linac_tps_2014_08_25_12_38_37_775791_5c6c80e1b17771208e9402b49beb6241
idx: _design/ui
[Mon, 25 Aug 2014 22:56:12 GMT] [info] [0.7589.2] 172.22.22.5 - - GET
/_config/native_query_servers/ 200
[Mon, 25 Aug 2014 22:56:15 GMT] [info] [0.7589.2] 172.22.22.5 - - GET
/_utils/image/grippie.gif 200
[Mon, 25 Aug 2014 22:56:16 GMT] [error] [0.8064.2] OS Process Error
0.8067.2 :: {os_process_error,
{exit_status,1}}
[Mon, 25 Aug 2014 22:56:16 GMT] [error] [emulator] Error in process
0.8064.2 with exit value:
{{nocatch,{os_process_error,{exit_status,1}}},[{couch_os_process,prompt,2},{couch_query_servers,map_doc_raw,2},{couch_mrview_updater,'-map_docs/2-fun-0-',3},{lists,foldl,3},{couch_mrview_updater,map_docs,2}]}
[Mon, 25 Aug 2014 22:56:16 GMT] [info] [0.7264.2] 172.22.22.5 - - GET
/elisdev_tanker__linac_tps_2014_08_25_12_38_37_775791_5c6c80e1b17771208e9402b49beb6241/_design/ui/_view/requestCid-attachmentFilename-contentType--to--null?limit=101
500
[Mon, 25 Aug 2014 22:56:16 GMT] [error] [0.7264.2] httpd 500 error response:
{error:os_process_error,reason:{exit_status,1}}
{code}
Creating this DB is non-trivial, but we have the capability to do so
repeatedly by running our product on a certain dataset.
The DB and couch.log excerpt will be attached shortly.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (COUCHDB-2304) changes_reader_died when trying to replicate 25mb DB locally
[
https://issues.apache.org/jira/browse/COUCHDB-2304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Eli Stevens updated COUCHDB-2304:
-
Description:
I get the following every time I try to replicate this DB under 1.5.0
(installed from the DCH Ubuntu PPA):
{code}
elis@build6:~$ time curl -X POST -H Content-Type: application/json -d
'{source:elisdev_tanker__linac_tps_2014_08_25_12_38_37_775791_5c6c80e1b17771208e9402b49beb6241,target:elisdev_tmp,create_target:true,filter:static/content}'
http://localhost:5984/_replicate
{error:changes_reader_died}
real0m4.278s
user0m0.008s
sys 0m0.004s
{code}
In addition, trying to access the
_design/ui/_view/requestCid-attachmentFilename-contentType--to--null view in
Futon results in this log file entry:
{code}
[Mon, 25 Aug 2014 22:56:12 GMT] [info] [0.8054.2] Opening index for db:
elisdev_tanker__linac_tps_2014_08_25_12_38_37_775791_5c6c80e1b17771208e9402b49beb6241
idx: _design/ui sig: 78a645c43cae59e1ea40a75e34b65629
[Mon, 25 Aug 2014 22:56:12 GMT] [info] [0.8059.2] Starting index update for
db:
elisdev_tanker__linac_tps_2014_08_25_12_38_37_775791_5c6c80e1b17771208e9402b49beb6241
idx: _design/ui
[Mon, 25 Aug 2014 22:56:12 GMT] [info] [0.7589.2] 172.22.22.5 - - GET
/_config/native_query_servers/ 200
[Mon, 25 Aug 2014 22:56:15 GMT] [info] [0.7589.2] 172.22.22.5 - - GET
/_utils/image/grippie.gif 200
[Mon, 25 Aug 2014 22:56:16 GMT] [error] [0.8064.2] OS Process Error
0.8067.2 :: {os_process_error,
{exit_status,1}}
[Mon, 25 Aug 2014 22:56:16 GMT] [error] [emulator] Error in process 0.8064.2
with exit value:
{{nocatch,{os_process_error,{exit_status,1}}},[{couch_os_process,prompt,2},{couch_query_servers,map_doc_raw,2},{couch_mrview_updater,'-map_docs/2-fun-0-',3},{lists,foldl,3},{couch_mrview_updater,map_docs,2}]}
[Mon, 25 Aug 2014 22:56:16 GMT] [info] [0.7264.2] 172.22.22.5 - - GET
/elisdev_tanker__linac_tps_2014_08_25_12_38_37_775791_5c6c80e1b17771208e9402b49beb6241/_design/ui/_view/requestCid-attachmentFilename-contentType--to--null?limit=101
500
[Mon, 25 Aug 2014 22:56:16 GMT] [error] [0.7264.2] httpd 500 error response:
{error:os_process_error,reason:{exit_status,1}}
{code}
Creating this DB is non-trivial, but we have the capability to do so repeatedly
by running our product on a certain dataset.
The DB and couch.log excerpt will be attached shortly.
Edit: .couch file is too large to attach:
https://www.dropbox.com/s/o492tun8vn01xjx/elisdev_tanker__linac_tps_2014_08_25_12_38_37_775791_5c6c80e1b17771208e9402b49beb6241.couch?dl=0
was:
I get the following every time I try to replicate this DB under 1.5.0
(installed from the DCH Ubuntu PPA):
{code}
elis@build6:~$ time curl -X POST -H Content-Type: application/json -d
'{source:elisdev_tanker__linac_tps_2014_08_25_12_38_37_775791_5c6c80e1b17771208e9402b49beb6241,target:elisdev_tmp,create_target:true,filter:static/content}'
http://localhost:5984/_replicate
{error:changes_reader_died}
real0m4.278s
user0m0.008s
sys 0m0.004s
{code}
In addition, trying to access the
_design/ui/_view/requestCid-attachmentFilename-contentType--to--null view in
Futon results in this log file entry:
{code}
[Mon, 25 Aug 2014 22:56:12 GMT] [info] [0.8054.2] Opening index for db:
elisdev_tanker__linac_tps_2014_08_25_12_38_37_775791_5c6c80e1b17771208e9402b49beb6241
idx: _design/ui sig: 78a645c43cae59e1ea40a75e34b65629
[Mon, 25 Aug 2014 22:56:12 GMT] [info] [0.8059.2] Starting index update for
db:
elisdev_tanker__linac_tps_2014_08_25_12_38_37_775791_5c6c80e1b17771208e9402b49beb6241
idx: _design/ui
[Mon, 25 Aug 2014 22:56:12 GMT] [info] [0.7589.2] 172.22.22.5 - - GET
/_config/native_query_servers/ 200
[Mon, 25 Aug 2014 22:56:15 GMT] [info] [0.7589.2] 172.22.22.5 - - GET
/_utils/image/grippie.gif 200
[Mon, 25 Aug 2014 22:56:16 GMT] [error] [0.8064.2] OS Process Error
0.8067.2 :: {os_process_error,
{exit_status,1}}
[Mon, 25 Aug 2014 22:56:16 GMT] [error] [emulator] Error in process 0.8064.2
with exit value:
{{nocatch,{os_process_error,{exit_status,1}}},[{couch_os_process,prompt,2},{couch_query_servers,map_doc_raw,2},{couch_mrview_updater,'-map_docs/2-fun-0-',3},{lists,foldl,3},{couch_mrview_updater,map_docs,2}]}
[Mon, 25 Aug 2014 22:56:16 GMT] [info] [0.7264.2] 172.22.22.5 - - GET
/elisdev_tanker__linac_tps_2014_08_25_12_38_37_775791_5c6c80e1b17771208e9402b49beb6241/_design/ui/_view/requestCid-attachmentFilename-contentType--to--null?limit=101
500
[Mon, 25 Aug 2014 22:56:16 GMT] [error] [0.7264.2] httpd 500 error response:
{error:os_process_error,reason:{exit_status,1}}
{code}
Creating this DB is non-trivial, but we have the capability to do so repeatedly
by running our product on a certain dataset.
The DB and couch.log excerpt will be attached shortly.
changes_reader_died when trying to replicate 25mb DB
[jira] [Commented] (COUCHDB-2304) changes_reader_died when trying to replicate 25mb DB locally
[
https://issues.apache.org/jira/browse/COUCHDB-2304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14110015#comment-14110015
]
Eli Stevens commented on COUCHDB-2304:
--
I get the same behavior under 1.6.0, FWIW (I haven't verified the content of
couch.log, but the curl command behaves the same).
changes_reader_died when trying to replicate 25mb DB locally
Key: COUCHDB-2304
URL: https://issues.apache.org/jira/browse/COUCHDB-2304
Project: CouchDB
Issue Type: Bug
Security Level: public(Regular issues)
Components: Replication
Reporter: Eli Stevens
Attachments: couch.error.log
I get the following every time I try to replicate this DB under 1.5.0
(installed from the DCH Ubuntu PPA):
{code}
elis@build6:~$ time curl -X POST -H Content-Type: application/json -d
'{source:elisdev_tanker__linac_tps_2014_08_25_12_38_37_775791_5c6c80e1b17771208e9402b49beb6241,target:elisdev_tmp,create_target:true,filter:static/content}'
http://localhost:5984/_replicate
{error:changes_reader_died}
real0m4.278s
user0m0.008s
sys 0m0.004s
{code}
In addition, trying to access the
_design/ui/_view/requestCid-attachmentFilename-contentType--to--null view in
Futon results in this log file entry:
{code}
[Mon, 25 Aug 2014 22:56:12 GMT] [info] [0.8054.2] Opening index for db:
elisdev_tanker__linac_tps_2014_08_25_12_38_37_775791_5c6c80e1b17771208e9402b49beb6241
idx: _design/ui sig: 78a645c43cae59e1ea40a75e34b65629
[Mon, 25 Aug 2014 22:56:12 GMT] [info] [0.8059.2] Starting index update for
db:
elisdev_tanker__linac_tps_2014_08_25_12_38_37_775791_5c6c80e1b17771208e9402b49beb6241
idx: _design/ui
[Mon, 25 Aug 2014 22:56:12 GMT] [info] [0.7589.2] 172.22.22.5 - - GET
/_config/native_query_servers/ 200
[Mon, 25 Aug 2014 22:56:15 GMT] [info] [0.7589.2] 172.22.22.5 - - GET
/_utils/image/grippie.gif 200
[Mon, 25 Aug 2014 22:56:16 GMT] [error] [0.8064.2] OS Process Error
0.8067.2 :: {os_process_error,
{exit_status,1}}
[Mon, 25 Aug 2014 22:56:16 GMT] [error] [emulator] Error in process
0.8064.2 with exit value:
{{nocatch,{os_process_error,{exit_status,1}}},[{couch_os_process,prompt,2},{couch_query_servers,map_doc_raw,2},{couch_mrview_updater,'-map_docs/2-fun-0-',3},{lists,foldl,3},{couch_mrview_updater,map_docs,2}]}
[Mon, 25 Aug 2014 22:56:16 GMT] [info] [0.7264.2] 172.22.22.5 - - GET
/elisdev_tanker__linac_tps_2014_08_25_12_38_37_775791_5c6c80e1b17771208e9402b49beb6241/_design/ui/_view/requestCid-attachmentFilename-contentType--to--null?limit=101
500
[Mon, 25 Aug 2014 22:56:16 GMT] [error] [0.7264.2] httpd 500 error response:
{error:os_process_error,reason:{exit_status,1}}
{code}
Creating this DB is non-trivial, but we have the capability to do so
repeatedly by running our product on a certain dataset.
The DB and couch.log excerpt will be attached shortly.
Edit: .couch file is too large to attach:
https://www.dropbox.com/s/o492tun8vn01xjx/elisdev_tanker__linac_tps_2014_08_25_12_38_37_775791_5c6c80e1b17771208e9402b49beb6241.couch?dl=0
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (COUCHDB-2304) changes_reader_died when trying to replicate 25mb DB locally
[
https://issues.apache.org/jira/browse/COUCHDB-2304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14110325#comment-14110325
]
Eli Stevens commented on COUCHDB-2304:
--
There is one document in the DB that is 1.9MB of JSON. Is that expected to be
problematic?
changes_reader_died when trying to replicate 25mb DB locally
Key: COUCHDB-2304
URL: https://issues.apache.org/jira/browse/COUCHDB-2304
Project: CouchDB
Issue Type: Bug
Security Level: public(Regular issues)
Components: Replication
Reporter: Eli Stevens
Attachments: couch.error.log
I get the following every time I try to replicate this DB under 1.5.0
(installed from the DCH Ubuntu PPA):
{code}
elis@build6:~$ time curl -X POST -H Content-Type: application/json -d
'{source:elisdev_tanker__linac_tps_2014_08_25_12_38_37_775791_5c6c80e1b17771208e9402b49beb6241,target:elisdev_tmp,create_target:true,filter:static/content}'
http://localhost:5984/_replicate
{error:changes_reader_died}
real0m4.278s
user0m0.008s
sys 0m0.004s
{code}
In addition, trying to access the
_design/ui/_view/requestCid-attachmentFilename-contentType--to--null view in
Futon results in this log file entry:
{code}
[Mon, 25 Aug 2014 22:56:12 GMT] [info] [0.8054.2] Opening index for db:
elisdev_tanker__linac_tps_2014_08_25_12_38_37_775791_5c6c80e1b17771208e9402b49beb6241
idx: _design/ui sig: 78a645c43cae59e1ea40a75e34b65629
[Mon, 25 Aug 2014 22:56:12 GMT] [info] [0.8059.2] Starting index update for
db:
elisdev_tanker__linac_tps_2014_08_25_12_38_37_775791_5c6c80e1b17771208e9402b49beb6241
idx: _design/ui
[Mon, 25 Aug 2014 22:56:12 GMT] [info] [0.7589.2] 172.22.22.5 - - GET
/_config/native_query_servers/ 200
[Mon, 25 Aug 2014 22:56:15 GMT] [info] [0.7589.2] 172.22.22.5 - - GET
/_utils/image/grippie.gif 200
[Mon, 25 Aug 2014 22:56:16 GMT] [error] [0.8064.2] OS Process Error
0.8067.2 :: {os_process_error,
{exit_status,1}}
[Mon, 25 Aug 2014 22:56:16 GMT] [error] [emulator] Error in process
0.8064.2 with exit value:
{{nocatch,{os_process_error,{exit_status,1}}},[{couch_os_process,prompt,2},{couch_query_servers,map_doc_raw,2},{couch_mrview_updater,'-map_docs/2-fun-0-',3},{lists,foldl,3},{couch_mrview_updater,map_docs,2}]}
[Mon, 25 Aug 2014 22:56:16 GMT] [info] [0.7264.2] 172.22.22.5 - - GET
/elisdev_tanker__linac_tps_2014_08_25_12_38_37_775791_5c6c80e1b17771208e9402b49beb6241/_design/ui/_view/requestCid-attachmentFilename-contentType--to--null?limit=101
500
[Mon, 25 Aug 2014 22:56:16 GMT] [error] [0.7264.2] httpd 500 error response:
{error:os_process_error,reason:{exit_status,1}}
{code}
Creating this DB is non-trivial, but we have the capability to do so
repeatedly by running our product on a certain dataset.
The DB and couch.log excerpt will be attached shortly.
Edit: .couch file is too large to attach:
https://www.dropbox.com/s/o492tun8vn01xjx/elisdev_tanker__linac_tps_2014_08_25_12_38_37_775791_5c6c80e1b17771208e9402b49beb6241.couch?dl=0
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (COUCHDB-2304) changes_reader_died when trying to replicate 25mb DB locally
[
https://issues.apache.org/jira/browse/COUCHDB-2304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14110331#comment-14110331
]
Eli Stevens commented on COUCHDB-2304:
--
The CouchDB server is working fine otherwise; other data runs execute as
expected.
changes_reader_died when trying to replicate 25mb DB locally
Key: COUCHDB-2304
URL: https://issues.apache.org/jira/browse/COUCHDB-2304
Project: CouchDB
Issue Type: Bug
Security Level: public(Regular issues)
Components: Replication
Reporter: Eli Stevens
Attachments: couch.error.log
I get the following every time I try to replicate this DB under 1.5.0
(installed from the DCH Ubuntu PPA):
{code}
elis@build6:~$ time curl -X POST -H Content-Type: application/json -d
'{source:elisdev_tanker__linac_tps_2014_08_25_12_38_37_775791_5c6c80e1b17771208e9402b49beb6241,target:elisdev_tmp,create_target:true,filter:static/content}'
http://localhost:5984/_replicate
{error:changes_reader_died}
real0m4.278s
user0m0.008s
sys 0m0.004s
{code}
In addition, trying to access the
_design/ui/_view/requestCid-attachmentFilename-contentType--to--null view in
Futon results in this log file entry:
{code}
[Mon, 25 Aug 2014 22:56:12 GMT] [info] [0.8054.2] Opening index for db:
elisdev_tanker__linac_tps_2014_08_25_12_38_37_775791_5c6c80e1b17771208e9402b49beb6241
idx: _design/ui sig: 78a645c43cae59e1ea40a75e34b65629
[Mon, 25 Aug 2014 22:56:12 GMT] [info] [0.8059.2] Starting index update for
db:
elisdev_tanker__linac_tps_2014_08_25_12_38_37_775791_5c6c80e1b17771208e9402b49beb6241
idx: _design/ui
[Mon, 25 Aug 2014 22:56:12 GMT] [info] [0.7589.2] 172.22.22.5 - - GET
/_config/native_query_servers/ 200
[Mon, 25 Aug 2014 22:56:15 GMT] [info] [0.7589.2] 172.22.22.5 - - GET
/_utils/image/grippie.gif 200
[Mon, 25 Aug 2014 22:56:16 GMT] [error] [0.8064.2] OS Process Error
0.8067.2 :: {os_process_error,
{exit_status,1}}
[Mon, 25 Aug 2014 22:56:16 GMT] [error] [emulator] Error in process
0.8064.2 with exit value:
{{nocatch,{os_process_error,{exit_status,1}}},[{couch_os_process,prompt,2},{couch_query_servers,map_doc_raw,2},{couch_mrview_updater,'-map_docs/2-fun-0-',3},{lists,foldl,3},{couch_mrview_updater,map_docs,2}]}
[Mon, 25 Aug 2014 22:56:16 GMT] [info] [0.7264.2] 172.22.22.5 - - GET
/elisdev_tanker__linac_tps_2014_08_25_12_38_37_775791_5c6c80e1b17771208e9402b49beb6241/_design/ui/_view/requestCid-attachmentFilename-contentType--to--null?limit=101
500
[Mon, 25 Aug 2014 22:56:16 GMT] [error] [0.7264.2] httpd 500 error response:
{error:os_process_error,reason:{exit_status,1}}
{code}
Creating this DB is non-trivial, but we have the capability to do so
repeatedly by running our product on a certain dataset.
The DB and couch.log excerpt will be attached shortly.
Edit: .couch file is too large to attach:
https://www.dropbox.com/s/o492tun8vn01xjx/elisdev_tanker__linac_tps_2014_08_25_12_38_37_775791_5c6c80e1b17771208e9402b49beb6241.couch?dl=0
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (COUCHDB-1515) _stats: open_databases and open_os_files are zero if queried with a range-parameter
[
https://issues.apache.org/jira/browse/COUCHDB-1515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13999234#comment-13999234
]
Eli Stevens commented on COUCHDB-1515:
--
CouchDB 1.5.0 (from the Ubuntu 12.04 LTS ppa built by dch):
http://localhost:15984/_stats/couchdb/open_databases?range=60
{
couchdb: {
open_databases: {
description: number of open databases,
current: -59,
sum: -59,
mean: 2.966,
stddev: 26.009,
min: -181,
max: 72
}
}
}
Kxepal on IRC says that querying this with ?range= doesn't make sense, since
these are absolute values. http://wiki.apache.org/couchdb/Runtime_Statistics
doesn't call out any keys as being special, and
http://couchdb.readthedocs.org/en/latest/api/server/common.html#stats doesn't
mention range at all. It is unclear what's supposed to happen here.
Querying without range:
{
couchdb: {
open_databases: {
description: number of open databases,
current: 137,
sum: 137,
mean: 0.028,
stddev: 7.227,
min: -227,
max: 89
}
}
}
Which also doesn't make sense. How is min negative? How is max less than
current?
_stats: open_databases and open_os_files are zero if queried with a
range-parameter
---
Key: COUCHDB-1515
URL: https://issues.apache.org/jira/browse/COUCHDB-1515
Project: CouchDB
Issue Type: Bug
Affects Versions: 1.2
Environment: Mac OS X (built with homebrew), Ubuntu 12.04 Server
(build with build-couchdb)
Reporter: Andreas Lappe
Priority: Minor
I don't know if this intentionally but I thought it would make sense that
couchdb also reports the number of open databases and files in the last
specified range…
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Created] (COUCHDB-2239) Log error XXX database out of sync. Try to increase max_dbs_open at the YYY's server wrong
Eli Stevens created COUCHDB-2239: Summary: Log error XXX database out of sync. Try to increase max_dbs_open at the YYY's server wrong Key: COUCHDB-2239 URL: https://issues.apache.org/jira/browse/COUCHDB-2239 Project: CouchDB Issue Type: Bug Security Level: public (Regular issues) Reporter: Eli Stevens After inquiring about the above error message, and linking http://stackoverflow.com/questions/13403295/why-after-increasing-max-dbs-open-do-replications-still-fail-with-increase-ma in IRC, I was pointed to: https://github.com/apache/couchdb/blob/master/src/couch_replicator/src/couch_replicator.erl#L776 And told that the exception catching there is over-broad for the error message it produces. The error message should be correct for the situation. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Created] (COUCHDB-2240) Many continuous replications cause DOS
Eli Stevens created COUCHDB-2240:
Summary: Many continuous replications cause DOS
Key: COUCHDB-2240
URL: https://issues.apache.org/jira/browse/COUCHDB-2240
Project: CouchDB
Issue Type: Bug
Security Level: public (Regular issues)
Reporter: Eli Stevens
Currently, I can configure an arbitrary number of replications between
localhost DBs (in my case, they are in the _replicator DB with continuous set
to true). However, there is a limit beyond which requests to the DB start to
fail. Trying to do another replication fails with the error:
ServerError: (500, ('checkpoint_commit_failure', Target database out of sync.
Try to increase max_dbs_open at the target's server.))
Due to COUCHDB-2239, it's not clear what the actual issue is.
I also believe that while the DB was in this state GET requests to documents
were also failing, but the machine that has the logs of this has already had
it's drives wiped. If need be, I can recreate the situation and provide those
logs as well.
I think that instead of there being a single fixed pool of resources that cause
errors when exhausted, the system should have a per-task-type pool of resources
that result in performance degradation when exhausted. N replication workers
with P DB connections, and if that's not enough they start to round-robin; that
sort of thing. When a user has too much to replicate, it gets slow instead of
failing.
As it stands now, I have a potentially large number of continuous replications
that produce a fixed rate of data to replicate (because there's a fixed
application worker pool that writes the data in the first place). We use a
DB+replication per batch of data to process, and if we receive a burst of
batches, then couchdb starts failing. The current setup means that I'm always
going to be playing chicken between burst size and whatever setting limit we're
hitting. That sucks, and isn't acceptable for a production system, so we're
going to have to re-architect how we do replication, and basically implement
poor-man's continuous by doing one off replications at various points of our
data processing runs.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (COUCHDB-2240) Many continuous replications cause DOS
[
https://issues.apache.org/jira/browse/COUCHDB-2240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14000535#comment-14000535
]
Eli Stevens commented on COUCHDB-2240:
--
Depends on your settings. My understanding of what's causing the issue is that
the default value of max_open_dbs is 100. I can raise that value arbitrarily
high and hope that I never need to process a burst of activity greater than my
arbitrarily high value, but that's not really desirable either, since I would
much prefer to have an arbitrarily high number of replications going that only
use a fixed pool of resources (and if those resources end up taxed, then
performance degrades).
I understand that this is not a small change that I'm suggesting.
Many continuous replications cause DOS
--
Key: COUCHDB-2240
URL: https://issues.apache.org/jira/browse/COUCHDB-2240
Project: CouchDB
Issue Type: Bug
Security Level: public(Regular issues)
Reporter: Eli Stevens
Currently, I can configure an arbitrary number of replications between
localhost DBs (in my case, they are in the _replicator DB with continuous set
to true). However, there is a limit beyond which requests to the DB start to
fail. Trying to do another replication fails with the error:
ServerError: (500, ('checkpoint_commit_failure', Target database out of
sync. Try to increase max_dbs_open at the target's server.))
Due to COUCHDB-2239, it's not clear what the actual issue is.
I also believe that while the DB was in this state GET requests to documents
were also failing, but the machine that has the logs of this has already had
it's drives wiped. If need be, I can recreate the situation and provide those
logs as well.
I think that instead of there being a single fixed pool of resources that
cause errors when exhausted, the system should have a per-task-type pool of
resources that result in performance degradation when exhausted. N
replication workers with P DB connections, and if that's not enough they
start to round-robin; that sort of thing. When a user has too much to
replicate, it gets slow instead of failing.
As it stands now, I have a potentially large number of continuous
replications that produce a fixed rate of data to replicate (because there's
a fixed application worker pool that writes the data in the first place). We
use a DB+replication per batch of data to process, and if we receive a burst
of batches, then couchdb starts failing. The current setup means that I'm
always going to be playing chicken between burst size and whatever setting
limit we're hitting. That sucks, and isn't acceptable for a production
system, so we're going to have to re-architect how we do replication, and
basically implement poor-man's continuous by doing one off replications at
various points of our data processing runs.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (COUCHDB-2218) gen_server timeout has unhelpful error message
[
https://issues.apache.org/jira/browse/COUCHDB-2218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13957192#comment-13957192
]
Eli Stevens commented on COUCHDB-2218:
--
This commit was added to address the underlying issue:
https://github.com/apache/couchdb/commit/d81619033206f09a774072dc2f84f7f275b12496
Can progress be made when this occurs? IOW, is it possible for retrying to
eventually work through task and allow the HTTP API call to succeed?
gen_server timeout has unhelpful error message
--
Key: COUCHDB-2218
URL: https://issues.apache.org/jira/browse/COUCHDB-2218
Project: CouchDB
Issue Type: Bug
Security Level: public(Regular issues)
Components: JavaScript View Server
Reporter: Eli Stevens
It's not clear if this error is something that's actionable by the user or
not:
https://gist.github.com/wickedgrey/d2c87e6ee1e7c695abae
It sounds like there's a bug (sloppy) here, but I'm not certain.
From IRC discussion:
4:40 PM rnewson: oh that's annoying
4:40 PM Wohali: gen_server timeout
4:40 PM rnewson: yeah, do we do a default timeout there for a view build?
that's sloppy
4:40 PM Wohali: i bet, as it's not an os_process timeout
4:41 PM Wohali: which is what i would have expected to see
4:42 PM rnewson:
run(Pid, IdxState) -
gen_server:call(Pid, {update, IdxState}).
4:42 PM wickedgrey: the machine was in the middle of a test run at the time,
which processes a lot of data under somewhat high load (roughly
O(num_cores)), but we do this regularly and don't usually see this
4:42 PM rnewson:
couch_index_updater
ah, this one;
update(Mod, State) -
update(nil, Mod, State).
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Created] (COUCHDB-2218) gen_server timeout has unhelpful error message
Eli Stevens created COUCHDB-2218:
Summary: gen_server timeout has unhelpful error message
Key: COUCHDB-2218
URL: https://issues.apache.org/jira/browse/COUCHDB-2218
Project: CouchDB
Issue Type: Bug
Security Level: public (Regular issues)
Components: JavaScript View Server
Reporter: Eli Stevens
It's not clear if this error is something that's actionable by the user or not:
https://gist.github.com/wickedgrey/d2c87e6ee1e7c695abae
It sounds like there's a bug (sloppy) here, but I'm not certain.
From IRC discussion:
4:40 PM rnewson: oh that's annoying
4:40 PM Wohali: gen_server timeout
4:40 PM rnewson: yeah, do we do a default timeout there for a view build?
that's sloppy
4:40 PM Wohali: i bet, as it's not an os_process timeout
4:41 PM Wohali: which is what i would have expected to see
4:42 PM rnewson:
run(Pid, IdxState) -
gen_server:call(Pid, {update, IdxState}).
4:42 PM wickedgrey: the machine was in the middle of a test run at the time,
which processes a lot of data under somewhat high load (roughly O(num_cores)),
but we do this regularly and don't usually see this
4:42 PM rnewson:
couch_index_updater
ah, this one;
update(Mod, State) -
update(nil, Mod, State).
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (COUCHDB-2035) Stop user from removing _id and _rev from doc
[ https://issues.apache.org/jira/browse/COUCHDB-2035?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13878886#comment-13878886 ] Eli Stevens commented on COUCHDB-2035: -- A use case to consider: I will often edit a doc in futon, remove the _rev, change the _id, and re-save (typically because the doc is complex and I would like to have a near-copy). It's not clear from the above if the intent is to remove the ability to do that, but it's a use case that's useful to at least this user. Thanks! Stop user from removing _id and _rev from doc - Key: COUCHDB-2035 URL: https://issues.apache.org/jira/browse/COUCHDB-2035 Project: CouchDB Issue Type: New Feature Reporter: Garren Smith Assignee: Garren Smith When a user is editing a doc, warn them and stop them from being able to remove the _id and _rev from a doc. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (COUCHDB-2007) Building docs under CI fails
[
https://issues.apache.org/jira/browse/COUCHDB-2007?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13853160#comment-13853160
]
Eli Stevens commented on COUCHDB-2007:
--
I'm not 100% certain that I fully understand what parts of the final text came
from which subkey of _info, but I think that you could do something like this
(starting at line 55):
if _info.get('LOCAL_VERSION_RELEASE') == '.%revision%':
release += '-dev'
elif _info.get('LOCAL_VERSION_RELEASE'):
if _info['LOCAL_VERSION_STAGE'].find('jenkins') 0:
release += '.' + _info['LOCAL_VERSION_RELEASE'][-9:]
else:
release += _info['LOCAL_VERSION_STAGE'] + _info['LOCAL_VERSION_RELEASE']
Building docs under CI fails
Key: COUCHDB-2007
URL: https://issues.apache.org/jira/browse/COUCHDB-2007
Project: CouchDB
Issue Type: Bug
Components: Build System, Documentation
Reporter: Jan Lehnardt
Building the PDF version of the docs fails under CI due to release name:
The CouchDB.tex file that gets generated includes a line that looks like this:
\release{1.6.0+build.jenkins-ERLANG_VERSION=R14B04,label=Mac-OS-10-8-2-832-76-g2996574}
It isn’t clear whether this is plain to long or whether there are invalid
characters in there.
I don’t quite understand how docs inherit the release name, so I need to ask
for help here.
The release name otherwise is valid, as it encodes the different build matrix
settings.
--
This message was sent by Atlassian JIRA
(v6.1.4#6159)
[jira] [Commented] (COUCHDB-1906) Removing DB referenced by _replicator should halt replication / remove doc
[ https://issues.apache.org/jira/browse/COUCHDB-1906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13791918#comment-13791918 ] Eli Stevens commented on COUCHDB-1906: -- I think that viewpoint is overly focused on the implementation of how persistent replication is accomplished, rather than what makes sense for the feature itself. If document removal specifically is an issue, perhaps the _replicator doc could be flagged as endpoint_removed or similar. Depending on replication topology it might be impossible for the entity doing the DB deletion to know about all of the persistent replications involved, and even if it is possible, having to perform N actions to cleanly remove a DB doesn't make sense to me when it could be done with one. Removing DB referenced by _replicator should halt replication / remove doc --- Key: COUCHDB-1906 URL: https://issues.apache.org/jira/browse/COUCHDB-1906 Project: CouchDB Issue Type: Improvement Components: Replication Reporter: Eli Stevens Removing a DB that is involved in replication via the _replicator DB should remove the corresponding _replicator document. Currently, couch.log includes errors when the DB is removed. My perspective is that replication is just a feature of a DB in the same way that attachments are; one doesn't need to separately remove attachments after deleting a DB, and similarly I've already expressed that I'm no longer interested the DB, its attachments, its replications, etc. by deleting it. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Issue Comment Deleted] (COUCHDB-1906) Removing DB referenced by _replicator should halt replication / remove doc
[ https://issues.apache.org/jira/browse/COUCHDB-1906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eli Stevens updated COUCHDB-1906: - Comment: was deleted (was: I think that viewpoint is overly focused on the implementation of how persistent replication is accomplished, rather than what makes sense for the feature itself. If document removal specifically is an issue, perhaps the _replicator doc could be flagged as endpoint_removed or similar. Depending on replication topology it might be impossible for the entity doing the DB deletion to know about all of the persistent replications involved, and even if it is possible, having to perform N actions to cleanly remove a DB doesn't make sense to me when it could be done with one. ) Removing DB referenced by _replicator should halt replication / remove doc --- Key: COUCHDB-1906 URL: https://issues.apache.org/jira/browse/COUCHDB-1906 Project: CouchDB Issue Type: Improvement Components: Replication Reporter: Eli Stevens Removing a DB that is involved in replication via the _replicator DB should remove the corresponding _replicator document. Currently, couch.log includes errors when the DB is removed. My perspective is that replication is just a feature of a DB in the same way that attachments are; one doesn't need to separately remove attachments after deleting a DB, and similarly I've already expressed that I'm no longer interested the DB, its attachments, its replications, etc. by deleting it. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (COUCHDB-1817) Exceeding the couchjs stack size does not have a clear error message
[
https://issues.apache.org/jira/browse/COUCHDB-1817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Eli Stevens updated COUCHDB-1817:
-
Summary: Exceeding the couchjs stack size does not have a clear error
message (was: OS Process Error 0.21247.103 :: {os_process_error,
{exit_status,0}})
Exceeding the couchjs stack size does not have a clear error message
Key: COUCHDB-1817
URL: https://issues.apache.org/jira/browse/COUCHDB-1817
Project: CouchDB
Issue Type: Bug
Components: JavaScript View Server
Reporter: Eli Stevens
Priority: Critical
Attachments: couchdb__couchdb_files.png,
couchdb__httpd_status_codes.png, couchdb_mem.png, loadavg.png, memory2.png
We have started seeing errors crop up in our application that we have not
seen before, and we're at a loss for how to start debugging it.
[~dch] Said that we might look into system resource limits, so we started
collecting all of the output from _stats into RRD (along with memory, load,
etc. that we were already collecting), but nothing is jumping out at us as
obviously problematic.
We can semi-reliably reproduce the problem, but it's far from a minimal test
case (basically, we load up several large chunks of data, and then halfway
through the processing run, we get the error). The error doesn't seem to
happen if we load up each chunk by itself.
The DB in question has about 100 docs in it, none particularly large (nothing
over a couple KB would be my guess), with a couple hundred MB in attachments.
10ish design docs, coffeescript. In general, there isn't anything that
seems obviously resource intensive.
We have seen this issue on 1.2.0, 1.2.1, and we're working on getting a
machine with 1.3.0 set up (the PPA we'd been using hasn't been updated yet).
Ubuntu 12.04, spinning disk, etc. The system is under load when it happens,
but the load isn't more than 1.5x the number of cores. I don't have disk IO
numbers at hand, but I'd be surprised if that was being strained.
Error as it appears in couch.log:
https://gist.github.com/wickedgrey/e7fd3fc14b6d43e95564
The design doc in question:
https://gist.github.com/wickedgrey/db41b0c3c75a590e2109
An example document: https://gist.github.com/wickedgrey/a8422aab261ddd2ce4fe
We have some preliminary evidence that the problem persists after the system
goes quiet, but we're not certain.
Either CouchDB isn't handling things correctly, in which case this bug is
prz fix or we're doing something wrong (hitting a resource limit, or
something), in which case this bug is prz make the error message more
informative.
Thanks!
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (COUCHDB-1817) Exceeding the couchjs stack size does not have a clear error message
[ https://issues.apache.org/jira/browse/COUCHDB-1817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eli Stevens updated COUCHDB-1817: - Description: Updated description: When a large document is uploaded (in this case 45MB of JSON), if the couchjs process runs out of stack, the error message that gets produced does not make it easy to debug what is going wrong. We have started seeing errors crop up in our application that we have not seen before, and we're at a loss for how to start debugging it. [~dch] Said that we might look into system resource limits, so we started collecting all of the output from _stats into RRD (along with memory, load, etc. that we were already collecting), but nothing is jumping out at us as obviously problematic. We can semi-reliably reproduce the problem, but it's far from a minimal test case (basically, we load up several large chunks of data, and then halfway through the processing run, we get the error). The error doesn't seem to happen if we load up each chunk by itself. The DB in question has about 100 docs in it, none particularly large (nothing over a couple KB would be my guess), with a couple hundred MB in attachments. 10ish design docs, coffeescript. In general, there isn't anything that seems obviously resource intensive. We have seen this issue on 1.2.0, 1.2.1, and we're working on getting a machine with 1.3.0 set up (the PPA we'd been using hasn't been updated yet). Ubuntu 12.04, spinning disk, etc. The system is under load when it happens, but the load isn't more than 1.5x the number of cores. I don't have disk IO numbers at hand, but I'd be surprised if that was being strained. Error as it appears in couch.log: https://gist.github.com/wickedgrey/e7fd3fc14b6d43e95564 The design doc in question: https://gist.github.com/wickedgrey/db41b0c3c75a590e2109 An example document: https://gist.github.com/wickedgrey/a8422aab261ddd2ce4fe We have some preliminary evidence that the problem persists after the system goes quiet, but we're not certain. Either CouchDB isn't handling things correctly, in which case this bug is prz fix or we're doing something wrong (hitting a resource limit, or something), in which case this bug is prz make the error message more informative. Thanks! was: We have started seeing errors crop up in our application that we have not seen before, and we're at a loss for how to start debugging it. [~dch] Said that we might look into system resource limits, so we started collecting all of the output from _stats into RRD (along with memory, load, etc. that we were already collecting), but nothing is jumping out at us as obviously problematic. We can semi-reliably reproduce the problem, but it's far from a minimal test case (basically, we load up several large chunks of data, and then halfway through the processing run, we get the error). The error doesn't seem to happen if we load up each chunk by itself. The DB in question has about 100 docs in it, none particularly large (nothing over a couple KB would be my guess), with a couple hundred MB in attachments. 10ish design docs, coffeescript. In general, there isn't anything that seems obviously resource intensive. We have seen this issue on 1.2.0, 1.2.1, and we're working on getting a machine with 1.3.0 set up (the PPA we'd been using hasn't been updated yet). Ubuntu 12.04, spinning disk, etc. The system is under load when it happens, but the load isn't more than 1.5x the number of cores. I don't have disk IO numbers at hand, but I'd be surprised if that was being strained. Error as it appears in couch.log: https://gist.github.com/wickedgrey/e7fd3fc14b6d43e95564 The design doc in question: https://gist.github.com/wickedgrey/db41b0c3c75a590e2109 An example document: https://gist.github.com/wickedgrey/a8422aab261ddd2ce4fe We have some preliminary evidence that the problem persists after the system goes quiet, but we're not certain. Either CouchDB isn't handling things correctly, in which case this bug is prz fix or we're doing something wrong (hitting a resource limit, or something), in which case this bug is prz make the error message more informative. Thanks! Exceeding the couchjs stack size does not have a clear error message Key: COUCHDB-1817 URL: https://issues.apache.org/jira/browse/COUCHDB-1817 Project: CouchDB Issue Type: Bug Components: JavaScript View Server Reporter: Eli Stevens Priority: Critical Attachments: couchdb__couchdb_files.png, couchdb__httpd_status_codes.png, couchdb_mem.png, loadavg.png, memory2.png Updated description: When a large document is uploaded (in this case 45MB of JSON), if the couchjs process runs out of stack, the error message that gets produced does
[jira] [Updated] (COUCHDB-1817) Exceeding the couchjs stack size does not have a clear error message
[ https://issues.apache.org/jira/browse/COUCHDB-1817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eli Stevens updated COUCHDB-1817: - Priority: Minor (was: Critical) Exceeding the couchjs stack size does not have a clear error message Key: COUCHDB-1817 URL: https://issues.apache.org/jira/browse/COUCHDB-1817 Project: CouchDB Issue Type: Bug Components: JavaScript View Server Reporter: Eli Stevens Priority: Minor Attachments: couchdb__couchdb_files.png, couchdb__httpd_status_codes.png, couchdb_mem.png, loadavg.png, memory2.png Updated description: When a large document is uploaded (in this case 45MB of JSON), if the couchjs process runs out of stack, the error message that gets produced does not make it easy to debug what is going wrong. We have started seeing errors crop up in our application that we have not seen before, and we're at a loss for how to start debugging it. [~dch] Said that we might look into system resource limits, so we started collecting all of the output from _stats into RRD (along with memory, load, etc. that we were already collecting), but nothing is jumping out at us as obviously problematic. We can semi-reliably reproduce the problem, but it's far from a minimal test case (basically, we load up several large chunks of data, and then halfway through the processing run, we get the error). The error doesn't seem to happen if we load up each chunk by itself. The DB in question has about 100 docs in it, none particularly large (nothing over a couple KB would be my guess), with a couple hundred MB in attachments. 10ish design docs, coffeescript. In general, there isn't anything that seems obviously resource intensive. We have seen this issue on 1.2.0, 1.2.1, and we're working on getting a machine with 1.3.0 set up (the PPA we'd been using hasn't been updated yet). Ubuntu 12.04, spinning disk, etc. The system is under load when it happens, but the load isn't more than 1.5x the number of cores. I don't have disk IO numbers at hand, but I'd be surprised if that was being strained. Error as it appears in couch.log: https://gist.github.com/wickedgrey/e7fd3fc14b6d43e95564 The design doc in question: https://gist.github.com/wickedgrey/db41b0c3c75a590e2109 An example document: https://gist.github.com/wickedgrey/a8422aab261ddd2ce4fe We have some preliminary evidence that the problem persists after the system goes quiet, but we're not certain. Either CouchDB isn't handling things correctly, in which case this bug is prz fix or we're doing something wrong (hitting a resource limit, or something), in which case this bug is prz make the error message more informative. Thanks! -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (COUCHDB-1817) Exceeding the couchjs stack size does not have a clear error message
[ https://issues.apache.org/jira/browse/COUCHDB-1817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eli Stevens updated COUCHDB-1817: - Description: Updated description: When a large document is uploaded and the couchjs process runs out of stack, the error message that gets produced does not make it easy to debug what is going wrong. In our case we had some data that should have been an attachment but was ending up in the document proper, which caused us to have 45MB documents, which triggered the issue. Having an error message that says which document ran the view server out of space, etc. would be nice. --- Original description: We have started seeing errors crop up in our application that we have not seen before, and we're at a loss for how to start debugging it. [~dch] Said that we might look into system resource limits, so we started collecting all of the output from _stats into RRD (along with memory, load, etc. that we were already collecting), but nothing is jumping out at us as obviously problematic. We can semi-reliably reproduce the problem, but it's far from a minimal test case (basically, we load up several large chunks of data, and then halfway through the processing run, we get the error). The error doesn't seem to happen if we load up each chunk by itself. The DB in question has about 100 docs in it, none particularly large (nothing over a couple KB would be my guess), with a couple hundred MB in attachments. 10ish design docs, coffeescript. In general, there isn't anything that seems obviously resource intensive. We have seen this issue on 1.2.0, 1.2.1, and we're working on getting a machine with 1.3.0 set up (the PPA we'd been using hasn't been updated yet). Ubuntu 12.04, spinning disk, etc. The system is under load when it happens, but the load isn't more than 1.5x the number of cores. I don't have disk IO numbers at hand, but I'd be surprised if that was being strained. Error as it appears in couch.log: https://gist.github.com/wickedgrey/e7fd3fc14b6d43e95564 The design doc in question: https://gist.github.com/wickedgrey/db41b0c3c75a590e2109 An example document: https://gist.github.com/wickedgrey/a8422aab261ddd2ce4fe We have some preliminary evidence that the problem persists after the system goes quiet, but we're not certain. Either CouchDB isn't handling things correctly, in which case this bug is prz fix or we're doing something wrong (hitting a resource limit, or something), in which case this bug is prz make the error message more informative. Thanks! was: Updated description: When a large document is uploaded (in this case 45MB of JSON), if the couchjs process runs out of stack, the error message that gets produced does not make it easy to debug what is going wrong. We have started seeing errors crop up in our application that we have not seen before, and we're at a loss for how to start debugging it. [~dch] Said that we might look into system resource limits, so we started collecting all of the output from _stats into RRD (along with memory, load, etc. that we were already collecting), but nothing is jumping out at us as obviously problematic. We can semi-reliably reproduce the problem, but it's far from a minimal test case (basically, we load up several large chunks of data, and then halfway through the processing run, we get the error). The error doesn't seem to happen if we load up each chunk by itself. The DB in question has about 100 docs in it, none particularly large (nothing over a couple KB would be my guess), with a couple hundred MB in attachments. 10ish design docs, coffeescript. In general, there isn't anything that seems obviously resource intensive. We have seen this issue on 1.2.0, 1.2.1, and we're working on getting a machine with 1.3.0 set up (the PPA we'd been using hasn't been updated yet). Ubuntu 12.04, spinning disk, etc. The system is under load when it happens, but the load isn't more than 1.5x the number of cores. I don't have disk IO numbers at hand, but I'd be surprised if that was being strained. Error as it appears in couch.log: https://gist.github.com/wickedgrey/e7fd3fc14b6d43e95564 The design doc in question: https://gist.github.com/wickedgrey/db41b0c3c75a590e2109 An example document: https://gist.github.com/wickedgrey/a8422aab261ddd2ce4fe We have some preliminary evidence that the problem persists after the system goes quiet, but we're not certain. Either CouchDB isn't handling things correctly, in which case this bug is prz fix or we're doing something wrong (hitting a resource limit, or something), in which case this bug is prz make the error message more informative. Thanks! Exceeding the couchjs stack size does not have a clear error message Key: COUCHDB-1817 URL:
[jira] [Commented] (COUCHDB-1817) Exceeding the couchjs stack size does not have a clear error message
[ https://issues.apache.org/jira/browse/COUCHDB-1817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13677314#comment-13677314 ] Eli Stevens commented on COUCHDB-1817: -- 45MB documents were running couchjs out of stack space. Removing the large documents caused the DB to be able to serve views again. See updated description. Exceeding the couchjs stack size does not have a clear error message Key: COUCHDB-1817 URL: https://issues.apache.org/jira/browse/COUCHDB-1817 Project: CouchDB Issue Type: Bug Components: JavaScript View Server Reporter: Eli Stevens Priority: Minor Attachments: couchdb__couchdb_files.png, couchdb__httpd_status_codes.png, couchdb_mem.png, loadavg.png, memory2.png Updated description: When a large document is uploaded and the couchjs process runs out of stack, the error message that gets produced does not make it easy to debug what is going wrong. In our case we had some data that should have been an attachment but was ending up in the document proper, which caused us to have 45MB documents, which triggered the issue. Having an error message that says which document ran the view server out of space, etc. would be nice. --- Original description: We have started seeing errors crop up in our application that we have not seen before, and we're at a loss for how to start debugging it. [~dch] Said that we might look into system resource limits, so we started collecting all of the output from _stats into RRD (along with memory, load, etc. that we were already collecting), but nothing is jumping out at us as obviously problematic. We can semi-reliably reproduce the problem, but it's far from a minimal test case (basically, we load up several large chunks of data, and then halfway through the processing run, we get the error). The error doesn't seem to happen if we load up each chunk by itself. The DB in question has about 100 docs in it, none particularly large (nothing over a couple KB would be my guess), with a couple hundred MB in attachments. 10ish design docs, coffeescript. In general, there isn't anything that seems obviously resource intensive. We have seen this issue on 1.2.0, 1.2.1, and we're working on getting a machine with 1.3.0 set up (the PPA we'd been using hasn't been updated yet). Ubuntu 12.04, spinning disk, etc. The system is under load when it happens, but the load isn't more than 1.5x the number of cores. I don't have disk IO numbers at hand, but I'd be surprised if that was being strained. Error as it appears in couch.log: https://gist.github.com/wickedgrey/e7fd3fc14b6d43e95564 The design doc in question: https://gist.github.com/wickedgrey/db41b0c3c75a590e2109 An example document: https://gist.github.com/wickedgrey/a8422aab261ddd2ce4fe We have some preliminary evidence that the problem persists after the system goes quiet, but we're not certain. Either CouchDB isn't handling things correctly, in which case this bug is prz fix or we're doing something wrong (hitting a resource limit, or something), in which case this bug is prz make the error message more informative. Thanks! -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (COUCHDB-1817) Exceeding the couchjs stack size does not have a clear error message
[ https://issues.apache.org/jira/browse/COUCHDB-1817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13677322#comment-13677322 ] Eli Stevens commented on COUCHDB-1817: -- Related: https://github.com/apache/couchdb/commit/501459c8b70efb814f5eef131201f7954781af1d Per conversations in IRC, apparently if the logs for couchjs could be viewed, it would make it apparent what's going on. Exceeding the couchjs stack size does not have a clear error message Key: COUCHDB-1817 URL: https://issues.apache.org/jira/browse/COUCHDB-1817 Project: CouchDB Issue Type: Bug Components: JavaScript View Server Reporter: Eli Stevens Priority: Minor Attachments: couchdb__couchdb_files.png, couchdb__httpd_status_codes.png, couchdb_mem.png, loadavg.png, memory2.png Updated description: When a large document is uploaded and the couchjs process runs out of stack, the error message that gets produced does not make it easy to debug what is going wrong. In our case we had some data that should have been an attachment but was ending up in the document proper, which caused us to have 45MB documents, which triggered the issue. Having an error message that says which document ran the view server out of space, etc. would be nice. --- Original description: We have started seeing errors crop up in our application that we have not seen before, and we're at a loss for how to start debugging it. [~dch] Said that we might look into system resource limits, so we started collecting all of the output from _stats into RRD (along with memory, load, etc. that we were already collecting), but nothing is jumping out at us as obviously problematic. We can semi-reliably reproduce the problem, but it's far from a minimal test case (basically, we load up several large chunks of data, and then halfway through the processing run, we get the error). The error doesn't seem to happen if we load up each chunk by itself. The DB in question has about 100 docs in it, none particularly large (nothing over a couple KB would be my guess), with a couple hundred MB in attachments. 10ish design docs, coffeescript. In general, there isn't anything that seems obviously resource intensive. We have seen this issue on 1.2.0, 1.2.1, and we're working on getting a machine with 1.3.0 set up (the PPA we'd been using hasn't been updated yet). Ubuntu 12.04, spinning disk, etc. The system is under load when it happens, but the load isn't more than 1.5x the number of cores. I don't have disk IO numbers at hand, but I'd be surprised if that was being strained. Error as it appears in couch.log: https://gist.github.com/wickedgrey/e7fd3fc14b6d43e95564 The design doc in question: https://gist.github.com/wickedgrey/db41b0c3c75a590e2109 An example document: https://gist.github.com/wickedgrey/a8422aab261ddd2ce4fe We have some preliminary evidence that the problem persists after the system goes quiet, but we're not certain. Either CouchDB isn't handling things correctly, in which case this bug is prz fix or we're doing something wrong (hitting a resource limit, or something), in which case this bug is prz make the error message more informative. Thanks! -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (COUCHDB-1817) OS Process Error 0.21247.103 :: {os_process_error, {exit_status,0}}
[
https://issues.apache.org/jira/browse/COUCHDB-1817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13676247#comment-13676247
]
Eli Stevens commented on COUCHDB-1817:
--
Our os_process_timeout was set to 5k. We changed it to 60k, and the next two
data runs didn't exhibit any issues (each run takes about an hour, so doing
lots of repeated trials is somewhat expensive). Given that our error rate was
about 75% previously, I'm cautiously optimistic. I'm worried that we'll run
into the non-timeout error and subsequent (what seems to be) corruption in
production, though, since we typically have higher loads there.
There is still the issue of the original error, which seems like it's not a
known / intended failure case.
Our DBs are something like 150 docs, with at most a megabyte of json spread
between them, and then on the order of a gig of attachments scattered around.
Would it help to have access to them? I can't post them publicly, but could
send them to someone.
OS Process Error 0.21247.103 :: {os_process_error, {exit_status,0}}
-
Key: COUCHDB-1817
URL: https://issues.apache.org/jira/browse/COUCHDB-1817
Project: CouchDB
Issue Type: Bug
Components: JavaScript View Server
Reporter: Eli Stevens
Attachments: couchdb__couchdb_files.png,
couchdb__httpd_status_codes.png, couchdb_mem.png, loadavg.png, memory2.png
We have started seeing errors crop up in our application that we have not
seen before, and we're at a loss for how to start debugging it.
[~dch] Said that we might look into system resource limits, so we started
collecting all of the output from _stats into RRD (along with memory, load,
etc. that we were already collecting), but nothing is jumping out at us as
obviously problematic.
We can semi-reliably reproduce the problem, but it's far from a minimal test
case (basically, we load up several large chunks of data, and then halfway
through the processing run, we get the error). The error doesn't seem to
happen if we load up each chunk by itself.
The DB in question has about 100 docs in it, none particularly large (nothing
over a couple KB would be my guess), with a couple hundred MB in attachments.
10ish design docs, coffeescript. In general, there isn't anything that
seems obviously resource intensive.
We have seen this issue on 1.2.0, 1.2.1, and we're working on getting a
machine with 1.3.0 set up (the PPA we'd been using hasn't been updated yet).
Ubuntu 12.04, spinning disk, etc. The system is under load when it happens,
but the load isn't more than 1.5x the number of cores. I don't have disk IO
numbers at hand, but I'd be surprised if that was being strained.
Error as it appears in couch.log:
https://gist.github.com/wickedgrey/e7fd3fc14b6d43e95564
The design doc in question:
https://gist.github.com/wickedgrey/db41b0c3c75a590e2109
An example document: https://gist.github.com/wickedgrey/a8422aab261ddd2ce4fe
We have some preliminary evidence that the problem persists after the system
goes quiet, but we're not certain.
Either CouchDB isn't handling things correctly, in which case this bug is
prz fix or we're doing something wrong (hitting a resource limit, or
something), in which case this bug is prz make the error message more
informative.
Thanks!
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (COUCHDB-1817) OS Process Error 0.21247.103 :: {os_process_error, {exit_status,0}}
[
https://issues.apache.org/jira/browse/COUCHDB-1817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13676584#comment-13676584
]
Eli Stevens commented on COUCHDB-1817:
--
Of course, the next run had an issue. :(
Here's the stack trace from the first server 500 for the data run:
https://gist.github.com/wickedgrey/ef7b59e6b0be7ec47692
Here's the stack trace from trying to *read* the view in question:
https://gist.github.com/wickedgrey/d6cb6e0fa2190882977f
We've copied the DB files off, and are ready to send them to someone who would
like to debug the issue.
OS Process Error 0.21247.103 :: {os_process_error, {exit_status,0}}
-
Key: COUCHDB-1817
URL: https://issues.apache.org/jira/browse/COUCHDB-1817
Project: CouchDB
Issue Type: Bug
Components: JavaScript View Server
Reporter: Eli Stevens
Attachments: couchdb__couchdb_files.png,
couchdb__httpd_status_codes.png, couchdb_mem.png, loadavg.png, memory2.png
We have started seeing errors crop up in our application that we have not
seen before, and we're at a loss for how to start debugging it.
[~dch] Said that we might look into system resource limits, so we started
collecting all of the output from _stats into RRD (along with memory, load,
etc. that we were already collecting), but nothing is jumping out at us as
obviously problematic.
We can semi-reliably reproduce the problem, but it's far from a minimal test
case (basically, we load up several large chunks of data, and then halfway
through the processing run, we get the error). The error doesn't seem to
happen if we load up each chunk by itself.
The DB in question has about 100 docs in it, none particularly large (nothing
over a couple KB would be my guess), with a couple hundred MB in attachments.
10ish design docs, coffeescript. In general, there isn't anything that
seems obviously resource intensive.
We have seen this issue on 1.2.0, 1.2.1, and we're working on getting a
machine with 1.3.0 set up (the PPA we'd been using hasn't been updated yet).
Ubuntu 12.04, spinning disk, etc. The system is under load when it happens,
but the load isn't more than 1.5x the number of cores. I don't have disk IO
numbers at hand, but I'd be surprised if that was being strained.
Error as it appears in couch.log:
https://gist.github.com/wickedgrey/e7fd3fc14b6d43e95564
The design doc in question:
https://gist.github.com/wickedgrey/db41b0c3c75a590e2109
An example document: https://gist.github.com/wickedgrey/a8422aab261ddd2ce4fe
We have some preliminary evidence that the problem persists after the system
goes quiet, but we're not certain.
Either CouchDB isn't handling things correctly, in which case this bug is
prz fix or we're doing something wrong (hitting a resource limit, or
something), in which case this bug is prz make the error message more
informative.
Thanks!
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (COUCHDB-1817) OS Process Error 0.21247.103 :: {os_process_error, {exit_status,0}}
[
https://issues.apache.org/jira/browse/COUCHDB-1817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13676594#comment-13676594
]
Eli Stevens commented on COUCHDB-1817:
--
I forgot to note that the problem exists for all views in the DB including
newly created views (_all_docs seems fine), and persists through both reboots
and removing the .dbname_xxx_design directory.
This implies to me a corruption of the .couch file. To me, this seems like a
critical issue.
OS Process Error 0.21247.103 :: {os_process_error, {exit_status,0}}
-
Key: COUCHDB-1817
URL: https://issues.apache.org/jira/browse/COUCHDB-1817
Project: CouchDB
Issue Type: Bug
Components: JavaScript View Server
Reporter: Eli Stevens
Priority: Critical
Attachments: couchdb__couchdb_files.png,
couchdb__httpd_status_codes.png, couchdb_mem.png, loadavg.png, memory2.png
We have started seeing errors crop up in our application that we have not
seen before, and we're at a loss for how to start debugging it.
[~dch] Said that we might look into system resource limits, so we started
collecting all of the output from _stats into RRD (along with memory, load,
etc. that we were already collecting), but nothing is jumping out at us as
obviously problematic.
We can semi-reliably reproduce the problem, but it's far from a minimal test
case (basically, we load up several large chunks of data, and then halfway
through the processing run, we get the error). The error doesn't seem to
happen if we load up each chunk by itself.
The DB in question has about 100 docs in it, none particularly large (nothing
over a couple KB would be my guess), with a couple hundred MB in attachments.
10ish design docs, coffeescript. In general, there isn't anything that
seems obviously resource intensive.
We have seen this issue on 1.2.0, 1.2.1, and we're working on getting a
machine with 1.3.0 set up (the PPA we'd been using hasn't been updated yet).
Ubuntu 12.04, spinning disk, etc. The system is under load when it happens,
but the load isn't more than 1.5x the number of cores. I don't have disk IO
numbers at hand, but I'd be surprised if that was being strained.
Error as it appears in couch.log:
https://gist.github.com/wickedgrey/e7fd3fc14b6d43e95564
The design doc in question:
https://gist.github.com/wickedgrey/db41b0c3c75a590e2109
An example document: https://gist.github.com/wickedgrey/a8422aab261ddd2ce4fe
We have some preliminary evidence that the problem persists after the system
goes quiet, but we're not certain.
Either CouchDB isn't handling things correctly, in which case this bug is
prz fix or we're doing something wrong (hitting a resource limit, or
something), in which case this bug is prz make the error message more
informative.
Thanks!
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (COUCHDB-1817) OS Process Error 0.21247.103 :: {os_process_error, {exit_status,0}}
[
https://issues.apache.org/jira/browse/COUCHDB-1817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Eli Stevens updated COUCHDB-1817:
-
Priority: Critical (was: Major)
OS Process Error 0.21247.103 :: {os_process_error, {exit_status,0}}
-
Key: COUCHDB-1817
URL: https://issues.apache.org/jira/browse/COUCHDB-1817
Project: CouchDB
Issue Type: Bug
Components: JavaScript View Server
Reporter: Eli Stevens
Priority: Critical
Attachments: couchdb__couchdb_files.png,
couchdb__httpd_status_codes.png, couchdb_mem.png, loadavg.png, memory2.png
We have started seeing errors crop up in our application that we have not
seen before, and we're at a loss for how to start debugging it.
[~dch] Said that we might look into system resource limits, so we started
collecting all of the output from _stats into RRD (along with memory, load,
etc. that we were already collecting), but nothing is jumping out at us as
obviously problematic.
We can semi-reliably reproduce the problem, but it's far from a minimal test
case (basically, we load up several large chunks of data, and then halfway
through the processing run, we get the error). The error doesn't seem to
happen if we load up each chunk by itself.
The DB in question has about 100 docs in it, none particularly large (nothing
over a couple KB would be my guess), with a couple hundred MB in attachments.
10ish design docs, coffeescript. In general, there isn't anything that
seems obviously resource intensive.
We have seen this issue on 1.2.0, 1.2.1, and we're working on getting a
machine with 1.3.0 set up (the PPA we'd been using hasn't been updated yet).
Ubuntu 12.04, spinning disk, etc. The system is under load when it happens,
but the load isn't more than 1.5x the number of cores. I don't have disk IO
numbers at hand, but I'd be surprised if that was being strained.
Error as it appears in couch.log:
https://gist.github.com/wickedgrey/e7fd3fc14b6d43e95564
The design doc in question:
https://gist.github.com/wickedgrey/db41b0c3c75a590e2109
An example document: https://gist.github.com/wickedgrey/a8422aab261ddd2ce4fe
We have some preliminary evidence that the problem persists after the system
goes quiet, but we're not certain.
Either CouchDB isn't handling things correctly, in which case this bug is
prz fix or we're doing something wrong (hitting a resource limit, or
something), in which case this bug is prz make the error message more
informative.
Thanks!
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (COUCHDB-1817) OS Process Error 0.21247.103 :: {os_process_error, {exit_status,0}}
[
https://issues.apache.org/jira/browse/COUCHDB-1817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13674586#comment-13674586
]
Eli Stevens commented on COUCHDB-1817:
--
$ erl +V
Erlang (SMP,ASYNC_THREADS) (BEAM) emulator version 5.8.5
$ erl
Erlang R14B04 (erts-5.8.5) [source] [64-bit] [smp:4:4] [rq:4] [async-threads:0]
[kernel-poll:false]
Eshell V5.8.5 (abort with ^G)
1
I can also confirm that a temporary view with the addition of a comment
character does not exhibit the same behavior, but the view continues to return
errors when viewed.
OS Process Error 0.21247.103 :: {os_process_error, {exit_status,0}}
-
Key: COUCHDB-1817
URL: https://issues.apache.org/jira/browse/COUCHDB-1817
Project: CouchDB
Issue Type: Bug
Components: JavaScript View Server
Reporter: Eli Stevens
We have started seeing errors crop up in our application that we have not
seen before, and we're at a loss for how to start debugging it.
[~dch] Said that we might look into system resource limits, so we started
collecting all of the output from _stats into RRD (along with memory, load,
etc. that we were already collecting), but nothing is jumping out at us as
obviously problematic.
We can semi-reliably reproduce the problem, but it's far from a minimal test
case (basically, we load up several large chunks of data, and then halfway
through the processing run, we get the error). The error doesn't seem to
happen if we load up each chunk by itself.
The DB in question has about 100 docs in it, none particularly large (nothing
over a couple KB would be my guess), with a couple hundred MB in attachments.
10ish design docs, coffeescript. In general, there isn't anything that
seems obviously resource intensive.
We have seen this issue on 1.2.0, 1.2.1, and we're working on getting a
machine with 1.3.0 set up (the PPA we'd been using hasn't been updated yet).
Ubuntu 12.04, spinning disk, etc. The system is under load when it happens,
but the load isn't more than 1.5x the number of cores. I don't have disk IO
numbers at hand, but I'd be surprised if that was being strained.
Error as it appears in couch.log:
https://gist.github.com/wickedgrey/e7fd3fc14b6d43e95564
The design doc in question:
https://gist.github.com/wickedgrey/db41b0c3c75a590e2109
An example document: https://gist.github.com/wickedgrey/a8422aab261ddd2ce4fe
We have some preliminary evidence that the problem persists after the system
goes quiet, but we're not certain.
Either CouchDB isn't handling things correctly, in which case this bug is
prz fix or we're doing something wrong (hitting a resource limit, or
something), in which case this bug is prz make the error message more
informative.
Thanks!
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (COUCHDB-1817) OS Process Error 0.21247.103 :: {os_process_error, {exit_status,0}}
Eli Stevens created COUCHDB-1817:
Summary: OS Process Error 0.21247.103 :: {os_process_error,
{exit_status,0}}
Key: COUCHDB-1817
URL: https://issues.apache.org/jira/browse/COUCHDB-1817
Project: CouchDB
Issue Type: Bug
Components: JavaScript View Server
Reporter: Eli Stevens
We have started seeing errors crop up in our application that we have not seen
before, and we're at a loss for how to start debugging it.
[~dch] Said that we might look into system resource limits, so we started
collecting all of the output from _stats into RRD (along with memory, load,
etc. that we were already collecting), but nothing is jumping out at us as
obviously problematic.
We can semi-reliably reproduce the problem, but it's far from a minimal test
case (basically, we load up several large chunks of data, and then halfway
through the processing run, we get the error). The error doesn't seem to
happen if we load up each chunk by itself.
The DB in question has about 100 docs in it, none particularly large (nothing
over a couple KB would be my guess), with a couple hundred MB in attachments.
10ish design docs, coffeescript. In general, there isn't anything that seems
obviously resource intensive.
We have seen this issue on 1.2.0, 1.2.1, and we're working on getting a machine
with 1.3.0 set up (the PPA we'd been using hasn't been updated yet). Ubuntu
12.04, spinning disk, etc. The system is under load when it happens, but the
load isn't more than 1.5x the number of cores. I don't have disk IO numbers at
hand, but I'd be surprised if that was being strained.
Error as it appears in couch.log:
https://gist.github.com/wickedgrey/e7fd3fc14b6d43e95564
The design doc in question:
https://gist.github.com/wickedgrey/db41b0c3c75a590e2109
An example document: https://gist.github.com/wickedgrey/a8422aab261ddd2ce4fe
We have some preliminary evidence that the problem persists after the system
goes quiet, but we're not certain.
Either CouchDB isn't handling things correctly, in which case this bug is prz
fix or we're doing something wrong (hitting a resource limit, or something),
in which case this bug is prz make the error message more informative.
Thanks!
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (COUCHDB-1493) null byte in string data dropped
Eli Stevens created COUCHDB-1493:
Summary: null byte in string data dropped
Key: COUCHDB-1493
URL: https://issues.apache.org/jira/browse/COUCHDB-1493
Project: CouchDB
Issue Type: Bug
Components: Database Core
Affects Versions: 1.2
Reporter: Eli Stevens
curl -X POST http://localhost:5984/test/ -H Content-Type: application/json
-d '{b:b\ub}'
Results in the following document:
{
_id: abd492c1f26d0b404111b2324c004537,
_rev: 1-b7d9864e38fd32678f0e7994f6672709,
b: bb
}
I'm not sure what the intended behavior is, but it's a change from 1.1 (1.1.1?
I don't recall the exact version I had before upgrading).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (COUCHDB-1192) Attachment upload speed varies widely based on how it is uploaded
Attachment upload speed varies widely based on how it is uploaded - Key: COUCHDB-1192 URL: https://issues.apache.org/jira/browse/COUCHDB-1192 Project: CouchDB Issue Type: Question Components: HTTP Interface Affects Versions: 1.0.2 Environment: OSX 10.6.7 MacBook Pro (7200 RPM disk) CouchDBX 1.0.2 couchdb-python used as client code Reporter: Eli Stevens Priority: Minor Running the following code on a macbook pro, using CouchDBX 1.0.2 (everything local), we're seeing the following output when trying to attach a file with 10MB of random data: Code: https://gist.github.com/bc0c36f36be0c85e2a36 Output: Using put_attachment: 0.309157133102 post time: 2.5557808876 Using multipart: 2.61283898354 Encoding base64: 0.0497629642487 Updating: 5.0550069809 Server log: https://gist.github.com/a80a495fd35049ff871f (there's a HEAD/DELETE/PUT/GET cycle that's just cleanup) The calls in question are: Using put_attachment: 0.309157133102 1 [info] [0.27809.7] 127.0.0.1 - - 'PUT' /benchmark_entity/bigfile/smallfile?rev=81-c538b38a8463952f0136143cfa49e9fa 201 Using multipart: 2.61283898354 (post time: 2.5557808876) 1 [info] [0.27809.7] 127.0.0.1 - - 'POST' /benchmark_entity/bigfile 201 Updating: 5.0550069809 1 [info] [0.27809.7] 127.0.0.1 - - 'POST' /benchmark_entity/_bulk_docs 201 Profiling our code shows 1.5 sec of CPU usage in our code (which covers setup / cleanup code that's not included in the times above), and 11.8 sec of total run time, which roughly matches up with the PUT/POST times above. Basically, I feel pretty confident that the bulk of the times above are not in our client code, and are instead due to couchdb's handling time. We haven't conclusively ruled out couchdb-python behaving very oddly, though it seems very unlikely. Why is the form/multipart handler so much slower than using a bare PUT on the attachment? Why is the base64 approach even slower? Is it due to bandwidth issues, couchdb CPU usage...? If needed, we can update to 1.1 and test there. Note that the curl code doesn't seem to result in the same MD5 when we get the attachment back out, so I've snipped the output related to that. Thanks for any help, Eli -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (COUCHDB-1168) The multipart API is not documented
[ https://issues.apache.org/jira/browse/COUCHDB-1168?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13039244#comment-13039244 ] Eli Stevens commented on COUCHDB-1168: -- [10:56am] jan: wickedgrey: first you gotta figure out how the API works, I'd look at the API handler inside couch_httpd_db.erl that handles the multipart formdata call The multipart API is not documented --- Key: COUCHDB-1168 URL: https://issues.apache.org/jira/browse/COUCHDB-1168 Project: CouchDB Issue Type: Bug Components: Documentation Reporter: Eli Stevens Priority: Minor Based on answers to questions like is it possible to upload documents and attachments at the same time? in IRC, it seems that there is a multipart API that allows that sort of thing, but I've been told that it's not documented (and haven't been able to find any mention of it via searching). - How can it be invoked? - Can it be used instead of bulk_docs? Ie. for multiple documents at once. - What are the _rev semantics (ATM each attachment increments the _rev by one; would it be possible to have a document at _rev 1 with all attachments?)? -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (COUCHDB-1168) The multipart API is not documented
The multipart API is not documented --- Key: COUCHDB-1168 URL: https://issues.apache.org/jira/browse/COUCHDB-1168 Project: CouchDB Issue Type: Bug Components: Documentation Reporter: Eli Stevens Priority: Minor Based on answers to questions like is it possible to upload documents and attachments at the same time? in IRC, it seems that there is a multipart API that allows that sort of thing, but I've been told that it's not documented (and haven't been able to find any mention of it via searching). - How can it be invoked? - Can it be used instead of bulk_docs? Ie. for multiple documents at once. - What are the _rev semantics (ATM each attachment increments the _rev by one; would it be possible to have a document at _rev 1 with all attachments?)? -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
