[jira] [Commented] (COUCHDB-1309) File descriptor leaks on design document update and view cleanup
[ https://issues.apache.org/jira/browse/COUCHDB-1309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13129520#comment-13129520 ] Benoit Chesneau commented on COUCHDB-1309: -- Ok, the update detection can't happen in updater, if this isn't triggered. Anyway I think we could remove this other listener by reusing the update notification process in couch_index_server, something like (not tested): http://friendpaste.com/7CwKM5clixdyuq6HWHjDDv I'm also not sure we need to know the exact design id, isn't emitting an 'refresh_indexes' event enough since we have all the info in states to detect signature change if it happends? It woud remove this list in memory and extra event notifications. Something like : http://friendpaste.com/44q84nzKSjaU15vX2sVkfF I will try to send a complete patch later in the day, but not sure I will have time. Btw I'm only worrying about the trunk here. I think we have some time to solve this problem on trunk. No need for a quick fix. > File descriptor leaks on design document update and view cleanup > > > Key: COUCHDB-1309 > URL: https://issues.apache.org/jira/browse/COUCHDB-1309 > Project: CouchDB > Issue Type: Bug >Reporter: Filipe Manana >Assignee: Filipe Manana > Attachments: couchdb-1309_12x.patch, couchdb-1309_trunk.patch > > > If we add a design document with views defined in it, open the corresponding > view group (by querying one of its views for e.g.), then update the design > document in such a way that the view signature changes (changing a view's map > function code for e.g), the old view group remains open forever (unless a > server restart happens) and keeps it's view file reference counter active > forever. > If a view cleanup request comes, the old view file is not deleted since the > reference counter is not dropped by the old view group, keeping the file > descriptor in use forever. > This leakage is different from what is reported in COUCHDB-1129 but it's > somehow related. > The attached patch, simply shutdowns a view group when the design document is > updated and the new view signature changes, releasing the old view file > descriptor (as soon as no more clients are using the old view). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (COUCHDB-1310) '/_restart' closes request socket before sending a response
[ https://issues.apache.org/jira/browse/COUCHDB-1310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13129408#comment-13129408 ] Paul Joseph Davis commented on COUCHDB-1310: quite possible that it makes the hangs better. The test that we currently have for detecting when the server came back is less than robust. I've tried to fix it a couple times but never managed to find something that was significantly less hacky. > '/_restart' closes request socket before sending a response > --- > > Key: COUCHDB-1310 > URL: https://issues.apache.org/jira/browse/COUCHDB-1310 > Project: CouchDB > Issue Type: Bug > Components: HTTP Interface >Affects Versions: 1.1 > Environment: All platforms. Tested on Ubuntu 10.10. >Reporter: J. Lee Coltrane >Priority: Minor > Fix For: 1.1 > > Attachments: > 0001-fixed-_restart-closes-socket-before-sending-request.patch > > Original Estimate: 0h > Remaining Estimate: 0h > > In 'couchdb_misc_handlers.erl', `handle_restart_req(...)` calls > `couch_server_sup:restart_core_server()` before sending the HTTP response. > This causes the current request's socket to be closed without sending any > response back to the client. From the perspective of the HTTP client, the > connection is simply dropped - no response is ever received. > In addition to the obvious aesthetic problems here, the HTTP1.1 spec suggests > a specific (and non-desirable) client-side behavior for connections that drop > like this. From RFC-2616 Section-8.2.4: "...if the client sees the connection > close before receiving any status from the server, the client SHOULD retry > the request..." (http://tools.ietf.org/html/rfc2616#section-8.2.4). Any HTTP > client that actually obeys this direction, would end up restarting the server > multiple times. > I have a patch that fixes this issue. I will attach it to this report. > This issue may be related to COUCHDB-946 > (https://issues.apache.org/jira/browse/COUCHDB-946). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (COUCHDB-1310) '/_restart' closes request socket before sending a response
[ https://issues.apache.org/jira/browse/COUCHDB-1310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13129390#comment-13129390 ] J. Lee Coltrane commented on COUCHDB-1310: -- I agree: If '/_restart' were to become a truly "useful" part of the api, it would need to do a graceful shutdown. This might be a useful feature to add, but it's certainly not provided by the attached patch. However, as it stands today, the tests *do* depend on the '/_restart' interface, and the issue reported here appears to be the cause of a lot of the flakey test-suite behavior that I (and others) have observed in 1.1.0. This is especially true when accessing couchdb through a proxy, as the proxy's handling of the dropped connection varies depending on the proxy implementation. Unfortunately, this is also one of the cases where the test suite would be most useful -- verifying the correctness of a custom reverse-proxy implementation for use in-front-of couchdb. There are various notes in the dev-list archives about the tests hanging (and other flakey behavior). With this patch in place, I'm no longer seeing tests hang like that. (I'm still seeing some scattered issues from the test suite -- but no hanging) > '/_restart' closes request socket before sending a response > --- > > Key: COUCHDB-1310 > URL: https://issues.apache.org/jira/browse/COUCHDB-1310 > Project: CouchDB > Issue Type: Bug > Components: HTTP Interface >Affects Versions: 1.1 > Environment: All platforms. Tested on Ubuntu 10.10. >Reporter: J. Lee Coltrane >Priority: Minor > Fix For: 1.1 > > Attachments: > 0001-fixed-_restart-closes-socket-before-sending-request.patch > > Original Estimate: 0h > Remaining Estimate: 0h > > In 'couchdb_misc_handlers.erl', `handle_restart_req(...)` calls > `couch_server_sup:restart_core_server()` before sending the HTTP response. > This causes the current request's socket to be closed without sending any > response back to the client. From the perspective of the HTTP client, the > connection is simply dropped - no response is ever received. > In addition to the obvious aesthetic problems here, the HTTP1.1 spec suggests > a specific (and non-desirable) client-side behavior for connections that drop > like this. From RFC-2616 Section-8.2.4: "...if the client sees the connection > close before receiving any status from the server, the client SHOULD retry > the request..." (http://tools.ietf.org/html/rfc2616#section-8.2.4). Any HTTP > client that actually obeys this direction, would end up restarting the server > multiple times. > I have a patch that fixes this issue. I will attach it to this report. > This issue may be related to COUCHDB-946 > (https://issues.apache.org/jira/browse/COUCHDB-946). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (COUCHDB-1310) '/_restart' closes request socket before sending a response
[ https://issues.apache.org/jira/browse/COUCHDB-1310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13129373#comment-13129373 ] Paul Joseph Davis commented on COUCHDB-1310: I'd point out that _restart is really only meant for test code to get the server into a known state. This patch merely hides the issue and doesn't really address the underlying problem, namely, that init:restart is a hard core reset of the VM. If there are concurrent clients accessing the server they will be just as exposed to the original problem. About the only thing to do would be to figure out how to gracefully quit mochiweb before calling init:restart. Its not out of the question, but we moved to this approach when the graceful one didn't work as reliably. > '/_restart' closes request socket before sending a response > --- > > Key: COUCHDB-1310 > URL: https://issues.apache.org/jira/browse/COUCHDB-1310 > Project: CouchDB > Issue Type: Bug > Components: HTTP Interface >Affects Versions: 1.1 > Environment: All platforms. Tested on Ubuntu 10.10. >Reporter: J. Lee Coltrane >Priority: Minor > Fix For: 1.1 > > Attachments: > 0001-fixed-_restart-closes-socket-before-sending-request.patch > > Original Estimate: 0h > Remaining Estimate: 0h > > In 'couchdb_misc_handlers.erl', `handle_restart_req(...)` calls > `couch_server_sup:restart_core_server()` before sending the HTTP response. > This causes the current request's socket to be closed without sending any > response back to the client. From the perspective of the HTTP client, the > connection is simply dropped - no response is ever received. > In addition to the obvious aesthetic problems here, the HTTP1.1 spec suggests > a specific (and non-desirable) client-side behavior for connections that drop > like this. From RFC-2616 Section-8.2.4: "...if the client sees the connection > close before receiving any status from the server, the client SHOULD retry > the request..." (http://tools.ietf.org/html/rfc2616#section-8.2.4). Any HTTP > client that actually obeys this direction, would end up restarting the server > multiple times. > I have a patch that fixes this issue. I will attach it to this report. > This issue may be related to COUCHDB-946 > (https://issues.apache.org/jira/browse/COUCHDB-946). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (COUCHDB-1310) '/_restart' closes request socket before sending a response
[ https://issues.apache.org/jira/browse/COUCHDB-1310?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] J. Lee Coltrane updated COUCHDB-1310: - Attachment: 0001-fixed-_restart-closes-socket-before-sending-request.patch here is a patch that fixes this issue (COUCHDB-1310) > '/_restart' closes request socket before sending a response > --- > > Key: COUCHDB-1310 > URL: https://issues.apache.org/jira/browse/COUCHDB-1310 > Project: CouchDB > Issue Type: Bug > Components: HTTP Interface >Affects Versions: 1.1 > Environment: All platforms. Tested on Ubuntu 10.10. >Reporter: J. Lee Coltrane >Priority: Minor > Fix For: 1.1 > > Attachments: > 0001-fixed-_restart-closes-socket-before-sending-request.patch > > Original Estimate: 0h > Remaining Estimate: 0h > > In 'couchdb_misc_handlers.erl', `handle_restart_req(...)` calls > `couch_server_sup:restart_core_server()` before sending the HTTP response. > This causes the current request's socket to be closed without sending any > response back to the client. From the perspective of the HTTP client, the > connection is simply dropped - no response is ever received. > In addition to the obvious aesthetic problems here, the HTTP1.1 spec suggests > a specific (and non-desirable) client-side behavior for connections that drop > like this. From RFC-2616 Section-8.2.4: "...if the client sees the connection > close before receiving any status from the server, the client SHOULD retry > the request..." (http://tools.ietf.org/html/rfc2616#section-8.2.4). Any HTTP > client that actually obeys this direction, would end up restarting the server > multiple times. > I have a patch that fixes this issue. I will attach it to this report. > This issue may be related to COUCHDB-946 > (https://issues.apache.org/jira/browse/COUCHDB-946). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (COUCHDB-1310) '/_restart' closes request socket before sending a response
'/_restart' closes request socket before sending a response --- Key: COUCHDB-1310 URL: https://issues.apache.org/jira/browse/COUCHDB-1310 Project: CouchDB Issue Type: Bug Components: HTTP Interface Affects Versions: 1.1 Environment: All platforms. Tested on Ubuntu 10.10. Reporter: J. Lee Coltrane Priority: Minor Fix For: 1.1 In 'couchdb_misc_handlers.erl', `handle_restart_req(...)` calls `couch_server_sup:restart_core_server()` before sending the HTTP response. This causes the current request's socket to be closed without sending any response back to the client. From the perspective of the HTTP client, the connection is simply dropped - no response is ever received. In addition to the obvious aesthetic problems here, the HTTP1.1 spec suggests a specific (and non-desirable) client-side behavior for connections that drop like this. From RFC-2616 Section-8.2.4: "...if the client sees the connection close before receiving any status from the server, the client SHOULD retry the request..." (http://tools.ietf.org/html/rfc2616#section-8.2.4). Any HTTP client that actually obeys this direction, would end up restarting the server multiple times. I have a patch that fixes this issue. I will attach it to this report. This issue may be related to COUCHDB-946 (https://issues.apache.org/jira/browse/COUCHDB-946). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (COUCHDB-1129) file descriptors sometimes not closed after compaction
[ https://issues.apache.org/jira/browse/COUCHDB-1129?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13129246#comment-13129246 ] Dan Everton commented on COUCHDB-1129: -- No, I don't think it's that bug. The CouchDB instance has been restarted several times to free up the file descriptors without changing the design documents. > file descriptors sometimes not closed after compaction > -- > > Key: COUCHDB-1129 > URL: https://issues.apache.org/jira/browse/COUCHDB-1129 > Project: CouchDB > Issue Type: Bug > Components: Database Core >Affects Versions: 1.0.2 >Reporter: Randall Leeds > Fix For: 1.2 > > > It seems there are still cases where file descriptors are not released upon > compaction finishing. > When I asked on IRC rnewson confirmed he'd seen the behavior also and the > last comment on 926 also suggests there are still times where this occurs. > Someone needs to take a careful eye to any race conditions we might have > between opening the file and subscribing to the compaction notification. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (COUCHDB-1309) File descriptor leaks on design document update and view cleanup
[ https://issues.apache.org/jira/browse/COUCHDB-1309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13129120#comment-13129120 ] Paul Joseph Davis commented on COUCHDB-1309: @Filipe After reading further I think you've probably got the best approach. The only alternative I've contemplated would be to have a monitor process that wakes up every so often and checks that its sig is still valid which I don't think is as clean as this version. Although, I would like to see this patch as at least two if not more commits. Specifically, at least one for the new ddoc event broadcasting and then one for the indexer/view stuff. > File descriptor leaks on design document update and view cleanup > > > Key: COUCHDB-1309 > URL: https://issues.apache.org/jira/browse/COUCHDB-1309 > Project: CouchDB > Issue Type: Bug >Reporter: Filipe Manana >Assignee: Filipe Manana > Attachments: couchdb-1309_12x.patch, couchdb-1309_trunk.patch > > > If we add a design document with views defined in it, open the corresponding > view group (by querying one of its views for e.g.), then update the design > document in such a way that the view signature changes (changing a view's map > function code for e.g), the old view group remains open forever (unless a > server restart happens) and keeps it's view file reference counter active > forever. > If a view cleanup request comes, the old view file is not deleted since the > reference counter is not dropped by the old view group, keeping the file > descriptor in use forever. > This leakage is different from what is reported in COUCHDB-1129 but it's > somehow related. > The attached patch, simply shutdowns a view group when the design document is > updated and the new view signature changes, releasing the old view file > descriptor (as soon as no more clients are using the old view). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (COUCHDB-1172) replicator_db futon test failing
[ https://issues.apache.org/jira/browse/COUCHDB-1172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robby Morgan updated COUCHDB-1172: -- Attachment: couch.log I am seeing the exact same failure, and have attached my own log. I'm testing on: * CentOS 5.5 64-bit server * Erlang R14B04 * libcurl 7.21.7 * Browser is Chrome running on Ubuntu > replicator_db futon test failing > > > Key: COUCHDB-1172 > URL: https://issues.apache.org/jira/browse/COUCHDB-1172 > Project: CouchDB > Issue Type: Bug > Components: Test Suite >Affects Versions: 1.1 > Environment: Ubuntu 10.04.2 64-bit server > Browser is FF3.6.17 running on Win7 > Running in dev mode (make dev; utils/run) >Reporter: Jake Levirne > Attachments: couch.log, couch.log > > > Received the following error when running the Futon test suite: > replicator_db failure 164016ms > 1. Run with debuggerAssertion failed: typeof > repDoc2._replication_state === "undefined" > 2. Assertion failed: typeof repDoc2._replication_state_time === "undefined" -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (COUCHDB-1309) File descriptor leaks on design document update and view cleanup
[ https://issues.apache.org/jira/browse/COUCHDB-1309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13129080#comment-13129080 ] Benoit Chesneau commented on COUCHDB-1309: -- I'm pretty sure things could be handled with a less offensive tone. Thanks. Tickets are here to let other review. Like I said I will have a closer look *later*. I don't think we need another asynchronous process and more I'm not sure for a performance reason that the change in update_notifier is really wanted. Even if we take this patch the notifier may need to be monitored, etc. > File descriptor leaks on design document update and view cleanup > > > Key: COUCHDB-1309 > URL: https://issues.apache.org/jira/browse/COUCHDB-1309 > Project: CouchDB > Issue Type: Bug >Reporter: Filipe Manana >Assignee: Filipe Manana > Attachments: couchdb-1309_12x.patch, couchdb-1309_trunk.patch > > > If we add a design document with views defined in it, open the corresponding > view group (by querying one of its views for e.g.), then update the design > document in such a way that the view signature changes (changing a view's map > function code for e.g), the old view group remains open forever (unless a > server restart happens) and keeps it's view file reference counter active > forever. > If a view cleanup request comes, the old view file is not deleted since the > reference counter is not dropped by the old view group, keeping the file > descriptor in use forever. > This leakage is different from what is reported in COUCHDB-1129 but it's > somehow related. > The attached patch, simply shutdowns a view group when the design document is > updated and the new view signature changes, releasing the old view file > descriptor (as soon as no more clients are using the old view). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (COUCHDB-1309) File descriptor leaks on design document update and view cleanup
[ https://issues.apache.org/jira/browse/COUCHDB-1309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13129056#comment-13129056 ] Paul Joseph Davis commented on COUCHDB-1309: Filipe, Good call. That code I'm remember is probably from before we used signatures for view groups instead of just the design name. > File descriptor leaks on design document update and view cleanup > > > Key: COUCHDB-1309 > URL: https://issues.apache.org/jira/browse/COUCHDB-1309 > Project: CouchDB > Issue Type: Bug >Reporter: Filipe Manana >Assignee: Filipe Manana > Attachments: couchdb-1309_12x.patch, couchdb-1309_trunk.patch > > > If we add a design document with views defined in it, open the corresponding > view group (by querying one of its views for e.g.), then update the design > document in such a way that the view signature changes (changing a view's map > function code for e.g), the old view group remains open forever (unless a > server restart happens) and keeps it's view file reference counter active > forever. > If a view cleanup request comes, the old view file is not deleted since the > reference counter is not dropped by the old view group, keeping the file > descriptor in use forever. > This leakage is different from what is reported in COUCHDB-1129 but it's > somehow related. > The attached patch, simply shutdowns a view group when the design document is > updated and the new view signature changes, releasing the old view file > descriptor (as soon as no more clients are using the old view). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (COUCHDB-1309) File descriptor leaks on design document update and view cleanup
[ https://issues.apache.org/jira/browse/COUCHDB-1309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13129046#comment-13129046 ] Filipe Manana commented on COUCHDB-1309: Benoit, you can't rely on the updater to detect that the ddoc changed. Maybe I wasn't clear enough before. Imagine the following scenario: 1) Create a ddoc 2) Query one of its views 3) Update the ddoc If all subsequent view query requests arrive after the update (3), they will get routed to the new view group - therefore the old one will not get its updater running again and will stay alive forever. Same issue happens if when the update happens clients who get "into" the old view group are querying only with ?stale=ok. If you're so convinced that doing it in the updater works for these 2 cases, please provide a working code prototype. > File descriptor leaks on design document update and view cleanup > > > Key: COUCHDB-1309 > URL: https://issues.apache.org/jira/browse/COUCHDB-1309 > Project: CouchDB > Issue Type: Bug >Reporter: Filipe Manana >Assignee: Filipe Manana > Attachments: couchdb-1309_12x.patch, couchdb-1309_trunk.patch > > > If we add a design document with views defined in it, open the corresponding > view group (by querying one of its views for e.g.), then update the design > document in such a way that the view signature changes (changing a view's map > function code for e.g), the old view group remains open forever (unless a > server restart happens) and keeps it's view file reference counter active > forever. > If a view cleanup request comes, the old view file is not deleted since the > reference counter is not dropped by the old view group, keeping the file > descriptor in use forever. > This leakage is different from what is reported in COUCHDB-1129 but it's > somehow related. > The attached patch, simply shutdowns a view group when the design document is > updated and the new view signature changes, releasing the old view file > descriptor (as soon as no more clients are using the old view). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (COUCHDB-1309) File descriptor leaks on design document update and view cleanup
[ https://issues.apache.org/jira/browse/COUCHDB-1309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13129045#comment-13129045 ] Benoit Chesneau commented on COUCHDB-1309: -- @davisp mostly what I had in head. > File descriptor leaks on design document update and view cleanup > > > Key: COUCHDB-1309 > URL: https://issues.apache.org/jira/browse/COUCHDB-1309 > Project: CouchDB > Issue Type: Bug >Reporter: Filipe Manana >Assignee: Filipe Manana > Attachments: couchdb-1309_12x.patch, couchdb-1309_trunk.patch > > > If we add a design document with views defined in it, open the corresponding > view group (by querying one of its views for e.g.), then update the design > document in such a way that the view signature changes (changing a view's map > function code for e.g), the old view group remains open forever (unless a > server restart happens) and keeps it's view file reference counter active > forever. > If a view cleanup request comes, the old view file is not deleted since the > reference counter is not dropped by the old view group, keeping the file > descriptor in use forever. > This leakage is different from what is reported in COUCHDB-1129 but it's > somehow related. > The attached patch, simply shutdowns a view group when the design document is > updated and the new view signature changes, releasing the old view file > descriptor (as soon as no more clients are using the old view). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (COUCHDB-1309) File descriptor leaks on design document update and view cleanup
[ https://issues.apache.org/jira/browse/COUCHDB-1309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13129042#comment-13129042 ] Paul Joseph Davis commented on COUCHDB-1309: Hrm. Good catch Filipe. I haven't done more than skim the patch, but I'm wondering if it might be easier to just shutdown the view group when it sees the new design doc during an update instead of adding all of the mechanics to listen for update notifications. I'll read through this more when I get done figuring out how deb packages work. > File descriptor leaks on design document update and view cleanup > > > Key: COUCHDB-1309 > URL: https://issues.apache.org/jira/browse/COUCHDB-1309 > Project: CouchDB > Issue Type: Bug >Reporter: Filipe Manana >Assignee: Filipe Manana > Attachments: couchdb-1309_12x.patch, couchdb-1309_trunk.patch > > > If we add a design document with views defined in it, open the corresponding > view group (by querying one of its views for e.g.), then update the design > document in such a way that the view signature changes (changing a view's map > function code for e.g), the old view group remains open forever (unless a > server restart happens) and keeps it's view file reference counter active > forever. > If a view cleanup request comes, the old view file is not deleted since the > reference counter is not dropped by the old view group, keeping the file > descriptor in use forever. > This leakage is different from what is reported in COUCHDB-1129 but it's > somehow related. > The attached patch, simply shutdowns a view group when the design document is > updated and the new view signature changes, releasing the old view file > descriptor (as soon as no more clients are using the old view). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (COUCHDB-1309) File descriptor leaks on design document update and view cleanup
[ https://issues.apache.org/jira/browse/COUCHDB-1309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13129039#comment-13129039 ] Benoit Chesneau commented on COUCHDB-1309: -- Sending a synchronous call doesn't mean it need to stay alive, just that we send a message to the parent synchronously before dying. On the other hand it allows to not spawn another process. A custom exit signal could be enough. > File descriptor leaks on design document update and view cleanup > > > Key: COUCHDB-1309 > URL: https://issues.apache.org/jira/browse/COUCHDB-1309 > Project: CouchDB > Issue Type: Bug >Reporter: Filipe Manana >Assignee: Filipe Manana > Attachments: couchdb-1309_12x.patch, couchdb-1309_trunk.patch > > > If we add a design document with views defined in it, open the corresponding > view group (by querying one of its views for e.g.), then update the design > document in such a way that the view signature changes (changing a view's map > function code for e.g), the old view group remains open forever (unless a > server restart happens) and keeps it's view file reference counter active > forever. > If a view cleanup request comes, the old view file is not deleted since the > reference counter is not dropped by the old view group, keeping the file > descriptor in use forever. > This leakage is different from what is reported in COUCHDB-1129 but it's > somehow related. > The attached patch, simply shutdowns a view group when the design document is > updated and the new view signature changes, releasing the old view file > descriptor (as soon as no more clients are using the old view). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (COUCHDB-1309) File descriptor leaks on design document update and view cleanup
[ https://issues.apache.org/jira/browse/COUCHDB-1309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13129025#comment-13129025 ] Filipe Manana commented on COUCHDB-1309: Yes, that's assuming the updater will run again after the design document update. What if it doesn't? You still end up in the current situation. > File descriptor leaks on design document update and view cleanup > > > Key: COUCHDB-1309 > URL: https://issues.apache.org/jira/browse/COUCHDB-1309 > Project: CouchDB > Issue Type: Bug >Reporter: Filipe Manana >Assignee: Filipe Manana > Attachments: couchdb-1309_12x.patch, couchdb-1309_trunk.patch > > > If we add a design document with views defined in it, open the corresponding > view group (by querying one of its views for e.g.), then update the design > document in such a way that the view signature changes (changing a view's map > function code for e.g), the old view group remains open forever (unless a > server restart happens) and keeps it's view file reference counter active > forever. > If a view cleanup request comes, the old view file is not deleted since the > reference counter is not dropped by the old view group, keeping the file > descriptor in use forever. > This leakage is different from what is reported in COUCHDB-1129 but it's > somehow related. > The attached patch, simply shutdowns a view group when the design document is > updated and the new view signature changes, releasing the old view file > descriptor (as soon as no more clients are using the old view). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: Mouch is Make Couchapps with GNU Make
On Oct 13, 2011, at 22:09 , Johannes J. Schmidt wrote: > Dear developers, > > I just released Mouch [1] (Make Couch) on github. > Mouch is a little helper to build Couchapps. I am using it since half a > year now almost every day and it does quiet well. > I hope someone else find it useful. > > Mouch follows a radical minimalistic approach (125 LOC). It consists > only of three files, which you simply copy inside your project. Mouch > has only a few requirements: Ruby, a Ruby JSON lib, Make and cURL. You > might already have most of them installed. > With Mouch you build a JSON file and push it to the server. You talk to > Mouch via GNU Make. > > I would love to hear discussion about Mouchs approach. Hi Johannes, this is pretty cool, I like minimal dependencies :) I'll play with this on my next CouchApp project. Cheers Jan --
[jira] [Commented] (COUCHDB-1309) File descriptor leaks on design document update and view cleanup
[ https://issues.apache.org/jira/browse/COUCHDB-1309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13128985#comment-13128985 ] Benoit Chesneau commented on COUCHDB-1309: -- Or maybe I'm not. I'm not speaking about redesigning how the view system works neither I don't want to fix this issue and I'm not sure why you are suggesting that. I'm trying to remove the need of this another asynchronous process. You don't really need it. Once the updater detect that this particular design doc has changed (and it will) it can simply send a synchronous signal to its parent. then the parent will do exactly the same you are doing in your patch. That's all. I don't see why it couldn't work. I will have a closer look later on it. > File descriptor leaks on design document update and view cleanup > > > Key: COUCHDB-1309 > URL: https://issues.apache.org/jira/browse/COUCHDB-1309 > Project: CouchDB > Issue Type: Bug >Reporter: Filipe Manana >Assignee: Filipe Manana > Attachments: couchdb-1309_12x.patch, couchdb-1309_trunk.patch > > > If we add a design document with views defined in it, open the corresponding > view group (by querying one of its views for e.g.), then update the design > document in such a way that the view signature changes (changing a view's map > function code for e.g), the old view group remains open forever (unless a > server restart happens) and keeps it's view file reference counter active > forever. > If a view cleanup request comes, the old view file is not deleted since the > reference counter is not dropped by the old view group, keeping the file > descriptor in use forever. > This leakage is different from what is reported in COUCHDB-1129 but it's > somehow related. > The attached patch, simply shutdowns a view group when the design document is > updated and the new view signature changes, releasing the old view file > descriptor (as soon as no more clients are using the old view). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (COUCHDB-1309) File descriptor leaks on design document update and view cleanup
[ https://issues.apache.org/jira/browse/COUCHDB-1309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13128936#comment-13128936 ] Filipe Manana commented on COUCHDB-1309: Benoit, what I meant by synchronous is the handle_call vs handle_cast in couch_index.erl. I'm afraid you're missing some details about how the view system works. There's more than the updater process that depends on the design document properties and view signature, like couch_index_server. Spawning a new group when the ddoc signature changes is not a new behaviour I'm adding - it was always like this afaik. I'm concentrated on fixing this particular issue and not redesigning a big part of the view system. Feel free to do it and provide a working patch. > File descriptor leaks on design document update and view cleanup > > > Key: COUCHDB-1309 > URL: https://issues.apache.org/jira/browse/COUCHDB-1309 > Project: CouchDB > Issue Type: Bug >Reporter: Filipe Manana >Assignee: Filipe Manana > Attachments: couchdb-1309_12x.patch, couchdb-1309_trunk.patch > > > If we add a design document with views defined in it, open the corresponding > view group (by querying one of its views for e.g.), then update the design > document in such a way that the view signature changes (changing a view's map > function code for e.g), the old view group remains open forever (unless a > server restart happens) and keeps it's view file reference counter active > forever. > If a view cleanup request comes, the old view file is not deleted since the > reference counter is not dropped by the old view group, keeping the file > descriptor in use forever. > This leakage is different from what is reported in COUCHDB-1129 but it's > somehow related. > The attached patch, simply shutdowns a view group when the design document is > updated and the new view signature changes, releasing the old view file > descriptor (as soon as no more clients are using the old view). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (COUCHDB-1309) File descriptor leaks on design document update and view cleanup
[ https://issues.apache.org/jira/browse/COUCHDB-1309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13128841#comment-13128841 ] Benoit Chesneau commented on COUCHDB-1309: -- using the db update notifier is an asynchronous process fpr me ... By crash I mean in the OTP way. ie crash the updater then the manager restart one with new ddoc. That what you do anyway but it could be done directly in the updater, the manager should take care of such event. > File descriptor leaks on design document update and view cleanup > > > Key: COUCHDB-1309 > URL: https://issues.apache.org/jira/browse/COUCHDB-1309 > Project: CouchDB > Issue Type: Bug >Reporter: Filipe Manana >Assignee: Filipe Manana > Attachments: couchdb-1309_12x.patch, couchdb-1309_trunk.patch > > > If we add a design document with views defined in it, open the corresponding > view group (by querying one of its views for e.g.), then update the design > document in such a way that the view signature changes (changing a view's map > function code for e.g), the old view group remains open forever (unless a > server restart happens) and keeps it's view file reference counter active > forever. > If a view cleanup request comes, the old view file is not deleted since the > reference counter is not dropped by the old view group, keeping the file > descriptor in use forever. > This leakage is different from what is reported in COUCHDB-1129 but it's > somehow related. > The attached patch, simply shutdowns a view group when the design document is > updated and the new view signature changes, releasing the old view file > descriptor (as soon as no more clients are using the old view). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (COUCHDB-1309) File descriptor leaks on design document update and view cleanup
[ https://issues.apache.org/jira/browse/COUCHDB-1309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13128814#comment-13128814 ] Filipe Manana commented on COUCHDB-1309: Not sure what you mean: "I'm not sure it need to be asynchronous. Couldn't we test it here : " It's synchronous to avoid getting the process' mailbox flooded in the unlikely case there's a high rate of updates to the same ddoc. And the updater shouldn't be crashed. If it's running it means there are clients waiting for it before folding the view. Those clients should not get an error. > File descriptor leaks on design document update and view cleanup > > > Key: COUCHDB-1309 > URL: https://issues.apache.org/jira/browse/COUCHDB-1309 > Project: CouchDB > Issue Type: Bug >Reporter: Filipe Manana >Assignee: Filipe Manana > Attachments: couchdb-1309_12x.patch, couchdb-1309_trunk.patch > > > If we add a design document with views defined in it, open the corresponding > view group (by querying one of its views for e.g.), then update the design > document in such a way that the view signature changes (changing a view's map > function code for e.g), the old view group remains open forever (unless a > server restart happens) and keeps it's view file reference counter active > forever. > If a view cleanup request comes, the old view file is not deleted since the > reference counter is not dropped by the old view group, keeping the file > descriptor in use forever. > This leakage is different from what is reported in COUCHDB-1129 but it's > somehow related. > The attached patch, simply shutdowns a view group when the design document is > updated and the new view signature changes, releasing the old view file > descriptor (as soon as no more clients are using the old view). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (COUCHDB-1309) File descriptor leaks on design document update and view cleanup
[ https://issues.apache.org/jira/browse/COUCHDB-1309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13128811#comment-13128811 ] Benoit Chesneau commented on COUCHDB-1309: -- I'm not sure it need to be asynchronous. Couldn't we test it here : http://git-wip-us.apache.org/repos/asf?p=couchdb.git;a=blob;f=src/couch_index/src/couch_index_updater.erl;h=853f3d11130c903650450e97b23a692762b3bba1;hb=HEAD#l118 and then crash the updater? > File descriptor leaks on design document update and view cleanup > > > Key: COUCHDB-1309 > URL: https://issues.apache.org/jira/browse/COUCHDB-1309 > Project: CouchDB > Issue Type: Bug >Reporter: Filipe Manana >Assignee: Filipe Manana > Attachments: couchdb-1309_12x.patch, couchdb-1309_trunk.patch > > > If we add a design document with views defined in it, open the corresponding > view group (by querying one of its views for e.g.), then update the design > document in such a way that the view signature changes (changing a view's map > function code for e.g), the old view group remains open forever (unless a > server restart happens) and keeps it's view file reference counter active > forever. > If a view cleanup request comes, the old view file is not deleted since the > reference counter is not dropped by the old view group, keeping the file > descriptor in use forever. > This leakage is different from what is reported in COUCHDB-1129 but it's > somehow related. > The attached patch, simply shutdowns a view group when the design document is > updated and the new view signature changes, releasing the old view file > descriptor (as soon as no more clients are using the old view). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (COUCHDB-1302) Fix couchjs
[
https://issues.apache.org/jira/browse/COUCHDB-1302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13128809#comment-13128809
]
Jan Lehnardt commented on COUCHDB-1302:
---
+1, what Randall said.
Great job Paul!
One caveat, thinking about how the comments are read over time, "recent" will
not convey they same meaning as it does today, maybe we can make the messages
more explicit and have them include version numbers of what works and what
doesn't?
> Fix couchjs
> ---
>
> Key: COUCHDB-1302
> URL: https://issues.apache.org/jira/browse/COUCHDB-1302
> Project: CouchDB
> Issue Type: Improvement
> Components: JavaScript View Server
>Affects Versions: 1.1.1, 1.2, 1.3
>Reporter: Paul Joseph Davis
>Priority: Blocker
> Attachments:
> 0001-Prevent-linking-against-too-new-of-a-SpiderMonkey.patch,
> 0002-Minor-fixes-to-link-agianst-SpiderMonkey-trunk.patch
>
>
> Figure out why some spidermonkeys have an error when doing:
> eval("function(){}")
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (COUCHDB-1129) file descriptors sometimes not closed after compaction
[ https://issues.apache.org/jira/browse/COUCHDB-1129?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13128801#comment-13128801 ] Filipe Manana commented on COUCHDB-1129: Dan, can it be that, without restarting the server, you updated the design document (potentially several times) and then issued a /db/_view_cleanup request? (COUCHDB-1309) > file descriptors sometimes not closed after compaction > -- > > Key: COUCHDB-1129 > URL: https://issues.apache.org/jira/browse/COUCHDB-1129 > Project: CouchDB > Issue Type: Bug > Components: Database Core >Affects Versions: 1.0.2 >Reporter: Randall Leeds > Fix For: 1.2 > > > It seems there are still cases where file descriptors are not released upon > compaction finishing. > When I asked on IRC rnewson confirmed he'd seen the behavior also and the > last comment on 926 also suggests there are still times where this occurs. > Someone needs to take a careful eye to any race conditions we might have > between opening the file and subscribing to the compaction notification. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (COUCHDB-1309) File descriptor leaks on design document update and view cleanup
[ https://issues.apache.org/jira/browse/COUCHDB-1309?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Filipe Manana updated COUCHDB-1309: --- Attachment: couchdb-1309_12x.patch couchdb-1309_trunk.patch the patches for trunk and 1.2.x/1.1.x > File descriptor leaks on design document update and view cleanup > > > Key: COUCHDB-1309 > URL: https://issues.apache.org/jira/browse/COUCHDB-1309 > Project: CouchDB > Issue Type: Bug >Reporter: Filipe Manana >Assignee: Filipe Manana > Attachments: couchdb-1309_12x.patch, couchdb-1309_trunk.patch > > > If we add a design document with views defined in it, open the corresponding > view group (by querying one of its views for e.g.), then update the design > document in such a way that the view signature changes (changing a view's map > function code for e.g), the old view group remains open forever (unless a > server restart happens) and keeps it's view file reference counter active > forever. > If a view cleanup request comes, the old view file is not deleted since the > reference counter is not dropped by the old view group, keeping the file > descriptor in use forever. > This leakage is different from what is reported in COUCHDB-1129 but it's > somehow related. > The attached patch, simply shutdowns a view group when the design document is > updated and the new view signature changes, releasing the old view file > descriptor (as soon as no more clients are using the old view). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (COUCHDB-1309) File descriptor leaks on design document update and view cleanup
File descriptor leaks on design document update and view cleanup Key: COUCHDB-1309 URL: https://issues.apache.org/jira/browse/COUCHDB-1309 Project: CouchDB Issue Type: Bug Reporter: Filipe Manana Assignee: Filipe Manana If we add a design document with views defined in it, open the corresponding view group (by querying one of its views for e.g.), then update the design document in such a way that the view signature changes (changing a view's map function code for e.g), the old view group remains open forever (unless a server restart happens) and keeps it's view file reference counter active forever. If a view cleanup request comes, the old view file is not deleted since the reference counter is not dropped by the old view group, keeping the file descriptor in use forever. This leakage is different from what is reported in COUCHDB-1129 but it's somehow related. The attached patch, simply shutdowns a view group when the design document is updated and the new view signature changes, releasing the old view file descriptor (as soon as no more clients are using the old view). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
