[Couchdb Wiki] Update of Installation by RandallLeeds
Dear Wiki user, You have subscribed to a wiki page or wiki category on Couchdb Wiki for change notification. The Installation page has been changed by RandallLeeds. http://wiki.apache.org/couchdb/Installation?action=diffrev1=22rev2=23 -- == General Production Deployment Tips == '''Third Party Tool Configuration''' + * [[Performance| Tips for getting more performance out of CouchDB]] * [[Apache_As_a_Reverse_Proxy| How to configure Apache as a reverse proxy.]]
[Couchdb Wiki] Update of CouchDB_in_the_wild by Linicks
Dear Wiki user, You have subscribed to a wiki page or wiki category on Couchdb Wiki for change notification. The CouchDB_in_the_wild page has been changed by Linicks. The comment on this change is: Add link to Anologue IRC like web app.. http://wiki.apache.org/couchdb/CouchDB_in_the_wild?action=diffrev1=88rev2=89 -- == Software == * [[http://www.ajatus.info/|Ajatus]] A CRM that runs as a local Ajax web application + * [[http://www.anologue.com/|Anologue]] is like comments, meets im, meets irc, meets your favorite paste app, meets instant coffee. * [[http://www.aptela.com/|Aptela]] Hosted VoIP/PBX Service that uses Couch for persistence services * [[http://www.couch.it/|Couchit]], minimal site creator ([[http://bitbucket.org/benoitc/couchit/|source code]]) * [[http://github.com/benoitc/couchdbproxy|CouchdbProxy]], Simple multinode couchdb proxy
[Couchdb Wiki] Update of Performance by RandallLeeds
Dear Wiki user,
You have subscribed to a wiki page or wiki category on Couchdb Wiki for
change notification.
The Performance page has been changed by RandallLeeds.
http://wiki.apache.org/couchdb/Performance?action=diffrev1=3rev2=4
--
As an example one of my benchmarks turned out to be mostly limited by the
json module's encoding and decoding speed. The process was using 40% of a CPU.
Switching to simplejson with no other changes resulted in 5% of a CPU.
Switching from threads to processes (using multiprocessing module) gave yet
another performance improvement finally pushing CouchDB to consume more than
100% of a CPU (this is on a multi-processor machine).
+ = Resource Limits =
+ One of the problems that administrators run into as their deployments become
large are resource limits imposed by the system and by the application
configuration. Raising these limits can allow your deployment to grow beyond
what the default configuration will support.
+ == CouchDB Configuration Options ==
+ In your configuration (local.ini or similar) familiarize yourself with the
following options:{{{
+ [couchdb]
+ max_dbs_open = 100
+
+ [httpd]
+ max_connections = 2048}}}
+ The first option places an upper bound on the number of databases that can be
open at one time. CouchDB reference counts database accesses internally and
will close idle databases when it must. Sometimes it is necessary to keep more
than the default open at once, such as in deployments where many databases will
be continuously replicating.
+ The second option limits how many client connections the HTTP server will
service at a time. Again, heavy replication scenarios are good candidates for
increased {{{max_connections}}} since the replicator opens several connections
to the source database.
+ == System Resource Limits ==
+ === Erlang ===
+ Even if you've increased the maximum connections CouchDB will allow, the
Erlang runtime system will not allow more than 1024 connections by default.
Setting the following option in {{{(prefix)/etc/default/couchdb}}} (or
equivalent) will increase this limit (in this case to 4096): {{{export
ERL_MAX_PORTS=4096}}}
+ === PAM and ulimit ===
+ Finally, most *nix operating systems impose various resource limits on every
process. If your system is set up to use the Pluggable Authentication Modules
(PAM) system, increasing this limit is straightforward. For example, creating a
file named {{{/etc/security/limits.d/100-couchdb.conf}}} with the following
contents will ensure that CouchDB can open enough file descriptors to service
your increased maximum open databases and Erlang ports:{{{
+ #domaintypeitemvalue
+ couchdb hard nofile4096
+ couchdb soft nofile4096}}}
+ If your system does not use PAM, a {{{ulimit}}} command is usually available
for use in a custom script to launch CouchDB with increased resource limits.
+ If necessary, feel free to increase this limits as long as your hardware can
handle the load.
+
[Couchdb Wiki] Update of Performance by RandallLeeds
Dear Wiki user,
You have subscribed to a wiki page or wiki category on Couchdb Wiki for
change notification.
The Performance page has been changed by RandallLeeds.
http://wiki.apache.org/couchdb/Performance?action=diffrev1=4rev2=5
--
The second option limits how many client connections the HTTP server will
service at a time. Again, heavy replication scenarios are good candidates for
increased {{{max_connections}}} since the replicator opens several connections
to the source database.
== System Resource Limits ==
=== Erlang ===
- Even if you've increased the maximum connections CouchDB will allow, the
Erlang runtime system will not allow more than 1024 connections by default.
Setting the following option in {{{(prefix)/etc/default/couchdb}}} (or
equivalent) will increase this limit (in this case to 4096): {{{export
ERL_MAX_PORTS=4096}}}
+ Even if you've increased the maximum connections CouchDB will allow, the
Erlang runtime system will not allow more than 1024 connections by default.
Adding the following directive to {{{(prefix)/etc/default/couchdb}}} (or
equivalent) will increase this limit (in this case to 4096):{{{
+ export ERL_MAX_PORTS=4096}}}
=== PAM and ulimit ===
Finally, most *nix operating systems impose various resource limits on every
process. If your system is set up to use the Pluggable Authentication Modules
(PAM) system, increasing this limit is straightforward. For example, creating a
file named {{{/etc/security/limits.d/100-couchdb.conf}}} with the following
contents will ensure that CouchDB can open enough file descriptors to service
your increased maximum open databases and Erlang ports:{{{
#domaintypeitemvalue
[Couchdb Wiki] Update of Performance by RandallLeeds
Dear Wiki user,
You have subscribed to a wiki page or wiki category on Couchdb Wiki for
change notification.
The Performance page has been changed by RandallLeeds.
The comment on this change is: Add information on +A n flag.
http://wiki.apache.org/couchdb/Performance?action=diffrev1=5rev2=6
--
If your system does not use PAM, a {{{ulimit}}} command is usually available
for use in a custom script to launch CouchDB with increased resource limits.
If necessary, feel free to increase this limits as long as your hardware can
handle the load.
+ = Disk and File System Performance =
+ Using faster disks, striped RAID arrays and modern file systems can all speed
up your CouchDB deployment. However, there is one option that can increase the
responsiveness of your CouchDB server when disk performance is a bottleneck.
From the erlang documentation for the file module: {{{
+ On operating systems with thread support, it is possible to let file
operations be performed in threads of their own, allowing other Erlang
processes to continue executing in parallel with the file operations. See the
command line flag +A in erl(1).}}}
+ Setting this argument to a number greater than zero can keep your CouchDB
installation responsive even during periods of heavy disk utilization. The
easiest way to set this option is through the {{{ERL_FLAGS}}} environment
variable. For example, to give Erlang four threads with which to perform i/o
operations add the following to {{{(prefix)/etc/defaults/couchdb}}} (or
equivalent): {{{
+ export ERL_FLAGS=+A 4}}}
+
svn commit: r955773 - in /couchdb/branches/new_replicator/src/couchdb: couch_api_wrap.erl couch_api_wrap.hrl couch_doc.erl couch_httpd_rep.erl couch_replicate.erl couch_replicate.hrl
Author: damien
Date: Thu Jun 17 22:35:49 2010
New Revision: 955773
URL: http://svn.apache.org/viewvc?rev=955773view=rev
Log:
Reorganized and commented the new replicator code for easier understanding.
Added:
couchdb/branches/new_replicator/src/couchdb/couch_api_wrap.hrl
- copied unchanged from r955001,
couchdb/branches/new_replicator/src/couchdb/couch_replicate.hrl
Removed:
couchdb/branches/new_replicator/src/couchdb/couch_replicate.hrl
Modified:
couchdb/branches/new_replicator/src/couchdb/couch_api_wrap.erl
couchdb/branches/new_replicator/src/couchdb/couch_doc.erl
couchdb/branches/new_replicator/src/couchdb/couch_httpd_rep.erl
couchdb/branches/new_replicator/src/couchdb/couch_replicate.erl
Modified: couchdb/branches/new_replicator/src/couchdb/couch_api_wrap.erl
URL:
http://svn.apache.org/viewvc/couchdb/branches/new_replicator/src/couchdb/couch_api_wrap.erl?rev=955773r1=955772r2=955773view=diff
==
--- couchdb/branches/new_replicator/src/couchdb/couch_api_wrap.erl (original)
+++ couchdb/branches/new_replicator/src/couchdb/couch_api_wrap.erl Thu Jun 17
22:35:49 2010
@@ -12,13 +12,29 @@
-module(couch_api_wrap).
+% This module wraps the native erlang API, and allows for performing
+% operations on a remote vs. local databases via the same API.
+%
+% Notes:
+% Many options and apis aren't yet supported here, they are added as needed.
+%
+% This file neesds a lot of work to robustify the common failures, and
+% convert the json errors back to Erlang style errors.
+%
+% Also, we open a new connection for every HTTP call, to avoid the
+% problems when requests are pipelined over a single connection and earlier
+% requests that fail and disconnect don't cause network errors for other
+% requests. This should eventually be optimized so each process has it's own
+% connection that's kept alive between requests.
+%
-include(couch_db.hrl).
--include(couch_replicate.hrl).
+-include(couch_api_wrap.hrl).
-include(../ibrowse/ibrowse.hrl).
-export([
db_open/2,
+db_close/1,
get_db_info/1,
open_doc/3,
update_doc/3,
@@ -34,6 +50,12 @@ db_open(#httpdb{}=Db, _Options) -
db_open(DbName, Options) -
couch_db:open(DbName,Options).
+db_close(#httpdb{}) -
+ok;
+db_close(DbName) -
+couch_db:close(DbName).
+
+
get_db_info(#httpdb{url=Url,oauth=OAuth,headers=Headers}) -
Headers2 = oauth_header(Url, [], get, OAuth) ++ Headers,
case ibrowse:send_req(Url, Headers2, get, [], [
@@ -67,9 +89,11 @@ open_doc(#httpdb{url=Url,oauth=OAuth,hea
open_doc(Db, DocId, Options) -
couch_db:open_doc(Db, DocId, Options).
+
update_doc(Db, Doc, Options) -
update_doc(Db,Doc,Options,interactive_edit).
+
ensure_full_commit(#httpdb{url=Url,oauth=OAuth,headers=Headers}) -
Headers2 = oauth_header(Url, [], post, OAuth) ++ Headers,
#url{host=Host,port=Port}=ibrowse_lib:parse_url(Url),
@@ -85,6 +109,7 @@ ensure_full_commit(#httpdb{url=Url,oauth
ensure_full_commit(Db) -
couch_db:ensure_full_commit(Db).
+
get_missing_revs(#httpdb{url=Url,oauth=OAuth,headers=Headers}, IdRevs) -
Json = [{Id, couch_doc:revs_to_strs(Revs)} || {Id, Revs} - IdRevs],
Headers2 = oauth_header(Url, [], post, OAuth) ++ Headers,
@@ -109,22 +134,6 @@ get_missing_revs(Db, IdRevs) -
couch_db:get_missing_revs(Db, IdRevs).
-options_to_query_args([], Acc) -
-lists:reverse(Acc);
-options_to_query_args([delay_commit|Rest], Acc) -
-options_to_query_args(Rest, Acc);
-options_to_query_args([{atts_since,[]}|Rest], Acc) -
-options_to_query_args(Rest, Acc);
-options_to_query_args([{atts_since,PossibleAncestors}|Rest], Acc) -
-options_to_query_args(Rest, [{atts_since,?JSON_ENCODE(
-couch_doc:revs_to_strs(PossibleAncestors))} | Acc]).
-
-query_args_to_string([], []) -
-;
-query_args_to_string([], Acc) -
-? ++ string:join(lists:reverse(Acc), );
-query_args_to_string([{K,V}|Rest], Acc) -
-query_args_to_string(Rest, [(K ++ = ++ V) | Acc]).
open_doc_revs(#httpdb{url=Url,oauth=OAuth,headers=Headers}, Id, Revs,
Options, Fun, Acc) -
@@ -163,6 +172,107 @@ open_doc_revs(Db, Id, Revs, Options, Fun
{ok, Results} = couch_db:open_doc_revs(Db, Id, Revs, Options),
{ok, lists:foldl(Fun, Acc, Results)}.
+
+
+update_doc(#httpdb{url=Url,headers=Headers,oauth=OAuth},Doc,Options,Type) -
+QArgs = if Type == replicated_changes -
+[{new_edits, false}]; true - [] end ++
+options_to_query_args(Options, []),
+
+Boundary = couch_uuids:random(),
+JsonBytes = ?JSON_ENCODE(couch_doc:to_json_obj(Doc,
[revs,attachments,follows|Options])),
+{ContentType, Len} = couch_doc:len_doc_to_multi_part_stream(Boundary,
+JsonBytes, Doc#doc.atts, false),
+Self = self(),
+Headers2 = case lists:member(delay_commit, Options) of
+true - [{X-Couch-Full-Commit, false}];
+false - []
+
svn commit: r955782 - /couchdb/trunk/share/www/script/jquery.couch.js
Author: jchris
Date: Thu Jun 17 23:17:15 2010
New Revision: 955782
URL: http://svn.apache.org/viewvc?rev=955782view=rev
Log:
jquery.couch.js works with PUT to delete.
Modified:
couchdb/trunk/share/www/script/jquery.couch.js
Modified: couchdb/trunk/share/www/script/jquery.couch.js
URL:
http://svn.apache.org/viewvc/couchdb/trunk/share/www/script/jquery.couch.js?rev=955782r1=955781r2=955782view=diff
==
--- couchdb/trunk/share/www/script/jquery.couch.js [utf-8] (original)
+++ couchdb/trunk/share/www/script/jquery.couch.js [utf-8] Thu Jun 17 23:17:15
2010
@@ -372,7 +372,7 @@
beforeSend : beforeSend,
complete: function(req) {
var resp = $.httpData(req, json);
- if (req.status == 201 || req.status == 202) {
+ if (req.status == 200 || req.status == 201 || req.status == 202)
{
doc._id = resp.id;
doc._rev = resp.rev;
if (versioned) {
