all_dbs_active error and continuous replications

2011-12-06 Thread Gregor Martynus
Hey everybody,

I've an app where an own db gets created for each user which gets
replicated continuously to a big central db.
This works great, it gives me fantastic possibilities that wouldn't be
possible with other databases, I love it.

My problem is though, that at some point I run into the all_dbs_active
error. Once I got there, everything freezes. I have to restart couchDB,
that's really bad. From what I can tell, continuous replications to count
against this limit, is that correct? Does that mean, that every continuous
replication keeps the db file open, even if there is no change for a longer
time?

My big question is: is there anything I can do about it? I mean to make
this setup scale better? I know I can increase the limit in configuration,
but that would just give me some more time, not solve my problem.

I'm really concerned ... and I asked around everywhere, but nobody could
give me an answer yet. Please help, I'd really appreciate it.

Thanks!

× Gregor


Re: all_dbs_active error and continuous replications

2011-12-06 Thread Benoit Chesneau
On Tue, Dec 6, 2011 at 9:56 AM, Gregor Martynus gre...@martynus.net wrote:
 Hey everybody,

 I've an app where an own db gets created for each user which gets
 replicated continuously to a big central db.
 This works great, it gives me fantastic possibilities that wouldn't be
 possible with other databases, I love it.

 My problem is though, that at some point I run into the all_dbs_active
 error. Once I got there, everything freezes. I have to restart couchDB,
 that's really bad. From what I can tell, continuous replications to count
 against this limit, is that correct? Does that mean, that every continuous
 replication keeps the db file open, even if there is no change for a longer
 time?

 My big question is: is there anything I can do about it? I mean to make
 this setup scale better? I know I can increase the limit in configuration,
 but that would just give me some more time, not solve my problem.

 I'm really concerned ... and I asked around everywhere, but nobody could
 give me an answer yet. Please help, I'd really appreciate it.

 Thanks!

 × Gregor

You can increase max_dbs_open in [couchdb] section of the ini. It's by
default 100.

- benoît


Re: all_dbs_active error and continuous replications

2011-12-06 Thread Gregor Martynus
Thanks Benoit, I know about this setting. But at some point, this won't
work anymore. My question is if there is any way to stop continuous
replications to keep the db files active changes have been replicated? Does
that make sense?


On Tue, Dec 6, 2011 at 10:08 AM, Benoit Chesneau bchesn...@gmail.comwrote:

 On Tue, Dec 6, 2011 at 9:56 AM, Gregor Martynus gre...@martynus.net
 wrote:
  Hey everybody,
 
  I've an app where an own db gets created for each user which gets
  replicated continuously to a big central db.
  This works great, it gives me fantastic possibilities that wouldn't be
  possible with other databases, I love it.
 
  My problem is though, that at some point I run into the all_dbs_active
  error. Once I got there, everything freezes. I have to restart couchDB,
  that's really bad. From what I can tell, continuous replications to count
  against this limit, is that correct? Does that mean, that every
 continuous
  replication keeps the db file open, even if there is no change for a
 longer
  time?
 
  My big question is: is there anything I can do about it? I mean to make
  this setup scale better? I know I can increase the limit in
 configuration,
  but that would just give me some more time, not solve my problem.
 
  I'm really concerned ... and I asked around everywhere, but nobody could
  give me an answer yet. Please help, I'd really appreciate it.
 
  Thanks!
 
  × Gregor

 You can increase max_dbs_open in [couchdb] section of the ini. It's by
 default 100.

 - benoît



Re: all_dbs_active error and continuous replications

2011-12-06 Thread Benoit Chesneau
On Tue, Dec 6, 2011 at 10:11 AM, Gregor Martynus gre...@martynus.net wrote:
 Thanks Benoit, I know about this setting. But at some point, this won't
 work anymore. My question is if there is any way to stop continuous
 replications to keep the db files active changes have been replicated? Does
 that make sense?



other way would be to queue changes events and when replicated dbs 
max dbs open a new task if a change happened on this . Something like
it.

Otherwise to answer to your question, if you use the _replicator db
you can interrupt a task by just deleting the doc.

- benoît


Re: git commit: Fix distcheck make target

2011-12-06 Thread Filipe David Manana
On Tue, Dec 6, 2011 at 1:27 AM, Noah Slater nsla...@tumbolia.org wrote:
 Can you prefix these paths with $(top_builddir)/ please?

Done. Thanks for the top Noah.


 On Mon, Dec 5, 2011 at 4:58 PM, fdman...@apache.org wrote:

 Updated Branches:
  refs/heads/master b68edd1bb - 8f38cbed2


 Fix distcheck make target

 The check target now starts the utils/run script to execute
 the JavaScript test suite. This script leaves the files
 couchdb.stdout and couchdb.stderr which caused the distcheck
 target to fail.


 Project: http://git-wip-us.apache.org/repos/asf/couchdb/repo
 Commit: http://git-wip-us.apache.org/repos/asf/couchdb/commit/8f38cbed
 Tree: http://git-wip-us.apache.org/repos/asf/couchdb/tree/8f38cbed
 Diff: http://git-wip-us.apache.org/repos/asf/couchdb/diff/8f38cbed

 Branch: refs/heads/master
 Commit: 8f38cbed2b952e3eb27e36deeb59c15abdad344a
 Parents: b68edd1
 Author: Filipe David Borba Manana fdman...@apache.org
 Authored: Mon Dec 5 16:55:46 2011 +
 Committer: Filipe David Borba Manana fdman...@apache.org
 Committed: Mon Dec 5 16:55:46 2011 +

 --
  Makefile.am |    2 ++
  1 files changed, 2 insertions(+), 0 deletions(-)
 --


 http://git-wip-us.apache.org/repos/asf/couchdb/blob/8f38cbed/Makefile.am
 --
 diff --git a/Makefile.am b/Makefile.am
 index 01879dc..98c7be9 100644
 --- a/Makefile.am
 +++ b/Makefile.am
 @@ -113,6 +113,8 @@ distclean-local:
        rm -fr $(top_builddir)/etc/couchdb/default.d
        rm -fr $(top_builddir)/etc/couchdb/local.d
        rm -fr $(top_builddir)/tmp
 +       rm -f couchdb.stdout
 +       rm -f couchdb.stderr

  .PHONY: local-clean
  local-clean: maintainer-clean





-- 
Filipe David Manana,

Reasonable men adapt themselves to the world.
 Unreasonable men adapt the world to themselves.
 That's why all progress depends on unreasonable men.


Re: all_dbs_active error and continuous replications

2011-12-06 Thread Johannes J. Schmidt
Thanks Gregor for raising this question. I have a very similar setup and
I am also concerned about how it will scale.

g jo
Am Dienstag, den 06.12.2011, 10:29 +0100 schrieb Gregor Martynus:
 Thanks Benoit!
 
 I'm sorry to keep bugging ...
 
 The problem is, at least for me, that I can't simply delete docs from the
 _replicator db. It's a critical part of my couchDB setup. It is something
 that excited me a lot a about couchDB, I just want to setup these
 replications and then don't think about it any more, that's something that
 the DB should just do – and let me relax.
 
 What I really don't understand is, _why_ the continuous replication keep
 the db files open? I'm not an Erlang developer at all, sorry if this is a
 dump question/thought, but shouldn't the continuous replication be
 triggered only if a change happens in the db, instead of keeping it alive
 all the time? Is this a bug or probably something that might be improved in
 future?
 
 Thanks
 
 × Gregor
 
 On Tue, Dec 6, 2011 at 10:16 AM, Benoit Chesneau bchesn...@gmail.comwrote:
 
  On Tue, Dec 6, 2011 at 10:11 AM, Gregor Martynus gre...@martynus.net
  wrote:
   Thanks Benoit, I know about this setting. But at some point, this won't
   work anymore. My question is if there is any way to stop continuous
   replications to keep the db files active changes have been replicated?
  Does
   that make sense?
  
  
 
  other way would be to queue changes events and when replicated dbs 
  max dbs open a new task if a change happened on this . Something like
  it.
 
  Otherwise to answer to your question, if you use the _replicator db
  you can interrupt a task by just deleting the doc.
 
  - benoît
 




Re: all_dbs_active error and continuous replications

2011-12-06 Thread Filipe David Manana
On Tue, Dec 6, 2011 at 9:29 AM, Gregor Martynus gre...@martynus.net wrote:
 What I really don't understand is, _why_ the continuous replication keep
 the db files open? I'm not an Erlang developer at all, sorry if this is a
 dump question/thought, but shouldn't the continuous replication be
 triggered only if a change happens in the db, instead of keeping it alive
 all the time? Is this a bug or probably something that might be improved in
 future?

Actually it doesn't keep the source and database open when there's no
changes to transfer from the source to the target.
What happens is that when started, the replication records the
databases' instance start times - they're timestamps that tell when
the databases where open by the couchdb server.

Whenever the replication commits a checkpoint (it is committed against
the source and the target), it verifies if the endpoints' current
instance start times are the same as when it started. If they're not
the same, then you'll get that max_dbs_open error message from the
replicator.

When the system has max_dbs_open and a request to open another
database X comes, it will close the least recently used database Y in
order to open the new one. If later on a request to open the database
X that was previously closed (because we reached max_dbs_open)
arrives, the new least recently used database will be closed, and X
will be reopened - at this point it gets a new instance start
timestamp.

The reason to abort if the instance start timestamps changed, it to
prevent you from getting into an inconsistent state. For example,
during an idle period (when there's no changes to transfer from source
to target) someone could swap database files - if the replicator
ignored the timestamps, it could mean in the end your target would be
missing all the changes transferred before the file swap.

One thing I have been thinking for sometime is to have the server
remember the timestamps and header md5 of each database opened since
it started. When it reopens a database that was previously closed by
the LRU system (due to max_dbs_open), it would use the cached
timestamp if the database's header md5 is the same as the one that's
cached along with the timestamp.
I haven't convinced myself this is a flawless/good idea however and
never made extensive tests.



 Thanks

 × Gregor

 On Tue, Dec 6, 2011 at 10:16 AM, Benoit Chesneau bchesn...@gmail.comwrote:

 On Tue, Dec 6, 2011 at 10:11 AM, Gregor Martynus gre...@martynus.net
 wrote:
  Thanks Benoit, I know about this setting. But at some point, this won't
  work anymore. My question is if there is any way to stop continuous
  replications to keep the db files active changes have been replicated?
 Does
  that make sense?
 
 

 other way would be to queue changes events and when replicated dbs 
 max dbs open a new task if a change happened on this . Something like
 it.

 Otherwise to answer to your question, if you use the _replicator db
 you can interrupt a task by just deleting the doc.

 - benoît




-- 
Filipe David Manana,

Reasonable men adapt themselves to the world.
 Unreasonable men adapt the world to themselves.
 That's why all progress depends on unreasonable men.


Re: git commit: Fix distcheck make target

2011-12-06 Thread Noah Slater
Thanks. :)

On Tue, Dec 6, 2011 at 9:29 AM, Filipe David Manana fdman...@apache.orgwrote:

 On Tue, Dec 6, 2011 at 1:27 AM, Noah Slater nsla...@tumbolia.org wrote:
  Can you prefix these paths with $(top_builddir)/ please?

 Done. Thanks for the top Noah.

 
  On Mon, Dec 5, 2011 at 4:58 PM, fdman...@apache.org wrote:
 
  Updated Branches:
   refs/heads/master b68edd1bb - 8f38cbed2
 
 
  Fix distcheck make target
 
  The check target now starts the utils/run script to execute
  the JavaScript test suite. This script leaves the files
  couchdb.stdout and couchdb.stderr which caused the distcheck
  target to fail.
 
 
  Project: http://git-wip-us.apache.org/repos/asf/couchdb/repo
  Commit: http://git-wip-us.apache.org/repos/asf/couchdb/commit/8f38cbed
  Tree: http://git-wip-us.apache.org/repos/asf/couchdb/tree/8f38cbed
  Diff: http://git-wip-us.apache.org/repos/asf/couchdb/diff/8f38cbed
 
  Branch: refs/heads/master
  Commit: 8f38cbed2b952e3eb27e36deeb59c15abdad344a
  Parents: b68edd1
  Author: Filipe David Borba Manana fdman...@apache.org
  Authored: Mon Dec 5 16:55:46 2011 +
  Committer: Filipe David Borba Manana fdman...@apache.org
  Committed: Mon Dec 5 16:55:46 2011 +
 
  --
   Makefile.am |2 ++
   1 files changed, 2 insertions(+), 0 deletions(-)
  --
 
 
 
 http://git-wip-us.apache.org/repos/asf/couchdb/blob/8f38cbed/Makefile.am
  --
  diff --git a/Makefile.am b/Makefile.am
  index 01879dc..98c7be9 100644
  --- a/Makefile.am
  +++ b/Makefile.am
  @@ -113,6 +113,8 @@ distclean-local:
 rm -fr $(top_builddir)/etc/couchdb/default.d
 rm -fr $(top_builddir)/etc/couchdb/local.d
 rm -fr $(top_builddir)/tmp
  +   rm -f couchdb.stdout
  +   rm -f couchdb.stderr
 
   .PHONY: local-clean
   local-clean: maintainer-clean
 
 



 --
 Filipe David Manana,

 Reasonable men adapt themselves to the world.
  Unreasonable men adapt the world to themselves.
  That's why all progress depends on unreasonable men.



[jira] [Created] (COUCHDB-1357) Authentication failure after updating password in user document

2011-12-06 Thread Filipe Manana (Created) (JIRA)
Authentication failure after updating password in user document
---

 Key: COUCHDB-1357
 URL: https://issues.apache.org/jira/browse/COUCHDB-1357
 Project: CouchDB
  Issue Type: Bug
Affects Versions: 1.1.1
Reporter: Filipe Manana


From the report at the users mailing list:

http://s.apache.org/9OG

Seems like after updating the password in a user doc, the user is not able to 
login with the new password unless Couch is restarted. Sounds like a caching 
issue.
The only case of getting the cache consistent with the _users database content 
is if the _users database processes crash and after the crash user documents 
are updated. The cache daemon is ignoring the database crash.

The following patch updates the daemon to monitor the _users database and crash 
(letting the supervisor restart it) if the database process crashes.
Etap test included.

This might be related to COUCHDB-1212.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (COUCHDB-1357) Authentication failure after updating password in user document

2011-12-06 Thread Pete Vander Giessen (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/COUCHDB-1357?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13163942#comment-13163942
 ] 

Pete Vander Giessen commented on COUCHDB-1357:
--

We ran into a similar issue, with 1.1.1. The problem seemed to start with a log 
entry that looked like so:

[Thu, 01 Dec 2011 19:22:48 GMT] [error] [0.31233.1889] Uncaught
error in HTTP request: {error,system_limit}
[Thu, 01 Dec 2011 19:22:48 GMT] [info] [0.31233.1889] Stacktrace:
[{erlang,open_port,
 [{spawn,couch_icu_driver},[]]},
{couch_util,drv_port,0},
{couch_util,collate,3},
{couch_view,less_json_ids,2},
{couch_btree,drop_nodes,4},
{couch_btree,stream_kp_node,8},
{couch_btree,fold,4},
{couch_view,fold,4}]

We had a cluster of 7 or so Windows users replicating 2 databases apiece from a 
master server running on a Red Hat box. Each user also has a separate user 
database that they use to fetch oauth credentials, so there were perhaps 16 
databases in use at the time.

Next time this happens, I'll run netstat on the machine and save the output. 
For now, I've saved off the logs, and can provide more complete logs upon 
request.

 Authentication failure after updating password in user document
 ---

 Key: COUCHDB-1357
 URL: https://issues.apache.org/jira/browse/COUCHDB-1357
 Project: CouchDB
  Issue Type: Bug
Affects Versions: 1.1.1
Reporter: Filipe Manana
 Attachments: COUCHDB-1357-master.patch


 From the report at the users mailing list:
 http://s.apache.org/9OG
 Seems like after updating the password in a user doc, the user is not able to 
 login with the new password unless Couch is restarted. Sounds like a caching 
 issue.
 The only case of getting the cache consistent with the _users database 
 content is if the _users database processes crash and after the crash user 
 documents are updated. The cache daemon is ignoring the database crash.
 The following patch updates the daemon to monitor the _users database and 
 crash (letting the supervisor restart it) if the database process crashes.
 Etap test included.
 This might be related to COUCHDB-1212.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




The perfect logger for development

2011-12-06 Thread Jason Smith
Hi, all. Iris Couch urgently needs improved logging facilities in
CouchDB. My goal is to make something we all love and get it accepted
upstream, God willing. Or committers willing. But I repeat myself!

This is the brainstorming and requirements gathering phase. In the
CouchDB of your dreams, logging system fits you like an old pair of
sneakers. It's perfect. Now, what characteristics does that system
exhibit? I will compile feedback into a spec on the wiki.

I hope to avoid bikeshedding. Seriously, please don't even mention a
product or project by name. At this phase I hope to stick to
descriptions of functionality, goals, and non-goals. I want to
evaluate tools later.

To start the discussion: logging is viewed differently based on your
relationship with CouchDB:

1. Developers of CouchDB
2. Administrators of a couch
3. Application developers

My roles are administration, and a little bit of development.
Requirements, in no order.

* Idiomatic Erlang

* Is a compelling place for new people to contribute. Miguel de Icaza
talks about this. It's not enough that the source code is public. You
have to provide a smooth on-ramp, where people people get great bang
for their buck. They write a modest feature, and are rewarded by
seeing its effects immediately. In other words: plugins. Or maybe a
behaviour. Or some way to swap in formatters and data sinks. I don't
want to write a Loggly target (http://loggly.com). Loggly should be
begging me to merge their module.

* 1st cut, no change to the default behavior. You still get the that
peculiar log file you know and love. People are parsing their log
files, and might expect version 1.x not to change.

* Existing code still works. No sweeping changes hitting every
?LOG_INFO and ?LOG_DEBUG.

(Filipe, would you please share your thoughts on these? I think you
struggled with the conflict between them recently.)
* No performance impact (non-blocking)...
* ... but also, impossible or difficult to overwhelm or crash or lose logs.

(The next few points sort of fit together)

* Logs are not strings, but data structures, with data (the log
message) and metadata (severity, line number, maybe the call stack,
etc.)

* More log levels. Roughly: trace, debug, info, warn, error, fatal

* Maybe automatic trace logs upon function entry/exit with return
values. Not sure if this is doable. Maybe a compile option, for `make
dev`

* When you log something, your module, line number, and maybe PID are known

* Components or categories, or tags, where multiple .erl files or
individual log events can share a common property (http, views,

* A policy decides what to do with logs based on level, module, or
component. You can set the policy either via configuration or
programatically.

* There is a formatter API to serialize log data. Built-in formatters
include the legacy format, and Jan's Apache httpd combined format.

* There is a transport API to receive logs and DTRT.

* I know this is insane, but kill -HUP pid should make couch reopen
its log files. Okay, I'll settle down now.

= Non Goals =

* Log rotation. I have never seen a rotation feature in an application
which was better than the OS tools. And you can get problem where both
the server and the OS are rotating the same logs. I have seen that
happen, twice. Or was it once? Of course, people could write a
rotating file transport.

-- 
Iris Couch


Re: The perfect logger for development

2011-12-06 Thread kowsik
As a CouchDB administrator, I would want *all* exception dumps to be
prefaced by the inbound request URL with the query parameters
(assuming it's a web request that caused the exception). There are
case where I've seen a stack trace but couldn't tell which inbound
request caused the problem.

K.
---
http://blitz.io
@pcapr

On Tue, Dec 6, 2011 at 5:51 PM, Jason Smith j...@iriscouch.com wrote:
 Hi, all. Iris Couch urgently needs improved logging facilities in
 CouchDB. My goal is to make something we all love and get it accepted
 upstream, God willing. Or committers willing. But I repeat myself!

 This is the brainstorming and requirements gathering phase. In the
 CouchDB of your dreams, logging system fits you like an old pair of
 sneakers. It's perfect. Now, what characteristics does that system
 exhibit? I will compile feedback into a spec on the wiki.

 I hope to avoid bikeshedding. Seriously, please don't even mention a
 product or project by name. At this phase I hope to stick to
 descriptions of functionality, goals, and non-goals. I want to
 evaluate tools later.

 To start the discussion: logging is viewed differently based on your
 relationship with CouchDB:

 1. Developers of CouchDB
 2. Administrators of a couch
 3. Application developers

 My roles are administration, and a little bit of development.
 Requirements, in no order.

 * Idiomatic Erlang

 * Is a compelling place for new people to contribute. Miguel de Icaza
 talks about this. It's not enough that the source code is public. You
 have to provide a smooth on-ramp, where people people get great bang
 for their buck. They write a modest feature, and are rewarded by
 seeing its effects immediately. In other words: plugins. Or maybe a
 behaviour. Or some way to swap in formatters and data sinks. I don't
 want to write a Loggly target (http://loggly.com). Loggly should be
 begging me to merge their module.

 * 1st cut, no change to the default behavior. You still get the that
 peculiar log file you know and love. People are parsing their log
 files, and might expect version 1.x not to change.

 * Existing code still works. No sweeping changes hitting every
 ?LOG_INFO and ?LOG_DEBUG.

 (Filipe, would you please share your thoughts on these? I think you
 struggled with the conflict between them recently.)
 * No performance impact (non-blocking)...
 * ... but also, impossible or difficult to overwhelm or crash or lose logs.

 (The next few points sort of fit together)

 * Logs are not strings, but data structures, with data (the log
 message) and metadata (severity, line number, maybe the call stack,
 etc.)

 * More log levels. Roughly: trace, debug, info, warn, error, fatal

 * Maybe automatic trace logs upon function entry/exit with return
 values. Not sure if this is doable. Maybe a compile option, for `make
 dev`

 * When you log something, your module, line number, and maybe PID are known

 * Components or categories, or tags, where multiple .erl files or
 individual log events can share a common property (http, views,

 * A policy decides what to do with logs based on level, module, or
 component. You can set the policy either via configuration or
 programatically.

 * There is a formatter API to serialize log data. Built-in formatters
 include the legacy format, and Jan's Apache httpd combined format.

 * There is a transport API to receive logs and DTRT.

 * I know this is insane, but kill -HUP pid should make couch reopen
 its log files. Okay, I'll settle down now.

 = Non Goals =

 * Log rotation. I have never seen a rotation feature in an application
 which was better than the OS tools. And you can get problem where both
 the server and the OS are rotating the same logs. I have seen that
 happen, twice. Or was it once? Of course, people could write a
 rotating file transport.

 --
 Iris Couch


[jira] [Commented] (COUCHDB-1357) Authentication failure after updating password in user document

2011-12-06 Thread Randall Leeds (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/COUCHDB-1357?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13164121#comment-13164121
 ] 

Randall Leeds commented on COUCHDB-1357:


Patch looks good.

 Authentication failure after updating password in user document
 ---

 Key: COUCHDB-1357
 URL: https://issues.apache.org/jira/browse/COUCHDB-1357
 Project: CouchDB
  Issue Type: Bug
Affects Versions: 1.1.1
Reporter: Filipe Manana
 Attachments: COUCHDB-1357-master.patch


 From the report at the users mailing list:
 http://s.apache.org/9OG
 Seems like after updating the password in a user doc, the user is not able to 
 login with the new password unless Couch is restarted. Sounds like a caching 
 issue.
 The only case of getting the cache consistent with the _users database 
 content is if the _users database processes crash and after the crash user 
 documents are updated. The cache daemon is ignoring the database crash.
 The following patch updates the daemon to monitor the _users database and 
 crash (letting the supervisor restart it) if the database process crashes.
 Etap test included.
 This might be related to COUCHDB-1212.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




Re: The perfect logger for development

2011-12-06 Thread Jason Smith
Brilliant, thanks!

I think this is possible if the Req object is part of the log object.
Then a formatter can access it there. Off the top of my head, it would
have access to the source IP address, the HTTP method, the path and
query string, and headers.

On Wed, Dec 7, 2011 at 10:44 AM, kowsik kow...@gmail.com wrote:
 As a CouchDB administrator, I would want *all* exception dumps to be
 prefaced by the inbound request URL with the query parameters
 (assuming it's a web request that caused the exception). There are
 case where I've seen a stack trace but couldn't tell which inbound
 request caused the problem.

 K.
 ---
 http://blitz.io
 @pcapr

 On Tue, Dec 6, 2011 at 5:51 PM, Jason Smith j...@iriscouch.com wrote:
 Hi, all. Iris Couch urgently needs improved logging facilities in
 CouchDB. My goal is to make something we all love and get it accepted
 upstream, God willing. Or committers willing. But I repeat myself!

 This is the brainstorming and requirements gathering phase. In the
 CouchDB of your dreams, logging system fits you like an old pair of
 sneakers. It's perfect. Now, what characteristics does that system
 exhibit? I will compile feedback into a spec on the wiki.

 I hope to avoid bikeshedding. Seriously, please don't even mention a
 product or project by name. At this phase I hope to stick to
 descriptions of functionality, goals, and non-goals. I want to
 evaluate tools later.

 To start the discussion: logging is viewed differently based on your
 relationship with CouchDB:

 1. Developers of CouchDB
 2. Administrators of a couch
 3. Application developers

 My roles are administration, and a little bit of development.
 Requirements, in no order.

 * Idiomatic Erlang

 * Is a compelling place for new people to contribute. Miguel de Icaza
 talks about this. It's not enough that the source code is public. You
 have to provide a smooth on-ramp, where people people get great bang
 for their buck. They write a modest feature, and are rewarded by
 seeing its effects immediately. In other words: plugins. Or maybe a
 behaviour. Or some way to swap in formatters and data sinks. I don't
 want to write a Loggly target (http://loggly.com). Loggly should be
 begging me to merge their module.

 * 1st cut, no change to the default behavior. You still get the that
 peculiar log file you know and love. People are parsing their log
 files, and might expect version 1.x not to change.

 * Existing code still works. No sweeping changes hitting every
 ?LOG_INFO and ?LOG_DEBUG.

 (Filipe, would you please share your thoughts on these? I think you
 struggled with the conflict between them recently.)
 * No performance impact (non-blocking)...
 * ... but also, impossible or difficult to overwhelm or crash or lose logs.

 (The next few points sort of fit together)

 * Logs are not strings, but data structures, with data (the log
 message) and metadata (severity, line number, maybe the call stack,
 etc.)

 * More log levels. Roughly: trace, debug, info, warn, error, fatal

 * Maybe automatic trace logs upon function entry/exit with return
 values. Not sure if this is doable. Maybe a compile option, for `make
 dev`

 * When you log something, your module, line number, and maybe PID are known

 * Components or categories, or tags, where multiple .erl files or
 individual log events can share a common property (http, views,

 * A policy decides what to do with logs based on level, module, or
 component. You can set the policy either via configuration or
 programatically.

 * There is a formatter API to serialize log data. Built-in formatters
 include the legacy format, and Jan's Apache httpd combined format.

 * There is a transport API to receive logs and DTRT.

 * I know this is insane, but kill -HUP pid should make couch reopen
 its log files. Okay, I'll settle down now.

 = Non Goals =

 * Log rotation. I have never seen a rotation feature in an application
 which was better than the OS tools. And you can get problem where both
 the server and the OS are rotating the same logs. I have seen that
 happen, twice. Or was it once? Of course, people could write a
 rotating file transport.

 --
 Iris Couch



-- 
Iris Couch


[jira] [Commented] (COUCHDB-1356) POST _session responds with name: null if _admin user and no _users doc present

2011-12-06 Thread Jason Smith (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/COUCHDB-1356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13164200#comment-13164200
 ] 

Jason Smith commented on COUCHDB-1356:
--

Hi, Johannes. I believe the userCtx.name value indicates the name of the 
document in the _users database, or `null` to indicate no corresponding 
document. Thus, CouchDB is communicating that this request is authorized but 
not to any specific user on the server. (Something similar happens if you have 
an /etc/passwd, NIS, or LDAP error; or if you remove a Unix user and look at 
their old files. User and group ownership will be indicated by the underlying 
integer. Both the type and value communicate information.)

CouchDB also uses null to indicate that it is in Admin Party mode. If you query 
/_session without authorization data, the name will be null. If the roles 
include _admin, then Admin Party mode is active.

 POST _session responds with name: null if _admin user and no _users doc 
 present
 ---

 Key: COUCHDB-1356
 URL: https://issues.apache.org/jira/browse/COUCHDB-1356
 Project: CouchDB
  Issue Type: Bug
Affects Versions: 1.1.1
Reporter: Johannes J. Schmidt
Priority: Minor

 When logging in with admin credentials (and no corresponding _users doc, if 
 that is important), the response of the POST to _session has the name 
 property set to null:
   {ok:true,name:null,roles:[_admin]}
 It should be the name of the admin instead, like it does when logging in with 
 a standard user:
   {ok:true,name:standarduser,roles:[]}
 Requesting the _session object after logging in with an admin, the name is 
 proper set:
 {ok:true,userCtx:{name:adminuser,roles:[_admin]},info:{authentication_db:_users,authentication_handlers:[oauth,cookie,default],authenticated:cookie}}
 Johannes

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira