from:"Paul Joseph Davis $Commented$ $JIRA$"

[jira] [Commented] (COUCHDB-602) Cannot use NFS for storage

2012-04-11 Thread Paul Joseph Davis (Commented) (JIRA)

[
https://issues.apache.org/jira/browse/COUCHDB-602?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13251646#comment-13251646
]

Paul Joseph Davis commented on COUCHDB-602:
---

Cool. Though a quick note that we still don't recommend running over NFS. Users
doing such need to be aware of the issues involved.

Cannot use NFS for storage
--

Key: COUCHDB-602
URL: https://issues.apache.org/jira/browse/COUCHDB-602
Project: CouchDB
Issue Type: Bug
Components: Test Suite
Affects Versions: 0.10
Reporter: Nils Breunese
Attachments: error.gz

Our hosting provider claims they cannot upgrade our CouchDB installation from
version 0.9.1 to 0.10 as the test suite for 0.10 is failing.
This is a translation from their report:

CouchDB can't handle having its data directory on NFS.
It seems CouchDB tries to do this:
1 mkdir test
2 start process X with fd under test/foo
3 do the work
4 rm test/foo
5 rmdir test
6 kill process X
When test/ is on NFS:
In step 4 an open file is being removed. NFS doesn't really support this
and the workaround is that the client implementation does
mv file .nfsXXX
However, the 'rmdir test' in step 5 then cannot succeed anymore, since
the directory is not empty. This eventually causes CouchDB to crash.
In earlier versions of CouchDB the open/rm/rmdir/kill order may have
been different, for instance open/rm/kill/rmdir.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (COUCHDB-1445) CouchDB deletes .view files if it can't open them, even if the error is emfile.

2012-03-21 Thread Paul Joseph Davis (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/COUCHDB-1445?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13235361#comment-13235361
 ] 

Paul Joseph Davis commented on COUCHDB-1445:


It'd probably have to be a manual merge of the diff from 1.2 since the view 
engine is so different.

 CouchDB deletes .view files if it can't open them, even if the error is 
 emfile.
 -

 Key: COUCHDB-1445
 URL: https://issues.apache.org/jira/browse/COUCHDB-1445
 Project: CouchDB
  Issue Type: Bug
  Components: JavaScript View Server
Affects Versions: 1.1.1, 1.2, 1.3
Reporter: Jan Lehnardt
Assignee: Randall Leeds
Priority: Blocker
 Fix For: 1.2, 1.3, 1.1.2


 Via Stefan Kögl on dev@:
 Another thing I noticed during my tests of CouchDB 1.2.x. I redirected
 live traffic to the instance and after a rather short time, requests
 were failing with the following information in the logs:
 [Sun, 18 Mar 2012 16:39:24 GMT] [error] [0.27554.2]
 {error_report,0.31.0,
{0.27554.2,std_error,
 [{application,mochiweb},
  Accept failed error,
  {error,emfile}]}}
 [Sun, 18 Mar 2012 16:39:24 GMT] [error] [0.27554.2]
 {error_report,0.31.0,
  {0.27554.2,crash_report,
   [[{initial_call,
 {mochiweb_acceptor,init,
 ['Argument__1','Argument__2',
  'Argument__3']}},
 {pid,0.27554.2},
 {registered_name,[]},
 {error_info,
 {exit,
 {error,accept_failed},
 [{mochiweb_acceptor,init,3},
  {proc_lib,init_p_do_apply,3}]}},
 {ancestors,
 [couch_httpd,couch_secondary_services,
  couch_server_sup,0.32.0]},
 {messages,[]},
 {links,[0.129.0]},
 {dictionary,[]},
 {trap_exit,false},
 {status,running},
 {heap_size,233},
 {stack_size,24},
 {reductions,244}],
[]]}}
 I think emfile means that CouchDB (or mochiweb?) couldn't open any
 more files / connections. I've set the (hard and soft) nofile limit for
 user couchdb to 4096, but didn't raise the ERL_MAX_PORTS accordingly.
 Anyway, as soon as the error occured, CouchDB started writing most of my
 view files from scratch, rendering the instance unusable.
 I'd expect CouchDB to fail more gracefully when the maximum number of
 open files is reached. Is this a bug or expected behaviour?

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (COUCHDB-1446) Config settings input validation

2012-03-20 Thread Paul Joseph Davis (Commented) (JIRA)

[
https://issues.apache.org/jira/browse/COUCHDB-1446?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13233271#comment-13233271
]

Paul Joseph Davis commented on COUCHDB-1446:

I understand your concern, but (at least at this hour after a few beers) I'm
not seeing much of a distinction. A validate all the inputs doesn't seem far
off the (IMO simpler) reorganization to Erlang term format.

IMO, the immediate solution to this patch is fixing the bug the user encounter
at the source where the config value is used and the fancier validation split
off as its own thing.

Config settings input validation

Key: COUCHDB-1446
URL: https://issues.apache.org/jira/browse/COUCHDB-1446
Project: CouchDB
Issue Type: Bug
Components: Database Core
Affects Versions: 1.2
Reporter: Jason Smith
Assignee: Jason Smith
Priority: Minor
Labels: config

Today I received a bug report from a user who thought CouchDB might be
relaxing and evaluate an arithmetic expression in the config. (That makes
sense, since it seems to evaluate erlang terms elsewhere.)
https://getsatisfaction.com/iriscouch/topics/my_couchdb_is_broken_after_putting_a_badarg_for_a_configuration_value
It would be nice for CouchDB to validate config input, particularly when bad
values permanently take it offline. (In this case, it returns 500 for all
subsequent requests.)
IMO, a good bang-for-buck is to have three basic value types:
1. String
2. Erlang tuple
3. Integer
And each setting knows what type it must be.

[jira] [Commented] (COUCHDB-1446) Config settings input validation

2012-03-19 Thread Paul Joseph Davis (Commented) (JIRA)

[
https://issues.apache.org/jira/browse/COUCHDB-1446?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13233207#comment-13233207
]

Paul Joseph Davis commented on COUCHDB-1446:

I've seen similar reports but nothing scary enough to worry too much. Though
your comment sparks an idea:

What if all config values are specified as Erlang terms, perhaps even so much
as going to a full on Erlang term file format and then for each setting that's
available we create a function that does input validation. Also, similarly we
can store help information along with the function that does validation and
then use that in building docs (we have a similar thing in Gunicorn that seems
to work well enough).

Also, this would allow apps to register functions at boot to allow the system
to be extensible. Settings that don't have a validator are stored but not used
(or rather, if they they shouldn't cause failures, etc). Only thing I see is
what to do if you register a handler for a value that exists but is invalid.
Perhaps replace it?

Anyway, good idea, just not sure on the details (ie, tuples only? what about
tuples that contain terms? Or our handful that are lists?)

Config settings input validation

[jira] [Commented] (COUCHDB-1446) Config settings input validation

2012-03-19 Thread Paul Joseph Davis (Commented) (JIRA)

[
https://issues.apache.org/jira/browse/COUCHDB-1446?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13233218#comment-13233218
]

Paul Joseph Davis commented on COUCHDB-1446:

Also, while I do agree this is a good idea, the setting that borked is a bug
with how that value is used when it's found. Regardless of how strict you are
on type information it'll always be possible to throw a wrench in the gears
with an unexpected value in the config. But having some sort of internal
tooling to handle the parsing and presentation of these values that is separate
from the code where the value is used seems like it'd be quite beneficial from
an engineering perspective.

Config settings input validation

[jira] [Commented] (COUCHDB-1445) CouchDB deletes .view files if it can't open them, even if the error is emfile.

2012-03-18 Thread Paul Joseph Davis (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/COUCHDB-1445?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13232398#comment-13232398
 ] 

Paul Joseph Davis commented on COUCHDB-1445:


Yep. This is a pretty good explanation and solution. This behavior has been 
there forever. I always understood the intent to be what this ticket decided 
(rewriting views is fine, so just trying to blow them away and rewriting to 
solve errors is acceptable). I agree that we can switch this to a log and abort 
the open. The exact implementation of which can be in a couple places but I'm 
not overly tied to any particular one.

One caveat, I'm pretty sure there's a function that works out to be the 
equivalent of strerror. We should make use of that rather than trying to 
generate our own posix error messages.

 CouchDB deletes .view files if it can't open them, even if the error is 
 emfile.
 -

 Key: COUCHDB-1445
 URL: https://issues.apache.org/jira/browse/COUCHDB-1445
 Project: CouchDB
  Issue Type: Bug
  Components: JavaScript View Server
Affects Versions: 1.2
Reporter: Jan Lehnardt
Assignee: Randall Leeds
 Fix For: 1.2, 1.1.2


 Via Stefan Kögl on dev@:
 Another thing I noticed during my tests of CouchDB 1.2.x. I redirected
 live traffic to the instance and after a rather short time, requests
 were failing with the following information in the logs:
 [Sun, 18 Mar 2012 16:39:24 GMT] [error] [0.27554.2]
 {error_report,0.31.0,
{0.27554.2,std_error,
 [{application,mochiweb},
  Accept failed error,
  {error,emfile}]}}
 [Sun, 18 Mar 2012 16:39:24 GMT] [error] [0.27554.2]
 {error_report,0.31.0,
  {0.27554.2,crash_report,
   [[{initial_call,
 {mochiweb_acceptor,init,
 ['Argument__1','Argument__2',
  'Argument__3']}},
 {pid,0.27554.2},
 {registered_name,[]},
 {error_info,
 {exit,
 {error,accept_failed},
 [{mochiweb_acceptor,init,3},
  {proc_lib,init_p_do_apply,3}]}},
 {ancestors,
 [couch_httpd,couch_secondary_services,
  couch_server_sup,0.32.0]},
 {messages,[]},
 {links,[0.129.0]},
 {dictionary,[]},
 {trap_exit,false},
 {status,running},
 {heap_size,233},
 {stack_size,24},
 {reductions,244}],
[]]}}
 I think emfile means that CouchDB (or mochiweb?) couldn't open any
 more files / connections. I've set the (hard and soft) nofile limit for
 user couchdb to 4096, but didn't raise the ERL_MAX_PORTS accordingly.
 Anyway, as soon as the error occured, CouchDB started writing most of my
 view files from scratch, rendering the instance unusable.
 I'd expect CouchDB to fail more gracefully when the maximum number of
 open files is reached. Is this a bug or expected behaviour?

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (COUCHDB-1445) CouchDB deletes .view files if it can't open them, even if the error is emfile.

2012-03-18 Thread Paul Joseph Davis (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/COUCHDB-1445?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13232400#comment-13232400
 ] 

Paul Joseph Davis commented on COUCHDB-1445:


Wow. Took me a while to find it, but its (appropriately named) 
erl_posix_msg:message/1 which takes the reason from the posix tuple.

 CouchDB deletes .view files if it can't open them, even if the error is 
 emfile.
 -

 Key: COUCHDB-1445
 URL: https://issues.apache.org/jira/browse/COUCHDB-1445
 Project: CouchDB
  Issue Type: Bug
  Components: JavaScript View Server
Affects Versions: 1.2
Reporter: Jan Lehnardt
Assignee: Randall Leeds
 Fix For: 1.2, 1.1.2


 Via Stefan Kögl on dev@:
 Another thing I noticed during my tests of CouchDB 1.2.x. I redirected
 live traffic to the instance and after a rather short time, requests
 were failing with the following information in the logs:
 [Sun, 18 Mar 2012 16:39:24 GMT] [error] [0.27554.2]
 {error_report,0.31.0,
{0.27554.2,std_error,
 [{application,mochiweb},
  Accept failed error,
  {error,emfile}]}}
 [Sun, 18 Mar 2012 16:39:24 GMT] [error] [0.27554.2]
 {error_report,0.31.0,
  {0.27554.2,crash_report,
   [[{initial_call,
 {mochiweb_acceptor,init,
 ['Argument__1','Argument__2',
  'Argument__3']}},
 {pid,0.27554.2},
 {registered_name,[]},
 {error_info,
 {exit,
 {error,accept_failed},
 [{mochiweb_acceptor,init,3},
  {proc_lib,init_p_do_apply,3}]}},
 {ancestors,
 [couch_httpd,couch_secondary_services,
  couch_server_sup,0.32.0]},
 {messages,[]},
 {links,[0.129.0]},
 {dictionary,[]},
 {trap_exit,false},
 {status,running},
 {heap_size,233},
 {stack_size,24},
 {reductions,244}],
[]]}}
 I think emfile means that CouchDB (or mochiweb?) couldn't open any
 more files / connections. I've set the (hard and soft) nofile limit for
 user couchdb to 4096, but didn't raise the ERL_MAX_PORTS accordingly.
 Anyway, as soon as the error occured, CouchDB started writing most of my
 view files from scratch, rendering the instance unusable.
 I'd expect CouchDB to fail more gracefully when the maximum number of
 open files is reached. Is this a bug or expected behaviour?

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (COUCHDB-1445) CouchDB deletes .view files if it can't open them, even if the error is emfile.

2012-03-18 Thread Paul Joseph Davis (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/COUCHDB-1445?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13232401#comment-13232401
 ] 

Paul Joseph Davis commented on COUCHDB-1445:


And then Randall reminds me its actually file:format_error. Note to future 
selves, don't bother trying to grep for strerror or perror in Erlang sources.

 CouchDB deletes .view files if it can't open them, even if the error is 
 emfile.
 -

 Key: COUCHDB-1445
 URL: https://issues.apache.org/jira/browse/COUCHDB-1445
 Project: CouchDB
  Issue Type: Bug
  Components: JavaScript View Server
Affects Versions: 1.2
Reporter: Jan Lehnardt
Assignee: Randall Leeds
 Fix For: 1.2, 1.1.2


 Via Stefan Kögl on dev@:
 Another thing I noticed during my tests of CouchDB 1.2.x. I redirected
 live traffic to the instance and after a rather short time, requests
 were failing with the following information in the logs:
 [Sun, 18 Mar 2012 16:39:24 GMT] [error] [0.27554.2]
 {error_report,0.31.0,
{0.27554.2,std_error,
 [{application,mochiweb},
  Accept failed error,
  {error,emfile}]}}
 [Sun, 18 Mar 2012 16:39:24 GMT] [error] [0.27554.2]
 {error_report,0.31.0,
  {0.27554.2,crash_report,
   [[{initial_call,
 {mochiweb_acceptor,init,
 ['Argument__1','Argument__2',
  'Argument__3']}},
 {pid,0.27554.2},
 {registered_name,[]},
 {error_info,
 {exit,
 {error,accept_failed},
 [{mochiweb_acceptor,init,3},
  {proc_lib,init_p_do_apply,3}]}},
 {ancestors,
 [couch_httpd,couch_secondary_services,
  couch_server_sup,0.32.0]},
 {messages,[]},
 {links,[0.129.0]},
 {dictionary,[]},
 {trap_exit,false},
 {status,running},
 {heap_size,233},
 {stack_size,24},
 {reductions,244}],
[]]}}
 I think emfile means that CouchDB (or mochiweb?) couldn't open any
 more files / connections. I've set the (hard and soft) nofile limit for
 user couchdb to 4096, but didn't raise the ERL_MAX_PORTS accordingly.
 Anyway, as soon as the error occured, CouchDB started writing most of my
 view files from scratch, rendering the instance unusable.
 I'd expect CouchDB to fail more gracefully when the maximum number of
 open files is reached. Is this a bug or expected behaviour?

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (COUCHDB-1426) error while building with 2 spidermonkey installed

2012-03-16 Thread Paul Joseph Davis (Commented) (JIRA)

[
https://issues.apache.org/jira/browse/COUCHDB-1426?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13231664#comment-13231664
]

Paul Joseph Davis commented on COUCHDB-1426:

Last decision is to see if we want to check if we can detect this nspr issue
using PKG_CHECK_EXISTS(mozjs185) ! PKG_CHECK_EXISTS(nspr) and print a
warning message if so (assuming that check even works). Other than that, I
think we're good to go.

error while building with 2 spidermonkey installed
--

Key: COUCHDB-1426
URL: https://issues.apache.org/jira/browse/COUCHDB-1426
Project: CouchDB
Issue Type: Bug
Components: Build System
Affects Versions: 1.1.1, 1.2, 1.3
Reporter: Benoit Chesneau
Assignee: Benoit Chesneau
Priority: Critical
Attachments: 0001-close-COUCHDB-1426.patch,
0001-fix-build-with-custom-path-close-COUCHDB-1426.patch,
0001-fix-build-with-custom-path-close-COUCHDB-1426.patch,
0001-fix-build-with-custom-path-close-COUCHDB-1426.patch,
0001-fix-build-with-custom-path-close-COUCHDB-1426.patch,
0001-fix-build-with-custom-path-close-COUCHDB-1426.patch, COUCHDB-1426.patch

Context:
To bench the differences between different versions of couchdb I had to test
against spidermonkey 1.7 and 1.8.5 . 1.8.5 is installed globally in
/usr/local while the 1.7 version is installed on a temporary path.
Problem:
Using --witth-js-include --with-js-lib configure options aren't enough to
use the 1.7 version it still want to use spidermonkey 1.8.5 . Removing
js-config from the path doesn't change anything. I had to uninstall
spidermonkey 1.8.5 to have these setting working.
Error result:
$ ./configure
--with-erlang=/Users/benoitc/local/otp-r14b04/lib/erlang/usr/include
--with-js-include=/Users/benoitc/local/js-1.7.0/include
--with-js-lib=/Users/benoitc/local/js-1.7.0/lib64
checking for a BSD-compatible install... /usr/bin/install -c
checking whether build environment is sane... yes
checking for a thread-safe mkdir -p... build-aux/install-sh -c -d
checking for gawk... no
checking for mawk... no
checking for nawk... no
checking for awk... awk
checking whether make sets $(MAKE)... yes
checking for gcc... gcc
checking for C compiler default output file name... a.out
checking whether the C compiler works... yes
checking whether we are cross compiling... no
checking for suffix of executables...
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether gcc accepts -g... yes
checking for gcc option to accept ISO C89... none needed
checking for style of include used by make... GNU
checking dependency style of gcc... gcc3
checking build system type... i386-apple-darwin11.3.0
checking host system type... i386-apple-darwin11.3.0
checking for a sed that does not truncate output... /usr/bin/sed
checking for grep that handles long lines and -e... /usr/bin/grep
checking for egrep... /usr/bin/grep -E
checking for fgrep... /usr/bin/grep -F
checking for ld used by gcc...
/usr/llvm-gcc-4.2/libexec/gcc/i686-apple-darwin11/4.2.1/ld
checking if the linker
(/usr/llvm-gcc-4.2/libexec/gcc/i686-apple-darwin11/4.2.1/ld) is GNU ld... no
checking for BSD- or MS-compatible name lister (nm)... /usr/bin/nm
checking the name lister (/usr/bin/nm) interface... BSD nm
checking whether ln -s works... yes
checking the maximum length of command line arguments... 196608
checking whether the shell understands some XSI constructs... yes
checking whether the shell understands +=... yes
checking for /usr/llvm-gcc-4.2/libexec/gcc/i686-apple-darwin11/4.2.1/ld
option to reload object files... -r
checking how to recognize dependent libraries... pass_all
checking for ar... ar
checking for strip... strip
checking for ranlib... ranlib
checking command to parse /usr/bin/nm output from gcc object... ok
checking for dsymutil... dsymutil
checking for nmedit... nmedit
checking for lipo... lipo
checking for otool... otool
checking for otool64... no
checking for -single_module linker flag... yes
checking for -exported_symbols_list linker flag... yes
checking how to run the C preprocessor... gcc -E
checking for ANSI C header files... yes
checking for sys/types.h... yes
checking for sys/stat.h... yes
checking for stdlib.h... yes
checking for string.h... yes
checking for memory.h... yes
checking for strings.h... yes
checking for inttypes.h... yes
checking for stdint.h... yes
checking for unistd.h... yes
checking for dlfcn.h... yes
checking for objdir... .libs
checking if gcc supports -fno-rtti -fno-exceptions... no
checking for gcc option to produce PIC... -fno-common -DPIC
checking if gcc PIC flag -fno-common -DPIC works... yes
checking if gcc static flag -static works...

[jira] [Commented] (COUCHDB-1426) error while building with 2 spidermonkey installed

2012-03-14 Thread Paul Joseph Davis (Commented) (JIRA)

[
https://issues.apache.org/jira/browse/COUCHDB-1426?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13229816#comment-13229816
]

Paul Joseph Davis commented on COUCHDB-1426:

New version of the patch [1] is on the COUCHDB-1426 branch on the ASF Git repo.
Sorry about the mass emails earlier. Jason had a branch up there that was based
on a weird commit and I didn't check close enough what I was overwriting. (Note
to self, we should start a best practices using commiter names in branches).

So, Dave gave me access to a machine he had that was demonstrating the issue.
It turns out with some serendipity from Benoit's diff I figured the issue is
that the way we're running PKG_CHECK_MODULE multiple times caches some values
that short circuit its internal logic in later invocations. So, when we run it
once for the lib detection we cache some values that are reused when checking
headers. This means that the include header detection ends up thinking it had
185 installed which ends up adding nothing to the includes.

I've also used pkg-config --cflags for both mozilla-js and mozjs185 instead of
our --variable=includedir shenanigans.

Randall had one last compilation issue with 1.8.5. I'll try and reproduce here
shortly but I might have to wait for him to come back before I can top that off.

[1]
http://git-wip-us.apache.org/repos/asf/couchdb/diff/0150670afa3dda957fd4ce9ab3b58495106f62f2

error while building with 2 spidermonkey installed
--

[jira] [Commented] (COUCHDB-1426) error while building with 2 spidermonkey installed

2012-03-14 Thread Paul Joseph Davis (Commented) (JIRA)

[
https://issues.apache.org/jira/browse/COUCHDB-1426?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13229822#comment-13229822
]

Paul Joseph Davis commented on COUCHDB-1426:

No go reproducing Randall's issue. Checked the command lines for the linking to
no avail. I know I've seen his error before but I can't remember what the
screwy fix was.

His build errors found here:

http://friendpaste.com/7PHtN13NFGH7q3OT6J9Ega

error while building with 2 spidermonkey installed
--

[jira] [Commented] (COUCHDB-1426) error while building with 2 spidermonkey installed

2012-03-14 Thread Paul Joseph Davis (Commented) (JIRA)

[
https://issues.apache.org/jira/browse/COUCHDB-1426?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13229823#comment-13229823
]

Paul Joseph Davis commented on COUCHDB-1426:

Whoops. To no avail means they're different. I'm on OS X 10.7.something with
whatever that uses. Occurred to me it could be an llvm-gcc vs. gnu gcc on Linux
thing or something vs should be g++ or something.

error while building with 2 spidermonkey installed
--

[jira] [Commented] (COUCHDB-1437) Refactor configure.ac

2012-03-09 Thread Paul Joseph Davis (Commented) (JIRA)

[
https://issues.apache.org/jira/browse/COUCHDB-1437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13226737#comment-13226737
]

Paul Joseph Davis commented on COUCHDB-1437:

Some WIP here:

https://github.com/davisp/couchdb/tree/wip-new-configure

Should show enough of the basic idea to communicate what I was heading for.
Should have a mostly working version tomorrow.

Refactor configure.ac
-

Key: COUCHDB-1437
URL: https://issues.apache.org/jira/browse/COUCHDB-1437
Project: CouchDB
Issue Type: Improvement
Components: Build System
Affects Versions: 1.3
Reporter: Paul Joseph Davis

Working on COUCHDB-1426 today has made me hate life a little bit more. Our
build system shouldn't be this complex. Mostly this has to do with us not
using m4 in a way that doesn't lead to horrendous spaghetti code. New plan is
to create a number of new files like:
* m4/couch_erlang.m4
* m4/couch_icu_drv.m4
* m4/couch_js.m4
* m4/couch_snappy.m4
* m4/couch_windows.m4 (maybe? not sure how hard this will be to extract)
Given these files the plan will be to make configure.ac just run a top-level
macro from each file. Hopefully we can then have much cleaner definitions
through out the configure stuff so that when something breaks it'll be more
obvious why and where to look for an answer.

[jira] [Commented] (COUCHDB-1431) DDoc name with underscore as first char produce invalid filters

2012-03-06 Thread Paul Joseph Davis (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/COUCHDB-1431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13224028#comment-13224028
 ] 

Paul Joseph Davis commented on COUCHDB-1431:


Wonky. Although as far as I see it, the bug here is that we allow the 
underscore after _design/ not that we can't use filters from it. Anyone have 
other thoughts?

 DDoc name with underscore as first char produce invalid filters
 ---

 Key: COUCHDB-1431
 URL: https://issues.apache.org/jira/browse/COUCHDB-1431
 Project: CouchDB
  Issue Type: Bug
  Components: Database Core
Affects Versions: 1.2, 1.3
 Environment: Apache CouchDB 1.3.0a-5cece68-git
 Apache CouchDB 1.2.0a-0d8ddc8-git
Reporter: Alexander Shorin

 CouchDB allows to create design document with id as `_design/_private`, but 
 there is no way to use filters from it:
 $ curl -v http://localhost:5984/test/_changes?filter=_private/confidential
 * About to connect() to localhost port 5984 (#0)
 *   Trying 127.0.0.1...
 * connected
 * Connected to localhost (127.0.0.1) port 5984 (#0)
  GET /test/_changes?filter=_private/confidential HTTP/1.1
  User-Agent: curl/7.24.0 (i686-pc-linux-gnu) libcurl/7.24.0 GnuTLS/2.10.5 
  zlib/1.2.5
  Host: localhost:5984
  Accept: */*
  
  HTTP/1.1 400 Bad Request
  Server: CouchDB/1.3.0a-5cece68-git (Erlang OTP/R14B04)
  Date: Wed, 07 Mar 2012 04:51:06 GMT
  Content-Type: text/plain; charset=utf-8
  Content-Length: 63
  Cache-Control: must-revalidate
  
 {error:bad_request,reason:unknown builtin filter name}
 * Connection #0 to host localhost left intact
 * Closing connection #0

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (COUCHDB-1431) DDoc name with underscore as first char produce invalid filters

2012-03-06 Thread Paul Joseph Davis (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/COUCHDB-1431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13224044#comment-13224044
 ] 

Paul Joseph Davis commented on COUCHDB-1431:


Interesting. If only someone had raised concerns about having such things as 
HTTP accessible databases that mandated API compatibility with non-system 
databases.

 DDoc name with underscore as first char produce invalid filters
 ---

 Key: COUCHDB-1431
 URL: https://issues.apache.org/jira/browse/COUCHDB-1431
 Project: CouchDB
  Issue Type: Bug
  Components: Database Core
Affects Versions: 1.2, 1.3
 Environment: Apache CouchDB 1.3.0a-5cece68-git
 Apache CouchDB 1.2.0a-0d8ddc8-git
Reporter: Alexander Shorin

 CouchDB allows to create design document with id as `_design/_private`, but 
 there is no way to use filters from it:
 $ curl -v http://localhost:5984/test/_changes?filter=_private/confidential
 * About to connect() to localhost port 5984 (#0)
 *   Trying 127.0.0.1...
 * connected
 * Connected to localhost (127.0.0.1) port 5984 (#0)
  GET /test/_changes?filter=_private/confidential HTTP/1.1
  User-Agent: curl/7.24.0 (i686-pc-linux-gnu) libcurl/7.24.0 GnuTLS/2.10.5 
  zlib/1.2.5
  Host: localhost:5984
  Accept: */*
  
  HTTP/1.1 400 Bad Request
  Server: CouchDB/1.3.0a-5cece68-git (Erlang OTP/R14B04)
  Date: Wed, 07 Mar 2012 04:51:06 GMT
  Content-Type: text/plain; charset=utf-8
  Content-Length: 63
  Cache-Control: must-revalidate
  
 {error:bad_request,reason:unknown builtin filter name}
 * Connection #0 to host localhost left intact
 * Closing connection #0

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (COUCHDB-1426) error while building with 2 spidermonkey installed

2012-03-02 Thread Paul Joseph Davis (Commented) (JIRA)

[
https://issues.apache.org/jira/browse/COUCHDB-1426?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13220966#comment-13220966
]

Paul Joseph Davis commented on COUCHDB-1426:

I'm so confused. JSOPTION_ANONFUNFIX is an option to the SpiderMonkey VM that
appeared and then disappeared (along with its functionality). The test has a
bug because 1.7 doesn't have the option (though has the expected behavior).
There is a bug in that test, but we should just reorder things so that we only
test for JSOPTION_ANONFUNFIX after we test for JS_ThrowStopIteration. And I
swear that I've already fixed this at least once. Perhaps I screwed the pooch
on a backport or something.

error while building with 2 spidermonkey installed
--

Key: COUCHDB-1426
URL: https://issues.apache.org/jira/browse/COUCHDB-1426
Project: CouchDB
Issue Type: Bug
Components: Build System
Affects Versions: 1.1.1, 1.2, 1.3
Reporter: Benoit Chesneau
Priority: Critical
Attachments:
0001-fix-build-with-custom-path-close-COUCHDB-1426.patch,
0001-fix-build-with-custom-path-close-COUCHDB-1426.patch,
0001-fix-build-with-custom-path-close-COUCHDB-1426.patch,
0001-fix-build-with-custom-path-close-COUCHDB-1426.patch

[jira] [Commented] (COUCHDB-1426) error while building with 2 spidermonkey installed

2012-03-01 Thread Paul Joseph Davis (Commented) (JIRA)

[
https://issues.apache.org/jira/browse/COUCHDB-1426?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13220392#comment-13220392
]

Paul Joseph Davis commented on COUCHDB-1426:

@benoit

How about we match the rest of the file for formatting? There are zero cases of
indentation matching the argument column. Fixed width indents are used
throughout and should be used here as well.

As to the actual test in that AC_COMPILE_IFELSE I finally realized that its
relying on an error being generated due to an invalid preprocessing
instruction. It would be better to use the error preprocessor directive there.

The more I think on this the less I think its fixing either issue directly. The
globally installed SM 1.8.5 should have put its headers into $PREFIX/js/ which
was being picked up by the three lines you removed that I explicitly added in
response to a different error report. Seeing as the header checks haven't
changed I'm still not convinced that this isn't introducing a regression for
some combination of headers installed.

On the other hand, there's also this large rearrangement to fix the detection
of SM170 which didn't have the anonymous function option. It sure does seem
like there's quite a bit of change for this.

IOW, this patch should be at least two patches. And I'm not convinced we should
even attempt at supporting building against multiple installed versions of a
library. Picking up headers from system directories is going to be a bitch
regardless and trying to teach configure.ac to do a dance itself into knots
figuring this out seems like a losing proposition.

error while building with 2 spidermonkey installed
--

Key: COUCHDB-1426
URL: https://issues.apache.org/jira/browse/COUCHDB-1426
Project: CouchDB
Issue Type: Bug
Components: Build System
Affects Versions: 1.1.1, 1.2
Reporter: Benoit Chesneau
Priority: Blocker
Attachments:
0001-fix-build-with-custom-path-close-COUCHDB-1426.patch,
0001-fix-build-with-custom-path-close-COUCHDB-1426.patch,
0001-fix-build-with-custom-path-close-COUCHDB-1426.patch,
0001-fix-build-with-custom-path-close-COUCHDB-1426.patch

[jira] [Commented] (COUCHDB-1426) error while building with 2 spidermonkey installed

2012-03-01 Thread Paul Joseph Davis (Commented) (JIRA)

[
https://issues.apache.org/jira/browse/COUCHDB-1426?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13220442#comment-13220442
]

Paul Joseph Davis commented on COUCHDB-1426:

I don't see the interrest to specify a lib and include paths if they are not
handled in priority.

These are there for people to specify extra search paths when the library is
not installed in the system wide search paths.

This isn't common at all.

I assume you mean uncommon, and I'm not sure which you're referring to. The
priority of handling extra paths?

If so, then I would disagree a bit in regards to include and link search paths.
The compiler already has a default set of paths that it searches and we're
trying to work around that to a certain extent. It doesn't help that
SpiderMonkey makes this harder than it has to be, but at a certain point that's
what we get for using a language that was designed to be used in a browser.

On the other hand, there's also this large rearrangement to fix the
detection of SM170 which didn't have the anonymous function option. It sure
does seem like there's quite a bit of change for this. Can you elaborate ?

This is a fairly large change and a lot of it seems unnecessary. There are
definitely things we should include. For instance, I like that you reordered
the pkg-config check after the command line args checks. But the whole
use_js170 flag stuff seems extremely brittle and adds a fair amount of
complexity. Unless I'm mistaken we should be able to fix the bug by just
reordering the tests so we find defualt = 17.0; check for 1.8.0 - 1.8.5 -
too_new_version

The patch can't be really splitted since all this change is needed.

Sure it can and if it can't then it just means that we've done it wrong. There
are two issues here:

1. Accidentally picking up a system wide SpiderMonkey
2. Checking for JS_ANON_FUNFIX in 1.7.0 which is a bug.

Item #2 should be simple enough to fix by changing the ordering when we test
for the version detection.

Item #1 OTOH is a bit of a pain. Its definitely possible to make this work, but
I'm not entirely convinced that this patch accomplishes anything more than
allowing 1.8.5 installed globally which is different than 1.7 installed
globally.

error while building with 2 spidermonkey installed
--

[jira] [Commented] (COUCHDB-1415) Re-insering a document silently fails after compact is executed

2012-02-23 Thread Paul Joseph Davis (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/COUCHDB-1415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13214867#comment-13214867
 ] 

Paul Joseph Davis commented on COUCHDB-1415:


Should affect both as well as _bulk_docs. The actual issue is quite deep in the 
revision merging and conflict checks.

 Re-insering a document silently fails after compact is executed
 ---

 Key: COUCHDB-1415
 URL: https://issues.apache.org/jira/browse/COUCHDB-1415
 Project: CouchDB
  Issue Type: Bug
  Components: Database Core
Affects Versions: 1.1.1
 Environment: Tested on multiple linux platforms
Reporter: Viktor Szabo
 Fix For: 1.3


 When a document is re-inserted after a compact operation using the same 
 contents it was originally created, the insert operation is silently ignored, 
 leaving the client unaware of the fact it's document is not available in the 
 database.
 Can be reproduced using the following sequence of steps:
 alias curl='curl -H Content-Type: application/json'
 url=http://localhost:5984/database;
 1 curl -X PUT $url
 2 curl -X POST $url -d '{_id: bug, key: value}'
 3 curl -X DELETE $url/bug?rev=1-59414e77c768bc202142ac82c2f129de
 4 curl -X POST $url/_compact
 5 curl -X POST $url -d '{_id: bug, key: value}'
 6 curl -X GET $url/bug
   (bug here)
 1 {ok:true}
   201
 2 [{ok:true,id:bug,rev:1-59414e77c768bc202142ac82c2f129de}]
   201
 3 {ok:true,id:bug,rev:2-9b2e3bcc3752a3a952a3570b2ed4d27e}
   200
 4 {ok:true}
   202
 5 [{ok:true,id:bug,rev:1-59414e77c768bc202142ac82c2f129de}]
   201
 6 {error:not_found,reason:deleted}
   404
 CouchDB shouldn't report ok on step 5 and then go on to claim that the doc 
 is deleted. Also, it seems to work on second try:
 7 curl -X POST $url -d '{_id: bug, key: value}'
 8 curl -X GET $url/bug
 7 {ok:true,id:bug,rev:3-674f864b73df1c80925e48436e21d550}
   201
 8 {_id:bug,_rev:3-674f864b73df1c80925e48436e21d550,key:value}
   200

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (COUCHDB-1407) JSON encoding of number changes

2012-02-22 Thread Paul Joseph Davis (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/COUCHDB-1407?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13213819#comment-13213819
 ] 

Paul Joseph Davis commented on COUCHDB-1407:


I'll get to this today. It shouldn't be that hard.

 JSON encoding of number changes
 ---

 Key: COUCHDB-1407
 URL: https://issues.apache.org/jira/browse/COUCHDB-1407
 Project: CouchDB
  Issue Type: Bug
  Components: HTTP Interface
Affects Versions: 1.2
 Environment: Ubuntu 12.04 (alpha)
Reporter: Adam Lofts
 Attachments: 0001-Only-validate-numbers-on-JSON-decoding.patch


 JSON encoding of Number has changed from 1.0.2 to 1.2. JSON only defines 
 Number but this change causes issues in my app because python decodes the 
 number as an int in 1.2.
 Test case:
 PORT=5985
 curl -X DELETE http://localhost:$PORT/test-floats/
 curl -X PUT http://localhost:$PORT/test-floats/
 curl -X PUT http://localhost:$PORT/test-floats/doc1 -H Content-Type: 
 application/json -d { \a\: 1.0 }
 curl http://localhost:$PORT/test-floats/doc1
 Run against 1.0.2:
 {ok:true}
 {ok:true}
 {ok:true,id:doc1,rev:1-78e61304147429d3d500aee7806fd26d}
 {_id:doc1,_rev:1-78e61304147429d3d500aee7806fd26d,a:1.0}
 Run against 1.2:
 {ok:true}
 {ok:true}
 {ok:true,id:doc1,rev:1-78e61304147429d3d500aee7806fd26d}
 {_id:doc1,_rev:1-78e61304147429d3d500aee7806fd26d,a:1}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (COUCHDB-1410) Formally define number support

2012-02-14 Thread Paul Joseph Davis (Commented) (JIRA)

[
https://issues.apache.org/jira/browse/COUCHDB-1410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13207783#comment-13207783
]

Paul Joseph Davis commented on COUCHDB-1410:

So this whole thing has really gotten blown out of proportion. While we have
never formally documented what's going on internally, it can be described as
such:

A number is parsed into one of two forms:

If the number contains a decimal point (.) or an exponent (e or E) then
the number is internally converted into an IEEE-754 floating point
representation. This means that numbers containing either a decimal point or
exponent are subject to the constraints of having a finite number of bits
representing the number as is standard operating procedure.

If a number does not contain a decimal point or exponent then it is parsed as
an integer with (theoretically) no loss of precision (I think precision is
bound by the amount of RAM IIRC but I don't promise there aren't any bugs).
(Side note for Jiffy, technically, if a number fits in a signed 64bit
representation, that is used. If not then parsing is deferred back to Erlang
which handles parsing as a bignum).

Literally, the only thing that's wrong in COUCHDB-1407 is that number
formatting for doubles changed a wee bit and it has a simple fix and now people
are getting all crazy about numbers and ignoring other places that JSON is
munged. Blergh.

Formally define number support
--

Key: COUCHDB-1410
URL: https://issues.apache.org/jira/browse/COUCHDB-1410
Project: CouchDB
Issue Type: Improvement
Affects Versions: 1.2
Reporter: Robert Newson
Priority: Blocker
Fix For: 1.3

The JSON spec has a very loose definition of Number. CouchDB, as a database,
should have well-defined and first class support for numbers (both integral
and decimal). The precision of number support should be formally specified as
should the algorithm used to represent floating-point values, especially
where an approximation must be made in the conversion.

[jira] [Commented] (COUCHDB-1409) Look into manual GC

2012-02-14 Thread Paul Joseph Davis (Commented) (JIRA)

[
https://issues.apache.org/jira/browse/COUCHDB-1409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13207863#comment-13207863
]

Paul Joseph Davis commented on COUCHDB-1409:

Was just about to say that we have seen a couple places with similar behavior
and similar results in BigCouch. Specifically couch_os_process pids can have
this effect. The article is missing the underlying reason but the HN comment is
close enough. Simple fix is to call erlang:garbage_collect() when processes are
returned. I think there might be a few other places we do similar things but
nothing as significant as a couchjs processes.

Also, its possible that this might not be much of an issue for CouchDB.
BigCouch deployments can run in the hundreds to a thousand or so couchjs
processes so it was a bit more of an issue with those pids stealing RAM when
they went idle. If the total number is smaller and they're reused quite quickly
then this issue shouldn't manifest, or at least not as drastically.

Look into manual GC
---

Key: COUCHDB-1409
URL: https://issues.apache.org/jira/browse/COUCHDB-1409
Project: CouchDB
Issue Type: Improvement
Reporter: Jan Lehnardt

This just as a reminder to consider
http://andy.wordpress.com/2012/02/13/erlang-is-a-hoarder/ (incl. comments)

[jira] [Commented] (COUCHDB-1407) JSON encoding of number changes

2012-02-13 Thread Paul Joseph Davis (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/COUCHDB-1407?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13207022#comment-13207022
 ] 

Paul Joseph Davis commented on COUCHDB-1407:


As mentioned on the dev@ thread, I'm pretty dead set against this approach. 
While there seems to be some sort of general consensus that storing numbers as 
uninterpreted bytes and repeating them back is the way to go it really misses 
the entirety of the issue.

First, CouchDB has never claimed to pass numbers around while keeping byte 
identical representations. This patch attempts to change that drastically with 
a very large number of consequences that we haven't begun to investigate.

Secondly, if we were to actually consider going this route then we'd also be 
obliged to start looking at every other place where we change representations 
internally as well.

Thirdly, if we were to do that then we'd also have to get into all of the cases 
where we're stricter than JSON specifically allows and then try and address all 
of those issues as well.

Basically, how about we just fix the encoder to spit out a decimal point and an 
appropriate amount of precision and then start documenting our round tripping 
limitations.

 JSON encoding of number changes
 ---

 Key: COUCHDB-1407
 URL: https://issues.apache.org/jira/browse/COUCHDB-1407
 Project: CouchDB
  Issue Type: Bug
  Components: HTTP Interface
Affects Versions: 1.2
 Environment: Ubuntu 12.04 (alpha)
Reporter: Adam Lofts
 Attachments: 0001-Only-validate-numbers-on-JSON-decoding.patch


 JSON encoding of Number has changed from 1.0.2 to 1.2. JSON only defines 
 Number but this change causes issues in my app because python decodes the 
 number as an int in 1.2.
 Test case:
 PORT=5985
 curl -X DELETE http://localhost:$PORT/test-floats/
 curl -X PUT http://localhost:$PORT/test-floats/
 curl -X PUT http://localhost:$PORT/test-floats/doc1 -H Content-Type: 
 application/json -d { \a\: 1.0 }
 curl http://localhost:$PORT/test-floats/doc1
 Run against 1.0.2:
 {ok:true}
 {ok:true}
 {ok:true,id:doc1,rev:1-78e61304147429d3d500aee7806fd26d}
 {_id:doc1,_rev:1-78e61304147429d3d500aee7806fd26d,a:1.0}
 Run against 1.2:
 {ok:true}
 {ok:true}
 {ok:true,id:doc1,rev:1-78e61304147429d3d500aee7806fd26d}
 {_id:doc1,_rev:1-78e61304147429d3d500aee7806fd26d,a:1}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (COUCHDB-1397) Function expressions, evals in SpiderMonkey

2012-02-03 Thread Paul Joseph Davis (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/COUCHDB-1397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13199668#comment-13199668
 ] 

Paul Joseph Davis commented on COUCHDB-1397:


@Jason

Of course I'd look at a patch if you have one. Though as I've mentioned before, 
anything that relies on parsing JavaScript or other source-to-source transforms 
is going to be heavily discouraged. Also, while I understand that this will 
change things from the old method, I'm not overly swayed by the relaxing 
arguments here. Fact of the matter is that this is invalid JavaScript, so 
regardless of what we do, it should at least be valid JavaScript. (And jslint 
and other command line tools would finally start working as well).

@Jason, @Max, @James

I tried doing some hacky shortcuts to account for this. Specifically, ( + 
map_source + ) because the error is literally anonymous function at top 
level scope or some such and the parens make it kosher. But this ends up 
breaking all of the code that looks like var f = 1; function(doc) 
{emit(doc._id, f);}; which is a noticeable amount of JavaScript.

@Jason, @Max, @James

I did just think of an alternative solution which would be to make the new 
version use scripts instead of functions. The resulting syntax would be 
something like:

var doc = next_doc(); emit(doc._id, null)

Which would then just be executed once for every doc. Though going this route, 
we'd need some sort of context for the functions so people could precompile 
their CommonJS libs and the like.

 Function expressions, evals in SpiderMonkey
 ---

 Key: COUCHDB-1397
 URL: https://issues.apache.org/jira/browse/COUCHDB-1397
 Project: CouchDB
  Issue Type: Bug
  Components: JavaScript View Server
Affects Versions: 1.2.1
 Environment: All
Reporter: Jason Smith

 New SpiderMonkey releases do not eval() a sole anonymous function expression. 
 That is not a valid JavaScript statement, and so it is not a valid JavaScript 
 script.
 COUCHDB-1302 addressed this for 1.1 and the 1.1.x branch. This ticket is for 
 1.2. (Sorry to spam COUCHDB-1302. I saw Unassigned and read Unresolved.)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (COUCHDB-1397) Function expressions, evals in SpiderMonkey

2012-02-03 Thread Paul Joseph Davis (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/COUCHDB-1397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13199714#comment-13199714
 ] 

Paul Joseph Davis commented on COUCHDB-1397:


@Jason

I reject the premise that there's a distinction to be made between valid 
JavaScript and valid CouchDB programming. At this point I would say that if 
its not possible to have whatever we decide pass jslint in a copy/paste manner, 
then its not valid anything.

Also, the function body thing is the same thing I suggested as an aside except 
that the wrapping it in a function part is useless and will lead to more issues 
than it solves. Also, it has the same issues that if you did do it, the 
included code would be executed everytime (ie, if you pulled in a large helper 
it would redefine everything again (also note, this is not the same as 
compilation)).

 Function expressions, evals in SpiderMonkey
 ---

 Key: COUCHDB-1397
 URL: https://issues.apache.org/jira/browse/COUCHDB-1397
 Project: CouchDB
  Issue Type: Bug
  Components: JavaScript View Server
Affects Versions: 1.2.1
 Environment: All
Reporter: Jason Smith

 New SpiderMonkey releases do not eval() a sole anonymous function expression. 
 That is not a valid JavaScript statement, and so it is not a valid JavaScript 
 script.
 COUCHDB-1302 addressed this for 1.1 and the 1.1.x branch. This ticket is for 
 1.2. (Sorry to spam COUCHDB-1302. I saw Unassigned and read Unresolved.)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (COUCHDB-1399) Rename _rev to _mvcc or _lock

2012-02-03 Thread Paul Joseph Davis (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/COUCHDB-1399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13199717#comment-13199717
 ] 

Paul Joseph Davis commented on COUCHDB-1399:


@Alexander

Yes, all the other bits would need to be renamed to match the new name as well. 
I honestly don't care what it's renamed to as long as it doesn't have a 
revision connotation.

@Dirkjan

If by exceedingly limited you mean removes an entire class of misunderstanding 
in the community, then yes. :) But I do think this is something we should at 
least be considering. Also, I'm assuming that 2.0 will have other similarly 
large breaks in API so I don't see this as much of an issue since its mostly 
just a renaming of a concept (with no current plans at actually changing 
behavior).

 Rename _rev to _mvcc or _lock
 -

 Key: COUCHDB-1399
 URL: https://issues.apache.org/jira/browse/COUCHDB-1399
 Project: CouchDB
  Issue Type: Improvement
  Components: Database Core
Affects Versions: 2.0
Reporter: Paul Joseph Davis
 Fix For: 2.0


 We should rename the revisions to lock or token or some other suitably 
 opaque term so we stop getting people asking questions about treating them as 
 revisions.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (COUCHDB-1397) Function expressions, evals in SpiderMonkey

2012-02-03 Thread Paul Joseph Davis (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/COUCHDB-1397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13199766#comment-13199766
 ] 

Paul Joseph Davis commented on COUCHDB-1397:


Erm, I'd like to point out that the string function(doc) {} actually *is* 
invalid JavaScript. Anonymous functions at global scope are specifically not 
ok. Although anonymous functions in a statement context *are* ok. Its a subtle 
distinction but its quite important.

Also, just today I came across a scenario of compiling a regular expression 
outside the function scope as a valid use of the helper pattern.

 Function expressions, evals in SpiderMonkey
 ---

 Key: COUCHDB-1397
 URL: https://issues.apache.org/jira/browse/COUCHDB-1397
 Project: CouchDB
  Issue Type: Bug
  Components: JavaScript View Server
Affects Versions: 1.2.1
 Environment: All
Reporter: Jason Smith

 New SpiderMonkey releases do not eval() a sole anonymous function expression. 
 That is not a valid JavaScript statement, and so it is not a valid JavaScript 
 script.
 COUCHDB-1302 addressed this for 1.1 and the 1.1.x branch. This ticket is for 
 1.2. (Sorry to spam COUCHDB-1302. I saw Unassigned and read Unresolved.)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (COUCHDB-1397) Function expressions, evals in SpiderMonkey

2012-02-03 Thread Paul Joseph Davis (Commented) (JIRA)

[
https://issues.apache.org/jira/browse/COUCHDB-1397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13199838#comment-13199838
]

Paul Joseph Davis commented on COUCHDB-1397:

Firstly, I'm not thinking of eval, I'm thinking of
JS_CompileScript/JS_CompileUCScript which is where this restriction is actually
applied. Secondly, I didn't say transforms are out of the question, I said they
are heavily discouraged. The reason for discouraging them is because they break
shit as demonstrated by all the people that wrote in to tell me I broke shit
with the simple paren wrapping approach not to mention that this is the sort of
engineering that leads to SQL injection.

Just because it's the easy answer for some folks doesn't necessarily mean its
the right call.

Function expressions, evals in SpiderMonkey
---

Key: COUCHDB-1397
URL: https://issues.apache.org/jira/browse/COUCHDB-1397
Project: CouchDB
Issue Type: Bug
Components: JavaScript View Server
Affects Versions: 1.2.1
Environment: All
Reporter: Jason Smith

New SpiderMonkey releases do not eval() a sole anonymous function expression.
That is not a valid JavaScript statement, and so it is not a valid JavaScript
script.
COUCHDB-1302 addressed this for 1.1 and the 1.1.x branch. This ticket is for
1.2. (Sorry to spam COUCHDB-1302. I saw Unassigned and read Unresolved.)

[jira] [Commented] (COUCHDB-1397) Function expressions, evals in SpiderMonkey

2012-02-03 Thread Paul Joseph Davis (Commented) (JIRA)

[
https://issues.apache.org/jira/browse/COUCHDB-1397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13200071#comment-13200071
]

Paul Joseph Davis commented on COUCHDB-1397:

@Benoit

That's what I was trying to describe earlier. The draw backs that come to mind
is that it's quite a break with the current behavior (and harder to detect the
difference) and we'd probably want to add some sort of context member that is
compiled and executed once when the map or reduce is run (only once for maps,
and once per reduce call). Or we can just force people to abuse the commonjs
cache maybe?

Function expressions, evals in SpiderMonkey
---

[jira] [Commented] (COUCHDB-1397) Function expressions, evals in SpiderMonkey

2012-02-02 Thread Paul Joseph Davis (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/COUCHDB-1397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13198936#comment-13198936
 ] 

Paul Joseph Davis commented on COUCHDB-1397:


Describing this as exploiting a bug is incorrect. SpiderMonkey had a 
non-standard extension to JavaScript that allowed for this syntax to be 
considered valid. It was enabled by default.

Recent SpiderMonkey trunk has removed this option so it is no longer an error. 
The logic underneath is interesting in that it has to specifically check for 
this condition (ie, its completely valid except for a single is anonymous 
function as statement check.

The nicest (and probably least likely) solution would be for SpiderMonkey to 
add this option back but just disable it by default. I asked the SM team in IRC 
shortly after we found the issue and they expressed a very negative view of 
this approach so I doubt it's a possibility.

For reference, Jason's example:

{ views:{people:{map: 
  function(doc) { emit(doc.name, 1) } 
}}} 

And a fixed example that could work (with modifications to couchjs):

{ views:{people:{map: 
  function map(doc) { emit(doc.name, 1) } 
}}}

and alternatively: 

{ views:{people:{map: 
  var map = function(doc) { emit(doc.name, 1) } 
}}}

We can upgrade couchjs to accept either form quite easily. I think a reasonable 
plan would be to have a version of couchjs that accepts both and logs a 
deprecation warning about the old version. Then we just need to go on a PR 
campaign to get everyone to make the upgrade so they're prepared for when their 
code eventually needs to run on a newer SpiderMonkey.

Also, I'm not convinced that this needs to be in 1.2. We still have a bit of 
time to prepare a 1.2.1 that includes this. 1.2 should've been out a long time 
ago and I'd rather we focus on getting that out and then worrying about this 
afterwards since we have a plausible backwards compatible upgrade path.



 Function expressions, evals in SpiderMonkey
 ---

 Key: COUCHDB-1397
 URL: https://issues.apache.org/jira/browse/COUCHDB-1397
 Project: CouchDB
  Issue Type: Bug
  Components: JavaScript View Server
Affects Versions: 1.2
 Environment: All
Reporter: Jason Smith

 New SpiderMonkey releases do not eval() a sole anonymous function expression. 
 That is not a valid JavaScript statement, and so it is not a valid JavaScript 
 script.
 COUCHDB-1302 addressed this for 1.1 and the 1.1.x branch. This ticket is for 
 1.2. (Sorry to spam COUCHDB-1302. I saw Unassigned and read Unresolved.)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (COUCHDB-1279) Deleted documents persist in View results

2012-01-30 Thread Paul Joseph Davis (Commented) (JIRA)

[
https://issues.apache.org/jira/browse/COUCHDB-1279?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13196751#comment-13196751
]

Paul Joseph Davis commented on COUCHDB-1279:

@Matt,

Can you get me a copy of the db file that exhibits this or shell access to a
machine that I can build CouchDB on that has a copy of this file? Sounds
deterministic enough but the fact that so few people have seen it means its
probably quite the subtle condition.

Deleted documents persist in View results
-

Key: COUCHDB-1279
URL: https://issues.apache.org/jira/browse/COUCHDB-1279
Project: CouchDB
Issue Type: Bug
Components: Database Core
Affects Versions: 1.0.3
Environment: CouchDB 1.0.3, Centos 64 bit
Reporter: Chris Gomez
Priority: Blocker

Previously deleted documents are returned as results in a View.
View results yield { error: not_found, reason: deleted } when queried by id
and are not returned in _all_docs.
Compacting the database does not fix the issue. Deleting the View and
rebuilding did not fix the issue. Deleting the view then compacting, then
rebuilding the view did not fix the issue. Currently, the only way I know how
to get around it is to delete/rebuild the database.
These documents would have been deleted programmatically, so I can't be
certain about the exact create/update/delete scenario the have gone through.
These particular documents could have been around for a long while before I
even noticed them. The biggest issue I have with this bug is that it means I
can't bulk delete any documents when this bug occurs because of the update
conflict it creates.
May be related:
http://stackoverflow.com/questions/7323285/couchdb-views-not-being-updated-after-delete

[jira] [Commented] (COUCHDB-1396) Deleting a conflict revision will return a wrong current doc revision

2012-01-30 Thread Paul Joseph Davis (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/COUCHDB-1396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13196753#comment-13196753
 ] 

Paul Joseph Davis commented on COUCHDB-1396:


What Randall said. Kind of a bit of a WTF but I really don't know how to better 
answer this without both massive changes and rather harmful effects to 
performance. I'd almost just close as not a bug, but I'm willing to wait and 
see if someone else has a good idea here.

 Deleting a conflict revision will return a wrong current doc revision
 -

 Key: COUCHDB-1396
 URL: https://issues.apache.org/jira/browse/COUCHDB-1396
 Project: CouchDB
  Issue Type: Bug
  Components: Replication
Reporter: Oliver Kurowski
Priority: Minor
  Labels: conflict, replication, revision

 After deleting the current revision to make the conflict revision the winning 
 one, 
 the returned revisionnummer (2-f4a2a6b0c7742a5e6dd7eea7a4b625a1) is not the 
 revision number of the documen (1-45c1922c1b51dc5ee945218d63ce2ab4) 
 echo $COUCH curly -X 
 DELETE $COUCH/db1 curly -X 
 DELETE $COUCH/db2 curly -X 
 PUT $COUCH/db1 
 curly -X PUT $COUCH/db2 
 curly -X PUT $COUCH/db1/doc -d '{wert:1}' 
 curly -X PUT $COUCH/db2/doc -d '{wert:2}' 
 curly -X POST $COUCH/_replicate -d '{source:db1,target:db2}' 
 curly $COUCH/db2/doc\?conflicts=true 
 curly $COUCH/db2/doc\?revs=true 
 curly -X DELETE $COUCH/db2/doc\?rev=1-97cea70a7b8b41aae9e5732f4ac7 
 curly $COUCH/db2/doc\?conflicts=true 
 curly $COUCH/db2/doc\?revs=true curly $COUCH/db2/doc
 sendai ~ % echo $COUCH http://localhost:5984 
 sendai ~ % curly -X DELETE $COUCH/db1 
 {ok : true } 
 sendai ~ % curly -X DELETE $COUCH/db2 
 {ok : true } 
 sendai ~ % curly -X PUT $COUCH/db1 
 {ok : true } 
 sendai ~ % curly -X PUT $COUCH/db2 
 {ok : true } 
 sendai ~ % curly -X PUT $COUCH/db1/doc -d '{wert:1}' 
 {ok : true,rev : 1-97cea70a7b8b41aae9e5732f4ac7,id : 
 doc } 
 sendai ~ % curly -X PUT $COUCH/db2/doc -d '{wert:2}' 
 {ok : true,rev : 1-45c1922c1b51dc5ee945218d63ce2ab4,id : 
 doc } 
 sendai ~ % curly -X POST $COUCH/_replicate -d 
 '{source:db1,target:db2}' 
 {ok : true,history : [   {  docs_read : 1,  
 session_id : 7c74c9bb9f322e10ad072215e823c2d8,  recorded_seq : 
 1,  end_last_seq : 1,  doc_write_failures : 0,  
 start_time : Mon, 30 Jan 2012 20:34:18 GMT,  start_last_seq : 
 0,  end_time : Mon, 30 Jan 2012 20:34:18 GMT,  
 missing_checked : 0,  docs_written : 1,  missing_found 
 : 1   }],session_id : 7c74c9bb9f322e10ad072215e823c2d8,
 source_last_seq : 1,replication_id_version : 2 } 
 sendai ~ % curly $COUCH/db2/doc\?conflicts=true 
 {_id : doc,wert : 1,_conflicts : [   
 1-45c1922c1b51dc5ee945218d63ce2ab4],_rev : 
 1-97cea70a7b8b41aae9e5732f4ac7 } 
 sendai ~ % curly $COUCH/db2/doc\?revs=true 
 {_id : doc,wert : 1,_revisions : {   ids : [
   97cea70a7b8b41aae9e5732f4ac7   ],   start : 1},
 _rev : 1-97cea70a7b8b41aae9e5732f4ac7 } 
 sendai ~ % curly -X DELETE 
 $COUCH/db2/doc\?rev=1-97cea70a7b8b41aae9e5732f4ac7
 {ok : true,rev : 2-f4a2a6b0c7742a5e6dd7eea7a4b625a1,id : 
 doc } 

 
 sendai ~ % curly $COUCH/db2/doc\?conflicts=true {_id : doc,wert 
 : 2,_rev : 1-45c1922c1b51dc5ee945218d63ce2ab4 } 
 sendai ~ % curly $COUCH/db2/doc\?revs=true {_id : doc,wert : 2, 
_revisions : {   ids : [  
 45c1922c1b51dc5ee945218d63ce2ab4   ],   start : 1},_rev 
 : 1-45c1922c1b51dc5ee945218d63ce2ab4 } 
 sendai ~ % curly $COUCH/db2/doc {_id : doc,wert : 2,_rev 
 : 1-45c1922c1b51dc5ee945218d63ce2ab4 } 
 sendai ~ % 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (COUCHDB-1391) Implement _security as _local doc with revision trees

2012-01-27 Thread Paul Joseph Davis (Commented) (JIRA)

[
https://issues.apache.org/jira/browse/COUCHDB-1391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13194961#comment-13194961
]

Paul Joseph Davis commented on COUCHDB-1391:

Can you explain couch_db.erl:705? Why did the list brackets appear around the
doc? I'm guessing that's the format new_revs wants.

Lots of code I reused wants to have lists of docs with the same _id. Instead of
rewriting everything else I just used the same structure.

add a clause to get_doc_infos/2.

Do you mean get_full_doc_infos? If so, that seems like it should work quite
nicely. I'll take a crack at that this afternoon.

the line Db1#db{security_ptr=local}; is redundant

Good catch. Will fix.

Am I right that at these lines update a db as soon as it's opened?

Yep.

I was wondering why I didn't see more backwards-compatibility code anywhere.

That's mostly all there is. I realized last night that I have to wrap the local
docs btree split fun to upgrade docs on the fly, but that should be simple
enough.

couch_db_updater:init_security/2 needs to be fixed

Good catch. That should be passing Db0 to update_local_docs/2.

Any reason why copy_docs can't be changed to take a btree argument?

Yes, copy_docs is working with the by_seq btree which doesn't work for local
docs. I contemplated refactoring here, but for now I'd just erred on the side
of caution. After having spelunked through this code I've decided that a real
fix is going to require me groking couch_db.erl and couch_db_updater.erl and
doing similar level refactor to what I did for the view engine.

What's the change in couch_replicator?

This was me not understanding the replicator / thinking very hard. I'm going to
rewrite the split fun for the local docs btree to automatically upgrade old
_local docs records on the fly which should make this code unnecessary.

Thanks for the review. I'll get a revised version up tonight or tomorrow
depending on time.

Implement _security as _local doc with revision trees
-

Key: COUCHDB-1391
URL: https://issues.apache.org/jira/browse/COUCHDB-1391
Project: CouchDB
Issue Type: Improvement
Components: Database Core
Reporter: Paul Joseph Davis
Attachments: 0001-Refactor-revision-merging.patch,
0002-Replace-_local-doc-sequence-with-revision-trees.patch,
0003-Prevent-multiple-updates-to-a-single-_local-doc.patch,
0004-Implement-_security-as-a-_local-doc.patch

We had a discussion [1] a while back about updating the _security object so
that it could be replicated (internally) in a cluster or similar environment.
The basic gist was update _local docs to have a revision tree, update
_security to be a _local doc with docid _local/_security and keep the
current _security API for a version or two for backwards compatibility (or
forever, what color is your bike shed?)
So I did that.
Basic patch progression is:
1. Refactor revision merging logic so that we can split it out of
couch_db_updater's code path for updating normal docs.
2. Implement _local docs with #full_doc_info{} records (and thus revision
trees)
3. Implement _security as _local/_security
These things are done. Tests should theoretically pass after each patch but I
haven't gone back and tried. They definitely pass (minus auth_cache which I
just submitted a fix for) now except for replication.js appears to fail for
random reasons. I can't quite decide if I've introduced this or if it just
fails randomly. Rather than run it a lot more times and continue to be
confused I'm starting this ticket so I can have other people test and tell me
their results.
Also, the test suite is rather wonky on trunk with segfaults. We should
really look into that more.
Patches forth coming. I've also pushed the branch to [2].
[1]
http://grokbase.com/t/couchdb.apache.org/dev/2011/08/the-security-object-should-be-versioned/17rfmmtlu3lagqvgyq7cay26dqk4
[2]
http://git-wip-us.apache.org/repos/asf?p=couchdb.git;a=log;h=refs/heads/new-security-object

[jira] [Commented] (COUCHDB-1391) Implement _security as _local doc with revision trees

2012-01-27 Thread Paul Joseph Davis (Commented) (JIRA)

[
https://issues.apache.org/jira/browse/COUCHDB-1391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13195380#comment-13195380
]

Paul Joseph Davis commented on COUCHDB-1391:

Yeah, I looked harder at init_security and realized what I needed was to just
return the db but replace the security context like you had originally. I
cribbed part of that function from a different place and then didn't think hard
enough. Should be fixed in a commit I haven't pushed quiet yet.

Implement _security as _local doc with revision trees
-

[jira] [Commented] (COUCHDB-1395) Not well-formed JSON in changes feed filtered by view.

2012-01-26 Thread Paul Joseph Davis (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/COUCHDB-1395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13194156#comment-13194156
 ] 

Paul Joseph Davis commented on COUCHDB-1395:


Huh. We've dealt with this exact issue in BigCouch but most of our errors were 
related to errors between machines. Hadn't seen a report of it happening in 
CouchDB itself. Either way, its definitely something we should fix.

The issue as Alexander quite well diagnosed is basically that we encounter an 
error after the response code and HTTP headers have already been sent to the 
client. The current code catches an error and then doesn't act appropriately 
(it just says, Oh hai! You are a lovely exception! Off you go to the client! 
but that just rewrites a 500 response into the middle of a chunked response 
which is obviously fail.

So, the error is obvious, but the solution is not. Internally we argued quite 
extensively on the best way to do this. For BigCouch we ended up agreeing 
(after much persuasion) that the best solution is to just immediately close the 
socket (thereby creating invalid JSON (on purpose)) and log the error. The 
reasoning for this was that HTTP has no direct way of signaling an error after 
the request has started. It might be possible to send a X-CouchDB-Status 
trailer or something but that's just ridiculous cause no one would support such 
a thing and it would be quite surprising in the oddity.

The only other obvious place to put the error is into the response body itself 
similar to what Alexander suggested. While it would be theoretically beneficial 
for a client to know information about what went wrong, I argued long and hard 
that it would end up causing more problems than it solved. First, the response 
has already started successfully, so the odds that the error is something the 
client can do anything about are minimal (ie, it was probably an ephemeral 
error from something else crashing). Secondly, if its valid JSON and and the 
client lib doesn't expect an error response, at best it'll throw some sort of 
error because of the mismatched JSON (ie, we send an {error: reason} 
instead of the next changes row). But my worst fear here is that the client lib 
doesn't throw an error at all. It just says oh hay, no moar changes! and 
carries on. This I argued would be super bad in that an error occurred yet the 
user was unaware of the issue.

Thus the only way to be super certain that a user would know that an error 
occurred was to close the connection leaving invalid JSON which would trigger 
an error deep in the stack that wouldn't be accidentally ignored.

Not entirely sure what everyone else would expect but I figured I'd try and 
give as much background on the issue as possible.

Also the fix is reasonable easy either way. We pass an HTTP response around in 
a couple places. To fix we just wrapped it with a special delayed response 
structure for long requests so that the first thing that tries to write a 
response to the client flushes the status and headers. Then when an exception 
is caught we have a boolean on whether to report the error or not (ie, is this 
response object a delayed response? Yes? Send error response instead, else kill 
connection).

 Not well-formed JSON in changes feed filtered by view.
 --

 Key: COUCHDB-1395
 URL: https://issues.apache.org/jira/browse/COUCHDB-1395
 Project: CouchDB
  Issue Type: Bug
Affects Versions: 1.2, 1.3
 Environment: Apache CouchDB 1.3.0a-d59cdd7-git
 Apache CouchDB 1.2.0a-0d8ddc8-git
Reporter: Alexander Shorin
Priority: Critical
 Attachments: filtered_view_bug.sh


 CouchDB returns invalid JSON response for _changes feed if filter view have 
 failed somehow: by code mistake, bug, os timeout etc.
 Steps to reproduce:
 1. Create db and fill it with some documents.
 2. Create buggy view that would make view server crush for some document. For 
 javascript one segfault and os timeout errors are actual due to any 
 exceptions raised from map function is silenced.  You view could be fine 
 however for normal usage.
 3.  Request changes feed and apply this buggy view as filter.
 Story:
 View function had never proceed design documents and mostly they are created 
 with that knowledge. But in changes feed they have to process every document 
 in database and DDocs too, so they breaks their original logic and creates 
 unexpectable situation. Another side of problem is about custom query servers 
 that could prevent view processing on first occurred exception or due dynamic 
 code execution nature.
 Certainly,  it's all about invalid view function that generates exception for 
 some document in some case, but should _changes feed return malformed data in 
 this case or just notify about problem somehow

[jira] [Commented] (COUCHDB-1390) Fix auth_cache etap test

2012-01-25 Thread Paul Joseph Davis (Commented) (JIRA)

[
https://issues.apache.org/jira/browse/COUCHDB-1390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13193122#comment-13193122
]

Paul Joseph Davis commented on COUCHDB-1390:

@Filipe

This assertion was the failing one:

https://github.com/apache/couchdb/blob/master/test/etap/075-auth-cache.t#L182

It was failing because the _revisions section of the document had a single
revision in one and a list of revisions in the other. If you trace the code
back through the auth_cache to couch_db:open_doc_int you'll see that the result
of an open call changes depending on whether it's passed a #doc_info{} vs a
binary Id. (Ie, one keeps the entire path in _revision and one just keeps the
latest.

In the auth cache this is a difference between refreshing an entry in the cache
and loading a new entry.

As to the options to the open I just made them the same so that I didn't have
to think about possible differences between the two code paths involving those
options.

Fix auth_cache etap test

Key: COUCHDB-1390
URL: https://issues.apache.org/jira/browse/COUCHDB-1390
Project: CouchDB
Issue Type: Bug
Reporter: Paul Joseph Davis
Attachments: COUCHDB-1390.patch

The auth_cache etap tests were failing for me. Debugged this to make sure it
wasn't related to something else. Commit message is:
Fix for the auth_cache etap

As it turns out, opening a doc by id is different than opening it using
a #doc_info record due to the inclusion of the full revision path. This
ended up breaking the auth_cache tests. This way includes the entire
revision path for all docs and not just first doc loads.
Patching attaching in a few moments.

[jira] [Commented] (COUCHDB-1389) Improve JS CLI test error reports

2012-01-25 Thread Paul Joseph Davis (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/COUCHDB-1389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13193123#comment-13193123
 ] 

Paul Joseph Davis commented on COUCHDB-1389:


Mine changes the .toString() to a .toSource() which is important for errors to 
be printed as something other than [object Object]. Yours includes the stack 
which is probably helpful as well. We should definitely combine the two.

Also, on yours we need to make sure we follow TAP in case we ever pass this 
output through prove. Basically just requires prefixing the first non test 
report line with # and then making sure the rest are all indented.

 Improve JS CLI test error reports
 -

 Key: COUCHDB-1389
 URL: https://issues.apache.org/jira/browse/COUCHDB-1389
 Project: CouchDB
  Issue Type: Improvement
Reporter: Paul Joseph Davis
 Attachments: COUCHDB-1389.patch


 About to attach a simple patch that gives better error output when JS tests 
 fail.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (COUCHDB-1390) Fix auth_cache etap test

2012-01-25 Thread Paul Joseph Davis (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/COUCHDB-1390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13193151#comment-13193151
 ] 

Paul Joseph Davis commented on COUCHDB-1390:


Hrm. That's odd as it was deterministically failing for me. I'll try and 
reproduce this outside of the patches for COUCHDB-1391 which is where I 
discovered it. I thought I'd tracked it down to something specific but perhaps 
my updates changed something subtle. Will know more in a few minutes.

 Fix auth_cache etap test
 

 Key: COUCHDB-1390
 URL: https://issues.apache.org/jira/browse/COUCHDB-1390
 Project: CouchDB
  Issue Type: Bug
Reporter: Paul Joseph Davis
 Attachments: COUCHDB-1390.patch


 The auth_cache etap tests were failing for me. Debugged this to make sure it 
 wasn't related to something else. Commit message is:
 Fix for the auth_cache etap
 
 As it turns out, opening a doc by id is different than opening it using
 a #doc_info record due to the inclusion of the full revision path. This
 ended up breaking the auth_cache tests. This way includes the entire
 revision path for all docs and not just first doc loads.
 Patching attaching in a few moments.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (COUCHDB-1393) badmatch on big binary

2012-01-25 Thread Paul Joseph Davis (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/COUCHDB-1393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13193181#comment-13193181
 ] 

Paul Joseph Davis commented on COUCHDB-1393:


I'd be quite interested to know what sort of fsync options were being used on 
the database/server in question where this occurred.

 badmatch on big binary
 --

 Key: COUCHDB-1393
 URL: https://issues.apache.org/jira/browse/COUCHDB-1393
 Project: CouchDB
  Issue Type: Bug
Reporter: Jan Lehnardt

 via dev@ by Peta Bogdan bog...@gmail.com
 Hello,
 I have a small database around 120 MB with approximately 16,000 documents.
 However, it happens (also from futon) that I get this error:
 [Tue, 17 Jan 2012 07:22:01 GMT] [error] [0.185.0] {error_report,0.30.0,
 {0.185.0,crash_report,
  [[{initial_call,{couch_file,init,['Argument__1']}},
{pid,0.185.0},
{registered_name,[]},
{error_info,
 {exit,
  {{badmatch,
{ok,
 9_MEGABYTES_BINARY}},
   [{couch_file,read_raw_iolist_int,3},
{couch_file,maybe_read_more_iolist,4},
{couch_file,handle_call,3},
{gen_server,handle_msg,5},
{proc_lib,init_p_do_apply,3}]},
  [{gen_server,terminate,6},
   {proc_lib,init_p_do_apply,3}]}},
{ancestors,[0.184.0]},
{messages,
 [{'$gen_call',
   {0.10840.18,#Ref0.0.3.20907},
   bytes}]},
{links,[0.190.0]},
{dictionary,[]},
{trap_exit,true},
{status,running},
{heap_size,1597},
{stack_size,24},
{reductions,65666}],
   [{neighbour,
 [{pid,0.190.0},
  {registered_name,[]},
  {initial_call,
   {couch_ref_counter,init,['Argument__1']}},
  {current_function,{gen_server,loop,6}},
  {ancestors,[0.188.0,0.187.0,0.184.0]},
  {messages,[]},
  {links,[0.185.0]},
  {dictionary,[]},
  {trap_exit,false},
  {status,waiting},
  {heap_size,610},
  {stack_size,9},
  {reductions,362}]}]]}}
 If this error occurs to frequently causes couch_server to reach its max
 restart frequency causing the entire supervision tree to shutdown and hence
 the database server instance disappears.
 The function couch_file:read_raw_iolist_int/3 calls file:pread which
 returns {ok, Binary}. This Binary has almost 9 megabytes in size, which is
 very strange.
 I think this does mean that the function file:pread/3 is instructed to read
 from wrong position.
 The only reason I can think of is that the value of 'TotalBytes' from line
 (1) doesn't match the value of 'TotalBytes' from line (2)
 (1) TotalBytes = calculate_total_read_len(BlockOffset, Len),
 (2) {ok, RawBin:TotalBytes/binary} = file:pread(Fd, Pos, TotalBytes),
 The possible answer would be that in certain conditions the function
 calculate_total_read_len/2 doesn't return the expected value.
 Server: CouchDB/1.1.1 (Erlang OTP/R14B04)
 OS: OpenBSD 5.0 GENERIC.MP#63 amd64
 Now, the trouble is how to circumvent this situation.
 Thank you in advance,
 Bogdan

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (COUCHDB-1391) Implement _security as _local doc with revision trees

2012-01-24 Thread Paul Joseph Davis (Commented) (JIRA)

[
https://issues.apache.org/jira/browse/COUCHDB-1391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13192913#comment-13192913
]

Paul Joseph Davis commented on COUCHDB-1391:

Just realized the subject of the third patch is silly. Updated on the Git
branch. Too lazy to re upload here for a commit message change.

Implement _security as _local doc with revision trees
-

[jira] [Commented] (COUCHDB-890) Option to use a persistent CommonJS module cache

2012-01-23 Thread Paul Joseph Davis (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/COUCHDB-890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13190952#comment-13190952
 ] 

Paul Joseph Davis commented on COUCHDB-890:
---

While true, the awesomely scary aspect is code loaded into the SM VM when code 
is run by different users/dbs.

 Option to use a persistent CommonJS module cache
 

 Key: COUCHDB-890
 URL: https://issues.apache.org/jira/browse/COUCHDB-890
 Project: CouchDB
  Issue Type: Improvement
  Components: JavaScript View Server
 Environment: All
Reporter: mikeal
 Attachments: commonjscache.diff


 Since COUCHDB-1075, there is a CommonJS module cache used for resolving 
 circular CommonJS dependencies. However, Caolan reports a 10x speed 
 improvement by not clearing this cache between requests. An option to not 
 clear the cache could be a great tool for performance-interested power users 
 who know their CommonJS modules are sane. The improvement will be even 
 greater when we turn on the JIT compiler in SpiderMonkey since cached modules 
 will benefit from being pre-JIT'd.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (COUCHDB-1371) configure erroneously warns against using a new spidermonkey with old spidermonkeys

2011-12-27 Thread Paul Joseph Davis (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/COUCHDB-1371?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13176267#comment-13176267
 ] 

Paul Joseph Davis commented on COUCHDB-1371:


While I understand your general frustration, deleting code just so we don't 
have to test a couple versions isn't a convincing argument to me. Though it 
does suck having to go back and test older spidermonkeys. I would be up for 
not officially supported, use at your own risk type of support, though that 
just makes me worry that we'll piss off anyone that happens to have an old 
SpiderMonkey.

Bottom line, we chose a funky ass dependency years ago and the chickens have 
come home to roost. If we want to maintain any semblance of It just works 
then we'll need to keep maintaining this dependency for quite some time.

 configure erroneously warns against using a new spidermonkey with old 
 spidermonkeys
 ---

 Key: COUCHDB-1371
 URL: https://issues.apache.org/jira/browse/COUCHDB-1371
 Project: CouchDB
  Issue Type: Bug
Reporter: Randall Leeds
Assignee: Randall Leeds
Priority: Minor
 Fix For: 1.2, 1.3, 1.1.2

 Attachments: 0001-fix-bad-configure-warning-on-old-SpiderMonkey.patch


 Paul added the check for JSOPTION_ANONFUNFIX in 7ce9e103e, but js-1.7 doesn't 
 have this constant so configure gives a warning.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (COUCHDB-1371) configure erroneously warns against using a new spidermonkey with old spidermonkeys

2011-12-27 Thread Paul Joseph Davis (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/COUCHDB-1371?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13176306#comment-13176306
 ] 

Paul Joseph Davis commented on COUCHDB-1371:


+1

 configure erroneously warns against using a new spidermonkey with old 
 spidermonkeys
 ---

 Key: COUCHDB-1371
 URL: https://issues.apache.org/jira/browse/COUCHDB-1371
 Project: CouchDB
  Issue Type: Bug
Reporter: Randall Leeds
Assignee: Randall Leeds
Priority: Minor
 Fix For: 1.2, 1.3, 1.1.2

 Attachments: 0001-fix-bad-configure-warning-on-old-SpiderMonkey.patch


 Paul added the check for JSOPTION_ANONFUNFIX in 7ce9e103e, but js-1.7 doesn't 
 have this constant so configure gives a warning.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (COUCHDB-1371) configure erroneously warns against using a new spidermonkey with old spidermonkeys

2011-12-25 Thread Paul Joseph Davis (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/COUCHDB-1371?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13175812#comment-13175812
 ] 

Paul Joseph Davis commented on COUCHDB-1371:


We have the code. I'd prefer to fix the INSTALL files and make it a recommends 
instead of requires.

 configure erroneously warns against using a new spidermonkey with old 
 spidermonkeys
 ---

 Key: COUCHDB-1371
 URL: https://issues.apache.org/jira/browse/COUCHDB-1371
 Project: CouchDB
  Issue Type: Bug
Reporter: Randall Leeds
Assignee: Randall Leeds
Priority: Minor
 Fix For: 1.2, 1.3, 1.1.2

 Attachments: 0001-fix-bad-configure-warning-on-old-SpiderMonkey.patch


 Paul added the check for JSOPTION_ANONFUNFIX in 7ce9e103e, but js-1.7 doesn't 
 have this constant so configure gives a warning.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (COUCHDB-1367) When settings revs_limit on db - the db increases its update_seq counter when viewing stats - but not when getting changes

2011-12-25 Thread Paul Joseph Davis (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/COUCHDB-1367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13175813#comment-13175813
 ] 

Paul Joseph Davis commented on COUCHDB-1367:


@Randall

If there's no technical reason why we need to bump the seq for changes that 
don't modify a document we could just stop doing that. What operations do this 
currently? How many of these resources have we at one time or another 
discussed making into a full document?

I think _purge, _revs_limit, and _security. Maybe other things. Not sure about 
the strictly confirming to theoretical model for _revs_limit and _security, but 
_purge has obvious semantics that need to be updated in views. Though as I 
think about this, I'm fairly certain we have a bug in the indexer that no one 
has ever reported. Granted, that seems to be fairly true of things that touch 
_purge code.

 When settings revs_limit on db - the db increases its update_seq counter when 
 viewing stats - but not when getting changes
 --

 Key: COUCHDB-1367
 URL: https://issues.apache.org/jira/browse/COUCHDB-1367
 Project: CouchDB
  Issue Type: Bug
  Components: HTTP Interface
Affects Versions: 1.1.1
 Environment: Any
Reporter: Henrik Hofmeister
Priority: Minor
  Labels: revs_limit

 If you put a number to _revs_limit on a db (to update it) - the 
 http://host/dbname/ info document gets an increase in update_seq number - 
 however the changes feed does not contain this change (while its not a 
 change). This causes the update_seq in the dbinfo doc and the last seq in the 
 changes feed to differ - which breaks any application depending on the 
 update_seq number as the expected sequence size of the db (in my case - 
 couchdb-lucene that will only respond to stale requests because it thinks its 
 not up to date)
 I know this is an edge case - but still its something fairly fundamental - 
 that clearly is not working as intended. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (COUCHDB-1367) When settings revs_limit on db - the db increases its update_seq counter when viewing stats - but not when getting changes

2011-12-19 Thread Paul Joseph Davis (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/COUCHDB-1367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13172432#comment-13172432
 ] 

Paul Joseph Davis commented on COUCHDB-1367:


Quite right. update_seq will be incremented for a number of API's that make 
changes while not updating documents. There's even an increment_update_seq API 
that just bumps it directly.

Having a last_update in the db info blob that lists the last update for 
managing changes feeds might be useful in general and seems like it'd be an 
easy enough addition.

 When settings revs_limit on db - the db increases its update_seq counter when 
 viewing stats - but not when getting changes
 --

 Key: COUCHDB-1367
 URL: https://issues.apache.org/jira/browse/COUCHDB-1367
 Project: CouchDB
  Issue Type: Bug
  Components: HTTP Interface
Affects Versions: 1.1.1
 Environment: Any
Reporter: Henrik Hofmeister
Assignee: Bob Dionne
Priority: Minor
  Labels: revs_limit

 If you put a number to _revs_limit on a db (to update it) - the 
 http://host/dbname/ info document gets an increase in update_seq number - 
 however the changes feed does not contain this change (while its not a 
 change). This causes the update_seq in the dbinfo doc and the last seq in the 
 changes feed to differ - which breaks any application depending on the 
 update_seq number as the expected sequence size of the db (in my case - 
 couchdb-lucene that will only respond to stale requests because it thinks its 
 not up to date)
 I know this is an edge case - but still its something fairly fundamental - 
 that clearly is not working as intended. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (COUCHDB-1365) Fix merging of document with attachment stubs

2011-12-16 Thread Paul Joseph Davis (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/COUCHDB-1365?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13171022#comment-13171022
 ] 

Paul Joseph Davis commented on COUCHDB-1365:


I don't get it. Does this patch fix replication or make it fail with an error?

 Fix merging of document with attachment stubs
 -

 Key: COUCHDB-1365
 URL: https://issues.apache.org/jira/browse/COUCHDB-1365
 Project: CouchDB
  Issue Type: Bug
  Components: Database Core, Replication
Reporter: Filipe Manana
Priority: Blocker
 Fix For: 1.2, 1.1.2

 Attachments: 
 0001-Fix-merging-of-documents-with-attachment-stubs.patch, 
 reproduce-CBMA-22.sh


 This issue was found by Marty Schoch and is reproducible the following 
 attached script.
 The commit message in the patch explains the issue:
 During replicated updates, merging of documents with
 attachment stubs will fail if, after merging the received
 document's revision tree with the current on disk revision
 tree, produces a revision tree which doesn't contain the revision
 that immediately precedes the received document's revision.
 This happens when the received document doesn't contain in its
 revision history any of the revisions in the revision tree
 of the currently on disk document. This is possible when the
 document had a number of updates higher than the value of revs
 limit defined for the source database.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (COUCHDB-1365) Fix merging of document with attachment stubs

2011-12-16 Thread Paul Joseph Davis (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/COUCHDB-1365?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13171039#comment-13171039
 ] 

Paul Joseph Davis commented on COUCHDB-1365:


Ah. Sounds good to me then.

 Fix merging of document with attachment stubs
 -

 Key: COUCHDB-1365
 URL: https://issues.apache.org/jira/browse/COUCHDB-1365
 Project: CouchDB
  Issue Type: Bug
  Components: Database Core, Replication
Reporter: Filipe Manana
Priority: Blocker
 Fix For: 1.2, 1.1.2

 Attachments: 
 0001-Fix-merging-of-documents-with-attachment-stubs.patch, 
 reproduce-CBMA-22.sh


 This issue was found by Marty Schoch and is reproducible the following 
 attached script.
 The commit message in the patch explains the issue:
 During replicated updates, merging of documents with
 attachment stubs will fail if, after merging the received
 document's revision tree with the current on disk revision
 tree, produces a revision tree which doesn't contain the revision
 that immediately precedes the received document's revision.
 This happens when the received document doesn't contain in its
 revision history any of the revisions in the revision tree
 of the currently on disk document. This is possible when the
 document had a number of updates higher than the value of revs
 limit defined for the source database.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (COUCHDB-1366) update of the Getting started with PL/SQL page

2011-12-16 Thread Paul Joseph Davis (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/COUCHDB-1366?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13171062#comment-13171062
 ] 

Paul Joseph Davis commented on COUCHDB-1366:


What's your wikipedia user name? I'll add you to the editors list.

 update of the Getting started with PL/SQL page
 --

 Key: COUCHDB-1366
 URL: https://issues.apache.org/jira/browse/COUCHDB-1366
 Project: CouchDB
  Issue Type: Task
  Components: Documentation
 Environment: couchdb wiki 
Reporter: zekeriya koç
Priority: Minor
  Labels: documentaion, getting_started, wiki

 my link to the couchdb client library for Oracle PL/SQL has changed to: 
 https://github.com/zekzekus/couch_orcl 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (COUCHDB-1366) update of the Getting started with PL/SQL page

2011-12-16 Thread Paul Joseph Davis (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/COUCHDB-1366?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13171065#comment-13171065
 ] 

Paul Joseph Davis commented on COUCHDB-1366:


And by wikipedia I meant wiki.apache.org. I am apparently still not awake.

 update of the Getting started with PL/SQL page
 --

 Key: COUCHDB-1366
 URL: https://issues.apache.org/jira/browse/COUCHDB-1366
 Project: CouchDB
  Issue Type: Task
  Components: Documentation
 Environment: couchdb wiki 
Reporter: zekeriya koç
Priority: Minor
  Labels: documentaion, getting_started, wiki

 my link to the couchdb client library for Oracle PL/SQL has changed to: 
 https://github.com/zekzekus/couch_orcl 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (COUCHDB-1355) split code create couch_httpd application

2011-12-13 Thread Paul Joseph Davis (Commented) (JIRA)

[
https://issues.apache.org/jira/browse/COUCHDB-1355?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13169037#comment-13169037
]

Paul Joseph Davis commented on COUCHDB-1355:

Commencing brain dump review:

0001

* src/Makefile.am indentation looks wrong for src/couch_httpd

* include files is empty, but this might change in the next patch so just a
note to self

* We do a thing for couch.app.src in couchdb so we can set the module list
automatically. Should we duplicate that here?

* couch_httpd.app is missing the module to start the app

0002

* Yep, record moved. Looks good.

* Whitespace only hunk at the bottom of 0002

0003

* I'm not entirely certain about this patch. What's the circular dependency?
If it's just records in include files I don't think I'd bother making the split
as it seems a bit awkward.

* The couch_changes.app reminds me we should be listing dependencies better
as we do this. Not super urgent but a thing to think about.

This also brings up how we want to handle things like the httpd code for views.
If anyone has good ideas we should open a ticket and discuss a plan of some
sort on how to handle this going forward. I'm thinking specifically that the
couch_httpd is separate from couch, but couch_changes and couch_mrview have
their httpd code in those apps.

split code create couch_httpd application
---

Key: COUCHDB-1355
URL: https://issues.apache.org/jira/browse/COUCHDB-1355
Project: CouchDB
Issue Type: Improvement
Components: Database Core, HTTP Interface
Affects Versions: 1.3
Reporter: Benoit Chesneau
Assignee: Benoit Chesneau
Fix For: 1.3

Attachments: 0001-couch_httpd-application.patch,
0002-move-httpd-record-in-couch_httpd-include-couch_httpd.patch,
0003-create-couch_changes-application.-Fix-tests-and-allo.patch

Couchdb is still to monolithic. This tcicket track changes to extract CouchDB
HTTP API as a full application.

[jira] [Commented] (COUCHDB-1166) See if the U+2028 character breaks JSONP

2011-12-12 Thread Paul Joseph Davis (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/COUCHDB-1166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13168047#comment-13168047
 ] 

Paul Joseph Davis commented on COUCHDB-1166:


Not paying much attention but I think that ejson has an issue here. we should 
think of a JS or Etap test for this to be safe I think.

 See if the U+2028 character breaks JSONP
 

 Key: COUCHDB-1166
 URL: https://issues.apache.org/jira/browse/COUCHDB-1166
 Project: CouchDB
  Issue Type: Improvement
Reporter: Jan Lehnardt
Assignee: Paul Joseph Davis
Priority: Minor

 We should see if we need to fix our JSON responses over JSONP according to 
 http://timelessrepo.com/json-isnt-a-javascript-subset

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (COUCHDB-282) Abort views that have exceptions in the view server

2011-12-12 Thread Paul Joseph Davis (Commented) (JIRA)

[
https://issues.apache.org/jira/browse/COUCHDB-282?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13168048#comment-13168048
]

Paul Joseph Davis commented on COUCHDB-282:
---

Good call on the distinction between compile time and runtime syntax errors.

Also, am I the only person that just got slammed in the forehead with awesome
ideas on eval'ing javascript on documents in the view engine? I mean, that's
kinda dirty, but sounds like something that could be useful scenarios.

Abort views that have exceptions in the view server
---

Key: COUCHDB-282
URL: https://issues.apache.org/jira/browse/COUCHDB-282
Project: CouchDB
Issue Type: Bug
Components: JavaScript View Server
Affects Versions: 0.8, 0.8.1
Environment: All
Reporter: kowsik

I'm incrementally experimenting with new views on a 100K document db. If
there's a typo in the javascript view or some other exception, there's a spew
of logs in couch.log while each of the couch documents continue to go through
the map/reduce code (which is buggy). This takes about 15 minutes before I
can update the view and re-query again. It would be awesome if couch can
handle exceptions in the view server and stop processing so that the fix can
be made.

[jira] [Commented] (COUCHDB-1362) couchjs doesn't clean out global namespace between views calls

2011-12-12 Thread Paul Joseph Davis (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/COUCHDB-1362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13168065#comment-13168065
 ] 

Paul Joseph Davis commented on COUCHDB-1362:


Except sealing has issues in some versions. And I'm not sure if that even gets 
us out of crazy town. We already use sub-contexts that are sealed if possible 
and I'm not entirely certain on the interplay there. I've also heard from other 
SpiderMonkey embeders that do things like scan k/v pairs on global, run js, 
delete unknown keys and reset values which sounds like a best effort but not 
full proof way of dealing with this.

Basically, there's a lot to this and don't do that has served us well enough 
for long enough that unless someone wants to take on the task of making these 
things work for our supported versions of SpiderMonkey I'd be a bit hesitant on 
allocating cycles to it right now.

 couchjs doesn't clean out global namespace between views calls
 --

 Key: COUCHDB-1362
 URL: https://issues.apache.org/jira/browse/COUCHDB-1362
 Project: CouchDB
  Issue Type: Bug
Affects Versions: 1.1.1
 Environment: Linux
Reporter: jay groven
Priority: Minor

 Sometimes a view can access the functions that were defined in a previous 
 view.  This can lead a developer to think that function sharing has been 
 implemented, but it's not reliable behaviour.  It would be better if the 
 global namespace were cleared between map/reduce runs so that it's clear that 
 there is no reliable way to share functions between views.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (COUCHDB-1362) couchjs doesn't clean out global namespace between views calls

2011-12-11 Thread Paul Joseph Davis (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/COUCHDB-1362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13167235#comment-13167235
 ] 

Paul Joseph Davis commented on COUCHDB-1362:


What version of SpiderMonkey are you using?

In general I agree but I've never seen an efficient method not to mention a way 
to support it across all the SpiderMonkey versions we use.

 couchjs doesn't clean out global namespace between views calls
 --

 Key: COUCHDB-1362
 URL: https://issues.apache.org/jira/browse/COUCHDB-1362
 Project: CouchDB
  Issue Type: Bug
Affects Versions: 1.1.1
 Environment: Linux
Reporter: jay groven
Priority: Minor

 Sometimes a view can access the functions that were defined in a previous 
 view.  This can lead a developer to think that function sharing has been 
 implemented, but it's not reliable behaviour.  It would be better if the 
 global namespace were cleared between map/reduce runs so that it's clear that 
 there is no reliable way to share functions between views.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (COUCHDB-1355) split code create couch_httpd application

2011-12-09 Thread Paul Joseph Davis (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/COUCHDB-1355?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13166221#comment-13166221
 ] 

Paul Joseph Davis commented on COUCHDB-1355:


I'll try and get a review of this done today. I skimmed and didn't see anything 
major but I want to go over some of the build stuff and startup in a bit more 
detail before I can comment more thoroughly.

 split code  create couch_httpd application
 ---

 Key: COUCHDB-1355
 URL: https://issues.apache.org/jira/browse/COUCHDB-1355
 Project: CouchDB
  Issue Type: Improvement
  Components: Database Core, HTTP Interface
Affects Versions: 1.3
Reporter: Benoit Chesneau
Assignee: Benoit Chesneau
 Fix For: 1.3

 Attachments: 0001-couch_httpd-application.patch, 
 0002-move-httpd-record-in-couch_httpd-include-couch_httpd.patch, 
 0003-create-couch_changes-application.-Fix-tests-and-allo.patch


 Couchdb is still to monolithic. This tcicket track changes to extract CouchDB 
 HTTP API as a full application.  

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (COUCHDB-1358) Only delete files with .view extension

2011-12-08 Thread Paul Joseph Davis (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/COUCHDB-1358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13165234#comment-13165234
 ] 

Paul Joseph Davis commented on COUCHDB-1358:


I'm not super against it by any means and was mostly thinking of the new 
indexers. If no one else objects I'll commit this.

 Only delete files with .view extension
 --

 Key: COUCHDB-1358
 URL: https://issues.apache.org/jira/browse/COUCHDB-1358
 Project: CouchDB
  Issue Type: Improvement
  Components: Database Core
Affects Versions: 1.1.1
Reporter: Volker Mische
Priority: Minor
 Fix For: 1.1.2

 Attachments: cleanup-views-only.patch


 Currently all files in the view directory that are not used by any View group 
 are deleted by the _view_cleanup endpoint. If other indexers (like GeoCouch) 
 store indexes in the same directory, they get deleted as well.
 The attached patch changes the bahaviour and only deletes files ending with 
 .view. Leftovers from compaction are deletes as well as they end with 
 .compact.view.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (COUCHDB-1358) Only delete files with .view extension

2011-12-07 Thread Paul Joseph Davis (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/COUCHDB-1358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13164630#comment-13164630
 ] 

Paul Joseph Davis commented on COUCHDB-1358:


I haven't thought about it too hard, but I've been running under the assumption 
that indexers would just store their files in their own directories and be 
responsible for their own cleanup. Is there a good reason to munge these things 
together?

 Only delete files with .view extension
 --

 Key: COUCHDB-1358
 URL: https://issues.apache.org/jira/browse/COUCHDB-1358
 Project: CouchDB
  Issue Type: Improvement
  Components: Database Core
Affects Versions: 1.1.1
Reporter: Volker Mische
Priority: Minor
 Fix For: 1.1.2

 Attachments: cleanup-views-only.patch


 Currently all files in the view directory that are not used by any View group 
 are deleted by the _view_cleanup endpoint. If other indexers (like GeoCouch) 
 store indexes in the same directory, they get deleted as well.
 The attached patch changes the bahaviour and only deletes files ending with 
 .view. Leftovers from compaction are deletes as well as they end with 
 .compact.view.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (COUCHDB-326) Occasional {error:error,reason:eacces} errors deleting a database on Windows

2011-11-22 Thread Paul Joseph Davis (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/COUCHDB-326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13155606#comment-13155606
 ] 

Paul Joseph Davis commented on COUCHDB-326:
---

@Dave

Why would an audio/visual program be interfering with CouchDB's data 
directories?

 Occasional {error:error,reason:eacces} errors deleting a database on 
 Windows
 

 Key: COUCHDB-326
 URL: https://issues.apache.org/jira/browse/COUCHDB-326
 Project: CouchDB
  Issue Type: Bug
Affects Versions: 0.9
 Environment: Windows, couch 0.9, erlang R12B 5.6.5
Reporter: Mark Hammond

 On Windows, occasionally you will see errors attempting to delete a database. 
   This manifests itself by 10-30% of the test suite failing on Windows.  If 
 you retry the tests that failed, they will usually pass on subsequent 
 attempts.  Running the tests individually causes them to fail roughly 10% of 
 the time.
  The log output shown is:
   [debug] [0.18650.6] httpd 500 error response:
{error:error,reason:eacces}
   [info] [0.18650.6] 127.0.0.1 - - 'DELETE' /test_suite_db/ 500
 A slightly snipped transcript from IRC:
 (2:58:32 PM) markh: I see a number of INFO logs Shutting down view group 
 server, monitored db is closing. directly before the error.  I was guessing 
 the file may be unlink'd before one of those workers actually closes its 
 handle?
 (2:58:54 PM) alisdair: yeah, it's probably a race condition
 (2:59:13 PM) alisdair: where the delete is tried before the fd is let go
 (2:59:26 PM) alisdair: the reader fd that is
 (2:59:32 PM) markh: yeah
 ...
 (3:11:47 PM) alisdair: i can't find an obvious deadlock
 (3:12:18 PM) alisdair: couch_server:delete explicitly waits for the db 
 process to exit
 (3:12:23 PM) alisdair: before deleting it
 (3:15:15 PM) alisdair: i think i found the problem
 (3:15:23 PM) alisdair: but i need a windows machine to confirm
 (3:15:30 PM) alisdair: i'll look into it tomorrow

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (COUCHDB-1342) Asynchronous file writes

2011-11-17 Thread Paul Joseph Davis (Commented) (JIRA)

[
https://issues.apache.org/jira/browse/COUCHDB-1342?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13152507#comment-13152507
]

Paul Joseph Davis commented on COUCHDB-1342:

@Damien

However, there is another optimization coming where a raw erlang FD is used
in a calling process to avoid messaging overhead (another big performance
improvement in certain long operations), which will maybe make it necessary
again.

I'm not sure what you mean here. Something along the lines of a file:open call
in the couch_db_updater process (and couch_mrview_updater)? If so that's an
interesting idea. Seems like we could make couch_file handle that quite easily
along the lines of how file handles #prim_file vs #file (if I recall those
record names correctly). This could also solve some of the fd duplication if we
only need an extra fd for views that are updating.

The concern with doubling the # non-db file descriptors is a real one. How
big of a concern of this?

The thing is, I'm not certain how it'll behave. Hence why it concerns me. Is it
a matter of just making sure that ulimit is set sufficiently high? How high is
sufficient? If I'm running in production, and I upgrade to a version of CouchDB
that has this patch, can I at least guestimate how configs might need to
change? Maybe I'm being overly paranoid and its not an issue. I dunno. Hence
why it concerns me.

The 5th concern would definitely make code more complicated for callers

I agree. I should've prefaced that bit with a I wonder if in the future
there's a follow up direction we can go. It only occurred as I was finishing
that comment so I figured I'd write it down.

Your 3rd and 4th concerns aren't Apache user concerns, but can be easily
addressed after check-in.

I have no idea what you mean by Apache user concerns here. If you're
referring to no one cares how the sausage is made so long as its faster then
I'm going to have to disagree. Strongly. Saying that databases are complicated
so we shouldn't concern ourselves with code quality is just going to leave us
with a source tree in an even worse state than it already is.

And I'd like to address this argument about progress and the desires of users.
This patch was submitted to JIRA yesterday. My initial review was up within
3.5h. This patch changes how the file abstraction works. In a database. As far
as I'm concerned development on this started yesterday at 13:28 when Jan
uploaded the patch to JIRA. If you wanted things to be moving more quickly at
this stage you should have been developing this on a branch in git and asking
for input from the community.

Secondly, while I understand that you're highly motivated to help users by
improving performance, what does that have to do with the conversation about
the technical merits of this patch? This sense of urgency that progress must be
made so lets address the issues I brought up after its in trunk is not a
convincing argument. You could address my comments by spending thirty minutes
in an editor and resubmitting the patch. Instead you're asking me to clean this
up for you after its committed.

Thirdly, every time someone asks, Can it wait till it's on trunk?, all I hear
is, Can I ignore what you just said and commit this anyway? If I point at
something and say that its broken its because I'm expecting the patch to change
or an explanation of why I'm wrong. And I'm fine being wrong. It happens quite
often. But this pattern of submitting patches and asking for all concerns to be
addressed after the patch is in trunk is starting to get a bit annoying. If we
want to adjust our policies around CTR vs RTC for larger patches, that's fine.
Perhaps adding an edge branch in git that will accept all our bigger somewhat
scary commits would be beneficial. If we start doing automated package building
then users could even pull bleeding edge code to test. But I digress.

Asynchronous file writes

Key: COUCHDB-1342
URL: https://issues.apache.org/jira/browse/COUCHDB-1342
Project: CouchDB
Issue Type: Improvement
Components: Database Core
Reporter: Jan Lehnardt
Fix For: 1.3

Attachments: COUCHDB-1342.patch

[jira] [Commented] (COUCHDB-1342) Asynchronous file writes

2011-11-17 Thread Paul Joseph Davis (Commented) (JIRA)

[
https://issues.apache.org/jira/browse/COUCHDB-1342?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13152557#comment-13152557
]

Paul Joseph Davis commented on COUCHDB-1342:

Damn it JIRA just ate a comment.

Short and skinny of it was that I just remembered that there's an (unmeasured)
optimization in zip_server that I learned from gen_server2. The basic idea is
to pull all messages out of the mailbox and then process them off a queue. The
original idea was that it'd prevent pattern matching from running away scanning
for messages though I think writer_loop's repeated implementation. Might be
worth a check but I only note this cause the thought occurred. Feel free to
ignore.

Also, just to be clear I'm not suggesting the use of zip_server or anything
nutty like that. It just occurred to me that it might help inform the current
discussion.

Asynchronous file writes

Key: COUCHDB-1342
URL: https://issues.apache.org/jira/browse/COUCHDB-1342
Project: CouchDB
Issue Type: Improvement
Components: Database Core
Reporter: Jan Lehnardt
Fix For: 1.3

Attachments: COUCHDB-1342.patch

This change updates the file module so that it can do
asynchronous writes. Basically it replies immediately
to process asking to write something to the file, with
the position where the chunks will be written to the
file, while a dedicated child process keeps collecting
chunks and write them to the file (and batching them
when possible). After issuing a series of write request
to the file module, the caller can call its 'flush'
function which will block the caller until all the
chunks it requested to write are effectively written
to the file.
This maximizes the IO subsystem, as for example, while
the updater is traversing and modifying the btrees and
doing CPU bound tasks, the writes are happening in
parallel.
Originally described at http://s.apache.org/TVu
Github Commit:
https://github.com/fdmanana/couchdb/commit/e82a673f119b82dddf674ac2e6233cd78c123554

[jira] [Commented] (COUCHDB-1342) Asynchronous file writes

2011-11-17 Thread Paul Joseph Davis (Commented) (JIRA)

[
https://issues.apache.org/jira/browse/COUCHDB-1342?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13152685#comment-13152685
]

Paul Joseph Davis commented on COUCHDB-1342:

@Damien

For #3 the issue is that it's broken. Granted there's lots in CouchDB that
breaks Erlang idioms, but this is basic broken window theory. We *know* this is
broken and its easily fixable. Why would we check in something that we know is
broken?

For #4 the patch is reimplementing parts of the standard library. In Erlang
there's a Right Way™ to start a process synchronously that might fail to
initialize. It reduces the code that we have to write and maintain and its
considerably more tested than the custom implementation we have here. Trying to
minimize this by calling it a style issue is not conducive to the technical
discourse.

You're also forgetting #2 which you agreed can be done. And if done it not only
reduces the size of this patch in terms of total lines, but reduces the number
of files that are touched, as well as removes the need for people to scour the
source for places that we might need to add calls to couch_file:flush/1 and
also prevents the need for devs to remember to add it in the future. And it
also makes the flushing more optimal in terms of decreased message passing
latency.

I'm sorry, but the culture issue here is that you continue to be subtly
derogatory by deriding technical comments as a pet concerns. This is
especially disheartening when I've spent so much time and effort on trying to
help by reviewing this patch and responding to this conversation. The sad part
is that most of the things I've voiced an opinion on are easy to fix or
abstract enough that a brief discussion on the issue with a list of things to
watch out for would have been enough to assuage my concerns.

This patch is less than 48 hours old and it changes a core aspect of how we
write data to disk in a database that prides itself on not losing data. What
are you honestly expecting for a time table here? It hasn't been two days and
you've already referenced speed of development and project progress multiple
times in response to technical points.

You ask for my help yet you remain dismissive of anything I'm offering. I'm
sorry you're frustrated but this is how code review works. I just recently
finished maintaing a 7K line refactor for over a month while people reviewed it
and even made changes to code that I was refactoring which I had to then merge
(behavior, not code) into my branch. I understand how painful review can be,
but that's how this system works and I, for one, thinks it makes us stronger.

Asynchronous file writes

Key: COUCHDB-1342
URL: https://issues.apache.org/jira/browse/COUCHDB-1342
Project: CouchDB
Issue Type: Improvement
Components: Database Core
Reporter: Jan Lehnardt
Fix For: 1.3

Attachments: COUCHDB-1342.patch

[jira] [Commented] (COUCHDB-1342) Asynchronous file writes

2011-11-16 Thread Paul Joseph Davis (Commented) (JIRA)

[
https://issues.apache.org/jira/browse/COUCHDB-1342?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13151314#comment-13151314
]

Paul Joseph Davis commented on COUCHDB-1342:

This patch looks like its a mishmash of a couple patches that are applied
slightly differently. Not sure if that's a branch thing or what. But for
instance, there's a lot of id_btree(Db) - Db#db.by_docid_btree type changes.
There's also a switch between couch_file:sync(Filepath) and
couch_file:sync(Fd). I'm not sure if that's on purpose or not because we
switched to Filepath on purpose. I see it as being possible but I don't see a
reference to it.

I'm really not a fan of exposing the inner workings of couch_file to every
client as can be seen by the massive number of flush calls that this creates. I
find it quite likely that we'll eventually end up missing one of these flush
calls and causing a nasty bug because the error would look like data hadn't
been written. I'd want to see things rewritten so that the usage of couch_file
doesn't change. Easiest solution I see would be to automatically flush buffers
if a read request comes in for something that's not on disk.

I'm pretty sure we talked about this at one point, but can someone describe the
differences between this and Erlang's delayed_write mode for files?

couch_file becomes much more complicated with this patch. I'd prefer to see it
simplified if at all possible. I'm not entirely certain how it'd look but I
might start by making a couch_file_reader and couch_file_writer gen_servers
instead of having bare processes in couch_file.

I have to say that this patch scares me a bit. If we make the switch to
something like this then we're opening up a whole new type of failure scenario
where file writes appear to succeed but then end up failing after the caller
has moved on. While there are certainly cases where I can see this being a fine
tradeoff (view updaters, compactors, etc) it worries me a bit for the main
database file. The fact is that I can't (yet) reasonably consider all of the
possible new failure modes and convince myself that things are correct. I've
already seen some super weird reactions to things like running out of disk
space, it seems like not knowing about such an error until you've written a
megabyte of data seems awkward.

Asynchronous file writes

Key: COUCHDB-1342
URL: https://issues.apache.org/jira/browse/COUCHDB-1342
Project: CouchDB
Issue Type: Improvement
Components: Database Core
Reporter: Jan Lehnardt
Fix For: 1.3

Attachments: COUCHDB-1342.patch

[jira] [Commented] (COUCHDB-1323) couch_replicator

2011-11-16 Thread Paul Joseph Davis (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/COUCHDB-1323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13151389#comment-13151389
 ] 

Paul Joseph Davis commented on COUCHDB-1323:


Looks like I should've clicked through to the file view. Apparently GitHub has 
decided to start representing tabs as two space indents in diffs. +1 from me 
then.

 couch_replicator
 

 Key: COUCHDB-1323
 URL: https://issues.apache.org/jira/browse/COUCHDB-1323
 Project: CouchDB
  Issue Type: Improvement
  Components: Replication
Affects Versions: 2.0, 1.3
Reporter: Benoit Chesneau
  Labels: replication
 Attachments: 
 0001-move-couchdb-replication-to-a-standalone-application.patch, 
 0001-move-couchdb-replication-to-a-standalone-application.patch, 
 0001-move-couchdb-replication-to-a-standalone-application.patch, 
 0001-move-couchdb-replication-to-a-standalone-application.patch


 patch to move couch replication modules in in a standalone couch_replicator 
 application. It's also available on my github:
 https://github.com/benoitc/couchdb/compare/master...couch_replicator

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (COUCHDB-1342) Asynchronous file writes

2011-11-16 Thread Paul Joseph Davis (Commented) (JIRA)

[
https://issues.apache.org/jira/browse/COUCHDB-1342?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13151430#comment-13151430
]

Paul Joseph Davis commented on COUCHDB-1342:

These are deliberate. Currently we are maintaining an Fd/couch_file pair for
reads and writes in a bit of a clumsy way in the #btree.fd record and the
couch_file module and do a switcheroo for writes vs. reads. Now all of that
is moved into couch_file and couch_db doesn't have to concern itself with the
details of efficient file access.

Oh I hadn't noticed that weirdness before, but I see it now.

Good point, API-wise this is a bit leaky and we should definitely look into
making this better, but I don't think this blocks the bulk of the
improvements that this patch introduces. I'm happy to open a new ticket about
this. Good starting points are calling flush() inside couch_file 1) after
writing a header and 2) if a read fails with {error, eof} (fwiw, Damien tried
the latter before but removed it again, the details elude me).

I disagree fairly strongly on this point. couch_file is about as core of an API
as it gets. couch_file:flush/1 shouldn't even exist as far as I'm concerned.
This isn't a mediocre API that should be improved later, its a bad API that
shouldn't go in at all. Its 100% foot-gun.

The difference is that delayed_write does buffering whereas we just want to
keep writing to the file all the time. delayed_write would not write until
the buffer is full or the timeout is hit. In addition, we wouldn't be able to
tell for delayed_commits=false writes when data hit the file reliably.

Oh right, I remember that discussion about clearing the buffer being an issue
now. Good call.

The whole point of this patch is to make couch_file smarter and move the
reader/writer abstraction further down the stack and reap the associated
performance benefits. Unless we have an alternate patch, we can't really
compare this.

Code Review 2.0:

Q: This looks complicated, can we think about trying to simplify some of the
organization?
A: No. Write it yourself if you care that much.

I don't need another patch to know that this one is complicated and could be
less complicated.

This doesn't change the error scenarios. We already rely on monitoring to
tell the request process a couch_file write didn't work

You're pointing at code that's about three abstractions away from couch_file's
writer loop. Suffice it to say, erlang:monitor/2 on a random process in the
write path does little to assuage my fears that we're entering dangerous
territory relying on our own file writing buffers.

Asynchronous file writes

Key: COUCHDB-1342
URL: https://issues.apache.org/jira/browse/COUCHDB-1342
Project: CouchDB
Issue Type: Improvement
Components: Database Core
Reporter: Jan Lehnardt
Fix For: 1.3

Attachments: COUCHDB-1342.patch

[jira] [Commented] (COUCHDB-1342) Asynchronous file writes

2011-11-16 Thread Paul Joseph Davis (Commented) (JIRA)

[
https://issues.apache.org/jira/browse/COUCHDB-1342?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13151723#comment-13151723
]

Paul Joseph Davis commented on COUCHDB-1342:

@Damien

That's an awful lot of disappointment packed into a single comment. First,
resorting to an ad hominem attack to insinuate that I'm not intelligent enough
to work on databases is quite disconcerting. Secondly, its an egregious fallacy
to suggest that because a patch appears to be technically correct that it
should be committed. Thirdly, declaring what is and isn't a valid reason to
hold up a patch is not how the ASF works.

And now back to the regularly scheduled technical discussion.

First, couch_file:flush/1. Unless I'm missing something extremely subtle here,
it's existence is so that clients can read their own writes. Yet the couch_file
gen_server has all the knowledge it needs to know if it has to flush to service
a write call. If the requested read position is between #file.eof and #file.eof
+ #file.queued_write_bytes, then it can call flush and move on with its life.
Not only does this mean that clients don't have remember to call flush, but it
removes unnecessary message passing that every unconditional call to flush
would generate.

Second, this is doubling the number of file descriptors required for anything
that isn't a database. On the first production machine I checked that's an
increase of 75% from 40K to 70K file descriptors. That's a fairly serious
change that ought to be discussed. At the very least it ought to be mentioned
somewhere so ops teams know to expect it.

Third, this is spawning long lived processes that aren't looping on exported
functions. After two code upgrades this would crash every couch_file in the VM
simultaneously.

Fourth, as I've mentioned numerous times before, the proper way to
synchronously start a process that might fail to initialize is to use
proc_lib:start_link and proc_lib:init_ack.

Fifth, has anyone considered using a write buffer outside of the couch_file API
that would allow clients more precise control. For instance, thinking briefly
on the view updater, you could buffer writes for a single add_remove call. This
also leads to the possibility that mostly read views aren't needlessly holding
open a writer fd for no reason.

Asynchronous file writes

Key: COUCHDB-1342
URL: https://issues.apache.org/jira/browse/COUCHDB-1342
Project: CouchDB
Issue Type: Improvement
Components: Database Core
Reporter: Jan Lehnardt
Fix For: 1.3

Attachments: COUCHDB-1342.patch

[jira] [Commented] (COUCHDB-1342) Asynchronous file writes

2011-11-16 Thread Paul Joseph Davis (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/COUCHDB-1342?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13151734#comment-13151734
 ] 

Paul Joseph Davis commented on COUCHDB-1342:


Also, I'd point out that I've not so much as implied that this patch shouldn't 
go in. The patch came to JIRA, I reviewed it and I've been trying to discuss 
the technical aspects of this patch. Somehow that got dragged off into the 
weeds about voting and other silly things. I'd prefer if we could avoid 
spending time squabbling like raccoons in a dumpster. There's a good idea here 
and we should be spending our time working on the engineering to make it happen.

 Asynchronous file writes
 

 Key: COUCHDB-1342
 URL: https://issues.apache.org/jira/browse/COUCHDB-1342
 Project: CouchDB
  Issue Type: Improvement
  Components: Database Core
Reporter: Jan Lehnardt
 Fix For: 1.3

 Attachments: COUCHDB-1342.patch


 This change updates the file module so that it can do
 asynchronous writes. Basically it replies immediately
 to process asking to write something to the file, with
 the position where the chunks will be written to the
 file, while a dedicated child process keeps collecting
 chunks and write them to the file (and batching them
 when possible). After issuing a series of write request
 to the file module, the caller can call its 'flush'
 function which will block the caller until all the
 chunks it requested to write are effectively written
 to the file.
 This maximizes the IO subsystem, as for example, while
 the updater is traversing and modifying the btrees and
 doing CPU bound tasks, the writes are happening in
 parallel.
 Originally described at http://s.apache.org/TVu
 Github Commit: 
 https://github.com/fdmanana/couchdb/commit/e82a673f119b82dddf674ac2e6233cd78c123554

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (COUCHDB-1334) Indexer speedup (for non-native view servers)

2011-11-14 Thread Paul Joseph Davis (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/COUCHDB-1334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13149699#comment-13149699
 ] 

Paul Joseph Davis commented on COUCHDB-1334:


+1

 Indexer speedup (for non-native view servers)
 -

 Key: COUCHDB-1334
 URL: https://issues.apache.org/jira/browse/COUCHDB-1334
 Project: CouchDB
  Issue Type: Improvement
  Components: Database Core, JavaScript View Server, View Server 
 Support
Reporter: Filipe Manana
Assignee: Filipe Manana
 Fix For: 1.2

 Attachments: 0001-More-efficient-view-updater-writes.patch, 
 0002-More-efficient-communication-with-the-view-server.patch, 
 master-0002-More-efficient-communication-with-the-view-server.patch, 
 master-2-0002-More-efficient-communication-with-the-view-server.patch, 
 master-3-0002-More-efficient-communication-with-the-view-server.patch, 
 master-4-0002-More-efficient-communication-with-the-view-server.patch


 The following 2 patches significantly improve view index generation/update 
 time and reduce CPU consumption.
 The first patch makes the view updater's batching more efficient, by ensuring 
 each btree bulk insertion adds/removes a minimum of N (=100) key/value 
 pairts. This also makes the index file size grow not so fast with old data 
 (old btree nodes basically). This behaviour is already done in master/trunk 
 in the new indexer (by Paul Davis).
 The second patch maximizes the throughput with an external view server (such 
 as couchjs). Basically it makes the pipe (erlang port) communication between 
 the Erlang VM (couch_os_process basically) and the view server more efficient 
 since the 2 sides spend less time block on reading from the pipe.
 Here follow some benchmarks.
 test database at  http://fdmanana.iriscouch.com/test_db  (1 million documents)
 branch 1.2.x
 $ echo 3  /proc/sys/vm/drop_caches
 $ time curl http://localhost:5984/test_db/_design/test/_view/test1
 {rows:[
 {key:null,value:100}
 ]}
 real  2m45.097s
 user  0m0.006s
 sys   0m0.007s
 view file size: 333Mb
 CPU usage:
 $ sar 1 60
 22:27:20  %usr  %nice   %sys   %idle
 22:27:21   38  0 12 50
 ()
 22:28:21   39  0 13 49
 Average: 39  0 13 47   
 branch 1.2.x + batch patch (first patch)
 $ echo 3  /proc/sys/vm/drop_caches
 $ time curl http://localhost:5984/test_db/_design/test/_view/test1
 {rows:[
 {key:null,value:100}
 ]}
 real  2m12.736s
 user  0m0.006s
 sys   0m0.005s
 view file size 72Mb
 branch 1.2.x + batch patch + os_process patch
 $ echo 3  /proc/sys/vm/drop_caches
 $ time curl http://localhost:5984/test_db/_design/test/_view/test1
 {rows:[
 {key:null,value:100}
 ]}
 real  1m9.330s
 user  0m0.006s
 sys   0m0.004s
 view file size:  72Mb
 CPU usage:
 $ sar 1 60
 22:22:55  %usr  %nice   %sys   %idle
 22:23:53   22  0  6 72
 ()
 22:23:55   22  0  6 72
 Average: 22  0  7 70   
 master/trunk
 $ echo 3  /proc/sys/vm/drop_caches
 $ time curl http://localhost:5984/test_db/_design/test/_view/test1
 {rows:[
 {key:null,value:100}
 ]}
 real  1m57.296s
 user  0m0.006s
 sys   0m0.005s
 master/trunk + os_process patch
 $ echo 3  /proc/sys/vm/drop_caches
 $ time curl http://localhost:5984/test_db/_design/test/_view/test1
 {rows:[
 {key:null,value:100}
 ]}
 real  0m53.768s
 user  0m0.006s
 sys   0m0.006s

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (COUCHDB-1309) File descriptor leaks on design document update and view cleanup

2011-11-13 Thread Paul Joseph Davis (Commented) (JIRA)

[
https://issues.apache.org/jira/browse/COUCHDB-1309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13149361#comment-13149361
]

Paul Joseph Davis commented on COUCHDB-1309:

So reviewing this again, my last comment still stands. There's no reason to
introduce the ddoc_changed function into the indexer API. These changes should
be limited entirely to couch_index (minus the tests anyway). This is a simple
fix. Just use Mod:get(signature, Mod:init(Db, DDoc)) (or similar) in the
with_db call in couch_index:handle_cast/2 call for ddoc_updated.

The other thing that occurred to me over lunch is if we might want to
reimplement this by just having couch_index close itself after a period of
inactivity. At the moment there's no limit on the number of groups that are
opened like we have for databases. Some sort of mechanism to close these things
down after lack of use seems like a good idea. The patch to do this would be
pretty trivial. All you have to do is add a Timeout to each reply/noreply
response in the couch_index gen_server, and then have a handle_info(timeout)
clause that returns {stop, normal, State}.

Other than that, I've heard other people wanting the update notifications for
design docs so that could go in regardless. Another thing that we might want to
think about is best practices for commits and whether commits should span
applications. I don't have super strong feelings either way though I have heard
the argument that not spanning applications decreases the chance that code
starts getting super entangled.

Good work, Filipe

File descriptor leaks on design document update and view cleanup

Key: COUCHDB-1309
URL: https://issues.apache.org/jira/browse/COUCHDB-1309
Project: CouchDB
Issue Type: Bug
Reporter: Filipe Manana
Assignee: Filipe Manana
Attachments: 12x-0001-Add-ddoc_updated-event.patch,
12x-0002-Shutdown-view-group-on-ddoc-update.patch, couchdb-1309_12x.patch,
couchdb-1309_trunk.patch, master-0001-Add-ddoc_updated-event.patch,
master-0002-Shutdown-view-group-on-ddoc-update.patch

If we add a design document with views defined in it, open the corresponding
view group (by querying one of its views for e.g.), then update the design
document in such a way that the view signature changes (changing a view's map
function code for e.g), the old view group remains open forever (unless a
server restart happens) and keeps it's view file reference counter active
forever.
If a view cleanup request comes, the old view file is not deleted since the
reference counter is not dropped by the old view group, keeping the file
descriptor in use forever.
This leakage is different from what is reported in COUCHDB-1129 but it's
somehow related.
The attached patch, simply shutdowns a view group when the design document is
updated and the new view signature changes, releasing the old view file
descriptor (as soon as no more clients are using the old view).

[jira] [Commented] (COUCHDB-1334) Indexer speedup (for non-native view servers)

2011-11-09 Thread Paul Joseph Davis (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/COUCHDB-1334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13147375#comment-13147375
 ] 

Paul Joseph Davis commented on COUCHDB-1334:


@Filipe, Awesome, this is considerably better than the old version.

As to this part:

+% Can throw badarg error, when OsProc Pid is dead.
+(catch port_connect(OsProc#os_proc.port, Pid))

That looks like the key to what I hadn't managed to track down when I tried 
something similar. I'm pretty sure we should be fine with couchspawnkillable, 
but do we need to close the port here and/or ignore some port exit status 
messages in this process? The other thing that is a bit confusing is why 
OsProc's Pid would be dead while its sitting idle. I haven't thought through 
all the implications here I guess.

Does this version maintain the same speedups as before? When I tried this 
approach I was actually doing it slightly differently by having the doc reader 
process send docs to the port which would then forward them directly to the 
writer process. There was some stuff that got a bit funky when I tried this 
though. IIRC it was something like, I had to pass deleted doc update_seq's 
directly to the writer process or it'd break  if the last update was a deletion 
(cause the writer would never see the update seq). Anyway, just a thought.

Good work on this.

 Indexer speedup (for non-native view servers)
 -

 Key: COUCHDB-1334
 URL: https://issues.apache.org/jira/browse/COUCHDB-1334
 Project: CouchDB
  Issue Type: Improvement
  Components: Database Core, JavaScript View Server, View Server 
 Support
Reporter: Filipe Manana
Assignee: Filipe Manana
 Fix For: 1.2

 Attachments: 0001-More-efficient-view-updater-writes.patch, 
 0002-More-efficient-communication-with-the-view-server.patch, 
 master-0002-More-efficient-communication-with-the-view-server.patch, 
 master-2-0002-More-efficient-communication-with-the-view-server.patch


 The following 2 patches significantly improve view index generation/update 
 time and reduce CPU consumption.
 The first patch makes the view updater's batching more efficient, by ensuring 
 each btree bulk insertion adds/removes a minimum of N (=100) key/value 
 pairts. This also makes the index file size grow not so fast with old data 
 (old btree nodes basically). This behaviour is already done in master/trunk 
 in the new indexer (by Paul Davis).
 The second patch maximizes the throughput with an external view server (such 
 as couchjs). Basically it makes the pipe (erlang port) communication between 
 the Erlang VM (couch_os_process basically) and the view server more efficient 
 since the 2 sides spend less time block on reading from the pipe.
 Here follow some benchmarks.
 test database at  http://fdmanana.iriscouch.com/test_db  (1 million documents)
 branch 1.2.x
 $ echo 3  /proc/sys/vm/drop_caches
 $ time curl http://localhost:5984/test_db/_design/test/_view/test1
 {rows:[
 {key:null,value:100}
 ]}
 real  2m45.097s
 user  0m0.006s
 sys   0m0.007s
 view file size: 333Mb
 CPU usage:
 $ sar 1 60
 22:27:20  %usr  %nice   %sys   %idle
 22:27:21   38  0 12 50
 ()
 22:28:21   39  0 13 49
 Average: 39  0 13 47   
 branch 1.2.x + batch patch (first patch)
 $ echo 3  /proc/sys/vm/drop_caches
 $ time curl http://localhost:5984/test_db/_design/test/_view/test1
 {rows:[
 {key:null,value:100}
 ]}
 real  2m12.736s
 user  0m0.006s
 sys   0m0.005s
 view file size 72Mb
 branch 1.2.x + batch patch + os_process patch
 $ echo 3  /proc/sys/vm/drop_caches
 $ time curl http://localhost:5984/test_db/_design/test/_view/test1
 {rows:[
 {key:null,value:100}
 ]}
 real  1m9.330s
 user  0m0.006s
 sys   0m0.004s
 view file size:  72Mb
 CPU usage:
 $ sar 1 60
 22:22:55  %usr  %nice   %sys   %idle
 22:23:53   22  0  6 72
 ()
 22:23:55   22  0  6 72
 Average: 22  0  7 70   
 master/trunk
 $ echo 3  /proc/sys/vm/drop_caches
 $ time curl http://localhost:5984/test_db/_design/test/_view/test1
 {rows:[
 {key:null,value:100}
 ]}
 real  1m57.296s
 user  0m0.006s
 sys   0m0.005s
 master/trunk + os_process patch
 $ echo 3  /proc/sys/vm/drop_caches
 $ time curl http://localhost:5984/test_db/_design/test/_view/test1
 {rows:[
 {key:null,value:100}
 ]}
 real  0m53.768s
 user  0m0.006s
 sys   0m0.006s

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (COUCHDB-1334) Indexer speedup (for non-native view servers)

2011-11-08 Thread Paul Joseph Davis (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/COUCHDB-1334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13146441#comment-13146441
 ] 

Paul Joseph Davis commented on COUCHDB-1334:


@Randall, now that 1.8.5 is out, we should be able to lean on that for quite a 
long time. If people want to package versions of SpiderMonkey trunk in their 
distro, I'm disinclined to put too much immediate effort into supporting those 
random versions.

@Filipe, reading the patch I think the idea is pretty good in general but I'd 
implement it a bit differently. Firstly, the logic for whether or not 
pipelining is used shouldn't be exposed to the client. That's just going to 
entangle a whole bunch of API knowledge in the wrong place. I've been meaning 
to go back and finish the refactoring of couch_(os|native)_process and 
couch_query_servers which would make this behavior possible.

The other part of this that might be interesting is the erlang:port_connect/2 
call that can set the destination Pid for that port. I played with it a bit 
during my refactoring work but couldn't get it to work quite right. I didn't 
spend too much time figuring it out, but it might be a way to skip the 
intermediary process and extra message passing.

http://erlang.org/doc/man/erlang.html#port_connect-2

 Indexer speedup (for non-native view servers)
 -

 Key: COUCHDB-1334
 URL: https://issues.apache.org/jira/browse/COUCHDB-1334
 Project: CouchDB
  Issue Type: Improvement
  Components: Database Core, JavaScript View Server, View Server 
 Support
Reporter: Filipe Manana
Assignee: Filipe Manana
 Fix For: 1.2

 Attachments: 0001-More-efficient-view-updater-writes.patch, 
 0002-More-efficient-communication-with-the-view-server.patch, 
 master-0002-More-efficient-communication-with-the-view-server.patch


 The following 2 patches significantly improve view index generation/update 
 time and reduce CPU consumption.
 The first patch makes the view updater's batching more efficient, by ensuring 
 each btree bulk insertion adds/removes a minimum of N (=100) key/value 
 pairts. This also makes the index file size grow not so fast with old data 
 (old btree nodes basically). This behaviour is already done in master/trunk 
 in the new indexer (by Paul Davis).
 The second patch maximizes the throughput with an external view server (such 
 as couchjs). Basically it makes the pipe (erlang port) communication between 
 the Erlang VM (couch_os_process basically) and the view server more efficient 
 since the 2 sides spend less time block on reading from the pipe.
 Here follow some benchmarks.
 test database at  http://fdmanana.iriscouch.com/test_db  (1 million documents)
 branch 1.2.x
 $ echo 3  /proc/sys/vm/drop_caches
 $ time curl http://localhost:5984/test_db/_design/test/_view/test1
 {rows:[
 {key:null,value:100}
 ]}
 real  2m45.097s
 user  0m0.006s
 sys   0m0.007s
 view file size: 333Mb
 CPU usage:
 $ sar 1 60
 22:27:20  %usr  %nice   %sys   %idle
 22:27:21   38  0 12 50
 ()
 22:28:21   39  0 13 49
 Average: 39  0 13 47   
 branch 1.2.x + batch patch (first patch)
 $ echo 3  /proc/sys/vm/drop_caches
 $ time curl http://localhost:5984/test_db/_design/test/_view/test1
 {rows:[
 {key:null,value:100}
 ]}
 real  2m12.736s
 user  0m0.006s
 sys   0m0.005s
 view file size 72Mb
 branch 1.2.x + batch patch + os_process patch
 $ echo 3  /proc/sys/vm/drop_caches
 $ time curl http://localhost:5984/test_db/_design/test/_view/test1
 {rows:[
 {key:null,value:100}
 ]}
 real  1m9.330s
 user  0m0.006s
 sys   0m0.004s
 view file size:  72Mb
 CPU usage:
 $ sar 1 60
 22:22:55  %usr  %nice   %sys   %idle
 22:23:53   22  0  6 72
 ()
 22:23:55   22  0  6 72
 Average: 22  0  7 70   
 master/trunk
 $ echo 3  /proc/sys/vm/drop_caches
 $ time curl http://localhost:5984/test_db/_design/test/_view/test1
 {rows:[
 {key:null,value:100}
 ]}
 real  1m57.296s
 user  0m0.006s
 sys   0m0.005s
 master/trunk + os_process patch
 $ echo 3  /proc/sys/vm/drop_caches
 $ time curl http://localhost:5984/test_db/_design/test/_view/test1
 {rows:[
 {key:null,value:100}
 ]}
 real  0m53.768s
 user  0m0.006s
 sys   0m0.006s

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (COUCHDB-1337) Use MD5 for attachment ETag

2011-11-07 Thread Paul Joseph Davis (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/COUCHDB-1337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13145981#comment-13145981
 ] 

Paul Joseph Davis commented on COUCHDB-1337:


+1

Also, I just realized why you hopped onto this ticket so fast. Had I looked at 
the number harder I would've been fighting for it.

 Use MD5 for attachment ETag
 ---

 Key: COUCHDB-1337
 URL: https://issues.apache.org/jira/browse/COUCHDB-1337
 Project: CouchDB
  Issue Type: Improvement
Affects Versions: 1.1.1
Reporter: Jacob Wright
Assignee: Robert Newson
Priority: Minor
 Attachments: 
 0001-CouchDB-1337-Use-attachments-md5-as-ETag-if-availabl.patch


 Currently attachments use the revision number of the document they're 
 attached to for their ETag. This can be inefficient if many attachments are 
 added, removed, or modified on the same document. Example: a website's assets 
 might be stored on a document. Whenever one file is changed every file on the 
 website drops out of cache.
 If we could use the MD5 of the attachment for the ETag then any caches 
 (including the browser's) will not drop unchanged files, even if the document 
 to which they are attached is modified.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (COUCHDB-464) Allow POST to _log for external processes

2011-10-31 Thread Paul Joseph Davis (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/COUCHDB-464?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13140562#comment-13140562
 ] 

Paul Joseph Davis commented on COUCHDB-464:
---

As a matter of fact, all control characters should probably be stripped from 
user supplied log messages. There's a CVE out there somewhere that discusses 
how to inject control sequences so that if someone tails the log as root it'll 
execute arbitrary commands. Technically more of a bug in the terminal, but 
still something to be aware of.

 Allow POST to _log for external processes
 -

 Key: COUCHDB-464
 URL: https://issues.apache.org/jira/browse/COUCHDB-464
 Project: CouchDB
  Issue Type: New Feature
Reporter: Robert Newson
 Fix For: 1.3

 Attachments: 0001-Add-POST-support-to-_log.patch, 
 0001-Add-POST-support-to-_log.patch, 0001-Add-POST-support-to-_log.patch


 Add POST support to _log so that external processes can also log to 
 couch.log. This would allow couchdb-lucene (to pick a random example) to log 
 consistently. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (COUCHDB-1319) Headers larger than 4k cannot be retrieved

2011-10-30 Thread Paul Joseph Davis (Commented) (JIRA)

[
https://issues.apache.org/jira/browse/COUCHDB-1319?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13139723#comment-13139723
]

Paul Joseph Davis commented on COUCHDB-1319:

It's a straightforward bug and has a test. I wouldn't have even bothered with a
ticket.

Headers larger than 4k cannot be retrieved
--

Key: COUCHDB-1319
URL: https://issues.apache.org/jira/browse/COUCHDB-1319
Project: CouchDB
Issue Type: Bug
Affects Versions: 1.1, 1.1.1
Reporter: Adam Kocoloski
Assignee: Adam Kocoloski
Fix For: 1.2, 1.1.2

Our headers start with a 1 and then four bytes indicating the length of
the header and its checksum. When the header is larger than 4090 bytes it
will be split across multiple blocks in the file and will need to be
reassembled on read. The reassembly consists of stripping out 0 from the
beginning of each subsequent block in the remove_block_prefixes/2 function.
The bug here is that we tell remove_block_prefixes that we're starting 1 byte
into the current block instead of 5, so it ends up removing one or more good
bytes from the header and injecting one or more random 0s.
Headers larger than 4k are very rare and generally require a view group with
a huge number of indexes or indexes with fairly large reductions, which
explains why this bug has gone undetected until now.
Patch forthcoming.

[jira] [Commented] (COUCHDB-1309) File descriptor leaks on design document update and view cleanup

2011-10-20 Thread Paul Joseph Davis (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/COUCHDB-1309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13131954#comment-13131954
 ] 

Paul Joseph Davis commented on COUCHDB-1309:


I've been thinking about this and it occurs to me that adding the new function 
to the index seems like the wrong way to go about this. Seems to me that 
couch_index_server and couch_index should take care of all of this without 
requiring each index to reimplement this simple code. I'll take a quick pass 
through later to try and sketch out what i mean.

 File descriptor leaks on design document update and view cleanup
 

 Key: COUCHDB-1309
 URL: https://issues.apache.org/jira/browse/COUCHDB-1309
 Project: CouchDB
  Issue Type: Bug
Reporter: Filipe Manana
Assignee: Filipe Manana
 Attachments: couchdb-1309_12x.patch, couchdb-1309_trunk.patch


 If we add a design document with views defined in it, open the corresponding 
 view group (by querying one of its views for e.g.), then update the design 
 document in such a way that the view signature changes (changing a view's map 
 function code for e.g), the old view group remains open forever (unless a 
 server restart happens) and keeps it's view file reference counter active 
 forever.
 If a view cleanup request comes, the old view file is not deleted since the 
 reference counter is not dropped by the old view group, keeping the file 
 descriptor in use forever.
 This leakage is different from what is reported in COUCHDB-1129 but it's 
 somehow related.
 The attached patch, simply shutdowns a view group when the design document is 
 updated and the new view signature changes, releasing the old view file 
 descriptor (as soon as no more clients are using the old view).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (COUCHDB-860) Futon appends wrong version number to files

2011-10-20 Thread Paul Joseph Davis (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/COUCHDB-860?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13132144#comment-13132144
 ] 

Paul Joseph Davis commented on COUCHDB-860:
---

No, that busts cache on everything served from the utils directory. What you 
want to do is to generate the files that reference the version with Autotools.

 Futon appends wrong version number to files
 ---

 Key: COUCHDB-860
 URL: https://issues.apache.org/jira/browse/COUCHDB-860
 Project: CouchDB
  Issue Type: Bug
  Components: Futon
Affects Versions: 1.0.1
Reporter: Volker Mische
Assignee: Randall Leeds
Priority: Minor
 Attachments: 
 0001-remove-version-number-from-futon-static-resources.patch, 
 0002-Futon-Cache-Control.patch


 Futon appends the CouchDB version number to the JavaScript files it loads. In 
 1.0.1 it still appends 0.11

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (COUCHDB-860) Futon appends wrong version number to files

2011-10-20 Thread Paul Joseph Davis (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/COUCHDB-860?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13132196#comment-13132196
 ] 

Paul Joseph Davis commented on COUCHDB-860:
---

Hrm, seems like something that we should reasonably expect. But its browsers, 
so I'd double check that its still conditional. If so, then I'm fine with that 
approach.

 Futon appends wrong version number to files
 ---

 Key: COUCHDB-860
 URL: https://issues.apache.org/jira/browse/COUCHDB-860
 Project: CouchDB
  Issue Type: Bug
  Components: Futon
Affects Versions: 1.0.1
Reporter: Volker Mische
Assignee: Randall Leeds
Priority: Minor
 Attachments: 
 0001-remove-version-number-from-futon-static-resources.patch, 
 0002-Futon-Cache-Control.patch


 Futon appends the CouchDB version number to the JavaScript files it loads. In 
 1.0.1 it still appends 0.11

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (COUCHDB-1309) File descriptor leaks on design document update and view cleanup

2011-10-17 Thread Paul Joseph Davis (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/COUCHDB-1309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13129042#comment-13129042
 ] 

Paul Joseph Davis commented on COUCHDB-1309:


Hrm. Good catch Filipe.

I haven't done more than skim the patch, but I'm wondering if it might be 
easier to just shutdown the view group when it sees the new design doc during 
an update instead of adding all of the mechanics to listen for update 
notifications. I'll read through this more when I get done figuring out how deb 
packages work.

 File descriptor leaks on design document update and view cleanup
 

 Key: COUCHDB-1309
 URL: https://issues.apache.org/jira/browse/COUCHDB-1309
 Project: CouchDB
  Issue Type: Bug
Reporter: Filipe Manana
Assignee: Filipe Manana
 Attachments: couchdb-1309_12x.patch, couchdb-1309_trunk.patch


 If we add a design document with views defined in it, open the corresponding 
 view group (by querying one of its views for e.g.), then update the design 
 document in such a way that the view signature changes (changing a view's map 
 function code for e.g), the old view group remains open forever (unless a 
 server restart happens) and keeps it's view file reference counter active 
 forever.
 If a view cleanup request comes, the old view file is not deleted since the 
 reference counter is not dropped by the old view group, keeping the file 
 descriptor in use forever.
 This leakage is different from what is reported in COUCHDB-1129 but it's 
 somehow related.
 The attached patch, simply shutdowns a view group when the design document is 
 updated and the new view signature changes, releasing the old view file 
 descriptor (as soon as no more clients are using the old view).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (COUCHDB-1309) File descriptor leaks on design document update and view cleanup

2011-10-17 Thread Paul Joseph Davis (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/COUCHDB-1309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13129056#comment-13129056
 ] 

Paul Joseph Davis commented on COUCHDB-1309:


Filipe,

Good call. That code I'm remember is probably from before we used signatures 
for view groups instead of just the design name.

 File descriptor leaks on design document update and view cleanup
 

 Key: COUCHDB-1309
 URL: https://issues.apache.org/jira/browse/COUCHDB-1309
 Project: CouchDB
  Issue Type: Bug
Reporter: Filipe Manana
Assignee: Filipe Manana
 Attachments: couchdb-1309_12x.patch, couchdb-1309_trunk.patch


 If we add a design document with views defined in it, open the corresponding 
 view group (by querying one of its views for e.g.), then update the design 
 document in such a way that the view signature changes (changing a view's map 
 function code for e.g), the old view group remains open forever (unless a 
 server restart happens) and keeps it's view file reference counter active 
 forever.
 If a view cleanup request comes, the old view file is not deleted since the 
 reference counter is not dropped by the old view group, keeping the file 
 descriptor in use forever.
 This leakage is different from what is reported in COUCHDB-1129 but it's 
 somehow related.
 The attached patch, simply shutdowns a view group when the design document is 
 updated and the new view signature changes, releasing the old view file 
 descriptor (as soon as no more clients are using the old view).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (COUCHDB-1309) File descriptor leaks on design document update and view cleanup

2011-10-17 Thread Paul Joseph Davis (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/COUCHDB-1309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13129120#comment-13129120
 ] 

Paul Joseph Davis commented on COUCHDB-1309:


@Filipe

After reading further I think you've probably got the best approach. The only 
alternative I've contemplated would be to have a monitor process that wakes up 
every so often and checks that its sig is still valid which I don't think is as 
clean as this version.

Although, I would like to see this patch as at least two if not more commits. 
Specifically, at least one for the new ddoc event broadcasting and then one for 
the indexer/view stuff.

 File descriptor leaks on design document update and view cleanup
 

 Key: COUCHDB-1309
 URL: https://issues.apache.org/jira/browse/COUCHDB-1309
 Project: CouchDB
  Issue Type: Bug
Reporter: Filipe Manana
Assignee: Filipe Manana
 Attachments: couchdb-1309_12x.patch, couchdb-1309_trunk.patch


 If we add a design document with views defined in it, open the corresponding 
 view group (by querying one of its views for e.g.), then update the design 
 document in such a way that the view signature changes (changing a view's map 
 function code for e.g), the old view group remains open forever (unless a 
 server restart happens) and keeps it's view file reference counter active 
 forever.
 If a view cleanup request comes, the old view file is not deleted since the 
 reference counter is not dropped by the old view group, keeping the file 
 descriptor in use forever.
 This leakage is different from what is reported in COUCHDB-1129 but it's 
 somehow related.
 The attached patch, simply shutdowns a view group when the design document is 
 updated and the new view signature changes, releasing the old view file 
 descriptor (as soon as no more clients are using the old view).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (COUCHDB-1310) '/_restart' closes request socket before sending a response

2011-10-17 Thread Paul Joseph Davis (Commented) (JIRA)

[
https://issues.apache.org/jira/browse/COUCHDB-1310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13129373#comment-13129373
]

Paul Joseph Davis commented on COUCHDB-1310:

I'd point out that _restart is really only meant for test code to get the
server into a known state. This patch merely hides the issue and doesn't really
address the underlying problem, namely, that init:restart is a hard core reset
of the VM. If there are concurrent clients accessing the server they will be
just as exposed to the original problem. About the only thing to do would be to
figure out how to gracefully quit mochiweb before calling init:restart. Its not
out of the question, but we moved to this approach when the graceful one didn't
work as reliably.

'/_restart' closes request socket before sending a response
---

Key: COUCHDB-1310
URL: https://issues.apache.org/jira/browse/COUCHDB-1310
Project: CouchDB
Issue Type: Bug
Components: HTTP Interface
Affects Versions: 1.1
Environment: All platforms. Tested on Ubuntu 10.10.
Reporter: J. Lee Coltrane
Priority: Minor
Fix For: 1.1

Attachments:
0001-fixed-_restart-closes-socket-before-sending-request.patch

Original Estimate: 0h
Remaining Estimate: 0h

In 'couchdb_misc_handlers.erl', `handle_restart_req(...)` calls
`couch_server_sup:restart_core_server()` before sending the HTTP response.
This causes the current request's socket to be closed without sending any
response back to the client. From the perspective of the HTTP client, the
connection is simply dropped - no response is ever received.
In addition to the obvious aesthetic problems here, the HTTP1.1 spec suggests
a specific (and non-desirable) client-side behavior for connections that drop
like this. From RFC-2616 Section-8.2.4: ...if the client sees the connection
close before receiving any status from the server, the client SHOULD retry
the request... (http://tools.ietf.org/html/rfc2616#section-8.2.4). Any HTTP
client that actually obeys this direction, would end up restarting the server
multiple times.
I have a patch that fixes this issue. I will attach it to this report.
This issue may be related to COUCHDB-946
(https://issues.apache.org/jira/browse/COUCHDB-946).

[jira] [Commented] (COUCHDB-1129) file descriptors sometimes not closed after compaction

2011-10-14 Thread Paul Joseph Davis (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/COUCHDB-1129?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13127312#comment-13127312
 ] 

Paul Joseph Davis commented on COUCHDB-1129:


Are there any view updates running?

 file descriptors sometimes not closed after compaction
 --

 Key: COUCHDB-1129
 URL: https://issues.apache.org/jira/browse/COUCHDB-1129
 Project: CouchDB
  Issue Type: Bug
  Components: Database Core
Affects Versions: 1.0.2
Reporter: Randall Leeds
 Fix For: 1.2


 It seems there are still cases where file descriptors are not released upon 
 compaction finishing.
 When I asked on IRC rnewson confirmed he'd seen the behavior also and the 
 last comment on 926 also suggests there are still times where this occurs.
 Someone needs to take a careful eye to any race conditions we might have 
 between opening the file and subscribing to the compaction notification.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (COUCHDB-1129) file descriptors sometimes not closed after compaction

2011-10-14 Thread Paul Joseph Davis (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/COUCHDB-1129?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13127788#comment-13127788
 ] 

Paul Joseph Davis commented on COUCHDB-1129:


@Dan Your compaction files won't go away until anything using them has also 
gone away. So I'd first make sure you don't have a long view read, view 
updater, or other long lived connection holding open the database.

 file descriptors sometimes not closed after compaction
 --

 Key: COUCHDB-1129
 URL: https://issues.apache.org/jira/browse/COUCHDB-1129
 Project: CouchDB
  Issue Type: Bug
  Components: Database Core
Affects Versions: 1.0.2
Reporter: Randall Leeds
 Fix For: 1.2


 It seems there are still cases where file descriptors are not released upon 
 compaction finishing.
 When I asked on IRC rnewson confirmed he'd seen the behavior also and the 
 last comment on 926 also suggests there are still times where this occurs.
 Someone needs to take a careful eye to any race conditions we might have 
 between opening the file and subscribing to the compaction notification.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (COUCHDB-1305) keep db crashes from bubbling through couch_server when possible

2011-10-13 Thread Paul Joseph Davis (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/COUCHDB-1305?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13127064#comment-13127064
 ] 

Paul Joseph Davis commented on COUCHDB-1305:


Sounds like a good idea to me.

 keep db crashes from bubbling through couch_server when possible
 

 Key: COUCHDB-1305
 URL: https://issues.apache.org/jira/browse/COUCHDB-1305
 Project: CouchDB
  Issue Type: Improvement
  Components: Database Core
Reporter: Randall Leeds
Assignee: Randall Leeds
Priority: Minor
 Fix For: 1.3


 COUCHDB-1050 shows how an error in a db process can cause unexpected 'EXIT' 
 messages to reach couch_server. These messages cause couch_server to restart, 
 forcing all open databases to close, which has nasty effects like killing 
 replications and index generation in unrelated databases. I suggest we use 
 the couch_dbs_by_pid table to swallow the error, cleanup the ETS tables, log 
 a message and continue. If a match isn't found in the table then the current 
 unexpected message crash should be preserved.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (COUCHDB-577) couchdb @ winxp: Simple RegExp's can not be executed

2011-10-11 Thread Paul Joseph Davis (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/COUCHDB-577?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13125444#comment-13125444
 ] 

Paul Joseph Davis commented on COUCHDB-577:
---

@Chas,

The issue is that SpiderMonkey 1.8.5 means about squat in the real life. 
There's a revision in trunk that removes the JSOPTION_ANONFUNFIX option for 
contexts which breaks all the things. The SpiderMonkey 1.8.5 from the tarball 
on ftp.mozilla.org does work with CouchDB. And there's a patch waiting for a 
couple tests that'll prevent couchdb from building if it finds a SpiderMonkey 
that doesn't have the anonfunfix thing.

Also, that 1.8.0 version is actually not as old as the 1.7 that's still used in 
lots of places.

 couchdb @ winxp: Simple RegExp's can not be executed
 

 Key: COUCHDB-577
 URL: https://issues.apache.org/jira/browse/COUCHDB-577
 Project: CouchDB
  Issue Type: Bug
  Components: JavaScript View Server
Affects Versions: 0.10, 0.11
 Environment: WinXP, setup-couchdb-0.11.0b880589.exe, 
 setup-couchdb-0.10.0.exe
Reporter: Thomas Mohaupt

 This show function failed on WinXP but pass on Linux:
 function (doc, req) {
   var rr2 = new RegExp(\n);
   rr2.exec(fafdadsds);
   var rr = /\n/;
   rr.exec(fafdadsds);
   return ok;
 }
 The first RegExp is ok, but the second one (simple form) can not be executed.
 1 [debug] [0.73.0] 'GET' /c2/_design/dash/_show/overview {1,1}
 Headers: 
 [{'Accept',text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8},
   {'Accept-Charset',ISO-8859-1,utf-8;q=0.7,*;q=0.7},
   {'Accept-Encoding',gzip,deflate},
   {'Accept-Language',de-de,de;q=0.8,en-us;q=0.5,en;q=0.3},
   {'Cache-Control',max-age=0},
   {'Connection',keep-alive},
   {'Host',127.0.0.1:5984},
   {'Keep-Alive',300},
   {'User-Agent',Mozilla/5.0 (Windows; U; Windows NT 5.1; de; 
 rv:1.9.1.5) Gecko/20091102 Firefox/3.5.5 (.NET CLR
  3.5.30729)}]
 1 [debug] [0.73.0] OAuth Params: []
 1 [info] [0.98.0] OS Process :: function raised error: TypeError: rr.exec 
 is not a function
 1 [info] [0.98.0] OS Process :: stacktrace: (null,[object Object])@:4
 runShow(function (doc, req) {var rr2 = new 
 RegExp(\n);rr2.exec(fafdadsds);var rr = /\n/;rr.exec(fafdadsds);return
 ok;},null,[object Object],function (doc, req) {\n  var rr2 = new 
 RegExp(\\\n\);\n  rr2.exec(\fafdadsds\);\n  var
 rr = /\\n/;\n  rr.exec(\fafdadsds\);\n\n  return 
 \ok\;\n}\n\n)@../share/couchdb/server/main.js:388
 (function (doc, req) {\n  var rr2 = new RegExp(\\\n\);\n  
 rr2.exec(\fafdadsds\);\n  var rr = /\\n/;\n  rr.exec(\fa
 fdadsds\);\n\n  return \ok\;\n}\n\n,null,[object 
 Object])@../share/couchdb/server/main.js:358
 @../share/couchdb/server/main.js:842

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (COUCHDB-577) couchdb @ winxp: Simple RegExp's can not be executed

2011-10-10 Thread Paul Joseph Davis (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/COUCHDB-577?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13124300#comment-13124300
 ] 

Paul Joseph Davis commented on COUCHDB-577:
---

@Chas

Do you know what version of SpiderMonkey that couchbase package is using?

 couchdb @ winxp: Simple RegExp's can not be executed
 

 Key: COUCHDB-577
 URL: https://issues.apache.org/jira/browse/COUCHDB-577
 Project: CouchDB
  Issue Type: Bug
  Components: JavaScript View Server
Affects Versions: 0.10, 0.11
 Environment: WinXP, setup-couchdb-0.11.0b880589.exe, 
 setup-couchdb-0.10.0.exe
Reporter: Thomas Mohaupt

 This show function failed on WinXP but pass on Linux:
 function (doc, req) {
   var rr2 = new RegExp(\n);
   rr2.exec(fafdadsds);
   var rr = /\n/;
   rr.exec(fafdadsds);
   return ok;
 }
 The first RegExp is ok, but the second one (simple form) can not be executed.
 1 [debug] [0.73.0] 'GET' /c2/_design/dash/_show/overview {1,1}
 Headers: 
 [{'Accept',text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8},
   {'Accept-Charset',ISO-8859-1,utf-8;q=0.7,*;q=0.7},
   {'Accept-Encoding',gzip,deflate},
   {'Accept-Language',de-de,de;q=0.8,en-us;q=0.5,en;q=0.3},
   {'Cache-Control',max-age=0},
   {'Connection',keep-alive},
   {'Host',127.0.0.1:5984},
   {'Keep-Alive',300},
   {'User-Agent',Mozilla/5.0 (Windows; U; Windows NT 5.1; de; 
 rv:1.9.1.5) Gecko/20091102 Firefox/3.5.5 (.NET CLR
  3.5.30729)}]
 1 [debug] [0.73.0] OAuth Params: []
 1 [info] [0.98.0] OS Process :: function raised error: TypeError: rr.exec 
 is not a function
 1 [info] [0.98.0] OS Process :: stacktrace: (null,[object Object])@:4
 runShow(function (doc, req) {var rr2 = new 
 RegExp(\n);rr2.exec(fafdadsds);var rr = /\n/;rr.exec(fafdadsds);return
 ok;},null,[object Object],function (doc, req) {\n  var rr2 = new 
 RegExp(\\\n\);\n  rr2.exec(\fafdadsds\);\n  var
 rr = /\\n/;\n  rr.exec(\fafdadsds\);\n\n  return 
 \ok\;\n}\n\n)@../share/couchdb/server/main.js:388
 (function (doc, req) {\n  var rr2 = new RegExp(\\\n\);\n  
 rr2.exec(\fafdadsds\);\n  var rr = /\\n/;\n  rr.exec(\fa
 fdadsds\);\n\n  return \ok\;\n}\n\n,null,[object 
 Object])@../share/couchdb/server/main.js:358
 @../share/couchdb/server/main.js:842

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (COUCHDB-577) couchdb @ winxp: Simple RegExp's can not be executed

2011-10-10 Thread Paul Joseph Davis (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/COUCHDB-577?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13124338#comment-13124338
 ] 

Paul Joseph Davis commented on COUCHDB-577:
---

@Chas

Most likely there's some option that SpiderMonkey has that affects the way that 
RegEx initializers work. I'll do a bit of looking but the SM docs aren't broken 
up between versions very well.

 couchdb @ winxp: Simple RegExp's can not be executed
 

 Key: COUCHDB-577
 URL: https://issues.apache.org/jira/browse/COUCHDB-577
 Project: CouchDB
  Issue Type: Bug
  Components: JavaScript View Server
Affects Versions: 0.10, 0.11
 Environment: WinXP, setup-couchdb-0.11.0b880589.exe, 
 setup-couchdb-0.10.0.exe
Reporter: Thomas Mohaupt

 This show function failed on WinXP but pass on Linux:
 function (doc, req) {
   var rr2 = new RegExp(\n);
   rr2.exec(fafdadsds);
   var rr = /\n/;
   rr.exec(fafdadsds);
   return ok;
 }
 The first RegExp is ok, but the second one (simple form) can not be executed.
 1 [debug] [0.73.0] 'GET' /c2/_design/dash/_show/overview {1,1}
 Headers: 
 [{'Accept',text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8},
   {'Accept-Charset',ISO-8859-1,utf-8;q=0.7,*;q=0.7},
   {'Accept-Encoding',gzip,deflate},
   {'Accept-Language',de-de,de;q=0.8,en-us;q=0.5,en;q=0.3},
   {'Cache-Control',max-age=0},
   {'Connection',keep-alive},
   {'Host',127.0.0.1:5984},
   {'Keep-Alive',300},
   {'User-Agent',Mozilla/5.0 (Windows; U; Windows NT 5.1; de; 
 rv:1.9.1.5) Gecko/20091102 Firefox/3.5.5 (.NET CLR
  3.5.30729)}]
 1 [debug] [0.73.0] OAuth Params: []
 1 [info] [0.98.0] OS Process :: function raised error: TypeError: rr.exec 
 is not a function
 1 [info] [0.98.0] OS Process :: stacktrace: (null,[object Object])@:4
 runShow(function (doc, req) {var rr2 = new 
 RegExp(\n);rr2.exec(fafdadsds);var rr = /\n/;rr.exec(fafdadsds);return
 ok;},null,[object Object],function (doc, req) {\n  var rr2 = new 
 RegExp(\\\n\);\n  rr2.exec(\fafdadsds\);\n  var
 rr = /\\n/;\n  rr.exec(\fafdadsds\);\n\n  return 
 \ok\;\n}\n\n)@../share/couchdb/server/main.js:388
 (function (doc, req) {\n  var rr2 = new RegExp(\\\n\);\n  
 rr2.exec(\fafdadsds\);\n  var rr = /\\n/;\n  rr.exec(\fa
 fdadsds\);\n\n  return \ok\;\n}\n\n,null,[object 
 Object])@../share/couchdb/server/main.js:358
 @../share/couchdb/server/main.js:842

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (COUCHDB-1302) Fix couchjs

2011-10-07 Thread Paul Joseph Davis (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/COUCHDB-1302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13122622#comment-13122622
 ] 

Paul Joseph Davis commented on COUCHDB-1302:


I'm not entirely sure what you mean about encapsulation, but just evaling code 
and hoping for the best doesn't seem like the best solution here.

When you eval something, it runs. So this case means we'd have to eval the 
function once per doc per view.

Plus, other things.

 Fix couchjs
 ---

 Key: COUCHDB-1302
 URL: https://issues.apache.org/jira/browse/COUCHDB-1302
 Project: CouchDB
  Issue Type: Improvement
  Components: JavaScript View Server
Affects Versions: 1.1.1, 1.2, 1.3
Reporter: Paul Joseph Davis
Priority: Blocker

 Figure out why some spidermonkeys have an error when doing: 
 eval(function(){})

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (COUCHDB-1302) Fix couchjs

2011-10-07 Thread Paul Joseph Davis (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/COUCHDB-1302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13122643#comment-13122643
 ] 

Paul Joseph Davis commented on COUCHDB-1302:


Source manipulations worry me. That's basically what I tired doing with the 
paren hack before thinking better of it and that's turned out to be quite a bad 
idea. As far as I'm concerned, if it can't be implemented at the level of the 
JSAPI, then its nothing more than hack that is just going to lead to more pain 
down the road.

I'm all for tools that we don't rely on directly to help users find 
incompatible code, but those sorts of things don't strike me as having a place 
in trunk.

 Fix couchjs
 ---

 Key: COUCHDB-1302
 URL: https://issues.apache.org/jira/browse/COUCHDB-1302
 Project: CouchDB
  Issue Type: Improvement
  Components: JavaScript View Server
Affects Versions: 1.1.1, 1.2, 1.3
Reporter: Paul Joseph Davis
Priority: Blocker

 Figure out why some spidermonkeys have an error when doing: 
 eval(function(){})

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (COUCHDB-1302) Fix couchjs

2011-10-07 Thread Paul Joseph Davis (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/COUCHDB-1302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13122652#comment-13122652
 ] 

Paul Joseph Davis commented on COUCHDB-1302:


On the face of it, I'm not a fan of this. It seems... wrong. But its late so 
maybe I'm just not seeing it.

 Fix couchjs
 ---

 Key: COUCHDB-1302
 URL: https://issues.apache.org/jira/browse/COUCHDB-1302
 Project: CouchDB
  Issue Type: Improvement
  Components: JavaScript View Server
Affects Versions: 1.1.1, 1.2, 1.3
Reporter: Paul Joseph Davis
Priority: Blocker

 Figure out why some spidermonkeys have an error when doing: 
 eval(function(){})

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (COUCHDB-1302) Fix couchjs

2011-10-06 Thread Paul Joseph Davis (Commented) (JIRA)

[
https://issues.apache.org/jira/browse/COUCHDB-1302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13121748#comment-13121748
]

Paul Joseph Davis commented on COUCHDB-1302:

From Benoit on the ML:

if we do that it would imply the release of a major version since the change
is major. I'm +1 to introduce correct behaviour in trunk rather than trying
to hack arouind our own insanity.

I think its a good idea to revert this on 1.1.x as it could break user code for
no good reason. Though 1.1.x will have the ability to use newer SpiderMonkey's,
they'll have to deal with upgrading their JS code.

What we do about 1.2.x and trunk I'm less certain. On the one hand, we could
save some people some effort. On the other hand, we'd also be breaking things
for people that use helper functions that haven't upgraded to 1.8.5 or w/e
version of SM introduces this.

Bottom line is that there's no good answer other than making sure we note it in
our NEWS files, and on the website and also try and document it in error
messages.

Quite an unfortunate situation that I see no good answer to.

Fix couchjs
---

Key: COUCHDB-1302
URL: https://issues.apache.org/jira/browse/COUCHDB-1302
Project: CouchDB
Issue Type: Improvement
Components: JavaScript View Server
Affects Versions: 1.1.1, 1.2, 1.3
Reporter: Paul Joseph Davis
Priority: Blocker

Figure out why some spidermonkeys have an error when doing:
eval(function(){})

[jira] [Commented] (COUCHDB-1302) Fix couchjs

2011-10-06 Thread Paul Joseph Davis (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/COUCHDB-1302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13122206#comment-13122206
 ] 

Paul Joseph Davis commented on COUCHDB-1302:


Ok. Here's the situation:

First, an anonymous function at the root of a scope is invalid JavaScript. Ie, 
the following is invalid:

function(doc) {emit(doc._id, 1);}

Is broken JavaScript.

But, SpiderMonkey has had an option for years [1] that allowed this. 
SpiderMonkey happened to be the only interpreter that has had this. The setting 
JSOPTION_ANONFUNFIX existed to enforce the error mechanism. The thing is, it 
was off by default (ie, the error was not triggered by default) in older js 
shells.

Recently, this appears to have caused an error for SpiderMonkey passing the 
JavaScript test suite [2]. Along with this patch, the js shell from SM 1.8.5 
enables JSOPTION_ANONFUNFIX so all of the js shell tests we were doing were 
misleading. couchjs works with the 1.8.5 tarball from Mozilla. It does not work 
starting when [2] was applied to trunk (after the 1.8.5 tarball was created).

So bottom line, this isn't an issue for all of the tarballs from Mozilla's 
official FTP site, but it will be an issue moving forward.

So the plan I'm proposing is this:

1. Re-enable support for 1.8.5 on 1.1.x for 1.1.1

2. Revert the paren hack across the board.

3. Add a configure check for JSOPTION_ANONFUNFIX so we can detect if
   user code will break (and perhaps add an option to override it)

4. Update 1.2.x with some new behavior that will force people to upgrade
   all of their function definitions to be correct moving forward. Also note 
that
   this would be backwards compatible if the named function is the last 
statement
   as we currently require which should be enforceable in the future (if we so
   desire).

I think 1, 2, and 3 are relatively uncontroversial at this point. We now know 
how to detect exactly when old code will break and can warn about it (ie, by 
refusing to build, (the option to override I'm less certain about, I'd be +0.5 
at this point)).

As to 4, the behavior I would propose at this point is that all of our JS 
definitions would require a name in the future. For instance:

 function(doc) {emit(doc._id, 1);}

becomes:

function map(doc) {emit(doc._id, 1);}

And then couchjs+main.js gets modified to look for these specific names. I'm 
thinking we'd end up with map, reduce, filter, validate, show, 
list, and update given our current set of user definable functions. For 
users we'd also need to make our error messages better. Currently, the error 
without these names would be something generic like expression does not 
evaluate to a function or similar which could be change to something like No 
function named 'map' or similar (though, that obviously needs work cause it 
could be a real syntax error as well).

So, 1, 2, and 3 have my +1. I think 4 is probably best going forward, but we 
can open a new ticket about how to deal with the it for 1.2.

[1] https://bugzilla.mozilla.org/show_bug.cgi?id=377433
[2] https://bugzilla.mozilla.org/show_bug.cgi?id=665835

 Fix couchjs
 ---

 Key: COUCHDB-1302
 URL: https://issues.apache.org/jira/browse/COUCHDB-1302
 Project: CouchDB
  Issue Type: Improvement
  Components: JavaScript View Server
Affects Versions: 1.1.1, 1.2, 1.3
Reporter: Paul Joseph Davis
Priority: Blocker

 Figure out why some spidermonkeys have an error when doing: 
 eval(function(){})

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (COUCHDB-1302) Fix couchjs

2011-10-06 Thread Paul Joseph Davis (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/COUCHDB-1302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13122519#comment-13122519
 ] 

Paul Joseph Davis commented on COUCHDB-1302:


I agree with Randall. I think we have a plan for 1.1.1 that's sane. The 1.2 can 
of worms is going to be a big one so we should make sure and separate the 
issues there.

 Fix couchjs
 ---

 Key: COUCHDB-1302
 URL: https://issues.apache.org/jira/browse/COUCHDB-1302
 Project: CouchDB
  Issue Type: Improvement
  Components: JavaScript View Server
Affects Versions: 1.1.1, 1.2, 1.3
Reporter: Paul Joseph Davis
Priority: Blocker

 Figure out why some spidermonkeys have an error when doing: 
 eval(function(){})

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (COUCHDB-1302) Fix couchjs

2011-10-05 Thread Paul Joseph Davis (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/COUCHDB-1302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13121578#comment-13121578
 ] 

Paul Joseph Davis commented on COUCHDB-1302:


So I sat down today to try and figure out this SpiderMonkey issue. After a chat 
with the guys on #jsapi they specifically said wrap it in parens when I 
asked. Though the issue is that it gets a bit more tricky.

The situation is basically that we've apparently been using invalid JavaScript 
for the last four years. Specifically, a script like such is invalid:

function(){}

Sadly, that's how *all* of our callbacks are expected to be defined.

When I mentioned this, the suggestion was exactly to wrap it in parens which is 
what I'd discovered on my own. This works fine except for that people that have 
defined functions outside the callback scope then hit issues. Ie:

var f = function() {};
function() {f();};

Breaks when wrapped in parens. And it even breaks when not in parens because 
the anonymous function still needs to be wrapped like such:

var f = function() {};
(function(){f();})

So, we're kinda up shit creek here. I'm still trying to figure out if there's 
anything we can do to save users pain, but the recommendation was basically, 
It was fixed on purpose and its a hardcoded change that can't be configured.

So, either we tell users fix your functions or we try and do some sort of 
source level analysis to wrap that last function, or something entirely 
different.

Anyone else have Ideas?

 Fix couchjs
 ---

 Key: COUCHDB-1302
 URL: https://issues.apache.org/jira/browse/COUCHDB-1302
 Project: CouchDB
  Issue Type: Improvement
  Components: JavaScript View Server
Affects Versions: 1.1.1, 1.2, 1.3
Reporter: Paul Joseph Davis
Priority: Blocker

 Figure out why some spidermonkeys have an error when doing: 
 eval(function(){})

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (COUCHDB-1302) Fix couchjs

2011-10-05 Thread Paul Joseph Davis (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/COUCHDB-1302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13121591#comment-13121591
 ] 

Paul Joseph Davis commented on COUCHDB-1302:


I thought to ask the SpiderMonkey hackers how crazy it was to try and do source 
level analysis to automatically wrap that last anonymous function in parens. 
Jason Orendorff is on record as saying it would be hopeless to try and hack 
that outside of the SpiderMonkey engine.

The only other thing I can think of would be to try and hack JSLint's parser to 
do this for us. Other than that, I think we really are up shit creek here.

 Fix couchjs
 ---

 Key: COUCHDB-1302
 URL: https://issues.apache.org/jira/browse/COUCHDB-1302
 Project: CouchDB
  Issue Type: Improvement
  Components: JavaScript View Server
Affects Versions: 1.1.1, 1.2, 1.3
Reporter: Paul Joseph Davis
Priority: Blocker

 Figure out why some spidermonkeys have an error when doing: 
 eval(function(){})

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (COUCHDB-1285) Allow configuration of vendor and module version in the welcome message

2011-10-01 Thread Paul Joseph Davis (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/COUCHDB-1285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13118908#comment-13118908
 ] 

Paul Joseph Davis commented on COUCHDB-1285:


+1 though do you need a ?l2b(Key) when grabbing stuff from the config? The 
values have it. I don't remember autoconverting keys from strings, but I could 
be mistaken.

 Allow configuration of vendor and module version in the welcome message
 ---

 Key: COUCHDB-1285
 URL: https://issues.apache.org/jira/browse/COUCHDB-1285
 Project: CouchDB
  Issue Type: Improvement
Reporter: Jan Lehnardt
 Attachments: vendor_and_modules_objects.patch


 The patch below allows to configure vendor and module version information 
 into the GET / welcome message.
 E.g.
   [vendor]
   name = refuge
   version = 2.0.0
   
   [modules]
   geocouch = 1.2.1
 would produce:
   
 {couchdb:Welcome,version:1.2.0,refuge:2.0.0,modules:{geocouch:1.2.1}}
 --
 --- a/src/couchdb/couch_httpd_misc_handlers.erl
 +++ b/src/couchdb/couch_httpd_misc_handlers.erl
 @@ -30,9 +30,23 @@
  % httpd global handlers
  
  handle_welcome_req(#httpd{method='GET'}=Req, WelcomeMessage) -
 +Vendor = case couch_config:get(vendor) of
 +  [] - [];
 +  Vendor1 - [{
 +proplists:get_value(name, Vendor1),
 +?l2b(proplists:get_value(version, Vendor1))
 +  }]
 +end,
 +
 +Modules = lists:map(fun({Key, Value}) -
 +  {Key, ?l2b(Value)}
 +end, couch_config:get(modules)),
 +
  send_json(Req, {[
  {couchdb, WelcomeMessage},
 -{version, list_to_binary(couch_server:get_version())}
 +{version, list_to_binary(couch_server:get_version())}]
 +++ Vendor
 +++ [{modules, {Modules}}
  ]});
  handle_welcome_req(Req, _) -
  send_method_not_allowed(Req, GET,HEAD).
 -- 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (COUCHDB-893) Error: os_process_error {exit_status,0} when rendering view on 17 mb doc, couchapp and data attached

2011-10-01 Thread Paul Joseph Davis (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/COUCHDB-893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13118957#comment-13118957
 ] 

Paul Joseph Davis commented on COUCHDB-893:
---

Get rid of couchjs_dev. Its never used. And if it is, we should fix that 
instead.

What's this change doing there?

-AC_INIT([LOCAL_PACKAGE_NAME], [LOCAL_VERSION], [], [LOCAL_PACKAGE_TARNAME])
+AC_INIT([LOCAL_PACKAGE_NAME], [LOCAL_VERSION], [LOCAL_BUG_URI],
+[LOCAL_PACKAGE_TARNAME])

This is odd:

+   $(MAKE) -f Makefile couchjs;

Is it really necessary to have a sub-process make build couchjs when the build 
target already depends on it? That seems wrong.


I'd tab in the continuation lines in help.h string literals. Like you said its 
gnarly, but its not like we'll be looking at it that much.


Your macro formatting annoys me.


#define couch_usage(basename)   \
fprintf(\
stdout, \
USAGE_TEMPLATE, \
basename,   \
basename,   \
PACKAGE_NAME,   \
basename,   \
PACKAGE_BUGREPORT   \
)

Not that my slashes will line up with the comment formatting.


Left over debugging?

+fprintf(stdout,
+Script name:\t%s\n
+Stack size:\t%d\n
+HTTP enabled:\t%d\n,
+args-script_name,
+args-stack_size,
+args-use_http);
+


Not sure about libgen here. Normally I'd be all for basename of argv[0] but I 
tend to wonder if this is gonna brea on Windows. I'd go ahead and leave it 
though. If it doesn't work on windows we can just #ifdef hardcode it there.

 Error: os_process_error  {exit_status,0}  when rendering view on 17 mb doc,  
 couchapp and data attached
 ---

 Key: COUCHDB-893
 URL: https://issues.apache.org/jira/browse/COUCHDB-893
 Project: CouchDB
  Issue Type: Bug
Affects Versions: 1.0.1
 Environment: I repeated this on windows and linux (64 bit)
Reporter: Michael Schneider
Assignee: Randall Leeds
 Fix For: 1.1.1, 1.2

 Attachments: 0001-improve-argument-parsing-in-couchjs.patch, 
 bugdoc.tar.gz


 I have a large set of documents that  I harvesting data from.  All docs  9mb 
 render fine (with doc size increased)
 Attached is a simple couchapp and one doc.
 To reproduce, untar and:
  reproduce
 1)  untar file
 2) cd bugdoc/couchapp/bugreport
 3) couchapp push bugreport
 4) cd bugdoc
 5) python submitbadjson.py
 open url 
 http://127.0.0.1:5984/_utils/database.html?bugreport/_design/bugreport/_view/buggy
 You should see a popup
 (Error: os_process_error
 {exit_status,0})
 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (COUCHDB-893) Error: os_process_error {exit_status,0} when rendering view on 17 mb doc, couchapp and data attached

2011-10-01 Thread Paul Joseph Davis (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/COUCHDB-893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13118958#comment-13118958
 ] 

Paul Joseph Davis commented on COUCHDB-893:
---

Oh, and otherwise, +1

 Error: os_process_error  {exit_status,0}  when rendering view on 17 mb doc,  
 couchapp and data attached
 ---

 Key: COUCHDB-893
 URL: https://issues.apache.org/jira/browse/COUCHDB-893
 Project: CouchDB
  Issue Type: Bug
Affects Versions: 1.0.1
 Environment: I repeated this on windows and linux (64 bit)
Reporter: Michael Schneider
Assignee: Randall Leeds
 Fix For: 1.1.1, 1.2

 Attachments: 0001-improve-argument-parsing-in-couchjs.patch, 
 bugdoc.tar.gz


 I have a large set of documents that  I harvesting data from.  All docs  9mb 
 render fine (with doc size increased)
 Attached is a simple couchapp and one doc.
 To reproduce, untar and:
  reproduce
 1)  untar file
 2) cd bugdoc/couchapp/bugreport
 3) couchapp push bugreport
 4) cd bugdoc
 5) python submitbadjson.py
 open url 
 http://127.0.0.1:5984/_utils/database.html?bugreport/_design/bugreport/_view/buggy
 You should see a popup
 (Error: os_process_error
 {exit_status,0})
 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (COUCHDB-1294) Use Jiffy for JSON serialization

2011-09-29 Thread Paul Joseph Davis (Commented) (JIRA)

[
https://issues.apache.org/jira/browse/COUCHDB-1294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13117872#comment-13117872
]

Paul Joseph Davis commented on COUCHDB-1294:

Most odd. I think this is a bit tied into that last fix I did. Either way, this
is a SM error and not a Jiffy error. I'm quite perplexed on what the interplay
might be though.

Also, the SM fix I did the other day seems like less and less of a fix.

Use Jiffy for JSON serialization

Key: COUCHDB-1294
URL: https://issues.apache.org/jira/browse/COUCHDB-1294
Project: CouchDB
Issue Type: Improvement
Components: Database Core, HTTP Interface
Affects Versions: 1.2
Reporter: Paul Joseph Davis
Labels: json
Attachments: 0001-Import-Jiffy.patch, 0002-Remove-old-ejson-app.patch

Jiffy is a NIF for JSON serialization. I wrote it quite a while ago but have
finally gotten around to writing the integration for CouchDB. This is a
direct replacement for ejson which does much less parsing in C. Jiffy is as
close to mochijson2 (with our custom objects) as humanly possible. At the
moment there are no known differences in output. Jiffy itself has been tested
fairly extensively by a couple different groups so I'm fairly confident there
aren't any major bugs left.

[jira] [Commented] (COUCHDB-1294) Use Jiffy for JSON serialization

2011-09-29 Thread Paul Joseph Davis (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/COUCHDB-1294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13117879#comment-13117879
 ] 

Paul Joseph Davis commented on COUCHDB-1294:


The fallback is based on configure declaring a compile time definition that 
couch_db.hrl uses to determine which JSON encoder to use.

 Use Jiffy for JSON serialization
 

 Key: COUCHDB-1294
 URL: https://issues.apache.org/jira/browse/COUCHDB-1294
 Project: CouchDB
  Issue Type: Improvement
  Components: Database Core, HTTP Interface
Affects Versions: 1.2
Reporter: Paul Joseph Davis
  Labels: json
 Attachments: 0001-Import-Jiffy.patch, 0002-Remove-old-ejson-app.patch


 Jiffy is a NIF for JSON serialization. I wrote it quite a while ago but have 
 finally gotten around to writing the integration for CouchDB. This is a 
 direct replacement for ejson which does much less parsing in C. Jiffy is as 
 close to mochijson2 (with our custom objects) as humanly possible. At the 
 moment there are no known differences in output. Jiffy itself has been tested 
 fairly extensively by a couple different groups so I'm fairly confident there 
 aren't any major bugs left.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

1 2 >

1 - 100 of 101 matches

Mail list logo