Re: [Windows] proposed binary build changes
Hi,
It seems to be ok, since I get:
> All 2 tests passed.
I also tried compaction again, but it didn't compress the 800Mb .couch
file, so I guess something else might be needed aswell?
Best regards,
Jeroen
On Sun, Jan 22, 2012 at 11:53 PM, Dave Cottlehuber wrote:
> On 22 January 2012 15:49, Dave Cottlehuber wrote:
>> On 22 January 2012 13:28, Jeroen Janssen wrote:
>>> Hi,
>>>
>>> Just a quick check since I am not able to get (snappy) compression
>>> working on windows.
>>
>> Hi Jeroen,
>>
>> Thanks for reporting this. AFAICT the snappy_nif.dll is not being copied
>> during
>> make dist or install.
>>
>> IIRC you ran into this last year on master and resulted in COUCHDB-1197
>> fixes.
>> So I'm not sure what is missing on 1.2.x for this, or if its more
>> likely an issue with
>> my packaging steps.
>>
>> I'll keep you posted.
>>
>> BTW compression is on by default so no need to tweak ini files.
>>
>> A+
>> Dave
>
> Hi Jeroen,
>
> While I look for why this is not working in the build, can you copy
> snappy_nif.dll [1] into $COUCH/lib/snappy-1.0.3/priv/ and then test
> with snappy_test.erl courtesy of Filipe [2]?
>
> 3> pwd().
> C:/couch/1.2.0a-8d83b39-git_otp_R15B_SDK7.1/bin
> ok
> 4> c("/couch/snappy_tests").
> {ok,snappy_tests}
> 5> snappy_tests:test().
> All 2 tests passed.
> ok
>
> Your higher-level compression tests should be fine.
>
> [1]: https://www.dropbox.com/s/jeifcxpbtpo78ak/snapshots/20120116?v=l
> [2]
> https://github.com/fdmanana/snappy-erlang-nif/blob/master/test/snappy_tests.erl
>
> A+
> Dave
[jira] [Closed] (COUCHDB-1387) couch_index_server:reset_indexes/2 does not use the correct utility function
[
https://issues.apache.org/jira/browse/COUCHDB-1387?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Randall Leeds closed COUCHDB-1387.
--
Resolution: Fixed
Fix Version/s: 1.3
879a1bc..50ae62f master -> master
Thanks, Jason.
> couch_index_server:reset_indexes/2 does not use the correct utility function
>
>
> Key: COUCHDB-1387
> URL: https://issues.apache.org/jira/browse/COUCHDB-1387
> Project: CouchDB
> Issue Type: Bug
> Components: Database Core
>Affects Versions: 1.3
>Reporter: Jason Smith
>Priority: Trivial
> Fix For: 1.3
>
> Attachments:
> 0001-Use-the-correct-utility-function-to-get-the-index-di.patch
>
>
> It looks like couch_index_util:index_dir(Module, DbName) is the new way to
> get the path to the .db_design/ directory.
> Passing an empty string as the "module" gives the desired result. So why not
> use that?
> 1> couch_index_util:index_dir("", <<"mydb">>).
> "/Users/jhs/src/iris/couchdb/tmp/lib/.mydb_design"
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: Issues blocking 1.2.0 release
Yep, that seems reasonable. Let me know when I can test again. :)
On Mon, Jan 23, 2012 at 1:34 AM, Filipe David Manana wrote:
> Noah, was able to reproduce your issue by tweaking the test to create
> more leaf revisions for a document:
>
> diff --git a/test/etap/242-replication-many-leaves.t
> b/test/etap/242-replication-many-leaves.t
> index d8d3eb9..4eb4765 100755
> --- a/test/etap/242-replication-many-leaves.t
> +++ b/test/etap/242-replication-many-leaves.t
> @@ -56,7 +56,7 @@ doc_ids() ->
>
> doc_num_conflicts(<<"doc1">>) -> 10;
> doc_num_conflicts(<<"doc2">>) -> 100;
> -doc_num_conflicts(<<"doc3">>) -> 286.
> +doc_num_conflicts(<<"doc3">>) -> 500.
>
>
> main(_) ->
>
>
> With that change, I get exactly the same timeout as you get when the
> test runs the push replication. It turns out that some _bulk_docs
> requests are taking more than 30 seconds (default replication
> connection timeout) therefore the replication request retry messages.
> Verified this by timing the _bulk_docs handler to log the time it
> takes:
>
> diff --git a/src/couchdb/couch_httpd_db.erl
> b/src/couchdb/couch_httpd_db.erl
> index d7ecb4a..442571d 100644
> --- a/src/couchdb/couch_httpd_db.erl
> +++ b/src/couchdb/couch_httpd_db.erl
> @@ -297,6 +297,7 @@
> db_req(#httpd{path_parts=[_,<<"_ensure_full_commit">>]}=Req, _Db) ->
> send_method_not_allowed(Req, "POST");
>
> db_req(#httpd{method='POST',path_parts=[_,<<"_bulk_docs">>]}=Req, Db) ->
> +T0 = now(),
> couch_stats_collector:increment({httpd, bulk_requests}),
> couch_httpd:validate_ctype(Req, "application/json"),
> {JsonProps} = couch_httpd:json_body_obj(Req),
> @@ -357,7 +358,9 @@
> db_req(#httpd{method='POST',path_parts=[_,<<"_bulk_docs">>]}=Req, Db)
> ->
> {ok, Errors} = couch_db:update_docs(Db, Docs, Options,
> replicated_changes),
> ErrorsJson =
> lists:map(fun update_doc_result_to_json/1, Errors),
> -send_json(Req, 201, ErrorsJson)
> +Rr = send_json(Req, 201, ErrorsJson),
> +?LOG_ERROR("BULK DOCS took ~p ms~n",
> [timer:now_diff(now(), T0) / 1000]),
> +Rr
> end
> end;
> db_req(#httpd{path_parts=[_,<<"_bulk_docs">>]}=Req, _Db) ->
>
>
> I was getting _bulk_docs response times after 50 seconds.
>
> This convinces me there's nothing wrong with the codebase, the
> timeouts just needs to be increased:
>
> diff --git a/test/etap/242-replication-many-leaves.t
> b/test/etap/242-replication-many-leaves.t
> index d8d3eb9..6508112 100755
> --- a/test/etap/242-replication-many-leaves.t
> +++ b/test/etap/242-replication-many-leaves.t
> @@ -77,6 +77,7 @@ test() ->
> couch_server_sup:start_link(test_util:config_files()),
> ibrowse:start(),
> crypto:start(),
> +couch_config:set("replicator", "connection_timeout", "9", false),
>
> Pairs = [
> {source_db_name(), target_db_name()},
> @@ -287,6 +288,6 @@ replicate(Source, Target) ->
> receive
> {'DOWN', MonRef, process, Pid, Reason} ->
> etap:is(Reason, normal, "Replication finished successfully")
> -after 30 ->
> +after 90 ->
> etap:bail("Timeout waiting for replication to finish")
> end.
>
> Alternatively the test can be updated to create less revisions for the
> document doc3. The current revisions # is 286 but for the tests'
> purpose 205+ is enough, which should make it faster - 7000 (max url
> length) / length(DocRevision) = 205
>
> If it's ok for you, updating the timeouts plus reducing the # from 286
> to 210 is fine for me.
>
>
>
> On Mon, Jan 23, 2012 at 12:00 AM, Noah Slater
> wrote:
> > I'm just the dumb QA guy.
> >
> > If you have some diagnostics you want me to run on my machine, I am happy
> > to.
> >
> > On Sun, Jan 22, 2012 at 11:31 PM, Filipe David Manana
> > wrote:
> >
> >> On Sun, Jan 22, 2012 at 7:20 PM, Noah Slater
> wrote:
> >> > Works. How do we proceed?
> >>
> >> For how much time does the test runs? On 2 different physical
> >> machines, it takes about 1 minute and 10 seconds for me.
> >>
> >> Perhaps some manual replication tests could confirm if there's
> >> something wrong with the codebase, your environment or if simply
> >> increasing the timeout is not alarming.
> >>
> >> >
> >> > On Sun, Jan 22, 2012 at 7:05 PM, Noah Slater
> >> wrote:
> >> >
> >> >> OVAR 9000! (Testing now...)
> >> >>
> >> >>
> >> >> On Sun, Jan 22, 2012 at 6:56 PM, Filipe David Manana <
> >> fdman...@apache.org>wrote:
> >> >>
> >> >>> On Sun, Jan 22, 2012 at 6:47 PM, Noah Slater
> >> >>> wrote:
> >> >>> > No change, still fails.
> >> >>>
> >> >>> Noah, to try to find out if it's due to slowness of the machine or
> >> >>> some other issue, do you think you can try to increase the following
> >> >>> timeout in the test?
> >> >>>
> >> >>> diff --git a/test/etap/242-replication-many-leaves.t
> >> >>> b/test/etap/242-replication-many-leaves.t
> >> >>> index d8d3eb9..737cd31 100755
> >> >>> --- a/test/etap/242-replication-many-leaves.t
> >> >>>
Custom Erlang compiler frontend
I spent a few hours today writing a custom frontend to the Erlang compiler to see what would happen if we built beam files in parallel in a single Erlang VM. I've got it building CouchDB although its not super well tested. Some times for building: couch_erlc: 0m19.481s master: 1m11.445s master -j4: 0m51.648s Each build was run with: $ git clean -fxd && ./bootstrap && ./configure && time make Except for "master -j4" which just adds -j4 to the make command. It doesn't pass make distcheck yet but I figured I'd throw it out there in case anyone was interested. I'm not entirely convinced its worth the amount of code, but just in case anyone was interested in what about we'd get from using Rebar this should be a pretty close approximation for build times. Code is up here: https://github.com/davisp/couchdb/tree/couch_erlc
[jira] [Updated] (COUCHDB-1387) couch_index_server:reset_indexes/2 does not use the correct utility function
[
https://issues.apache.org/jira/browse/COUCHDB-1387?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jason Smith updated COUCHDB-1387:
-
Attachment: 0001-Use-the-correct-utility-function-to-get-the-index-di.patch
Attached one-liner fixes this. The same is at
https://github.com/jhs/couchdb/tree/COUCHDB-1387
> couch_index_server:reset_indexes/2 does not use the correct utility function
>
>
> Key: COUCHDB-1387
> URL: https://issues.apache.org/jira/browse/COUCHDB-1387
> Project: CouchDB
> Issue Type: Bug
> Components: Database Core
>Affects Versions: 1.3
>Reporter: Jason Smith
>Priority: Trivial
> Attachments:
> 0001-Use-the-correct-utility-function-to-get-the-index-di.patch
>
>
> It looks like couch_index_util:index_dir(Module, DbName) is the new way to
> get the path to the .db_design/ directory.
> Passing an empty string as the "module" gives the desired result. So why not
> use that?
> 1> couch_index_util:index_dir("", <<"mydb">>).
> "/Users/jhs/src/iris/couchdb/tmp/lib/.mydb_design"
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (COUCHDB-1387) couch_index_server:reset_indexes/2 does not use the correct utility function
couch_index_server:reset_indexes/2 does not use the correct utility function
Key: COUCHDB-1387
URL: https://issues.apache.org/jira/browse/COUCHDB-1387
Project: CouchDB
Issue Type: Bug
Components: Database Core
Affects Versions: 1.3
Reporter: Jason Smith
Priority: Trivial
Attachments:
0001-Use-the-correct-utility-function-to-get-the-index-di.patch
It looks like couch_index_util:index_dir(Module, DbName) is the new way to get
the path to the .db_design/ directory.
Passing an empty string as the "module" gives the desired result. So why not
use that?
1> couch_index_util:index_dir("", <<"mydb">>).
"/Users/jhs/src/iris/couchdb/tmp/lib/.mydb_design"
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: Issues blocking 1.2.0 release
Noah, was able to reproduce your issue by tweaking the test to create
more leaf revisions for a document:
diff --git a/test/etap/242-replication-many-leaves.t
b/test/etap/242-replication-many-leaves.t
index d8d3eb9..4eb4765 100755
--- a/test/etap/242-replication-many-leaves.t
+++ b/test/etap/242-replication-many-leaves.t
@@ -56,7 +56,7 @@ doc_ids() ->
doc_num_conflicts(<<"doc1">>) -> 10;
doc_num_conflicts(<<"doc2">>) -> 100;
-doc_num_conflicts(<<"doc3">>) -> 286.
+doc_num_conflicts(<<"doc3">>) -> 500.
main(_) ->
With that change, I get exactly the same timeout as you get when the
test runs the push replication. It turns out that some _bulk_docs
requests are taking more than 30 seconds (default replication
connection timeout) therefore the replication request retry messages.
Verified this by timing the _bulk_docs handler to log the time it
takes:
diff --git a/src/couchdb/couch_httpd_db.erl b/src/couchdb/couch_httpd_db.erl
index d7ecb4a..442571d 100644
--- a/src/couchdb/couch_httpd_db.erl
+++ b/src/couchdb/couch_httpd_db.erl
@@ -297,6 +297,7 @@
db_req(#httpd{path_parts=[_,<<"_ensure_full_commit">>]}=Req, _Db) ->
send_method_not_allowed(Req, "POST");
db_req(#httpd{method='POST',path_parts=[_,<<"_bulk_docs">>]}=Req, Db) ->
+T0 = now(),
couch_stats_collector:increment({httpd, bulk_requests}),
couch_httpd:validate_ctype(Req, "application/json"),
{JsonProps} = couch_httpd:json_body_obj(Req),
@@ -357,7 +358,9 @@
db_req(#httpd{method='POST',path_parts=[_,<<"_bulk_docs">>]}=Req, Db)
->
{ok, Errors} = couch_db:update_docs(Db, Docs, Options,
replicated_changes),
ErrorsJson =
lists:map(fun update_doc_result_to_json/1, Errors),
-send_json(Req, 201, ErrorsJson)
+Rr = send_json(Req, 201, ErrorsJson),
+?LOG_ERROR("BULK DOCS took ~p ms~n",
[timer:now_diff(now(), T0) / 1000]),
+Rr
end
end;
db_req(#httpd{path_parts=[_,<<"_bulk_docs">>]}=Req, _Db) ->
I was getting _bulk_docs response times after 50 seconds.
This convinces me there's nothing wrong with the codebase, the
timeouts just needs to be increased:
diff --git a/test/etap/242-replication-many-leaves.t
b/test/etap/242-replication-many-leaves.t
index d8d3eb9..6508112 100755
--- a/test/etap/242-replication-many-leaves.t
+++ b/test/etap/242-replication-many-leaves.t
@@ -77,6 +77,7 @@ test() ->
couch_server_sup:start_link(test_util:config_files()),
ibrowse:start(),
crypto:start(),
+couch_config:set("replicator", "connection_timeout", "9", false),
Pairs = [
{source_db_name(), target_db_name()},
@@ -287,6 +288,6 @@ replicate(Source, Target) ->
receive
{'DOWN', MonRef, process, Pid, Reason} ->
etap:is(Reason, normal, "Replication finished successfully")
-after 30 ->
+after 90 ->
etap:bail("Timeout waiting for replication to finish")
end.
Alternatively the test can be updated to create less revisions for the
document doc3. The current revisions # is 286 but for the tests'
purpose 205+ is enough, which should make it faster - 7000 (max url
length) / length(DocRevision) = 205
If it's ok for you, updating the timeouts plus reducing the # from 286
to 210 is fine for me.
On Mon, Jan 23, 2012 at 12:00 AM, Noah Slater wrote:
> I'm just the dumb QA guy.
>
> If you have some diagnostics you want me to run on my machine, I am happy
> to.
>
> On Sun, Jan 22, 2012 at 11:31 PM, Filipe David Manana
> wrote:
>
>> On Sun, Jan 22, 2012 at 7:20 PM, Noah Slater wrote:
>> > Works. How do we proceed?
>>
>> For how much time does the test runs? On 2 different physical
>> machines, it takes about 1 minute and 10 seconds for me.
>>
>> Perhaps some manual replication tests could confirm if there's
>> something wrong with the codebase, your environment or if simply
>> increasing the timeout is not alarming.
>>
>> >
>> > On Sun, Jan 22, 2012 at 7:05 PM, Noah Slater
>> wrote:
>> >
>> >> OVAR 9000! (Testing now...)
>> >>
>> >>
>> >> On Sun, Jan 22, 2012 at 6:56 PM, Filipe David Manana <
>> fdman...@apache.org>wrote:
>> >>
>> >>> On Sun, Jan 22, 2012 at 6:47 PM, Noah Slater
>> >>> wrote:
>> >>> > No change, still fails.
>> >>>
>> >>> Noah, to try to find out if it's due to slowness of the machine or
>> >>> some other issue, do you think you can try to increase the following
>> >>> timeout in the test?
>> >>>
>> >>> diff --git a/test/etap/242-replication-many-leaves.t
>> >>> b/test/etap/242-replication-many-leaves.t
>> >>> index d8d3eb9..737cd31 100755
>> >>> --- a/test/etap/242-replication-many-leaves.t
>> >>> +++ b/test/etap/242-replication-many-leaves.t
>> >>> @@ -287,6 +287,6 @@ replicate(Source, Target) ->
>> >>> receive
>> >>> {'DOWN', MonRef, process, Pid, Reason} ->
>> >>> etap:is(Reason, normal, "Replication finished successfully")
>> >>> - after 30 ->
>> >>> + after 90 ->
>> >>> etap:bail("Timeout waiting for replication to f
Re: Issues blocking 1.2.0 release
I'm just the dumb QA guy.
If you have some diagnostics you want me to run on my machine, I am happy
to.
On Sun, Jan 22, 2012 at 11:31 PM, Filipe David Manana
wrote:
> On Sun, Jan 22, 2012 at 7:20 PM, Noah Slater wrote:
> > Works. How do we proceed?
>
> For how much time does the test runs? On 2 different physical
> machines, it takes about 1 minute and 10 seconds for me.
>
> Perhaps some manual replication tests could confirm if there's
> something wrong with the codebase, your environment or if simply
> increasing the timeout is not alarming.
>
> >
> > On Sun, Jan 22, 2012 at 7:05 PM, Noah Slater
> wrote:
> >
> >> OVAR 9000! (Testing now...)
> >>
> >>
> >> On Sun, Jan 22, 2012 at 6:56 PM, Filipe David Manana <
> fdman...@apache.org>wrote:
> >>
> >>> On Sun, Jan 22, 2012 at 6:47 PM, Noah Slater
> >>> wrote:
> >>> > No change, still fails.
> >>>
> >>> Noah, to try to find out if it's due to slowness of the machine or
> >>> some other issue, do you think you can try to increase the following
> >>> timeout in the test?
> >>>
> >>> diff --git a/test/etap/242-replication-many-leaves.t
> >>> b/test/etap/242-replication-many-leaves.t
> >>> index d8d3eb9..737cd31 100755
> >>> --- a/test/etap/242-replication-many-leaves.t
> >>> +++ b/test/etap/242-replication-many-leaves.t
> >>> @@ -287,6 +287,6 @@ replicate(Source, Target) ->
> >>> receive
> >>> {'DOWN', MonRef, process, Pid, Reason} ->
> >>> etap:is(Reason, normal, "Replication finished successfully")
> >>> -after 30 ->
> >>> +after 90 ->
> >>> etap:bail("Timeout waiting for replication to finish")
> >>> end.
> >>>
> >>> >
> >>> > On Sun, Jan 22, 2012 at 6:08 PM, Noah Slater
> >>> wrote:
> >>> >
> >>> >>
> >>> >> On Sun, Jan 22, 2012 at 6:01 PM, Filipe David Manana <
> >>> fdman...@apache.org>wrote:
> >>> >>
> >>> >>> Noah, does it fail occasionally or every time for you?
> >>> >>>
> >>> >>
> >>> >> Fails every time.
> >>> >>
> >>> >>
> >>> >>> I'm assuming you're with a slow machine or the machine is a bit
> >>> >>> overloaded.
> >>> >>>
> >>> >>
> >>> >> Shouldn't be, I'm not doing anything else right now, and this is a
> new
> >>> MBA.
> >>> >>
> >>> >>
> >>> >>> Can you try with the following patch?
> >>> >>>
> >>> >>
> >>> >> Yes. Will report back.
> >>> >>
> >>> >>
> >>>
> >>>
> >>>
> >>> --
> >>> Filipe David Manana,
> >>>
> >>> "Reasonable men adapt themselves to the world.
> >>> Unreasonable men adapt the world to themselves.
> >>> That's why all progress depends on unreasonable men."
> >>>
> >>
> >>
>
>
>
> --
> Filipe David Manana,
>
> "Reasonable men adapt themselves to the world.
> Unreasonable men adapt the world to themselves.
> That's why all progress depends on unreasonable men."
>
[jira] [Commented] (COUCHDB-1342) Asynchronous file writes
[
https://issues.apache.org/jira/browse/COUCHDB-1342?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13190821#comment-13190821
]
Filipe Manana commented on COUCHDB-1342:
Made a few more tests comparing latest master against COUCHDB-1342 branch
(after merging master into it).
Database writes
=
* 1Kb documents
(https://github.com/fdmanana/basho_bench_couch/blob/master/couch_docs/doc_1kb.json)
http://graphs.mikeal.couchone.com/#/graph/7c13e2bdebfcd17aab424e68f225fe9a
* 2Kb documents
(https://github.com/fdmanana/basho_bench_couch/blob/master/couch_docs/doc_2kb.json)
http://graphs.mikeal.couchone.com/#/graph/7c13e2bdebfcd17aab424e68f2261504
* 11Kb documents
(https://github.com/fdmanana/basho_bench_couch/blob/master/couch_docs/doc_11kb.json)
http://graphs.mikeal.couchone.com/#/graph/7c13e2bdebfcd17aab424e68f2262e98
View indexer
==
Test database: http://fdmanana.iriscouch.com/_utils/many_docs
* master
$ echo 3 > /proc/sys/vm/drop_caches
$ time curl http://localhost:5984/many_docs/_design/test/_view/test1
{"rows":[
{"key":null,"value":2000}
]}
real29m42.041s
user0m0.016s
sys 0m0.036s
* master + patch (branch COUCHDB-1342)
$ echo 3 > /proc/sys/vm/drop_caches
$ time curl http://localhost:5984/many_docs/_design/test/_view/test1
{"rows":[
{"key":null,"value":2000}
]}
real26m13.112s
user0m0.008s
sys 0m0.036s
Before COUCHDB-1334, and possibly the refactored indexer as well, the
difference used to be more significant (like in the results presented in the
dev mail http://s.apache.org/TVu).
> Asynchronous file writes
>
>
> Key: COUCHDB-1342
> URL: https://issues.apache.org/jira/browse/COUCHDB-1342
> Project: CouchDB
> Issue Type: Improvement
> Components: Database Core
>Reporter: Jan Lehnardt
> Fix For: 1.3
>
> Attachments: COUCHDB-1342.patch
>
>
> This change updates the file module so that it can do
> asynchronous writes. Basically it replies immediately
> to process asking to write something to the file, with
> the position where the chunks will be written to the
> file, while a dedicated child process keeps collecting
> chunks and write them to the file (and batching them
> when possible). After issuing a series of write request
> to the file module, the caller can call its 'flush'
> function which will block the caller until all the
> chunks it requested to write are effectively written
> to the file.
> This maximizes the IO subsystem, as for example, while
> the updater is traversing and modifying the btrees and
> doing CPU bound tasks, the writes are happening in
> parallel.
> Originally described at http://s.apache.org/TVu
> Github Commit:
> https://github.com/fdmanana/couchdb/commit/e82a673f119b82dddf674ac2e6233cd78c123554
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: Issues blocking 1.2.0 release
On Sun, Jan 22, 2012 at 7:20 PM, Noah Slater wrote:
> Works. How do we proceed?
For how much time does the test runs? On 2 different physical
machines, it takes about 1 minute and 10 seconds for me.
Perhaps some manual replication tests could confirm if there's
something wrong with the codebase, your environment or if simply
increasing the timeout is not alarming.
>
> On Sun, Jan 22, 2012 at 7:05 PM, Noah Slater wrote:
>
>> OVAR 9000! (Testing now...)
>>
>>
>> On Sun, Jan 22, 2012 at 6:56 PM, Filipe David Manana
>> wrote:
>>
>>> On Sun, Jan 22, 2012 at 6:47 PM, Noah Slater
>>> wrote:
>>> > No change, still fails.
>>>
>>> Noah, to try to find out if it's due to slowness of the machine or
>>> some other issue, do you think you can try to increase the following
>>> timeout in the test?
>>>
>>> diff --git a/test/etap/242-replication-many-leaves.t
>>> b/test/etap/242-replication-many-leaves.t
>>> index d8d3eb9..737cd31 100755
>>> --- a/test/etap/242-replication-many-leaves.t
>>> +++ b/test/etap/242-replication-many-leaves.t
>>> @@ -287,6 +287,6 @@ replicate(Source, Target) ->
>>> receive
>>> {'DOWN', MonRef, process, Pid, Reason} ->
>>> etap:is(Reason, normal, "Replication finished successfully")
>>> - after 30 ->
>>> + after 90 ->
>>> etap:bail("Timeout waiting for replication to finish")
>>> end.
>>>
>>> >
>>> > On Sun, Jan 22, 2012 at 6:08 PM, Noah Slater
>>> wrote:
>>> >
>>> >>
>>> >> On Sun, Jan 22, 2012 at 6:01 PM, Filipe David Manana <
>>> fdman...@apache.org>wrote:
>>> >>
>>> >>> Noah, does it fail occasionally or every time for you?
>>> >>>
>>> >>
>>> >> Fails every time.
>>> >>
>>> >>
>>> >>> I'm assuming you're with a slow machine or the machine is a bit
>>> >>> overloaded.
>>> >>>
>>> >>
>>> >> Shouldn't be, I'm not doing anything else right now, and this is a new
>>> MBA.
>>> >>
>>> >>
>>> >>> Can you try with the following patch?
>>> >>>
>>> >>
>>> >> Yes. Will report back.
>>> >>
>>> >>
>>>
>>>
>>>
>>> --
>>> Filipe David Manana,
>>>
>>> "Reasonable men adapt themselves to the world.
>>> Unreasonable men adapt the world to themselves.
>>> That's why all progress depends on unreasonable men."
>>>
>>
>>
--
Filipe David Manana,
"Reasonable men adapt themselves to the world.
Unreasonable men adapt the world to themselves.
That's why all progress depends on unreasonable men."
Re: [Windows] proposed binary build changes
On 22 January 2012 15:49, Dave Cottlehuber wrote:
> On 22 January 2012 13:28, Jeroen Janssen wrote:
>> Hi,
>>
>> Just a quick check since I am not able to get (snappy) compression
>> working on windows.
>
> Hi Jeroen,
>
> Thanks for reporting this. AFAICT the snappy_nif.dll is not being copied
> during
> make dist or install.
>
> IIRC you ran into this last year on master and resulted in COUCHDB-1197 fixes.
> So I'm not sure what is missing on 1.2.x for this, or if its more
> likely an issue with
> my packaging steps.
>
> I'll keep you posted.
>
> BTW compression is on by default so no need to tweak ini files.
>
> A+
> Dave
Hi Jeroen,
While I look for why this is not working in the build, can you copy
snappy_nif.dll [1] into $COUCH/lib/snappy-1.0.3/priv/ and then test
with snappy_test.erl courtesy of Filipe [2]?
3> pwd().
C:/couch/1.2.0a-8d83b39-git_otp_R15B_SDK7.1/bin
ok
4> c("/couch/snappy_tests").
{ok,snappy_tests}
5> snappy_tests:test().
All 2 tests passed.
ok
Your higher-level compression tests should be fine.
[1]: https://www.dropbox.com/s/jeifcxpbtpo78ak/snapshots/20120116?v=l
[2]
https://github.com/fdmanana/snappy-erlang-nif/blob/master/test/snappy_tests.erl
A+
Dave
Re: git commit: Remove dead _all_docs code
Oh that's pretty. Thanks, Bob.
On Sun, Jan 22, 2012 at 06:25, wrote:
> Updated Branches:
> refs/heads/master 6dba2e911 -> d59cdd71b
>
>
> Remove dead _all_docs code
>
>
> Project: http://git-wip-us.apache.org/repos/asf/couchdb/repo
> Commit: http://git-wip-us.apache.org/repos/asf/couchdb/commit/d59cdd71
> Tree: http://git-wip-us.apache.org/repos/asf/couchdb/tree/d59cdd71
> Diff: http://git-wip-us.apache.org/repos/asf/couchdb/diff/d59cdd71
>
> Branch: refs/heads/master
> Commit: d59cdd71b356a454eff36b52bca0c212b2f03984
> Parents: 6dba2e9
> Author: Robert Newson
> Authored: Sun Jan 22 14:07:49 2012 +
> Committer: Robert Newson
> Committed: Sun Jan 22 14:07:49 2012 +
>
> --
> src/couchdb/couch_httpd_db.erl | 143 ---
> 1 files changed, 0 insertions(+), 143 deletions(-)
> --
>
>
> http://git-wip-us.apache.org/repos/asf/couchdb/blob/d59cdd71/src/couchdb/couch_httpd_db.erl
> --
> diff --git a/src/couchdb/couch_httpd_db.erl b/src/couchdb/couch_httpd_db.erl
> index 1bcfeff..f669643 100644
> --- a/src/couchdb/couch_httpd_db.erl
> +++ b/src/couchdb/couch_httpd_db.erl
> @@ -340,26 +340,6 @@
> db_req(#httpd{method='POST',path_parts=[_,<<"_purge">>]}=Req, Db) ->
> db_req(#httpd{path_parts=[_,<<"_purge">>]}=Req, _Db) ->
> send_method_not_allowed(Req, "POST");
>
> -db_req(#httpd{method='GET',path_parts=[_,<<"_all_docs">>]}=Req, Db) ->
> - Keys = couch_httpd:qs_json_value(Req, "keys", nil),
> - all_docs_view(Req, Db, Keys);
> -
> -db_req(#httpd{method='POST',path_parts=[_,<<"_all_docs">>]}=Req, Db) ->
> - couch_httpd:validate_ctype(Req, "application/json"),
> - {Fields} = couch_httpd:json_body_obj(Req),
> - case couch_util:get_value(<<"keys">>, Fields, nil) of
> - nil ->
> - ?LOG_DEBUG("POST to _all_docs with no keys member.", []),
> - all_docs_view(Req, Db, nil);
> - Keys when is_list(Keys) ->
> - all_docs_view(Req, Db, Keys);
> - _ ->
> - throw({bad_request, "`keys` member must be a array."})
> - end;
> -
> -db_req(#httpd{path_parts=[_,<<"_all_docs">>]}=Req, _Db) ->
> - send_method_not_allowed(Req, "GET,HEAD,POST");
> -
> db_req(#httpd{method='POST',path_parts=[_,<<"_missing_revs">>]}=Req, Db) ->
> {JsonDocIdRevs} = couch_httpd:json_body_obj(Req),
> JsonDocIdRevs2 = [{Id, [couch_doc:parse_rev(RevStr) || RevStr <-
> RevStrs]} || {Id, RevStrs} <- JsonDocIdRevs],
> @@ -458,129 +438,6 @@ db_req(#httpd{path_parts=[_, DocId]}=Req, Db) ->
> db_req(#httpd{path_parts=[_, DocId | FileNameParts]}=Req, Db) ->
> db_attachment_req(Req, Db, DocId, FileNameParts).
>
> -all_docs_view(Req, Db, Keys) ->
> - case couch_db:is_system_db(Db) of
> - true ->
> - case (catch couch_db:check_is_admin(Db)) of
> - ok ->
> - do_all_docs_view(Req, Db, Keys);
> - _ ->
> - throw({forbidden, <<"Only admins can access _all_docs",
> - " of system databases.">>})
> - end;
> - false ->
> - do_all_docs_view(Req, Db, Keys)
> - end.
> -
> -do_all_docs_view(Req, Db, Keys) ->
> - RawCollator = fun(A, B) -> A < B end,
> - #view_query_args{
> - start_key = StartKey,
> - start_docid = StartDocId,
> - end_key = EndKey,
> - end_docid = EndDocId,
> - limit = Limit,
> - skip = SkipCount,
> - direction = Dir,
> - inclusive_end = Inclusive
> - } = QueryArgs
> - = couch_httpd_view:parse_view_params(Req, Keys, map, RawCollator),
> - {ok, Info} = couch_db:get_db_info(Db),
> - CurrentEtag = couch_httpd:make_etag(Info),
> - couch_httpd:etag_respond(Req, CurrentEtag, fun() ->
> -
> - TotalRowCount = couch_util:get_value(doc_count, Info),
> - StartId = if is_binary(StartKey) -> StartKey;
> - true -> StartDocId
> - end,
> - EndId = if is_binary(EndKey) -> EndKey;
> - true -> EndDocId
> - end,
> - FoldAccInit = {Limit, SkipCount, undefined, []},
> - UpdateSeq = couch_db:get_update_seq(Db),
> - JsonParams = case couch_httpd:qs_value(Req, "update_seq") of
> - "true" ->
> - [{update_seq, UpdateSeq}];
> - _Else ->
> - []
> - end,
> - case Keys of
> - nil ->
> - FoldlFun = couch_httpd_view:make_view_fold_fun(Req, QueryArgs,
> CurrentEtag, Db, UpdateSeq,
> - TotalRowCount, #view_fold_helper_funs{
> - reduce_count = fun couch_db:enum_docs_reduce_to_count/1,
> - send_row = fun all_docs_send_json_view_row/6
> - }),
> - AdapterFun = fun(#full_doc_info{id=Id}=FullDocInfo, Offset, Acc)
> ->
> - case couch_doc:to_doc_info(FullDocInfo) of
> - #doc_info{revs=[#rev_in
Re: git commit: Fix SpiderMonkey version detection
Dave confirmed. Backported to 1.1.x and 1.2.x. Sorry for the dumb mistake on that one. On Sun, Jan 22, 2012 at 2:47 PM, Paul Davis wrote: > Dave pasted me part of his. I realized that I moved the header check > above the spot where we assign the JS_CPPFLAGS which was dumb of me. > Pushed a fix in 572b561adbf852e08c7397519070f299d0b401e4 to master. > Soon as I have confirmation it's correct I'll backport to 1.2.x and > 1.1.x > > On Sun, Jan 22, 2012 at 2:38 PM, Paul Davis > wrote: >> Most odd since I only added three directories to search for. >> >> Can you all post me a config.log so I can see the compiler options being >> used? >> >> On Sun, Jan 22, 2012 at 1:56 PM, Dave Cottlehuber wrote: >>> On 22 January 2012 19:15, Filipe David Manana wrote: Paul, after this change I'm no longer able to build master (haven't tried other branches however). >>> >>> Ditto. >>> configure can't find js/jsapi.h, this is because my jsapi.h doesn't live inside a js directory. Ubuntu 10.10 jsapi.h full path: /usr/include/xulrunner-1.9.2.24/jsapi.h flags passed to configure: configure --with-js-include=/usr/include/xulrunner-1.9.2.24 --with-js-lib=/usr/lib/xulrunner-1.9.2.24 >>> >>> Windows:: >>> /relax/js-1.8.5/js/src/dist/include/jsapi.h >>> >>> maas@sendai /relax/couchdb >>> $ more /relax/bin/couchdb_config_js185.sh >>> #!/bin/sh >>> COUCH_TOP=`pwd` >>> export COUCH_TOP >>> >>> ./configure \ >>> --with-js-lib=/relax/js-1.8.5/js/src/dist/lib \ >>> --with-js-include=/relax/js-1.8.5/js/src/dist/include \ >>> ... >>> >>> A+ >>> Dave
Re: git commit: Fix SpiderMonkey version detection
Dave pasted me part of his. I realized that I moved the header check above the spot where we assign the JS_CPPFLAGS which was dumb of me. Pushed a fix in 572b561adbf852e08c7397519070f299d0b401e4 to master. Soon as I have confirmation it's correct I'll backport to 1.2.x and 1.1.x On Sun, Jan 22, 2012 at 2:38 PM, Paul Davis wrote: > Most odd since I only added three directories to search for. > > Can you all post me a config.log so I can see the compiler options being used? > > On Sun, Jan 22, 2012 at 1:56 PM, Dave Cottlehuber wrote: >> On 22 January 2012 19:15, Filipe David Manana wrote: >>> Paul, after this change I'm no longer able to build master (haven't >>> tried other branches however). >> >> Ditto. >> >>> configure can't find js/jsapi.h, this is because my jsapi.h doesn't >>> live inside a js directory. >>> >>> Ubuntu 10.10 >>> jsapi.h full path: /usr/include/xulrunner-1.9.2.24/jsapi.h >>> >>> flags passed to configure: >>> >>> configure --with-js-include=/usr/include/xulrunner-1.9.2.24 >>> --with-js-lib=/usr/lib/xulrunner-1.9.2.24 >> >> Windows:: >> /relax/js-1.8.5/js/src/dist/include/jsapi.h >> >> maas@sendai /relax/couchdb >> $ more /relax/bin/couchdb_config_js185.sh >> #!/bin/sh >> COUCH_TOP=`pwd` >> export COUCH_TOP >> >> ./configure \ >> --with-js-lib=/relax/js-1.8.5/js/src/dist/lib \ >> --with-js-include=/relax/js-1.8.5/js/src/dist/include \ >> ... >> >> A+ >> Dave
Re: git commit: Fix SpiderMonkey version detection
Most odd since I only added three directories to search for. Can you all post me a config.log so I can see the compiler options being used? On Sun, Jan 22, 2012 at 1:56 PM, Dave Cottlehuber wrote: > On 22 January 2012 19:15, Filipe David Manana wrote: >> Paul, after this change I'm no longer able to build master (haven't >> tried other branches however). > > Ditto. > >> configure can't find js/jsapi.h, this is because my jsapi.h doesn't >> live inside a js directory. >> >> Ubuntu 10.10 >> jsapi.h full path: /usr/include/xulrunner-1.9.2.24/jsapi.h >> >> flags passed to configure: >> >> configure --with-js-include=/usr/include/xulrunner-1.9.2.24 >> --with-js-lib=/usr/lib/xulrunner-1.9.2.24 > > Windows:: > /relax/js-1.8.5/js/src/dist/include/jsapi.h > > maas@sendai /relax/couchdb > $ more /relax/bin/couchdb_config_js185.sh > #!/bin/sh > COUCH_TOP=`pwd` > export COUCH_TOP > > ./configure \ > --with-js-lib=/relax/js-1.8.5/js/src/dist/lib \ > --with-js-include=/relax/js-1.8.5/js/src/dist/include \ > ... > > A+ > Dave
Re: Do we need 2 entry points for the replication?
On 19 January 2012 22:56, Paul Davis wrote: > On Thu, Jan 19, 2012 at 3:52 PM, Robert Newson wrote: >> I would prefer to see a single /_replicate entrypoint, with, say, >> "persistent":true to indicate that the replication settings should be >> stored. We would also need an API to list all persistent replication >> tasks and one to delete them. Which would look a lot like the >> _replicator database, though much more controlled (no public passwords >> for those jobs that require auth). +1 while I understand *why* we have different APIs at the moment, it is definitely confusing for people. >> I think it's too late, though. There's work on master to fix the >> issues with _replicator now (and the similar ones in _user). While I >> don't like the approach, it does solve the problem. >> > > We can break it eventually and I think we should consider it sooner > rather than later. As Damien observed, sometimes its not until you tried already that you understand the requirements better. ATM we are concerned that the short-term user impact of reverting an API decision is greater than the damage done long-term by having something that *every* future user will struggle with. Either we release new APIs more frequently, perhaps with an experimental tag, to get real-world feedback, or we need to batch these things up for a larger 2.0 release where we revert a lot of existing functionality. Are there other options that help get features to users quickly, and enable tidying up in future? >> Bottom line: It's my opinion that _replicator (and _user) were wrongly >> exposed as full-blooded databases when all we needed to use was the >> database format (and carefully curate API endpoints). But, alas, that >> train has sailed. >> > > I seem to recall someone else with a similar opinion even when these > things were being designed. ;) > > Also, what kind of crazy sailing trains do you brits have over there > and how do I get a ticket to ride on one? http://farm4.staticflickr.com/3008/5840702274_bd17fe8dee_z.jpg A+ Dave
Re: git commit: Fix SpiderMonkey version detection
On 22 January 2012 19:15, Filipe David Manana wrote: > Paul, after this change I'm no longer able to build master (haven't > tried other branches however). Ditto. > configure can't find js/jsapi.h, this is because my jsapi.h doesn't > live inside a js directory. > > Ubuntu 10.10 > jsapi.h full path: /usr/include/xulrunner-1.9.2.24/jsapi.h > > flags passed to configure: > > configure --with-js-include=/usr/include/xulrunner-1.9.2.24 > --with-js-lib=/usr/lib/xulrunner-1.9.2.24 Windows:: /relax/js-1.8.5/js/src/dist/include/jsapi.h maas@sendai /relax/couchdb $ more /relax/bin/couchdb_config_js185.sh #!/bin/sh COUCH_TOP=`pwd` export COUCH_TOP ./configure \ --with-js-lib=/relax/js-1.8.5/js/src/dist/lib \ --with-js-include=/relax/js-1.8.5/js/src/dist/include \ ... A+ Dave
Re: Issues blocking 1.2.0 release
Works. How do we proceed?
On Sun, Jan 22, 2012 at 7:05 PM, Noah Slater wrote:
> OVAR 9000! (Testing now...)
>
>
> On Sun, Jan 22, 2012 at 6:56 PM, Filipe David Manana
> wrote:
>
>> On Sun, Jan 22, 2012 at 6:47 PM, Noah Slater
>> wrote:
>> > No change, still fails.
>>
>> Noah, to try to find out if it's due to slowness of the machine or
>> some other issue, do you think you can try to increase the following
>> timeout in the test?
>>
>> diff --git a/test/etap/242-replication-many-leaves.t
>> b/test/etap/242-replication-many-leaves.t
>> index d8d3eb9..737cd31 100755
>> --- a/test/etap/242-replication-many-leaves.t
>> +++ b/test/etap/242-replication-many-leaves.t
>> @@ -287,6 +287,6 @@ replicate(Source, Target) ->
>> receive
>> {'DOWN', MonRef, process, Pid, Reason} ->
>> etap:is(Reason, normal, "Replication finished successfully")
>> -after 30 ->
>> +after 90 ->
>> etap:bail("Timeout waiting for replication to finish")
>> end.
>>
>> >
>> > On Sun, Jan 22, 2012 at 6:08 PM, Noah Slater
>> wrote:
>> >
>> >>
>> >> On Sun, Jan 22, 2012 at 6:01 PM, Filipe David Manana <
>> fdman...@apache.org>wrote:
>> >>
>> >>> Noah, does it fail occasionally or every time for you?
>> >>>
>> >>
>> >> Fails every time.
>> >>
>> >>
>> >>> I'm assuming you're with a slow machine or the machine is a bit
>> >>> overloaded.
>> >>>
>> >>
>> >> Shouldn't be, I'm not doing anything else right now, and this is a new
>> MBA.
>> >>
>> >>
>> >>> Can you try with the following patch?
>> >>>
>> >>
>> >> Yes. Will report back.
>> >>
>> >>
>>
>>
>>
>> --
>> Filipe David Manana,
>>
>> "Reasonable men adapt themselves to the world.
>> Unreasonable men adapt the world to themselves.
>> That's why all progress depends on unreasonable men."
>>
>
>
Re: Issues blocking 1.2.0 release
OVAR 9000! (Testing now...)
On Sun, Jan 22, 2012 at 6:56 PM, Filipe David Manana wrote:
> On Sun, Jan 22, 2012 at 6:47 PM, Noah Slater wrote:
> > No change, still fails.
>
> Noah, to try to find out if it's due to slowness of the machine or
> some other issue, do you think you can try to increase the following
> timeout in the test?
>
> diff --git a/test/etap/242-replication-many-leaves.t
> b/test/etap/242-replication-many-leaves.t
> index d8d3eb9..737cd31 100755
> --- a/test/etap/242-replication-many-leaves.t
> +++ b/test/etap/242-replication-many-leaves.t
> @@ -287,6 +287,6 @@ replicate(Source, Target) ->
> receive
> {'DOWN', MonRef, process, Pid, Reason} ->
> etap:is(Reason, normal, "Replication finished successfully")
> -after 30 ->
> +after 90 ->
> etap:bail("Timeout waiting for replication to finish")
> end.
>
> >
> > On Sun, Jan 22, 2012 at 6:08 PM, Noah Slater
> wrote:
> >
> >>
> >> On Sun, Jan 22, 2012 at 6:01 PM, Filipe David Manana <
> fdman...@apache.org>wrote:
> >>
> >>> Noah, does it fail occasionally or every time for you?
> >>>
> >>
> >> Fails every time.
> >>
> >>
> >>> I'm assuming you're with a slow machine or the machine is a bit
> >>> overloaded.
> >>>
> >>
> >> Shouldn't be, I'm not doing anything else right now, and this is a new
> MBA.
> >>
> >>
> >>> Can you try with the following patch?
> >>>
> >>
> >> Yes. Will report back.
> >>
> >>
>
>
>
> --
> Filipe David Manana,
>
> "Reasonable men adapt themselves to the world.
> Unreasonable men adapt the world to themselves.
> That's why all progress depends on unreasonable men."
>
Re: Issues blocking 1.2.0 release
On Sun, Jan 22, 2012 at 6:47 PM, Noah Slater wrote:
> No change, still fails.
Noah, to try to find out if it's due to slowness of the machine or
some other issue, do you think you can try to increase the following
timeout in the test?
diff --git a/test/etap/242-replication-many-leaves.t
b/test/etap/242-replication-many-leaves.t
index d8d3eb9..737cd31 100755
--- a/test/etap/242-replication-many-leaves.t
+++ b/test/etap/242-replication-many-leaves.t
@@ -287,6 +287,6 @@ replicate(Source, Target) ->
receive
{'DOWN', MonRef, process, Pid, Reason} ->
etap:is(Reason, normal, "Replication finished successfully")
-after 30 ->
+after 90 ->
etap:bail("Timeout waiting for replication to finish")
end.
>
> On Sun, Jan 22, 2012 at 6:08 PM, Noah Slater wrote:
>
>>
>> On Sun, Jan 22, 2012 at 6:01 PM, Filipe David Manana
>> wrote:
>>
>>> Noah, does it fail occasionally or every time for you?
>>>
>>
>> Fails every time.
>>
>>
>>> I'm assuming you're with a slow machine or the machine is a bit
>>> overloaded.
>>>
>>
>> Shouldn't be, I'm not doing anything else right now, and this is a new MBA.
>>
>>
>>> Can you try with the following patch?
>>>
>>
>> Yes. Will report back.
>>
>>
--
Filipe David Manana,
"Reasonable men adapt themselves to the world.
Unreasonable men adapt the world to themselves.
That's why all progress depends on unreasonable men."
Re: Do we need 2 entry points for the replication?
On Thu, Jan 19, 2012 at 11:46 PM, Jason Smith wrote: > On Fri, Jan 20, 2012 at 9:04 AM, Randall Leeds > wrote: >>> Exposing features as manipulations to normal documents makes CouchDB's >>> API simpler and more orthogonal. >> >> On the other hand, exposing features via special-purpose APIs hides >> the implementation and frees us to change how it works under the hood. > > Thank you for not saying "under the covers." Activity under the > hood--or bonnet--hardly resembles activity under the covers. > > All agree that state is stored in a database. So the question is, have > a database and an API defined as changes to it (perhaps via _show, > _list, and _update); or, have a database and an API defined otherwise. > Either way, you have to bite the bullet and make a breaking change; so > is hiding the implementation a different matter? +1 for hiding the implementation. Who says that those other API endpoints that you would use to manage the replication db will be around in a year? We protect ourselves from the future by hiding the implementation details, thereby not making the same mistake twice. Cheers, -- Sam Bisbee
Re: git commit: Fix SpiderMonkey version detection
same issue here, jsapi.h isn't in a js directory. MBA running Lion
On Jan 22, 2012, at 1:15 PM, Filipe David Manana wrote:
> Paul, after this change I'm no longer able to build master (haven't
> tried other branches however).
>
> configure can't find js/jsapi.h, this is because my jsapi.h doesn't
> live inside a js directory.
>
> Ubuntu 10.10
> jsapi.h full path: /usr/include/xulrunner-1.9.2.24/jsapi.h
>
> flags passed to configure:
>
> configure --with-js-include=/usr/include/xulrunner-1.9.2.24
> --with-js-lib=/usr/lib/xulrunner-1.9.2.24
>
> It's the only spidermonkey version I have installed.
>
> Without this commit, configure doesn't complain and everything works fine.
> Is this a local issue or something missing in the autotools config?
>
> On Sat, Jan 21, 2012 at 10:10 PM, wrote:
>> Updated Branches:
>> refs/heads/master da33e3447 -> 10047e759
>>
>>
>> Fix SpiderMonkey version detection
>>
>> Randall's last patch to only test for JSOPTION_ANONFUNFIX ended up
>> reordering the test before the headers were located. This ran into
>> errors in version detection. This patch reorders the header location as
>> well as adds a few more default search paths when no --with-js-include
>> option is specified to account for newer SpiderMonkeys that puth their
>> headers into $PREFIX/include/js.
>>
>>
>> Project: http://git-wip-us.apache.org/repos/asf/couchdb/repo
>> Commit: http://git-wip-us.apache.org/repos/asf/couchdb/commit/10047e75
>> Tree: http://git-wip-us.apache.org/repos/asf/couchdb/tree/10047e75
>> Diff: http://git-wip-us.apache.org/repos/asf/couchdb/diff/10047e75
>>
>> Branch: refs/heads/master
>> Commit: 10047e75935818e0421bdd9ac96dc21334f90e95
>> Parents: da33e34
>> Author: Paul Joseph Davis
>> Authored: Sat Jan 21 16:08:58 2012 -0600
>> Committer: Paul Joseph Davis
>> Committed: Sat Jan 21 16:08:58 2012 -0600
>>
>> --
>> configure.ac | 41 ++---
>> 1 files changed, 22 insertions(+), 19 deletions(-)
>> --
>>
>>
>> http://git-wip-us.apache.org/repos/asf/couchdb/blob/10047e75/configure.ac
>> --
>> diff --git a/configure.ac b/configure.ac
>> index c6d564a..adfd740 100644
>> --- a/configure.ac
>> +++ b/configure.ac
>> @@ -177,8 +177,11 @@ AS_CASE([$(uname -s)],
>> [CYGWIN*], [] ,
>> [*], [
>> CPPFLAGS="$CPPFLAGS -I/opt/local/include"
>> +CPPFLAGS="$CPPFLAGS -I/opt/local/include/js"
>> CPPFLAGS="$CPPFLAGS -I/usr/local/include"
>> +CPPFLAGS="$CPPFLAGS -I/usr/local/include/js"
>> CPPFLAGS="$CPPFLAGS -I/usr/include"
>> +CPPFLAGS="$CPPFLAGS -I/usr/include/js"
>> LDFLAGS="$LDFLAGS -L/opt/local/lib"
>> LDFLAGS="$LDFLAGS -L/usr/local/lib"
>> ])
>> @@ -203,6 +206,17 @@ AS_CASE([$(uname -s)],
>>
>> AM_CONDITIONAL([WINDOWS], [test x$IS_WINDOWS = xTRUE])
>>
>> +AC_CHECK_HEADER([jsapi.h], [], [
>> +AC_CHECK_HEADER([js/jsapi.h],
>> +[
>> +CPPFLAGS="$CPPFLAGS -I$JS_INCLUDE/js"
>> +],
>> +[
>> +AC_MSG_ERROR([Could not find the jsapi header.
>> +
>> +Are the Mozilla SpiderMonkey headers installed?])
>> +])])
>> +
>> OLD_LIBS="$LIBS"
>> LIBS="$JS_LIBS $LIBS"
>> OLD_CPPFLAGS="$CPPFLAGS"
>> @@ -247,6 +261,14 @@ AC_CHECK_LIB([$JS_LIB_BASE],
>> [JS_GetStringCharsAndLength],
>>
>> # Else, hope that 1.7.0 works
>>
>> +# Deal with JSScript -> JSObject -> JSScript switcheroo
>> +
>> +AC_CHECK_TYPE([JSScript*],
>> +[AC_DEFINE([JSSCRIPT_TYPE], [JSScript*], [Use JSObject* for scripts])],
>> +[AC_DEFINE([JSSCRIPT_TYPE], [JSObject*], [Use JSScript* for scripts])],
>> +[[#include ]]
>> +)
>> +
>> AC_DEFINE([COUCHJS_NAME], ["couchjs"], ["CouchJS executable name."])
>>
>> if test x${IS_WINDOWS} = xTRUE; then
>> @@ -298,25 +320,6 @@ fi
>> JS_LIBS="-l$JS_LIB_BASE -lm $JS_LIBS"
>> AC_SUBST(JS_LIBS)
>>
>> -AC_CHECK_HEADER([jsapi.h], [], [
>> -AC_CHECK_HEADER([js/jsapi.h],
>> -[
>> -CPPFLAGS="$CPPFLAGS -I$JS_INCLUDE/js"
>> -],
>> -[
>> -AC_MSG_ERROR([Could not find the jsapi header.
>> -
>> -Are the Mozilla SpiderMonkey headers installed?])
>> -])])
>> -
>> -# Deal with JSScript -> JSObject -> JSScript switcheroo
>> -
>> -AC_CHECK_TYPE([JSScript*],
>> -[AC_DEFINE([JSSCRIPT_TYPE], [JSScript*], [Use JSObject* for scripts])],
>> -[AC_DEFINE([JSSCRIPT_TYPE], [JSObject*], [Use JSScript* for scripts])],
>> -[[#include ]]
>> -)
>> -
>> LIBS="$OLD_LIBS"
>> CPPFLAGS="$OLD_CPPFLAGS"
>>
>>
>
>
>
> --
> Filipe David Manana,
>
> "Reasonable men adapt themselves to the world.
> Unreasonable men adapt the world to themselves.
> That's why all progress depends on unreasonable men."
Re: Issues blocking 1.2.0 release
No change, still fails. On Sun, Jan 22, 2012 at 6:08 PM, Noah Slater wrote: > > On Sun, Jan 22, 2012 at 6:01 PM, Filipe David Manana > wrote: > >> Noah, does it fail occasionally or every time for you? >> > > Fails every time. > > >> I'm assuming you're with a slow machine or the machine is a bit >> overloaded. >> > > Shouldn't be, I'm not doing anything else right now, and this is a new MBA. > > >> Can you try with the following patch? >> > > Yes. Will report back. > >
Re: Unique instance IDs?
Sorry to bump this old thread, but just going through my backlog. With regard to URLs, I think there is some confusion about the purpose of a URL here. If I write a a cool essay, say, and I stick that up at nslater.org/my-cool-essay, then I can link to it from other places on the web using that address. I might also want to put my cool essay on Dropbox, or post it to Tumblr, or send it in an email. Now my cool essay has lots of URLs. Each one of them perfectly valid. I don't have to go and edit the original copy at nslater.org/my-cool-essay, because I am making copies of it. My cool essay is completely unaware of the URLs that are being used to point to it. And it doesn't care that many URLs point to it. Yes, URLs can be used as identifiers. But when you do this, you tie the thing you're naming to the place you're hosting it. Sometimes that is useful, other times it will cripple you. There is nothing about URLs that requires you to do this. I would hazard a guess that 99% of URLs are de-coupled from the things they point to. WebArch is much more robust when the identity of the object is de-coupled from the URL. Look at Atom, the ID element is supposed to be a URL, but they recommend a non-dereferencable format, precisely to decouple posts from the location you happen to be hosting them this month. Hey, if we're gonna use URLs, maybe we want to go down the same route? http://en.wikipedia.org/wiki/Tag_URI At this point, I'm not sure what they buy us over UUIDs. Thoughts? Thanks, N
Re: git commit: ignore .*
Ah, okay. Thanks. On Tue, Jan 3, 2012 at 5:26 PM, Randall Leeds wrote: > On Tue, Jan 3, 2012 at 08:03, Paul Davis > wrote: > > There was a discussion about it for editor temp files and the like. No > > one objected and it was added quickly cause Gavin said "quick make a > > commit to test buildbot". > > > > I haven't seen any objections so I'd go ahead and remove the entries > > it made obsolete. > > I don't see any obsolete entries. > Most of the entries at the top are "star dot something", but this one > is "dot anything", meaning dotfiles (beginning with the dot). > This is the only line in .gitignore starting with a dot. > > > > > On Sun, Jan 1, 2012 at 5:27 PM, Noah Slater > wrote: > >> Why was this added? I see it's still there. Was it a temporary commit? > If > >> we want this in properly, then the first lot of entries could be > removed. > >> > >> On Wed, Dec 7, 2011 at 10:33 PM, wrote: > >> > >>> Updated Branches: > >>> refs/heads/master 5be3eb3b5 -> 1aafbf4b3 > >>> > >>> > >>> ignore .* > >>> > >>> Great excuse to help gmacdonald check JIRA integration, too. > >>> Fix TEST-1 > >>> > >>> > >>> Project: http://git-wip-us.apache.org/repos/asf/couchdb/repo > >>> Commit: http://git-wip-us.apache.org/repos/asf/couchdb/commit/1aafbf4b > >>> Tree: http://git-wip-us.apache.org/repos/asf/couchdb/tree/1aafbf4b > >>> Diff: http://git-wip-us.apache.org/repos/asf/couchdb/diff/1aafbf4b > >>> > >>> Branch: refs/heads/master > >>> Commit: 1aafbf4b33884b4b63ca7a2790a071fe346b2cc4 > >>> Parents: 5be3eb3 > >>> Author: Randall Leeds > >>> Authored: Wed Dec 7 14:32:24 2011 -0800 > >>> Committer: Randall Leeds > >>> Committed: Wed Dec 7 14:32:27 2011 -0800 > >>> > >>> -- > >>> .gitignore |2 ++ > >>> 1 files changed, 2 insertions(+), 0 deletions(-) > >>> -- > >>> > >>> > >>> > http://git-wip-us.apache.org/repos/asf/couchdb/blob/1aafbf4b/.gitignore > >>> -- > >>> diff --git a/.gitignore b/.gitignore > >>> index f0ebdef..a24f0a1 100644 > >>> --- a/.gitignore > >>> +++ b/.gitignore > >>> @@ -15,6 +15,8 @@ configure > >>> autom4te.cache > >>> build-aux > >>> *.diff > >>> +!.gitignore > >>> +.* > >>> > >>> # ./configure > >>> > >>> > >>> >
Re: git commit: Fix SpiderMonkey version detection
Paul, after this change I'm no longer able to build master (haven't
tried other branches however).
configure can't find js/jsapi.h, this is because my jsapi.h doesn't
live inside a js directory.
Ubuntu 10.10
jsapi.h full path: /usr/include/xulrunner-1.9.2.24/jsapi.h
flags passed to configure:
configure --with-js-include=/usr/include/xulrunner-1.9.2.24
--with-js-lib=/usr/lib/xulrunner-1.9.2.24
It's the only spidermonkey version I have installed.
Without this commit, configure doesn't complain and everything works fine.
Is this a local issue or something missing in the autotools config?
On Sat, Jan 21, 2012 at 10:10 PM, wrote:
> Updated Branches:
> refs/heads/master da33e3447 -> 10047e759
>
>
> Fix SpiderMonkey version detection
>
> Randall's last patch to only test for JSOPTION_ANONFUNFIX ended up
> reordering the test before the headers were located. This ran into
> errors in version detection. This patch reorders the header location as
> well as adds a few more default search paths when no --with-js-include
> option is specified to account for newer SpiderMonkeys that puth their
> headers into $PREFIX/include/js.
>
>
> Project: http://git-wip-us.apache.org/repos/asf/couchdb/repo
> Commit: http://git-wip-us.apache.org/repos/asf/couchdb/commit/10047e75
> Tree: http://git-wip-us.apache.org/repos/asf/couchdb/tree/10047e75
> Diff: http://git-wip-us.apache.org/repos/asf/couchdb/diff/10047e75
>
> Branch: refs/heads/master
> Commit: 10047e75935818e0421bdd9ac96dc21334f90e95
> Parents: da33e34
> Author: Paul Joseph Davis
> Authored: Sat Jan 21 16:08:58 2012 -0600
> Committer: Paul Joseph Davis
> Committed: Sat Jan 21 16:08:58 2012 -0600
>
> --
> configure.ac | 41 ++---
> 1 files changed, 22 insertions(+), 19 deletions(-)
> --
>
>
> http://git-wip-us.apache.org/repos/asf/couchdb/blob/10047e75/configure.ac
> --
> diff --git a/configure.ac b/configure.ac
> index c6d564a..adfd740 100644
> --- a/configure.ac
> +++ b/configure.ac
> @@ -177,8 +177,11 @@ AS_CASE([$(uname -s)],
> [CYGWIN*], [] ,
> [*], [
> CPPFLAGS="$CPPFLAGS -I/opt/local/include"
> + CPPFLAGS="$CPPFLAGS -I/opt/local/include/js"
> CPPFLAGS="$CPPFLAGS -I/usr/local/include"
> + CPPFLAGS="$CPPFLAGS -I/usr/local/include/js"
> CPPFLAGS="$CPPFLAGS -I/usr/include"
> + CPPFLAGS="$CPPFLAGS -I/usr/include/js"
> LDFLAGS="$LDFLAGS -L/opt/local/lib"
> LDFLAGS="$LDFLAGS -L/usr/local/lib"
> ])
> @@ -203,6 +206,17 @@ AS_CASE([$(uname -s)],
>
> AM_CONDITIONAL([WINDOWS], [test x$IS_WINDOWS = xTRUE])
>
> +AC_CHECK_HEADER([jsapi.h], [], [
> + AC_CHECK_HEADER([js/jsapi.h],
> + [
> + CPPFLAGS="$CPPFLAGS -I$JS_INCLUDE/js"
> + ],
> + [
> + AC_MSG_ERROR([Could not find the jsapi header.
> +
> +Are the Mozilla SpiderMonkey headers installed?])
> + ])])
> +
> OLD_LIBS="$LIBS"
> LIBS="$JS_LIBS $LIBS"
> OLD_CPPFLAGS="$CPPFLAGS"
> @@ -247,6 +261,14 @@ AC_CHECK_LIB([$JS_LIB_BASE],
> [JS_GetStringCharsAndLength],
>
> # Else, hope that 1.7.0 works
>
> +# Deal with JSScript -> JSObject -> JSScript switcheroo
> +
> +AC_CHECK_TYPE([JSScript*],
> + [AC_DEFINE([JSSCRIPT_TYPE], [JSScript*], [Use JSObject* for scripts])],
> + [AC_DEFINE([JSSCRIPT_TYPE], [JSObject*], [Use JSScript* for scripts])],
> + [[#include ]]
> +)
> +
> AC_DEFINE([COUCHJS_NAME], ["couchjs"], ["CouchJS executable name."])
>
> if test x${IS_WINDOWS} = xTRUE; then
> @@ -298,25 +320,6 @@ fi
> JS_LIBS="-l$JS_LIB_BASE -lm $JS_LIBS"
> AC_SUBST(JS_LIBS)
>
> -AC_CHECK_HEADER([jsapi.h], [], [
> - AC_CHECK_HEADER([js/jsapi.h],
> - [
> - CPPFLAGS="$CPPFLAGS -I$JS_INCLUDE/js"
> - ],
> - [
> - AC_MSG_ERROR([Could not find the jsapi header.
> -
> -Are the Mozilla SpiderMonkey headers installed?])
> - ])])
> -
> -# Deal with JSScript -> JSObject -> JSScript switcheroo
> -
> -AC_CHECK_TYPE([JSScript*],
> - [AC_DEFINE([JSSCRIPT_TYPE], [JSScript*], [Use JSObject* for scripts])],
> - [AC_DEFINE([JSSCRIPT_TYPE], [JSObject*], [Use JSScript* for scripts])],
> - [[#include ]]
> -)
> -
> LIBS="$OLD_LIBS"
> CPPFLAGS="$OLD_CPPFLAGS"
>
>
--
Filipe David Manana,
"Reasonable men adapt themselves to the world.
Unreasonable men adapt the world to themselves.
That's why all progress depends on unreasonable men."
Re: Issues blocking 1.2.0 release
On Sun, Jan 22, 2012 at 6:01 PM, Filipe David Manana wrote: > Noah, does it fail occasionally or every time for you? > Fails every time. > I'm assuming you're with a slow machine or the machine is a bit overloaded. > Shouldn't be, I'm not doing anything else right now, and this is a new MBA. > Can you try with the following patch? > Yes. Will report back.
Re: Issues blocking 1.2.0 release
Noah, does it fail occasionally or every time for you?
I'm assuming you're with a slow machine or the machine is a bit overloaded.
Can you try with the following patch?
diff --git a/test/etap/242-replication-many-leaves.t
b/test/etap/242-replication-many-leaves.t
index d8d3eb9..ad9d180 100755
--- a/test/etap/242-replication-many-leaves.t
+++ b/test/etap/242-replication-many-leaves.t
@@ -77,6 +77,7 @@ test() ->
couch_server_sup:start_link(test_util:config_files()),
ibrowse:start(),
crypto:start(),
+couch_config:set("replicator", "worker_processes", "1", false),
Pairs = [
{source_db_name(), target_db_name()},
On Sun, Jan 22, 2012 at 5:41 PM, Noah Slater wrote:
> Hello,
>
> The following test fails:
>
> ./test/etap/run test/etap/242-replication-many-leaves.t
>
>
> Log is:
>
> ok 38 - Document revisions updated with 2 attachments
> # Triggering replication again
> [info] [<0.3567.0>] 127.0.0.1 - - HEAD /couch_test_rep_db_b/ 200
> [info] [<0.3567.0>] 127.0.0.1 - - GET /couch_test_rep_db_b/ 200
> [info] [<0.3567.0>] 127.0.0.1 - - GET
> /couch_test_rep_db_b/_local/4296cc7705ed9d0c0cf63593b42a10b7 200
> [info] [<0.6460.0>] Replication `"4296cc7705ed9d0c0cf63593b42a10b7"` is
> using:
> 4 worker processes
> a worker batch size of 500
> 20 HTTP connections
> a connection timeout of 3 milliseconds
> 10 retries per request
> socket options are: [{keepalive,true},{nodelay,false}]
> source start sequence 6
> [info] [<0.2.0>] starting new replication
> `4296cc7705ed9d0c0cf63593b42a10b7` at <0.6460.0> (`couch_test_rep_db_a` -> `
> http://127.0.0.1:53314/couch_test_rep_db_b/`)
> [info] [<0.3573.0>] 127.0.0.1 - - POST /couch_test_rep_db_b/_revs_diff 200
> [info] [<0.3569.0>] 127.0.0.1 - - POST /couch_test_rep_db_b/_revs_diff 200
> [info] [<0.3573.0>] 127.0.0.1 - - POST /couch_test_rep_db_b/_bulk_docs 201
> [info] [<0.3567.0>] 127.0.0.1 - - POST /couch_test_rep_db_b/_revs_diff 200
> [info] [<0.3567.0>] 127.0.0.1 - - POST /couch_test_rep_db_b/_bulk_docs 201
> [info] [<0.6468.0>] Retrying POST request to
> http://127.0.0.1:53314/couch_test_rep_db_b/_bulk_docs in 0.25 seconds due
> to error req_timedout
> [info] [<0.3573.0>] 127.0.0.1 - - POST /couch_test_rep_db_b/_bulk_docs 201
> [info] [<0.6468.0>] Retrying POST request to
> http://127.0.0.1:53314/couch_test_rep_db_b/_bulk_docs in 0.5 seconds due to
> error req_timedout
> [info] [<0.3568.0>] 127.0.0.1 - - POST /couch_test_rep_db_b/_bulk_docs 201
> [info] [<0.6468.0>] Retrying POST request to
> http://127.0.0.1:53314/couch_test_rep_db_b/_bulk_docs in 1.0 seconds due to
> error req_timedout
> [info] [<0.3571.0>] 127.0.0.1 - - POST /couch_test_rep_db_b/_bulk_docs 201
> [info] [<0.6468.0>] Retrying POST request to
> http://127.0.0.1:53314/couch_test_rep_db_b/_bulk_docs in 2.0 seconds due to
> error req_timedout
> [info] [<0.6463.0>] 127.0.0.1 - - POST /couch_test_rep_db_b/_bulk_docs 201
> [info] [<0.6468.0>] Retrying POST request to
> http://127.0.0.1:53314/couch_test_rep_db_b/_bulk_docs in 4.0 seconds due to
> error req_timedout
> [info] [<0.8537.0>] 127.0.0.1 - - POST /couch_test_rep_db_b/_bulk_docs 201
> [info] [<0.6468.0>] Retrying POST request to
> http://127.0.0.1:53314/couch_test_rep_db_b/_bulk_docs in 8.0 seconds due to
> error req_timedout
> [info] [<0.9151.0>] 127.0.0.1 - - POST /couch_test_rep_db_b/_bulk_docs 201
> [info] [<0.6468.0>] Retrying POST request to
> http://127.0.0.1:53314/couch_test_rep_db_b/_bulk_docs in 16.0 seconds due
> to error req_timedout
> [info] [<0.9768.0>] 127.0.0.1 - - POST /couch_test_rep_db_b/_bulk_docs 201
> [info] [<0.6468.0>] Retrying POST request to
> http://127.0.0.1:53314/couch_test_rep_db_b/_bulk_docs in 32.0 seconds due
> to error req_timedout
> [info] [<0.10390.0>] 127.0.0.1 - - POST /couch_test_rep_db_b/_bulk_docs 201
> Bail out! Timeout waiting for replication to finish
> escript: exception error: bad argument
> in function etap:diag/1
> in call from erl_eval:do_apply/5
> in call from erl_eval:exprs/5
> in call from escript:eval_exprs/5
> in call from erl_eval:local_func/5
> in call from escript:interpret/4
> in call from escript:start/1
> in call from init:start_it/1
>
>
> Please advise.
>
> Thanks,
>
> N
--
Filipe David Manana,
"Reasonable men adapt themselves to the world.
Unreasonable men adapt the world to themselves.
That's why all progress depends on unreasonable men."
dev@couchdb.apache.org
As long as the source distribution can build without needing an internet connection, I'll be happy. On Sat, Jan 7, 2012 at 2:25 PM, Benoit Chesneau wrote: > Hi all, > > I would like to start some work on my own to test how I can merge our > autotool build system and rebar to provide an erlang release and > distribute couchdb more easily. In refuge I consider that all the apps > in src/ are standalone app and each have their own history (old > couchdb history is kept) . To create a release, the make distdir > collect all the deps and in one source dir and the archive is created > from that source directory. At the end the created release is the same > you have with couchdb and you don't need to have git installed to > build it. Packages created (deb, rpm & osx) are based on this source > release. > > Can we do the same in the apache project ? Ie having one git repo / > app ? Or do we still need to handle them in the same repo ? Just > wanted to ask to know how I should handle that or at least start the > integration of rebar. > > Let me know, > > - benoît >
Re: [Windows] proposed binary build changes
This all sounds great, Dave. Once we get the source distribution working for 1.2.x, I'd like to co-ordinate with you to get the Windows build prepped at the same time. We should aim to have you uploading the .exe, .exe.md5, and .exe.sha files to your p.a.o space, and calling a vote on the artefacts like we do for the source artefacts. We can then announce both distributions at the same time. Thanks, On Sun, Jan 22, 2012 at 2:49 PM, Dave Cottlehuber wrote: > On 22 January 2012 13:28, Jeroen Janssen wrote: > > Hi, > > > > Just a quick check since I am not able to get (snappy) compression > > working on windows. > > Hi Jeroen, > > Thanks for reporting this. AFAICT the snappy_nif.dll is not being copied > during > make dist or install. > > IIRC you ran into this last year on master and resulted in COUCHDB-1197 > fixes. > So I'm not sure what is missing on 1.2.x for this, or if its more > likely an issue with > my packaging steps. > > I'll keep you posted. > > BTW compression is on by default so no need to tweak ini files. > > A+ > Dave >
Issues blocking 1.2.0 release
Hello,
The following test fails:
./test/etap/run test/etap/242-replication-many-leaves.t
Log is:
ok 38 - Document revisions updated with 2 attachments
# Triggering replication again
[info] [<0.3567.0>] 127.0.0.1 - - HEAD /couch_test_rep_db_b/ 200
[info] [<0.3567.0>] 127.0.0.1 - - GET /couch_test_rep_db_b/ 200
[info] [<0.3567.0>] 127.0.0.1 - - GET
/couch_test_rep_db_b/_local/4296cc7705ed9d0c0cf63593b42a10b7 200
[info] [<0.6460.0>] Replication `"4296cc7705ed9d0c0cf63593b42a10b7"` is
using:
4 worker processes
a worker batch size of 500
20 HTTP connections
a connection timeout of 3 milliseconds
10 retries per request
socket options are: [{keepalive,true},{nodelay,false}]
source start sequence 6
[info] [<0.2.0>] starting new replication
`4296cc7705ed9d0c0cf63593b42a10b7` at <0.6460.0> (`couch_test_rep_db_a` -> `
http://127.0.0.1:53314/couch_test_rep_db_b/`)
[info] [<0.3573.0>] 127.0.0.1 - - POST /couch_test_rep_db_b/_revs_diff 200
[info] [<0.3569.0>] 127.0.0.1 - - POST /couch_test_rep_db_b/_revs_diff 200
[info] [<0.3573.0>] 127.0.0.1 - - POST /couch_test_rep_db_b/_bulk_docs 201
[info] [<0.3567.0>] 127.0.0.1 - - POST /couch_test_rep_db_b/_revs_diff 200
[info] [<0.3567.0>] 127.0.0.1 - - POST /couch_test_rep_db_b/_bulk_docs 201
[info] [<0.6468.0>] Retrying POST request to
http://127.0.0.1:53314/couch_test_rep_db_b/_bulk_docs in 0.25 seconds due
to error req_timedout
[info] [<0.3573.0>] 127.0.0.1 - - POST /couch_test_rep_db_b/_bulk_docs 201
[info] [<0.6468.0>] Retrying POST request to
http://127.0.0.1:53314/couch_test_rep_db_b/_bulk_docs in 0.5 seconds due to
error req_timedout
[info] [<0.3568.0>] 127.0.0.1 - - POST /couch_test_rep_db_b/_bulk_docs 201
[info] [<0.6468.0>] Retrying POST request to
http://127.0.0.1:53314/couch_test_rep_db_b/_bulk_docs in 1.0 seconds due to
error req_timedout
[info] [<0.3571.0>] 127.0.0.1 - - POST /couch_test_rep_db_b/_bulk_docs 201
[info] [<0.6468.0>] Retrying POST request to
http://127.0.0.1:53314/couch_test_rep_db_b/_bulk_docs in 2.0 seconds due to
error req_timedout
[info] [<0.6463.0>] 127.0.0.1 - - POST /couch_test_rep_db_b/_bulk_docs 201
[info] [<0.6468.0>] Retrying POST request to
http://127.0.0.1:53314/couch_test_rep_db_b/_bulk_docs in 4.0 seconds due to
error req_timedout
[info] [<0.8537.0>] 127.0.0.1 - - POST /couch_test_rep_db_b/_bulk_docs 201
[info] [<0.6468.0>] Retrying POST request to
http://127.0.0.1:53314/couch_test_rep_db_b/_bulk_docs in 8.0 seconds due to
error req_timedout
[info] [<0.9151.0>] 127.0.0.1 - - POST /couch_test_rep_db_b/_bulk_docs 201
[info] [<0.6468.0>] Retrying POST request to
http://127.0.0.1:53314/couch_test_rep_db_b/_bulk_docs in 16.0 seconds due
to error req_timedout
[info] [<0.9768.0>] 127.0.0.1 - - POST /couch_test_rep_db_b/_bulk_docs 201
[info] [<0.6468.0>] Retrying POST request to
http://127.0.0.1:53314/couch_test_rep_db_b/_bulk_docs in 32.0 seconds due
to error req_timedout
[info] [<0.10390.0>] 127.0.0.1 - - POST /couch_test_rep_db_b/_bulk_docs 201
Bail out! Timeout waiting for replication to finish
escript: exception error: bad argument
in function etap:diag/1
in call from erl_eval:do_apply/5
in call from erl_eval:exprs/5
in call from escript:eval_exprs/5
in call from erl_eval:local_func/5
in call from escript:interpret/4
in call from escript:start/1
in call from init:start_it/1
Please advise.
Thanks,
N
Re: [Windows] proposed binary build changes
On 22 January 2012 13:28, Jeroen Janssen wrote: > Hi, > > Just a quick check since I am not able to get (snappy) compression > working on windows. Hi Jeroen, Thanks for reporting this. AFAICT the snappy_nif.dll is not being copied during make dist or install. IIRC you ran into this last year on master and resulted in COUCHDB-1197 fixes. So I'm not sure what is missing on 1.2.x for this, or if its more likely an issue with my packaging steps. I'll keep you posted. BTW compression is on by default so no need to tweak ini files. A+ Dave
[jira] [Commented] (COUCHDB-1342) Asynchronous file writes
[ https://issues.apache.org/jira/browse/COUCHDB-1342?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13190665#comment-13190665 ] Filipe Manana commented on COUCHDB-1342: I really appreciate Randall's pushing for collaboration rather than expect a single person to do it all or let this fall into oblivion. I will do some updates to the branch soon as well repost some performance benchmarks, and instructions how to reproduce them as usual, in comparison to latest master (the results posted months ago don't account for many improvements that came after such as COUCHDB-1334). > Asynchronous file writes > > > Key: COUCHDB-1342 > URL: https://issues.apache.org/jira/browse/COUCHDB-1342 > Project: CouchDB > Issue Type: Improvement > Components: Database Core >Reporter: Jan Lehnardt > Fix For: 1.3 > > Attachments: COUCHDB-1342.patch > > > This change updates the file module so that it can do > asynchronous writes. Basically it replies immediately > to process asking to write something to the file, with > the position where the chunks will be written to the > file, while a dedicated child process keeps collecting > chunks and write them to the file (and batching them > when possible). After issuing a series of write request > to the file module, the caller can call its 'flush' > function which will block the caller until all the > chunks it requested to write are effectively written > to the file. > This maximizes the IO subsystem, as for example, while > the updater is traversing and modifying the btrees and > doing CPU bound tasks, the writes are happening in > parallel. > Originally described at http://s.apache.org/TVu > Github Commit: > https://github.com/fdmanana/couchdb/commit/e82a673f119b82dddf674ac2e6233cd78c123554 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (COUCHDB-1384) File descriptor leak if view compaction is cancelled
[ https://issues.apache.org/jira/browse/COUCHDB-1384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Filipe Manana resolved COUCHDB-1384. Resolution: Fixed Applied to 1.2.x > File descriptor leak if view compaction is cancelled > > > Key: COUCHDB-1384 > URL: https://issues.apache.org/jira/browse/COUCHDB-1384 > Project: CouchDB > Issue Type: Bug >Affects Versions: 1.2 >Reporter: Filipe Manana >Priority: Critical > Fix For: 1.2 > > Attachments: > 0001-Close-view-compaction-file-when-compaction-is-cancel.patch > > > If a view compaction is canceled, the compact file's fd remains open as long > as the view group is alive. > This is because the couch_file the compactor uses is spawn_linked by the view > group and not linked to the compactor process, therefore when the compactor > is shutdown the corresponding couch_file is not shutdown. The view group > doesn't keep track of the compact file's couch_file, so it can't explicitly > shutdown it either. > This affects only the 1.2.x branch and is addressed by simply linking the > file process to the compactor process. Patch attached. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
How are key values stored in B+ tree nodes on-disk?
In my ever-growing need to ask the most esoteric file-format questions possible, I am curious if anyone can point me in the direction of the answer here... Background - Consider the _id and _seq B+ indices whose change-paths are appended to the end of the Couch data file after every update. Each of the nodes written have a basic structure of [key, value] where the "key" is the _id or _seq value being indexed, and the "value" is an offset pointer to the next node in the tree to seek to (or in the case of a leaf, an offset directly into the datafile where the updated record lives). General --- Efficiently searching the keys stored in a node of a B+ tree after it has been paged in require (let's assume) that the values are in sorted order; then the keys can be binary-searched to find the next offset or block ID on-disk to jump to. To be able to efficiently jump around key values like that, they must all be of the same size. For example, let's say we store 133 keys per node (assume it is full). That means our first comparison after paging in that block should be at the 67th key. The only way I can jump to that position is to know every key value is say 16 bytes and then jump forward 67 * 16 = 1072 bytes before decoding the key from raw bytes and comparing. Question - Given the massive potential for document _id values to vary in size, it seems impossible to have a hard-coded "key size" that couch utilizes here. Even something sufficiently large like 32 or 64 bytes would still run into situations where a longer key was being indexed or a *very short* key is being indexed and there are multiple magnitudes of wasted space in the index. The few solutions I've found to variable-length keys online require a sequential processing of the keys in a node because the keys will be written like: [key length][key data][offset] This is certainly doable, but less than optimal. I am curious how CouchDB handles this? Thank you for any pointers. -R
[jira] [Commented] (COUCHDB-1342) Asynchronous file writes
[ https://issues.apache.org/jira/browse/COUCHDB-1342?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13190660#comment-13190660 ] Bob Dionne commented on COUCHDB-1342: - I should add that I'd like to get Filipe's opinion on this as this was a collaborative effort > Asynchronous file writes > > > Key: COUCHDB-1342 > URL: https://issues.apache.org/jira/browse/COUCHDB-1342 > Project: CouchDB > Issue Type: Improvement > Components: Database Core >Reporter: Jan Lehnardt > Fix For: 1.3 > > Attachments: COUCHDB-1342.patch > > > This change updates the file module so that it can do > asynchronous writes. Basically it replies immediately > to process asking to write something to the file, with > the position where the chunks will be written to the > file, while a dedicated child process keeps collecting > chunks and write them to the file (and batching them > when possible). After issuing a series of write request > to the file module, the caller can call its 'flush' > function which will block the caller until all the > chunks it requested to write are effectively written > to the file. > This maximizes the IO subsystem, as for example, while > the updater is traversing and modifying the btrees and > doing CPU bound tasks, the writes are happening in > parallel. > Originally described at http://s.apache.org/TVu > Github Commit: > https://github.com/fdmanana/couchdb/commit/e82a673f119b82dddf674ac2e6233cd78c123554 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: [Windows] proposed binary build changes
Hi, Just a quick check since I am not able to get (snappy) compression working on windows. Steps I did: *) install couchdb 1.2.0a from [5] (disabled service/autostart) *) edit local.ini add "file_compression = snappy" to the [couchdb] section *) copied an old (800Mb) 1.0.1 database to var\lib\couchdb *) start couchdb\bin\couchdb.bat *) run compaction on the old database. still 800Mb size additional test I did was to: *) create a new database *) created a 200kb document and saved *) database file is ~200kb in size similar results with "file_compression = deflate_9" Should I do additional steps to enable compression? Can anyone else confirm this behaviour on windows? Thanks in advance, Jeroen On Tue, Jan 17, 2012 at 8:16 PM, Dave Cottlehuber wrote: > Hi folks, > > Quick questions followed by a long update. > > 1. Does anybody rely on the compiler that I use to build CouchDB? > I'm considering dropping VS2008/SDK7.0 builds in favour of the newer > VS2010/SDK7.1, mainly because I don't really have the space to build both. > Although feel free to send me a 500GiB SSD… > > 2. Unless there are strenuous objections, I'm proposing to bump the > Windows binaries and dependencies to enable that, details below. You can > test a variety of 1.1.1 combos if this is important to you [1]. > > 3. Any further volunteers for testing & voting on windows binary builds? > - Kevin Coombes > - Nick North > - Ryan Ramage > - pblakez > - Dirkjan > > It should take ~ 1h per release & I'm not expecting a commitment from you! > > 4. Feedback is hard to come by so, I'd love some use cases, success stories, > and even the occasional abject failure for Couch on Windows. > > Some background details follow, mainly to ensure we have these > archived on the list. > > HISTORY: > Up until Nov 2011, the official binary release from Erlang/OTP team was > done with VS2005 / VC++6. > > However Erlang & CouchDB require ICU, OpenSSL, wxWidgets, Mozilla > SpiderMonkey and optionally cURL, none of which compile cleanly under > VS2005 these days. > > So I've been rolling my own using VS2008 / SDK 7.0, which is not so > difficult now, and has also enabled me to provide fixes for uncommitted > upstream patches where necessary. > > To work around some of the above, I've built binaries against a single, > very specific release of the vcredist_x86.exe package. Thanks to some > advice from Pierre at Microsoft, I can now remove this requirement. > This should simplify requirements for anybody who needs to repackage. > He's also given excellent suggestions on improving the performance of > Apache CouchDB on Windows, and sharing building the dependencies > with projects like PHP in future. Let's hope this progresses further. > > TODAY: > Erlang/OTP have a VS2010 set of binaries available, with current > components, and no outstanding critical bugs as yet. R15B has error > messages with line numbers which is an immense help for troubleshooting. > > The build scripts I use are now largely compatible between SDKs, so it's > possible to create SDK 7.0 and 7.1 versions when I merge & document it > properly. Thanks Wohali for the suggestion & some help along the way! > > LIBRARIES INCLUDED IN THE BUILD: > Unless there are strenuous objections, I'm proposing to move from/to: > > Compiler: > VS2008/ SDK 7.0 -> VS2010 / SDK 7.1 compiler & Visual C++ runtime > > Erlang/OTP R15B > - wxWidgets 2.8.11 -> 2.8.12 > - OpenSSL 1.0.0e -> 0.9.8r to align with OTP team's release > > For CouchDB 1.2.0 and the future: > - ICU 4.4.2 -> 4.6.1 as it compiles cleanly under both SDKs. > - cURL 7.23.1 -> no change > - SpiderMonkey 1.8.0-rc1 -> 1.8.5 > > This sets us up nicely for providing a much simpler build process, > perhaps without needing cygwin at all, just common Windows tools. > > I don't see any issues with this in the last month of testing, YMMV! > > The artefacts used to build these (& their debug symbols) are all > available [4], and there's a build off the 1.2.x branch including these [5]. > > FUTURE: > Feel free to add but this is where I'm thinking of going, in no particular > order: > - Makefile target for a trimmed OTP + Couch build & upx compression > - MSI support (thanks in advance to Wohali!!) > - add a GeoCouchified version with erica + kanso support built in > - performance improvements c/- Microsoft's PGO [2] > - simpler build process perhaps using Mozilla tool chain only > - stabilise my glazier build scripts [3] > - use Windows-approved locations for datafiles > - aim for Microsoft Certification - good general CouchDB publicity > - enable JS and ETAP test framework on Windows > - look at 64 bit builds in June or July once OTP end has settled down > - move at least the .sha and .md5 files onto apache.org pages to > ensure people have a trusted source of binaries. > > Feel free to prioritise this, or add to it. > > [1]: https://www.dropbox.com/s/jeifcxpbtpo78ak/snapshots/20120117?v=l > [2]: http://msdn.micro
[jira] [Commented] (COUCHDB-1342) Asynchronous file writes
[ https://issues.apache.org/jira/browse/COUCHDB-1342?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13190655#comment-13190655 ] Bob Dionne commented on COUCHDB-1342: - I revisited this a bit this morning. I tried to rebase master just to see how far it's moved, it's not too bad, couch_db had some conflicts mostly relating to a new field #db.updater_fd Given Damien's abandonment of the project I'm not sure whether we should push on this or not. I suppose it's worth cleaning up and using if in fact the improvements are substantial > Asynchronous file writes > > > Key: COUCHDB-1342 > URL: https://issues.apache.org/jira/browse/COUCHDB-1342 > Project: CouchDB > Issue Type: Improvement > Components: Database Core >Reporter: Jan Lehnardt > Fix For: 1.3 > > Attachments: COUCHDB-1342.patch > > > This change updates the file module so that it can do > asynchronous writes. Basically it replies immediately > to process asking to write something to the file, with > the position where the chunks will be written to the > file, while a dedicated child process keeps collecting > chunks and write them to the file (and batching them > when possible). After issuing a series of write request > to the file module, the caller can call its 'flush' > function which will block the caller until all the > chunks it requested to write are effectively written > to the file. > This maximizes the IO subsystem, as for example, while > the updater is traversing and modifying the btrees and > doing CPU bound tasks, the writes are happening in > parallel. > Originally described at http://s.apache.org/TVu > Github Commit: > https://github.com/fdmanana/couchdb/commit/e82a673f119b82dddf674ac2e6233cd78c123554 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (COUCHDB-687) Add md5 hash to _attachments properties for documents
[
https://issues.apache.org/jira/browse/COUCHDB-687?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Robert Newson resolved COUCHDB-687.
---
Resolution: Fixed
Fix Version/s: (was: 1.3)
(was: 2.0)
1.1
This happened in COUCHDB-1173
> Add md5 hash to _attachments properties for documents
> -
>
> Key: COUCHDB-687
> URL: https://issues.apache.org/jira/browse/COUCHDB-687
> Project: CouchDB
> Issue Type: Improvement
> Environment: CouchDB
>Reporter: mikeal
>Assignee: Filipe Manana
> Fix For: 1.1
>
> Attachments: couchdb-md5-in-attachment-COUCHDB-687-v2.patch,
> couchdb-md5-in-attachment-COUCHDB-687-v3.patch,
> couchdb-md5-in-attachment-COUCHDB-687.patch, md5.patch
>
>
> The current attachment information looks like this:
> GET /dbname/docid
> "_attachments": {
> "jquery-1.4.1.min.js": {
> "content_type": "text/javascript"
> "revpos": 138
> "length": 70844
> "stub": true
> }
> }
> If a client wanted to sync local files as attachments with a document it
> would not currently be able to do so without keeping a local store of the
> revpos. If this information included an md5 hash of the attachment clients
> could compare it against a hash of the local file to see if they match.
> -Mikeal
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
