[couchdb] branch opentracing deleted (was aa359ee)
This is an automated email from the ASF dual-hosted git repository. iilyak pushed a change to branch opentracing in repository https://gitbox.apache.org/repos/asf/couchdb.git. was aa359ee Remove compiler warning The revisions that were on this branch are still contained in other references; therefore, this change does not discard any commits from the repository.
[couchdb] branch jenkins-opentracing created (now aa359ee)
This is an automated email from the ASF dual-hosted git repository. iilyak pushed a change to branch jenkins-opentracing in repository https://gitbox.apache.org/repos/asf/couchdb.git. at aa359ee Remove compiler warning No new revisions were added by this update.
[couchdb] branch jenkins-opentracing deleted (was aa359ee)
This is an automated email from the ASF dual-hosted git repository. iilyak pushed a change to branch jenkins-opentracing in repository https://gitbox.apache.org/repos/asf/couchdb.git. was aa359ee Remove compiler warning The revisions that were on this branch are still contained in other references; therefore, this change does not discard any commits from the repository.
[couchdb-documentation] branch master updated (8aeeaa7 -> d85235b)
This is an automated email from the ASF dual-hosted git repository. iilyak pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/couchdb-documentation.git. from 8aeeaa7 Remove delayed commits (#445) new 4fa6ebb Remove 417 response codes new 982f335 Merge branch 'master' into patch-1 new d85235b Merge pull request #447 from vmatyusGitHub/patch-1 The 1004 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: src/api/database/bulk-api.rst | 19 +-- 1 file changed, 9 insertions(+), 10 deletions(-)
[couchdb] branch master updated: Return headers from _changes feed when there are no changes
This is an automated email from the ASF dual-hosted git repository.
iilyak pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/couchdb.git
The following commit(s) were added to refs/heads/master by this push:
new abe586e Return headers from _changes feed when there are no changes
new 6382374 Merge pull request #2240 from
cloudant/issue/985-continious-feed-blocking
abe586e is described below
commit abe586e04c6a78f7abffe6afcefbadb39ff94c2a
Author: ILYA Khlopotov
AuthorDate: Mon Oct 7 10:35:55 2019 +
Return headers from _changes feed when there are no changes
Problem
---
The request of continious _changes feed doesn't return until either:
- new change is made to the database
- the heartbeat interval is reached
This causes clients to block on subscription call.
Solution
Introduce a counter to account for number of chunks sent.
Send '\n' exactly once on `waiting_for_updates` when `chunks_sent`
is still 0.
The implementation is suggested by @davisp
[here](https://github.com/apache/couchdb/issues/985#issuecomment-537150907).
There is only one difference from his proposal which is:
```
diff --git a/src/chttpd/src/chttpd_db.erl b/src/chttpd/src/chttpd_db.erl
index aba1bd22f..9cd6944d2 100644
--- a/src/chttpd/src/chttpd_db.erl
+++ b/src/chttpd/src/chttpd_db.erl
@@ -215,7 +215,7 @@ changes_callback(waiting_for_updates, #cacc{buffer =
[]} = Acc) ->
true ->
{ok, Acc};
false ->
-{ok, Resp1} = chttpd:send_delayed_chunk(Resp, []),
+{ok, Resp1} = chttpd:send_delayed_chunk(Resp, <<"\n">>),
{ok, Acc#cacc{mochi = Resp1, chunks_sent = 1}}
end;
changes_callback(waiting_for_updates, Acc) ->
```
---
src/chttpd/src/chttpd_db.erl | 33 +++---
src/chttpd/test/eunit/chttpd_db_test.erl | 40
2 files changed, 54 insertions(+), 19 deletions(-)
diff --git a/src/chttpd/src/chttpd_db.erl b/src/chttpd/src/chttpd_db.erl
index c6404b0..aba1bd2 100644
--- a/src/chttpd/src/chttpd_db.erl
+++ b/src/chttpd/src/chttpd_db.erl
@@ -46,6 +46,7 @@
mochi,
prepend = "",
responding = false,
+chunks_sent = 0,
buffer = [],
bufsize = 0,
threshold
@@ -170,10 +171,10 @@ changes_callback({change, {ChangeProp}=Change},
#cacc{feed = eventsource} = Acc)
Len = iolist_size(Chunk),
maybe_flush_changes_feed(Acc, Chunk, Len);
changes_callback(timeout, #cacc{feed = eventsource} = Acc) ->
-#cacc{mochi = Resp} = Acc,
+#cacc{mochi = Resp, chunks_sent = ChunksSet} = Acc,
Chunk = "event: heartbeat\ndata: \n\n",
{ok, Resp1} = chttpd:send_delayed_chunk(Resp, Chunk),
-{ok, Acc#cacc{mochi = Resp1}};
+{ok, Acc#cacc{mochi = Resp1, chunks_sent = ChunksSet + 1}};
changes_callback({stop, _EndSeq}, #cacc{feed = eventsource} = Acc) ->
#cacc{mochi = Resp, buffer = Buf} = Acc,
{ok, Resp1} = chttpd:send_delayed_chunk(Resp, Buf),
@@ -209,14 +210,27 @@ changes_callback({stop, EndSeq, Pending}, Acc) ->
chttpd:end_delayed_json_response(Resp1);
changes_callback(waiting_for_updates, #cacc{buffer = []} = Acc) ->
-{ok, Acc};
+#cacc{mochi = Resp, chunks_sent = ChunksSent} = Acc,
+case ChunksSent > 0 of
+true ->
+{ok, Acc};
+false ->
+{ok, Resp1} = chttpd:send_delayed_chunk(Resp, <<"\n">>),
+{ok, Acc#cacc{mochi = Resp1, chunks_sent = 1}}
+end;
changes_callback(waiting_for_updates, Acc) ->
-#cacc{buffer = Buf, mochi = Resp} = Acc,
+#cacc{buffer = Buf, mochi = Resp, chunks_sent = ChunksSent} = Acc,
{ok, Resp1} = chttpd:send_delayed_chunk(Resp, Buf),
-{ok, Acc#cacc{buffer = [], bufsize = 0, mochi = Resp1}};
+{ok, Acc#cacc{
+buffer = [],
+bufsize = 0,
+mochi = Resp1,
+chunks_sent = ChunksSent + 1
+}};
changes_callback(timeout, Acc) ->
-{ok, Resp1} = chttpd:send_delayed_chunk(Acc#cacc.mochi, "\n"),
-{ok, Acc#cacc{mochi = Resp1}};
+#cacc{mochi = Resp, chunks_sent = ChunksSent} = Acc,
+{ok, Resp1} = chttpd:send_delayed_chunk(Resp, "\n"),
+{ok, Acc#cacc{mochi = Resp1, chunks_sent = ChunksSent + 1}};
changes_callback({error, Reason}, #cacc{mochi = #httpd{}} = Acc) ->
#cacc{mochi = Req} = Acc,
chttpd:send_error(Req, Reason);
@@ -232,11 +246,12 @@ maybe_flush_changes_feed(#cacc{bufsize=Size,
threshold=Max} = Acc, Data, Len)
{ok, R1} = chttpd:send_delayed_chunk(Resp, Buffer),
{ok, Acc#cacc{prepend = ",\r\n", buffer = Data, bufsize=Len, mochi = R1}};
maybe_flush_changes_feed(Acc0, Data, Len) ->
-#cacc{buffer = Buf, bufsize = Size} = Acc0,
[couchdb] branch master updated: Make changes feed return bad request for invalid heartbeat values
This is an automated email from the ASF dual-hosted git repository.
iilyak pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/couchdb.git
The following commit(s) were added to refs/heads/master by this push:
new a47f0fa Make changes feed return bad request for invalid heartbeat
values
new be2364d Merge pull request #2270 from
bessbd/changes-feed-input-validation
a47f0fa is described below
commit a47f0fa8dde5f3b8d5c06649d8d896778c708f7e
Author: Bessenyei Balázs Donát
AuthorDate: Wed Oct 23 17:04:05 2019 +0200
Make changes feed return bad request for invalid heartbeat values
Using a negative heartbeat value does not return a 400 bad request, instead
getting just an empty response with no status code at all.
This commit adds extra checks so that negative and non-integer heartbeat
values return 400 bad request responses.
This fixes #2234
---
src/chttpd/src/chttpd_db.erl | 9 +++-
test/elixir/test/changes_test.exs | 43 +++
2 files changed, 51 insertions(+), 1 deletion(-)
diff --git a/src/chttpd/src/chttpd_db.erl b/src/chttpd/src/chttpd_db.erl
index aba1bd2..970495f 100644
--- a/src/chttpd/src/chttpd_db.erl
+++ b/src/chttpd/src/chttpd_db.erl
@@ -1816,7 +1816,14 @@ parse_changes_query(Req) ->
{"heartbeat", "true"} ->
Args#changes_args{heartbeat=true};
{"heartbeat", _} ->
-Args#changes_args{heartbeat=list_to_integer(Value)};
+try list_to_integer(Value) of
+HeartbeatInteger when HeartbeatInteger > 0 ->
+Args#changes_args{heartbeat=HeartbeatInteger};
+_ ->
+throw({bad_request, <<"The heartbeat value should be a
positive integer (in milliseconds).">>})
+catch error:badarg ->
+throw({bad_request, <<"Invalid heartbeat value. Expecting a
positive integer value (in milliseconds).">>})
+end;
{"timeout", _} ->
Args#changes_args{timeout=list_to_integer(Value)};
{"include_docs", "true"} ->
diff --git a/test/elixir/test/changes_test.exs
b/test/elixir/test/changes_test.exs
new file mode 100644
index 000..b554508
--- /dev/null
+++ b/test/elixir/test/changes_test.exs
@@ -0,0 +1,43 @@
+defmodule ChangesTest do
+ use CouchTestCase
+
+ @moduletag :changes
+
+ @moduledoc """
+ Test CouchDB /{db}/_changes
+ """
+
+ @tag :with_db
+ test "Changes feed negative heartbeat", context do
+db_name = context[:db_name]
+
+resp = Couch.get(
+ "/#{db_name}/_changes",
+ query: %{
+:feed => "continuous",
+:heartbeat => -1000
+ }
+)
+
+assert resp.status_code == 400
+assert resp.body["error"] == "bad_request"
+assert resp.body["reason"] == "The heartbeat value should be a positive
integer (in milliseconds)."
+ end
+
+ @tag :with_db
+ test "Changes feed non-integer heartbeat", context do
+db_name = context[:db_name]
+
+resp = Couch.get(
+ "/#{db_name}/_changes",
+ query: %{
+:feed => "continuous",
+:heartbeat => "a1000"
+ }
+)
+
+assert resp.status_code == 400
+assert resp.body["error"] == "bad_request"
+assert resp.body["reason"] == "Invalid heartbeat value. Expecting a
positive integer value (in milliseconds)."
+ end
+end
[couchdb] branch prototype/fdb-layer updated: Remove old clause which is no longer used
This is an automated email from the ASF dual-hosted git repository.
iilyak pushed a commit to branch prototype/fdb-layer
in repository https://gitbox.apache.org/repos/asf/couchdb.git
The following commit(s) were added to refs/heads/prototype/fdb-layer by this
push:
new 797fe08 Remove old clause which is no longer used
new c3ef462 Merge pull request #2275 from
cloudant/remove-ints-client-remains
797fe08 is described below
commit 797fe0811fa52dad211b5881a049df536ae65337
Author: ILYA Khlopotov
AuthorDate: Wed Sep 25 11:03:45 2019 +
Remove old clause which is no longer used
The history of `send_error(_Req, {already_sent, Resp, _Error})`
clause is bellow:
- it was added on
[2009/04/18](https://svn.apache.org/viewvc/couchdb/trunk/src/couchdb/couch_httpd.erl?r1=762574&r2=765819&diff_format=h)
- we triggered that clause [in
couch_httpd:do](https://svn.apache.org/viewvc/couchdb/trunk/src/couchdb/couch_httpd.erl?revision=642432&view=markup#l88)
- at that time we were using inets webserver [see use of
`httpd_response/3`](https://svn.apache.org/viewvc/couchdb/trunk/src/couchdb/couch_httpd.erl?revision=642432&view=markup#l170)
- The inets OTP codebase uses `already_sent` messages
[here](https://github.com/erlang/otp/blob/50214f02501926fee6ec286efa68a57a47c2e531/lib/inets/src/http_server/httpd_response.erl#L220)
It should be safe to remove this clause because we are not using inets
anymore
and search of `already_sent` term across all dependencies doesn't return
any results.
---
src/chttpd/src/chttpd.erl | 3 ---
src/couch/src/couch_httpd.erl | 3 ---
2 files changed, 6 deletions(-)
diff --git a/src/chttpd/src/chttpd.erl b/src/chttpd/src/chttpd.erl
index fb227ae..2f4bbcf 100644
--- a/src/chttpd/src/chttpd.erl
+++ b/src/chttpd/src/chttpd.erl
@@ -1038,9 +1038,6 @@ error_headers(#httpd{mochi_req=MochiReq}=Req, 401=Code,
ErrorStr, ReasonStr) ->
error_headers(_, Code, _, _) ->
{Code, []}.
-send_error(_Req, {already_sent, Resp, _Error}) ->
-{ok, Resp};
-
send_error(#httpd{} = Req, Error) ->
update_timeout_stats(Error, Req),
diff --git a/src/couch/src/couch_httpd.erl b/src/couch/src/couch_httpd.erl
index 3cdfc0c..10b44d1 100644
--- a/src/couch/src/couch_httpd.erl
+++ b/src/couch/src/couch_httpd.erl
@@ -977,9 +977,6 @@ error_headers(#httpd{mochi_req=MochiReq}=Req, Code,
ErrorStr, ReasonStr) ->
{Code, []}
end.
-send_error(_Req, {already_sent, Resp, _Error}) ->
-{ok, Resp};
-
send_error(Req, Error) ->
{Code, ErrorStr, ReasonStr} = error_info(Error),
{Code1, Headers} = error_headers(Req, Code, ErrorStr, ReasonStr),
[couchdb] branch prototype/fdb-layer updated: Remove compiler warning
This is an automated email from the ASF dual-hosted git repository.
iilyak pushed a commit to branch prototype/fdb-layer
in repository https://gitbox.apache.org/repos/asf/couchdb.git
The following commit(s) were added to refs/heads/prototype/fdb-layer by this
push:
new 3f322a5 Remove compiler warning
new ed1c3d7 Merge pull request #2274 from cloudant/fix-warning
3f322a5 is described below
commit 3f322a554cd0a29b8d4d0976a0dd4c8701ef355a
Author: ILYA Khlopotov
AuthorDate: Tue Oct 1 16:10:26 2019 +
Remove compiler warning
---
src/chttpd/src/chttpd_db.erl | 2 --
1 file changed, 2 deletions(-)
diff --git a/src/chttpd/src/chttpd_db.erl b/src/chttpd/src/chttpd_db.erl
index 5a7f060..8cfa1f4 100644
--- a/src/chttpd/src/chttpd_db.erl
+++ b/src/chttpd/src/chttpd_db.erl
@@ -333,8 +333,6 @@ update_partition_stats(PathParts) ->
handle_design_req(#httpd{
path_parts=[_DbName, _Design, Name, <<"_",_/binary>> = Action | _Rest]
}=Req, Db) ->
-DbName = fabric2_db:name(Db),
-%%case ddoc_cache:open(DbName, <<"_design/", Name/binary>>) of
case fabric2_db:open_doc(Db, <<"_design/", Name/binary>>) of
{ok, DDoc} ->
Handler = chttpd_handlers:design_handler(Action, fun bad_action_req/3),
[couchdb] branch master updated: Remove old clause which is no longer used
This is an automated email from the ASF dual-hosted git repository.
iilyak pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/couchdb.git
The following commit(s) were added to refs/heads/master by this push:
new fc6cc98 Remove old clause which is no longer used
new b99d6e0 Merge pull request #2276 from
cloudant/remove-inets-client-remains
fc6cc98 is described below
commit fc6cc980cf3cdd9163ae64ddd7215b6d60dd8bd9
Author: ILYA Khlopotov
AuthorDate: Wed Sep 25 11:03:45 2019 +
Remove old clause which is no longer used
The history of `send_error(_Req, {already_sent, Resp, _Error})`
clause is bellow:
- it was added on
[2009/04/18](https://svn.apache.org/viewvc/couchdb/trunk/src/couchdb/couch_httpd.erl?r1=762574&r2=765819&diff_format=h)
- we triggered that clause [in
couch_httpd:do](https://svn.apache.org/viewvc/couchdb/trunk/src/couchdb/couch_httpd.erl?revision=642432&view=markup#l88)
- at that time we were using inets webserver [see use of
`httpd_response/3`](https://svn.apache.org/viewvc/couchdb/trunk/src/couchdb/couch_httpd.erl?revision=642432&view=markup#l170)
- The inets OTP codebase uses `already_sent` messages
[here](https://github.com/erlang/otp/blob/50214f02501926fee6ec286efa68a57a47c2e531/lib/inets/src/http_server/httpd_response.erl#L220)
It should be safe to remove this clause because we are not using inets
anymore
and search of `already_sent` term across all dependencies doesn't return
any results.
---
src/chttpd/src/chttpd.erl | 3 ---
src/couch/src/couch_httpd.erl | 3 ---
2 files changed, 6 deletions(-)
diff --git a/src/chttpd/src/chttpd.erl b/src/chttpd/src/chttpd.erl
index 1e1d638..87fb341 100644
--- a/src/chttpd/src/chttpd.erl
+++ b/src/chttpd/src/chttpd.erl
@@ -1029,9 +1029,6 @@ error_headers(#httpd{mochi_req=MochiReq}=Req, 401=Code,
ErrorStr, ReasonStr) ->
error_headers(_, Code, _, _) ->
{Code, []}.
-send_error(_Req, {already_sent, Resp, _Error}) ->
-{ok, Resp};
-
send_error(#httpd{} = Req, Error) ->
update_timeout_stats(Error, Req),
diff --git a/src/couch/src/couch_httpd.erl b/src/couch/src/couch_httpd.erl
index 3cdfc0c..10b44d1 100644
--- a/src/couch/src/couch_httpd.erl
+++ b/src/couch/src/couch_httpd.erl
@@ -977,9 +977,6 @@ error_headers(#httpd{mochi_req=MochiReq}=Req, Code,
ErrorStr, ReasonStr) ->
{Code, []}
end.
-send_error(_Req, {already_sent, Resp, _Error}) ->
-{ok, Resp};
-
send_error(Req, Error) ->
{Code, ErrorStr, ReasonStr} = error_info(Error),
{Code1, Headers} = error_headers(Req, Code, ErrorStr, ReasonStr),
[couchdb] branch prototype/fdb-layer updated: Pass contexts to fabric2_db functions
This is an automated email from the ASF dual-hosted git repository.
iilyak pushed a commit to branch prototype/fdb-layer
in repository https://gitbox.apache.org/repos/asf/couchdb.git
The following commit(s) were added to refs/heads/prototype/fdb-layer by this
push:
new 583d7fe Pass contexts to fabric2_db functions
new 8d28d85 Merge pull request #2279 from
cloudant/refactor-user-ctx-handling
583d7fe is described below
commit 583d7feeaa6fbe3359474ec743ab960fe4a0dc27
Author: ILYA Khlopotov
AuthorDate: Tue Oct 1 16:01:08 2019 +
Pass contexts to fabric2_db functions
Since the db structure returned from fabric2_db:open and fabric2_db:create
includes `user_ctx` there is no need to pass it explicitly
in every `fabric2_db` call. This means we can simplify few things:
- Don't pass user_ctx in `chttpd_db:db_req/2` since we pass db already
- Don't have to use `db_open_options` in `chttpd_changes`
- Don't have to pass `user_ctx` to `fabric2_db:open_doc` and
`fabric2_db:update_doc`
---
src/chttpd/src/chttpd_db.erl | 103 ++-
1 file changed, 42 insertions(+), 61 deletions(-)
diff --git a/src/chttpd/src/chttpd_db.erl b/src/chttpd/src/chttpd_db.erl
index 5a7f060..88e7fa9 100644
--- a/src/chttpd/src/chttpd_db.erl
+++ b/src/chttpd/src/chttpd_db.erl
@@ -92,10 +92,7 @@
handle_changes_req(#httpd{path_parts=[_,<<"_changes">>]}=Req, _Db) ->
send_method_not_allowed(Req, "GET,POST,HEAD").
handle_changes_req1(#httpd{}=Req, Db) ->
-#changes_args{filter=Raw, style=Style} = Args0 = parse_changes_query(Req),
-ChangesArgs = Args0#changes_args{
-db_open_options = [{user_ctx, fabric2_db:get_user_ctx(Db)}]
-},
+ChangesArgs = parse_changes_query(Req),
ChangesFun = chttpd_changes:handle_db_changes(ChangesArgs, Req, Db),
Max = chttpd:chunked_response_buffer_size(),
case ChangesArgs#changes_args.feed of
@@ -360,10 +357,10 @@ handle_design_info_req(#httpd{method='GET'}=Req, Db,
#doc{} = DDoc) ->
handle_design_info_req(Req, _Db, _DDoc) ->
send_method_not_allowed(Req, "GET").
-create_db_req(#httpd{}=Req, DbName) ->
+create_db_req(#httpd{user_ctx=Ctx}=Req, DbName) ->
couch_httpd:verify_is_server_admin(Req),
DocUrl = absolute_uri(Req, "/" ++ couch_util:url_encode(DbName)),
-case fabric2_db:create(DbName, []) of
+case fabric2_db:create(DbName, [{user_ctx, Ctx}]) of
{ok, _} ->
send_json(Req, 201, [{"Location", DocUrl}], {[{ok, true}]});
{error, file_exists} ->
@@ -372,9 +369,9 @@ create_db_req(#httpd{}=Req, DbName) ->
throw(Error)
end.
-delete_db_req(#httpd{}=Req, DbName) ->
+delete_db_req(#httpd{user_ctx=Ctx}=Req, DbName) ->
couch_httpd:verify_is_server_admin(Req),
-case fabric2_db:delete(DbName, []) of
+case fabric2_db:delete(DbName, [{user_ctx, Ctx}]) of
ok ->
send_json(Req, 200, {[{ok, true}]});
Error ->
@@ -393,11 +390,9 @@ db_req(#httpd{method='GET',path_parts=[_DbName]}=Req, Db)
->
couch_stats:update_histogram([couchdb, dbinfo], DeltaT),
send_json(Req, {DbInfo});
-db_req(#httpd{method='POST', path_parts=[DbName], user_ctx=Ctx}=Req, Db) ->
+db_req(#httpd{method='POST', path_parts=[DbName]}=Req, Db) ->
chttpd:validate_ctype(Req, "application/json"),
-Options = [{user_ctx,Ctx}],
-
Doc0 = chttpd:json_body(Req),
Doc1 = couch_doc:from_json_obj_validate(Doc0, fabric2_db:name(Db)),
Doc2 = case Doc1#doc.id of
@@ -411,7 +406,7 @@ db_req(#httpd{method='POST', path_parts=[DbName],
user_ctx=Ctx}=Req, Db) ->
"ok" ->
% async_batching
spawn(fun() ->
-case catch(fabric2_db:update_doc(Db, Doc2, Options)) of
+case catch(fabric2_db:update_doc(Db, Doc2, [])) of
{ok, _} ->
chttpd_stats:incr_writes(),
ok;
@@ -431,7 +426,7 @@ db_req(#httpd{method='POST', path_parts=[DbName],
user_ctx=Ctx}=Req, Db) ->
% normal
DocUrl = absolute_uri(Req, [$/, couch_util:url_encode(DbName),
$/, couch_util:url_encode(DocId)]),
-case fabric2_db:update_doc(Db, Doc2, Options) of
+case fabric2_db:update_doc(Db, Doc2, []) of
{ok, NewRev} ->
chttpd_stats:incr_writes(),
HttpCode = 201;
@@ -449,8 +444,8 @@ db_req(#httpd{method='POST', path_parts=[DbName],
user_ctx=Ctx}=Req, Db) ->
db_req(#httpd{path_parts=[_DbName]}=Req, _Db) ->
send_method_not_allowed(Req, "DELETE,GET,HEAD,POST");
-db_req(#httpd{method='POST', path_parts=[_DbName, <<"_ensure_full_commit">>],
-user_ctx=Ctx}=Req, Db) ->
+db_re
[couchdb] branch master updated (cda9430 -> 0ef71cb)
This is an automated email from the ASF dual-hosted git repository. iilyak pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/couchdb.git. from cda9430 Fix multidb changes test new a66998e Make sure we fetch test dependencies new 1953c33 Integrate excoverals into test stack new 0ef71cb Merge pull request #2353 from cloudant/exunit-cover The 11934 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: .credo.exs | 8 ++ .gitignore | 10 Makefile | 5 ++-- Makefile.win | 2 +- mix.exs | 79 +++- mix.lock | 9 +++ 6 files changed, 109 insertions(+), 4 deletions(-)
[couchdb] branch prototype/fdb-layer updated: Add `external` tag to opentrace events
This is an automated email from the ASF dual-hosted git repository.
iilyak pushed a commit to branch prototype/fdb-layer
in repository https://gitbox.apache.org/repos/asf/couchdb.git
The following commit(s) were added to refs/heads/prototype/fdb-layer by this
push:
new 963f84b Add `external` tag to opentrace events
new 6b1da76 Merge pull request #2451 from cloudant/tracing-external
963f84b is described below
commit 963f84bad08d0a0e5587ae2f1869c3563d4182d6
Author: ILYA Khlopotov
AuthorDate: Tue Jan 14 02:41:50 2020 -0800
Add `external` tag to opentrace events
This PR adds an ability to selectively enable opentracing for HTTP requests
with X-B3-... headers. This is helpful in following cases:
- tracing all requests with X-B3-... headers
`all = (#{external := E}) when E == true -> true`
- tracing all requests to specific database with X-B3-... headers
```
all = (#{external := E, 'db.name' := Db})
when E == true andalso Db == <<"foo">> -> true
```
- tracing requests to specific endpoint with X-B3-... headers
```
db.design.view.read = (#{external := E, 'design.id' := Name})
when E == true andalso Name == <<"bar">> -> true
```
I want to remind that we support following X-B3-... headers:
- X-B3-TraceId
- X-B3-SpanId
- X-B3-ParentSpanId
- B3 which is in the following format
--<1 | 0>-
---
src/chttpd/src/chttpd.erl | 12 +++-
src/ctrace/README.md | 3 +++
2 files changed, 10 insertions(+), 5 deletions(-)
diff --git a/src/chttpd/src/chttpd.erl b/src/chttpd/src/chttpd.erl
index c4bfa60..d5e7314 100644
--- a/src/chttpd/src/chttpd.erl
+++ b/src/chttpd/src/chttpd.erl
@@ -1250,6 +1250,7 @@ start_span(Req) ->
[] -> <<"">>;
[_ | _] -> filename:join(PathParts)
end,
+{IsExternalSpan, RootOptions} = root_span_options(MochiReq),
Tags = maps:merge(#{
peer => Peer,
'http.method' => Method,
@@ -1257,21 +1258,22 @@ start_span(Req) ->
'http.url' => MochiReq:get(raw_path),
path_parts => Path,
'span.kind' => <<"server">>,
-component => <<"couchdb.chttpd">>
+component => <<"couchdb.chttpd">>,
+external => IsExternalSpan
}, ExtraTags),
ctrace:start_span(OperationName, [
{tags, Tags},
{time, Begin}
-] ++ maybe_root_span(MochiReq)).
+] ++ RootOptions).
-maybe_root_span(MochiReq) ->
+root_span_options(MochiReq) ->
case get_trace_headers(MochiReq) of
[undefined, _, _] ->
-[];
+{false, []};
[TraceId, SpanId, ParentSpanId] ->
Span = ctrace:external_span(TraceId, SpanId, ParentSpanId),
-[{root, Span}]
+{true, [{root, Span}]}
end.
parse_trace_id(undefined) ->
diff --git a/src/ctrace/README.md b/src/ctrace/README.md
index 6e40b43..3172f26 100644
--- a/src/ctrace/README.md
+++ b/src/ctrace/README.md
@@ -146,7 +146,10 @@ and logged.
```ini
[tracing.filters]
+; trace all events
; all = (#{}) -> true
+; trace all events with X-B3-... headers
+; all = (#{external := External}) when External == true -> true
; database-info.read = (#{'http.method' := Method}) when Method == 'GET' ->
true
; view.build = (#{'view.name' := Name}) when Name == "foo" -> 0.25
```
[couchdb-documentation] branch master updated: Fix typo in `_scheduler/docs`
This is an automated email from the ASF dual-hosted git repository. iilyak pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/couchdb-documentation.git The following commit(s) were added to refs/heads/master by this push: new 7466c11 Fix typo in `_scheduler/docs` new 3216391 Merge pull request #483 from bessbd/patch-2 7466c11 is described below commit 7466c1166f7c1826fc0bc57a2eb90aae6d28cacf Author: Bessenyei Balázs Donát AuthorDate: Wed Jan 22 13:57:49 2020 +0100 Fix typo in `_scheduler/docs` The response description for `_scheduler/docs` says there's a key `last_update`, however there isn't one. There is a `last_updated` instead, just as it shows in the example right below. This commit fixes the issue described above. --- src/api/server/common.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/api/server/common.rst b/src/api/server/common.rst index 415e45f..a5fb478 100644 --- a/src/api/server/common.rst +++ b/src/api/server/common.rst @@ -1086,7 +1086,7 @@ error. :>json string source: Replication source :>json string target: Replication target :>json string start_time: Timestamp of when the replication was started -:>json string last_update: Timestamp of last state update +:>json string last_updated: Timestamp of last state update :>json object info: Will contain additional information about the state. For errors, this will be an object with an `"error"` field and string value. For
[couchdb-thrift-protocol] 02/07: Merge pull request #1 from iilyak/export-struct-function-for-binary
This is an automated email from the ASF dual-hosted git repository. iilyak pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/couchdb-thrift-protocol.git commit 0cf27dc6a1076b46975dffec71941f993bde3c8d Merge: 1c86c26 b90dacd Author: Takeru Ohta AuthorDate: Sat Jan 18 13:16:11 2020 +0900 Merge pull request #1 from iilyak/export-struct-function-for-binary Export encode_struct and decode_struct functions src/thrift_protocol_binary.erl | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
[couchdb-thrift-protocol] 01/07: Export encode_struct and decode_struct functions
This is an automated email from the ASF dual-hosted git repository.
iilyak pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/couchdb-thrift-protocol.git
commit b90dacd2968a14a0cd925cd644204cf97dcb3212
Author: ILYA Khlopotov
AuthorDate: Wed Jan 15 07:47:33 2020 -0800
Export encode_struct and decode_struct functions
---
src/thrift_protocol_binary.erl | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/src/thrift_protocol_binary.erl b/src/thrift_protocol_binary.erl
index 301a9f1..ec55680 100644
--- a/src/thrift_protocol_binary.erl
+++ b/src/thrift_protocol_binary.erl
@@ -3,7 +3,7 @@
-include("thrift_protocol.hrl").
--export([encode_message/1, decode_message/1]).
+-export([encode_message/1, decode_message/1, encode_struct/1,
decode_struct/2]).
-define(VERSION, 1).
[couchdb-thrift-protocol] 03/07: Remove deprecated field
This is an automated email from the ASF dual-hosted git repository.
iilyak pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/couchdb-thrift-protocol.git
commit 6a6df1ea58d1a78750d20e6e9c2bd6da115c8ef5
Author: Takeru Ohta
AuthorDate: Sat Jan 18 13:20:20 2020 +0900
Remove deprecated field
---
src/thrift_protocol.app.src | 1 -
1 file changed, 1 deletion(-)
diff --git a/src/thrift_protocol.app.src b/src/thrift_protocol.app.src
index 1c7896a..2e4ef67 100644
--- a/src/thrift_protocol.app.src
+++ b/src/thrift_protocol.app.src
@@ -5,6 +5,5 @@
{applications,[kernel,stdlib]},
{env,[]},
{modules,[]},
- {maintainers,["Takeru Ohta"]},
{licenses,["MIT"]},
{links,[{"GitHub","https://github.com/sile/thrift_protocol"}]}]}.
[couchdb-thrift-protocol] 07/07: v0.1.5
This is an automated email from the ASF dual-hosted git repository.
iilyak pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/couchdb-thrift-protocol.git
commit 217438a913e7406848f92a8f742bf5c58bcf88bf
Author: Takeru Ohta
AuthorDate: Sat Jan 18 13:37:22 2020 +0900
v0.1.5
---
src/thrift_protocol.app.src | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/src/thrift_protocol.app.src b/src/thrift_protocol.app.src
index 114dd06..1c532b6 100644
--- a/src/thrift_protocol.app.src
+++ b/src/thrift_protocol.app.src
@@ -1,6 +1,6 @@
{application,thrift_protocol,
[{description,"An Erlang implementation of Thrift protocol."},
- {vsn,"0.1.4"},
+ {vsn,"0.1.5"},
{registered,[]},
{applications,[kernel,stdlib]},
{env,[]},
[couchdb-thrift-protocol] 05/07: Add `thrift_protocol:{decode_struct/2, encode_struct/2}`
This is an automated email from the ASF dual-hosted git repository.
iilyak pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/couchdb-thrift-protocol.git
commit 6395e2724fe2e71b03c60960e9703100d4fb456c
Author: Takeru Ohta
AuthorDate: Sat Jan 18 13:29:40 2020 +0900
Add `thrift_protocol:{decode_struct/2, encode_struct/2}`
---
src/thrift_protocol.erl | 16 +++-
src/thrift_protocol_compact.erl | 2 +-
2 files changed, 16 insertions(+), 2 deletions(-)
diff --git a/src/thrift_protocol.erl b/src/thrift_protocol.erl
index 4c80023..f9ca1ef 100644
--- a/src/thrift_protocol.erl
+++ b/src/thrift_protocol.erl
@@ -33,7 +33,7 @@
-include("thrift_protocol.hrl").
--export([encode_message/2, decode_message/2]).
+-export([encode_message/2, decode_message/2, encode_struct/2,
decode_struct/2]).
-export([data_type/1]).
-export_type([message/0, message_type/0]).
@@ -114,6 +114,13 @@ encode_message(Message, binary) ->
encode_message(Message, compact) ->
thrift_protocol_compact:encode_message(Message).
+%% @doc Encodes `Struct'.
+-spec encode_struct(struct(), Format :: format()) -> iodata().
+encode_struct(Struct, binary) ->
+thrift_protocol_binary:encode_struct(Struct);
+encode_struct(Struct, compact) ->
+thrift_protocol_compact:encode_struct(Struct).
+
%% @doc Decodes a message from the given binary.
-spec decode_message(binary(), Format :: format()) -> {message(), binary()}.
decode_message(Bin, binary) ->
@@ -121,6 +128,13 @@ decode_message(Bin, binary) ->
decode_message(Bin, compact) ->
thrift_protocol_compact:decode_message(Bin).
+%% @doc Decodes a struct from the given binary.
+-spec decode_struct(binary(), Format :: format()) -> {struct(), binary()}.
+decode_struct(Bin, binary) ->
+thrift_protocol_binary:decode_struct(Bin, #{});
+decode_struct(Bin, compact) ->
+thrift_protocol_compact:decode_struct(Bin, 0, #{}).
+
%% @doc Returns the type of `Data'.
-spec data_type(Data :: data()) -> data_type().
data_type(X) when is_boolean(X) ->
diff --git a/src/thrift_protocol_compact.erl b/src/thrift_protocol_compact.erl
index 4d12e81..ca44a05 100644
--- a/src/thrift_protocol_compact.erl
+++ b/src/thrift_protocol_compact.erl
@@ -11,7 +11,7 @@
-include("thrift_protocol.hrl").
--export([decode_message/1, encode_message/1]).
+-export([decode_message/1, encode_message/1, encode_struct/1,
decode_struct/3]).
-define(PROTOCOL_ID, 16#82).
-define(VERSION, 1).
[couchdb-thrift-protocol] branch master updated (1c86c26 -> 217438a)
This is an automated email from the ASF dual-hosted git repository.
iilyak pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/couchdb-thrift-protocol.git.
from 1c86c26 v0.1.3
new b90dacd Export encode_struct and decode_struct functions
new 0cf27dc Merge pull request #1 from
iilyak/export-struct-function-for-binary
new 6a6df1e Remove deprecated field
new eded491 v0.1.4
new 6395e27 Add `thrift_protocol:{decode_struct/2, encode_struct/2}`
new 9f75747 Update .gitignore
new 217438a v0.1.5
The 7 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
.gitignore | 1 +
src/thrift_protocol.app.src | 3 +--
src/thrift_protocol.erl | 16 +++-
src/thrift_protocol_binary.erl | 2 +-
src/thrift_protocol_compact.erl | 2 +-
5 files changed, 19 insertions(+), 5 deletions(-)
[couchdb-thrift-protocol] 04/07: v0.1.4
This is an automated email from the ASF dual-hosted git repository.
iilyak pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/couchdb-thrift-protocol.git
commit eded4916f0091362fe011a397091dc1a335b9e14
Author: Takeru Ohta
AuthorDate: Sat Jan 18 13:20:41 2020 +0900
v0.1.4
---
src/thrift_protocol.app.src | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/src/thrift_protocol.app.src b/src/thrift_protocol.app.src
index 2e4ef67..114dd06 100644
--- a/src/thrift_protocol.app.src
+++ b/src/thrift_protocol.app.src
@@ -1,6 +1,6 @@
{application,thrift_protocol,
[{description,"An Erlang implementation of Thrift protocol."},
- {vsn,"0.1.3"},
+ {vsn,"0.1.4"},
{registered,[]},
{applications,[kernel,stdlib]},
{env,[]},
[couchdb-thrift-protocol] 06/07: Update .gitignore
This is an automated email from the ASF dual-hosted git repository. iilyak pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/couchdb-thrift-protocol.git commit 9f75747fd482187b271502f1036d060b05e83aa0 Author: Takeru Ohta AuthorDate: Sat Jan 18 13:30:09 2020 +0900 Update .gitignore --- .gitignore | 1 + 1 file changed, 1 insertion(+) diff --git a/.gitignore b/.gitignore index 40ca652..41889bf 100644 --- a/.gitignore +++ b/.gitignore @@ -16,3 +16,4 @@ _build .idea *.iml rebar3.crashdump +doc/ \ No newline at end of file
[couchdb-thrift-protocol] tag 0.1.4 created (now eded491)
This is an automated email from the ASF dual-hosted git repository. iilyak pushed a change to tag 0.1.4 in repository https://gitbox.apache.org/repos/asf/couchdb-thrift-protocol.git. at eded491 (commit) No new revisions were added by this update.
[couchdb-thrift-protocol] tag 0.1.5 created (now 217438a)
This is an automated email from the ASF dual-hosted git repository. iilyak pushed a change to tag 0.1.5 in repository https://gitbox.apache.org/repos/asf/couchdb-thrift-protocol.git. at 217438a (commit) No new revisions were added by this update.
[couchdb-jaeger-passage] 04/09: Use httpc by default
This is an automated email from the ASF dual-hosted git repository.
iilyak pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/couchdb-jaeger-passage.git
commit dd36ad40df5e198bda39901e1779ed1e9f51e775
Author: ILYA Khlopotov
AuthorDate: Wed Jan 22 02:53:16 2020 -0800
Use httpc by default
---
src/jaeger_passage_reporter_http.erl | 18 +++---
test/jaeger_passage_repoter_tests.erl | 3 ++-
2 files changed, 17 insertions(+), 4 deletions(-)
diff --git a/src/jaeger_passage_reporter_http.erl
b/src/jaeger_passage_reporter_http.erl
index 095f3a1..0b0097c 100644
--- a/src/jaeger_passage_reporter_http.erl
+++ b/src/jaeger_passage_reporter_http.erl
@@ -74,7 +74,7 @@
| {process_tags, passage:tags()}.
%%
%% endpoint: The jaeger endpoint URL for sending thrift messages.
The default value is `http://127.0.0.1:14268'.
-%% http_client: The callback to call to send span to jaeger.
+%% http_client: The callback to call to send span to jaeger. The
httpc client is used by default.
%% default_service_name: The default service name. If a reporting
span has `location.application' tag, the value is used as the service name
instead of this. The default value is `ReporterId'.
%% process_tags: The tags of the reporting process. The default
value is `#{}'.
%%
@@ -112,8 +112,8 @@ init({ReporterId, Options}) ->
Endpoint = proplists:get_value(endpoint, Options,
"http://127.0.0.1:14268";),
EndpointURL = Endpoint ++ "/api/traces",
-HttpClient = proplists:get_value(http_client, Options),
-HttpClient =/= undefined orelse error(badarg, [ReporterId, Options]),
+HttpClient = proplists:get_value(http_client, Options, fun httpc_client/5),
+is_function(HttpClient, 5) orelse error(badarg, [ReporterId, Options]),
DefaultServiceName = proplists:get_value(default_service_name, Options,
ReporterId),
Tags0 = proplists:get_value(process_tags, Options, #{}),
@@ -171,3 +171,15 @@ handle_report(Span, State = #?STATE{default_service_name =
DefaultName, process_
Headers = [?CONTENT_TYPE],
HttpClient(URI, post, Headers, Encoded, Options),
{noreply, State}.
+
+-spec httpc_client(
+Url:: string(),
+Method :: post,
+Headers :: [{string(), string()}],
+Body :: string() | binary(),
+ReporterOptions :: start_options()) ->
+ok.
+
+httpc_client(Url, Method, _Headers, Body, _ReporterOptions) ->
+httpc:request(Method, {Url, [], "application/x-thrift", Body}, [], []),
+ok.
diff --git a/test/jaeger_passage_repoter_tests.erl
b/test/jaeger_passage_repoter_tests.erl
index 7f034bb..cdc25a5 100644
--- a/test/jaeger_passage_repoter_tests.erl
+++ b/test/jaeger_passage_repoter_tests.erl
@@ -64,7 +64,8 @@ error_http_test() ->
%% Starts `http_reporter'
?assertMatch({error, {{badarg, _}, _}},
jaeger_passage_reporter:start(http_reporter, [
-{protocol, http}
+{protocol, http},
+{http_client, undefined}
])),
%% Starts `http_reporter'
[couchdb-jaeger-passage] 08/09: Add OTP-22 as a CI target
This is an automated email from the ASF dual-hosted git repository. iilyak pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/couchdb-jaeger-passage.git commit c04ce6aee005e9d3633f43cf20241750d371ff3a Author: Takeru Ohta AuthorDate: Thu Jan 23 22:29:03 2020 +0900 Add OTP-22 as a CI target --- .travis.yml | 1 + 1 file changed, 1 insertion(+) diff --git a/.travis.yml b/.travis.yml index 175deb4..72329f7 100644 --- a/.travis.yml +++ b/.travis.yml @@ -2,6 +2,7 @@ language: erlang os: linux otp_release: + - 22.0 - 21.0 - 20.0 - 19.3
[couchdb-jaeger-passage] 09/09: v0.1.14
This is an automated email from the ASF dual-hosted git repository.
iilyak pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/couchdb-jaeger-passage.git
commit 8613f138ebd14e238c4e9ef9c5c24b9ff653e8ba
Author: Takeru Ohta
AuthorDate: Thu Jan 23 22:31:43 2020 +0900
v0.1.14
---
src/jaeger_passage.app.src | 3 +--
1 file changed, 1 insertion(+), 2 deletions(-)
diff --git a/src/jaeger_passage.app.src b/src/jaeger_passage.app.src
index 0cc50ac..ad3701f 100644
--- a/src/jaeger_passage.app.src
+++ b/src/jaeger_passage.app.src
@@ -1,12 +1,11 @@
{application,jaeger_passage,
[{description,"Jaeger client library for Erlang"},
- {vsn,"0.1.13"},
+ {vsn,"0.1.14"},
{registered,[]},
{mod,{jaeger_passage_app,[]}},
{applications,[kernel,stdlib,inets,local,passage,
thrift_protocol]},
{env,[]},
{modules,[]},
- {maintainers,["Takeru Ohta"]},
{licenses,["MIT"]},
{links,[{"GitHub","https://github.com/sile/jaeger_passage"}]}]}.
[couchdb-jaeger-passage] 03/09: Add test case for http_reporter
This is an automated email from the ASF dual-hosted git repository.
iilyak pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/couchdb-jaeger-passage.git
commit 367238681f376c3d899118806c9efb530990cff4
Author: ILYA Khlopotov
AuthorDate: Tue Jan 21 03:55:35 2020 -0800
Add test case for http_reporter
---
test/jaeger_passage_repoter_tests.erl | 59 ++-
1 file changed, 52 insertions(+), 7 deletions(-)
diff --git a/test/jaeger_passage_repoter_tests.erl
b/test/jaeger_passage_repoter_tests.erl
index 7aa1dd0..7f034bb 100644
--- a/test/jaeger_passage_repoter_tests.erl
+++ b/test/jaeger_passage_repoter_tests.erl
@@ -6,27 +6,72 @@
%%--
%% Test Cases
%%--
-basic_test() ->
+basic_udp_test() ->
{ok, _} = application:ensure_all_started(jaeger_passage),
-%% Starts `test_reporter'
-{ok, Reporter} = jaeger_passage_reporter:start(test_reporter),
-[test_reporter] = jaeger_passage_reporter:which_reporters(),
+%% Starts `udp_reporter'
+{ok, Reporter} = jaeger_passage_reporter:start(udp_reporter),
+[udp_reporter] = jaeger_passage_reporter:which_reporters(),
%% Registers `test_tracer'
Context = jaeger_passage_span_context,
Sampler = passage_sampler_all:new(),
-ok = passage_tracer_registry:register(test_tracer, Context, Sampler,
Reporter),
+ok = passage_tracer_registry:register(udp_tracer, Context, Sampler,
Reporter),
%% Starts and finishes spans
-passage_pd:start_span(test_root, [{tracer, test_tracer}]),
+passage_pd:start_span(test_root, [{tracer, udp_tracer}]),
passage_pd:start_span(test_child),
passage_pd:log(#{message => "Hello World"}),
passage_pd:finish_span(),
passage_pd:finish_span(),
timer:sleep(50),
-ok = jaeger_passage_reporter:stop(test_reporter),
+ok = jaeger_passage_reporter:stop(udp_reporter),
[] = jaeger_passage_reporter:which_reporters(),
ok = application:stop(jaeger_passage).
+
+basic_http_test() ->
+{ok, _} = application:ensure_all_started(jaeger_passage),
+
+%% Starts `http_reporter'
+{ok, Reporter} = jaeger_passage_reporter:start(http_reporter, [
+{protocol, http},
+{http_client, fun http_client/5}
+]),
+[http_reporter] = jaeger_passage_reporter:which_reporters(),
+
+%% Registers `test_tracer'
+Context = jaeger_passage_span_context,
+Sampler = passage_sampler_all:new(),
+ok = passage_tracer_registry:register(http_tracer, Context, Sampler,
Reporter),
+
+%% Starts and finishes spans
+passage_pd:start_span(test_root, [{tracer, http_tracer}]),
+passage_pd:start_span(test_child),
+passage_pd:log(#{message => "Hello World"}),
+passage_pd:finish_span(),
+passage_pd:finish_span(),
+timer:sleep(50),
+
+ok = jaeger_passage_reporter:stop(http_reporter),
+[] = jaeger_passage_reporter:which_reporters(),
+
+ok = application:stop(jaeger_passage).
+
+error_http_test() ->
+{ok, _} = application:ensure_all_started(jaeger_passage),
+
+%% Starts `http_reporter'
+?assertMatch({error, {{badarg, _}, _}},
jaeger_passage_reporter:start(http_reporter, [
+{protocol, http}
+])),
+
+%% Starts `http_reporter'
+?assertError(badarg, jaeger_passage_reporter:start(http_reporter, [
+{protocol, undefined}
+])).
+
+
+http_client(_URI, _Method, _Headers, _Encoded, _Options) ->
+ok.
\ No newline at end of file
[couchdb-jaeger-passage] 06/09: Use default value.
This is an automated email from the ASF dual-hosted git repository.
iilyak pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/couchdb-jaeger-passage.git
commit 732163262cea64b8c365e296f627ac170744e09b
Author: Takeru Ohta
AuthorDate: Thu Jan 23 21:40:42 2020 +0900
Use default value.
---
src/jaeger_passage_reporter.erl | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/src/jaeger_passage_reporter.erl b/src/jaeger_passage_reporter.erl
index 1287f48..a2a4a68 100644
--- a/src/jaeger_passage_reporter.erl
+++ b/src/jaeger_passage_reporter.erl
@@ -87,7 +87,7 @@
-spec start(reporter_id()) -> {ok, passage_reporter:reporter()} | {error,
Reason} when
Reason :: {already_started, pid()} | term().
start(ReporterId) ->
-start(ReporterId, [{protocol, udp}]).
+start(ReporterId, []).
%% @doc Starts a reporter process.
-spec start(reporter_id(), start_options()) -> {ok, Reporter} | {error,
Reason} when
[couchdb-jaeger-passage] 01/09: Update thrift_protocol to 0.1.5
This is an automated email from the ASF dual-hosted git repository.
iilyak pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/couchdb-jaeger-passage.git
commit 076192ba899dbd9b635948d7ba47ed3fb261bc78
Author: ILYA Khlopotov
AuthorDate: Tue Jan 21 03:32:46 2020 -0800
Update thrift_protocol to 0.1.5
---
rebar.config.script | 2 +-
rebar.lock | 4 ++--
2 files changed, 3 insertions(+), 3 deletions(-)
diff --git a/rebar.config.script b/rebar.config.script
index 4454cdc..74eb314 100644
--- a/rebar.config.script
+++ b/rebar.config.script
@@ -5,7 +5,7 @@ Rebar2Deps =
[
{local, ".*", {git, "https://github.com/sile/local.git";, {tag, "0.2.1"}}},
{passage, ".*", {git, "https://github.com/sile/passage.git";, {tag,
"0.2.6"}}},
- {thrift_protocol, ".*", {git,
"https://github.com/sile/thrift_protocol.git";, {tag, "0.1.3"}}}
+ {thrift_protocol, ".*", {git,
"https://github.com/sile/thrift_protocol.git";, {tag, "0.1.5"}}}
],
case IsRebar3 of
diff --git a/rebar.lock b/rebar.lock
index c9ba4ec..9d5110a 100644
--- a/rebar.lock
+++ b/rebar.lock
@@ -1,10 +1,10 @@
{"1.1.0",
[{<<"local">>,{pkg,<<"local">>,<<"0.2.1">>},0},
{<<"passage">>,{pkg,<<"passage">>,<<"0.2.6">>},0},
- {<<"thrift_protocol">>,{pkg,<<"thrift_protocol">>,<<"0.1.3">>},0}]}.
+ {<<"thrift_protocol">>,{pkg,<<"thrift_protocol">>,<<"0.1.5">>},0}]}.
[
{pkg_hash,[
{<<"local">>,
<<"F82483CD6DB6A39B0E4C59B37C2FCCCF5B96D90A746AFC3D79A81D31E3D40963">>},
{<<"passage">>,
<<"7B0A6F0A6806B056DC3323A6A0243503642E6425F45E33B87277EA0BE88BD130">>},
- {<<"thrift_protocol">>,
<<"CDD0C06BFC235159D789353CBB4F01F9FF04592F30329CB3DF2C77E28D92CCC0">>}]}
+ {<<"thrift_protocol">>,
<<"300DB7CA06BED397406A4680AD3DD3A0212AC7BEABDC81FA03C3C3DADEA13673">>}]}
].
[couchdb-jaeger-passage] branch master updated (690c699 -> 8613f13)
This is an automated email from the ASF dual-hosted git repository.
iilyak pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/couchdb-jaeger-passage.git.
from 690c699 v0.1.13
new 076192b Update thrift_protocol to 0.1.5
new c4edcbf Add HTTP reporter
new 3672386 Add test case for http_reporter
new dd36ad4 Use httpc by default
new e080c97 Merge pull request #3 from iilyak/add-http-reporter
new 7321632 Use default value.
new 82f8fd0 Update documentation
new c04ce6a Add OTP-22 as a CI target
new 8613f13 v0.1.14
The 9 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
.travis.yml| 1 +
README.md | 18 ++
rebar.config.script| 2 +-
rebar.lock | 4 +-
src/jaeger_passage.app.src | 3 +-
src/jaeger_passage.erl | 2 +-
src/jaeger_passage_reporter.erl| 132 ++-
src/jaeger_passage_reporter_http.erl | 183 +
src/jaeger_passage_reporter_sup.erl| 9 +-
...eporter.erl => jaeger_passage_reporter_udp.erl} | 98 ++-
src/jaeger_passage_thrift.erl | 8 +-
test/jaeger_passage_repoter_tests.erl | 60 ++-
12 files changed, 296 insertions(+), 224 deletions(-)
create mode 100644 src/jaeger_passage_reporter_http.erl
copy src/{jaeger_passage_reporter.erl => jaeger_passage_reporter_udp.erl} (64%)
[couchdb-jaeger-passage] 05/09: Merge pull request #3 from iilyak/add-http-reporter
This is an automated email from the ASF dual-hosted git repository. iilyak pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/couchdb-jaeger-passage.git commit e080c975226e85f22b47d7fa9543d0ca45456e0e Merge: 690c699 dd36ad4 Author: Takeru Ohta AuthorDate: Thu Jan 23 21:15:27 2020 +0900 Merge pull request #3 from iilyak/add-http-reporter Add HTTP reporter README.md | 18 +++ rebar.config.script| 2 +- rebar.lock | 4 +- src/jaeger_passage.erl | 2 +- src/jaeger_passage_reporter.erl| 138 +++--- ...porter.erl => jaeger_passage_reporter_http.erl} | 154 - src/jaeger_passage_reporter_sup.erl| 9 +- ...eporter.erl => jaeger_passage_reporter_udp.erl} | 79 +-- src/jaeger_passage_thrift.erl | 8 +- test/jaeger_passage_repoter_tests.erl | 60 +++- 10 files changed, 168 insertions(+), 306 deletions(-)
[couchdb-jaeger-passage] 02/09: Add HTTP reporter
This is an automated email from the ASF dual-hosted git repository.
iilyak pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/couchdb-jaeger-passage.git
commit c4edcbf558e939e3279dc88fc7beb528befa2be0
Author: ILYA Khlopotov
AuthorDate: Wed Jan 15 08:13:07 2020 -0800
Add HTTP reporter
The reporter can be configured using the following:
```
% Define http client callback
Client = fun(Url, Method, Headers, Body, ReporterOptions) ->
ibrowse:send_req(Url, Headers, Method, Body, [])
end.
[
{protocol, http},
{endpoint, "http://127.0.0.1:14268"},
{http_client, HttpClient}
]
```
---
README.md | 18 +++
src/jaeger_passage.erl | 2 +-
src/jaeger_passage_reporter.erl| 138
...porter.erl => jaeger_passage_reporter_http.erl} | 142 ++---
src/jaeger_passage_reporter_sup.erl| 9 +-
...eporter.erl => jaeger_passage_reporter_udp.erl} | 79 +---
src/jaeger_passage_thrift.erl | 8 +-
7 files changed, 100 insertions(+), 296 deletions(-)
diff --git a/README.md b/README.md
index ddfd2a6..078eee6 100644
--- a/README.md
+++ b/README.md
@@ -45,6 +45,23 @@ Browses the tracing result:
$ firefox http://localhost:16686/
```
+Selecting reporter
+--
+
+By default 'compact' jaeger.thrift over UDP reporter is used. However it is
+possible to select different reporter. Bellow is a configuration matrics for
+available options:
+
+| protocol | thrift_protocol | jaeger port | description |
+|--|-|-|--|
+| udp | compact | 6831| accept jaeger.thrift over compact
thrift protocol (default) |
+| udp | binary | 6832| accept jaeger.thrift over binary
thrift protocol |
+| http | N/A | 14268 | accept jaeger.thrift directly
from clients |
+
+The HTTP version is beneficial if you don't have jaeger agents deployed or your
+spans are greater than max udp packet size (65Kb).
+Otherwise it is better to use default.
+
References
---
@@ -52,3 +69,4 @@ References
- [Jaeger](https://uber.github.io/jaeger/)
-
[jaeger-client-go/README.md](https://github.com/jaegertracing/jaeger-client-go/blob/v2.9.0/README.md)
- [Jaeger Client
Library](https://github.com/jaegertracing/jaeger/blob/master/docs/client_libraries.md)
+- [Jaeger default
ports](https://www.jaegertracing.io/docs/1.8/getting-started/)
\ No newline at end of file
diff --git a/src/jaeger_passage.erl b/src/jaeger_passage.erl
index 50c5aa5..cf27120 100644
--- a/src/jaeger_passage.erl
+++ b/src/jaeger_passage.erl
@@ -33,7 +33,7 @@
Tracer :: passage:tracer_id(),
Sampler :: passage_sampler:sampler().
start_tracer(Tracer, Sampler) ->
-start_tracer(Tracer, Sampler, []).
+start_tracer(Tracer, Sampler, [{protocol, udp}]).
%% @doc Starts a tracer for Jaeger.
%%
diff --git a/src/jaeger_passage_reporter.erl b/src/jaeger_passage_reporter.erl
index 32f537e..1287f48 100644
--- a/src/jaeger_passage_reporter.erl
+++ b/src/jaeger_passage_reporter.erl
@@ -29,7 +29,6 @@
-module(jaeger_passage_reporter).
-behaviour(passage_reporter).
--behaviour(gen_server).
-include("constants.hrl").
@@ -45,36 +44,11 @@
-export_type([start_option/0, start_options/0]).
%%--
-%% Application Internal API
-%%--
--export([start_link/2]).
-
-%%--
%% 'passage_reporter' Callback API
%%--
-export([report/2]).
%%--
-%% 'gen_server' Callback API
-%%--
--export([init/1, handle_call/3, handle_cast/2, handle_info/2, terminate/2,
code_change/3]).
-
-%%--
-%% Macros & Records
-%%--
--define(STATE, ?MODULE).
-
--record(?STATE,
-{
- socket :: gen_udp:socket(),
- thrift_format:: thrift_protocol:format(),
- agent_host :: inet:hostname(),
- agent_port :: inet:port_number(),
- default_service_name :: atom(),
- process_tags :: passage:tags()
-}).
-
-%%--
[couchdb-jaeger-passage] 07/09: Update documentation
This is an automated email from the ASF dual-hosted git repository.
iilyak pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/couchdb-jaeger-passage.git
commit 82f8fd037325a2ceb6695a350181e41c5331f0cc
Author: Takeru Ohta
AuthorDate: Thu Jan 23 22:21:43 2020 +0900
Update documentation
---
src/jaeger_passage_reporter.erl | 24 +++-
src/jaeger_passage_reporter_http.erl | 56 +---
src/jaeger_passage_reporter_udp.erl | 21 ++
3 files changed, 40 insertions(+), 61 deletions(-)
diff --git a/src/jaeger_passage_reporter.erl b/src/jaeger_passage_reporter.erl
index a2a4a68..5054ae9 100644
--- a/src/jaeger_passage_reporter.erl
+++ b/src/jaeger_passage_reporter.erl
@@ -1,6 +1,6 @@
%% @copyright 2017 Takeru Ohta
%%
-%% @doc A reporter that sends the spans to an jaeger agent
+%% @doc A reporter that sends the spans to an jaeger agent (UDP) or collector
(HTTP).
%%
%% === Examples ===
%%
@@ -57,27 +57,11 @@
-type start_options() :: [start_option()].
%% Options for {@link start/2}.
--type start_option() :: jaeger_passage_reporter_udp:start_option()
- | jaeger_passage_reporter_http:start_option()
- | {protocol, udp | http}
- | {default_service_name, atom()}
- | {process_tags, passage:tags()}.
-
-%% Common reporter options
+-type start_option() :: {protocol, udp | http}
+ | jaeger_passage_reporter_udp:start_option()
+ | jaeger_passage_reporter_http:start_option().
%%
%% protocol: Communication protocol used to connect to jaeger.
The value is used to select reporter module. Possible values are: `udp' |
`http'. The default value is `udp'.
-%% default_service_name: The default service name. If a reporting
span has `location.application' tag, the value is used as the service name
instead of this. The default value is `ReporterId'.
-%% process_tags: The tags of the reporting process. The default
value is `#{}'.
-%%
-%% UDP reporter specific options
-%%
-%% thrift_format: The format for encoding thrift messages. The
default value is `compact'.
-%% agent_host: The hostname of the jaeger agent. The default
value is `"127.0.0.1"'.
-%% agent_port: The port of the jaeger agent. The default values
for the thrift format `compact' and `binary' are `6831' and `6832'
respectively.
-%%
-%% HTTP reporter specific options
-%%
-%% endpoint: The jaeger endpoint URL for sending thrift messages.
The default value is `http://127.0.0.1:14268'.
%%
%%--
diff --git a/src/jaeger_passage_reporter_http.erl
b/src/jaeger_passage_reporter_http.erl
index 0b0097c..e226d6c 100644
--- a/src/jaeger_passage_reporter_http.erl
+++ b/src/jaeger_passage_reporter_http.erl
@@ -1,4 +1,7 @@
-%% @doc A reporter that sends the spans to an jaeger agent
+%% @doc A reporter that sends the spans to an jaeger collector using HTTP.
+%%
+%% To start a reporter process, please use {@link
jaeger_passage_reporter:start/1} or {@link jaeger_passage_reporter:start/2}.
+%%
%%
%% === Examples ===
%%
@@ -15,15 +18,8 @@
%% %% Starts and finishes a span
%% Span = passage:start_span(example, [{tracer, example_tracer}]).
%%
-%% passage:finish_span(Span). % The span will send to the jaeger agent on the
localhost
+%% passage:finish_span(Span). % The span will send to the jaeger collector on
the localhost
%% '''
-%%
-%% === Refereces ===
-%%
-%%
-%% http://jaeger.readthedocs.io/en/latest/architecture/#agent";>Jaeger -
Architecture - Agent
-%% http://jaeger.readthedocs.io/en/latest/deployment/#agent";>Jaeger -
Deployment - Agent
-%%
-module(jaeger_passage_reporter_http).
-behaviour(gen_server).
@@ -33,6 +29,8 @@
%%--
%% Exported API
%%--
+-export([httpc_client/5]).
+
-export_type([start_option/0, start_options/0]).
%%--
@@ -66,25 +64,28 @@
%%--
-type start_options() :: [start_option()].
-%% Options for {@link start/2}.
+%% Options for {@link jaeger_passage_reporter:start/2}.
--type start_option() :: {endpoint, string()}
+-type start_option() :: {default_service_name, atom()}
+ | {process_tags, passage:tags()}
+ | {endpoint, string()}
| {http_client, http_client()}
- | {default_service_name, atom()}
- | {process_tags, passage:tags()}.
+ | (HttpClientSpecificOption :: any()).
%%
-%% en
[couchdb-jaeger-passage] tag 0.1.14 created (now 8613f13)
This is an automated email from the ASF dual-hosted git repository. iilyak pushed a change to tag 0.1.14 in repository https://gitbox.apache.org/repos/asf/couchdb-jaeger-passage.git. at 8613f13 (commit) No new revisions were added by this update.
[couchdb-jaeger-passage] branch couchdb-0.1.14 created (now 8613f13)
This is an automated email from the ASF dual-hosted git repository. iilyak pushed a change to branch couchdb-0.1.14 in repository https://gitbox.apache.org/repos/asf/couchdb-jaeger-passage.git. at 8613f13 v0.1.14 No new revisions were added by this update.
[couchdb-jaeger-passage] 01/01: Use Apache CouchDB mirrors for dependencies
This is an automated email from the ASF dual-hosted git repository.
iilyak pushed a commit to branch point-to-couchdb-deps
in repository https://gitbox.apache.org/repos/asf/couchdb-jaeger-passage.git
commit 7f56a93393070fb6e3778615a2d040573b10a6f3
Author: ILYA Khlopotov
AuthorDate: Thu Jan 23 07:05:24 2020 -0800
Use Apache CouchDB mirrors for dependencies
---
rebar.config.script | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)
diff --git a/rebar.config.script b/rebar.config.script
index 74eb314..56e94b9 100644
--- a/rebar.config.script
+++ b/rebar.config.script
@@ -3,9 +3,9 @@ IsRebar3 = erlang:function_exported(rebar3, main, 1),
Rebar2Deps =
[
- {local, ".*", {git, "https://github.com/sile/local.git";, {tag, "0.2.1"}}},
- {passage, ".*", {git, "https://github.com/sile/passage.git";, {tag,
"0.2.6"}}},
- {thrift_protocol, ".*", {git,
"https://github.com/sile/thrift_protocol.git";, {tag, "0.1.5"}}}
+ {local, ".*", {git, "https://github.com/apache/couchdb-local.git";, {tag,
"0.2.1"}}},
+ {passage, ".*", {git, "https://github.com/apache/couchdb-passage.git";,
{tag, "0.2.6"}}},
+ {thrift_protocol, ".*", {git,
"https://github.com/apache/couchdb-thrift-protocol.git";, {tag, "0.1.5"}}}
],
case IsRebar3 of
[couchdb-jaeger-passage] branch point-to-couchdb-deps created (now 7f56a93)
This is an automated email from the ASF dual-hosted git repository. iilyak pushed a change to branch point-to-couchdb-deps in repository https://gitbox.apache.org/repos/asf/couchdb-jaeger-passage.git. at 7f56a93 Use Apache CouchDB mirrors for dependencies This branch includes the following new commits: new 7f56a93 Use Apache CouchDB mirrors for dependencies The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference.
[couchdb-jaeger-passage] branch upstream created (now 8613f13)
This is an automated email from the ASF dual-hosted git repository. iilyak pushed a change to branch upstream in repository https://gitbox.apache.org/repos/asf/couchdb-jaeger-passage.git. at 8613f13 v0.1.14 No new revisions were added by this update.
[couchdb-jaeger-passage] 01/01: Merge pull request #2 from apache/point-to-couchdb-deps
This is an automated email from the ASF dual-hosted git repository. iilyak pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/couchdb-jaeger-passage.git commit 4aa2f4975799bad70d2188fc1ae6334b25181862 Merge: 8613f13 7f56a93 Author: iilyak AuthorDate: Thu Jan 30 04:48:15 2020 -0800 Merge pull request #2 from apache/point-to-couchdb-deps Use Apache CouchDB mirrors for dependencies rebar.config.script | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-)
[couchdb-jaeger-passage] branch master updated (8613f13 -> 4aa2f49)
This is an automated email from the ASF dual-hosted git repository. iilyak pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/couchdb-jaeger-passage.git. from 8613f13 v0.1.14 add 7f56a93 Use Apache CouchDB mirrors for dependencies new 4aa2f49 Merge pull request #2 from apache/point-to-couchdb-deps The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: rebar.config.script | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-)
[couchdb-jaeger-passage] annotated tag CouchDB-0.1.14-1 updated (4aa2f49 -> d8d106b)
This is an automated email from the ASF dual-hosted git repository. iilyak pushed a change to annotated tag CouchDB-0.1.14-1 in repository https://gitbox.apache.org/repos/asf/couchdb-jaeger-passage.git. *** WARNING: tag CouchDB-0.1.14-1 was modified! *** from 4aa2f49 (commit) to d8d106b (tag) tagging 4aa2f4975799bad70d2188fc1ae6334b25181862 (commit) by ILYA Khlopotov on Thu Jan 30 05:26:11 2020 -0800 - Log - * Add HTTP reporter * Update thrift_protocol to 0.1.5 --- No new revisions were added by this update. Summary of changes:
[couchdb] branch prototype/fdb-layer updated: Support jaeger http reporter
This is an automated email from the ASF dual-hosted git repository.
iilyak pushed a commit to branch prototype/fdb-layer
in repository https://gitbox.apache.org/repos/asf/couchdb.git
The following commit(s) were added to refs/heads/prototype/fdb-layer by this
push:
new bd3c021 Support jaeger http reporter
new 7e0c5bb Merge pull request #2494 from cloudant/add-http-reporter
bd3c021 is described below
commit bd3c021bd887f4233c6c9a9fff121fa9c22af84e
Author: ILYA Khlopotov
AuthorDate: Fri Jan 17 09:37:54 2020 -0800
Support jaeger http reporter
---
rebar.config.script | 4 ++--
rel/overlay/etc/default.ini | 12 +---
src/ctrace/README.md | 18 --
src/ctrace/src/ctrace_config.erl | 38 +-
4 files changed, 56 insertions(+), 16 deletions(-)
diff --git a/rebar.config.script b/rebar.config.script
index 2fde9c5..f7ebcb5 100644
--- a/rebar.config.script
+++ b/rebar.config.script
@@ -121,13 +121,13 @@ DepDescs = [
{folsom, "folsom", {tag, "CouchDB-0.8.3"}},
{hyper,"hyper",{tag, "CouchDB-2.2.0-4"}},
{ibrowse, "ibrowse", {tag, "CouchDB-4.0.1-1"}},
-{jaeger_passage, "jaeger-passage", {tag, "CouchDB-0.1.13-1"}},
+{jaeger_passage, "jaeger-passage", {tag, "CouchDB-0.1.14-1"}},
{jiffy,"jiffy",{tag, "CouchDB-0.14.11-2"}},
{local,"local",{tag, "0.2.1"}},
{mochiweb, "mochiweb", {tag, "v2.19.0"}},
{meck, "meck", {tag, "0.8.8"}},
{passage, "passage", {tag, "0.2.6"}},
-{thrift_protocol, "thrift-protocol", {tag, "0.1.3"}},
+{thrift_protocol, "thrift-protocol", {tag, "0.1.5"}},
%% TMP - Until this is moved to a proper Apache repo
{erlfdb, "erlfdb", {branch, "master"}}
diff --git a/rel/overlay/etc/default.ini b/rel/overlay/etc/default.ini
index 63cb443..592445c 100644
--- a/rel/overlay/etc/default.ini
+++ b/rel/overlay/etc/default.ini
@@ -568,13 +568,19 @@ min_priority = 2.0
[tracing]
;
; Configuration settings for the `ctrace` OpenTracing
-; API.
-;
+; API. There are two reporter which we support.
+; - jaeger.thrift over udp
+; - jaeger.thrift over http
+; ## Common settings
; enabled = false ; true | false
+; app_name = couchdb ; value to use for the `location.application` tag
+; protocol = udp ; udp | http - which reporter to use
+; ## jaeger.thrift over udp reporter
; thrift_format = compact ; compact | binary
; agent_host = 127.0.0.1
; agent_port = 6831
-; app_name = couchdb ; value to use for the `location.application` tag
+; ## jaeger.thrift over udp reporter
+; endpoint = http://127.0.0.1:14268
[tracing.filters]
;
diff --git a/src/ctrace/README.md b/src/ctrace/README.md
index 3172f26..4b0238b 100644
--- a/src/ctrace/README.md
+++ b/src/ctrace/README.md
@@ -120,9 +120,23 @@ Configuration
Traces are configured using standard CouchDB ini file based configuration.
There is a global toggle `[tracing] enabled = true | false` that switches
tracing on or off completely. The `[tracing]` section also includes
-configuration for where to send trace data.
+configuration for where to send trace data. There are two reporters which we
+support.
-An example `[tracing]` section
+The thrift over udp reporter (this is the default) has following configuration
+options:
+
+- protocol = udp
+- thrift_format = compact | binary
+- agent_host = 127.0.0.1
+- agent_port = 6831
+
+The thrift over http has following options
+
+- protocol = http
+- endpoint = http://127.0.0.1:14268
+
+An example of `[tracing]` section
```ini
[tracing]
diff --git a/src/ctrace/src/ctrace_config.erl b/src/ctrace/src/ctrace_config.erl
index bc2a3df..c63c77f 100644
--- a/src/ctrace/src/ctrace_config.erl
+++ b/src/ctrace/src/ctrace_config.erl
@@ -98,17 +98,37 @@ maybe_start_main_tracer(TracerId) ->
start_main_tracer(TracerId) ->
-Sampler = passage_sampler_all:new(),
-Options = [
-{thrift_format,
-list_to_atom(config:get("tracing", "thrift_format", "compact"))},
-{agent_host, config:get("tracing", "agent_host", "127.0.0.1")},
-{agent_port, config:get_integer("tracing", "agent_port", 6831)},
-{default_service_name,
-list_to_atom(config:get("tracing", "app_name", "couchdb"))}
-],
+MaxQueueLen = config:get_integer("tracing", "max_queue_len", 1024),
+Sampler = jaeger_passage_sampler_queue_limit:new(
+passage_sampler_all:new(), TracerId, MaxQueueLen),
+Ser
[couchdb] branch prototype/fdb-layer updated (7e0c5bb -> f431566)
This is an automated email from the ASF dual-hosted git repository.
iilyak pushed a change to branch prototype/fdb-layer
in repository https://gitbox.apache.org/repos/asf/couchdb.git.
from 7e0c5bb Merge pull request #2494 from cloudant/add-http-reporter
new bf39736 Update httpotion to 3.1.3
new 8650a4e Support setting base_url in Couch test helper
new 3b9f04d fix b3 - Headers suppose to be strings
new d233f81 Add basic test case for b3 fix
new f431566 Merge pull request #2519 from cloudant/fix-b3-header
The 11895 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
mix.exs | 2 +-
mix.lock | 2 +-
src/chttpd/src/chttpd.erl | 2 +-
src/{couch => chttpd}/test/exunit/test_helper.exs | 0
src/chttpd/test/exunit/tracing_test.exs | 101 ++
test/elixir/lib/couch.ex | 124 +++---
6 files changed, 119 insertions(+), 112 deletions(-)
copy src/{couch => chttpd}/test/exunit/test_helper.exs (100%)
create mode 100644 src/chttpd/test/exunit/tracing_test.exs
[couchdb] branch prototype/fdb-layer updated: Add support for X-Couch-Trace header
This is an automated email from the ASF dual-hosted git repository.
iilyak pushed a commit to branch prototype/fdb-layer
in repository https://gitbox.apache.org/repos/asf/couchdb.git
The following commit(s) were added to refs/heads/prototype/fdb-layer by this
push:
new 782f4fb Add support for X-Couch-Trace header
new b3bd36b Merge pull request #2597 from cloudant/add-couch-trace-header
782f4fb is described below
commit 782f4fb92f257aa9ae621dd3fb8ea0b228ce1931
Author: ILYA Khlopotov
AuthorDate: Tue Feb 25 03:05:51 2020 -0800
Add support for X-Couch-Trace header
---
src/chttpd/src/chttpd.erl | 5 -
1 file changed, 4 insertions(+), 1 deletion(-)
diff --git a/src/chttpd/src/chttpd.erl b/src/chttpd/src/chttpd.erl
index 9888b46..2923a16 100644
--- a/src/chttpd/src/chttpd.erl
+++ b/src/chttpd/src/chttpd.erl
@@ -1251,6 +1251,9 @@ start_span(Req) ->
[_ | _] -> filename:join(PathParts)
end,
{IsExternalSpan, RootOptions} = root_span_options(MochiReq),
+
+CouchTrace = header_value(Req, "X-Couch-Trace") /= undefined,
+
Tags = maps:merge(#{
peer => Peer,
'http.method' => Method,
@@ -1259,7 +1262,7 @@ start_span(Req) ->
path_parts => Path,
'span.kind' => <<"server">>,
component => <<"couchdb.chttpd">>,
-external => IsExternalSpan
+external => IsExternalSpan orelse CouchTrace
}, ExtraTags),
ctrace:start_span(OperationName, [
[couchdb] branch prototype/fdb-layer updated: Use `couch_rate` application for `couch_view`
This is an automated email from the ASF dual-hosted git repository.
iilyak pushed a commit to branch prototype/fdb-layer
in repository https://gitbox.apache.org/repos/asf/couchdb.git
The following commit(s) were added to refs/heads/prototype/fdb-layer by this
push:
new 85f81d8 Use `couch_rate` application for `couch_view`
new d520d73 Merge pull request #2662 from cloudant/couch_view-rate_limit
85f81d8 is described below
commit 85f81d88018fe526f1f216d673a9bbc847cbd81c
Author: ILYA Khlopotov
AuthorDate: Thu Feb 13 12:23:42 2020 -0800
Use `couch_rate` application for `couch_view`
---
.credo.exs | 1 +
.gitignore | 1 +
Makefile | 1 +
mix.exs| 8 +-
mix.lock | 1 +
rebar.config.script| 1 +
rel/files/eunit.ini| 3 +
rel/overlay/etc/default.ini| 4 +
src/couch_rate/README.md | 155 +
.../src/couch_rate.app.src}| 18 +-
src/couch_rate/src/couch_rate.erl | 318 +
.../src/couch_rate.hrl}| 27 +-
.../src/couch_rate_app.erl}| 36 +-
src/couch_rate/src/couch_rate_config.erl | 66
src/couch_rate/src/couch_rate_ets.erl | 119 +++
src/couch_rate/src/couch_rate_limiter.erl | 387 +
src/couch_rate/src/couch_rate_pd.erl | 90 +
.../src/couch_rate_sup.erl}| 44 +--
.../test/exunit/couch_rate_config_test.exs | 88 +
.../test/exunit/couch_rate_limiter_test.exs| 350 +++
src/couch_rate/test/exunit/test_helper.exs | 14 +
src/couch_views/README.md | 33 ++
src/couch_views/src/couch_views.app.src| 3 +-
src/couch_views/src/couch_views_indexer.erl| 60 ++--
src/couch_views/test/couch_views_indexer_test.erl | 55 ++-
.../test/couch_views_trace_index_test.erl | 2 +-
26 files changed, 1784 insertions(+), 101 deletions(-)
diff --git a/.credo.exs b/.credo.exs
index bd26f40..112561b 100644
--- a/.credo.exs
+++ b/.credo.exs
@@ -37,6 +37,7 @@
~r"/src/metrics",
~r"/src/minerl",
~r"/src/parse_trans",
+ ~r"/src/stream_data",
~r"/src/ssl_verify_fun",
~r"/test/elixir/deps/"
]
diff --git a/.gitignore b/.gitignore
index bf45d1a..955403a 100644
--- a/.gitignore
+++ b/.gitignore
@@ -76,6 +76,7 @@ src/rebar/
src/recon/
src/smoosh/
src/snappy/
+src/stream_data/
src/ssl_verify_fun/
src/thrift_protocol/
src/triq/
diff --git a/Makefile b/Makefile
index 2f5df90..b3eb64c 100644
--- a/Makefile
+++ b/Makefile
@@ -162,6 +162,7 @@ endif
check-fdb:
make eunit
apps=couch_eval,couch_expiring_cache,ctrace,couch_jobs,couch_views,fabric
make elixir
tests=test/elixir/test/basics_test.exs,test/elixir/test/replication_test.exs,test/elixir/test/map_test.exs,test/elixir/test/all_docs_test.exs,test/elixir/test/bulk_docs_test.exs
+ make exunit tests=src/couch_rate/test/exunit/
.PHONY: eunit
# target: eunit - Run EUnit tests, use EUNIT_OPTS to provide custom options
diff --git a/mix.exs b/mix.exs
index 29c81fa..480d426 100644
--- a/mix.exs
+++ b/mix.exs
@@ -49,11 +49,14 @@ defmodule CouchDBTest.Mixfile do
# Run "mix help compile.app" to learn about applications.
def application do
[
- extra_applications: [:logger],
+ extra_applications: extra_applications(Mix.env()),
applications: [:httpotion]
]
end
+ defp extra_applications(:test), do: [:logger, :stream_data]
+ defp extra_applications(_), do: [:logger]
+
# Specifies which paths to compile per environment.
defp elixirc_paths(:test), do: ["test/elixir/lib",
"test/elixir/test/support"]
defp elixirc_paths(:integration), do: ["test/elixir/lib",
"test/elixir/test/support"]
@@ -68,7 +71,8 @@ defmodule CouchDBTest.Mixfile do
{:jiffy, path: Path.expand("src/jiffy", __DIR__)},
{:ibrowse,
path: Path.expand("src/ibrowse", __DIR__), override: true, compile:
false},
- {:credo, "~> 1.2.0", only: [:dev, :test, :integration], runtime: false}
+ {:credo, "~> 1.2.0", only: [:dev, :test, :integration], runtime: false},
+ {:stream_data, "~> 0.4.3", only: [:dev, :test, :integration], runtime:
false}
]
end
diff --git a/mix.lock b/mix.lock
index c03e11f..7a155c6 100644
--- a/mix.lock
+++ b/mix.lock
@@ -14,5 +14,6 @@
"mimerl": {:hex, :mimerl, "1.2.0",
"67e2d
[couchdb] branch prototype/fdb-layer updated: Add mango_plugin
This is an automated email from the ASF dual-hosted git repository.
iilyak pushed a commit to branch prototype/fdb-layer
in repository https://gitbox.apache.org/repos/asf/couchdb.git
The following commit(s) were added to refs/heads/prototype/fdb-layer by this
push:
new a14f62d Add mango_plugin
new 396a3b5 Merge pull request #2767 from
cloudant/prototype/fdb-layer-mango-plugin
a14f62d is described below
commit a14f62d3f0bbb16f57d692d63028579f96affc5e
Author: ILYA Khlopotov
AuthorDate: Tue Apr 7 09:46:30 2020 -0700
Add mango_plugin
Implement the following extention points:
- `before_find(Req) -> {ok, Req}`
- `after_find(Req, HttpResp, KVs) -> {ok, KVs}`
---
src/mango/src/mango_epi.erl| 4 +++-
src/mango/src/mango_httpd.erl | 20 +++-
src/mango/src/mango_plugin.erl | 43 ++
3 files changed, 57 insertions(+), 10 deletions(-)
diff --git a/src/mango/src/mango_epi.erl b/src/mango/src/mango_epi.erl
index 1fcd05b..d593d63 100644
--- a/src/mango/src/mango_epi.erl
+++ b/src/mango/src/mango_epi.erl
@@ -33,7 +33,9 @@ providers() ->
].
services() ->
-[].
+[
+{mango, mango_plugin}
+].
data_subscriptions() ->
[].
diff --git a/src/mango/src/mango_httpd.erl b/src/mango/src/mango_httpd.erl
index 94aa866..8d5a212 100644
--- a/src/mango/src/mango_httpd.erl
+++ b/src/mango/src/mango_httpd.erl
@@ -187,17 +187,18 @@ handle_explain_req(Req, _Db) ->
chttpd:send_method_not_allowed(Req, "POST").
-handle_find_req(#httpd{method='POST'}=Req, Db) ->
-chttpd:validate_ctype(Req, "application/json"),
-Body = chttpd:json_body_obj(Req),
+handle_find_req(#httpd{method='POST'}=Req0, Db) ->
+{ok, Req1} = mango_plugin:before_find(Req0),
+chttpd:validate_ctype(Req1, "application/json"),
+Body = chttpd:json_body_obj(Req1),
{ok, Opts0} = mango_opts:validate_find(Body),
{value, {selector, Sel}, Opts} = lists:keytake(selector, 1, Opts0),
-{ok, Resp0} = start_find_resp(Req),
+{ok, Resp0} = start_find_resp(Req1),
case run_find(Resp0, Db, Sel, Opts) of
{ok, AccOut} ->
-end_find_resp(AccOut);
+end_find_resp(Req1, AccOut);
{error, Error} ->
-chttpd:send_error(Req, Error)
+chttpd:send_error(Req1, Error)
end;
@@ -225,14 +226,15 @@ start_find_resp(Req) ->
chttpd:start_delayed_json_response(Req, 200, [], "{\"docs\":[").
-end_find_resp(Acc0) ->
-#vacc{resp=Resp00, buffer=Buf, kvs=KVs, threshold=Max} = Acc0,
+end_find_resp(Req, Acc0) ->
+#vacc{resp=Resp00, buffer=Buf, kvs=KVs0, threshold=Max} = Acc0,
{ok, Resp0} = chttpd:close_delayed_json_object(Resp00, Buf, "\r\n]", Max),
+{ok, KVs1} = mango_plugin:after_find(Req, Resp0, KVs0),
FinalAcc = lists:foldl(fun({K, V}, Acc) ->
JK = ?JSON_ENCODE(K),
JV = ?JSON_ENCODE(V),
[JV, ": ", JK, ",\r\n" | Acc]
-end, [], KVs),
+end, [], KVs1),
Chunk = lists:reverse(FinalAcc, ["}\r\n"]),
{ok, Resp1} = chttpd:send_delayed_chunk(Resp0, Chunk),
chttpd:end_delayed_json_response(Resp1).
diff --git a/src/mango/src/mango_plugin.erl b/src/mango/src/mango_plugin.erl
new file mode 100644
index 000..296a354
--- /dev/null
+++ b/src/mango/src/mango_plugin.erl
@@ -0,0 +1,43 @@
+% Licensed under the Apache License, Version 2.0 (the "License"); you may not
+% use this file except in compliance with the License. You may obtain a copy of
+% the License at
+%
+% http://www.apache.org/licenses/LICENSE-2.0
+%
+% Unless required by applicable law or agreed to in writing, software
+% distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+% WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+% License for the specific language governing permissions and limitations under
+% the License.
+
+-module(mango_plugin).
+
+-export([
+before_find/1,
+after_find/3
+]).
+
+-define(SERVICE_ID, mango).
+
+%% --
+%% API Function Definitions
+%% --
+
+before_find(HttpReq0) ->
+with_pipe(before_find, [HttpReq0]).
+
+
+after_find(HttpReq, HttpResp, Arg0) ->
+with_pipe(after_find, [HttpReq, HttpResp, Arg0]).
+
+%% --
+%% Internal Function Definitions
+%% --
+
+with_pipe(Func, Args) ->
+do_apply(Func, Args, [pipe]).
+
+
+do_apply(Func, Args, Opts) ->
+Handle = couch_epi:get_handle(?SERVICE_ID),
+couch_epi:apply(Handle, ?SERVICE_ID, Func, Args, Opts).
[couchdb] branch prototype/fdb-layer updated: Fix incorrect usage of couch_epi in mango plugin
This is an automated email from the ASF dual-hosted git repository.
iilyak pushed a commit to branch prototype/fdb-layer
in repository https://gitbox.apache.org/repos/asf/couchdb.git
The following commit(s) were added to refs/heads/prototype/fdb-layer by this
push:
new 2bb0ccd Fix incorrect usage of couch_epi in mango plugin
new aad871b Merge pull request #2775 from cloudant/mango-plugin-fixup
2bb0ccd is described below
commit 2bb0ccda2935c028b4fe7fc8edd28f0d1a7febf8
Author: ILYA Khlopotov
AuthorDate: Fri Apr 10 07:06:17 2020 -0700
Fix incorrect usage of couch_epi in mango plugin
Previously we used the value returned from couch_epi apply as is.
However it returns a list of arguments passed in the same order.
---
src/mango/src/mango_plugin.erl | 7 +--
1 file changed, 5 insertions(+), 2 deletions(-)
diff --git a/src/mango/src/mango_plugin.erl b/src/mango/src/mango_plugin.erl
index 296a354..de23f8e 100644
--- a/src/mango/src/mango_plugin.erl
+++ b/src/mango/src/mango_plugin.erl
@@ -24,11 +24,14 @@
%% --
before_find(HttpReq0) ->
-with_pipe(before_find, [HttpReq0]).
+[HttpReq1] = with_pipe(before_find, [HttpReq0]),
+{ok, HttpReq1}.
after_find(HttpReq, HttpResp, Arg0) ->
-with_pipe(after_find, [HttpReq, HttpResp, Arg0]).
+[_HttpReq, _HttpResp, Arg1] = with_pipe(after_find, [HttpReq, HttpResp,
Arg0]),
+{ok, Arg1}.
+
%% --
%% Internal Function Definitions
[couchdb] 01/01: Integrate emilio - erang linter
This is an automated email from the ASF dual-hosted git repository.
iilyak pushed a commit to branch integrate-emilio
in repository https://gitbox.apache.org/repos/asf/couchdb.git
commit 97a7c7a06e0e225b90b056de057845faa714ec0e
Author: ILYA Khlopotov
AuthorDate: Tue Feb 26 18:16:50 2019 +
Integrate emilio - erang linter
---
.gitignore| 1 +
Makefile | 6 ++-
Makefile.win | 6 ++-
bin/warnings_in_scope | 125 ++
configure | 14 ++
configure.ps1 | 14 ++
emilio.config | 20
7 files changed, 184 insertions(+), 2 deletions(-)
diff --git a/.gitignore b/.gitignore
index 60e6d14..3cfa372 100644
--- a/.gitignore
+++ b/.gitignore
@@ -45,6 +45,7 @@ src/couch/priv/couch_js/**/*.d
src/couch/priv/icu_driver/couch_icu_driver.d
src/mango/src/mango_cursor_text.nocompile
src/docs/
+src/emilio/
src/ets_lru/
src/excoveralls/
src/fauxton/
diff --git a/Makefile b/Makefile
index 97fc97c..fff1df5 100644
--- a/Makefile
+++ b/Makefile
@@ -147,6 +147,7 @@ fauxton: share/www
.PHONY: check
# target: check - Test everything
check: all python-black
+ @$(MAKE) emilio
@$(MAKE) eunit
@$(MAKE) javascript
@$(MAKE) mango-test
@@ -198,6 +199,9 @@ soak-eunit: couch
@$(REBAR) setup_eunit 2> /dev/null
while [ $$? -eq 0 ] ; do $(REBAR) -r eunit $(EUNIT_OPTS) ; done
+emilio:
+ @bin/emilio -c emilio.config src/ | bin/warnings_in_scope -s 3
+
.venv/bin/black:
@python3 -m venv .venv
@.venv/bin/pip3 install black || touch .venv/bin/black
@@ -260,7 +264,7 @@ elixir-credo: elixir-init
.PHONY: javascript
# target: javascript - Run JavaScript test suites or specific ones defined by
suites option
javascript: export COUCHDB_TEST_ADMIN_PARTY_OVERRIDE=1
-javascript:
+javascript:
@$(MAKE) devclean
@mkdir -p share/www/script/test
diff --git a/Makefile.win b/Makefile.win
index bdecc73..0fc4d91 100644
--- a/Makefile.win
+++ b/Makefile.win
@@ -134,6 +134,7 @@ fauxton: share\www
.PHONY: check
# target: check - Test everything
check: all python-black
+ @$(MAKE) emilio
@$(MAKE) eunit
@$(MAKE) javascript
@$(MAKE) mango-test
@@ -175,6 +176,9 @@ just-eunit: export ERL_AFLAGS = "-config $(shell echo
%cd%)/rel/files/eunit.conf
just-eunit:
@$(REBAR) -r eunit $(EUNIT_OPTS)
+emilio:
+ @bin\emilio -c emilio.config src\ | python.exe bin\warnings_in_scope -s
3
+
.venv/bin/black:
@python.exe -m venv .venv
@.venv\Scripts\pip3.exe install black || copy /b
.venv\Scripts\black.exe +,,
@@ -359,7 +363,7 @@ install: release
@echo .
@echo To install CouchDB into your system, copy the rel\couchdb
@echo to your desired installation location. For example:
- @echo xcopy /E rel\couchdb C:\CouchDB\
+ @echo xcopy /E rel\couchdb C:\CouchDB\
@echo .
diff --git a/bin/warnings_in_scope b/bin/warnings_in_scope
new file mode 100755
index 000..2a85421
--- /dev/null
+++ b/bin/warnings_in_scope
@@ -0,0 +1,125 @@
+#!/usr/bin/env python3
+import os
+import subprocess
+from pathlib import Path
+import optparse
+import sys
+import re
+
+def run(command, cwd=None):
+try:
+return subprocess.Popen(
+command, shell=True, cwd=cwd,
+stdout=subprocess.PIPE,
+stderr=subprocess.PIPE)
+except OSError as err:
+raise OSError("Error in command '{0}': {1}".format(command, err))
+
+def parse_location(line):
+# take substring between @@
+# take second part of it
+location = line.split(b'@@')[1].strip().split(b' ')[1]
+tokens = location.split(b',')
+if len(tokens) == 1:
+return (int(tokens[0][1:]), 1)
+elif len(tokens) == 2:
+return (int(tokens[0][1:]), int(tokens[1]))
+
+def changed_files(directory, scope):
+result = {}
+proc = run('git diff --no-prefix --unified={0}'.format(scope),
cwd=str(directory))
+file_path = None
+for line in iter(proc.stdout.readline, b''):
+if line.startswith(b'diff --git '):
+# this would be problematic if directory has space in the name
+file_name = line.split(b' ')[3].strip()
+file_path = str(directory.joinpath(str(file_name, 'utf-8')))
+result[file_path] = set()
+continue
+if line.startswith(b'@@'):
+start_pos, number_of_lines = parse_location(line)
+for line_number in range(start_pos, start_pos + number_of_lines):
+result[file_path].add(line_number)
+return result
+
+def print_changed(file_name, line_number):
+print('{0}:{1}'.format(str(file_name), str(line_nu
[couchdb] branch integrate-emilio created (now 97a7c7a)
This is an automated email from the ASF dual-hosted git repository. iilyak pushed a change to branch integrate-emilio in repository https://gitbox.apache.org/repos/asf/couchdb.git. at 97a7c7a Integrate emilio - erang linter This branch includes the following new commits: new 97a7c7a Integrate emilio - erang linter The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference.
[couchdb] branch master updated: Integrate emilio - erang linter
This is an automated email from the ASF dual-hosted git repository.
iilyak pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/couchdb.git
The following commit(s) were added to refs/heads/master by this push:
new 522627e Integrate emilio - erang linter
new 7f24add Merge pull request #1944 from cloudant/integrate-emilio
522627e is described below
commit 522627eb88d8a280b62a125cf008991438848865
Author: ILYA Khlopotov
AuthorDate: Tue Feb 26 18:16:50 2019 +
Integrate emilio - erang linter
---
.gitignore| 1 +
Makefile | 6 ++-
Makefile.win | 6 ++-
bin/warnings_in_scope | 125 ++
configure | 13 ++
configure.ps1 | 14 ++
emilio.config | 20
7 files changed, 183 insertions(+), 2 deletions(-)
diff --git a/.gitignore b/.gitignore
index 60e6d14..3cfa372 100644
--- a/.gitignore
+++ b/.gitignore
@@ -45,6 +45,7 @@ src/couch/priv/couch_js/**/*.d
src/couch/priv/icu_driver/couch_icu_driver.d
src/mango/src/mango_cursor_text.nocompile
src/docs/
+src/emilio/
src/ets_lru/
src/excoveralls/
src/fauxton/
diff --git a/Makefile b/Makefile
index 97fc97c..fff1df5 100644
--- a/Makefile
+++ b/Makefile
@@ -147,6 +147,7 @@ fauxton: share/www
.PHONY: check
# target: check - Test everything
check: all python-black
+ @$(MAKE) emilio
@$(MAKE) eunit
@$(MAKE) javascript
@$(MAKE) mango-test
@@ -198,6 +199,9 @@ soak-eunit: couch
@$(REBAR) setup_eunit 2> /dev/null
while [ $$? -eq 0 ] ; do $(REBAR) -r eunit $(EUNIT_OPTS) ; done
+emilio:
+ @bin/emilio -c emilio.config src/ | bin/warnings_in_scope -s 3
+
.venv/bin/black:
@python3 -m venv .venv
@.venv/bin/pip3 install black || touch .venv/bin/black
@@ -260,7 +264,7 @@ elixir-credo: elixir-init
.PHONY: javascript
# target: javascript - Run JavaScript test suites or specific ones defined by
suites option
javascript: export COUCHDB_TEST_ADMIN_PARTY_OVERRIDE=1
-javascript:
+javascript:
@$(MAKE) devclean
@mkdir -p share/www/script/test
diff --git a/Makefile.win b/Makefile.win
index bdecc73..0fc4d91 100644
--- a/Makefile.win
+++ b/Makefile.win
@@ -134,6 +134,7 @@ fauxton: share\www
.PHONY: check
# target: check - Test everything
check: all python-black
+ @$(MAKE) emilio
@$(MAKE) eunit
@$(MAKE) javascript
@$(MAKE) mango-test
@@ -175,6 +176,9 @@ just-eunit: export ERL_AFLAGS = "-config $(shell echo
%cd%)/rel/files/eunit.conf
just-eunit:
@$(REBAR) -r eunit $(EUNIT_OPTS)
+emilio:
+ @bin\emilio -c emilio.config src\ | python.exe bin\warnings_in_scope -s
3
+
.venv/bin/black:
@python.exe -m venv .venv
@.venv\Scripts\pip3.exe install black || copy /b
.venv\Scripts\black.exe +,,
@@ -359,7 +363,7 @@ install: release
@echo .
@echo To install CouchDB into your system, copy the rel\couchdb
@echo to your desired installation location. For example:
- @echo xcopy /E rel\couchdb C:\CouchDB\
+ @echo xcopy /E rel\couchdb C:\CouchDB\
@echo .
diff --git a/bin/warnings_in_scope b/bin/warnings_in_scope
new file mode 100755
index 000..2a85421
--- /dev/null
+++ b/bin/warnings_in_scope
@@ -0,0 +1,125 @@
+#!/usr/bin/env python3
+import os
+import subprocess
+from pathlib import Path
+import optparse
+import sys
+import re
+
+def run(command, cwd=None):
+try:
+return subprocess.Popen(
+command, shell=True, cwd=cwd,
+stdout=subprocess.PIPE,
+stderr=subprocess.PIPE)
+except OSError as err:
+raise OSError("Error in command '{0}': {1}".format(command, err))
+
+def parse_location(line):
+# take substring between @@
+# take second part of it
+location = line.split(b'@@')[1].strip().split(b' ')[1]
+tokens = location.split(b',')
+if len(tokens) == 1:
+return (int(tokens[0][1:]), 1)
+elif len(tokens) == 2:
+return (int(tokens[0][1:]), int(tokens[1]))
+
+def changed_files(directory, scope):
+result = {}
+proc = run('git diff --no-prefix --unified={0}'.format(scope),
cwd=str(directory))
+file_path = None
+for line in iter(proc.stdout.readline, b''):
+if line.startswith(b'diff --git '):
+# this would be problematic if directory has space in the name
+file_name = line.split(b' ')[3].strip()
+file_path = str(directory.joinpath(str(file_name, 'utf-8')))
+result[file_path] = set()
+continue
+if line.startswith(b'@@'):
+start_pos, number_of_lines = parse_location(line)
+for line_number in range(start_pos, sta
[couchdb] branch prototype/fdb-layer updated: Integrate emilio - erang linter
This is an automated email from the ASF dual-hosted git repository.
iilyak pushed a commit to branch prototype/fdb-layer
in repository https://gitbox.apache.org/repos/asf/couchdb.git
The following commit(s) were added to refs/heads/prototype/fdb-layer by this
push:
new 9da549e Integrate emilio - erang linter
new 6bc6f9c Merge pull request #2789 from cloudant/fdb-integrate-emilio
9da549e is described below
commit 9da549ee9a063287436c44711ce4fd99c3ebc03c
Author: ILYA Khlopotov
AuthorDate: Tue Feb 26 18:16:50 2019 +
Integrate emilio - erang linter
---
.gitignore| 1 +
Makefile | 4 ++
Makefile.win | 6 ++-
bin/warnings_in_scope | 125 ++
configure | 13 ++
configure.ps1 | 14 ++
emilio.config | 20
7 files changed, 182 insertions(+), 1 deletion(-)
diff --git a/.gitignore b/.gitignore
index 955403a..e2d3eff 100644
--- a/.gitignore
+++ b/.gitignore
@@ -47,6 +47,7 @@ src/couch/priv/couch_js/**/*.d
src/couch/priv/icu_driver/couch_icu_driver.d
src/mango/src/mango_cursor_text.nocompile
src/docs/
+src/emilio/
src/erlfdb/
src/ets_lru/
src/excoveralls/
diff --git a/Makefile b/Makefile
index ebdab22..2e3b338 100644
--- a/Makefile
+++ b/Makefile
@@ -162,6 +162,7 @@ endif
.PHONY: check
check: all
+ @$(MAKE) emilio
make eunit
apps=couch_eval,couch_expiring_cache,ctrace,couch_jobs,couch_views,fabric,mango,chttpd
make elixir
tests=test/elixir/test/basics_test.exs,test/elixir/test/replication_test.exs,test/elixir/test/map_test.exs,test/elixir/test/all_docs_test.exs,test/elixir/test/bulk_docs_test.exs
make exunit tests=src/couch_rate/test/exunit/
@@ -207,6 +208,9 @@ soak-eunit: couch
@$(REBAR) setup_eunit 2> /dev/null
while [ $$? -eq 0 ] ; do $(REBAR) -r eunit $(EUNIT_OPTS) ; done
+emilio:
+ @bin/emilio -c emilio.config src/ | bin/warnings_in_scope -s 3
+
.venv/bin/black:
@python3 -m venv .venv
@.venv/bin/pip3 install black || touch .venv/bin/black
diff --git a/Makefile.win b/Makefile.win
index 30ebe0e..885b774 100644
--- a/Makefile.win
+++ b/Makefile.win
@@ -134,6 +134,7 @@ fauxton: share\www
.PHONY: check
# target: check - Test everything
check: all python-black
+ @$(MAKE) emilio
@$(MAKE) eunit
@$(MAKE) javascript
@$(MAKE) mango-test
@@ -175,6 +176,9 @@ just-eunit: export ERL_AFLAGS = "-config $(shell echo
%cd%)/rel/files/eunit.conf
just-eunit:
@$(REBAR) -r eunit $(EUNIT_OPTS)
+emilio:
+ @bin\emilio -c emilio.config src\ | python.exe bin\warnings_in_scope -s
3
+
.venv/bin/black:
@python.exe -m venv .venv
@.venv\Scripts\pip3.exe install black || copy /b
.venv\Scripts\black.exe +,,
@@ -356,7 +360,7 @@ install: release
@echo .
@echo To install CouchDB into your system, copy the rel\couchdb
@echo to your desired installation location. For example:
- @echo xcopy /E rel\couchdb C:\CouchDB\
+ @echo xcopy /E rel\couchdb C:\CouchDB\
@echo .
diff --git a/bin/warnings_in_scope b/bin/warnings_in_scope
new file mode 100755
index 000..2a85421
--- /dev/null
+++ b/bin/warnings_in_scope
@@ -0,0 +1,125 @@
+#!/usr/bin/env python3
+import os
+import subprocess
+from pathlib import Path
+import optparse
+import sys
+import re
+
+def run(command, cwd=None):
+try:
+return subprocess.Popen(
+command, shell=True, cwd=cwd,
+stdout=subprocess.PIPE,
+stderr=subprocess.PIPE)
+except OSError as err:
+raise OSError("Error in command '{0}': {1}".format(command, err))
+
+def parse_location(line):
+# take substring between @@
+# take second part of it
+location = line.split(b'@@')[1].strip().split(b' ')[1]
+tokens = location.split(b',')
+if len(tokens) == 1:
+return (int(tokens[0][1:]), 1)
+elif len(tokens) == 2:
+return (int(tokens[0][1:]), int(tokens[1]))
+
+def changed_files(directory, scope):
+result = {}
+proc = run('git diff --no-prefix --unified={0}'.format(scope),
cwd=str(directory))
+file_path = None
+for line in iter(proc.stdout.readline, b''):
+if line.startswith(b'diff --git '):
+# this would be problematic if directory has space in the name
+file_name = line.split(b' ')[3].strip()
+file_path = str(directory.joinpath(str(file_name, 'utf-8')))
+result[file_path] = set()
+continue
+if line.startswith(b'@@'):
+start_pos, number_of_lines = parse_location(line)
+for line_number in range(start_pos, start_pos + number_of_lines):
+result[file_p
[couchdb] branch prototype/fdb-layer updated: Merge keys from rebar.config
This is an automated email from the ASF dual-hosted git repository.
iilyak pushed a commit to branch prototype/fdb-layer
in repository https://gitbox.apache.org/repos/asf/couchdb.git
The following commit(s) were added to refs/heads/prototype/fdb-layer by this
push:
new 4e2f18c Merge keys from rebar.config
new a527ad1 Merge pull request #2783 from cloudant/merge-rebar-config
4e2f18c is described below
commit 4e2f18c03e478855e927d9d0ae7bd757427c2edd
Author: ILYA Khlopotov
AuthorDate: Tue Apr 14 03:06:25 2020 -0700
Merge keys from rebar.config
This change allows creation of local `src/couch/rebar.config` and
`rebar.config`
files to set additional configuration options. This is useful for:
- disabling deprecation warnings `{nowarn_deprecated_function, MFAs}`
- control debugging in eunit tests
- `DEBUG` - `{eunit_compile_opts, [{d, DEBUG, true}]}`
- `NODEBUG` - `{eunit_compile_opts, [{d, NODEBUG, true}]}`
---
.gitignore| 1 +
rebar.config.script | 9 +++--
src/couch/.gitignore | 2 ++
src/couch/rebar.config.script | 7 ++-
4 files changed, 16 insertions(+), 3 deletions(-)
diff --git a/.gitignore b/.gitignore
index 955403a..cd46088 100644
--- a/.gitignore
+++ b/.gitignore
@@ -28,6 +28,7 @@ ebin/
erl_crash.dump
erln8.config
install.mk
+rebar.config
rel/*.config
rel/couchdb
rel/dev*
diff --git a/rebar.config.script b/rebar.config.script
index 6f9f65c..b3ea2c9 100644
--- a/rebar.config.script
+++ b/rebar.config.script
@@ -223,6 +223,11 @@ AddConfig = [
{post_hooks, [{compile, "escript support/build_js.escript"}]}
],
-C = lists:foldl(fun({K, V}, CfgAcc) ->
-lists:keystore(K, 1, CfgAcc, {K, V})
+lists:foldl(fun({K, V}, CfgAcc) ->
+case lists:keyfind(K, 1, CfgAcc) of
+{K, Existent} when is_list(Existent) andalso is_list(V) ->
+lists:keystore(K, 1, CfgAcc, {K, Existent ++ V});
+false ->
+lists:keystore(K, 1, CfgAcc, {K, V})
+end
end, CONFIG, AddConfig).
diff --git a/src/couch/.gitignore b/src/couch/.gitignore
index e1fa653..861974a 100644
--- a/src/couch/.gitignore
+++ b/src/couch/.gitignore
@@ -19,3 +19,5 @@ test/engines/log/
.rebar/
.eunit
+
+rebar.config
diff --git a/src/couch/rebar.config.script b/src/couch/rebar.config.script
index 91e24d9..80e6bd1 100644
--- a/src/couch/rebar.config.script
+++ b/src/couch/rebar.config.script
@@ -229,5 +229,10 @@ AddConfig = [
].
lists:foldl(fun({K, V}, CfgAcc) ->
-lists:keystore(K, 1, CfgAcc, {K, V})
+case lists:keyfind(K, 1, CfgAcc) of
+{K, Existent} when is_list(Existent) andalso is_list(V) ->
+lists:keystore(K, 1, CfgAcc, {K, Existent ++ V});
+false ->
+lists:keystore(K, 1, CfgAcc, {K, V})
+end
end, CONFIG, AddConfig).
[couchdb] branch prototype/fdb-layer updated: Fix typo in error message
This is an automated email from the ASF dual-hosted git repository.
iilyak pushed a commit to branch prototype/fdb-layer
in repository https://gitbox.apache.org/repos/asf/couchdb.git
The following commit(s) were added to refs/heads/prototype/fdb-layer by this
push:
new 45e0c30 Fix typo in error message
new 8554329 Merge pull request #2796 from cloudant/fix-typo
45e0c30 is described below
commit 45e0c30368cf071dc3de0df32efba95dde5abcd8
Author: ILYA Khlopotov
AuthorDate: Mon Apr 20 13:47:39 2020 -0700
Fix typo in error message
---
src/chttpd/src/chttpd_httpd_handlers.erl | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/src/chttpd/src/chttpd_httpd_handlers.erl
b/src/chttpd/src/chttpd_httpd_handlers.erl
index 79ec3db..d501159 100644
--- a/src/chttpd/src/chttpd_httpd_handlers.erl
+++ b/src/chttpd/src/chttpd_httpd_handlers.erl
@@ -514,5 +514,5 @@ not_supported(#httpd{} = Req, _Db) ->
not_implemented(#httpd{} = Req, _Db) ->
-Msg = <<"resouce is not implemented">>,
+Msg = <<"resource is not implemented">>,
chttpd:send_error(Req, 501, not_implemented, Msg).
[couchdb-documentation] branch rfc-014-pagination created (now bf3d28d)
This is an automated email from the ASF dual-hosted git repository. iilyak pushed a change to branch rfc-014-pagination in repository https://gitbox.apache.org/repos/asf/couchdb-documentation.git. at bf3d28d POST is not supported This branch includes the following new commits: new bf3d28d POST is not supported The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference.
[couchdb-documentation] 01/01: POST is not supported
This is an automated email from the ASF dual-hosted git repository. iilyak pushed a commit to branch rfc-014-pagination in repository https://gitbox.apache.org/repos/asf/couchdb-documentation.git commit bf3d28dc22f29ca80a94b5b433e107ab0743d1df Author: ILYA Khlopotov AuthorDate: Thu Apr 30 05:02:07 2020 -0700 POST is not supported --- rfcs/014-pagination.md | 9 + 1 file changed, 9 insertions(+) diff --git a/rfcs/014-pagination.md b/rfcs/014-pagination.md index 768c2a4..b6bf011 100644 --- a/rfcs/014-pagination.md +++ b/rfcs/014-pagination.md @@ -88,8 +88,17 @@ document are to be interpreted as described in "next": "/myddb/_all_docs?bookmark=12343tyekf3" ``` +## Limitations + +- The `first`/`next`/`last` keys in the response are represented as path which + includes the bookmark query key. This means the bookmark token size contributes + to total URI length and is subject to a max URL lenght (around 2000 characters). + This means storing `keys` in a bookmark is not an option. For that reason + `POST` method is not supported when pagination is enabled + ## Semantics of the implementation +- Only GET method would have pagination support - The bookmark would include information needed to ensure proper pagination without the need to repeat initial parameters of the request. - Don't use delayed responses when `bookmark` field is provided
[couchdb-erlfdb] 01/01: code:priv_dir/1 expects application name
This is an automated email from the ASF dual-hosted git repository.
iilyak pushed a commit to branch fix-monitor-path-detection
in repository https://gitbox.apache.org/repos/asf/couchdb-erlfdb.git
commit fb36e0c4c14a8118b613137e7c48a2fdad6fb2da
Author: ILYA Khlopotov
AuthorDate: Thu May 7 01:02:20 2020 -0700
code:priv_dir/1 expects application name
---
src/erlfdb_util.erl | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/src/erlfdb_util.erl b/src/erlfdb_util.erl
index e038f46..5a6224b 100644
--- a/src/erlfdb_util.erl
+++ b/src/erlfdb_util.erl
@@ -197,7 +197,7 @@ write_cluster_file(FileName, ClusterName, ClusterId,
IpAddr, Port) ->
get_monitor_path() ->
-PrivDir = case code:priv_dir(?MODULE) of
+PrivDir = case code:priv_dir(erlfdb) of
{error, _} ->
EbinDir = filename:dirname(code:which(?MODULE)),
AppPath = filename:dirname(EbinDir),
[couchdb-erlfdb] branch fix-monitor-path-detection created (now fb36e0c)
This is an automated email from the ASF dual-hosted git repository. iilyak pushed a change to branch fix-monitor-path-detection in repository https://gitbox.apache.org/repos/asf/couchdb-erlfdb.git. at fb36e0c code:priv_dir/1 expects application name This branch includes the following new commits: new fb36e0c code:priv_dir/1 expects application name The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference.
[couchdb-erlfdb] 01/01: Merge pull request #6 from apache/fix-monitor-path-detection
This is an automated email from the ASF dual-hosted git repository. iilyak pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/couchdb-erlfdb.git commit 52052a01b1b7d22087fa3da4cdb36a4571ef790c Merge: d4663cf fb36e0c Author: iilyak AuthorDate: Thu May 7 02:36:35 2020 -0700 Merge pull request #6 from apache/fix-monitor-path-detection code:priv_dir/1 expects application name src/erlfdb_util.erl | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
[couchdb-erlfdb] branch master updated (d4663cf -> 52052a0)
This is an automated email from the ASF dual-hosted git repository. iilyak pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/couchdb-erlfdb.git. from d4663cf Avoid the system key range in the eunit test random key generator. add fb36e0c code:priv_dir/1 expects application name new 52052a0 Merge pull request #6 from apache/fix-monitor-path-detection The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: src/erlfdb_util.erl | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
[couchdb-erlfdb] annotated tag v1.2.0 updated (52052a0 -> ed3d22c)
This is an automated email from the ASF dual-hosted git repository. iilyak pushed a change to annotated tag v1.2.0 in repository https://gitbox.apache.org/repos/asf/couchdb-erlfdb.git. *** WARNING: tag v1.2.0 was modified! *** from 52052a0 (commit) to ed3d22c (tag) tagging 52052a01b1b7d22087fa3da4cdb36a4571ef790c (commit) replaces v1.0.0 by ILYA Khlopotov on Thu May 7 02:42:01 2020 -0700 - Log - Fix monitor path detection in init_test_cluster_int --- No new revisions were added by this update. Summary of changes:
[couchdb] branch prototype/fdb-layer updated (7e7a3f6 -> 25bf5cb)
This is an automated email from the ASF dual-hosted git repository. iilyak pushed a change to branch prototype/fdb-layer in repository https://gitbox.apache.org/repos/asf/couchdb.git. from 7e7a3f6 add test to make sure type <<"text">> design docs are ignored (#2866) new 577be65 Re-enable ExUnit tests new 51b8cc1 Update erlfdb new 25bf5cb Merge pull request #2874 from cloudant/enable-exunit The 12432 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: Makefile| 3 ++- Makefile.win| 1 + rebar.config.script | 2 +- 3 files changed, 4 insertions(+), 2 deletions(-)
[couchdb] branch prototype/fdb-layer updated (c9cbcb4 -> d4a9723)
This is an automated email from the ASF dual-hosted git repository. iilyak pushed a change to branch prototype/fdb-layer in repository https://gitbox.apache.org/repos/asf/couchdb.git. from c9cbcb4 Fix compiler warning new 3846af7 Fix variable shadowing new 4041741 Move not_implemented check down to allow testing of validation new af502ea Add tests for legacy API before refactoring new b8a13a5 Implement pagination API new 02e4c3e Add tests for pagination API new d4a9723 Merge pull request #2870 from cloudant/pagination-api-2 The 12446 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: src/chttpd/src/chttpd_db.erl | 107 ++- src/chttpd/src/chttpd_view.erl | 162 - src/chttpd/test/exunit/pagination_test.exs | 1073 src/couch_mrview/include/couch_mrview.hrl |8 +- src/couch_mrview/src/couch_mrview_http.erl | 24 +- src/couch_views/src/couch_views_http.erl | 292 src/couch_views/src/couch_views_util.erl | 71 +- 7 files changed, 1683 insertions(+), 54 deletions(-) create mode 100644 src/chttpd/test/exunit/pagination_test.exs create mode 100644 src/couch_views/src/couch_views_http.erl
[couchdb] branch prototype/fdb-layer updated: Fix handling of limit query parameter
This is an automated email from the ASF dual-hosted git repository.
iilyak pushed a commit to branch prototype/fdb-layer
in repository https://gitbox.apache.org/repos/asf/couchdb.git
The following commit(s) were added to refs/heads/prototype/fdb-layer by this
push:
new fad3828 Fix handling of limit query parameter
new 9e3f475 Merge pull request #2896 from
cloudant/pagination-api-fix-limit
fad3828 is described below
commit fad38281474813f8479c7fb71862555b7f381755
Author: ILYA Khlopotov
AuthorDate: Wed May 20 12:50:46 2020 -0700
Fix handling of limit query parameter
---
src/chttpd/test/exunit/pagination_test.exs | 49 +++
src/couch_views/src/couch_views_http.erl | 63 +-
2 files changed, 93 insertions(+), 19 deletions(-)
diff --git a/src/chttpd/test/exunit/pagination_test.exs
b/src/chttpd/test/exunit/pagination_test.exs
index fcb8f9a..140a5dc 100644
--- a/src/chttpd/test/exunit/pagination_test.exs
+++ b/src/chttpd/test/exunit/pagination_test.exs
@@ -384,6 +384,55 @@ defmodule Couch.Test.Pagination do
assert resp.status_code == 200, "got error #{inspect(resp.body)}"
end
+test ": _all_docs?page_size=4 should respect limit", ctx do
+ %{session: session, db_name: db_name} = ctx
+
+ resp =
+Couch.Session.get(session, "/#{db_name}/_all_docs",
+ query: %{page_size: ctx.page_size, limit: ctx.page_size - 2}
+)
+
+ assert resp.status_code == 200, "got error #{inspect(resp.body)}"
+ assert length(resp.body["rows"]) == ctx.page_size - 2
+ assert not Map.has_key?(resp.body, "next")
+
+ resp =
+Couch.Session.get(session, "/#{db_name}/_all_docs",
+ query: %{page_size: ctx.page_size, limit: ctx.page_size - 1}
+)
+
+ assert resp.status_code == 200, "got error #{inspect(resp.body)}"
+ assert length(resp.body["rows"]) == ctx.page_size - 1
+ assert not Map.has_key?(resp.body, "next")
+
+ resp =
+Couch.Session.get(session, "/#{db_name}/_all_docs",
+ query: %{page_size: ctx.page_size, limit: ctx.page_size}
+)
+
+ assert resp.status_code == 200, "got error #{inspect(resp.body)}"
+ assert length(resp.body["rows"]) == ctx.page_size
+ assert not Map.has_key?(resp.body, "next")
+
+ resp =
+Couch.Session.get(session, "/#{db_name}/_all_docs",
+ query: %{page_size: ctx.page_size, limit: ctx.page_size + 1}
+)
+
+ assert resp.status_code == 200, "got error #{inspect(resp.body)}"
+ assert length(resp.body["rows"]) == ctx.page_size
+ assert Map.has_key?(resp.body, "next")
+
+ resp =
+Couch.Session.get(session, "/#{db_name}/_all_docs",
+ query: %{page_size: ctx.page_size, limit: ctx.page_size + 2}
+)
+
+ assert resp.status_code == 200, "got error #{inspect(resp.body)}"
+ assert length(resp.body["rows"]) == ctx.page_size
+ assert Map.has_key?(resp.body, "next")
+end
+
test ": _all_docs/queries should limit number of queries", ctx do
queries = %{
queries: [%{}, %{}, %{}, %{}, %{}]
diff --git a/src/couch_views/src/couch_views_http.erl
b/src/couch_views/src/couch_views_http.erl
index ae67256..b9bc2b3 100644
--- a/src/couch_views/src/couch_views_http.erl
+++ b/src/couch_views/src/couch_views_http.erl
@@ -108,7 +108,7 @@ paginated_cb({meta, Meta}, #vacc{}=VAcc) ->
paginated(Req, EtagTerm, #mrargs{page_size = PageSize} = Args, KeyFun, Fun) ->
Etag = couch_httpd:make_etag(EtagTerm),
chttpd:etag_respond(Req, Etag, fun() ->
-hd(do_paginated(PageSize, [set_limit(Args)], KeyFun, Fun))
+hd(do_paginated(PageSize, [Args], KeyFun, Fun))
end).
@@ -124,10 +124,10 @@ do_paginated(PageSize, QueriesArgs, KeyFun, Fun) when
is_list(QueriesArgs) ->
{_N, Results} = lists:foldl(fun(Args0, {Limit, Acc}) ->
case Limit > 0 of
true ->
-Args = set_limit(Args0#mrargs{page_size = Limit}),
+{OriginalLimit, Args} = set_limit(Args0#mrargs{page_size =
Limit}),
{Meta, Items} = Fun(Args),
Result = maybe_add_bookmark(
-PageSize, Args, Meta, Items, KeyFun),
+OriginalLimit, PageSize, Args, Meta, Items, KeyFun),
#{total_rows := Total} = Result,
{Limit - Total, [Result | Acc]};
false ->
@@ -143,10 +143,9 @@ do_paginated(PageSize, QueriesArgs, KeyFun, Fun) when
is_list(QueriesArgs) ->
lists:reverse(Results).
-maybe_add_bookmark(PageSize, Args0, Response, Items, KeyFun) ->
-#mrargs{page_size = Limit} = Args0,
-Args = Args0#mrargs{page_size = PageSize},
-
[couchdb] branch prototype/fdb-layer updated: Add max_bulk_get_count configuration option
This is an automated email from the ASF dual-hosted git repository.
iilyak pushed a commit to branch prototype/fdb-layer
in repository https://gitbox.apache.org/repos/asf/couchdb.git
The following commit(s) were added to refs/heads/prototype/fdb-layer by this
push:
new afbe32e Add max_bulk_get_count configuration option
new 5c7dbc4 Merge pull request #2960 from cloudant/add-max_bulk_get_count
afbe32e is described below
commit afbe32ed13c316acb188b2fdd6648e7991a04d07
Author: ILYA Khlopotov
AuthorDate: Mon Jun 22 14:04:00 2020 -0700
Add max_bulk_get_count configuration option
---
rel/overlay/etc/default.ini| 4
src/chttpd/src/chttpd.erl | 2 ++
src/chttpd/src/chttpd_db.erl | 5 +
src/chttpd/test/eunit/chttpd_db_doc_size_tests.erl | 24 +-
4 files changed, 34 insertions(+), 1 deletion(-)
diff --git a/rel/overlay/etc/default.ini b/rel/overlay/etc/default.ini
index 40a3b31..1c37765 100644
--- a/rel/overlay/etc/default.ini
+++ b/rel/overlay/etc/default.ini
@@ -59,6 +59,10 @@ max_document_size = 800 ; bytes
; returns a 413 error for the whole request
;max_bulk_docs_count = 1
;
+; Maximum number of documents in a _bulk_get request. Anything larger
+; returns a 413 error for the whole request
+;max_bulk_get_count = 1
+;
; Maximum attachment size.
; max_attachment_size = infinity
;
diff --git a/src/chttpd/src/chttpd.erl b/src/chttpd/src/chttpd.erl
index e8639ed..eca936f 100644
--- a/src/chttpd/src/chttpd.erl
+++ b/src/chttpd/src/chttpd.erl
@@ -958,6 +958,8 @@ error_info({request_entity_too_large, {attachment,
AttName}}) ->
{413, <<"attachment_too_large">>, AttName};
error_info({request_entity_too_large, {bulk_docs, Max}}) when is_integer(Max)
->
{413, <<"max_bulk_docs_count_exceeded">>, integer_to_binary(Max)};
+error_info({request_entity_too_large, {bulk_get, Max}}) when is_integer(Max) ->
+{413, <<"max_bulk_get_count_exceeded">>, integer_to_binary(Max)};
error_info({request_entity_too_large, DocID}) ->
{413, <<"document_too_large">>, DocID};
error_info({error, security_migration_updates_disabled}) ->
diff --git a/src/chttpd/src/chttpd_db.erl b/src/chttpd/src/chttpd_db.erl
index 5af98fe..fdaf4af 100644
--- a/src/chttpd/src/chttpd_db.erl
+++ b/src/chttpd/src/chttpd_db.erl
@@ -560,6 +560,11 @@ db_req(#httpd{method='POST', path_parts=[_,
<<"_bulk_get">>],
undefined ->
throw({bad_request, <<"Missing JSON list of 'docs'.">>});
Docs ->
+MaxDocs = config:get_integer("couchdb", "max_bulk_get_count",
1),
+case length(Docs) =< MaxDocs of
+true -> ok;
+false -> throw({request_entity_too_large, {bulk_get, MaxDocs}})
+end,
#doc_query_args{
options = Options
} = bulk_get_parse_doc_query(Req),
diff --git a/src/chttpd/test/eunit/chttpd_db_doc_size_tests.erl
b/src/chttpd/test/eunit/chttpd_db_doc_size_tests.erl
index 2b04050..2826cda 100644
--- a/src/chttpd/test/eunit/chttpd_db_doc_size_tests.erl
+++ b/src/chttpd/test/eunit/chttpd_db_doc_size_tests.erl
@@ -30,6 +30,7 @@ setup() ->
ok = config:set("admins", ?USER, ?b2l(Hashed), _Persist=false),
ok = config:set("couchdb", "max_document_size", "50"),
ok = config:set("couchdb", "max_bulk_docs_count", "2"),
+ok = config:set("couchdb", "max_bulk_get_count", "2"),
TmpDb = ?tempdb(),
Addr = config:get("chttpd", "bind_address", "127.0.0.1"),
Port = mochiweb_socket_server:get(chttpd, port),
@@ -41,7 +42,9 @@ teardown(Url) ->
delete_db(Url),
ok = config:delete("admins", ?USER, _Persist=false),
ok = config:delete("couchdb", "max_document_size"),
-ok = config:delete("couchdb", "max_bulk_docs_count").
+ok = config:delete("couchdb", "max_bulk_docs_count"),
+ok = config:delete("couchdb", "max_bulk_get_count"),
+ok.
create_db(Url) ->
{ok, Status, _, _} = test_request:put(Url, [?CONTENT_JSON, ?AUTH], "{}"),
@@ -70,6 +73,7 @@ all_test_() ->
fun put_single_doc/1,
fun bulk_doc/1,
fun bulk_docs_too_many_docs/1,
+fun bulk_get_too_many_docs/1,
fun put_post_doc_attach_inline/1,
fun put_multi_part_related/1,
fun post_multi_part_form/1
@@ -120,6 +124,24 @@ bulk_docs_too_many_docs(Url) ->
?_assertEqual({413, ExpectJson}, {Code, Result
[couchdb] branch prototype/fdb-layer updated: Start all required deps automatically in test
This is an automated email from the ASF dual-hosted git repository.
iilyak pushed a commit to branch prototype/fdb-layer
in repository https://gitbox.apache.org/repos/asf/couchdb.git
The following commit(s) were added to refs/heads/prototype/fdb-layer by this
push:
new 844ee87 Start all required deps automatically in test
new 7c3d752 Merge pull request #2972 from cloudant/start_deps_in_epi_tests
844ee87 is described below
commit 844ee87d8c2ecbc71a0979047bb4906bce8ddfbd
Author: ILYA Khlopotov
AuthorDate: Mon Jun 29 08:28:37 2020 -0700
Start all required deps automatically in test
---
src/couch_epi/test/eunit/couch_epi_tests.erl | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/src/couch_epi/test/eunit/couch_epi_tests.erl
b/src/couch_epi/test/eunit/couch_epi_tests.erl
index 12d8610..23b9e61 100644
--- a/src/couch_epi/test/eunit/couch_epi_tests.erl
+++ b/src/couch_epi/test/eunit/couch_epi_tests.erl
@@ -162,7 +162,8 @@ start_epi(Plugins) ->
Module
end, Plugins),
application:set_env(couch_epi, plugins, PluginsModules),
-application:start(couch_epi).
+{ok, _} = application:ensure_all_started(couch_epi),
+ok.
setup(data_file) ->
error_logger:tty(false),
[couchdb] branch prototype/fdb-layer updated: Do not fail when emilio detects errors
This is an automated email from the ASF dual-hosted git repository. iilyak pushed a commit to branch prototype/fdb-layer in repository https://gitbox.apache.org/repos/asf/couchdb.git The following commit(s) were added to refs/heads/prototype/fdb-layer by this push: new d7b0051 Do not fail when emilio detects errors new 6654b7f Merge pull request #2993 from cloudant/emilio-warnings d7b0051 is described below commit d7b005123dd31950692bd26ef246263d6dac7811 Author: ILYA Khlopotov AuthorDate: Fri Jul 10 08:18:43 2020 -0700 Do not fail when emilio detects errors --- Makefile | 2 +- Makefile.win | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/Makefile b/Makefile index eaa8b3d..2e3cc8a 100644 --- a/Makefile +++ b/Makefile @@ -210,7 +210,7 @@ soak-eunit: couch while [ $$? -eq 0 ] ; do $(REBAR) -r eunit $(EUNIT_OPTS) ; done emilio: - @bin/emilio -c emilio.config src/ | bin/warnings_in_scope -s 3 + @bin/emilio -c emilio.config src/ | bin/warnings_in_scope -s 3 || exit 0 .venv/bin/black: @python3 -m venv .venv diff --git a/Makefile.win b/Makefile.win index 265cdf3..16cf2ca 100644 --- a/Makefile.win +++ b/Makefile.win @@ -178,7 +178,7 @@ just-eunit: @$(REBAR) -r eunit $(EUNIT_OPTS) emilio: - @bin\emilio -c emilio.config src\ | python.exe bin\warnings_in_scope -s 3 + @bin\emilio -c emilio.config src\ | python.exe bin\warnings_in_scope -s 3 || exit 0 .venv/bin/black: @python.exe -m venv .venv
[couchdb] branch prototype/fdb-layer updated: Simplify using `req_body` for JSON requests
This is an automated email from the ASF dual-hosted git repository.
iilyak pushed a commit to branch prototype/fdb-layer
in repository https://gitbox.apache.org/repos/asf/couchdb.git
The following commit(s) were added to refs/heads/prototype/fdb-layer by this
push:
new ddbaf4f Simplify using `req_body` for JSON requests
new eda40cc Merge pull request #2991 from cloudant/req_body-json
ddbaf4f is described below
commit ddbaf4f0e88558ea00a9fb4e4cf9eb2365af9ef7
Author: ILYA Khlopotov
AuthorDate: Thu Jul 9 10:47:31 2020 -0700
Simplify using `req_body` for JSON requests
Currently the EPI plugins have no easy way to modify body of the document in
before request. There are complicated approaches via overiding compression
header.
This is due to the fact that `chttp:json_body/1` expects compressed body.
We can rely on the fact that `MochiReq:recv_body/1` returns binary to allow
passing of already parsed JSON terms (objects and lists).
---
src/chttpd/src/chttpd.erl | 10 ++
src/chttpd/src/chttpd_misc.erl | 3 +--
src/chttpd/test/eunit/chttpd_handlers_tests.erl | 2 +-
3 files changed, 8 insertions(+), 7 deletions(-)
diff --git a/src/chttpd/src/chttpd.erl b/src/chttpd/src/chttpd.erl
index eca936f..5a3e3fa 100644
--- a/src/chttpd/src/chttpd.erl
+++ b/src/chttpd/src/chttpd.erl
@@ -423,8 +423,7 @@ possibly_hack(#httpd{path_parts=[<<"_replicate">>]}=Req) ->
{Props0} = chttpd:json_body_obj(Req),
Props1 = fix_uri(Req, Props0, <<"source">>),
Props2 = fix_uri(Req, Props1, <<"target">>),
-put(post_body, {Props2}),
-Req;
+Req#httpd{req_body={Props2}};
possibly_hack(Req) ->
Req.
@@ -677,13 +676,16 @@ body(#httpd{mochi_req=MochiReq, req_body=ReqBody}) ->
validate_ctype(Req, Ctype) ->
couch_httpd:validate_ctype(Req, Ctype).
-json_body(Httpd) ->
+json_body(#httpd{req_body=undefined} = Httpd) ->
case body(Httpd) of
undefined ->
throw({bad_request, "Missing request body"});
Body ->
?JSON_DECODE(maybe_decompress(Httpd, Body))
-end.
+end;
+
+json_body(#httpd{req_body=ReqBody}) ->
+ReqBody.
json_body_obj(Httpd) ->
case json_body(Httpd) of
diff --git a/src/chttpd/src/chttpd_misc.erl b/src/chttpd/src/chttpd_misc.erl
index 565b121..07d5371 100644
--- a/src/chttpd/src/chttpd_misc.erl
+++ b/src/chttpd/src/chttpd_misc.erl
@@ -302,10 +302,9 @@ handle_task_status_req(#httpd{method='GET'}=Req) ->
handle_task_status_req(Req) ->
send_method_not_allowed(Req, "GET,HEAD").
-handle_replicate_req(#httpd{method='POST', user_ctx=Ctx} = Req) ->
+handle_replicate_req(#httpd{method='POST', user_ctx=Ctx, req_body=PostBody} =
Req) ->
chttpd:validate_ctype(Req, "application/json"),
%% see HACK in chttpd.erl about replication
-PostBody = get(post_body),
case replicate(PostBody, Ctx) of
{ok, {continuous, RepId}} ->
send_json(Req, 202, {[{ok, true}, {<<"_local_id">>, RepId}]});
diff --git a/src/chttpd/test/eunit/chttpd_handlers_tests.erl
b/src/chttpd/test/eunit/chttpd_handlers_tests.erl
index f3e8f5d..649d82e 100644
--- a/src/chttpd/test/eunit/chttpd_handlers_tests.erl
+++ b/src/chttpd/test/eunit/chttpd_handlers_tests.erl
@@ -70,7 +70,7 @@ request_replicate(Url, Body) ->
Headers = [{"Content-Type", "application/json"}],
Handler = {chttpd_misc, handle_replicate_req},
request(post, Url, Headers, Body, Handler, fun(Req) ->
-chttpd:send_json(Req, 200, get(post_body))
+chttpd:send_json(Req, 200, Req#httpd.req_body)
end).
request(Method, Url, Headers, Body, {M, F}, MockFun) ->
[couchdb] branch prototype/fdb-layer updated: Fix auth injection in test helper
This is an automated email from the ASF dual-hosted git repository.
iilyak pushed a commit to branch prototype/fdb-layer
in repository https://gitbox.apache.org/repos/asf/couchdb.git
The following commit(s) were added to refs/heads/prototype/fdb-layer by this
push:
new f1f8f60 Fix auth injection in test helper
new 733c357 Merge pull request #2985 from cloudant/fix-session-test-helper
f1f8f60 is described below
commit f1f8f602a4b31479688fe112c3fb3d1a69334ecc
Author: ILYA Khlopotov
AuthorDate: Mon Jul 6 08:10:57 2020 -0700
Fix auth injection in test helper
Previously the auth was injected unconidtionally. There were two problems:
1. no way to specify `basic_auth` credentials
2. no way to disable injection
Due to #2 it was impossible to use other user than hardcoded `adm` ()
in `Couch.login/3`.
---
test/elixir/lib/couch.ex | 12
1 file changed, 8 insertions(+), 4 deletions(-)
diff --git a/test/elixir/lib/couch.ex b/test/elixir/lib/couch.ex
index ed58623..a843941 100644
--- a/test/elixir/lib/couch.ex
+++ b/test/elixir/lib/couch.ex
@@ -129,7 +129,12 @@ defmodule Couch do
end
def set_auth_options(options) do
-if Keyword.get(options, :cookie) == nil do
+no_auth? = Keyword.get(options, :no_auth) == true
+cookie? = Keyword.has_key?(options, :cookie)
+basic_auth? = Keyword.has_key?(options, :basic_auth)
+if cookie? or no_auth? or basic_auth? do
+ Keyword.delete(options, :no_auth)
+else
headers = Keyword.get(options, :headers, [])
if headers[:basic_auth] != nil or headers[:authorization] != nil do
@@ -139,8 +144,6 @@ defmodule Couch do
password = System.get_env("EX_PASSWORD") || "pass"
Keyword.put(options, :basic_auth, {username, password})
end
-else
- options
end
end
@@ -177,7 +180,8 @@ defmodule Couch do
Couch.post(
"/_session",
body: %{:username => user, :password => pass},
-base_url: base_url
+base_url: base_url,
+no_auth: true
)
if Map.get(options, :expect, :success) == :success do
[couchdb-documentation] branch master updated: Rename missed_revs to missing_revs in {db}/_missing_revs result
This is an automated email from the ASF dual-hosted git repository.
iilyak pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/couchdb-documentation.git
The following commit(s) were added to refs/heads/master by this push:
new bc50598 Rename missed_revs to missing_revs in {db}/_missing_revs
result
new 46951b8 Merge pull request #570 from vmatyusGitHub/missing_rev_update
bc50598 is described below
commit bc505983712ffc4fab99bc7e6fb0fd10905a02d8
Author: Veronika
AuthorDate: Wed Jul 15 18:17:23 2020 +0200
Rename missed_revs to missing_revs in {db}/_missing_revs result
---
src/api/database/misc.rst | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/src/api/database/misc.rst b/src/api/database/misc.rst
index 7f5ef61..12e6dc6 100644
--- a/src/api/database/misc.rst
+++ b/src/api/database/misc.rst
@@ -328,7 +328,7 @@ following behavior:
Server: CouchDB (Erlang/OTP)
{
-"missed_revs":{
+"missing_revs":{
"c6114c65e295552ab1019e2b046b10e": [
"3-b06fcd1c1c9e0ec7c480ee8aa467bf3b"
]
[couchdb] branch prototype/fdb-layer updated: Add support for previous bookmark
This is an automated email from the ASF dual-hosted git repository.
iilyak pushed a commit to branch prototype/fdb-layer
in repository https://gitbox.apache.org/repos/asf/couchdb.git
The following commit(s) were added to refs/heads/prototype/fdb-layer by this
push:
new a081412 Add support for previous bookmark
new 46ee05e Merge pull request #2904 from
cloudant/support-previous-bookmark
a081412 is described below
commit a0814126bdae274f4260edd87e8b23736370885e
Author: ILYA Khlopotov
AuthorDate: Fri May 22 07:12:32 2020 -0700
Add support for previous bookmark
---
src/chttpd/test/exunit/pagination_test.exs | 47 ++
src/couch_views/src/couch_views_http.erl | 43 +++
2 files changed, 85 insertions(+), 5 deletions(-)
diff --git a/src/chttpd/test/exunit/pagination_test.exs
b/src/chttpd/test/exunit/pagination_test.exs
index 140a5dc..7fd9623 100644
--- a/src/chttpd/test/exunit/pagination_test.exs
+++ b/src/chttpd/test/exunit/pagination_test.exs
@@ -872,6 +872,18 @@ defmodule Couch.Test.Pagination do
assert Map.has_key?(resp.body, "next")
end
+test "first page should not return 'previous' bookmark", ctx do
+ resp =
+Couch.Session.get(
+ ctx.session,
+ "/#{ctx.db_name}/_design/#{ctx.ddoc_id}/_view/#{ctx.view_name}",
+ query: %{page_size: ctx.page_size, descending: ctx.descending}
+)
+
+ assert resp.status_code == 200, "got error #{inspect(resp.body)}"
+ assert not Map.has_key?(resp.body, "previous")
+end
+
test "total_rows matches the length of rows array", ctx do
resp =
Couch.Session.get(
@@ -919,6 +931,41 @@ defmodule Couch.Test.Pagination do
assert body["total_rows"] == length(body["rows"])
assert body["total_rows"] <= ctx.page_size
end
+
+test "can use 'previous' bookmark", ctx do
+ resp =
+Couch.Session.get(
+ ctx.session,
+ "/#{ctx.db_name}/_design/#{ctx.ddoc_id}/_view/#{ctx.view_name}",
+ query: %{page_size: ctx.page_size, descending: ctx.descending}
+)
+
+ assert resp.status_code == 200, "got error #{inspect(resp.body)}"
+ next_bookmark = resp.body["next"]
+
+ first_page_ids = Enum.map(resp.body["rows"], fn row -> row["id"] end)
+
+ resp =
+Couch.Session.get(
+ ctx.session,
+ "/#{ctx.db_name}/_design/#{ctx.ddoc_id}/_view/#{ctx.view_name}",
+ query: %{bookmark: next_bookmark}
+)
+
+ assert resp.status_code == 200, "got error #{inspect(resp.body)}"
+ assert Map.has_key?(resp.body, "previous")
+
+ resp =
+Couch.Session.get(
+ ctx.session,
+ "/#{ctx.db_name}/_design/#{ctx.ddoc_id}/_view/#{ctx.view_name}",
+ query: %{bookmark: resp.body["previous"]}
+)
+
+ assert resp.status_code == 200, "got error #{inspect(resp.body)}"
+ ids = Enum.map(resp.body["rows"], fn row -> row["id"] end)
+ assert first_page_ids == ids
+end
end
end
end
diff --git a/src/couch_views/src/couch_views_http.erl
b/src/couch_views/src/couch_views_http.erl
index b9bc2b3..8e12b24 100644
--- a/src/couch_views/src/couch_views_http.erl
+++ b/src/couch_views/src/couch_views_http.erl
@@ -126,8 +126,9 @@ do_paginated(PageSize, QueriesArgs, KeyFun, Fun) when
is_list(QueriesArgs) ->
true ->
{OriginalLimit, Args} = set_limit(Args0#mrargs{page_size =
Limit}),
{Meta, Items} = Fun(Args),
-Result = maybe_add_bookmark(
+Result0 = maybe_add_next_bookmark(
OriginalLimit, PageSize, Args, Meta, Items, KeyFun),
+Result = maybe_add_previous_bookmark(Args, Result0, KeyFun),
#{total_rows := Total} = Result,
{Limit - Total, [Result | Acc]};
false ->
@@ -143,8 +144,11 @@ do_paginated(PageSize, QueriesArgs, KeyFun, Fun) when
is_list(QueriesArgs) ->
lists:reverse(Results).
-maybe_add_bookmark(OriginalLimit, PageSize, Args0, Response, Items, KeyFun) ->
-#mrargs{page_size = RequestedLimit} = Args0,
+maybe_add_next_bookmark(OriginalLimit, PageSize, Args0, Response, Items,
KeyFun) ->
+#mrargs{
+page_size = RequestedLimit,
+extra = Extra
+} = Args0,
case check_completion(OriginalLimit, RequestedLimit, Items) of
{Rows, nil} ->
maps:merge(Response, #{
@@ -152,12 +156,17 @@ maybe_add_bookmark
[couchdb] branch prototype/fdb-layer updated: Fix 'first page should not return previous bookmark' test
This is an automated email from the ASF dual-hosted git repository.
iilyak pushed a commit to branch prototype/fdb-layer
in repository https://gitbox.apache.org/repos/asf/couchdb.git
The following commit(s) were added to refs/heads/prototype/fdb-layer by this
push:
new 822f278 Fix 'first page should not return previous bookmark' test
new 6875e7b Merge pull request #3027 from cloudant/fix-previous-bookmark
822f278 is described below
commit 822f2782ca3137fcbd710e3b87e95778fda5674b
Author: ILYA Khlopotov
AuthorDate: Thu Jul 23 07:13:54 2020 -0700
Fix 'first page should not return previous bookmark' test
---
src/couch_views/src/couch_views_http.erl | 10 +++---
1 file changed, 7 insertions(+), 3 deletions(-)
diff --git a/src/couch_views/src/couch_views_http.erl
b/src/couch_views/src/couch_views_http.erl
index 8e12b24..2aa9e9e 100644
--- a/src/couch_views/src/couch_views_http.erl
+++ b/src/couch_views/src/couch_views_http.erl
@@ -177,10 +177,14 @@ maybe_add_next_bookmark(OriginalLimit, PageSize, Args0,
Response, Items, KeyFun)
maybe_add_previous_bookmark(#mrargs{extra = Extra} = Args, #{rows := Rows} =
Result, KeyFun) ->
StartKey = couch_util:get_value(fk, Extra),
-case first_key(KeyFun, Rows) of
-undefined ->
+case {StartKey, first_key(KeyFun, Rows)} of
+{undefined, _} ->
Result;
-EndKey ->
+{_, undefined} ->
+Result;
+{StartKey, StartKey} ->
+Result;
+{StartKey, EndKey} ->
Bookmark = bookmark_encode(
Args#mrargs{
start_key = StartKey,
[couchdb-ibrowse] branch add-format-status created (now f4fd3f1)
This is an automated email from the ASF dual-hosted git repository. iilyak pushed a change to branch add-format-status in repository https://gitbox.apache.org/repos/asf/couchdb-ibrowse.git. at f4fd3f1 Strip sensitive data from state This branch includes the following new commits: new f4fd3f1 Strip sensitive data from state The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference.
[couchdb-ibrowse] 01/01: Strip sensitive data from state
This is an automated email from the ASF dual-hosted git repository.
iilyak pushed a commit to branch add-format-status
in repository https://gitbox.apache.org/repos/asf/couchdb-ibrowse.git
commit f4fd3f11747fd9b03fe64c94c7288653fd5f7320
Author: ILYA Khlopotov
AuthorDate: Thu Jul 23 08:15:02 2020 -0700
Strip sensitive data from state
---
src/ibrowse_http_client.erl | 39 ++-
1 file changed, 38 insertions(+), 1 deletion(-)
diff --git a/src/ibrowse_http_client.erl b/src/ibrowse_http_client.erl
index a1cf6eb..b66d0ec 100644
--- a/src/ibrowse_http_client.erl
+++ b/src/ibrowse_http_client.erl
@@ -31,7 +31,8 @@
handle_cast/2,
handle_info/2,
terminate/2,
- code_change/3
+ code_change/3,
+ format_status/2
]).
-include("ibrowse.hrl").
@@ -268,10 +269,46 @@ terminate(_Reason, State) ->
code_change(_OldVsn, State, _Extra) ->
{ok, State}.
+
+%%
+%% Function: format_status/3
+%% Purpose: Clean process state before logging
+%% Returns: key value list
+%%
+format_status(_Opt, [_PDict, State]) ->
+#state{
+reqs=Reqs,
+reply_buffer=ReplyBuf,
+recvd_headers=RCVDHeaders,
+raw_headers=RawHeaders,
+chunk_size_buffer=ChunkSizeBuf,
+cur_req=Request
+} = State,
+ScrubbedReq = Request#request{url=url_strip_password(Request#request.url)},
+Scrubbed = State#state{
+socks5_user=nil,
+socks5_password=nil,
+reqs=queue:len(Reqs),
+reply_buffer=byte_size(ReplyBuf),
+recvd_headers=lists:map(fun({K, _V}) -> K end, RCVDHeaders),
+raw_headers=byte_size(RawHeaders),
+chunk_size_buffer=byte_size(ChunkSizeBuf),
+cur_req=ScrubbedReq
+},
+[{data, [{"State",
+Scrubbed
+}]}].
+
%%
%%% Internal functions
%%
+url_strip_password(Url) ->
+re:replace(Url,
+"(http|https|socks5)://([^:]+):[^@]+@(.*)$",
+"\\1://\\2:*@\\3",
+[{return, list}]).
+
%%
%% Handles data recvd on the socket
%%
[couchdb-ibrowse] branch add-format-status updated (f4fd3f1 -> 88fcd4b)
This is an automated email from the ASF dual-hosted git repository. iilyak pushed a change to branch add-format-status in repository https://gitbox.apache.org/repos/asf/couchdb-ibrowse.git. from f4fd3f1 Strip sensitive data from state add 88fcd4b Add type tags to modified values and convert to proplist No new revisions were added by this update. Summary of changes: src/ibrowse_http_client.erl | 13 - 1 file changed, 8 insertions(+), 5 deletions(-)
[couchdb-ibrowse] branch add-format-status updated (88fcd4b -> c53f551)
This is an automated email from the ASF dual-hosted git repository. iilyak pushed a change to branch add-format-status in repository https://gitbox.apache.org/repos/asf/couchdb-ibrowse.git. from 88fcd4b Add type tags to modified values and convert to proplist add c53f551 fix typo No new revisions were added by this update. Summary of changes: src/ibrowse_http_client.erl | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
[couchdb-config] branch master updated: Refactor config:set to be able to pass sensitive flag
This is an automated email from the ASF dual-hosted git repository.
iilyak pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/couchdb-config.git
The following commit(s) were added to refs/heads/master by this push:
new b03084d Refactor config:set to be able to pass sensitive flag
new cabaf95 Merge pull request #30 from iilyak/implement-format-status
b03084d is described below
commit b03084dd3684d853cfd3c297ce9e7ee9668f71e4
Author: ILYA Khlopotov
AuthorDate: Mon Mar 2 04:25:25 2020 -0800
Refactor config:set to be able to pass sensitive flag
The sensitive flag wouldn't log the value.
---
src/config.erl | 41 -
1 file changed, 28 insertions(+), 13 deletions(-)
diff --git a/src/config.erl b/src/config.erl
index f109067..e8f7533 100644
--- a/src/config.erl
+++ b/src/config.erl
@@ -171,20 +171,22 @@ get(Section, Key, Default) when is_list(Section),
is_list(Key) ->
set(Section, Key, Value) ->
?MODULE:set(Section, Key, Value, true, nil).
+set(Sec, Key, Val, Opts) when is_binary(Sec) and is_binary(Key) ->
+?MODULE:set(binary_to_list(Sec), binary_to_list(Key), Val, Opts);
set(Section, Key, Value, Persist) when is_boolean(Persist) ->
-?MODULE:set(Section, Key, Value, Persist, nil);
-set(Section, Key, Value, Reason) ->
-?MODULE:set(Section, Key, Value, true, Reason).
-
-set(Sec, Key, Val, Persist, Reason) when is_binary(Sec) and is_binary(Key) ->
-?MODULE:set(binary_to_list(Sec), binary_to_list(Key), Val, Persist,
Reason);
-set(Section, Key, Value, Persist, Reason)
+?MODULE:set(Section, Key, Value, #{persist => Persist});
+set(Section, Key, Value, #{} = Opts)
when is_list(Section), is_list(Key), is_list(Value) ->
-gen_server:call(?MODULE, {set, Section, Key, Value, Persist, Reason},
-?TIMEOUT);
-set(_Sec, _Key, _Val, _Persist, _Reason) ->
+gen_server:call(?MODULE, {set, Section, Key, Value, Opts}, ?TIMEOUT);
+set(Section, Key, Value, Reason)
+when is_list(Section), is_list(Key), is_list(Value) ->
+?MODULE:set(Section, Key, Value, #{persist => true, reason => Reason});
+set(_Sec, _Key, _Val, _Options) ->
error(badarg).
+set(Section, Key, Value, Persist, Reason)
+when is_list(Section), is_list(Key), is_list(Value) ->
+?MODULE:set(Section, Key, Value, #{persist => Persist, reason => Reason}).
delete(Section, Key) when is_binary(Section) and is_binary(Key) ->
delete(binary_to_list(Section), binary_to_list(Key));
@@ -242,16 +244,29 @@ terminate(_Reason, _State) ->
handle_call(all, _From, Config) ->
Resp = lists:sort((ets:tab2list(?MODULE))),
{reply, Resp, Config};
-handle_call({set, Sec, Key, Val, Persist, Reason}, _From, Config) ->
+handle_call({set, Sec, Key, Val, Opts}, _From, Config) ->
+Persist = maps:get(persist, Opts, true),
+Reason = maps:get(reason, Opts, nil),
+IsSensitive = maps:get(sensitive, Opts, false),
case validate_config_update(Sec, Key, Val) of
+{error, ValidationError} when IsSensitive ->
+couch_log:error("~p: [~s] ~s = '' rejected for reason ~p",
+ [?MODULE, Sec, Key, Reason]),
+{reply, {error, ValidationError}, Config};
{error, ValidationError} ->
couch_log:error("~p: [~s] ~s = '~s' rejected for reason ~p",
[?MODULE, Sec, Key, Val, Reason]),
{reply, {error, ValidationError}, Config};
ok ->
true = ets:insert(?MODULE, {{Sec, Key}, Val}),
-couch_log:notice("~p: [~s] ~s set to ~s for reason ~p",
- [?MODULE, Sec, Key, Val, Reason]),
+case IsSensitive of
+false ->
+couch_log:notice("~p: [~s] ~s set to ~s for reason ~p",
+[?MODULE, Sec, Key, Val, Reason]);
+true ->
+couch_log:notice("~p: [~s] ~s set to '' for reason ~p",
+[?MODULE, Sec, Key, Reason])
+end,
ConfigWriteReturn = case {Persist, Config#config.write_filename} of
{true, undefined} ->
ok;
[couchdb-config] annotated tag 2.1.8 updated (cabaf95 -> 0ef49f6)
This is an automated email from the ASF dual-hosted git repository. iilyak pushed a change to annotated tag 2.1.8 in repository https://gitbox.apache.org/repos/asf/couchdb-config.git. *** WARNING: tag 2.1.8 was modified! *** from cabaf95 (commit) to 0ef49f6 (tag) tagging cabaf95d16c0dfe54c220eea6c0db6fc49b21651 (commit) replaces 2.1.6 by ILYA Khlopotov on Wed Jul 29 06:50:01 2020 -0700 - Log - Pass flags --- No new revisions were added by this update. Summary of changes:
[couchdb-ibrowse] 01/01: Merge pull request #4 from apache/add-format-status
This is an automated email from the ASF dual-hosted git repository. iilyak pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/couchdb-ibrowse.git commit a02f1feeccd8e6efda42aebf42ac926939c6215a Merge: 6c15e3e 7c3a61c Author: iilyak AuthorDate: Thu Jul 30 05:14:55 2020 -0700 Merge pull request #4 from apache/add-format-status Strip sensitive data from state src/ibrowse_http_client.erl | 42 +- 1 file changed, 41 insertions(+), 1 deletion(-)
[couchdb-ibrowse] branch add-format-status updated (c53f551 -> 7c3a61c)
This is an automated email from the ASF dual-hosted git repository. iilyak pushed a change to branch add-format-status in repository https://gitbox.apache.org/repos/asf/couchdb-ibrowse.git. discard c53f551 fix typo discard 88fcd4b Add type tags to modified values and convert to proplist discard f4fd3f1 Strip sensitive data from state add 7c3a61c Strip sensitive data from state This update added new revisions after undoing existing revisions. That is to say, some revisions that were in the old version of the branch are not in the new version. This situation occurs when a user --force pushes a change and generates a repository containing something like this: * -- * -- B -- O -- O -- O (c53f551) \ N -- N -- N refs/heads/add-format-status (7c3a61c) You should already have received notification emails for all of the O revisions, and so the following emails describe only the N revisions from the common base, B. Any revisions marked "omit" are not gone; other references still refer to them. Any revisions marked "discard" are gone forever. No new revisions were added by this update. Summary of changes: src/ibrowse_http_client.erl | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
[couchdb-ibrowse] branch master updated (6c15e3e -> a02f1fe)
This is an automated email from the ASF dual-hosted git repository. iilyak pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/couchdb-ibrowse.git. from 6c15e3e Merge pull request #2 from cloudant/handle-dead-ibrowse_lb-pids-backport add 7c3a61c Strip sensitive data from state new a02f1fe Merge pull request #4 from apache/add-format-status The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: src/ibrowse_http_client.erl | 42 +- 1 file changed, 41 insertions(+), 1 deletion(-)
[couchdb-ibrowse] annotated tag CouchDB-4.0.1-2 updated (a02f1fe -> dec5a43)
This is an automated email from the ASF dual-hosted git repository. iilyak pushed a change to annotated tag CouchDB-4.0.1-2 in repository https://gitbox.apache.org/repos/asf/couchdb-ibrowse.git. *** WARNING: tag CouchDB-4.0.1-2 was modified! *** from a02f1fe (commit) to dec5a43 (tag) tagging a02f1feeccd8e6efda42aebf42ac926939c6215a (commit) replaces CouchDB-4.0.1-1 by ILYA Khlopotov on Thu Jul 30 05:17:26 2020 -0700 - Log - Implement format_status/2 --- No new revisions were added by this update. Summary of changes:
[couchdb-ibrowse] branch master updated (6c15e3e -> a02f1fe)
This is an automated email from the ASF dual-hosted git repository. iilyak pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/couchdb-ibrowse.git. from 6c15e3e Merge pull request #2 from cloudant/handle-dead-ibrowse_lb-pids-backport add 7c3a61c Strip sensitive data from state add a02f1fe Merge pull request #4 from apache/add-format-status No new revisions were added by this update. Summary of changes: src/ibrowse_http_client.erl | 42 +- 1 file changed, 41 insertions(+), 1 deletion(-)
[couchdb-ibrowse] branch add-format-status updated (c53f551 -> 7c3a61c)
This is an automated email from the ASF dual-hosted git repository. iilyak pushed a change to branch add-format-status in repository https://gitbox.apache.org/repos/asf/couchdb-ibrowse.git. discard c53f551 fix typo discard 88fcd4b Add type tags to modified values and convert to proplist discard f4fd3f1 Strip sensitive data from state add 7c3a61c Strip sensitive data from state This update added new revisions after undoing existing revisions. That is to say, some revisions that were in the old version of the branch are not in the new version. This situation occurs when a user --force pushes a change and generates a repository containing something like this: * -- * -- B -- O -- O -- O (c53f551) \ N -- N -- N refs/heads/add-format-status (7c3a61c) You should already have received notification emails for all of the O revisions, and so the following emails describe only the N revisions from the common base, B. Any revisions marked "omit" are not gone; other references still refer to them. Any revisions marked "discard" are gone forever. No new revisions were added by this update. Summary of changes: src/ibrowse_http_client.erl | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
[couchdb] branch prototype/fdb-layer updated (46cff24 -> 6ff2f41)
This is an automated email from the ASF dual-hosted git repository. iilyak pushed a change to branch prototype/fdb-layer in repository https://gitbox.apache.org/repos/asf/couchdb.git. from 46cff24 Merge pull request #3045 from apache/prototype/fdb-layer-ebtree-enhancements new e1b4259 Strip last_msg from logs new 97e7a95 Add format_status/2 callback in gen_server implementations new 52d5327 Do not log sensitive data during _cluster_setup new 8360026 Do not log admin credentials new e4555a4 Update config app new 6ff2f41 Merge pull request #3031 from cloudant/clean-up-logs The 12601 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: rebar.config.script| 4 +- src/chttpd/src/chttpd_node.erl | 4 +- src/couch/include/couch_db.hrl | 3 + src/couch/src/couch_lru.erl| 5 +- src/couch/src/couch_multidb_changes.erl| 14 ++- src/couch/src/couch_native_process.erl | 17 ++- src/couch/src/couch_proc_manager.erl | 16 ++- src/couch/src/couch_server.erl | 6 +- src/couch/src/couch_stream.erl | 16 ++- src/couch/src/couch_work_queue.erl | 25 - src/couch_index/src/couch_index.erl| 19 +++- src/couch_jobs/src/couch_jobs_notifier.erl | 22 +++- src/couch_js/src/couch_js_native_process.erl | 18 +++- src/couch_js/src/couch_js_proc_manager.erl | 16 ++- src/couch_log/src/couch_log_config.erl | 11 +- src/couch_log/src/couch_log_config_dyn.erl | 3 +- src/couch_log/src/couch_log_formatter.erl | 24 - src/couch_log/src/couch_log_sup.erl| 2 + src/couch_log/test/eunit/couch_log_config_test.erl | 37 ++- .../test/eunit/couch_log_formatter_test.erl| 114 - src/couch_mrview/src/couch_mrview_index.erl| 12 +++ src/couch_peruser/src/couch_peruser.erl| 13 ++- .../src/couch_replicator_auth_session.erl | 2 +- .../src/couch_replicator_httpc_pool.erl| 14 ++- src/couch_stats/src/couch_stats_aggregator.erl | 17 ++- src/couch_views/src/couch_views_server.erl | 17 ++- src/ddoc_cache/src/ddoc_cache_entry.erl| 21 +++- src/dreyfus/src/dreyfus_index.erl | 26 - src/fabric/src/fabric2_txids.erl | 15 ++- src/global_changes/src/global_changes_server.erl | 11 +- src/ken/src/ken_server.erl | 16 ++- src/setup/src/setup.erl| 2 +- src/setup/src/setup_httpd.erl | 17 +-- 33 files changed, 511 insertions(+), 48 deletions(-)
[couchdb] branch prototype/fdb-layer updated (46cff24 -> 6ff2f41)
This is an automated email from the ASF dual-hosted git repository. iilyak pushed a change to branch prototype/fdb-layer in repository https://gitbox.apache.org/repos/asf/couchdb.git. from 46cff24 Merge pull request #3045 from apache/prototype/fdb-layer-ebtree-enhancements new e1b4259 Strip last_msg from logs new 97e7a95 Add format_status/2 callback in gen_server implementations new 52d5327 Do not log sensitive data during _cluster_setup new 8360026 Do not log admin credentials new e4555a4 Update config app new 6ff2f41 Merge pull request #3031 from cloudant/clean-up-logs The 12601 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: rebar.config.script| 4 +- src/chttpd/src/chttpd_node.erl | 4 +- src/couch/include/couch_db.hrl | 3 + src/couch/src/couch_lru.erl| 5 +- src/couch/src/couch_multidb_changes.erl| 14 ++- src/couch/src/couch_native_process.erl | 17 ++- src/couch/src/couch_proc_manager.erl | 16 ++- src/couch/src/couch_server.erl | 6 +- src/couch/src/couch_stream.erl | 16 ++- src/couch/src/couch_work_queue.erl | 25 - src/couch_index/src/couch_index.erl| 19 +++- src/couch_jobs/src/couch_jobs_notifier.erl | 22 +++- src/couch_js/src/couch_js_native_process.erl | 18 +++- src/couch_js/src/couch_js_proc_manager.erl | 16 ++- src/couch_log/src/couch_log_config.erl | 11 +- src/couch_log/src/couch_log_config_dyn.erl | 3 +- src/couch_log/src/couch_log_formatter.erl | 24 - src/couch_log/src/couch_log_sup.erl| 2 + src/couch_log/test/eunit/couch_log_config_test.erl | 37 ++- .../test/eunit/couch_log_formatter_test.erl| 114 - src/couch_mrview/src/couch_mrview_index.erl| 12 +++ src/couch_peruser/src/couch_peruser.erl| 13 ++- .../src/couch_replicator_auth_session.erl | 2 +- .../src/couch_replicator_httpc_pool.erl| 14 ++- src/couch_stats/src/couch_stats_aggregator.erl | 17 ++- src/couch_views/src/couch_views_server.erl | 17 ++- src/ddoc_cache/src/ddoc_cache_entry.erl| 21 +++- src/dreyfus/src/dreyfus_index.erl | 26 - src/fabric/src/fabric2_txids.erl | 15 ++- src/global_changes/src/global_changes_server.erl | 11 +- src/ken/src/ken_server.erl | 16 ++- src/setup/src/setup.erl| 2 +- src/setup/src/setup_httpd.erl | 17 +-- 33 files changed, 511 insertions(+), 48 deletions(-)
[couchdb] branch prototype/fdb-layer updated: Fix ordering of page_size based pagination for views
This is an automated email from the ASF dual-hosted git repository.
iilyak pushed a commit to branch prototype/fdb-layer
in repository https://gitbox.apache.org/repos/asf/couchdb.git
The following commit(s) were added to refs/heads/prototype/fdb-layer by this
push:
new 6235f0f Fix ordering of page_size based pagination for views
new 07e179f Merge pull request #3094 from cloudant/use-key_docid
6235f0f is described below
commit 6235f0f92b27f755cfea3cd6ab8464a71ca23ecb
Author: ILYA Khlopotov
AuthorDate: Fri Aug 21 08:32:03 2020 -0700
Fix ordering of page_size based pagination for views
The pagination relied on id of the document. However for views it should use
combination of key and id.
---
src/chttpd/src/chttpd_db.erl | 8 +-
src/chttpd/src/chttpd_view.erl | 8 +-
src/chttpd/test/exunit/pagination_test.exs | 242 +++--
src/couch_views/src/couch_views_http.erl | 31 ++--
4 files changed, 263 insertions(+), 26 deletions(-)
diff --git a/src/chttpd/src/chttpd_db.erl b/src/chttpd/src/chttpd_db.erl
index 8acccb4..c458cba 100644
--- a/src/chttpd/src/chttpd_db.erl
+++ b/src/chttpd/src/chttpd_db.erl
@@ -864,7 +864,9 @@ paginate_multi_all_docs_view(Req, Db, OP, Args0, Queries) ->
ArgQueries = chttpd_view:parse_queries(Req, Args1, Queries, fun(QArgs) ->
set_namespace(OP, QArgs)
end),
-KeyFun = fun({Props}) -> couch_util:get_value(id, Props) end,
+KeyFun = fun({Props}) ->
+{couch_util:get_value(id, Props), undefined}
+end,
#mrargs{page_size = PageSize} = Args0,
#httpd{path_parts = Parts} = Req,
UpdateSeq = fabric2_db:get_update_seq(Db),
@@ -911,7 +913,9 @@ paginate_all_docs_view(Req, Db, Args0, OP) ->
Args1 = Args0#mrargs{view_type=map},
Args2 = chttpd_view:validate_args(Req, Args1),
Args3 = set_namespace(OP, Args2),
-KeyFun = fun({Props}) -> couch_util:get_value(id, Props) end,
+KeyFun = fun({Props}) ->
+{couch_util:get_value(id, Props), undefined}
+end,
#httpd{path_parts = Parts} = Req,
UpdateSeq = fabric2_db:get_update_seq(Db),
EtagTerm = {Parts, UpdateSeq, Args3},
diff --git a/src/chttpd/src/chttpd_view.erl b/src/chttpd/src/chttpd_view.erl
index 8e2a08e..8d40101 100644
--- a/src/chttpd/src/chttpd_view.erl
+++ b/src/chttpd/src/chttpd_view.erl
@@ -58,7 +58,9 @@ paginate_multi_query_view(Req, Db, DDoc, ViewName, Args0,
Queries) ->
ArgQueries = parse_queries(Req, Args0, Queries, fun(QueryArg) ->
couch_mrview_util:set_view_type(QueryArg, ViewName, Views)
end),
-KeyFun = fun({Props}) -> couch_util:get_value(id, Props) end,
+KeyFun = fun({Props}) ->
+{couch_util:get_value(id, Props), couch_util:get_value(key, Props)}
+end,
#mrargs{page_size = PageSize} = Args0,
#httpd{path_parts = Parts} = Req,
UpdateSeq = fabric2_db:get_update_seq(Db),
@@ -100,7 +102,9 @@ stream_fabric_query_view(Db, Req, DDoc, ViewName, Args) ->
paginate_fabric_query_view(Db, Req, DDoc, ViewName, Args0) ->
-KeyFun = fun({Props}) -> couch_util:get_value(id, Props) end,
+KeyFun = fun({Props}) ->
+{couch_util:get_value(id, Props), couch_util:get_value(key, Props)}
+end,
#httpd{path_parts = Parts} = Req,
UpdateSeq = fabric2_db:get_update_seq(Db),
ETagTerm = {Parts, UpdateSeq, Args0},
diff --git a/src/chttpd/test/exunit/pagination_test.exs
b/src/chttpd/test/exunit/pagination_test.exs
index 7fd9623..6544017 100644
--- a/src/chttpd/test/exunit/pagination_test.exs
+++ b/src/chttpd/test/exunit/pagination_test.exs
@@ -68,6 +68,25 @@ defmodule Couch.Test.Pagination do
%{view_name: "all", ddoc_id: ddoc_id}
end
+ defp with_same_key_docs(context) do
+assert Map.has_key?(context, :n_docs), "Please define '@describetag
n_docs: 10'"
+
+docs =
+ for id <- 1..context.n_docs do
+str_id = docid(id)
+%{"_id" => str_id, "integer" => id, "string" => docid(div(id,
context.page_size))}
+ end
+
+docs =
+ docs
+ |> Enum.map(fn doc ->
+created_doc = create_doc(context.session, context.db_name, doc)
+Map.merge(doc, created_doc)
+ end)
+
+%{docs: docs}
+ end
+
defp all_docs(context) do
assert Map.has_key?(context, :page_size), "Please define '@describetag
page_size: 4'"
@@ -86,6 +105,50 @@ defmodule Couch.Test.Pagination do
}
end
+ defp paginate_queries(context, opts) do
+paginate_queries(context, [], opts)
+ end
+
+ defp paginate_queries(context, acc, opts) do
+{paginate_opts, client_opts} = Keyword.split(opts, [:url, :direction])
+
+resp =
+ Couch.Session.post(context.session, Keyword.get(paginate_opts, :url),
client_opts)
+
+results = resp.body["results"]
+view_url = String.replace_suffix(Keyword.get
[couchdb-documentation] branch master updated: Fix typo in query-servers.rst
This is an automated email from the ASF dual-hosted git repository. iilyak pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/couchdb-documentation.git The following commit(s) were added to refs/heads/master by this push: new 2b1547f Fix typo in query-servers.rst new a78d0bc Merge pull request #589 from bessbd/patch-5 2b1547f is described below commit 2b1547f0d76a49723f5ec45dc0b8001d7c1f1649 Author: Bessenyei Balázs Donát AuthorDate: Mon Sep 14 11:02:56 2020 +0200 Fix typo in query-servers.rst --- src/config/query-servers.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/config/query-servers.rst b/src/config/query-servers.rst index c770227..eb64f55 100644 --- a/src/config/query-servers.rst +++ b/src/config/query-servers.rst @@ -148,7 +148,7 @@ Native Erlang Query Server Due to security restrictions, the Erlang query server is disabled by default. -Unlike the JavaScript query server, the Erlang one does not runs in a +Unlike the JavaScript query server, the Erlang one does not run in a sandbox mode. This means that Erlang code has full access to your OS, file system and network, which may lead to security issues. While Erlang functions are faster than JavaScript ones, you need to be careful
[couchdb-documentation] 01/01: Merge pull request #593 from apache/typo
This is an automated email from the ASF dual-hosted git repository. iilyak pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/couchdb-documentation.git commit f6e3670108ff279dcd300ba5260a8c33445a3958 Merge: b59eec8 503aca4 Author: iilyak AuthorDate: Mon Sep 21 06:20:04 2020 -0700 Merge pull request #593 from apache/typo Fix spelling/typo mistake in /_up endpoint description src/api/server/common.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
[couchdb-documentation] branch master updated (b59eec8 -> f6e3670)
This is an automated email from the ASF dual-hosted git repository. iilyak pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/couchdb-documentation.git. from b59eec8 Bump version number to 3.1.1 (#592) add 503aca4 Fix spelling/typo mistake in /_up endpoint description new f6e3670 Merge pull request #593 from apache/typo The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: src/api/server/common.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
[couchdb-documentation] 01/02: Document pagination API
This is an automated email from the ASF dual-hosted git repository.
iilyak pushed a commit to branch pagination
in repository https://gitbox.apache.org/repos/asf/couchdb-documentation.git
commit d0c7dba9775a35897f17584db600171be3dba833
Author: ILYA Khlopotov
AuthorDate: Mon May 25 07:29:04 2020 -0700
Document pagination API
---
src/api/database/bulk-api.rst | 3 +++
src/api/ddoc/views.rst | 45 +
src/ddocs/views/pagination.rst | 56 --
3 files changed, 102 insertions(+), 2 deletions(-)
diff --git a/src/api/database/bulk-api.rst b/src/api/database/bulk-api.rst
index 14e72ba..15cc623 100644
--- a/src/api/database/bulk-api.rst
+++ b/src/api/database/bulk-api.rst
@@ -332,6 +332,9 @@ Sending multiple queries to a database
:param db: Database name
+:query number page_size: Specify the number of queries in the result.
+ Enables paginated reply for included queries.
+
:`
+based on ``offset``, ``limit`` and ``skip``.
+
+The user can request paginated mode by setting ``page_size`` query parameter.
When
+pagination is enabled the response would include ``next`` and ``previous``
tokens.
+Which can be used to retrieve next and previous page of the results.
+The maximum possible page size is configured via ``request_limits``
+section in one of the ``ini`` files.
+
+.. code-block:: ini
+
+[request_limits]
+_all_docs = 5000
+_all_docs/queries = 5000
+_all_dbs = 5000
+_view = 2500
+_view/queries = 2500
+
+
+Note that ``page_size`` for :ref:`Multiple queries
`
+endpoints limits number of queries the user can submit in the body of the
request.
+
+Compatibility notes
+---
+
+- ``page_size`` is forbidden in the query object passed in ``queries`` array \
+ submitted via :post:`/{db}/_design/{ddoc}/_view/{view}/queries` request.
+
+- ``keys`` propery is incompatible with ``page_size``.
+
+- value for ``skip`` property has to be in range of ``[0..]``.
+
+- ``bookmark`` is incompatible with any other query parameters
diff --git a/src/ddocs/views/pagination.rst b/src/ddocs/views/pagination.rst
index 6cae038..867f66c 100644
--- a/src/ddocs/views/pagination.rst
+++ b/src/ddocs/views/pagination.rst
@@ -123,8 +123,60 @@ Or in a pseudo-JavaScript snippet:
page.display_link('next');
}
-Paging
-==
+
+Paging (New Way)
+
+
+.. versionadded:: 4.0
+
+To get the first five rows from the view result, you use the ``?page_size=5``
+query parameter::
+
+curl -X GET
http://127.0.0.1:5984/artists/_design/artists/_view/by-name?page_size=5
+
+The result:
+
+.. code-block:: javascript
+
+{"total_rows":11,"offset":0,"rows":[
+{"id":"a0746072bba60a62b01209f467ca4fe2","key":"Biffy
Clyro","value":null},
+{"id":"b47d82284969f10cd1b6ea460ad62d00","key":"Foo
Fighters","value":null},
+{"id":"45ccde324611f86ad4932555dea7fce0","key":"Tenacious
D","value":null},
+{"id":"d7ab24bb3489a9010c7d1a2087a4a9e4","key":"Future of the
Left","value":null},
+{"id":"ad2f85ef87f5a9a65db5b3a75a03cd82","key":"Helmet","value":null}
+],
+"next": "an encoded string representing bookmark pointing to next page
of results"
+}
+
+By presence of ``next`` property we can determine if there are more pages to
display.
+
+To get next page from CouchDB we would use::
+
+curl -X GET
'http://127.0.0.1:5984/artists/_design/artists/_view/by-name?bookmark='
+
+The result:
+
+.. code-block:: javascript
+
+{"total_rows":11,"offset":5,"rows":[
+{"id":"a2f31cfa68118a6ae9d35444fcb1a3cf","key":"Nirvana","value":null},
+{"id":"67373171d0f626b811bdc34e92e77901","key":"Kerub","value":null},
+{"id":"3e1b84630c384f6aef1a5c50a81e4a34","key":"Perfect
Circle","value":null},
+{"id":"84a371a7b8414237fad1b6aaf68cd16a","key":"Queens of the Stone
Age",
+"value":null},
+
{"id":"dcdaf08242a4be7da1a36e25f4f0b022","key":"Silverchair","value":null}
+],
+"previous": "an encoded string representing bookmark pointing to
previous page of results",
+"next": "an encoded string representing bookmark pointing to next page
of results",
+}
+
+The ``previous`` property can be used to get the previous page of the results
if need to::
+
+curl -X GET
'http://127.0.0.1:5984/artists/_design/artists/_view/by-name?bookmark='
+
+
+Paging (Old Way)
+
To get the first five rows from the view result, you use the ``?limit=5``
query parameter::
[couchdb-documentation] branch pagination created (now e0eae75)
This is an automated email from the ASF dual-hosted git repository. iilyak pushed a change to branch pagination in repository https://gitbox.apache.org/repos/asf/couchdb-documentation.git. at e0eae75 Address problems identified durring review This branch includes the following new commits: new d0c7dba Document pagination API new e0eae75 Address problems identified durring review The 2 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference.
[couchdb-documentation] 02/02: Address problems identified durring review
This is an automated email from the ASF dual-hosted git repository.
iilyak pushed a commit to branch pagination
in repository https://gitbox.apache.org/repos/asf/couchdb-documentation.git
commit e0eae75fe698148e75c3a50c6e2b245621d7d86e
Author: ILYA Khlopotov
AuthorDate: Mon Sep 21 06:47:19 2020 -0700
Address problems identified durring review
---
src/api/database/bulk-api.rst | 2 +-
src/api/ddoc/views.rst | 13 ++---
src/ddocs/views/pagination.rst | 24 +++-
3 files changed, 18 insertions(+), 21 deletions(-)
diff --git a/src/api/database/bulk-api.rst b/src/api/database/bulk-api.rst
index 15cc623..4f2af0f 100644
--- a/src/api/database/bulk-api.rst
+++ b/src/api/database/bulk-api.rst
@@ -332,7 +332,7 @@ Sending multiple queries to a database
:param db: Database name
-:query number page_size: Specify the number of queries in the result.
+:query number page_size: The number of queries in the result.
Enables paginated reply for included queries.
:`
+CouchDB supports token-based pagination, which is an alternative to
+:ref:`pagination recipe `
based on ``offset``, ``limit`` and ``skip``.
The user can request paginated mode by setting ``page_size`` query parameter.
When
@@ -946,7 +946,6 @@ section in one of the ``ini`` files.
_view = 2500
_view/queries = 2500
-
Note that ``page_size`` for :ref:`Multiple queries
`
endpoints limits number of queries the user can submit in the body of the
request.
@@ -956,8 +955,8 @@ Compatibility notes
- ``page_size`` is forbidden in the query object passed in ``queries`` array \
submitted via :post:`/{db}/_design/{ddoc}/_view/{view}/queries` request.
-- ``keys`` propery is incompatible with ``page_size``.
+- The ``keys`` propery is incompatible with ``page_size``.
-- value for ``skip`` property has to be in range of ``[0..]``.
+- value for ``skip`` property must be in range of ``[0..]``.
- ``bookmark`` is incompatible with any other query parameters
diff --git a/src/ddocs/views/pagination.rst b/src/ddocs/views/pagination.rst
index 867f66c..bf5a736 100644
--- a/src/ddocs/views/pagination.rst
+++ b/src/ddocs/views/pagination.rst
@@ -123,9 +123,8 @@ Or in a pseudo-JavaScript snippet:
page.display_link('next');
}
-
-Paging (New Way)
-
+`page_size` based paging (recommended)
+==
.. versionadded:: 4.0
@@ -134,7 +133,7 @@ query parameter::
curl -X GET
http://127.0.0.1:5984/artists/_design/artists/_view/by-name?page_size=5
-The result:
+The result::
.. code-block:: javascript
@@ -148,13 +147,13 @@ The result:
"next": "an encoded string representing bookmark pointing to next page
of results"
}
-By presence of ``next`` property we can determine if there are more pages to
display.
+By presence of ``next`` property we can determine if there are more pages to
display.
-To get next page from CouchDB we would use::
+To get the next page from CouchDB we would use::
curl -X GET
'http://127.0.0.1:5984/artists/_design/artists/_view/by-name?bookmark='
-The result:
+The result::
.. code-block:: javascript
@@ -170,13 +169,12 @@ The result:
"next": "an encoded string representing bookmark pointing to next page
of results",
}
-The ``previous`` property can be used to get the previous page of the results
if need to::
+Likewise, the ``previous`` property can be used to get the previous page of
the results::
curl -X GET
'http://127.0.0.1:5984/artists/_design/artists/_view/by-name?bookmark='
-
-Paging (Old Way)
-
+Skip based paging
+=
To get the first five rows from the view result, you use the ``?limit=5``
query parameter::
@@ -243,8 +241,8 @@ straightforward:
return page != last_page;
}
-Paging (Alternate Method)
-=
+`startkey` based paging
+===
The method described above performed poorly with large skip values until
CouchDB 1.2. Additionally, some use cases may call for the following
[couchdb] branch main updated: Fix semantics of total_rows
This is an automated email from the ASF dual-hosted git repository.
iilyak pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/couchdb.git
The following commit(s) were added to refs/heads/main by this push:
new c0efba3 Fix semantics of total_rows
new cc9e261 Merge pull request #3227 from
cloudant/fix-total_rows-semantics
c0efba3 is described below
commit c0efba32fa1ced66b5baf63ef1ab0dc6188974c5
Author: ILYA Khlopotov
AuthorDate: Thu Oct 22 08:58:47 2020 -0700
Fix semantics of total_rows
The `total_rows` field suppose to be a number of documents in the
database/view.
See https://docs.couchdb.org/en/stable/api/ddoc/views.html.
When new pagination API was introduced the meaning of `total_rows` field was
changed to number of rows in the query results. The PR reverts this
accidental change.
---
src/chttpd/test/exunit/pagination_test.exs | 56 +-
src/couch_views/src/couch_views_http.erl | 17 -
2 files changed, 23 insertions(+), 50 deletions(-)
diff --git a/src/chttpd/test/exunit/pagination_test.exs
b/src/chttpd/test/exunit/pagination_test.exs
index 6544017..4e0a5d6 100644
--- a/src/chttpd/test/exunit/pagination_test.exs
+++ b/src/chttpd/test/exunit/pagination_test.exs
@@ -255,7 +255,7 @@ defmodule Couch.Test.Pagination do
@describetag descending: descending
setup [:with_session, :random_db, :with_docs]
- test "total_rows matches the length of rows array", ctx do
+ test "total_rows matches the number of docs", ctx do
resp =
Couch.Session.get(ctx.session, "/#{ctx.db_name}/_all_docs",
query: %{descending: ctx.descending}
@@ -263,7 +263,7 @@ defmodule Couch.Test.Pagination do
assert resp.status_code == 200, "got error #{inspect(resp.body)}"
body = resp.body
-assert body["total_rows"] == length(body["rows"])
+assert body["total_rows"] == ctx.n_docs
end
test "the rows are correctly sorted", ctx do
@@ -371,7 +371,7 @@ defmodule Couch.Test.Pagination do
@describetag page_size: 4
setup [:with_session, :random_db, :with_view, :with_docs]
-test "total_rows matches the length of rows array", ctx do
+test "total_rows matches the number of documents in view", ctx do
resp =
Couch.Session.get(
ctx.session,
@@ -381,7 +381,7 @@ defmodule Couch.Test.Pagination do
assert resp.status_code == 200, "got error #{inspect(resp.body)}"
body = resp.body
- assert body["total_rows"] == length(body["rows"])
+ assert body["total_rows"] == ctx.n_docs
end
end
@@ -593,14 +593,9 @@ defmodule Couch.Test.Pagination do
assert Map.has_key?(body, "next")
end
-test "total_rows matches the length of rows array", ctx do
+test "total_rows matches the number of documents", ctx do
body = ctx.response
- assert body["total_rows"] == length(body["rows"])
-end
-
-test "total_rows matches the requested page_size", ctx do
- body = ctx.response
- assert body["total_rows"] == ctx.page_size
+ assert body["total_rows"] == ctx.n_docs
end
test "can use 'next' bookmark to get remaining results", ctx do
@@ -613,8 +608,8 @@ defmodule Couch.Test.Pagination do
assert resp.status_code == 200, "got error #{inspect(resp.body)}"
body = resp.body
- assert body["total_rows"] == length(body["rows"])
- assert body["total_rows"] <= ctx.page_size
+ assert body["total_rows"] == ctx.n_docs
+ assert length(body["rows"]) <= ctx.page_size
end
end
@@ -721,7 +716,7 @@ defmodule Couch.Test.Pagination do
test "final page doesn't include 'next' bookmark", ctx do
assert not Map.has_key?(ctx.response, "next")
- assert ctx.response["total_rows"] == rem(ctx.n_docs, ctx.page_size)
+ assert length(ctx.response["rows"]) == rem(ctx.n_docs, ctx.page_size)
end
test "each but last page has page_size rows", ctx do
@@ -768,14 +763,9 @@ defmodule Couch.Test.Pagination do
assert not Map.has_key?(body, "next")
end
- test "total_rows matches the length of rows array", ctx do
-body = ctx.response
-assert body["total_rows"] == length(body["rows"])
- end
-
- test "total_rows less than the requested page_size", ctx do
+ test "total_rows matches the number of documents",
[couchdb] branch main updated: Use `req_body` field if present
This is an automated email from the ASF dual-hosted git repository.
iilyak pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/couchdb.git
The following commit(s) were added to refs/heads/main by this push:
new 1be2312 Use `req_body` field if present
new 5cfa4a8 Merge pull request #3268 from cloudant/reuse-json-body
1be2312 is described below
commit 1be23124b7a867a966ce39ec66231bc2d2779ca9
Author: ILYA Khlopotov
AuthorDate: Mon Nov 16 04:01:21 2020 -0800
Use `req_body` field if present
When we call `couch_httpd:json_body/1` we can have `req_body` already set.
In this case we should return the field as is without any attempt to
decompress or decode it. This PR brings the approach we use in `chttpd`
into `couch_httpd`.
---
src/couch/src/couch_httpd.erl | 7 +--
1 file changed, 5 insertions(+), 2 deletions(-)
diff --git a/src/couch/src/couch_httpd.erl b/src/couch/src/couch_httpd.erl
index 8f7fedd..53d14d7 100644
--- a/src/couch/src/couch_httpd.erl
+++ b/src/couch/src/couch_httpd.erl
@@ -599,13 +599,16 @@ body(#httpd{mochi_req=MochiReq, req_body=undefined}) ->
body(#httpd{req_body=ReqBody}) ->
ReqBody.
-json_body(Httpd) ->
+json_body(#httpd{req_body=undefined} = Httpd) ->
case body(Httpd) of
undefined ->
throw({bad_request, "Missing request body"});
Body ->
?JSON_DECODE(maybe_decompress(Httpd, Body))
-end.
+end;
+
+json_body(#httpd{req_body=ReqBody}) ->
+ReqBody.
json_body_obj(Httpd) ->
case json_body(Httpd) of
[couchdb] branch main updated (8c21567 -> 0b98b3b)
This is an automated email from the ASF dual-hosted git repository. iilyak pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/couchdb.git. from 8c21567 Bump erlfdb to v1.2.3 new 0eb1043 `q` and `n` are not needed for FDB based CouchDB new edd320e Disable TextIndex test since it is not implemeneted yet new 0b98b3b Merge pull request #3274 from cloudant/fix-mango-tests The 12935 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: src/mango/test/06-basic-text-test.py | 4 src/mango/test/mango.py | 2 +- 2 files changed, 5 insertions(+), 1 deletion(-)
[couchdb] branch 3.x updated (87a6b1a -> f611ffa)
This is an automated email from the ASF dual-hosted git repository. iilyak pushed a change to branch 3.x in repository https://gitbox.apache.org/repos/asf/couchdb.git. from 87a6b1a Merge pull request #3271 from apache/couch_att_erroneous_md5_mismatch new 76898af Use `req_body` field if present new d7c3524 Simplify using `req_body` for JSON requests new fb7eb38 'req_body' suppose to hold JSON obj new f611ffa Merge pull request #3277 from cloudant/req_body-json-3.x The 12308 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: src/chttpd/src/chttpd.erl | 10 ++ src/chttpd/src/chttpd_misc.erl | 3 +-- src/chttpd/src/chttpd_show.erl | 14 ++ src/chttpd/test/eunit/chttpd_handlers_tests.erl | 2 +- src/couch/src/couch_httpd.erl | 7 +-- 5 files changed, 19 insertions(+), 17 deletions(-)
[couchdb] branch specify-elixir-tests created (now eb2a5cf)
This is an automated email from the ASF dual-hosted git repository. iilyak pushed a change to branch specify-elixir-tests in repository https://gitbox.apache.org/repos/asf/couchdb.git. at eb2a5cf Add ability to control which Elixir integration tests to run This branch includes the following new commits: new eb2a5cf Add ability to control which Elixir integration tests to run The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference.
[couchdb] 01/01: Add ability to control which Elixir integration tests to run
This is an automated email from the ASF dual-hosted git repository. iilyak pushed a commit to branch specify-elixir-tests in repository https://gitbox.apache.org/repos/asf/couchdb.git commit eb2a5cfece0bb5212a21c4a9b36733c9646db4c9 Author: ILYA Khlopotov AuthorDate: Wed Dec 2 04:30:15 2020 -0800 Add ability to control which Elixir integration tests to run New `elixir-suite` Makefile target is added. It runs a predefined set of elixir integration tests. The feature is controled by two files: - test/elixir/test/config/suite.elixir - contains list of all awailable tests - test/elixir/test/config/skip.elixir - contains list of tests to skip In order to update the `test/elixir/test/config/suite.elixir` when new tests are added. The one would need to run the following command: ``` MIX_ENV=integration mix suite > test/elixir/test/config/suite.elixir ``` --- Makefile | 11 + mix.exs | 20 ++ test/elixir/lib/suite.ex | 212 + test/elixir/test/config/skip.elixir | 355 + test/elixir/test/config/suite.elixir | 592 +++ test/elixir/test/test_helper.exs | 17 +- 6 files changed, 1191 insertions(+), 16 deletions(-) diff --git a/Makefile b/Makefile index 8844b86..f84eebc 100644 --- a/Makefile +++ b/Makefile @@ -271,6 +271,17 @@ elixir-cluster-with-quorum: elixir-init elixir-check-formatted elixir-credo devc --degrade-cluster 1 \ --no-eval 'mix test --trace --only with_quorum_test $(EXUNIT_OPTS)' +.PHONY: elixir-suite +elixir-suite: export MIX_ENV=integration +elixir-suite: export COUCHDB_TEST_ADMIN_PARTY_OVERRIDE=1 +elixir-suite: elixir-init + @dev/run -n 1 -q -a adm:pass \ + --enable-erlang-views \ + --no-join \ + --locald-config test/elixir/test/config/test-config.ini \ + --erlang-config rel/files/eunit.config \ + --no-eval 'mix test --trace --include test/elixir/test/config/suite.elixir --exclude test/elixir/test/config/skip.elixir' + .PHONY: elixir-check-formatted elixir-check-formatted: elixir-init @mix format --check-formatted diff --git a/mix.exs b/mix.exs index ae42af5..042862b 100644 --- a/mix.exs +++ b/mix.exs @@ -20,6 +20,26 @@ defmodule CoverTool do end end +defmodule Mix.Tasks.Suite do + @moduledoc """ + Helper task to create `suites.elixir` file. It suppose to be used as follows + MIX_ENV=integration mix suite > test/elixir/test/config/suite.elixir + """ + use Mix.Task + @shortdoc "Outputs all availabe integration tests" + def run(_) do +Path.wildcard(Path.join(Mix.Project.build_path, "/**/ebin")) +|> Enum.filter(&File.dir?/1) +|> Enum.map(&Code.append_path/1) + +tests = + Couch.Test.Suite.list() + |> Enum.sort() + |> Couch.Test.Suite.group_by() +IO.puts(Couch.Test.Suite.pretty_print(tests)) + end +end + defmodule CouchDBTest.Mixfile do use Mix.Project diff --git a/test/elixir/lib/suite.ex b/test/elixir/lib/suite.ex new file mode 100644 index 000..8f8b849 --- /dev/null +++ b/test/elixir/lib/suite.ex @@ -0,0 +1,212 @@ +defmodule Couch.Test.Suite do + @moduledoc """ +Common code to configure ExUnit runner. +It replaces the usual invocation of `ExUnit.start()` in +`test_helper.exs` related to integration tests with: +``` +Couch.Test.Suite.start() +``` + """ + @doc """ + This helper function can be used to create `suite.elixir` + as + ``` + tests = +Couch.Test.Suite.list() +|> Enum.sort() +|> Couch.Test.Suite.group_by() + + IO.puts(Couch.Test.Suite.pretty_print(tests)) + + ``` + """ + def list() do +test_paths = Keyword.get(Mix.Project.config(), :test_paths, []) +Enum.reduce(test_paths, [], fn directory, acc -> + list(directory) ++ acc +end) + end + + @doc """ + This helper function can be used to create `suite.elixir` + as + ``` + tests = +Couch.Test.Suite.list(["test/elixir/test"]) +|> Enum.sort() +|> Couch.Test.Suite.group_by() + + IO.puts(Couch.Test.Suite.pretty_print(tests)) + ``` + """ + def list(directory) do +ensure_exunit_started() +Enum.reduce(test_files(directory), [], fn file_path, acc -> + tests_in_file(file_path) ++ acc +end) + end + + @doc """ + This helper function is used in a snippet to create `suite.elixir` + see list/1 + """ + def group_by(tests) do +tests |> Enum.group_by(&module_name/1, &test_name/1) + end + + @doc """ + This helper function is used in a snippet to create `suite.elixir` + see list/1 + """
[couchdb] branch main updated (c362b1c -> be87c40)
This is an automated email from the ASF dual-hosted git repository. iilyak pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/couchdb.git. from c362b1c Add missing default headers to responses (#3279) new 109f74f Add ability to control which Elixir integration tests to run new 5b8bf5a Use elixir-suite new be87c40 Merge pull request #3286 from cloudant/specify-elixir-tests The 12943 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: Makefile | 13 +- Makefile.win | 11 + mix.exs | 23 ++ test/elixir/lib/suite.ex | 213 + test/elixir/test/config/skip.elixir | 313 ++ test/elixir/test/config/suite.elixir | 592 +++ test/elixir/test/test_helper.exs | 17 +- 7 files changed, 1165 insertions(+), 17 deletions(-) create mode 100644 test/elixir/lib/suite.ex create mode 100644 test/elixir/test/config/skip.elixir create mode 100644 test/elixir/test/config/suite.elixir
[couchdb] branch clean-up-logs-3.x.v2 created (now 9d2e7af)
This is an automated email from the ASF dual-hosted git repository. iilyak pushed a change to branch clean-up-logs-3.x.v2 in repository https://gitbox.apache.org/repos/asf/couchdb.git. at 9d2e7af Strip proxy_url in replicator This branch includes the following new commits: new fc6292b Revert "fixup: Add format_status/2 callbacks..." new cbe90e3 Revert "fixup: Add support for pluguable sanitizer module..." new 8eee6ae Revert "fixup whitespace for format_status/2" new 4b08f74 Revert "fixup: Fix format_status dispattch in couch_db_engine" new acb25cf Revert "Add format_status/2 callbacks" new 5e54ec7 Revert "Add support for pluguable sanitizer module for logger" new 9d2e7af Strip proxy_url in replicator The 7 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference.
