[jira] [Commented] (COUCHDB-1144) oauth requests with non-percent-encoded realms result in function_clause error in HTTP request
[
https://issues.apache.org/jira/browse/COUCHDB-1144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13027247#comment-13027247
]
Paul Joseph Davis commented on COUCHDB-1144:
Anyone have an issue with me committing this?
> oauth requests with non-percent-encoded realms result in function_clause
> error in HTTP request
> --
>
> Key: COUCHDB-1144
> URL: https://issues.apache.org/jira/browse/COUCHDB-1144
> Project: CouchDB
> Issue Type: Bug
> Components: HTTP Interface
>Affects Versions: 1.0.2, 1.1
> Environment: OSX - branch 1.1 - r1095237
>Reporter: Joan Touzet
>Priority: Minor
> Attachments: oauth-realm-test.patch, oauth_uri.erl
>
>
> As illustrated in this Wireshark packet dump, lines 53ff, any OAuth request
> with a non-percent-encoded URL as the realm will return a 500 error:
> http://friendpaste.com/3vXPjHP6s7dLZjsj7DOfSH
> Erlang stacktrace is similar to:
> [error] [<0.189.0>] function_clause error in HTTP request [Wed, 27 Apr 2011
> 23:31:46 GMT] [info] [<0.189.0>] Stacktrace:
> [{oauth_uri,decode, ["://127.0.0.1:5984","ptth"]},
> {oauth_uri,param_from_header_string,1},
> {oauth_uri,
> '-params_from_header_string/1-lc$^0/1-0-',
> 1},
> {couch_httpd_oauth,serve_oauth,3},
> {couch_httpd,authenticate_request,2},
> {couch_httpd,handle_request_int,5},
> {mochiweb_http,headers,5},
> {proc_lib,init_p_do_apply,3}] [Wed, 27 Apr
> 2011 23:31:46 GMT] [info] [<0.189.0>] 127.0.0.1 - - 'PUT'
> /test_c_project/16f74c25-d641-4710-8f38-18295a8a69b1 500
> Chatted with benoitc today and he suggested this may be because the realm is
> not being properly encoded before being passed on.
> By default, some OAuth libraries such as python's popular oauth2 library
> always set the realm of a request to the URL of the resource being accessed,
> such as http://localhost:5984/ . (In fact, oauth2 library does not support
> overriding this setting.) The OAuth spec @ http://oauth.net/core/1.0/ also
> shows realms specified as URLs. RFC5849 states "The OPTIONAL "realm"
> parameter MAY be added and interpreted per [RFC2617] section 1.2.", which in
> turn says the realm is any quoted-string.
> It seems that this may already be fixed in trunk simply because trunk has a
> newer version of the upstream erlang-oauth.
> A JS test could be added in futon to validate this by modifying function
> oauthRequest to accept a realm parameter, then passing that down to
> OAuth.getAuthorizationHeader on line 56.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: Debian package 0.11.0 stable?
On Tue, Apr 26, 2011 at 7:44 PM, Benoit Chesneau wrote: > On Tue, Apr 26, 2011 at 7:35 PM, Noah Slater wrote: >> >> On 26 Apr 2011, at 18:34, Noah Slater wrote: >> >>> >>> On 26 Apr 2011, at 18:31, till wrote: >>> Why not link all? >>> >>> You would like me to exhaustively link to every part of the Debian >>> packaging infrastructure? Heh. That could take me hours to compile. It is >>> distributed across loads of different sites. Depending on what you're >>> packaging, with whom, and what you want to do, or know, about that effort. >> >> Having said that, this is a good place to start: >> >> http://packages.qa.debian.org/c/couchdb.html > > Or simply maintain a ppa on launchpad . > > - benoit > Didn't know you can have a debian PPA on launchpad. In this case, I can try to help too. We just went through the setup for some other stuff 'in house'. Sort of related -- does Apache maintain an apt-mirror or would they support that? That would be extremely nice if this launchpad PPA could be mirrored by them to sail around downtimes and capacity issues. Till
[jira] [Commented] (COUCHDB-1120) Snappy compression (databases, view indexes) + keeping doc bodies as ejson binaries
[
https://issues.apache.org/jira/browse/COUCHDB-1120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13027196#comment-13027196
]
Filipe Manana commented on COUCHDB-1120:
After chatting a bit with Paul Davis on IRC, having compression disabled, the
view indexer gets about the same benefit:
$ time curl
http://localhost:5984/snappy_complex_keys/_design/test/_view/view1?limit=1
{"total_rows":551200,"offset":0,"rows":[
{"id":"00d49881-7bcf-4c3d-a65d-e44435eeb513","key":["dwarf","assassin",2,1.1],"value":[{"x":174347.18,"y":127272.8},{"x":35179.93,"y":41550.55},{"x":157014.38,"y":172052.63},{"x":116185.83,"y":69871.73},{"x":153746.28,"y":190006.59}]}
]}
real13m18.866s
user0m0.012s
sys 0m0.020s
This is for the database with 551 200 documents (each with a size of about 1
Kb).
For much larger databases, the compression adds more positive impact, since the
OS will likely be able to cache more disk pages and the data to write to the
index file is smaller. I've observed this for very large databases/indexes.
> Snappy compression (databases, view indexes) + keeping doc bodies as ejson
> binaries
>
>
> Key: COUCHDB-1120
> URL: https://issues.apache.org/jira/browse/COUCHDB-1120
> Project: CouchDB
> Issue Type: Improvement
> Components: Database Core
> Environment: trunk
>Reporter: Filipe Manana
>Assignee: Filipe Manana
>
> The branch at:
> https://github.com/fdmanana/couchdb/compare/snappy
> Is an experiment which adds snappy compression to database files and view
> index files. Snappy is a very fast compressor/decompressor developed by and
> used by Google [1] - even for small data chunks like 100Kb it can be 2 orders
> of magnitude faster then zlib or Erlang's term_to_binary compression level 1.
> Somewhere at [1] there are benchmark results published by Google that compare
> against zlib's deflate, Erlang's term_to_binary compression, lzo, etc.
> Even small objects like database headers or btree nodes, still get smaller
> after compressing them with snappy, see the shell session at [2].
> Besides the compression, this branch also keeps the document bodies
> (#doc.body fields) as binaries (snappy compressed ejson binaries) and only
> converts them back to ejson when absolutely needed (done by
> couch_doc:to_json_obj/2 for e.g.) - this is similar to COUCHDB-1092 - but the
> bodies are EJSON compressed binaries and doesn't suffer from the same issue
> Paul identified before (which could be fixed without many changes) - on reads
> we decompress and still do the binary_to_term/1 + ?JSON_ENCODE calls as
> before.
> It also prepares the document summaries before sending the documents to the
> updater, so that we avoid copying EJSON terms and move this task outside of
> the updater to add more parallelism to concurrent updates.
> I made some tests, comparing trunk before and after the JSON parser NIF was
> added, against this snappy branch.
> I created databases with 1 000 000 documents of 4Kb each. The document
> template is this one: http://friendpaste.com/qdfyId8w1C5vkxROc5Thf
> The databases have this design document:
> {
> "_id": "_design/test",
> "language": "javascript",
> "views": {
> "simple": {
> "map": "function(doc) { emit(doc.data5.float1, [doc.strings[2],
> doc.strings[10]]); }"
> }
> }
> }
> == Results with trunk ==
> database file size after compaction: 7.5 Gb
> view index file size after compaction: 257 Mb
> ** Before JSON nif:
> $ time curl
> 'http://localhost:5985/trunk_db_1m/_design/test/_view/simple?limit=1'
> {"total_rows":100,"offset":0,"rows":[
> {"id":"0632-d25d-49c6-9b4e-e038b78ff97d","key":76.572,"value":["jURcBZ0vrJcmf2roZUMzZJQoTsKZDIdj7KhO7itskKvM80jBU9","fKYYthv8iFvaYoFoYZyB"]}
> ]}
> real 58m28.599s
> user 0m0.036s
> sys 0m0.056s
> ** After JSON nif:
> fdmanana 12:45:55 /opt/couchdb > time curl
> 'http://localhost:5985/trunk_db_1m/_design/test/_view/simple?limit=1'
> {"total_rows":100,"offset":0,"rows":[
> {"id":"0632-d25d-49c6-9b4e-e038b78ff97d","key":76.572,"value":["jURcBZ0vrJcmf2roZUMzZJQoTsKZDIdj7KhO7itskKvM80jBU9","fKYYthv8iFvaYoFoYZyB"]}
> ]}
> real 51m14.738s
> user 0m0.040s
> sys 0m0.044s
> == Results with the snappy branch ==
> database file size after compaction: 3.2 Gb (vs 7.5 Gb on trunk)
> view index file size after compaction: 100 Mb (vs 257 Mb on trunk)
> ** Before JSON nif:
> $ time curl
> 'http://localhost:5984/snappy_db_1m/_design/test/_view/simple?limit=1'
> {"total_rows":100,"offset":0,"rows":[
> {"id":"0632-d25d-49c6-9b4e-e038b78ff97d","key":76.572,"value":["jURcBZ0vrJcmf2roZUMzZJQoTsKZDIdj7KhO7itskKvM80jBU9","fKYYthv8iFvaYoFoYZyB"]}
> ]}
> real 32m29.854s
> user 0m0.008s
> sys
Re: Debian package 0.11.0 stable?
On 26/04/11 12:48 PM, Benoit Chesneau wrote: On Tue, Apr 26, 2011 at 8:38 PM, Noah Slater wrote: If you'd like code reviewing, I can help out. the same. Ditto. We're users of Couch on Debian and wouldn't mind testing packages and so forth. -- Matt Adams Radical Dynamic www.radicaldynamic.com
[jira] [Commented] (COUCHDB-1120) Snappy compression (databases, view indexes) + keeping doc bodies as ejson binaries
[
https://issues.apache.org/jira/browse/COUCHDB-1120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13027182#comment-13027182
]
Norman Barker commented on COUCHDB-1120:
I was referring to trade off between access speeds using snappy vs gzip and
file size. It works well.
> Snappy compression (databases, view indexes) + keeping doc bodies as ejson
> binaries
>
>
> Key: COUCHDB-1120
> URL: https://issues.apache.org/jira/browse/COUCHDB-1120
> Project: CouchDB
> Issue Type: Improvement
> Components: Database Core
> Environment: trunk
>Reporter: Filipe Manana
>Assignee: Filipe Manana
>
> The branch at:
> https://github.com/fdmanana/couchdb/compare/snappy
> Is an experiment which adds snappy compression to database files and view
> index files. Snappy is a very fast compressor/decompressor developed by and
> used by Google [1] - even for small data chunks like 100Kb it can be 2 orders
> of magnitude faster then zlib or Erlang's term_to_binary compression level 1.
> Somewhere at [1] there are benchmark results published by Google that compare
> against zlib's deflate, Erlang's term_to_binary compression, lzo, etc.
> Even small objects like database headers or btree nodes, still get smaller
> after compressing them with snappy, see the shell session at [2].
> Besides the compression, this branch also keeps the document bodies
> (#doc.body fields) as binaries (snappy compressed ejson binaries) and only
> converts them back to ejson when absolutely needed (done by
> couch_doc:to_json_obj/2 for e.g.) - this is similar to COUCHDB-1092 - but the
> bodies are EJSON compressed binaries and doesn't suffer from the same issue
> Paul identified before (which could be fixed without many changes) - on reads
> we decompress and still do the binary_to_term/1 + ?JSON_ENCODE calls as
> before.
> It also prepares the document summaries before sending the documents to the
> updater, so that we avoid copying EJSON terms and move this task outside of
> the updater to add more parallelism to concurrent updates.
> I made some tests, comparing trunk before and after the JSON parser NIF was
> added, against this snappy branch.
> I created databases with 1 000 000 documents of 4Kb each. The document
> template is this one: http://friendpaste.com/qdfyId8w1C5vkxROc5Thf
> The databases have this design document:
> {
> "_id": "_design/test",
> "language": "javascript",
> "views": {
> "simple": {
> "map": "function(doc) { emit(doc.data5.float1, [doc.strings[2],
> doc.strings[10]]); }"
> }
> }
> }
> == Results with trunk ==
> database file size after compaction: 7.5 Gb
> view index file size after compaction: 257 Mb
> ** Before JSON nif:
> $ time curl
> 'http://localhost:5985/trunk_db_1m/_design/test/_view/simple?limit=1'
> {"total_rows":100,"offset":0,"rows":[
> {"id":"0632-d25d-49c6-9b4e-e038b78ff97d","key":76.572,"value":["jURcBZ0vrJcmf2roZUMzZJQoTsKZDIdj7KhO7itskKvM80jBU9","fKYYthv8iFvaYoFoYZyB"]}
> ]}
> real 58m28.599s
> user 0m0.036s
> sys 0m0.056s
> ** After JSON nif:
> fdmanana 12:45:55 /opt/couchdb > time curl
> 'http://localhost:5985/trunk_db_1m/_design/test/_view/simple?limit=1'
> {"total_rows":100,"offset":0,"rows":[
> {"id":"0632-d25d-49c6-9b4e-e038b78ff97d","key":76.572,"value":["jURcBZ0vrJcmf2roZUMzZJQoTsKZDIdj7KhO7itskKvM80jBU9","fKYYthv8iFvaYoFoYZyB"]}
> ]}
> real 51m14.738s
> user 0m0.040s
> sys 0m0.044s
> == Results with the snappy branch ==
> database file size after compaction: 3.2 Gb (vs 7.5 Gb on trunk)
> view index file size after compaction: 100 Mb (vs 257 Mb on trunk)
> ** Before JSON nif:
> $ time curl
> 'http://localhost:5984/snappy_db_1m/_design/test/_view/simple?limit=1'
> {"total_rows":100,"offset":0,"rows":[
> {"id":"0632-d25d-49c6-9b4e-e038b78ff97d","key":76.572,"value":["jURcBZ0vrJcmf2roZUMzZJQoTsKZDIdj7KhO7itskKvM80jBU9","fKYYthv8iFvaYoFoYZyB"]}
> ]}
> real 32m29.854s
> user 0m0.008s
> sys 0m0.052s
> ** After JSON nif:
> fdmanana 15:40:39 /opt/couchdb > time curl
> 'http://localhost:5984/snappy_db_1m/_design/test/_view/simple?limit=1'
> {"total_rows":100,"offset":0,"rows":[
> {"id":"0632-d25d-49c6-9b4e-e038b78ff97d","key":76.572,"value":["jURcBZ0vrJcmf2roZUMzZJQoTsKZDIdj7KhO7itskKvM80jBU9","fKYYthv8iFvaYoFoYZyB"]}
> ]}
> real 18m39.240s
> user 0m0.012s
> sys 0m0.020s
> A writes-only relaximation test also shows a significant improvement in the
> writes response times / throughput:
> http://graphs.mikeal.couchone.com/#/graph/698bf36b6c64dbd19aa2bef63405480d
> These results are also in a file of this branch [3].
> Seems clear this, together with Paul's JSON NIF parser, has a very good
> impact in the view indexer, besides
[jira] [Commented] (COUCHDB-1120) Snappy compression (databases, view indexes) + keeping doc bodies as ejson binaries
[
https://issues.apache.org/jira/browse/COUCHDB-1120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13027175#comment-13027175
]
Filipe Manana commented on COUCHDB-1120:
I've created another branch on top of the previous one which makes compression
optional and also adds the possibility to use deflate (zlib) compression:
https://github.com/fdmanana/couchdb/compare/file_compression
By default, snappy compression is enabled. The compression is configured in the
section [couchdb]:
https://github.com/fdmanana/couchdb/compare/file_compression#diff-4
For those interested, after checking out snappy from google code (
http://code.google.com/p/snappy/ ), one can run some benchmark tests to compare
snappy against zlib, lzo and other algorithms. This is done by running:
$ ./snappy_unittest -run_microbenchmarks=false --zlib --lzo testdata/*
Output example at http://friendpaste.com/7YVC8jImnY2GbnOLJvce6x
The tests I presented before, as well as the following ones, show that snappy
has a positive impact on the database read/write performance and view indexer
performance.
Here are a few more tests against 2 different databases/views.
* Database with 551 200 documents, each with a size of about 1 Kb *
Database created with:
$ ./seatoncouch.rb --host localhost --port 5984 --docs 551200 --threads 20
--db-name complex_keys \
--bulk-batch 100 --doc-tpl complex_keys.tpl
The document template can be found here:
http://friendpaste.com/1cRdpfPyzWzoQKo8zb3fky
The database has following design document:
{
"_id": "_design/test",
"language": "javascript",
"views": {
"view1": {
"map": "function(doc) { emit([doc.type, doc.category, doc.level,
doc.ratio], doc.nested.coords); }"
},
"view2": {
"map": "function(doc) { emit(doc._id, {type: doc.type, cat:
doc.category, level: doc.level, ratio: doc.ratio}); }"
}
}
}
* trunk *
database file size after compaction: 1592 Mb
view file size after compaction: 520 Mb
view index build time:
$ time curl
http://localhost:5984/trunk_complex_keys/_design/test/_view/view1?limit=1
{"total_rows":551200,"offset":0,"rows":[
{"id":"00d49881-7bcf-4c3d-a65d-e44435eeb513","key":["dwarf","assassin",2,1.1],"value":[{"x":174347.18,"y":127272.8},{"x":35179.93,"y":41550.55},{"x":157014.38,"y":172052.63},{"x":116185.83,"y":69871.73},{"x":153746.28,"y":190006.59}]}
]}
real25m47.395s
user0m0.008s
sys 0m0.040s
*** branch file_compression with snappy compression enabled ***
database file size after compaction: 1215 Mb
view file size after compaction: 155 Mb
view index build time:
$ time curl
http://localhost:5985/snappy_complex_keys/_design/test/_view/view1?limit=1
{"total_rows":551200,"offset":0,"rows":[
{"id":"00d49881-7bcf-4c3d-a65d-e44435eeb513","key":["dwarf","assassin",2,1.1],"value":[{"x":174347.18,"y":127272.8},{"x":35179.93,"y":41550.55},{"x":157014.38,"y":172052.63},{"x":116185.83,"y":69871.73},{"x":153746.28,"y":190006.59}]}
]}
real12m11.829s
user0m0.004s
sys 0m0.020s
*** branch file_compression with deflate compression level 1 enabled ***
database file size after compaction: 1097 Mb
view file size after compaction: 123 Mb
view index build time:
$ time curl
http://localhost:5985/deflate1_complex_keys/_design/test/_view/view1?limit=1
{"total_rows":551200,"offset":0,"rows":[
{"id":"00d49881-7bcf-4c3d-a65d-e44435eeb513","key":["dwarf","assassin",2,1.1],"value":[{"x":174347.18,"y":127272.8},{"x":35179.93,"y":41550.55},{"x":157014.38,"y":172052.63},{"x":116185.83,"y":69871.73},{"x":153746.28,"y":190006.59}]}
]}
real19m32.945s
user0m0.000s
sys 0m0.036s
*** branch file_compression with deflate compression level 9 enabled ***
database file size after compaction: 1092 Mb
view file size after compaction: 118 Mb
view index build time:
$ time curl
http://localhost:5985/deflate9_complex_keys/_design/test/_view/view1?limit=1
{"total_rows":551200,"offset":0,"rows":[
{"id":"00d49881-7bcf-4c3d-a65d-e44435eeb513","key":["dwarf","assassin",2,1.1],"value":[{"x":174347.18,"y":127272.8},{"x":35179.93,"y":41550.55},{"x":157014.38,"y":172052.63},{"x":116185.83,"y":69871.73},{"x":153746.28,"y":190006.59}]}
]}
real21m50.390s
user0m0.012s
sys 0m0.036s
* Benoit's warlogs database (https://warlogs.upondata.com/warlogs), 391 835
documents *
*** trunk ***
database file size after compaction: 1090 Mb
view file size after compaction: 1.3 Gb
view index build time:
$ time curl
http://localhost:5984/warlogs_trunk/_design/warlogs/_view/by_date?limit=1
{"total_rows":391832,"offset":0,"rows":[
{"id":"0104D7FC-0219-4C16-8531-97C60A59C70C","key":["2004-01","0104D7FC-0219-4C16-8531-97C60A59C70C"],"value":null}
]}
real44m10.341s
user0m0.021s
sys 0m0.050s
*** branch file_compression with snappy compression enabled ***
database file size
Re: Development environment
On 29 Apr 2011, at 06:36, Andrey Somov wrote: > P.S. I have already identified some redundant code ( > https://issues.apache.org/jira/browse/COUCHDB-920). > I hope I can further improve the code base when I am able to debug the > running database. Thank you for your efforts Andrey and thanks for calling out an obvious deficiency in our setup for the way you work. I'm looking forward to your contributions :) I strongly believe, and you have already proven, that a new perspective on code brings previously unknown issues to light. This is going to be good :) Cheers Jan -- > > > > On Fri, Apr 29, 2011 at 1:34 PM, Robert Dionne > wrote: > >> Hi Andrey, >> >> I use Distel[1] (Distributed emacs lisp for Erlang), a set of emacs >> extensions that create a full development environment for Erlang. It >> connects to a running node so one gets full access to the syntax_tools and >> source code of Erlang, all at run time. As this brief white paper points out >> it goes further than typical elisp hacks as it imports the Erlang >> distributed model into emacs. I keep hoping to find some time to push it a >> little further and provide better support for working with BigCouch, our >> clustered CouchDB at Cloudant. >> >> I keep up my own fork of it as there aren't too many of us out there and I >> needed to fix a few things. I also include in that project some tweaks to >> the Erlang mode that ships with Erlang to accommodate the CouchDB format >> conventions. It provides a full symbolic debugging environment. Though it's >> useful and I think I've found a few CouchDB bugs with it, given the nature >> of OTP programming it's a little harder when you have breakpoints that hit >> 50 processes. It was great for stepping thru the btree code. >> >> The most useful features are the navigation (M-. M-,) and who_calls (C-c >> C-d w) The lack of use of who_calls I believe is the major reason we often >> discover dead code that's been there forever. As an aside the use of type >> specs and dialyzer go a long way towards finding errors at compile time. >> >> Regards, >> >> Bob >> >> [1] https://github.com/bdionne/distel/raw/master/doc/gorrie02distel.pdf >> [2] https://github.com/bdionne/distel >> >> >> >>
Re: Development environment
On 28 Apr 2011, at 19:17, Nathan Stott wrote: > Originally, (until 0.11 I think?) CouchDB was developed without unit > tests. That is not correct. The source has always contained testing code for the harder parts of CouchDB. They just weren't always accessible to the user, but only to developers. > I take it from this thread it was also developed then without > step-by-step debugging. It's amazing how stable it is and has been in > light of those two facts. What Noah said, there are many more ways to develop software :) Cheers Jan -- > On Thu, Apr 28, 2011 at 7:40 PM, till wrote: >> On Fri, Apr 29, 2011 at 2:38 AM, Miles Fidelman >> wrote: >>> Andrey Somov wrote: On Fri, Apr 29, 2011 at 1:01 AM, Paul Daviswrote: > > On Thu, Apr 28, 2011 at 6:57 PM, Nathan Stott wrote: > >> >> So couchdb is really developed without using step-by-step debugging at >> all? That seems quite odd. >> >> > > I never use step by step debugging on any project in any language. > I've never found it that useful. > > step by step debugging of a unit test is one the most effective ways to find bugs. >>> >>> Until you start running highly concurrent software, distributed across >>> multiple nodes. That takes debugging to a whole new level. >>> >> >> A unit test doesn't run on multiple nodes. >>
Re: Development environment
Hi Robert, unfortunately, I could not build distel (on Ubuntu 10.04): trop:~/distel-bdionne$ erl Erlang R13B03 (erts-5.7.4) [source] [rq:1] [async-threads:0] [hipe] [kernel-poll:false] Eshell V5.7.4 (abort with ^G) 1> trop:~/distel-bdionne$ make --debug GNU Make 3.81 Copyright (C) 2006 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. This program built for i486-pc-linux-gnu Reading makefiles... Updating goal targets File `base' does not exist. File `ebin/distel.beam' does not exist. Must remake target `ebin/distel.beam'. erlc -W -o ebin +debug_info src/distel.erl /home/trop/distel-bdionne/ebin/distel.bea#: error writing file make: *** [ebin/distel.beam] Error 1
Re: Development environment
Hello Robert, this is much closer to what I expect. I assume I should follow the procedure described here http://mail-archives.apache.org/mod_mbox/couchdb-dev/200811.mbox/%3cc397e43f-096f-4892-9519-383032622...@dionne-associates.com%3E I will give it a try. Thank you very much. - Andrey P.S. I have already identified some redundant code ( https://issues.apache.org/jira/browse/COUCHDB-920). I hope I can further improve the code base when I am able to debug the running database. On Fri, Apr 29, 2011 at 1:34 PM, Robert Dionne wrote: > Hi Andrey, > > I use Distel[1] (Distributed emacs lisp for Erlang), a set of emacs > extensions that create a full development environment for Erlang. It > connects to a running node so one gets full access to the syntax_tools and > source code of Erlang, all at run time. As this brief white paper points out > it goes further than typical elisp hacks as it imports the Erlang > distributed model into emacs. I keep hoping to find some time to push it a > little further and provide better support for working with BigCouch, our > clustered CouchDB at Cloudant. > > I keep up my own fork of it as there aren't too many of us out there and I > needed to fix a few things. I also include in that project some tweaks to > the Erlang mode that ships with Erlang to accommodate the CouchDB format > conventions. It provides a full symbolic debugging environment. Though it's > useful and I think I've found a few CouchDB bugs with it, given the nature > of OTP programming it's a little harder when you have breakpoints that hit > 50 processes. It was great for stepping thru the btree code. > > The most useful features are the navigation (M-. M-,) and who_calls (C-c > C-d w) The lack of use of who_calls I believe is the major reason we often > discover dead code that's been there forever. As an aside the use of type > specs and dialyzer go a long way towards finding errors at compile time. > > Regards, > > Bob > > [1] https://github.com/bdionne/distel/raw/master/doc/gorrie02distel.pdf > [2] https://github.com/bdionne/distel > > > >
Re: Development environment
Hi Andrey, I use Distel[1] (Distributed emacs lisp for Erlang), a set of emacs extensions that create a full development environment for Erlang. It connects to a running node so one gets full access to the syntax_tools and source code of Erlang, all at run time. As this brief white paper points out it goes further than typical elisp hacks as it imports the Erlang distributed model into emacs. I keep hoping to find some time to push it a little further and provide better support for working with BigCouch, our clustered CouchDB at Cloudant. I keep up my own fork of it as there aren't too many of us out there and I needed to fix a few things. I also include in that project some tweaks to the Erlang mode that ships with Erlang to accommodate the CouchDB format conventions. It provides a full symbolic debugging environment. Though it's useful and I think I've found a few CouchDB bugs with it, given the nature of OTP programming it's a little harder when you have breakpoints that hit 50 processes. It was great for stepping thru the btree code. The most useful features are the navigation (M-. M-,) and who_calls (C-c C-d w) The lack of use of who_calls I believe is the major reason we often discover dead code that's been there forever. As an aside the use of type specs and dialyzer go a long way towards finding errors at compile time. Regards, Bob [1] https://github.com/bdionne/distel/raw/master/doc/gorrie02distel.pdf [2] https://github.com/bdionne/distel On Apr 28, 2011, at 9:03 AM, Andrey Somov wrote: > Hi all, > in order to understand how CouchDB works I want be able to run the > application under debugger. Unfortunately it does not look like an easy > task. > The information provided on the wiiki ( > http://wiki.apache.org/couchdb/Running%20CouchDB%20in%20Dev%20Mode) may be > enough for a > professional Erlang developer but it is not enough for anyone who learns > Erlang together with CouchDB. > I could not find any resource which gives step-by-step instructions on how > to organise an effective development environment for CouchDB. > > Can someone point me to such a guide/tutorial/manual/screencast ? > > Thanks, > Andrey
Re: Development environment
On 29 Apr 2011, at 03:17, Nathan Stott wrote: > Originally, (until 0.11 I think?) CouchDB was developed without unit > tests. I take it from this thread it was also developed then without > step-by-step debugging. It's amazing how stable it is and has been in > light of those two facts. Depends how you're writing code, I think. Some people, me included, and especially in web development, tend to adopt the Throw Enough Shit, See What Sticks approach to software. Engineers and computer scientists tend to take the opposite approach. Knuth famously remarked "Beware of bugs in the above code; I have only proved it correct, not tried it." Any point on that spectrum is a trade-off, and some approaches are more suited to certain applications. I am guessing that database development is more up towards the Knuth end. Maybe that's how Damian codes? I doubt Knuth has any use for unit tests.
[jira] [Updated] (COUCHDB-1148) Remove unused single backslash from a regular expression in couch_config_writer.erl
[
https://issues.apache.org/jira/browse/COUCHDB-1148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrey Somov updated COUCHDB-1148:
--
Attachment: patch-1148.txt
> Remove unused single backslash from a regular expression in
> couch_config_writer.erl
> ---
>
> Key: COUCHDB-1148
> URL: https://issues.apache.org/jira/browse/COUCHDB-1148
> Project: CouchDB
> Issue Type: Improvement
> Components: Database Core
>Affects Versions: 1.1
>Reporter: Andrey Somov
>Priority: Minor
> Attachments: patch-1148.txt
>
> Original Estimate: 1h
> Remaining Estimate: 1h
>
> The pattern re:compile(["^(", Key, "\\s*=)|\\[[a-zA-Z0-9\_-]*\\]"]) contains
> a single backslash '\_' which can be removed.
> http://comments.gmane.org/gmane.comp.db.couchdb.devel/14534
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (COUCHDB-1148) Remove unused single backslash from a regular expression in couch_config_writer.erl
Remove unused single backslash from a regular expression in
couch_config_writer.erl
---
Key: COUCHDB-1148
URL: https://issues.apache.org/jira/browse/COUCHDB-1148
Project: CouchDB
Issue Type: Improvement
Components: Database Core
Affects Versions: 1.1
Reporter: Andrey Somov
Priority: Minor
The pattern re:compile(["^(", Key, "\\s*=)|\\[[a-zA-Z0-9\_-]*\\]"]) contains a
single backslash '\_' which can be removed.
http://comments.gmane.org/gmane.comp.db.couchdb.devel/14534
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
