Unicode collation
Hi,
I noticed that CouchDB uses ICU for Unicode collation. Great job on
that decision! I've been interested in Unicode for a while, so I
looked into the implementation of this. I saw a couple things that
confused me, though.
In the Version 0.3.0 changelog, it says that locale-specific collation
is supported, but I don't see how this works in the current
implementation. couch_icu_driver.c initializes a case-sensitive
collator and a case-insensitive collator both with calls to the ICU
function as ucol_open("", &status). But from the ICU documentation, it
looks like passing "" as the locale (the first argument) selects the
default collation rules as specified in the UCA and DUCET. Is there
some other way that the locale is being passed to ICU?
It looks like strings are being compared in CouchDB using the
col_strcollIter call. From what I understand, this is fine if used in
a simple binary comparison, but when comparing strings multiple times
(as in a B-tree), it can be more efficient to pre-calculate a
collation key using ucol_getSortKey (or, to be really fancy,
calculating only the used part of the collation key, on-demand, with
ucol_nextSortKeyPart, though this may be difficult to reconcile with
an append-only file structure). Has anyone evaluated this strategy
within CouchDB to see if it might yield better performance?
Dan
Re: [VOTE] Apache CouchDB 1.0.1 release, second round
+1 JS tests pass (except for cookie_auth, but the feature is fine and I blame the failure on my dev box, not CouchDB) make check passes Erlang R14A (erts-5.8) [source] [64-bit] [smp:2:2] [rq:2] [async-threads:0] [hipe] [kernel-poll:false] OS X Snow Leopard on an SSD MacBook Pro Chris On Aug 11, 2010, at 12:15 PM, Noah Slater wrote: > Hello, > > I would like call a vote for the Apache CouchDB 1.0.1 release, second round. > > Changes since the last round: > > * Fix data corruption bug COUCHDB-844. Please see > http://couchdb.apache.org/notice/1.0.1.html for details. > > We have added a regression test to prevent this problem from happening again. > > We encourage the whole community to download and test these release artifacts > so that any critical issues can be resolved before the release is made. > Everyone is free to vote on this release, so get stuck in! > > We are voting on the following release artifacts: > > http://people.apache.org/~nslater/dist/1.0.1/ > > These artifacts have been built from the 1.0.1 tag in Subversion: > > http://svn.apache.org/repos/asf/couchdb/tags/1.0.1/ > > Happy voting, > > N >
Re: data recovery tool progress
On Aug 13, 2010, at 9:26 AM, J Chris Anderson wrote:
> Here is my first pass at notes on the repair tool.
>
> I'd like to get this on the Apache website today so we can publicize it:
>
> http://wiki.couchone.com/page/repair-tool
>
Sitting on our hands wasn't doing anyone any good, so I tweeted this as CouchDB.
http://twitter.com/CouchDB/status/21083958040
I still think we should put this on the Apache site and give it a proper
announcement.
Chris
> Please read, test, edit, give feedback, etc.
>
> Thanks,
> Chris
>
> On Aug 13, 2010, at 7:05 AM, J Chris Anderson wrote:
>
>>
>> On Aug 12, 2010, at 11:38 PM, Mikeal Rogers wrote:
>>
>>> I tested the latest code in recover-couchdb and it looks great.
>>
>> We need to package this so that it is useable by end-users, and put a link
>> to it on http://couchdb.apache.org/notice/1.0.1.html
>>
>> I'm the last guy who knows what that would mean... anyone? I think we should
>> do this today.
>>
>> Do we need to do anything formal and time consuming before linking to the
>> recovery tool / process from that page?
>>
>> Also, someone needs to write up the how-to instructions, along with a
>> description of what to expect.
>>
>> Chris
>>
>>>
>>> -Mikeal
>>>
>>> On Thu, Aug 12, 2010 at 2:33 PM, J Chris Anderson wrote:
>>>
On Aug 12, 2010, at 2:15 PM, J Chris Anderson wrote:
>
> On Aug 12, 2010, at 12:36 PM, Adam Kocoloski wrote:
>
>> Right, and jchris' db_repair branch includes my patches for DB reader
_admin access and a more useful progress report in the replication phase of
the repair.
>>
>
> I've updated the repair branch with everyone's code. I think it is
faster, due to Adam's idea that if we run the merges in reverse order,
those
near the front of the file are more likely to be no-ops, so less work is
done over all.
>
> Mikeal will be testing for correctness. Could other's please use it and
test for usability as well. Latest code (with instructions) is here:
>
> http://github.com/jhs/recover-couchdb/
>
> Which points at http://github.com/jchris/couchdb/tree/db_repair for the
repair code.
>
> One thing I am not clear about (need better docs) is, do we need to
replicate the original db to the lost+found db (or vice-versa), after
recovery is complete?
>
Also, we should be clear about what the semantics for this are. It can
potentially introduce conflicts if some writes were repeated after
restarts.
Should it always be a noop on dbs that are clean w/r/t the bug?
Chris
> Chris
>
>> Adam
>>
>> On Aug 12, 2010, at 3:14 PM, Jason Smith wrote:
>>
>>> The code is updated with the following changes:
>>> 1. Adhere to the lost+found/databasename custom...
>>> 2. ...except databases starting with _, which goes into
>>> _system/databasename
>>> 3. Sync up with jchris's db_repair branch
>>>
>>> (About #2, I started with _/database but I think it's too easy to miss
at
>>> the command line.)
>>>
>>> On Fri, Aug 13, 2010 at 00:52, J Chris Anderson
wrote:
>>>
A few bug reports from my testing:
I launched with this command, as specified in the README:
find ~/code/couchdb/tmp/lib -type f -name '*.couch' -exec
./recover_couchdb
{} \;
First of all, it chokes on my _users and _replicator db:
[info] [<0.2.0>] couch_db_repair for _users - scanning 335961 bytes at
0
[error] [<0.2.0>] couch_db_repair merge node at 332061 {case_clause,
{error,illegal_database_name}}
That second [error] line is repeated many many times (once per merge I
think). I think the issue is that _users is hard-coded to be OK, but
_users_lost+found is not. So we should do something about that, maybe
if a
db-name starts with _ we should call the lost and found
a_users_lost+found
(_ sorts at the top, so "a" will be near it and legal).
When a database has readers defined in the security object, the tool
is
unable to open them (the reading part of the repair tool needs to have
the
_admin userCtx, not just the writer).
[debug] [<0.2.0>] Not a reader: UserCtx {user_ctx,null,[],undefined}
vs
Names [<<"joe">>] Roles [<<"_admin">>]
escript: exception throw: {unauthorized,<<"You are not authorized to
access
this db.">>}
in function couch_db:open/2
in call from couch_db_repair:make_lost_and_found/3
in call from recover_couchdb:main/1
in call from escript:run/2
in call from escript:start/1
in call from init:start_it/1
in ca
Re: data recovery tool progress
Here is my first pass at notes on the repair tool.
I'd like to get this on the Apache website today so we can publicize it:
http://wiki.couchone.com/page/repair-tool
Please read, test, edit, give feedback, etc.
Thanks,
Chris
On Aug 13, 2010, at 7:05 AM, J Chris Anderson wrote:
>
> On Aug 12, 2010, at 11:38 PM, Mikeal Rogers wrote:
>
>> I tested the latest code in recover-couchdb and it looks great.
>
> We need to package this so that it is useable by end-users, and put a link to
> it on http://couchdb.apache.org/notice/1.0.1.html
>
> I'm the last guy who knows what that would mean... anyone? I think we should
> do this today.
>
> Do we need to do anything formal and time consuming before linking to the
> recovery tool / process from that page?
>
> Also, someone needs to write up the how-to instructions, along with a
> description of what to expect.
>
> Chris
>
>>
>> -Mikeal
>>
>> On Thu, Aug 12, 2010 at 2:33 PM, J Chris Anderson wrote:
>>
>>>
>>> On Aug 12, 2010, at 2:15 PM, J Chris Anderson wrote:
>>>
On Aug 12, 2010, at 12:36 PM, Adam Kocoloski wrote:
> Right, and jchris' db_repair branch includes my patches for DB reader
>>> _admin access and a more useful progress report in the replication phase of
>>> the repair.
>
I've updated the repair branch with everyone's code. I think it is
>>> faster, due to Adam's idea that if we run the merges in reverse order, those
>>> near the front of the file are more likely to be no-ops, so less work is
>>> done over all.
Mikeal will be testing for correctness. Could other's please use it and
>>> test for usability as well. Latest code (with instructions) is here:
http://github.com/jhs/recover-couchdb/
Which points at http://github.com/jchris/couchdb/tree/db_repair for the
>>> repair code.
One thing I am not clear about (need better docs) is, do we need to
>>> replicate the original db to the lost+found db (or vice-versa), after
>>> recovery is complete?
>>>
>>> Also, we should be clear about what the semantics for this are. It can
>>> potentially introduce conflicts if some writes were repeated after restarts.
>>> Should it always be a noop on dbs that are clean w/r/t the bug?
>>>
>>> Chris
>>>
Chris
> Adam
>
> On Aug 12, 2010, at 3:14 PM, Jason Smith wrote:
>
>> The code is updated with the following changes:
>> 1. Adhere to the lost+found/databasename custom...
>> 2. ...except databases starting with _, which goes into
>> _system/databasename
>> 3. Sync up with jchris's db_repair branch
>>
>> (About #2, I started with _/database but I think it's too easy to miss
>>> at
>> the command line.)
>>
>> On Fri, Aug 13, 2010 at 00:52, J Chris Anderson
>>> wrote:
>>
>>> A few bug reports from my testing:
>>>
>>> I launched with this command, as specified in the README:
>>>
>>> find ~/code/couchdb/tmp/lib -type f -name '*.couch' -exec
>>> ./recover_couchdb
>>> {} \;
>>>
>>>
>>>
>>> First of all, it chokes on my _users and _replicator db:
>>>
>>> [info] [<0.2.0>] couch_db_repair for _users - scanning 335961 bytes at
>>> 0
>>> [error] [<0.2.0>] couch_db_repair merge node at 332061 {case_clause,
>>> {error,illegal_database_name}}
>>>
>>> That second [error] line is repeated many many times (once per merge I
>>> think). I think the issue is that _users is hard-coded to be OK, but
>>> _users_lost+found is not. So we should do something about that, maybe
>>> if a
>>> db-name starts with _ we should call the lost and found
>>> a_users_lost+found
>>> (_ sorts at the top, so "a" will be near it and legal).
>>>
>>>
>>>
>>> When a database has readers defined in the security object, the tool
>>> is
>>> unable to open them (the reading part of the repair tool needs to have
>>> the
>>> _admin userCtx, not just the writer).
>>>
>>> [debug] [<0.2.0>] Not a reader: UserCtx {user_ctx,null,[],undefined}
>>> vs
>>> Names [<<"joe">>] Roles [<<"_admin">>]
>>> escript: exception throw: {unauthorized,<<"You are not authorized to
>>> access
>>> this db.">>}
>>> in function couch_db:open/2
>>> in call from couch_db_repair:make_lost_and_found/3
>>> in call from recover_couchdb:main/1
>>> in call from escript:run/2
>>> in call from escript:start/1
>>> in call from init:start_it/1
>>> in call from init:start_em/1
>>>
>>>
>>> It would also be helpful if the status lines could say something more
>>> than
>>>
>>> [info] [<0.2.0>] couch_db_repair writing 15 updates to
>>> bench_lost+found
>>>
>>> Like maybe add a note like "about 23% complete" if at all possible.
>>>
>>>
>>> I will patch the first few, I'd love help from someone on the last
>>> one.
>>> I'll be on IRC.
>>>
Re: Migrating Wiki to a CouchApp
On Aug 13, 2010, at 7:56 AM, Robert Dionne wrote: >>> >>> >>> The home link is out of sync with the FrontPage. >> >> Thanks. I'll think about how to fix that. I'd like to avoid deploying the >> CouchDB version of the wiki as a fork of the basic Pages codebase, so maybe >> it's worth it to rename FrontPage to index, and put a pointer (or redirect) >> to index on the FrontPage. >> >>> >>> Oddly I'm not able to clone the github source without first cloning it. >> >> You broke my brain. Come again? > > sorry, I should have said "forking" When I tried to git clone your repo > locally it kept failing, so I forked in github and cloned that. > hmm, it worked for me (tried a fresh one from the anonymous url), but it took a while, so maybe you just hit a timeout. > I have a background in logic so I tend to use a lot of Yogi Beraisms :) >
Re: Migrating Wiki to a CouchApp
>> >> >> The home link is out of sync with the FrontPage. > > Thanks. I'll think about how to fix that. I'd like to avoid deploying the > CouchDB version of the wiki as a fork of the basic Pages codebase, so maybe > it's worth it to rename FrontPage to index, and put a pointer (or redirect) > to index on the FrontPage. > >> >> Oddly I'm not able to clone the github source without first cloning it. > > You broke my brain. Come again? sorry, I should have said "forking" When I tried to git clone your repo locally it kept failing, so I forked in github and cloned that. I have a background in logic so I tend to use a lot of Yogi Beraisms :)
Re: get emit(doc.name,doc.url) to javascript variable
On Aug 9, 2010, at 5:50 PM, whimsica wrote:
>
> So far I'm very confused about how to get couchdb to talk to javascript on
> the web page using show or view.
>
The challenge here is figuring out which javascript code runs in which context.
All the JS you've shown, runs on the server, not the browser. Here are some
tips about how to run JS in the browser and talk to CouchDB:
The simplest way to do this is with static HTML served as an attachment to the
browser, which makes Ajax queries to the CouchDB. This way you don't need to
fuss with a show function (which is awesome, but another layer of indirection.)
Take a look at the example code on this page (and follow the exercise):
http://couchapp.org/page/what-is-couchapp#/
Where it says:
$.couch.allDbs({
success : function(dbs) {
dbs.forEach(function(db) {
$("#databases").append(''+db+'');
});
}
});
you might try replacing it with this, to do a view query instead (assuming your
db is called "mydb" and your view is called "myview":
var db = $.couch.db("mydb");
db.view("myview",{
limit : 10,
success : function(resp) {
resp.rows.forEach(function(row) {
$("#databases").append(''+row.key+'');
});
}
});
Hope that helps.
> Let's say I have a database with things like names of users and a url that
> users submit
> I want to emit the results of a view in raw json form and alert it on the
> page using javascript alert()
>
> How do I do that.
>
> I tried putting a javascript function in a view (alert(1)) just to see but
> it gave back an error.
>
> So far I was able to call a javascript function in a show command below but
> it's not the json from the view.
>
> database
> {
> "_id": "81c76a15131be24115f53e2cafaea4e8",
> "_rev": "2-5df7ac370489e818b4e72d5293069bfb",
> "name": "moe",
> "url": "http://www.moeshomepage.com";
> }
>
>
> {
> "_id": "_design/yo",
> "_rev": "11-3ac3428c8d47bb485b67f262248c8192",
> "language": "javascript",
> "views": {
> "users": {
> "map": "function(doc) {emit(doc.name,doc.url)}"
> }
> },
> "shows": {
> "showall": "function(doc,req){return alert(1)}"
> }
> }
>
> Thank you,
>
> Dan
>
> --
> View this message in context:
> http://couchdb-development.1959287.n2.nabble.com/get-emit-doc-name-doc-url-to-javascript-variable-tp5391246p5391246.html
> Sent from the CouchDB Development mailing list archive at Nabble.com.
Re: data recovery tool progress
Also, an abrupt shutdown with delayed_commits=true might orphan some
data anyway, so the warning on startup might encourage more people to
disable it.
On Fri, Aug 13, 2010 at 3:49 PM, Robert Newson wrote:
> A user (#herman) on IRC today reported slow startups for couchdb. I
> speculated that he'd hit the data loss bug and that couchdb was
> scanning backwards for a header. This turned out to be the case.
> Interestingly this was verified with a strace call, watching the read
> calls use earlier and earlier offsets.
>
> Should we consider a tweak to the tool, or couchdb itself, to report a
> warning if we have to seek back very far to find a header? Obviously
> it would a heuristic but there would be no real downside to the odd
> false positive since the recovery tool and subsequent replication will
> amount to a no-op.
>
> fyi: Reading the couchdb database (90G) with 'dd' took 22 minutes, but
> couchdb's backward scanning took 3 hours.
>
> B.
>
> On Fri, Aug 13, 2010 at 3:05 PM, J Chris Anderson wrote:
>>
>> On Aug 12, 2010, at 11:38 PM, Mikeal Rogers wrote:
>>
>>> I tested the latest code in recover-couchdb and it looks great.
>>
>> We need to package this so that it is useable by end-users, and put a link
>> to it on http://couchdb.apache.org/notice/1.0.1.html
>>
>> I'm the last guy who knows what that would mean... anyone? I think we should
>> do this today.
>>
>> Do we need to do anything formal and time consuming before linking to the
>> recovery tool / process from that page?
>>
>> Also, someone needs to write up the how-to instructions, along with a
>> description of what to expect.
>>
>> Chris
>>
>>>
>>> -Mikeal
>>>
>>> On Thu, Aug 12, 2010 at 2:33 PM, J Chris Anderson wrote:
>>>
On Aug 12, 2010, at 2:15 PM, J Chris Anderson wrote:
>
> On Aug 12, 2010, at 12:36 PM, Adam Kocoloski wrote:
>
>> Right, and jchris' db_repair branch includes my patches for DB reader
_admin access and a more useful progress report in the replication phase of
the repair.
>>
>
> I've updated the repair branch with everyone's code. I think it is
faster, due to Adam's idea that if we run the merges in reverse order,
those
near the front of the file are more likely to be no-ops, so less work is
done over all.
>
> Mikeal will be testing for correctness. Could other's please use it and
test for usability as well. Latest code (with instructions) is here:
>
> http://github.com/jhs/recover-couchdb/
>
> Which points at http://github.com/jchris/couchdb/tree/db_repair for the
repair code.
>
> One thing I am not clear about (need better docs) is, do we need to
replicate the original db to the lost+found db (or vice-versa), after
recovery is complete?
>
Also, we should be clear about what the semantics for this are. It can
potentially introduce conflicts if some writes were repeated after
restarts.
Should it always be a noop on dbs that are clean w/r/t the bug?
Chris
> Chris
>
>> Adam
>>
>> On Aug 12, 2010, at 3:14 PM, Jason Smith wrote:
>>
>>> The code is updated with the following changes:
>>> 1. Adhere to the lost+found/databasename custom...
>>> 2. ...except databases starting with _, which goes into
>>> _system/databasename
>>> 3. Sync up with jchris's db_repair branch
>>>
>>> (About #2, I started with _/database but I think it's too easy to miss
at
>>> the command line.)
>>>
>>> On Fri, Aug 13, 2010 at 00:52, J Chris Anderson
wrote:
>>>
A few bug reports from my testing:
I launched with this command, as specified in the README:
find ~/code/couchdb/tmp/lib -type f -name '*.couch' -exec
./recover_couchdb
{} \;
First of all, it chokes on my _users and _replicator db:
[info] [<0.2.0>] couch_db_repair for _users - scanning 335961 bytes at
0
[error] [<0.2.0>] couch_db_repair merge node at 332061 {case_clause,
{error,illegal_database_name}}
That second [error] line is repeated many many times (once per merge I
think). I think the issue is that _users is hard-coded to be OK, but
_users_lost+found is not. So we should do something about that, maybe
if a
db-name starts with _ we should call the lost and found
a_users_lost+found
(_ sorts at the top, so "a" will be near it and legal).
When a database has readers defined in the security object, the tool
is
unable to open them (the reading part of the repair tool needs to have
the
_admin userCtx, not just the writer).
[debug] [<0.2.0>] Not a reader: UserCtx {user_ctx,null,[],undefined}
vs
Names [<<"joe">>] Roles [<<"_a
Re: data recovery tool progress
A user (#herman) on IRC today reported slow startups for couchdb. I
speculated that he'd hit the data loss bug and that couchdb was
scanning backwards for a header. This turned out to be the case.
Interestingly this was verified with a strace call, watching the read
calls use earlier and earlier offsets.
Should we consider a tweak to the tool, or couchdb itself, to report a
warning if we have to seek back very far to find a header? Obviously
it would a heuristic but there would be no real downside to the odd
false positive since the recovery tool and subsequent replication will
amount to a no-op.
fyi: Reading the couchdb database (90G) with 'dd' took 22 minutes, but
couchdb's backward scanning took 3 hours.
B.
On Fri, Aug 13, 2010 at 3:05 PM, J Chris Anderson wrote:
>
> On Aug 12, 2010, at 11:38 PM, Mikeal Rogers wrote:
>
>> I tested the latest code in recover-couchdb and it looks great.
>
> We need to package this so that it is useable by end-users, and put a link to
> it on http://couchdb.apache.org/notice/1.0.1.html
>
> I'm the last guy who knows what that would mean... anyone? I think we should
> do this today.
>
> Do we need to do anything formal and time consuming before linking to the
> recovery tool / process from that page?
>
> Also, someone needs to write up the how-to instructions, along with a
> description of what to expect.
>
> Chris
>
>>
>> -Mikeal
>>
>> On Thu, Aug 12, 2010 at 2:33 PM, J Chris Anderson wrote:
>>
>>>
>>> On Aug 12, 2010, at 2:15 PM, J Chris Anderson wrote:
>>>
On Aug 12, 2010, at 12:36 PM, Adam Kocoloski wrote:
> Right, and jchris' db_repair branch includes my patches for DB reader
>>> _admin access and a more useful progress report in the replication phase of
>>> the repair.
>
I've updated the repair branch with everyone's code. I think it is
>>> faster, due to Adam's idea that if we run the merges in reverse order, those
>>> near the front of the file are more likely to be no-ops, so less work is
>>> done over all.
Mikeal will be testing for correctness. Could other's please use it and
>>> test for usability as well. Latest code (with instructions) is here:
http://github.com/jhs/recover-couchdb/
Which points at http://github.com/jchris/couchdb/tree/db_repair for the
>>> repair code.
One thing I am not clear about (need better docs) is, do we need to
>>> replicate the original db to the lost+found db (or vice-versa), after
>>> recovery is complete?
>>>
>>> Also, we should be clear about what the semantics for this are. It can
>>> potentially introduce conflicts if some writes were repeated after restarts.
>>> Should it always be a noop on dbs that are clean w/r/t the bug?
>>>
>>> Chris
>>>
Chris
> Adam
>
> On Aug 12, 2010, at 3:14 PM, Jason Smith wrote:
>
>> The code is updated with the following changes:
>> 1. Adhere to the lost+found/databasename custom...
>> 2. ...except databases starting with _, which goes into
>> _system/databasename
>> 3. Sync up with jchris's db_repair branch
>>
>> (About #2, I started with _/database but I think it's too easy to miss
>>> at
>> the command line.)
>>
>> On Fri, Aug 13, 2010 at 00:52, J Chris Anderson
>>> wrote:
>>
>>> A few bug reports from my testing:
>>>
>>> I launched with this command, as specified in the README:
>>>
>>> find ~/code/couchdb/tmp/lib -type f -name '*.couch' -exec
>>> ./recover_couchdb
>>> {} \;
>>>
>>>
>>>
>>> First of all, it chokes on my _users and _replicator db:
>>>
>>> [info] [<0.2.0>] couch_db_repair for _users - scanning 335961 bytes at
>>> 0
>>> [error] [<0.2.0>] couch_db_repair merge node at 332061 {case_clause,
>>> {error,illegal_database_name}}
>>>
>>> That second [error] line is repeated many many times (once per merge I
>>> think). I think the issue is that _users is hard-coded to be OK, but
>>> _users_lost+found is not. So we should do something about that, maybe
>>> if a
>>> db-name starts with _ we should call the lost and found
>>> a_users_lost+found
>>> (_ sorts at the top, so "a" will be near it and legal).
>>>
>>>
>>>
>>> When a database has readers defined in the security object, the tool
>>> is
>>> unable to open them (the reading part of the repair tool needs to have
>>> the
>>> _admin userCtx, not just the writer).
>>>
>>> [debug] [<0.2.0>] Not a reader: UserCtx {user_ctx,null,[],undefined}
>>> vs
>>> Names [<<"joe">>] Roles [<<"_admin">>]
>>> escript: exception throw: {unauthorized,<<"You are not authorized to
>>> access
>>> this db.">>}
>>> in function couch_db:open/2
>>> in call from couch_db_repair:make_lost_and_found/3
>>> in call from recover_couchdb:main/1
>>> in call from escript:run/2
>>> in call from escript:start/1
>>> in call from init:start_it/
Re: Migrating Wiki to a CouchApp
Glad to see people are excited! I am willing to take (continue) ownership / maintenance of the Pages CouchApp. The project of keeping the current wiki in sync with the CouchApp version is a horse of a different color. Once the raw dump is done, there needs to be some cleanup. Obviously with that cleanup, reimporting new changes from the old wiki becomes harder. So at some point the new wiki will have a hard break from the old one. At that point it would make sense to encourage visitors to the old wiki, to visit the new wiki instead. I am not a fan of leaving ghost town wikis around. So one thing I think we need to do, is configure MoinMoin to make 301 redirects from eg http://wiki.apache.org/couchdb/Quirks_on_Windows to http://wiki.couchdb.couchdb.org/page/Quirks_on_Windows Also, the domain name wiki.couchdb.couchdb.org is a bit ridiculous, so before we set redirects up, we should move the new wiki to wiki.couchdb.org. I'm not sure if there's an issue here that this is not under an apache.org domain. It could be moved to wiki.couchdb.apache.org for all I care, except that doing that will probably involve more logistical headaches. Also currently the demo wiki is hosted on Couchio servers, which we are happy to provide. There is some question as to whether we should stay on Apache Infrastructure. I'm happy to defer to Apache old-timers here. My instinct is always toward expedience, so I'm maybe not the one to make this judgement call. IMHO, the issue of running on non-Apache infra is strongly mitigated by the fact that it's a CouchDB, not some godforsaken bundle of directories, permissions, and .htaccess files. If folks want to help, compiling a list of the pages on this new wiki that didn't translate well automatically, would be helpful. Fixing them by hand won't be necessary. If you see a broken page, just add it to this list: http://wiki.couchdb.couchdb.org/page/BrokenPages Once we have a few broken pages, I'll go back and patch the import script to do a better job of importing them. After a few iterations, we may be down to just a handful of manual edits that need to be done, when we flip the switch to the new wiki. As far as features on Pages that we need to complete before this switch is viable, the top ones to my mind are: * Atom feeds of recent changes and comments. * Some variety of safe-Markdown (currently it's wide open to script-injection attacks). * Editable history so that an old version can be promoted as the current version. None of these are that hard to write, and they are part of the natural course of things that the wiki software needs anyway. On Aug 13, 2010, at 3:39 AM, Robert Dionne wrote: > nice, it is very snappy. I could see this would encourage more use. > > The home link is out of sync with the FrontPage. Thanks. I'll think about how to fix that. I'd like to avoid deploying the CouchDB version of the wiki as a fork of the basic Pages codebase, so maybe it's worth it to rename FrontPage to index, and put a pointer (or redirect) to index on the FrontPage. > > Oddly I'm not able to clone the github source without first cloning it. You broke my brain. Come again? > Anyway the issue with the home link seems to be in > profileReady/mustache.html, it points to "index" which is a WIKI page not yet > created > > I can certainly live without email notifications. I don't need anymore email > > > > > > On Aug 13, 2010, at 1:48 AM, J Chris Anderson wrote: > >> Devs, >> >> With the help of code from Sebastian Cohnen and Paul Davis, I've imported >> the wiki currently at http://wiki.apache.org/couchdb to a CouchApp. >> >> Here it is: >> >> http://wiki.couchdb.couchdb.org/page/FrontPage >> >> The work is still preliminary. I haven't vetted all the content, and the >> wiki software itself still needs to be polished. But I think in the long run >> we will be better off as a project to host our wiki on CouchDB. >> >> First of all, the response time when you click a link will be faster (yay >> not being a slow cgi script!) Second, the code is a CouchApp, so not only >> will we all be able to help improve it, we can easily replicate the wiki >> offline for editing, etc. >> >> In the long run it would make sense to ship a copy of the wiki with CouchDB >> (or at least make replicating a local instance of it super-simple). >> >> There are some missing features. The most notable one that I don't plan to >> implement, is email notifications of changes to pages. I haven't added Atom >> feeds of recent-changes yet, but I think that can make up for the missing >> email feature. What do you think? If email is crucial to migrating away from >> MoinMoin, it is possible. >> >> The other missing feature that I think is critical, is some built-in way to >> revert to old points in the history of a page. Currently history is stored >> but to revert would require writing some more code. >> >> Code is here for those who want to hack:
Re: data recovery tool progress
On Aug 12, 2010, at 11:38 PM, Mikeal Rogers wrote:
> I tested the latest code in recover-couchdb and it looks great.
We need to package this so that it is useable by end-users, and put a link to
it on http://couchdb.apache.org/notice/1.0.1.html
I'm the last guy who knows what that would mean... anyone? I think we should do
this today.
Do we need to do anything formal and time consuming before linking to the
recovery tool / process from that page?
Also, someone needs to write up the how-to instructions, along with a
description of what to expect.
Chris
>
> -Mikeal
>
> On Thu, Aug 12, 2010 at 2:33 PM, J Chris Anderson wrote:
>
>>
>> On Aug 12, 2010, at 2:15 PM, J Chris Anderson wrote:
>>
>>>
>>> On Aug 12, 2010, at 12:36 PM, Adam Kocoloski wrote:
>>>
Right, and jchris' db_repair branch includes my patches for DB reader
>> _admin access and a more useful progress report in the replication phase of
>> the repair.
>>>
>>> I've updated the repair branch with everyone's code. I think it is
>> faster, due to Adam's idea that if we run the merges in reverse order, those
>> near the front of the file are more likely to be no-ops, so less work is
>> done over all.
>>>
>>> Mikeal will be testing for correctness. Could other's please use it and
>> test for usability as well. Latest code (with instructions) is here:
>>>
>>> http://github.com/jhs/recover-couchdb/
>>>
>>> Which points at http://github.com/jchris/couchdb/tree/db_repair for the
>> repair code.
>>>
>>> One thing I am not clear about (need better docs) is, do we need to
>> replicate the original db to the lost+found db (or vice-versa), after
>> recovery is complete?
>>>
>>
>> Also, we should be clear about what the semantics for this are. It can
>> potentially introduce conflicts if some writes were repeated after restarts.
>> Should it always be a noop on dbs that are clean w/r/t the bug?
>>
>> Chris
>>
>>> Chris
>>>
Adam
On Aug 12, 2010, at 3:14 PM, Jason Smith wrote:
> The code is updated with the following changes:
> 1. Adhere to the lost+found/databasename custom...
> 2. ...except databases starting with _, which goes into
> _system/databasename
> 3. Sync up with jchris's db_repair branch
>
> (About #2, I started with _/database but I think it's too easy to miss
>> at
> the command line.)
>
> On Fri, Aug 13, 2010 at 00:52, J Chris Anderson
>> wrote:
>
>> A few bug reports from my testing:
>>
>> I launched with this command, as specified in the README:
>>
>> find ~/code/couchdb/tmp/lib -type f -name '*.couch' -exec
>> ./recover_couchdb
>> {} \;
>>
>>
>>
>> First of all, it chokes on my _users and _replicator db:
>>
>> [info] [<0.2.0>] couch_db_repair for _users - scanning 335961 bytes at
>> 0
>> [error] [<0.2.0>] couch_db_repair merge node at 332061 {case_clause,
>> {error,illegal_database_name}}
>>
>> That second [error] line is repeated many many times (once per merge I
>> think). I think the issue is that _users is hard-coded to be OK, but
>> _users_lost+found is not. So we should do something about that, maybe
>> if a
>> db-name starts with _ we should call the lost and found
>> a_users_lost+found
>> (_ sorts at the top, so "a" will be near it and legal).
>>
>>
>>
>> When a database has readers defined in the security object, the tool
>> is
>> unable to open them (the reading part of the repair tool needs to have
>> the
>> _admin userCtx, not just the writer).
>>
>> [debug] [<0.2.0>] Not a reader: UserCtx {user_ctx,null,[],undefined}
>> vs
>> Names [<<"joe">>] Roles [<<"_admin">>]
>> escript: exception throw: {unauthorized,<<"You are not authorized to
>> access
>> this db.">>}
>> in function couch_db:open/2
>> in call from couch_db_repair:make_lost_and_found/3
>> in call from recover_couchdb:main/1
>> in call from escript:run/2
>> in call from escript:start/1
>> in call from init:start_it/1
>> in call from init:start_em/1
>>
>>
>> It would also be helpful if the status lines could say something more
>> than
>>
>> [info] [<0.2.0>] couch_db_repair writing 15 updates to
>> bench_lost+found
>>
>> Like maybe add a note like "about 23% complete" if at all possible.
>>
>>
>> I will patch the first few, I'd love help from someone on the last
>> one.
>> I'll be on IRC.
>>
>>
>> Cheers,
>> Chris
>>
>>
>>
>>
>>
>>
>>
>> On Aug 12, 2010, at 10:18 AM, J Chris Anderson wrote:
>>
>>>
>>> On Aug 11, 2010, at 2:14 PM, Jason Smith wrote:
>>>
Hi, Jason.
On Thu, Aug 12, 2010 at 04:14, Jason Smith wrote:
> On Wed, Aug 11, 2010 at 09:52, Adam Kocoloski >>
>> wrote:
>
>> Excellent
Re: get emit(doc.name,doc.url) to javascript variable
Return a string:
"shows": {
"showall": "function(doc,req){return 'alert(1)';}"
}
FB
On Mon, Aug 9, 2010 at 8:50 PM, whimsica wrote:
>
> So far I'm very confused about how to get couchdb to talk to javascript on
> the web page using show or view.
>
> Let's say I have a database with things like names of users and a url that
> users submit
> I want to emit the results of a view in raw json form and alert it on the
> page using javascript alert()
>
> How do I do that.
>
> I tried putting a javascript function in a view (alert(1)) just to see but
> it gave back an error.
>
> So far I was able to call a javascript function in a show command below but
> it's not the json from the view.
>
> database
> {
> "_id": "81c76a15131be24115f53e2cafaea4e8",
> "_rev": "2-5df7ac370489e818b4e72d5293069bfb",
> "name": "moe",
> "url": "http://www.moeshomepage.com";
> }
>
>
> {
> "_id": "_design/yo",
> "_rev": "11-3ac3428c8d47bb485b67f262248c8192",
> "language": "javascript",
> "views": {
> "users": {
> "map": "function(doc) {emit(doc.name,doc.url)}"
> }
> },
> "shows": {
> "showall": "function(doc,req){return alert(1)}"
> }
> }
>
> Thank you,
>
> Dan
>
> --
> View this message in context:
> http://couchdb-development.1959287.n2.nabble.com/get-emit-doc-name-doc-url-to-javascript-variable-tp5391246p5391246.html
> Sent from the CouchDB Development mailing list archive at Nabble.com.
>
Re: Migrating Wiki to a CouchApp
On Aug 13, 2010, at 8:00 AM, Noah Slater wrote: > > >> (or at least make replicating a local instance of it super-simple). > > > Bingo. > > We could form it like a tutorial even. When you first install CouchDB, you go > to Futon, and there's a documentation section. It explains to you that the > project wiki is hosted on CouchDB, and you can replicate from it. It explains > some core concepts, and gives you a big fat button to press. You press it, > the official wiki is replicated to your local machine. You could even make > edits and replicate back. A perfect way to show off some of CouchDB's > strengths. > +1 - great idea
Re: Migrating Wiki to a CouchApp
On 13 Aug 2010, at 06:48, J Chris Anderson wrote: > But I think in the long run we will be better off as a project to host our > wiki on CouchDB. Eating out own dogfood is fine, but for the same reason I objected to BugShrink, I have my concerns about this. Are we sure that maintenance of this project is not going to take away valuable time from CouchDB? Do we have anyone who is prepared to take ownership of the project. Perhaps we could incubate it for a while, to see if it gains a community around it, before we bless it as official. Much like we did for CouchDB itself. > In the long run it would make sense to ship a copy of the wiki with CouchDB I disagree that we should be shipping the wiki with the official CouchDB releases. We want to keep these as small as possible. Additionally, I'm not very happy about the idea of shipping data that has not been vetted properly by the committers. However, I was thinking about how to get around these concerns, but it seems you beat me to the punch. > (or at least make replicating a local instance of it super-simple). Bingo. We could form it like a tutorial even. When you first install CouchDB, you go to Futon, and there's a documentation section. It explains to you that the project wiki is hosted on CouchDB, and you can replicate from it. It explains some core concepts, and gives you a big fat button to press. You press it, the official wiki is replicated to your local machine. You could even make edits and replicate back. A perfect way to show off some of CouchDB's strengths.
Re: Migrating Wiki to a CouchApp
On Aug 13, 2010, at 6:44 AM, Mikhail A. Pokidko wrote: > On Fri, Aug 13, 2010 at 9:48 AM, J Chris Anderson wrote: >> Devs, >> >> With the help of code from Sebastian Cohnen and Paul Davis, I've imported >> the wiki currently at http://wiki.apache.org/couchdb to a CouchApp. > > That is a kind code nationalism - "you your own code" ))) > >> The other missing feature that I think is critical, is some built-in way to >> revert to old points in the history of a page. Currently history is stored >> but to revert would require writing some more code. > > You want exactly reverting? What about just moving HEAD version? I suppose the new Github wikis readily support that as they are git backed. > >> Hope you are as excited about this as me! > > Definitely yes! > > > -- > xmpp: pma AT altlinux DOT org
Re: [VOTE] Apache CouchDB 1.0.1 release, second round
+1 All tests (Futon, make check) pass successfully. Ubuntu 10.04 amd64, Erlang R13B03, xulrunner 1.9.2.8 - Klaus On Wed, 2010-08-11 at 20:15 +0100, Noah Slater wrote: > Hello, > > I would like call a vote for the Apache CouchDB 1.0.1 release, second round. > > Changes since the last round: > > * Fix data corruption bug COUCHDB-844. Please see > http://couchdb.apache.org/notice/1.0.1.html for details. > > We have added a regression test to prevent this problem from happening again. > > We encourage the whole community to download and test these release artifacts > so that any critical issues can be resolved before the release is made. > Everyone is free to vote on this release, so get stuck in! > > We are voting on the following release artifacts: > > http://people.apache.org/~nslater/dist/1.0.1/ > > These artifacts have been built from the 1.0.1 tag in Subversion: > > http://svn.apache.org/repos/asf/couchdb/tags/1.0.1/ > > Happy voting, > > N >
Re: Migrating Wiki to a CouchApp
On Fri, Aug 13, 2010 at 9:48 AM, J Chris Anderson wrote: > Devs, > > With the help of code from Sebastian Cohnen and Paul Davis, I've imported the > wiki currently at http://wiki.apache.org/couchdb to a CouchApp. That is a kind code nationalism - "you your own code" ))) > The other missing feature that I think is critical, is some built-in way to > revert to old points in the history of a page. Currently history is stored > but to revert would require writing some more code. You want exactly reverting? What about just moving HEAD version? > Hope you are as excited about this as me! Definitely yes! -- xmpp: pma AT altlinux DOT org
Re: Migrating Wiki to a CouchApp
On 13 August 2010 22:33, Klaus Trainer wrote: > On Thu, 2010-08-12 at 22:48 -0700, J Chris Anderson wrote: >> Hope you are as excited about this as me! > > Yes! Sounds pretty cool. Even there's still a bunch of work to do, it's > consequent. > > Looking forward to have the entire wiki in my CouchDB! > > - Klaus Yeah very exciting -- I could see this becoming the biggest public replicated couch ever -- looking forwards to having the entire wiki on my iPhone! Let us know when we should stop updating the old wiki... A+ Dave
Re: Migrating Wiki to a CouchApp
nice, it is very snappy. I could see this would encourage more use. The home link is out of sync with the FrontPage. Oddly I'm not able to clone the github source without first cloning it. Anyway the issue with the home link seems to be in profileReady/mustache.html, it points to "index" which is a WIKI page not yet created I can certainly live without email notifications. I don't need anymore email On Aug 13, 2010, at 1:48 AM, J Chris Anderson wrote: > Devs, > > With the help of code from Sebastian Cohnen and Paul Davis, I've imported the > wiki currently at http://wiki.apache.org/couchdb to a CouchApp. > > Here it is: > > http://wiki.couchdb.couchdb.org/page/FrontPage > > The work is still preliminary. I haven't vetted all the content, and the wiki > software itself still needs to be polished. But I think in the long run we > will be better off as a project to host our wiki on CouchDB. > > First of all, the response time when you click a link will be faster (yay not > being a slow cgi script!) Second, the code is a CouchApp, so not only will we > all be able to help improve it, we can easily replicate the wiki offline for > editing, etc. > > In the long run it would make sense to ship a copy of the wiki with CouchDB > (or at least make replicating a local instance of it super-simple). > > There are some missing features. The most notable one that I don't plan to > implement, is email notifications of changes to pages. I haven't added Atom > feeds of recent-changes yet, but I think that can make up for the missing > email feature. What do you think? If email is crucial to migrating away from > MoinMoin, it is possible. > > The other missing feature that I think is critical, is some built-in way to > revert to old points in the history of a page. Currently history is stored > but to revert would require writing some more code. > > Code is here for those who want to hack: > > http://github.com/couchone/pages > > Hope you are as excited about this as me! > > Chris >
Re: Migrating Wiki to a CouchApp
On Thu, 2010-08-12 at 22:48 -0700, J Chris Anderson wrote: > Hope you are as excited about this as me! Yes! Sounds pretty cool. Even there's still a bunch of work to do, it's consequent. Looking forward to have the entire wiki in my CouchDB! - Klaus
