On Aug 12, 2010, at 2:15 PM, J Chris Anderson wrote: > > On Aug 12, 2010, at 12:36 PM, Adam Kocoloski wrote: > >> Right, and jchris' db_repair branch includes my patches for DB reader _admin >> access and a more useful progress report in the replication phase of the >> repair. >> > > I've updated the repair branch with everyone's code. I think it is faster, > due to Adam's idea that if we run the merges in reverse order, those near the > front of the file are more likely to be no-ops, so less work is done over all. > > Mikeal will be testing for correctness. Could other's please use it and test > for usability as well. Latest code (with instructions) is here: > > http://github.com/jhs/recover-couchdb/ > > Which points at http://github.com/jchris/couchdb/tree/db_repair for the > repair code. > > One thing I am not clear about (need better docs) is, do we need to replicate > the original db to the lost+found db (or vice-versa), after recovery is > complete? >
Also, we should be clear about what the semantics for this are. It can potentially introduce conflicts if some writes were repeated after restarts. Should it always be a noop on dbs that are clean w/r/t the bug? Chris > Chris > >> Adam >> >> On Aug 12, 2010, at 3:14 PM, Jason Smith wrote: >> >>> The code is updated with the following changes: >>> 1. Adhere to the lost+found/databasename custom... >>> 2. ...except databases starting with _, which goes into >>> _system/databasename >>> 3. Sync up with jchris's db_repair branch >>> >>> (About #2, I started with _/database but I think it's too easy to miss at >>> the command line.) >>> >>> On Fri, Aug 13, 2010 at 00:52, J Chris Anderson <jch...@gmail.com> wrote: >>> >>>> A few bug reports from my testing: >>>> >>>> I launched with this command, as specified in the README: >>>> >>>> find ~/code/couchdb/tmp/lib -type f -name '*.couch' -exec ./recover_couchdb >>>> {} \; >>>> >>>> >>>> >>>> First of all, it chokes on my _users and _replicator db: >>>> >>>> [info] [<0.2.0>] couch_db_repair for _users - scanning 335961 bytes at 0 >>>> [error] [<0.2.0>] couch_db_repair merge node at 332061 {case_clause, >>>> {error,illegal_database_name}} >>>> >>>> That second [error] line is repeated many many times (once per merge I >>>> think). I think the issue is that _users is hard-coded to be OK, but >>>> _users_lost+found is not. So we should do something about that, maybe if a >>>> db-name starts with _ we should call the lost and found a_users_lost+found >>>> (_ sorts at the top, so "a" will be near it and legal). >>>> >>>> >>>> >>>> When a database has readers defined in the security object, the tool is >>>> unable to open them (the reading part of the repair tool needs to have the >>>> _admin userCtx, not just the writer). >>>> >>>> [debug] [<0.2.0>] Not a reader: UserCtx {user_ctx,null,[],undefined} vs >>>> Names [<<"joe">>] Roles [<<"_admin">>] >>>> escript: exception throw: {unauthorized,<<"You are not authorized to access >>>> this db.">>} >>>> in function couch_db:open/2 >>>> in call from couch_db_repair:make_lost_and_found/3 >>>> in call from recover_couchdb:main/1 >>>> in call from escript:run/2 >>>> in call from escript:start/1 >>>> in call from init:start_it/1 >>>> in call from init:start_em/1 >>>> >>>> >>>> It would also be helpful if the status lines could say something more than >>>> >>>> [info] [<0.2.0>] couch_db_repair writing 15 updates to bench_lost+found >>>> >>>> Like maybe add a note like "about 23% complete" if at all possible. >>>> >>>> >>>> I will patch the first few, I'd love help from someone on the last one. >>>> I'll be on IRC. >>>> >>>> >>>> Cheers, >>>> Chris >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> On Aug 12, 2010, at 10:18 AM, J Chris Anderson wrote: >>>> >>>>> >>>>> On Aug 11, 2010, at 2:14 PM, Jason Smith wrote: >>>>> >>>>>> Hi, Jason. >>>>>> >>>>>> On Thu, Aug 12, 2010 at 04:14, Jason Smith <j...@couch.io> wrote: >>>>>> >>>>>>> On Wed, Aug 11, 2010 at 09:52, Adam Kocoloski <kocol...@apache.org> >>>> wrote: >>>>>>> >>>>>>>> Excellent, thanks for testing. I caught Jason Smith saying on IRC >>>> that he >>>>>>>> had packaged the whole thing up as an escript + some .beams. If we >>>> can get >>>>>>>> it down to a single file a la rebar that would be a pretty sweet way >>>> to >>>>>>>> deliver the repair tool in my opinion. >>>>>>>> >>>>>>> >>>>>>> Please check out http://github.com/jhs/repair-couchdb >>>>>>> >>>>>> >>>>>> I think you mean http://github.com/jhs/recover-couchdb >>>>>> >>>>> >>>>> I think it is important that we package and release this, if it is ready. >>>> We should link to it from the bug description page, the project home page, >>>> as well as blog about it, etc. What is the point of working feverishly on a >>>> recovery tool if we don't go the last mile? >>>>> >>>>> I am testing it now on my database directory to make sure it doesn't harm >>>> anything (I was never subject to the bug, which is probably where most >>>> people are, but they might run it anyway.) >>>>> >>>>> As it stands the submodules thing can't be part of the release, we need >>>> to package it up as a single zip file or something. >>>>> >>>>> Is there anything else that needs to be done before we can release this? >>>>> >>>>> Chris >>>>> >>>>>> -- >>>>>> Jason Smith >>>>>> Couchio Hosting >>>>> >>>> >>>> >>> >>> >>> -- >>> Jason Smith >>> Couchio Hosting >> >