Not sure at all. I don't know how to check precisely if a live design doc
is pointing to a particular file. I was basing my statement off the fact
that I have my views declared and they were available pre-indexed before
compaction (but they were not physically opened as file handles by couch,
but they were opened on demand). Once I finish my current script, I will
test everything again and will spend some time tracing the code.
On Fri, Jan 31, 2014 at 6:52 PM, Robert Samuel Newson <rnew...@apache.org>wrote:
>
> Ownership is interesting. Would the bigcouch user have the right to delete
> the file but not open it for reading?
>
> There's definitely an issue in bigcouch (fixed long since in couchdb)
> where any failure to open a view file makes us delete it.
>
> OS/fs all check out fine, You see the filename that should be retained in
> that log output? you're 100% sure? You do have a live design doc pointing
> to it?
>
> B.
>
> On 31 Jan 2014, at 16:39, Vladimir Ralev <vladimir.ra...@gmail.com> wrote:
>
> > Thanks a lot. The database was moved from older machines so some other
> file
> > system metadata might be scrambled. But I don't see what can cause a
> > problem like this.
> >
> > Yes the debug output is seen "deleting unused view index files:" and it
> > deletes every view in every database, little doubt about it. It doesn't
> > delete fresh views though that are fully regenerated afterwards. I think
> > the original views somehow got corrupted, but I need to figure out why
> and
> > may be fix it manually with a script
> >
> > OS is Debian 64, file system is ext4, there is a little scramble of the
> > file ownership, some directories are owned by old bigcouch user, others
> by
> > root, so that's one thing I am investigating. I reset the ownership, but
> > will have to repeat it for my next tests.
> >
> >
> >
> >
> > On Fri, Jan 31, 2014 at 6:21 PM, Robert Samuel Newson <
> rnew...@apache.org>wrote:
> >
> >> and details of OS, filesystem, anything you think might be relevant.
> >>
> >> B.
> >>
> >> On 31 Jan 2014, at 16:20, Robert Samuel Newson <rnew...@apache.org>
> wrote:
> >>
> >>> First thing to note is that bigcouch development is over, but we can at
> >> least confirm this;
> >>>
> >>> This function fetches all the design docs of the database, grabs all
> the
> >> signatures from each (you'll have noticed view filenames look
> uuid/randomy,
> >> that's a 'sig'), and then sweeps the dir where all views for the given
> >> database should be and deletes those not in the 'keep' list.
> >>>
> >>> Can you enable debug level logging (curl
> >> localhost:5984/_config/log/level -X PUT -d '"debug"' to *all* bigcouch
> >> nodes) and tell us if ;
> >>>
> >>> ?LOG_DEBUG("deleting unused view index files: ~p",[DeleteFiles]),
> >>>
> >>> actually gets printed?
> >>>
> >>> B.
> >>>
> >>> On 31 Jan 2014, at 16:09, Vladimir Ralev <vladimir.ra...@gmail.com>
> >> wrote:
> >>>
> >>>> Hi guys,
> >>>>
> >>>> bigcouch 0.4.2 has the following code that handles view cleanup:
> >>>>
> >>>> cleanup_index_files(Db) ->
> >>>>
> >>>> % load all ddocs
> >>>>
> >>>> {ok, DesignDocs} = couch_db:get_design_docs(Db),
> >>>>
> >>>>
> >>>> % make unique list of group sigs
> >>>>
> >>>> Sigs = lists:map(fun(#doc{id = GroupId}) ->
> >>>>
> >>>> {ok, Info} = get_group_info(Db, GroupId),
> >>>>
> >>>> ?b2l(couch_util:get_value(signature, Info))
> >>>>
> >>>> end, [DD||DD <- DesignDocs, DD#doc.deleted == false]),
> >>>>
> >>>>
> >>>> FileList = list_index_files(Db),
> >>>>
> >>>>
> >>>> DeleteFiles =
> >>>>
> >>>> if length(Sigs) =:= 0 ->
> >>>>
> >>>> FileList;
> >>>>
> >>>> true ->
> >>>>
> >>>> % regex that matches all ddocs
> >>>>
> >>>> RegExp = "("++ string:join(Sigs, "|") ++")",
> >>>>
> >>>>
> >>>> % filter out the ones in use
> >>>>
> >>>> [FilePath || FilePath <- FileList,
> >>>>
> >>>> re:run(FilePath, RegExp, [{capture, none}]) =:= nomatch]
> >>>>
> >>>> end,
> >>>>
> >>>>
> >>>> % delete unused files
> >>>>
> >>>> ?LOG_DEBUG("deleting unused view index files: ~p",[DeleteFiles]),
> >>>>
> >>>> RootDir = couch_config:get("couchdb", "view_index_dir"),
> >>>>
> >>>> [couch_file:delete(RootDir,File,false)||File <- DeleteFiles],
> >>>>
> >>>> ok.
> >>>>
> >>>>
> >>>> From here
> >>>>
> >>
> https://github.com/cloudant/bigcouch/blob/master/apps/couch/src/couch_view.erl#L84
> >>>>
> >>>> It's supposed to delete only unused views, but in my case it deletes
> >>>> everything and then starts building from scratch. Can you help me
> >>>> understand the condition used here to filter the files that are
> >> currently
> >>>> in use? How is the regex supposed to work.
> >>>
> >>
> >>
>
>
