On Sun, Sep 13, 2020 at 6:58 PM Bulldog20630405 <[email protected]> wrote: > > we are currently using 1.9.3; however will probably upgrade to the new 1.10 > soon now that it is released (has some nice bug fixes) > > yes, i was using the shell; however, i have an admin tool which wraps the > api... i will look into that... > > from what i have seen from the api; you can merge using begin and end splits; > however, you cannot use size and force (unless you used the util merge > command)
That's right. You'd have to identify the splits you want to merge yourself, and then use the API to do each one directly. You may even be able to reuse the code in the Merge.java utility, or modify it to remove the restriction (carefully... since I don't know why the limitation was put in place, I'm not sure it would be 100% safe to do so). > > questions: > 1) will the utility merge command work with metadata table? The shell uses the Merge.java utility, so that utility would have the same restrictions. > 2) is there anything we should do before running the merge from the api? > (e.g. should we stop all ingest and query before running metadata merge?) If it were me, I would probably stop ingest and accumulo-gc and offline all the user tables, just to be safe, and then flush, compact, and back up the root and metadata tables. You are, after all, doing some metadata surgery, so you can't be too careful. It would also be good to experiment on a test system first. > 3) general question; why does the shell not allow it; when it is not an issue > with the metadata table? I'm not sure. It could be because the code used by the shell predates the ability to do merges on the metadata table, and it was never updated. It could also be the case that we didn't want to make it too easy for users to shoot themselves in the foot by automating a procedure that should be done more cautiously. Perhaps it's just riskier to make such changes on the metadata table while tables are being hosted in a live system. I'm not sure the exact reason. It's something we could look at changing for 2.1, if it's thoroughly tested. > 4) how can be update the documentation to explain this? Some places where the limitation could be documented are: docs/ directory in the 1.10 branch, the shell merge command's usage/help, the accumulo-website repo for 2.x documentation Right now, I'm not sure what would need to be updated, since the command itself clearly declares its limitation that it can't operate on the metadata table, interactively, when requested to do so. If I knew more about why the decision was made to restrict it, I could suggest some changes, but I don't really know the answer to that at this time. Feel free to submit a pull request with suggested changes. > > thanx > > > On Sun, Sep 13, 2020 at 3:32 AM Christopher <[email protected]> wrote: >> >> Can you provide more information, such as: >> * version of Accumulo >> * the actual error message you see >> * any stack trace corresponding to the error message, in full, if possible >> >> The shell command uses an admin utility that is unable to merge >> metadata tables. However, I believe the metadata table can be merged >> using the Java API directly, rather than the shell command. Have you >> tried that? >> >> On Sat, Sep 12, 2020 at 5:47 PM Bulldog20630405 >> <[email protected]> wrote: >> > >> > >> > >> > over time with table deleting, etc... accumulo.metadata "accumulates" zero >> > entry splits... currently we have about 25% splits in our >> > accumulo.metadata with zero entries... when trying to merge using --size >> > 1k --force we get an error saying we cannot merge the metadata table. >> > >> > how do you get rid of these worthless splits? >> > >> > the only way i know how to do it is to: >> > 1) backup all our tables >> > 2) rebuild accumulo >> > 3) import all our backuped tables >> > >> > however, that seems to be a lot of work! >> > >> > any help would be greatly appreciated >> > >> >
