Hi Keith,
Thanks for clarifying.
For now, it make sense to go with the init approach & look at the fix in
2.1 when released.
Cheers,
Karthick
-----Original Message-----
From: Keith Turner <ke...@deenlo.com>
Sent: Wednesday, November 20, 2019 7:28 AM
To: Accumulo Dev List <dev@accumulo.apache.org>
Subject: Re: Multiple instance volumes
On Tue, Nov 19, 2019 at 6:45 PM karthick rn <
karthick.narend...@gmail.com> wrote:
>
> Hi,
>
> When provisioning multiple volumes, for ex. HDFS & Azure Data Lake
> storage, it would be good to choose which volume we want the system
> tables like metadata, root, replication tables to be created.
> Currently, Accumulo randomly creates these tables on multiple volumes
> and the only way to control this is to run Accumulo init on 1st
volume
> so all system tables get created on this volume and then add the
other volumes.
>
> I have also tried running 'config -t accumulo.metadata -s
> table.custom.volume.preferred=hdfs://accucluster/accumulo' in an
> attempt to move the metadata table to a preferred volume, in this
case
> HDFS, but I don’t see metadata table under HDFS. Also, ‘config -f
> table.custom.volume.preferred’ does not show anything!
> In short, I was wondering if there is any provision to "move" these
> system tables across volumes, or is that a non-goal by design?
Accumulo volume choices are currently sticky. Each tablet chooses a
volume where it will put files and it remembers that choice (stored in the
metadata table under the srv:dir column). After that choice is made, the
tablet will always put files on that volume. I don't think changing the
volume choose config and then compacting will fix this.
I think the best option is to run init carefully as you mentioned
and/or look into fixing #1373.
For 2.1.0 this is fixed by #1389[1] and choices are no longer sticky.
So in 2.1, you should be able to change config and then compact.
Another option for 2.0 and earlier would be to perform surgery on
Accumulo's metadata, but this would be tricky for the root tablet.
The following is speculation on what needs to be done. For the
metadata table, you could change the srv:dir entries stored in the root
tablet (then restart and compact metadata table). For the root tablet
itself, you would need to change its dir entry in zookeeper AND move its
files to the location specified in zookeeper (doing all of this while
Accumulo is down). Accumulo has an admin utility to do this type of thing
(it was removed in #1389), but I looked at that utility and its really only
suited for user tables and not the metadata or root table. We could
possibly make that utility support the metadata and root tables, make it
know how to do the surgery I mentioned.
[1]:
https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Faccumulo%2Fpull%2F1389&data=02%7C01%7CKarthick.Narendran%40microsoft.com%7C46ab4e85c1a84749774608d76dd5633f%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637098635611706427&sdata=YRhlsKjnm7zTI1vNz0ZPiM2xH1JY1O7HX0DF47z1RSg%3D&reserved=0
>
> Many thanks
>
> Regards,
> Karthick
