I have never seen this happen, I have found zookeeper to be very reliable. I think Accumulo needs a utility to handle this case of reinitializing just zookeeper. Would you like to open a ticket?
Zookeeper does store some important persistent info, like mappings of table names to table ids, table config, users data, and FATE ops. On Thu, Jul 5, 2012 at 11:15 AM, Krishmin Rai <[email protected]> wrote: > Hi All, > We've recently encountered a strange situation on a small test cluster: > after an awkward crash, our ZooKeeper data was erased and we no longer have > the [accumulo] znode. The HDFS accumulo directory is intact, so all the > RFiles and etc are still there, but it's not clear how best to bring Accumulo > back up to its previous state. Obviously just starting Accumulo as-is > complains about the missing znode ("Waiting for accumulo to be initialized"), > whereas re-initializing is not possible over existing HDFS directories ("It > appears this location was previously initialized, exiting"). > > A couple of questions about recovery strategies: > > 1) Is there any way to re-create the znode for a previous instance-id? My > understanding is that ZK is mostly used to store ephemeral data (such as > which tserver is currently responsible for which tablets) and things like > users (which we could re-create), so perhaps this is plausible? > > 2) I imagine that I could init Accumulo with a new instance.dfs.dir, then > import the RFiles from the old installation back in. I see Patrick just asked > a related question, so, with the data integrity caveats, I would essentially > be following the last of the steps in ACCUMULO-456. > > 3) This is a vague question, but have any of you had experience with the > [accumulo] znode being entirely deleted? Aside from stopping/starting ZK > (3.3.5) and Accumulo 1.4.0 (possibly with a force-quit), I'm not sure what we > could have done to actually delete it. > > This is just a test instance, and the data could easily be recreated, but I > want to take this opportunity to learn a little more about Accumulo plumbing > and maintenance. > > Thanks, > Krishmin > > > >
