On Thu, Sep 17, 2015 at 12:38 AM, Josh Elser <josh.el...@gmail.com> wrote:
> MiniAccumuloCluster by default already uses the local file system. > > To get the correct sync semantics that Keith mentioned you need to set > "fs.file.impl" in the Hadoop Configuration you pass to the > MiniAccumuloConfig. > Yeah thats the main thing I was thinking of. Thanks for digging it up Josh. Its really important to set that hadoop config when having Accumulo use the local FS. Otherwise the default class hadoop uses for the local fs does nothing for flush. A few caveats to be aware of : * The default impl is LocalFileSystem and it used to do nothing for flush(). That may no longer be true, at one point it extended something from CheckSumFS which did nothing for flush. * RawLocalFileSystem may not checksum data like LocalFileSystem does * RawLocalFileSystem may not survive a power outage. It flushes the file to the OS... but the OS may hold it in its buffers for some time before really writing it to disk. So this means you would survive processes terminating, but not the OS terminating unexpectedly. When using HDFS Accumulo can be configured to make the hdfs output stream make java.nio calls to flush data to disk. > See > https://github.com/apache/accumulo/blob/master/test/src/main/java/org/apache/accumulo/test/VolumeIT.java#L113 > for code. > > mohit.kaushik wrote: > >> Keith, >> >> How can I configure Accumulo to use local file system??? >> >> On 09/17/2015 01:11 AM, Keith Turner wrote: >> >>> Would you be able to provide more informaiton about your use case? >>> Was wondering if other solutions could be of use, like configuring >>> regular Accumulo to use the local filesystem. This can be done, but >>> care needs to be taken to make walogs work correctly. If interested >>> I could provide more info about this configuration. >>> >>> On Wed, Sep 16, 2015 at 9:20 AM, Sven Hodapp >>> <sven.hod...@scai.fraunhofer.de >>> <mailto:sven.hod...@scai.fraunhofer.de>> wrote: >>> >>> Hi there, >>> >>> is it possible for MiniAccumuloCluster to reuse a given directory? >>> Sadly, I haven't found anything in the docs? >>> >>> I’ll fire up my instance like this: >>> >>> val dict = new File("/tmp/accumulo-mini-cluster") >>> val accumulo = new MiniAccumuloCluster(dict, "test“) >>> >>> If I’ll restart my JVM it will raise a error like this: >>> >>> Exception in thread "main" java.lang.IllegalArgumentException: >>> Directory /tmp/accumulo-mini-cluster is not empty >>> >>> It would be nice if the data can survive a JVM restart and the >>> folder structure must not be constructed every time. >>> >>> Thanks a lot! >>> >>> Regards, >>> Sven >>> >>> -- >>> Sven Hodapp M.Sc., >>> Fraunhofer Institute for Algorithms and Scientific Computing SCAI, >>> Department of Bioinformatics >>> Schloss Birlinghoven, 53754 Sankt Augustin, Germany >>> sven.hod...@scai.fraunhofer.de <mailto: >>> sven.hod...@scai.fraunhofer.de> >>> www.scai.fraunhofer.de <http://www.scai.fraunhofer.de> >>> >>> >>> >> >> -- >> >> *Mohit Kaushik* >> Software Engineer >> A Square,Plot No. 278, Udyog Vihar, Phase 2, Gurgaon 122016, India >> *Tel:*+91 (124) 4969352 | *Fax:*+91 (124) 4033553 >> >> <http://politicomapper.orkash.com>interactive social intelligence at >> work... >> >> <https://www.facebook.com/Orkash2012> >> <http://www.linkedin.com/company/orkash-services-private-limited> >> <https://twitter.com/Orkash> <http://www.orkash.com/blog/> >> <http://www.orkash.com> >> <http://www.orkash.com> ... ensuring Assurance in complexity and >> uncertainty >> >> /This message including the attachments, if any, is a confidential >> business communication. If you are not the intended recipient it may be >> unlawful for you to read, copy, distribute, disclose or otherwise use >> the information in this e-mail. If you have received it in error or are >> not the intended recipient, please destroy it and notify the sender >> immediately. Thank you / >> >>
