Re: Unit/Integration tests with Mini Accumulo Cluster

Jim, I've found that sharing a minicluster instance via a lazily-initialized static singleton is simple and effective within tests. The reason I lean toward this over the maven plugin is because it works both when executing tests as part of a maven build and when executing tests from an IDE.

Re: Unit/Integration tests with Mini Accumulo Cluster

One more point about Fluo. ITBase relies on maven setting a property. That is done by the following link. https://github.com/apache/fluo/blob/549d645addb330f4ae2e074447428cb86b5a9a3f/pom.xml#L333 On Fri, Dec 13, 2019 at 10:30 AM Jim Hughes wrote: > > Hi all, > > I work on GeoMesa and for the

Re: Unit/Integration tests with Mini Accumulo Cluster

You could use the Accumulo Maven Plugin which starts an Accumulo cluster for using MiniAccumulo. https://accumulo.apache.org/accumulo2-maven-plugin/ Fluo, uses this and below are a few pointers. The following is a base for Fluo ITs that uses the mini Accumulo started by the maven plugin if it

Re: Unit/Integration tests with Mini Accumulo Cluster

The Accumulo ITs (any java files whose name ends in IT) would be a good example since they are all run using MiniAccumuloCluster, with some being able to run against a standalone cluster[1]. Some ITs start and stop a cluster for every test, while others keep a cluster running for the whole IT.

Unit/Integration tests with Mini Accumulo Cluster

Hi all, I work on GeoMesa and for the Accumulo 1.x line, we have been using the MockAccumulo infrastructure for our unit/integration tests which run in a Maven build.  In Accumulo 2.x, since MockAccumulo is gone, we're looking at using the MiniAccumulo cluster infrastructure. Are there best

Re: Mini Accumulo Cluster reusing the directory

...@scai.fraunhofer.de www.scai.fraunhofer.de - Ursprüngliche Mail - > Von: "Keith Turner" <ke...@deenlo.com> > An: "user" <user@accumulo.apache.org> > Gesendet: Mittwoch, 16. September 2015 21:41:00 > Betreff: Re: Mini Accumulo Cluster reusing the di

Re: Mini Accumulo Cluster reusing the directory

Thanks Josh :) On 09/17/2015 10:09 AM, Josh Elser wrote: And, to be explicit, if you want to configure not-MiniAccumuloCluster to use the local FS (you only run on one node), just use file:///... in instance.volumes (or instance.dfs.uri) instead of hdfs://namenode:port/... Wasn't sure if it

Re: Mini Accumulo Cluster reusing the directory

On Thu, Sep 17, 2015 at 12:38 AM, Josh Elser wrote: > MiniAccumuloCluster by default already uses the local file system. > > To get the correct sync semantics that Keith mentioned you need to set > "fs.file.impl" in the Hadoop Configuration you pass to the >

Re: Mini Accumulo Cluster reusing the directory

I created a util for doing this, but i haven't worked on it in a while... You could see if it still works https://github.com/agwells0714/AccumuloDeveloperUtil On Wed, Sep 16, 2015 at 10:31 AM, Corey Nolet wrote: > Sven, > > What version of Accumulo are you running? We have

Re: Mini Accumulo Cluster reusing the directory

btw, my solution revolved around this issue: https://issues.apache.org/jira/browse/ACCUMULO-2851 which let users save their data off somewhere, then load it into the new mac cluster On Wed, Sep 16, 2015 at 10:37 AM, Andrew Wells wrote: > I created a util for doing this,

Re: Mini Accumulo Cluster reusing the directory

code from Christopher somewhere deployed? > > Currently I'm using version 1.7 > > Regards, > Sven > > - Ursprüngliche Mail - > > Von: "Corey Nolet" <cjno...@gmail.com> > > An: "user" <user@accumulo.apache.org> > > Ge

Re: Mini Accumulo Cluster reusing the directory

Sven, What version of Accumulo are you running? We have a ticket for this [1] which has had a lot of discussion on it. Christopher Tubbs mentioned that he had gotten this to work. [1] https://issues.apache.org/jira/browse/ACCUMULO-1378 On Wed, Sep 16, 2015 at 9:20 AM, Sven Hodapp

Re: Mini Accumulo Cluster reusing the directory

> An: "user" <user@accumulo.apache.org> > Gesendet: Mittwoch, 16. September 2015 16:31:02 > Betreff: Re: Mini Accumulo Cluster reusing the directory > Sven, > > What version of Accumulo are you running? We have a ticket for this [1] > which has had a lot of discussion

Re: Mini Accumulo Cluster reusing the directory

>> >> Currently I'm using version 1.7 >> >> Regards, >> Sven >> >> - Ursprüngliche Mail - >> > Von: "Corey Nolet" <cjno...@gmail.com> >> > An: "user" <user@accumulo.apache.org> >> > Gesendet: Mittwoc

Re: Mini Accumulo Cluster reusing the directory

n/m, didn't read the whole thread. Sounds like ACCUMULO-1378 does capture what you asked for :) Josh Elser wrote: Best as I can tell, we don't actually have the ability to do this. We can use external ZooKeeper servers and we can start a MAC instance on top of an already running ZK from a

Re: Mini Accumulo Cluster reusing the directory

Best as I can tell, we don't actually have the ability to do this. We can use external ZooKeeper servers and we can start a MAC instance on top of an already running ZK from a different MAC instance, but it doesn't look like we have the ability to reuse the same storage directory. That

Re: Mini Accumulo Cluster reusing the directory

Would you be able to provide more informaiton about your use case? Was wondering if other solutions could be of use, like configuring regular Accumulo to use the local filesystem. This can be done, but care needs to be taken to make walogs work correctly. If interested I could provide more

Re: Mini Accumulo Cluster reusing the directory

where deployed? >>> >>> Currently I'm using version 1.7 >>> >>> Regards, >>> Sven >>> >>> - Ursprüngliche Mail - >>> > Von: "Corey Nolet" <cjno...@gmail.com> >>> > An: "user" <user@accumu

Re: Mini Accumulo Cluster reusing the directory

And, to be explicit, if you want to configure not-MiniAccumuloCluster to use the local FS (you only run on one node), just use file:///... in instance.volumes (or instance.dfs.uri) instead of hdfs://namenode:port/... Wasn't sure if it would be implied to look at how MAC does this from my

Re: Mini Accumulo Cluster reusing the directory

MiniAccumuloCluster by default already uses the local file system. To get the correct sync semantics that Keith mentioned you need to set "fs.file.impl" in the Hadoop Configuration you pass to the MiniAccumuloConfig. See

Re: Mini Accumulo Cluster reusing the directory

Keith, How can I configure Accumulo to use local file system??? On 09/17/2015 01:11 AM, Keith Turner wrote: Would you be able to provide more informaiton about your use case? Was wondering if other solutions could be of use, like configuring regular Accumulo to use the local filesystem.

Re: Mini Accumulo cluster

Josh, Thanks for your response. My iterators will do the same number of seeks, they're only different in the implementation of the functions used to perform filtering, so I think I'll get a reasonable comparison but I won't read too much into the results. On 13 May 2015 at 21:19, Josh Elser

Re: Mini Accumulo cluster

As long as you're managing your expectations (which I sounds like you've considered well), there could be some worth. A concern would be how using a different filesystem implementation actually impacts the validity of your benchmark though. e.g. w/ a local FS (which is by default what MAC

Mini Accumulo cluster

Hi, Is it crazy to use a MiniAccumuloCluster to measure the *relative* performance of two different implementations of iterators? Obviously it would be better to do it on a real Accumulo cluster, but that's not possible for several reasons. The approach would be something like: - Fire up a Mini