Re: multi-table isolated batch scanner

2013-04-16 Thread Keith Turner
On Mon, Apr 15, 2013 at 6:19 PM, Adam Fuchs wrote: > Keith, > > In this case we're filling the buffer before we can amortize the search > cost. We're using a document-partitioned table design and we have to do a > local sort before we can get the first result. > I am not sure exactly what you are

Re: simple cluster setup

2013-04-16 Thread Josh Elser
Drew, I'm not an ec2 guru but I'll try to help: the IP addresses (elastic IP's, local ip's) to use in masters/slaves, You likely want to use the local ip's. The nodes just need to be routable to one another. are the slaves using a single zookeeper server or their own on localhost You li

Re: simple cluster setup

2013-04-16 Thread Kevin Faro
This may be a little out of date but will probably still be useful. http://www.accumulodata.com/ec2.html --Kevin On Tue, Apr 16, 2013 at 9:43 AM, Josh Elser wrote: > Drew, > > I'm not an ec2 guru but I'll try to help: > > > the IP addresses (elastic IP's, local ip's) to use in masters/slaves

Re: multi-table isolated batch scanner

2013-04-16 Thread Adam Fuchs
Keith, I'm essentially performing a local sort-merge inner join (local meaning within one tablet at a time). This is similar to what the intersecting iterator does, but I'm doing it on data that is not already sorted. Applications of this include any kind of indexed boolean logic predicate search

VFS class reloading?

2013-04-16 Thread John Vines
Maybe I missed something with the switch to the VFS classloader, but does dynamic loading out of lib/ext no longer work? I had accumulo 1.5 running, threw an iterator in there, but had to restart tserver to get the new iterator picked up. Was that an intentional change?

RE: VFS class reloading?

2013-04-16 Thread Dave Marion
The implementation changed several times, so the pre-1.5 layout may not work. In 1.5, using the bootstrap script, it should put the accumulo jars into HDFS and dynamic loading from there should occur. I'll try and test tonight if I have time. -Original Message- From: John Vines [mailto:vi.

RE: VFS class reloading?

2013-04-16 Thread Dave Marion
Looking at the code, it should work. Keith and I had several conversations about what the new classloader should do. I believe that he wanted it to behave like the old one and what I see in the code supports that. If it is not working, then I would say create a ticket for it for now. I'll try to re

Re: VFS class reloading?

2013-04-16 Thread Eric Newton
We have tests for dynamic loading of classes so I'm pretty sure it works. John, can you repeat the failure? -Eric On Tue, Apr 16, 2013 at 6:50 PM, Dave Marion wrote: > Looking at the code, it should work. Keith and I had several conversations > about what the new classloader should do. I bel

Re: MiniAccumuloCluster backport

2013-04-16 Thread Corey Nolet
Anyone against bumping the junit version in 1.4.4 to 4.11? On Apr 15, 2013, at 10:40 PM, Keith Turner wrote: > On Mon, Apr 15, 2013 at 10:07 PM, Corey Nolet wrote: >> I'm working through the back port from 1.5. I've got it running the basic >> operations that are supported in 1.4, however unle

Re: VFS class reloading?

2013-04-16 Thread dlmarion
Updated my local 1.5 branch and tried to build with "mvn clean package -P assemble -Dhadoop.version=0.20.205.0". I'm running CDH3 Update 3 locally, so that should work. A bunch of tests failed in core with: Caused by: java.lang.ClassNotFoundException: org.codehaus.jackson.map.JsonMappingExcep

Re: VFS class reloading?

2013-04-16 Thread Corey Nolet
I've run into version conflicts with Hadoop 0.20.2 CDH3 series and jackson versions < 1.9 in the past. On Apr 16, 2013, at 8:39 PM, dlmar...@comcast.net wrote: > > Updated my local 1.5 branch and tried to build with "mvn clean package -P > assemble -Dhadoop.version=0.20.205.0". I'm running CD

Re: VFS class reloading?

2013-04-16 Thread Josh Elser
I wish I could tell you that it worked on my machine, but alas the command below did fail on a bunch of core tests via Accumulo*FormatTest$MRTester. Looking into it currently. Maybe someone accidentally broke compatibility with the old Hadoop stuff and we need to add a new dependency with 0.2

Re: VFS class reloading?

2013-04-16 Thread dlmarion
I would believe that at runtime, but during the build? - Original Message - From: "Corey Nolet" To: dev@accumulo.apache.org Sent: Tuesday, April 16, 2013 8:47:13 PM Subject: Re: VFS class reloading? I've run into version conflicts with Hadoop 0.20.2 CDH3 series and jackson versions

Re: VFS class reloading?

2013-04-16 Thread dlmarion
Likely. I just followed the directions in the README as I have not kept up with all of the changes in 1.5. It seems that maybe we should have Jenkins build with all of the different profile variations, or at least the ones that we use as examples in the README. - Original Message - Fro

Re: VFS class reloading?

2013-04-16 Thread dlmarion
I was able to confirm that 'accumulo classpath' picked up jars in lib/ext. So it's configured to know about that location. - Original Message - From: "Eric Newton" To: dev@accumulo.apache.org Sent: Tuesday, April 16, 2013 7:52:09 PM Subject: Re: VFS class reloading? We have tests f

Re: VFS class reloading?

2013-04-16 Thread Josh Elser
Possibly. Being rather explicit about what we support is definitely good; finding out when we accidentally break something we're supposed to support... also a good idea. :D I'm almost of the mindset to not really encourage people to use 0.20.205 hadoop things anymore, maybe? I have no idea wha

Re: VFS class reloading?

2013-04-16 Thread Josh Elser
Dave, Patch attached which let me build against 0.20.205.0 Run: mvn package -Dhadoop.profile=0.20 -Passemble I took the extra second to follow the same convention we have for Hadoop 1 and 2. I completely guessed on the version of jackson-mapper-asl we should use, so that's why I didn't commit

Re: VFS class reloading?

2013-04-16 Thread Billie Rinaldi
On Tue, Apr 16, 2013 at 5:39 PM, wrote: > > Updated my local 1.5 branch and tried to build with "mvn clean package -P > assemble -Dhadoop.version=0.20.205.0". I'm running CDH3 Update 3 locally, > so that should work. A bunch of tests failed in core with: > > Caused by: java.lang.ClassNotFoundExce

Re: VFS class reloading?

2013-04-16 Thread Josh Elser
I think I'm ok with dropping 0.20.205.0 support, but, as Dave pointed out, we need to update the README to reflect such (really needs to happen before 1.5.0 drops). I'll drop a poll to see if anyone who's only subscribed to user@a.a.o has any concerns on the matter. On 04/16/2013 09:59 PM, B