Hi Mike, The username/password is validated, and there are privileges stored in accumulo for each user. For example, you may be able to read a table, but not write to it.
Zookeeper maintains a consistent, replicated set of data. Accumulo stores configuration information in zookeeper, including an all-important pointer to the root tablet, which allows clients to find out where every tablet is located in the cluster. We need a reference to at least one zookeeper replica in order to get this information. You can provide a comma separated list for redundancy: "zoohost1,zoohost2,zoohost3" I don't know where "accumulate" comes from... I was just +1 for using the full project name "accumulo". The InsertWithBatchWriter example is a pretty simple start: src/examples/src/main/java/org/apache/accumulo/examples/helloworld/InsertWithBatchWriter.java Sure, you can require folks to pass the table in the URI; I just wanted to point out that the table could also be passed as data. There isn't any semantic difference between write and batch-write. There is a bulk-load, which requires the input data to be sorted. The difference between read and batch-read is that the order of results is not guaranteed in batch-read. Accumulo does not have a mutate, in the traditional sense, so lets ignore that for now. There is a difference between read and isolated-read, but perhaps that should just be an option for the read uri? write bulk-load read batch-read What would make the most sense for the URI form: accumulo://instance/write?table=mytable&... or accumulo-write://instance?table=mytable& or accumulo://write?instance=myinstance&table=mytable& or accumulo://write/instance/mytable?... Can you create a ticket for switching to bundles? Since we have no releases of accumulo, now is an ideal time to change the packaging. -Eric On Wed, Oct 26, 2011 at 9:39 AM, <[email protected]> wrote: > > > Eric, > > > > I'm confused about the need for a username/password. Does Accumulo validate > against that, or does it accept a set of priveledges for a user? Also, in > your discussion of zookeeper, what does the "zookeepers" attribute > represent? > > > > For my first stab at this, I'm using the ReadandWrite example to show me > how to connect to Accumulo and perform reads/writes . Is there a better one > that I should be using? This example class seems pretty heavy-weight... > > > > For the table name, I'd like to keep the camel component as generic as > possible. So, to allow for this, I'm going to require folks to pass in the > table name on their accumulate url. Is this ok? > > > > Full Ack on the "accumulate" versus "accum" uri. The only reason I > attempted to shorten it is that looks like the trend for other > camel-components. That said, I can live with a longer name if it makes it > easier to use. Along these lines, I've thought that having a different uri > for reads, writes, and mutates would be an easy way to handle interactions > through the camel component. To help with this, I'm thinking that he > following uri's would be use ful (developed in this order) : > > accumulate-read > > accumulate-write > > accumulate-mutate > > accumulate-batch-read > > accumulate-batch-write > > > > How does that sound? > > > > Also, how do you feel about changing the build so that it creates bundles > instead of .jar files? All a bundle is, is a .jar file with a MANIFEST.MF > file that has some extra attributes. The maven-bundle-plugin can take care > of that pretty easily, but it will change the packaging type from jar to > bundle. This would really only impact folks who are looking for a packaging > type in thier .pom file dependencies, which is not common. > > > > Mike Van > > > > ----- Original Message ----- > > > From: "Eric Newton" <[email protected]> > To: [email protected] > Sent: Wednesday, October 26, 2011 9:06:19 AM > Subject: Re: Camel Accumulo > > Hi Mike, > > Scan auths are needed for reading, not for writing. > > Looking at constructors and factory methods looks like we'll need: > > ZooKeeperInstance: > zookeepers > instance name > > getConnector(): > username > password > > getMultiTableBatchWriter(): > memory > latency > write threads > > This would allow you to stream in tuples of (table, row, cf, cq, > visibility, > timestamp). If you already have a connector for HBase, we would probably > want something compatible with that, and that is probably table oriented > (just guessing, being unfamiliar with Camel). So, perhaps if you specify a > table you would accept tuples of (row, cf, cq, visibility, timestamp). You > could make visibility and timestamp optional, too. > > A minimal uri would look something like this? You could provide reasonable > defaults for the other arguments. > > > accumulo://instance/write?zookeeper=zoohost&userName=root&password=secret > > Unfortunately, "accumulo" doesn't have a nice short abbreviation. I would > lean towards using the whole word, but that's just a personal preference. > > -Eric > > On Wed, Oct 26, 2011 at 12:17 AM, <[email protected]> wrote: > > > > > > > Hi there. > > > > > > > > I recently read about your project and like the direction it is taking. > > Currently, I am a committer to another incubator project, Kalumet, and > have > > been a long-time contributor to the Karaf and Camel projects. After > talking > > with a few of the Accumulo project members, it looks like the most > immediate > > hurdle is growing the user community. I believe I can help with that. > > > > > > > > Making associations between incubator projects and top-level-projects has > > been a proven mechanism to pique developer interest and garner more > > dedicated contributors and commiters. Because of the wide integration of > > Camel and NOSQL databases, creating a Camel component for interaction > with > > Accumulo seems like a no-brainer. > > > > > > > > In order to help grow the Accumulo community, tonight I began writing a > > camel-accumulo component. This will allow Camel users to route files to > > Accumulo in the same manner as they currently route files to HDFS . > > > > > > > > > > > > For some background, Camel is the open-source implementation of > Enterprise > > Integration Patterns. Most modern ESB's use OSGI and Camel to perform > > routing and orchestration of data through endpoints. Camel has been > written > > to allow various technologies to create Camel Components that folks can > use > > when they define the route that a given file or data will be processed > > through. In this model, users would define a "route" in Camel that > contains > > various Accumulo endpoints for reading, writing, or mutating data > persisted > > through Accumulo. To make this work, I need to define a URI that folks > will > > use. Would you folks be able to help me define the URI and URL > parameters? > > > > > > > > Right now, I'm using the URI "accum". For the first iteration of this > > component, I'm thinking it would be simplest to create an endpoint folks > > could write a single re cord to. Then, follow it up with a scan, and > mutate > > components. Once these are done, I'd like to do the batch-versions of > these > > operations. > > > > > > > > In Camel, an endpoint usually looks similar to a web-service endpoint: > > URI://location/service ?[arg1=value1][&arg<x>=value<x>] > > > > > > > > With this in mind, I'm thinking the following would be the minimally > > acceptable Camel-Accumulo endpoint for simplistic write operations: > > > > accum://location /write ?zookeeperName=value&\ > > > > tableName=value& \ > > > > userName=value&\ > > > > password=value&\ > > > > userPrivs=value&\ > > > > scanAuths=value&\ > > > > debug=value > > > > > > > > Does this contain a c omplete listing of the properties? A m I missing > > anything, did I put something in that's not needed, or are there other > > options a user should be able to pass in? Also, is the URI of "accum" ok > for > > this camel component? > > > > > > > > Because Camel is written to play nicely inside of OSGI (like > Karaf/Felix), > > the .jar files this camel component relies on should be bundle-ized. This > > shouldn't be too hard to do, and as a Karaf contributor, I've done this > with > > hundreds of third party .jar files. Basically, we would replace the > > maven-jar-plugin with a light implementation of the maven-bundle-plugin > > along with some fairly generic attributes. If you folks would like, I > can > > do this for you on a seperate branch so that you can test it. > > > > > > > > Mike Van > > > > Committer - ASF Kalumet > > > > Contributor - ASF Karaf, Camel >
