Eric,
I'm confused about the need for a username/password. Does Accumulo validate against that, or does it accept a set of priveledges for a user? Also, in your discussion of zookeeper, what does the "zookeepers" attribute represent? For my first stab at this, I'm using the ReadandWrite example to show me how to connect to Accumulo and perform reads/writes . Is there a better one that I should be using? This example class seems pretty heavy-weight... For the table name, I'd like to keep the camel component as generic as possible. So, to allow for this, I'm going to require folks to pass in the table name on their accumulate url. Is this ok? Full Ack on the "accumulate" versus "accum" uri. The only reason I attempted to shorten it is that looks like the trend for other camel-components. That said, I can live with a longer name if it makes it easier to use. Along these lines, I've thought that having a different uri for reads, writes, and mutates would be an easy way to handle interactions through the camel component. To help with this, I'm thinking that he following uri's would be use ful (developed in this order) : accumulate-read accumulate-write accumulate-mutate accumulate-batch-read accumulate-batch-write How does that sound? Also, how do you feel about changing the build so that it creates bundles instead of .jar files? All a bundle is, is a .jar file with a MANIFEST.MF file that has some extra attributes. The maven-bundle-plugin can take care of that pretty easily, but it will change the packaging type from jar to bundle. This would really only impact folks who are looking for a packaging type in thier .pom file dependencies, which is not common. Mike Van ----- Original Message ----- From: "Eric Newton" <[email protected]> To: [email protected] Sent: Wednesday, October 26, 2011 9:06:19 AM Subject: Re: Camel Accumulo Hi Mike, Scan auths are needed for reading, not for writing. Looking at constructors and factory methods looks like we'll need: ZooKeeperInstance: zookeepers instance name getConnector(): username password getMultiTableBatchWriter(): memory latency write threads This would allow you to stream in tuples of (table, row, cf, cq, visibility, timestamp). If you already have a connector for HBase, we would probably want something compatible with that, and that is probably table oriented (just guessing, being unfamiliar with Camel). So, perhaps if you specify a table you would accept tuples of (row, cf, cq, visibility, timestamp). You could make visibility and timestamp optional, too. A minimal uri would look something like this? You could provide reasonable defaults for the other arguments. accumulo://instance/write?zookeeper=zoohost&userName=root&password=secret Unfortunately, "accumulo" doesn't have a nice short abbreviation. I would lean towards using the whole word, but that's just a personal preference. -Eric On Wed, Oct 26, 2011 at 12:17 AM, <[email protected]> wrote: > > > Hi there. > > > > I recently read about your project and like the direction it is taking. > Currently, I am a committer to another incubator project, Kalumet, and have > been a long-time contributor to the Karaf and Camel projects. After talking > with a few of the Accumulo project members, it looks like the most immediate > hurdle is growing the user community. I believe I can help with that. > > > > Making associations between incubator projects and top-level-projects has > been a proven mechanism to pique developer interest and garner more > dedicated contributors and commiters. Because of the wide integration of > Camel and NOSQL databases, creating a Camel component for interaction with > Accumulo seems like a no-brainer. > > > > In order to help grow the Accumulo community, tonight I began writing a > camel-accumulo component. This will allow Camel users to route files to > Accumulo in the same manner as they currently route files to HDFS . > > > > > > For some background, Camel is the open-source implementation of Enterprise > Integration Patterns. Most modern ESB's use OSGI and Camel to perform > routing and orchestration of data through endpoints. Camel has been written > to allow various technologies to create Camel Components that folks can use > when they define the route that a given file or data will be processed > through. In this model, users would define a "route" in Camel that contains > various Accumulo endpoints for reading, writing, or mutating data persisted > through Accumulo. To make this work, I need to define a URI that folks will > use. Would you folks be able to help me define the URI and URL parameters? > > > > Right now, I'm using the URI "accum". For the first iteration of this > component, I'm thinking it would be simplest to create an endpoint folks > could write a single re cord to. Then, follow it up with a scan, and mutate > components. Once these are done, I'd like to do the batch-versions of these > operations. > > > > In Camel, an endpoint usually looks similar to a web-service endpoint: > URI://location/service ?[arg1=value1][&arg<x>=value<x>] > > > > With this in mind, I'm thinking the following would be the minimally > acceptable Camel-Accumulo endpoint for simplistic write operations: > > accum://location /write ?zookeeperName=value&\ > > tableName=value& \ > > userName=value&\ > > password=value&\ > > userPrivs=value&\ > > scanAuths=value&\ > > debug=value > > > > Does this contain a c omplete listing of the properties? A m I missing > anything, did I put something in that's not needed, or are there other > options a user should be able to pass in? Also, is the URI of "accum" ok for > this camel component? > > > > Because Camel is written to play nicely inside of OSGI (like Karaf/Felix), > the .jar files this camel component relies on should be bundle-ized. This > shouldn't be too hard to do, and as a Karaf contributor, I've done this with > hundreds of third party .jar files. Basically, we would replace the > maven-jar-plugin with a light implementation of the maven-bundle-plugin > along with some fairly generic attributes. If you folks would like, I can > do this for you on a seperate branch so that you can test it. > > > > Mike Van > > Committer - ASF Kalumet > > Contributor - ASF Karaf, Camel
