Hi Mike,
Scan auths are needed for reading, not for writing.
Looking at constructors and factory methods looks like we'll need:
ZooKeeperInstance:
zookeepers
instance name
getConnector():
username
password
getMultiTableBatchWriter():
memory
latency
write threads
This would allow you to stream in tuples of (table, row, cf, cq, visibility,
timestamp). If you already have a connector for HBase, we would probably
want something compatible with that, and that is probably table oriented
(just guessing, being unfamiliar with Camel). So, perhaps if you specify a
table you would accept tuples of (row, cf, cq, visibility, timestamp). You
could make visibility and timestamp optional, too.
A minimal uri would look something like this? You could provide reasonable
defaults for the other arguments.
accumulo://instance/write?zookeeper=zoohost&userName=root&password=secret
Unfortunately, "accumulo" doesn't have a nice short abbreviation. I would
lean towards using the whole word, but that's just a personal preference.
-Eric
On Wed, Oct 26, 2011 at 12:17 AM, <[email protected]> wrote:
>
>
> Hi there.
>
>
>
> I recently read about your project and like the direction it is taking.
> Currently, I am a committer to another incubator project, Kalumet, and have
> been a long-time contributor to the Karaf and Camel projects. After talking
> with a few of the Accumulo project members, it looks like the most immediate
> hurdle is growing the user community. I believe I can help with that.
>
>
>
> Making associations between incubator projects and top-level-projects has
> been a proven mechanism to pique developer interest and garner more
> dedicated contributors and commiters. Because of the wide integration of
> Camel and NOSQL databases, creating a Camel component for interaction with
> Accumulo seems like a no-brainer.
>
>
>
> In order to help grow the Accumulo community, tonight I began writing a
> camel-accumulo component. This will allow Camel users to route files to
> Accumulo in the same manner as they currently route files to HDFS .
>
>
>
>
>
> For some background, Camel is the open-source implementation of Enterprise
> Integration Patterns. Most modern ESB's use OSGI and Camel to perform
> routing and orchestration of data through endpoints. Camel has been written
> to allow various technologies to create Camel Components that folks can use
> when they define the route that a given file or data will be processed
> through. In this model, users would define a "route" in Camel that contains
> various Accumulo endpoints for reading, writing, or mutating data persisted
> through Accumulo. To make this work, I need to define a URI that folks will
> use. Would you folks be able to help me define the URI and URL parameters?
>
>
>
> Right now, I'm using the URI "accum". For the first iteration of this
> component, I'm thinking it would be simplest to create an endpoint folks
> could write a single re cord to. Then, follow it up with a scan, and mutate
> components. Once these are done, I'd like to do the batch-versions of these
> operations.
>
>
>
> In Camel, an endpoint usually looks similar to a web-service endpoint:
> URI://location/service ?[arg1=value1][&arg<x>=value<x>]
>
>
>
> With this in mind, I'm thinking the following would be the minimally
> acceptable Camel-Accumulo endpoint for simplistic write operations:
>
> accum://location /write ?zookeeperName=value&\
>
> tableName=value& \
>
> userName=value&\
>
> password=value&\
>
> userPrivs=value&\
>
> scanAuths=value&\
>
> debug=value
>
>
>
> Does this contain a c omplete listing of the properties? A m I missing
> anything, did I put something in that's not needed, or are there other
> options a user should be able to pass in? Also, is the URI of "accum" ok for
> this camel component?
>
>
>
> Because Camel is written to play nicely inside of OSGI (like Karaf/Felix),
> the .jar files this camel component relies on should be bundle-ized. This
> shouldn't be too hard to do, and as a Karaf contributor, I've done this with
> hundreds of third party .jar files. Basically, we would replace the
> maven-jar-plugin with a light implementation of the maven-bundle-plugin
> along with some fairly generic attributes. If you folks would like, I can
> do this for you on a seperate branch so that you can test it.
>
>
>
> Mike Van
>
> Committer - ASF Kalumet
>
> Contributor - ASF Karaf, Camel
