I'm not a maven/packaging expert... so we'll have a discussion/vote in the ticket. A patch would be awesome.
I did see your ticket, and it should be fixed. If you find it works for you, please close the ticket. We only know of one other person using accumulo under cygwin: expect other glitches. Is it common to do a sandbox in an apache project? -Eric On Wed, Oct 26, 2011 at 11:08 AM, <[email protected]> wrote: > > > Jesse, > > > > The only difference between a bundle and a .jar is the inclusion of > additional fields in the MANIFEST.MF file. I think there's a way to allow > the maven-bundle-plugin to create this file, and then to include it into a > .jar file created by the maven-jar-plugin. Basically, after the code is > compiled, the maven-bundle-plugin goes through all of the compiled .class > files, and creates a set of Import-Package statements. Then, you can either > have the bundle plugin export everything automatically, or specificy the > packages you want to export. > > > > If you'd like, tonight I can implement this in the parent pom.xml file and > submit a patch for your review. > > > > Also, did anyone take a look at the Jira ticket I created last night > referencing exec plugin attempting to find /bin/bash? I'd really like to be > able to compile accumulo and get it working in a stand-alone mode with a > single hadoop and zookeeper instance to allow me to test the camel-accumulo > component once it is complete. > > > > Lastly, do you folks envision creating a "sandbox" area in your svn > repository? This would allow me to commit the camel component along with > examples and tests to allow you folks to review it. > > > > ----- Original Message ----- > > > From: "Jesse Yates" <[email protected]> > To: [email protected] > Sent: Wednesday, October 26, 2011 10:37:16 AM > Subject: Re: Camel Accumulo > > Comments inline. > ------------------- > Jesse Yates > 240-888-2200 > @jesse_yates > > On Wed, Oct 26, 2011 at 6:39 AM, <[email protected]> wrote: > > > > > > > Eric, > > > > > > > > I'm confused about the need for a username/password. Does Accumulo > validate > > against that, or does it accept a set of priveledges for a user? > > > It checks that user and password against an internal db of users, giving > the > user ability to read from a subset of the permissions on the system > ('Authorizations' in Accumulo parlance). > > > > Also, in your discussion of zookeeper, what does the "zookeepers" > attribute > > represent? > > > > The address of the zookeeper servers keeping track of the accumulo > instance. > This is the 'root' truth of where things are for clients. > > > > > > > > > > For my first stab at this, I'm using the ReadandWrite example to show me > > how to connect to Accumulo and perform reads/writes . Is there a better > one > > that I should be using? This example class seems pretty heavy-weight... > > > > > It might be a little bit heavy, but I would rather have a complete example, > than a bunch of scattered sub-pieces. I think once you have the connection > setup, the actual read/write shouldn't be too bad. > > > > > > > > For the table name, I'd like to keep the camel component as generic as > > possible. So, to allow for this, I'm going to require folks to pass in > the > > table name on their accumulate url. Is this ok? > > > > > > > > Full Ack on the "accumulate" versus "accum" uri. The only reason I > > attempted to shorten it is that looks like the trend for other > > camel-components. That said, I can live with a longer name if it makes > it > > easier to use. Along these lines, I've thought that having a different > uri > > for reads, writes, and mutates would be an easy way to handle > interactions > > through the camel component. To help with this, I'm thinking that he > > following uri's would be use ful (developed in this order) : > > > > accumulate-read > > > > accumulate-write > > > > accumulate-mutate > > > > accumulate-batch-read > > > > accumulate-batch-write > > > > > > > > How does that sound? > > > > shouldn't this be 'accumulo' or is this for implying accumulation of > information via Camel? > > > > > > > > > > Also, how do you feel about changing the build so that it creates bundles > > instead of .jar files? All a bundle is, is a .jar file with a > MANIFEST.MF > > file that has some extra attributes. The maven-bundle-plugin can take > care > > of that pretty easily, but it will change the packaging type from jar to > > bundle. This would really only impact folks who are looking for a > packaging > > type in thier .pom file dependencies, which is not common. > > > > > I would argue vehemently against changing the core build for an external > process (no offense mike). I think it would be okay if we have a profile in > the build that builds a bundle, but since that is a pretty uncommon distro, > people will easily get confused, leading to lower adoption, lots of > questions on user@, etc. Yes, its a low likelyhood, but I would rather > make > it is as easy as possible to 'do the right thing'. > > Camel (and other systems that need a bundle) are a special case, so already > people are going to be be a little more advanced and expect slightly > 'special' versions. > > > > > > > > > Mike Van > > > > > > > > ----- Original Message ----- > > > > > > From: "Eric Newton" <[email protected]> > > To: [email protected] > > Sent: Wednesday, October 26, 2011 9:06:19 AM > > Subject: Re: Camel Accumulo > > > > Hi Mike, > > > > Scan auths are needed for reading, not for writing. > > > > Looking at constructors and factory methods looks like we'll need: > > > > ZooKeeperInstance: > > zookeepers > > instance name > > > > getConnector(): > > username > > password > > > > getMultiTableBatchWriter(): > > memory > > latency > > write threads > > > > This would allow you to stream in tuples of (table, row, cf, cq, > > visibility, > > timestamp). If you already have a connector for HBase, we would probably > > want something compatible with that, and that is probably table oriented > > (just guessing, being unfamiliar with Camel). So, perhaps if you specify > a > > table you would accept tuples of (row, cf, cq, visibility, timestamp). > You > > could make visibility and timestamp optional, too. > > > > A minimal uri would look something like this? You could provide > reasonable > > defaults for the other arguments. > > > > > > accumulo://instance/write?zookeeper=zoohost&userName=root&password=secret > > > > Unfortunately, "accumulo" doesn't have a nice short abbreviation. I > would > > lean towards using the whole word, but that's just a personal preference. > > > > -Eric > > > > On Wed, Oct 26, 2011 at 12:17 AM, <[email protected]> wrote: > > > > > > > > > > > Hi there. > > > > > > > > > > > > I recently read about your project and like the direction it is taking. > > > Currently, I am a committer to another incubator project, Kalumet, and > > have > > > been a long-time contributor to the Karaf and Camel projects. After > > talking > > > with a few of the Accumulo project members, it looks like the most > > immediate > > > hurdle is growing the user community. I believe I can help with that. > > > > > > > > > > > > Making associations between incubator projects and top-level-projects > has > > > been a proven mechanism to pique developer interest and garner more > > > dedicated contributors and commiters. Because of the wide integration > of > > > Camel and NOSQL databases, creating a Camel component for interaction > > with > > > Accumulo seems like a no-brainer. > > > > > > > > > > > > In order to help grow the Accumulo community, tonight I began writing a > > > camel-accumulo component. This will allow Camel users to route files > to > > > Accumulo in the same manner as they currently route files to HDFS . > > > > > > > > > > > > > > > > > > For some background, Camel is the open-source implementation of > > Enterprise > > > Integration Patterns. Most modern ESB's use OSGI and Camel to perform > > > routing and orchestration of data through endpoints. Camel has been > > written > > > to allow various technologies to create Camel Components that folks can > > use > > > when they define the route that a given file or data will be processed > > > through. In this model, users would define a "route" in Camel that > > contains > > > various Accumulo endpoints for reading, writing, or mutating data > > persisted > > > through Accumulo. To make this work, I need to define a URI that folks > > will > > > use. Would you folks be able to help me define the URI and URL > > parameters? > > > > > > > > > > > > Right now, I'm using the URI "accum". For the first iteration of this > > > component, I'm thinking it would be simplest to create an endpoint > folks > > > could write a single re cord to. Then, follow it up with a scan, and > > mutate > > > components. Once these are done, I'd like to do the batch-versions of > > these > > > operations. > > > > > > > > > > > > In Camel, an endpoint usually looks similar to a web-service endpoint: > > > URI://location/service ?[arg1=value1][&arg<x>=value<x>] > > > > > > > > > > > > With this in mind, I'm thinking the following would be the minimally > > > acceptable Camel-Accumulo endpoint for simplistic write operations: > > > > > > accum://location /write ?zookeeperName=value&\ > > > > > > tableName=value& \ > > > > > > userName=value&\ > > > > > > password=value&\ > > > > > > userPrivs=value&\ > > > > > > scanAuths=value&\ > > > > > > debug=value > > > > > > > > > > > > Does this contain a c omplete listing of the properties? A m I missing > > > anything, did I put something in that's not needed, or are there other > > > options a user should be able to pass in? Also, is the URI of "accum" > ok > > for > > > this camel component? > > > > > > > > > > > > Because Camel is written to play nicely inside of OSGI (like > > Karaf/Felix), > > > the .jar files this camel component relies on should be bundle-ized. > This > > > shouldn't be too hard to do, and as a Karaf contributor, I've done this > > with > > > hundreds of third party .jar files. Basically, we would replace the > > > maven-jar-plugin with a light implementation of the maven-bundle-plugin > > > along with some fairly generic attributes. If you folks would like, I > > can > > > do this for you on a seperate branch so that you can test it. > > > > > > > > > > > > Mike Van > > > > > > Committer - ASF Kalumet > > > > > > Contributor - ASF Karaf, Camel > > >
