I might be interested in taking on the work. Can you help me find your patch?
I have a code base already in OSGi that works with MongoDB. I need to point it at Accumulo instead, and if I can do so without porting out of OSGi that would be nice. You say the community would be well-accepting of bundling up the Accumulo client. If that's the case, I'd like to hear from them. On Wed, Apr 9, 2014 at 2:09 PM, Corey Nolet <[email protected]> wrote: > Geoffry, > > > Interesting you have Hadoop working in Karaf. I'm using equinox. > > Sure, but are we talking Karaf-specific features here? You just want a > Hadoop Client bundle that works, right? The author of the Karaf-Hadoop > project already worked through the classpath issues so it's not a bad > starting place. Same with the ServiceMix features files. > > If I understand you correctly, I only need have Text available. I'll > look into that. It does answer my question and maybe I can avoid the JAAS > mishmash. > > Without needing a connection to the FileSystem, can the JAAS stuff be set > as an optional import? > > Close on to 100% of my troubles has always been with getting 3rd party > libraries working. Accumulo/Hadoop is only the latest round. > > You are getting no arguments from me. I've been using Karaf consistently > for a few years now and no doubt most of my time maintaining the services > is spent making sure 3rd party things work (and continue to work after > updates). I've come to expect it at this point. > > > It's security, scaleability, relationship to Hadoop's HDFS and MR all > conspire to make it attractive. But creating an uberbundle? > > Sure, packaging all the Accumulo classes into a single bundle got me up > and running. I still took time wrapping all of its transitive dependencies > in their own bundles and wiring up the imports/exports, but it works. > > As mentioned in previous posts, I think the community would be > well-excepting of a first-class Accumulo bundle. If you are interested in > taking on the work, the ticket I posted previously is open and ready for a > patch. > > Both of these were advised by one of the bndtools gurus--neither worked. > When I did the Import-Package other things broke. > > I had to import several nested packages to get mine to work properly. > Which other things broke when you did the Import Package of the JAAS > packaes? > > > > On Wed, Apr 9, 2014 at 11:49 AM, Geoffry Roberts > <[email protected]>wrote: > >> Corey, >> >> >> Interesting you have Hadoop working in Karaf. I'm using equinox. It >> also sounds as if I don't need to access HFDS in order to get Accumulo to >> work in OSGi. If I understand you correctly, I only need have Text >> available. I'll look into that. It does answer my question and maybe I >> can avoid the JAAS mishmash. >> >> >> I have been using OSGi off and on for a few years now. Close on to 100% >> of my troubles has always been with getting 3rd party libraries working. >> Accumulo/Hadoop is only the latest round. >> >> >> I'm gearing to do some major work based on Accumulo. It's security, >> scaleability, relationship to Hadoop's HDFS and MR all conspire to make it >> attractive. But creating an uberbundle? I'm sure I could get it working as >> a proof of concept, but will it play in prime-time? >> >> >> How hard would it be to make a proper Accumulo bundle? Would the >> community accept it? >> >> >> Thanks >> >> >> So far as my travails with JAAS is concerned, I did this in my bndtools >> *.bndrun file: >> >> >> -runproperties: >> org.osgi.framework.system.packages.extra=com.sun.security.auth.module >> >> I also tried: >> Import-Package: com.sun.security.auth.module >> in my bundle that calls Hadoop. >> >> Both of these were advised by one of the bndtools gurus--neither worked. >> When I did the Import-Package other things broke. >> >> On Wed, Apr 9, 2014 at 9:46 AM, Corey Nolet <[email protected]> wrote: >> >>> Geoffry, >>> >>> As Josh pointed out, you should only need the Hadoop libraries on the >>> client side to use the Text object. This means you won't have to go through >>> the pain of placing the xml files in your root bundles. >>> >>> Did you try the JAAS export from the packages in your container? Did >>> that help? >>> >>> I agree with your comment that we *should* be able to look at these >>> bundles as black boxes but we're not dealing with true bundles, yet. Once I >>> got my Hadoop client bundle ready to go, I haven't had to touch it (and the >>> Hadoop Karaf project solved most of the import/export guessing work for me >>> already). For Accumulo, on the other hand, it's up to you if you want to >>> create an uber-bundle with the necessary Accumulo packages or just wrap >>> each one and provide the necessary imports/exports across them. For my >>> needs, I just created one uber-bundle for Accumulo packages and imported >>> the necessary non-Accumulo packages (many of those I had to wrap as well). >>> I didn't find that part too painful on the Accumulo side. While not being >>> the ideal situation from the OSGI side, with lack of an existing bundle >>> artifact, it did solve my problem and I'm actively using both Accumulo and >>> Hadoop in OSGi. My recommendation would be that until we get a proper >>> bundle, that solution would certainly work in the short-term. >>> >>> I believe Josh posted this already but check out [1]. A ready-to-go OSGi >>> bundle for Accumulo would be useful but the Hadoop client dependency would >>> need to be wrapped (or exposed as its own bundle). IMO, with proper >>> documentation this shouldn't be too painful for users. Thoughts? >>> >>> >>> [1] https://issues.apache.org/jira/browse/ACCUMULO-2518 >>> >>> >>> On Mon, Apr 7, 2014 at 11:27 AM, Geoffry Roberts <[email protected] >>> > wrote: >>> >>>> Ahh, let me try and address where I might have gone off the linguistic >>>> reservation. >>>> >>>> bndtools -- is an eclipse plugin that is very helpful when developing >>>> OSGi bundles. It does a lot of grimy, boilerplate things for you. >>>> >>>> inlining -- is where one places dependent *.jar files inside the OSGi >>>> bundle and therefore on said bundle's class path. It tends to promote >>>> bloated bundles--not in the spirit of OSGi--but sometimes necessary. >>>> >>>> componentizing -- is the business of converting a class into a >>>> component. In the bndtools way of doing things, this can be a easy as >>>> annotating a class with @Component. >>>> >>>> bundle -- You probably know what this is already, but I'll include it >>>> for good measure. A bundle is a body of code that is on the same class >>>> path, and often acts as a service to there bundles. >>>> >>>> I don't know what could be done upstream other that making Accumulo's >>>> client OAGI ready. Would we like to do that? >>>> >>>> >>>> On Mon, Apr 7, 2014 at 11:02 AM, Josh Elser <[email protected]>wrote: >>>> >>>>> You just used a lot of words that don't mean anything to me :) >>>>> >>>>> Hopefully you don't have to do much on your own. If there are things >>>>> we can change upstream to make this process easier, please feel free to >>>>> let >>>>> us know. >>>>> >>>>> >>>>> On 4/7/14, 10:55 AM, Geoffry Roberts wrote: >>>>> >>>>>> Thanks Josh, >>>>>> >>>>>> My container for the moment is equinox, but all should work in Felix >>>>>> as >>>>>> well. I've been using bndtools for my other OSGi work so I'm faced >>>>>> with >>>>>> either annotating the Accumulo Code or wrapping it somehow. What do >>>>>> you >>>>>> want to bet I wind up inlining it? Still, the annotated (read >>>>>> componentized) approach would be less kloogy. I hesitate because I'd >>>>>> wind up maintaining my own code line. >>>>>> >>>>>> >>>>>> On Mon, Apr 7, 2014 at 10:28 AM, Josh Elser <[email protected] >>>>>> <mailto:[email protected]>> wrote: >>>>>> >>>>>> On 4/7/14, 10:07 AM, Geoffry Roberts wrote: >>>>>> >>>>>> My original question remains: Is the Accumulo Client dependent >>>>>> on the >>>>>> Hadoop Client fully? This determination can be made through >>>>>> trial and >>>>>> error. But I'm looking to leverage OPE (other people's >>>>>> experience) if >>>>>> it exists. >>>>>> >>>>>> >>>>>> I thought someone had already said this (but I may be confusing >>>>>> threads): the Accumulo API uses Text throughout. Hadoop is a >>>>>> required dependency. >>>>>> >>>>>> >>>>>> In the same spirit, does anyone know if all the following are >>>>>> required >>>>>> to run an Accumulo Client? core, fate, start, trace? If I >>>>>> attempt to >>>>>> OSGify, I'm trying to figure how much trouble am I getting >>>>>> into. >>>>>> >>>>>> >>>>>> Yes, that should be about it from within Accumulo. You might need >>>>>> some other foss dependencies also available, but I'm not aware on >>>>>> what your "container" (or w/e the proper terminology would be) >>>>>> provides. >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> There are ways and there are ways, >>>>>> >>>>>> Geoffry Roberts >>>>>> >>>>> >>>> >>>> >>>> -- >>>> There are ways and there are ways, >>>> >>>> Geoffry Roberts >>>> >>> >>> >> >> >> -- >> There are ways and there are ways, >> >> Geoffry Roberts >> > > -- There are ways and there are ways, Geoffry Roberts
