Re: client config files
I absolutely DO think they should be combined in a properties file located in $HOME/.accumulo/config I absolutely DO NOT think this client configuration should be exclusive to the shell, and I absolutely DO NOT think it should be XML. I would love to see all our clients/client code use commons-configuration to hold properties from the properties file, so that only a --config parameter is needed (with reasonable defaults, so even that is not absolutely necessary). I also think that every property that can exist in the file should be possible to override on the command-line. I personally prefer to use system properties, using commons-configuration's HierarchicalConfiguration, but jcommander may make it easier to do the same thing in a slightly different way. -- Christopher L Tubbs II http://gravatar.com/ctubbsii On Thu, Aug 1, 2013 at 12:25 PM, Michael Berman mber...@sqrrl.com wrote: As part of SSL, we need to introduce configuration so accumulo clients (such as ZooKeeperInstance) can find trust stores. It seems like this has a lot in common with shell config files in ACCUMULO-1397. Do people think these should be combined, or should the shell have its own separate config? I was imagining a simple java .properties-style key=value list. Does this seem reasonable? Or should the format be more like the xml of the site config? I was also imagining looking through a list of files that would each override settings, perhaps in the following order (from lowest to highest priority): /etc/accumulo/client.conf $ACCUMULO_HOME/conf/client.conf $HOME/.accumulo/config --client-config command line switch for shell or explicit parameter passed to ZooKeeperInstance Does this sound good to y'all? Should the explicit switch/parameter have per-property override semantics, or should it just be used as the exclusive source of properties if specified? Mike Drob, are you actively working on the shell side of this already? I see that bug is assigned to you... Thanks, Michael
Re: client config files
^ Another reason I like commons-configuration here is for property-interpolation with HierarchicalConfiguration. -- Christopher L Tubbs II http://gravatar.com/ctubbsii On Thu, Aug 1, 2013 at 3:07 PM, Christopher ctubb...@apache.org wrote: I absolutely DO think they should be combined in a properties file located in $HOME/.accumulo/config I absolutely DO NOT think this client configuration should be exclusive to the shell, and I absolutely DO NOT think it should be XML. I would love to see all our clients/client code use commons-configuration to hold properties from the properties file, so that only a --config parameter is needed (with reasonable defaults, so even that is not absolutely necessary). I also think that every property that can exist in the file should be possible to override on the command-line. I personally prefer to use system properties, using commons-configuration's HierarchicalConfiguration, but jcommander may make it easier to do the same thing in a slightly different way. -- Christopher L Tubbs II http://gravatar.com/ctubbsii On Thu, Aug 1, 2013 at 12:25 PM, Michael Berman mber...@sqrrl.com wrote: As part of SSL, we need to introduce configuration so accumulo clients (such as ZooKeeperInstance) can find trust stores. It seems like this has a lot in common with shell config files in ACCUMULO-1397. Do people think these should be combined, or should the shell have its own separate config? I was imagining a simple java .properties-style key=value list. Does this seem reasonable? Or should the format be more like the xml of the site config? I was also imagining looking through a list of files that would each override settings, perhaps in the following order (from lowest to highest priority): /etc/accumulo/client.conf $ACCUMULO_HOME/conf/client.conf $HOME/.accumulo/config --client-config command line switch for shell or explicit parameter passed to ZooKeeperInstance Does this sound good to y'all? Should the explicit switch/parameter have per-property override semantics, or should it just be used as the exclusive source of properties if specified? Mike Drob, are you actively working on the shell side of this already? I see that bug is assigned to you... Thanks, Michael
Re: client config files
I generally prefer properties files to XML, but there may be a argument for reusing Hadoop's SSL configuration system which is XML based. -Joey — Sent from Mailbox for iPhone On Thu, Aug 1, 2013 at 3:08 PM, Christopher ctubb...@apache.org wrote: ^ Another reason I like commons-configuration here is for property-interpolation with HierarchicalConfiguration. -- Christopher L Tubbs II http://gravatar.com/ctubbsii On Thu, Aug 1, 2013 at 3:07 PM, Christopher ctubb...@apache.org wrote: I absolutely DO think they should be combined in a properties file located in $HOME/.accumulo/config I absolutely DO NOT think this client configuration should be exclusive to the shell, and I absolutely DO NOT think it should be XML. I would love to see all our clients/client code use commons-configuration to hold properties from the properties file, so that only a --config parameter is needed (with reasonable defaults, so even that is not absolutely necessary). I also think that every property that can exist in the file should be possible to override on the command-line. I personally prefer to use system properties, using commons-configuration's HierarchicalConfiguration, but jcommander may make it easier to do the same thing in a slightly different way. -- Christopher L Tubbs II http://gravatar.com/ctubbsii On Thu, Aug 1, 2013 at 12:25 PM, Michael Berman mber...@sqrrl.com wrote: As part of SSL, we need to introduce configuration so accumulo clients (such as ZooKeeperInstance) can find trust stores. It seems like this has a lot in common with shell config files in ACCUMULO-1397. Do people think these should be combined, or should the shell have its own separate config? I was imagining a simple java .properties-style key=value list. Does this seem reasonable? Or should the format be more like the xml of the site config? I was also imagining looking through a list of files that would each override settings, perhaps in the following order (from lowest to highest priority): /etc/accumulo/client.conf $ACCUMULO_HOME/conf/client.conf $HOME/.accumulo/config --client-config command line switch for shell or explicit parameter passed to ZooKeeperInstance Does this sound good to y'all? Should the explicit switch/parameter have per-property override semantics, or should it just be used as the exclusive source of properties if specified? Mike Drob, are you actively working on the shell side of this already? I see that bug is assigned to you... Thanks, Michael
github mirror
Does anyone know what the process is for getting the github mirror pointed to the real git repo? Right now it's pointing at a year-old revision off SVN. I'd like to fork a branch for my SSL work so I can submit pull requests back, but it doesn't seem like there's a point if it's so far diverged from master.
Re: github mirror
Oh, actually I was looking at the wrong branch. It's only a month old, not a year old...but my question remains. On Thu, Aug 1, 2013 at 5:15 PM, Michael Berman mber...@sqrrl.com wrote: Does anyone know what the process is for getting the github mirror pointed to the real git repo? Right now it's pointing at a year-old revision off SVN. I'd like to fork a branch for my SSL work so I can submit pull requests back, but it doesn't seem like there's a point if it's so far diverged from master.
Re: github mirror
IIRC I don't believe a process is actually in place to accept pull requests off the chub. I'm open to being corrected by someone with better info, however. On Thu, Aug 1, 2013 at 5:17 PM, Michael Berman mber...@sqrrl.com wrote: Oh, actually I was looking at the wrong branch. It's only a month old, not a year old...but my question remains. On Thu, Aug 1, 2013 at 5:15 PM, Michael Berman mber...@sqrrl.com wrote: Does anyone know what the process is for getting the github mirror pointed to the real git repo? Right now it's pointing at a year-old revision off SVN. I'd like to fork a branch for my SSL work so I can submit pull requests back, but it doesn't seem like there's a point if it's so far diverged from master.
RE: Hadoop 2.0 Support for Accumulo 1.4 Branch
Any update? -Original Message- From: Joey Echeverria [mailto:j...@cloudera.com] Sent: Monday, July 29, 2013 1:24 PM To: dev@accumulo.apache.org Subject: Re: Hadoop 2.0 Support for Accumulo 1.4 Branch We're testing this today. I'll report back what we find. -Joey — Sent from Mailbox for iPhone On Fri, Jul 26, 2013 at 3:34 PM, null dlmar...@comcast.net wrote: Will 1.4 still work with 0.20 with these patches? Great point Billie. - Original Message - From: Billie Rinaldi billie.rina...@gmail.com To: dev@accumulo.apache.org Sent: Friday, July 26, 2013 3:02:41 PM Subject: Re: Hadoop 2.0 Support for Accumulo 1.4 Branch On Fri, Jul 26, 2013 at 11:33 AM, Joey Echeverria j...@cloudera.com wrote: If these patches are going to be included with 1.4.4 or 1.4.5, I would like to see the following test run using CDH4 on at least a 5 node cluster. More nodes would be better. * unit test * Functional test * 24 hr Continuous ingest + verification * 24 hr Continuous ingest + verification + agitation * 24 hr Random walk * 24 hr Random walk + agitation I may be able to assist with this, but I can not make any promises. Sure thing. Is there already a write-up on running this full battery of tests? I have a 10 node cluster that I can use for this. Great. I think this would be a good patch for 1.4. I assume that if a user stays with Hadoop 1 there are no dependency changes? Yup. It works the same way as 1.5 where all of the dependency changes are in a Hadoop 2.0 profile. In 1.5.0, we gave up on compatibility with 0.20 (and early versions of 1.0) to make the compatibility requirements simpler; we ended up without dependency changes in the hadoop version profiles. Will 1.4 still work with 0.20 with these patches? If there are dependency changes in the profiles, 1.4 would have to be compiled against a hadoop version compatible with the running version of hadoop, correct? We had some trouble in the 1.5 release process with figuring out how to provide multiple binary artifacts (each compiled against a different version of hadoop) for the same release. Just something we should consider before we are in the midst of releasing 1.4.4. Billie -Joey
Re: github mirror
On Thu, Aug 1, 2013 at 6:33 PM, Josh Elser josh.el...@gmail.com wrote: https://issues.apache.org/**jira/browse/INFRA-6560https://issues.apache.org/jira/browse/INFRA-6560 In the meantime you can clone https://git-wip-us.apache.org/repos/asf/accumulo.git. When that happens and you open a pull request, it's supposed to email this list (so we know it's there). On 08/01/2013 05:25 PM, William Slacum wrote: IIRC I don't believe a process is actually in place to accept pull requests off the chub. I'm open to being corrected by someone with better info, however. On Thu, Aug 1, 2013 at 5:17 PM, Michael Berman mber...@sqrrl.com wrote: Oh, actually I was looking at the wrong branch. It's only a month old, not a year old...but my question remains. On Thu, Aug 1, 2013 at 5:15 PM, Michael Berman mber...@sqrrl.com wrote: Does anyone know what the process is for getting the github mirror pointed to the real git repo? Right now it's pointing at a year-old revision off SVN. I'd like to fork a branch for my SSL work so I can submit pull requests back, but it doesn't seem like there's a point if it's so far diverged from master.
Re: github mirror
It's worse than that. It has not just diverged from master... it was based on a whole different git-svn, and it looks like git-svn is non-deterministic. So, even the stuff that is the same has completely different commit IDs. What I did (https://github.com/ctubbsii/accumulo) was fork apache/accumulo on github, then on my forked repo, I deleted all the tags and branches, and pushed the correct master to it. I try to keep my master as a mirror of the latest in our real master, and I have other branches for features I'm working on. I haven't tried to do a pull request back to apache/accumulo, but if what Josh says about mailing the dev list is correct, then it should work. Either way, you can always just mail the dev list directly, asking us to pull. -- Christopher L Tubbs II http://gravatar.com/ctubbsii On Thu, Aug 1, 2013 at 5:15 PM, Michael Berman mber...@sqrrl.com wrote: Does anyone know what the process is for getting the github mirror pointed to the real git repo? Right now it's pointing at a year-old revision off SVN. I'd like to fork a branch for my SSL work so I can submit pull requests back, but it doesn't seem like there's a point if it's so far diverged from master.
Re: client config files
I'm generally not a fan of the way some standard Java things have been reinvented in Hadoop. I've always seen JSSE SSL config as system properties, optionally stored in a properties file. Even if Hadoop is using an XML-based configuration for this purpose, I'd still steer clear of it, for this reason. -- Christopher L Tubbs II http://gravatar.com/ctubbsii On Thu, Aug 1, 2013 at 4:33 PM, Joey Echeverria j...@cloudera.com wrote: I generally prefer properties files to XML, but there may be a argument for reusing Hadoop's SSL configuration system which is XML based. -Joey — Sent from Mailbox for iPhone On Thu, Aug 1, 2013 at 3:08 PM, Christopher ctubb...@apache.org wrote: ^ Another reason I like commons-configuration here is for property-interpolation with HierarchicalConfiguration. -- Christopher L Tubbs II http://gravatar.com/ctubbsii On Thu, Aug 1, 2013 at 3:07 PM, Christopher ctubb...@apache.org wrote: I absolutely DO think they should be combined in a properties file located in $HOME/.accumulo/config I absolutely DO NOT think this client configuration should be exclusive to the shell, and I absolutely DO NOT think it should be XML. I would love to see all our clients/client code use commons-configuration to hold properties from the properties file, so that only a --config parameter is needed (with reasonable defaults, so even that is not absolutely necessary). I also think that every property that can exist in the file should be possible to override on the command-line. I personally prefer to use system properties, using commons-configuration's HierarchicalConfiguration, but jcommander may make it easier to do the same thing in a slightly different way. -- Christopher L Tubbs II http://gravatar.com/ctubbsii On Thu, Aug 1, 2013 at 12:25 PM, Michael Berman mber...@sqrrl.com wrote: As part of SSL, we need to introduce configuration so accumulo clients (such as ZooKeeperInstance) can find trust stores. It seems like this has a lot in common with shell config files in ACCUMULO-1397. Do people think these should be combined, or should the shell have its own separate config? I was imagining a simple java .properties-style key=value list. Does this seem reasonable? Or should the format be more like the xml of the site config? I was also imagining looking through a list of files that would each override settings, perhaps in the following order (from lowest to highest priority): /etc/accumulo/client.conf $ACCUMULO_HOME/conf/client.conf $HOME/.accumulo/config --client-config command line switch for shell or explicit parameter passed to ZooKeeperInstance Does this sound good to y'all? Should the explicit switch/parameter have per-property override semantics, or should it just be used as the exclusive source of properties if specified? Mike Drob, are you actively working on the shell side of this already? I see that bug is assigned to you... Thanks, Michael
