Regarding Kyle's question, that assessment is what I'm getting from the thread. We're not saying Metron has to be Kerberized, but that components that will be managed by the mpack need to support it (otherwise Ambari will have issues Kerberizing and running a cluster where a mandatory component isn't Kerberized).
I agree that if we're making Kerberos an important component, it would be nice to provide some utilities for actually testing it relatively easily. I don't want to gate off Kerberos testability to people willing to setup, and possibly pay for, a real cluster. Even beyond that, these things won't just need to be tested in advance of a release. Are we expecting reviewers to have to go through hoops or spin up a cluster to be able to test any Kerberos related change? That's not necessarily just an MPack issue, but a general issue if we have to change any code/scripts to handle things properly. We may run Kerberized components run outside of Ambari until Ambari support is ready. In terms of integration testing, I don't think adding integration tests for such an important interaction makes the tests bloated. Having Kerberos integration tested does provide us value, but it does mean that we may have to take a look at optimizing our builds so that we aren't running into Travis' build limits. It may also mean cleaning up some of our build so that integration tests are actually properly separated from our unit tests (i.e. conform to the conventions of Maven Failsafe so that the unit test phase actually does just run unit tests, instead of spinning up all the in memory components). This would let users easily just run quick builds with unit tests during development. Justin On Thu, Apr 13, 2017 at 5:36 PM, Kyle Richardson <kylerichards...@gmail.com> wrote: > Just to clarify, we're saying when adding a service to the mpack that > Kerberos support is required... but we're not saying that installing Metron > requires a kerberized cluster correct? I think we should support it but > should allow installation and use of Metron without it (for testing or > other reasons determined by the user). > > -Kyle > > > On Apr 13, 2017, at 12:55 PM, Otto Fowler <ottobackwa...@gmail.com> > wrote: > > > > My thought was that if it was a requirement, or blocker for contribution > we > > would want > > to provide something to help. > > > > I am not sure everyone will have a kerberos cluster to test with. Maybe > > they will. > > Maybe the answer is Docker or Vagrant as Casey suggests and not > integration > > testing. > > > > > > > > On April 13, 2017 at 12:02:08, James Sirota (jsir...@apache.org) wrote: > > > > Hi Guys, > > > > I don't like further bloating our integration tests so I am not sure I > like > > the idea. I think when people add new services they should test on real > > clusters using kerberos. Also, community members can take on end-to-end > > testing in advance of a release on a cluster using kerberos. But I think > > adding this to our integration test framework is just too much. > > > > 13.04.2017, 08:12, "Casey Stella" <ceste...@gmail.com>: > >> I honestly don't know if we can mock out a KDC for integration tests. If > >> we did move the integration tests to running against docker, that might > > be > >> an option as we could dockerize a KDC as well. > >> > >> Long story short, "probably, but not for free. ;)" > >> > >> On Thu, Apr 13, 2017 at 10:41 AM, Otto Fowler <ottobackwa...@gmail.com> > >> wrote: > >> > >>> Can we test kerberized support in integration? > >>> > >>> On April 13, 2017 at 10:24:43, Casey Stella (ceste...@gmail.com) > wrote: > >>> > >>> Agreed, +1 > >>> > >>> On Thu, Apr 13, 2017 at 10:14 AM, Otto Fowler <ottobackwa...@gmail.com > > > >>> wrote: > >>> > >>>> This should be in the dev guide and pr template > >>>> > >>>> On April 13, 2017 at 09:43:48, Casey Stella (ceste...@gmail.com) > > wrote: > >>>> > >>>> Based on my understanding, we have a few axioms that we're working > > from: > >>>> > >>>> - The installer should install a complete and workable product (i.e. > >>>> after install, everything should work). Afterall, that has to be the > >>>> sensible definition of 'working' for an installer > >>>> - Metron should support running in a Kerberized environment > >>>> > >>>> If we are going to support kerberos and the installer is going to > > install > >>>> the product, then I would consider lack of kerberos support for a > >>>> component > >>>> to block inclusion into the mpack. > >>>> > >>>> Casey > >>>> > >>>> On Thu, Apr 13, 2017 at 9:29 AM, Ryan Merriman <merrim...@gmail.com> > >>>> wrote: > >>>> > >>>>> There is a PR up for review ( > >>>>> https://github.com/apache/incubator-metron/pull/518) that updates > > our > >>>>> MPack > >>>>> to support a Kerberized environment. There is also a PR up for > > review > >>>> that > >>>>> adds the REST service to the MPack ( > >>>>> https://github.com/apache/incubator-metron/pull/500). > >>>>> > >>>>> However, the REST application currently does not work in a > > kerberized > >>>>> environment. That work has already started so it won't be an issue > > for > >>>>> long but how should we handle situations like this in the future > > where > >>>> we > >>>>> want to add a service but it's not quite ready for Kerberos? Should > >>>>> Kerberos support be a prerequisite before it's added to the MPack? > >>>> Should > >>>>> we look at ways to make these services optional? Any other thoughts > > or > >>>>> ideas? > >>>>> > >>>>> Ryan > >>>>> > > > > ------------------- > > Thank you, > > > > James Sirota > > PPMC- Apache Metron (Incubating) > > jsirota AT apache DOT org >
