Ryan, what's remote debugging look like for UI testing with Knox enabled? Anything we lose from a dev testability standpoint? The discussion of defaults sounds reasonable to me, and I'd like to understand any other tradeoffs there may be for non-prod deployments like full dev.
On Fri, Nov 16, 2018 at 7:20 AM Ryan Merriman <merrim...@gmail.com> wrote: > Most of the research I've done around adding Metron as a Knox service is > based on how other projects do it. The documentation is not easy to follow > so I learned by reading other service definition files. The assumption > that we are doing things drastically different is false. > > I completely agree with Simon. Why would we want to be dependent on Knox's > release cycle? How does that benefit us? It may reduce some operational > complexity but it makes our install process more complicated because we > require a certain version of Knox (who knows when that gets released). > What do we do in the meantime? I would also like to point out that Metron > is inherently different than other Hadoop stack services. We are a > full-blown application with multiple UIs so the way we expose services > through Knox may be a little different. > > I think this will be easier to discuss when we can all see what is actually > involved. I am working on a PR that adds Metron as a Knox service and will > have that out soon. That should give everyone more context. > > On Fri, Nov 16, 2018 at 7:39 AM Simon Elliston Ball < > si...@simonellistonball.com> wrote: > > > You could say the same thing about Ambari, but that provides mpacks. Knox > > is also designed to be extensible through Knox service stacks since they > > realized they can’t support every project. The challenge is that the docs > > have not made it as easy as they could for the ecosystem to plug into > Knox, > > which has led to some confusion around this being a recommended pattern > > (which it is). > > > > The danger of trying to get your bits into Knox is that that ties you to > > their release cycle (a problem Ambari has felt hard, hence their > community > > is moving away from the everything inside model towards everything is an > > mpack). > > > > A number of implementations of Knox also use the approach Ryan is > > suggesting for their own organization specific end points, so it’s not > like > > this is an uncommon, or anti-pattern, it’s more the way Knox is designed > to > > work in the future, than the legacy of it only being able to handle a > > subset of Hadoop projects. > > > > Knox remains optional In our scenario, but we keep control over the > > shipping of things like rewrite rules, which allows Metron to control its > > release destiny should things like url patterns in the ui need to change > > (with a new release of angular / new module / new rest endpoint etc) > > instead of making a Metron release dependent on a Knox release. > > > > Imagine how we would have done with the Ambari side if we’d had to wait > > for them to release every time we needed to change something in the > > mpack... we don’t want that happening with Knox. > > > > Simon > > > > > On 16 Nov 2018, at 13:22, Otto Fowler <ottobackwa...@gmail.com> wrote: > > > > > > > > > https://issues.apache.org/jira/browse/KNOX-841?jql=project%20%3D%20KNOX%20AND%20text%20~%20support > > > > > > Solr is angular for example. > > > > > > > > > On November 16, 2018 at 08:12:55, Otto Fowler (ottobackwa...@gmail.com > ) > > > wrote: > > > > > > Ok, here is something I don’t understand, but would like to. > > > > > > Knox comes configured with build in services for a number of other > apache > > > products and UI’s. > > > It would seem to me, that the best integration with Knox would be to do > > > what these other products have done. > > > > > > > > > 1. Do whatever you have to do to make your own stuff compatible. > > > 2. Create a knox service definition and provide it or try to get it > into > > > knox itself > > > > > > This would make the knox integration with metron optional and pluggable > > > wouldn’t it? > > > > > > Then knox with metron would just be the same as knox with anything > else. > > > Please help me if I am wrong, but we seem to be going our own way here. > > > Why don’t we just do what these other products have done? > > > Why don’t we try to get apache metron services accepted to the knox > > > project? Why don’t we model our knox integration with how XYZ does it? > > > Have we looked at how others integrate? Having all the code and being > > > able to track stuff is kind of the point of this whole thing isn’t it? > > > > > > Maybe this is implied and I’m missing it, if so I apologize. > > > > > > I think consistency with the rest of the hadoop stack with knox helps > us. > > > > > > > > > > > > On November 15, 2018 at 22:20:00, Ryan Merriman (merrim...@gmail.com) > > wrote: > > > > > > 1) Sorry I misspoke. I meant to say this is not possible in the Alerts > UI > > > as far as I know. I put up a PR with a proposed solution here: > > > https://github.com/apache/metron/pull/1266. > > > 2) Yes Knox is a service you can install with Ambari, similar to Ranger > > or > > > Spark. There are some things that are specifically configured in Knox > and > > > there are some things specific to Metron. I will put up a PR with the > > > changes needed so you can see exactly what is involved. > > > 3) I don't understand what you mean here. Is this a question? > > > 4) I think it's a little early to predict the Ambari changes required. > > > This will depend on how tasks 1-3 go. I imagine it's similar to other > > > mpack work: expose some parameters in ambari and bind those to config > > > files. My understanding from this thread so far is that we should focus > > on > > > a manual, documented approach to start. > > > > > > On Thu, Nov 15, 2018 at 7:53 PM Michael Miklavcic < > > > michael.miklav...@gmail.com> wrote: > > > > > >> Thanks Ryan, > > >> > > >> 1) Can you clarify "not a good way to do this?" Are you saying we > don't > > >> have a way to set this and need to add the config option, or that a > > >> solution is not obvious and it's unclear what to do? It seems to me > > you're > > >> saying the former, but I'd like to be sure. > > >> 2) Is Knox not a service made available by Ambari similar to Ranger or > > >> Spark? I'm assuming that similar to Kerberos, there are some things > that > > >> are specifically configured in Knox and others that are app-specific. > > Some > > >> explanation of what this looks like would be helpful. > > >> 3) Sounds like this follows pretty naturally from #1 > > >> 4) Relates to #2. I think we need some guidance on what a manual vs > > >> MPack/automated install would look like. > > >> > > >> Cheers, > > >> Mike > > >> > > >> > > >>> On Thu, Nov 15, 2018 at 4:07 PM Ryan Merriman <merrim...@gmail.com> > > wrote: > > >>> > > >>> Wanted to give an update on the context path issue. I investigated > > >>> rewriting url links in the outgoing static assets with Knox and it > was > > >> not > > >>> trivial. Fortunately I found a simple solution that works with or > > >> without > > >>> Knox. I changed the base tag in index.html from <base href="/"> to > > <base > > >>> href="./">, or in other words made the base href relative. > > >>> > > >>> I believe I am at the point where I can task this out and provide a > > high > > >>> level overview of the changes needed. I think that each task will be > a > > >>> manageable size and can stand alone so I don't think we need a > feature > > >>> branch. > > >>> > > >>> The first task involves a general change to the UI code. We need a > way > > >> to > > >>> set the path to the REST service with a configuration setting because > > it > > >> is > > >>> different with and without Knox. Currently there is not a good way to > > do > > >>> this in the UI. We can use the environment files but that is a build > > >> time > > >>> setting and is not flexible. I can see this capability being useful > for > > >>> other use cases in the future. I think we could even split this up > into > > >> 2 > > >>> separate tasks, one for the alerts UI and one for the management UI. > > >>> > > >>> The second task involves adding Knox to our stack either by default > as > > a > > >>> dependency in the mpack or with a documented approach. We would add > our > > >>> REST service, Alerts UI, and Management UI as services in Knox. > > >> Everything > > >>> would continue to function as it currently does but with all > > >> communication > > >>> going through Knox. LDAP authentication would be required when using > > >> Knox > > >>> and Knox will authenticate with the REST service by passing along an > > >>> Authorization header. Enabling Knox would be a manual process that > > >>> involves deploying assets (Knox descriptor files) and changing > > >>> configuration. There would be no change to how the UI functions by > > >> default > > >>> (without Knox) and either LDAP or JDBC authentication could still be > > >> used.. > > >>> > > >>> The third task involves enabling SSO with Knox. We would update the > > REST > > >>> service so that it can authenticate with a Knox SSO token. We would > > >>> provide documentation on how to update the Knox settings in Ambari to > > >>> enable SSO. The Alerts/Management UI would need to expose > configuration > > >>> properties for the REST url and login url since these would be > > different > > >>> when Knox is enabled. We would also need to provide documentation on > > how > > >>> to make these UI configuration changes, based on the work done in > task > > > 1. > > >>> > > >>> An optional forth task would be exposing configuration settings and > > >>> enabling Knox with Ambari. We would eliminate the manual steps > > necessary > > >>> for enabling Knox and instead automate those steps with an Ambari > input > > >>> control, similar to how LDAP is enabled. > > >>> > > >>> Thoughts on this plan? > > >>> > > >>> On Thu, Nov 15, 2018 at 10:22 AM James Sirota <jsir...@apache.org> > > >> wrote: > > >>> > > >>>> In my view Knox SSO is such a minor feature when it comes to > Metron's > > >>>> capabilities than it's not worth supporting multiple scenarios where > > > it > > >>>> works with Knox or without Knox. Where we should be configurable > (and > > >>> are > > >>>> configurable) is on the analytics and stream processing. But this? > As > > >>>> long as the UI authenticates securely I don't think anyone is going > to > > >>> care > > >>>> what proxy it's using. The code itself should be written in a way > that > > >>>> it's pluggable so if we ever wanted to use another proxy or disable > it > > >>> all > > >>>> together we could. But this should not be a configuration we pass on > > >> to > > >>>> the user. The added complexity is simply not worth it here. We have > > >> to > > >>>> start being opinionated about making sensible choices on behalf of > the > > >>>> user. A sensible choice here is to run with Knox and LDAP. The JDBC > > >>>> component should exist for another release to allow the community to > > >>>> migrate over to LDAP and then be deprecated. The code should still > be > > >>>> pluggable and if anyone wanted to extend it to work with JDBC they > > >> could, > > >>>> or if people wanted to plug in another proxy they could, but this is > > >> not > > >>>> something we would officially support. > > >>>> > > >>>> Thanks, > > >>>> James > > >>>> > > >>>> 12.11.2018, 07:36, "Ryan Merriman" <merrim...@gmail.com>: > > >>>>> Let me clarify on exposing both legacy and Knox URLs at the same > > >> time. > > >>>> The > > >>>>> base urls will look something like this: > > >>>>> > > >>>>> Legacy REST - http://node1:8082/api/v1 > > >>>>> Legacy Alerts UI - http://node1:4201:/alerts-list > > >>>>> > > >>>>> Knox REST - https://node1:8443/gateway/default/metron/api/v1 > > >>>>> Knox Alerts UI - > > >>>>> https://node1:8443/gateway/default/metron-alerts-ui/alerts-list > > >>>>> > > >>>>> If Knox were turned on and the alerts UI deployed as is, it would > > > not > > >>>>> work. This is because static assets are referenced with > > >>>>> http://node1:4201/assets/some-asset.js which does not include the > > >>>> correct > > >>>>> context path to the alerts UI in knox. To make it work, you have to > > >> set > > >>>>> the base ref to "/gateway/default/metron-alerts-ui" so that static > > >>> assets > > >>>>> are referenced at > > >>>>> > > >>> > > https://node1:8443/gateway/default/metron-alerts-ui/assets/some-asset.js > > >>>> . > > >>>>> When you do that, the legacy alerts UI will no longer work. I guess > > >> the > > >>>>> point I'm trying to make is that we would have to switch between > > > them > > >>> or > > >>>>> have 2 separate application running. I imagine most users only need > > >> one > > >>>> or > > >>>>> the other running so probably not an issue. > > >>>>> > > >>>>> Jon, the primary upgrade consideration I see is with > authentication. > > >> To > > >>>> be > > >>>>> able to use Knox, you would have to upgrade to LDAP-based > > >>> authentication > > >>>> if > > >>>>> you were still using JDBC-based authentication in REST. The urls > > >> would > > >>>>> also change obviously. > > >>>>> > > >>>>> On Sun, Nov 11, 2018 at 6:38 PM zeo...@gmail.com <zeo...@gmail.com > > > > >>>> wrote: > > >>>>> > > >>>>>> Phew, that was quite the thread to catch up on. > > >>>>>> > > >>>>>> I agree that this should be optional/pluggable to start, and I'm > > >>>> interested > > >>>>>> to hear the issues as they relate to upgrading an existing cluster > > >>>> (given > > >>>>>> the suggested approach) and exposing both legacy and knox URLs at > > >> the > > >>>> same > > >>>>>> time. > > >>>>>> > > >>>>>> Jon > > >>>>>> > > >>>>>> On Fri, Nov 9, 2018, 4:46 PM Michael Miklavcic < > > >>>>>> michael.miklav...@gmail.com> > > >>>>>> wrote: > > >>>>>> > > >>>>>>> A couple more things, and I think this goes without saying - > > >>>> whatever we > > >>>>>> do > > >>>>>>> with Knox should NOT > > >>>>>>> > > >>>>>>> 1. Require unit and integration tests to use Knox > > >>>>>>> 2. Break fulldev > > >>>>>>> > > >>>>>>> Also, I don't know that I saw you mention this, but I'm unsure > > >> how > > >>> we > > >>>>>>> should leverage Knox as a core piece of the platform. i.e. should > > >>> we > > >>>> make > > >>>>>>> this required or optional? I'm open to hearing opinions on this, > > >>> but > > >>>> I'm > > >>>>>>> inclined to keep this a pluggable option. > > >>>>>>> > > >>>>>>> Mike > > >>>>>>> > > >>>>>>> > > >>>>>>> On Fri, Nov 9, 2018 at 2:42 PM Michael Miklavcic < > > >>>>>>> michael.miklav...@gmail.com> wrote: > > >>>>>>> > > >>>>>>>> Thanks for the update Ryan. Per my earlier comments, I thought > > >> it > > >>>> might > > >>>>>>> be > > >>>>>>>> the case that we could dramatically simplify this by leveraging > > >>>> Knox's > > >>>>>>>> proxy capabilities, and per your research that appears to be > > >> the > > >>>> case. > > >>>>>>> This > > >>>>>>>> is a dramatic simplification and improvement of this feature > > >> imo, > > >>>> +1. > > >>>>>> I'm > > >>>>>>>> also +1 on a couple distinct steps that you've laid out: fix > > >> the > > >>> UI > > >>>>>>> issues > > >>>>>>>> in master, then add Knox for SSO. That should help mitigate > > >>> issues > > >>>> with > > >>>>>>>> merge conflicts with ongoing development. > > >>>>>>>> > > >>>>>>>>> I think it will be a challenge exposing the UIs through both > > >>> the > > >>>> Knox > > >>>>>>>> url and legacy urls at the same time. > > >>>>>>>> I'm not sure I understand the issue here. Are you referring to > > >>> this > > >>>>>>>> comment? "Added a ng build option to build the UI with base > > >> href > > >>>> set to > > >>>>>>>> Knox base path." Isn't it just a matter of URL > > >>>> rewriting/forwarding? I > > >>>>>>>> thought we'd be exposing the URL's directly in one context, and > > >>>> through > > >>>>>>>> Knox in the other. Either way, it seems like we should be able > > >> to > > >>>>>>> provide a > > >>>>>>>> dynamic base path through configuration in our web > > >> applications. > > >>>> I'd > > >>>>>>> expect > > >>>>>>>> to modify that property based on whether Knox is configured or > > >>> not. > > >>>>>>>> > > >>>>>>>>> I'm also not clear on how one would use Knox with REST set to > > >>>> legacy > > >>>>>>>> JDBC-based authentication. As far as I know Knox does not > > >> support > > >>>> JDBC > > >>>>>> so > > >>>>>>>> there would be a mismatch between Knox and REST. > > >>>>>>>> I'm OK with not having Knox work with JDBC. That's a feature of > > >>>> Knox > > >>>>>> and > > >>>>>>>> probably not something we care much about. > > >>>>>>>> > > >>>>>>>>> We could initially make Knox an optional feature that requires > > >>>> setup > > >>>>>>> with > > >>>>>>>> the help of some documentation (like Kerberos) while keeping > > >> the > > >>>> system > > >>>>>>> the > > >>>>>>>> way it is now by default. > > >>>>>>>> Sounds good to me. > > >>>>>>>> > > >>>>>>>>> I imagine we'll deprecate JDBC-based authentication at some > > >>>> point so > > >>>>>>>> that may be a good time to switch. > > >>>>>>>> I would like to announce deprecation in our next release and > > >> move > > >>>> to > > >>>>>>>> remove it in a following release. > > >>>>>>>> > > >>>>>>>> Thanks for taking this on and great job laying things out. > > >>>>>>>> > > >>>>>>>> Thanks, > > >>>>>>>> Mike > > >>>>>>>> > > >>>>>>>> On Fri, Nov 9, 2018 at 2:09 PM Ryan Merriman < > > >>> merrim...@gmail.com> > > >>>>>>> wrote: > > >>>>>>>> > > >>>>>>>>> I have spent some time recently reviewing this discussion and > > >>> the > > >>>>>>> feature > > >>>>>>>>> branch that Simon put out. I think this is an important > > >> feature > > >>>> and > > >>>>>>> want > > >>>>>>>>> to move it forward. I started another discussion on adding > > >> Knox > > >>> to > > >>>>>> our > > >>>>>>>>> stack but this discussion has a lot of good context so I will > > >>>> continue > > >>>>>>> it > > >>>>>>>>> here. > > >>>>>>>>> > > >>>>>>>>> I think the main point of contention was that this feature > > >>> branch > > >>>>>>> included > > >>>>>>>>> several different architectural changes and it was unclear if > > >>> they > > >>>>>> were > > >>>>>>>>> needed and if so, could be done separately. Fortunately LDAP > > >>>>>>>>> authentication has been accepted into master so we can cross > > >> it > > >>>> off > > >>>>>> the > > >>>>>>>>> list. From my understanding of the points people have made, > > >> that > > >>>>>> leaves > > >>>>>>>>> Knox related SSO changes and migrating expressjs to a > > >> different, > > >>>>>>> JVM-based > > >>>>>>>>> web server that includes proxying capabilities (Zuul). > > >>>>>>>>> > > >>>>>>>>> I think everyone agrees that if we can limit the scope to just > > >>>> Knox > > >>>>>>>>> related > > >>>>>>>>> SSO changes that would be ideal. I believe I have found a way > > >> to > > >>>> do > > >>>>>>> that > > >>>>>>>>> while working on a small POC this week. The key to this (Simon > > >>>>>> alluded > > >>>>>>> to > > >>>>>>>>> it earlier) is to put both REST and our UIs behind Knox. I > > >>>> initially > > >>>>>>> was > > >>>>>>>>> focused on just adding REST as a service in Knox and decided > > >> to > > >>>>>>> experiment > > >>>>>>>>> with also adding our UIs. After I did this it became clear > > >> that > > >>>> this > > >>>>>>>>> simplifies things considerably: > > >>>>>>>>> > > >>>>>>>>> - The REST app and the UIs are now served from the same host > > >> so > > >>>>>> CORS > > >>>>>>>>> concerns go away. > > >>>>>>>>> - We no longer need to worry about proxying REST requests from > > >>> the > > >>>>>>> UIs > > >>>>>>>>> with express or Zuul because Knox handles that for us. This > > >> will > > >>>>>>> make > > >>>>>>>>> our > > >>>>>>>>> express configuration even simpler. In fact, all we need is a > > >>>>>> simple > > >>>>>>>>> way > > >>>>>>>>> to serve static UI assets. > > >>>>>>>>> - We no longer need to check for SSO tokens and redirect in > > >> the > > >>> UI > > >>>>>>>>> web/app servers (or the REST app for that matter) because Knox > > >>>>>>> handles > > >>>>>>>>> that > > >>>>>>>>> for us. > > >>>>>>>>> - The UIs can now easily access any Knox service (not just our > > >>>> REST > > >>>>>>>>> app) > > >>>>>>>>> without any extra proxy configuration. > > >>>>>>>>> - SSO token authentication is only necessary in REST so there > > >> is > > >>>> no > > >>>>>>>>> need > > >>>>>>>>> to create shared Spring modules or split functionality out. > > >>>>>>>>> > > >>>>>>>>> The most significant change I had to make (borrowed from > > >> Simon's > > >>>>>> feature > > >>>>>>>>> branch) was the SSO token authentication mentioned above. The > > >>>> primary > > >>>>>>>>> short term benefit with this approach is that outside of some > > >>>> general > > >>>>>>>>> deficiencies unrelated to this our UI architecture doesn't > > >> need > > >>> to > > >>>>>>>>> fundamentally change. I could summarize the changes as: > > >>>>>>>>> > > >>>>>>>>> - Knox install and configuration (setting up REST and the > > >> alerts > > >>>> UI > > >>>>>>> as > > >>>>>>>>> Knox services) > > >>>>>>>>> - Added Knox SSO token authentication to REST > > >>>>>>>>> - Updated REST urls in the UI code (should be configurable) > > >>>>>>>>> - Fixed a few UI bugs where relative paths were not being used > > >>>>>>>>> - Added a ng build option to build the UI with base href set > > >> to > > >>>>>> Knox > > >>>>>>>>> base path ( > > >>>>>>>>> > > >>>>>>>>> > > >>>>>>> > > >>>>>> > > >>>> > > >>> > > >> > > > > > > https://github.com/angular/angular-cli/wiki/build#base-tag-handling-in-indexhtml > > >>>>>>>>> ) > > >>>>>>>>> > > >>>>>>>>> Most the UI changes are preexisting, minor issues that could > > >> be > > >>>> fixed > > >>>>>>>>> directly in master. We would need to think of an approach for > > >>> the > > >>>>>> base > > >>>>>>>>> href build requirement but I'm sure it's not that bad. > > >>>>>>>>> > > >>>>>>>>> However there will be some backwards compatibility issues we > > >>> would > > >>>>>> need > > >>>>>>> to > > >>>>>>>>> think through. I think it will be a challenge exposing the UIs > > >>>>>> through > > >>>>>>>>> both the Knox url and legacy urls at the same time. I'm also > > >> not > > >>>>>> clear > > >>>>>>> on > > >>>>>>>>> how one would use Knox with REST set to legacy JDBC-based > > >>>>>>> authentication. > > >>>>>>>>> As far as I know Knox does not support JDBC so there would be > > >> a > > >>>>>> mismatch > > >>>>>>>>> between Knox and REST. Knox does have the ability to pass > > >> along > > >>>> basic > > >>>>>>>>> authentication headers so LDAP in REST would work. We could > > >>>> initially > > >>>>>>>>> make > > >>>>>>>>> Knox an optional feature that requires setup with the help of > > >>> some > > >>>>>>>>> documentation (like Kerberos) while keeping the system the way > > >>> it > > >>>> is > > >>>>>> now > > >>>>>>>>> by > > >>>>>>>>> default. I imagine we'll deprecate JDBC-based authentication > > >> at > > >>>> some > > >>>>>>>>> point > > >>>>>>>>> so that may be a good time to switch. > > >>>>>>>>> > > >>>>>>>>> What do people think about this approach? Concerns? Are there > > >>> any > > >>>>>> huge > > >>>>>>>>> holes in this I'm not thinking about? > > >>>>>>>>> > > >>>>>>>>> I want to highlight that the work Simon did in his feature > > >>> branch > > >>>> was > > >>>>>>>>> crucial to better understanding this. I am pretty sure we'll > > >> end > > >>>> up > > >>>>>>>>> reusing a lot code from that branch. > > >>>>>>>>> > > >>>>>>>>> On Thu, Sep 27, 2018 at 6:30 PM Michael Miklavcic < > > >>>>>>>>> michael.miklav...@gmail.com> wrote: > > >>>>>>>>> > > >>>>>>>>>> Apparently, I hit send on my last email before finishing my > > >>>> synopsis > > >>>>>>>>> (per > > >>>>>>>>>> @Otto's Q in Slack). To summarize, based on my current > > >>>>>> understanding I > > >>>>>>>>>> believe that each of the feature branch changes I've outline > > >>>> above > > >>>>>> are > > >>>>>>>>>> units of work that are related, yet should be executed on > > >>>>>>> independently. > > >>>>>>>>>> Knox SSO in its own feature branch. Migrating technologies > > >>> like > > >>>>>> NodeJs > > >>>>>>>>> or > > >>>>>>>>>> migrating the auth DB to LDAP seem like they belong in their > > >>> own > > >>>>>>>>> separate > > >>>>>>>>>> PR's or feature branches. > > >>>>>>>>>> > > >>>>>>>>>> Thanks, > > >>>>>>>>>> Mike > > >>>>>>>>>> > > >>>>>>>>>> On Thu, Sep 27, 2018 at 4:08 PM Casey Stella < > > >>>> ceste...@gmail.com> > > >>>>>>>>> wrote: > > >>>>>>>>>> > > >>>>>>>>>>> I'm coming in late to the game here, but for my mind a > > >>> feature > > >>>>>>> branch > > >>>>>>>>>>> should involve the minimum architectural change to > > >>> accomplish > > >>>> a > > >>>>>>> given > > >>>>>>>>>>> feature. > > >>>>>>>>>>> The feature in question is SSO integration. It seems to me > > >>>> that > > >>>>>> the > > >>>>>>>>>>> operative question is can we do the feature without making > > >>> the > > >>>>>> OTHER > > >>>>>>>>>>> architectural change > > >>>>>>>>>>> (e.g. migrating from expressjs to spring boot + zuul). I > > >>> would > > >>>>>>> argue > > >>>>>>>>>> that > > >>>>>>>>>>> if we WANT to do that, then it should be a separate > > >> feature > > >>>>>> branch. > > >>>>>>>>>>> > > >>>>>>>>>>> Thus, I leave with a question: is there a way to > > >> accomplish > > >>>> this > > >>>>>>>>> feature > > >>>>>>>>>>> without ripping out expressjs? > > >>>>>>>>>>> > > >>>>>>>>>>> - If so and it is feasible, I would argue that we should > > >>>>>> decouple > > >>>>>>>>> this > > >>>>>>>>>>> into a separate feature branch. > > >>>>>>>>>>> - If so and it is infeasible, I'd like to hear an argument > > >>> as > > >>>>>> to > > >>>>>>>>> the > > >>>>>>>>>>> infeasibility and let's decide given that > > >>>>>>>>>>> - If it is not possible, then I'd argue that we should > > >> keep > > >>>>>> them > > >>>>>>>>>> coupled > > >>>>>>>>>>> and move this through as-is. > > >>>>>>>>>>> > > >>>>>>>>>>> On a side-note, it feels a bit weird that we're narrowing > > >>> to a > > >>>>>>> bundled > > >>>>>>>>>>> proxy, rather than having that be a pluggable thing. I'm > > >> not > > >>>>>> super > > >>>>>>>>>>> knowledgeable in this space, so I apologize > > >>>>>>>>>>> in advance if this is naive, but isn't this a pluggable, > > >>>> external > > >>>>>>>>>> component > > >>>>>>>>>>> (e.g. nginx)? > > >>>>>>>>>>> > > >>>>>>>>>>> On Thu, Sep 27, 2018 at 5:05 PM Michael Miklavcic < > > >>>>>>>>>>> michael.miklav...@gmail.com> wrote: > > >>>>>>>>>>> > > >>>>>>>>>>>> I've spent some more time reading through Simon's > > >> response > > >>>> and > > >>>>>> the > > >>>>>>>>>> added > > >>>>>>>>>>>> sequence diagram. This is definitely helpful - thank you > > >>>> Simon. > > >>>>>>>>>>>> > > >>>>>>>>>>>> I need to redact my initial list: > > >>>>>>>>>>>> > > >>>>>>>>>>>> 1. Node migrated to Spring Boot, expressjs migrated to a > > >>>>>>>>>>>> non-JS/non-NodeJs proxying mechanism (ie Zuul in this > > >>> case) > > >>>>>>>>>>>> 2. JDBC removed completely in favor of LDAP > > >>>>>>>>>>>> 3. Knox/SSO > > >>>>>>>>>>>> > > >>>>>>>>>>>> I'm a bit conflicted on the best way to move forward and > > >>>> would > > >>>>>>> like > > >>>>>>>>>> some > > >>>>>>>>>>>> thoughts from other community members on this. I think > > >> an > > >>>>>> argument > > >>>>>>>>> can > > >>>>>>>>>> be > > >>>>>>>>>>>> made that 1 and 2 are independent of 3, and should/could > > >>>> really > > >>>>>> be > > >>>>>>>>>>>> independent PR's against master. > > >>>>>>>>>>>> > > >>>>>>>>>>>> The need for a replacement for expressjs (Zuul in this > > >>>> case) is > > >>>>>> an > > >>>>>>>>>>> artifact > > >>>>>>>>>>>> that our request/response cycle for REST calls is a > > >> simple > > >>>>>> matter > > >>>>>>> of > > >>>>>>>>>>>> forwarding with some additional headers for > > >>> authentication. > > >>>>>>> There's > > >>>>>>>>> a > > >>>>>>>>>>>> JSESSIONID managed by the client browser in our current > > >>>>>>>>> architecture, > > >>>>>>>>>> for > > >>>>>>>>>>>> example. You login to the alerts or the management UI > > >>> which > > >>>>>>>>> forwards a > > >>>>>>>>>>>> request to REST, which looks up credentials in a backend > > >>>>>> database, > > >>>>>>>>> and > > >>>>>>>>>>>> passes the results back up the chain. All browser > > >> requests > > >>>> go > > >>>>>>>>> directly > > >>>>>>>>>> to > > >>>>>>>>>>>> the specific UI you're working with - this is the CORS > > >>>> problem. > > >>>>>>> You > > >>>>>>>>>>> can't, > > >>>>>>>>>>>> without some effort with headers for adding other > > >> domains > > >>>> to the > > >>>>>>>>> safe > > >>>>>>>>>>> list > > >>>>>>>>>>>> or disabling the security check for CORS, make remote > > >>> calls > > >>>>>>>>> directly to > > >>>>>>>>>>>> REST. That's why we proxy. Switching over to Spring Boot > > >>>> leaves > > >>>>>> a > > >>>>>>>>> gap > > >>>>>>>>>>> with > > >>>>>>>>>>>> expressjs having handled the proxying and filtering, > > >> since > > >>>> it's > > >>>>>>> only > > >>>>>>>>>>>> available to a NodeJs application (it's server-side > > >>>> javascript > > >>>>>> vs > > >>>>>>>>> the > > >>>>>>>>>>>> client side javascript deployed via our Angular > > >>>> applications). > > >>>>>>> Enter > > >>>>>>>>>>> Zuul, > > >>>>>>>>>>>> which now effectively handles that. At runtime, Zuul is > > >> a > > >>>> part > > >>>>>> of > > >>>>>>>>> the > > >>>>>>>>>>>> Spring app that serves up our UI's. It handles the > > >>> requests > > >>>> via > > >>>>>>>>>>> filtering, > > >>>>>>>>>>>> forwards them to REST, manages the response back to the > > >>>> client. > > >>>>>>> Very > > >>>>>>>>>>>> similar to what expressjs was doing, per my current > > >>>>>> understanding. > > >>>>>>>>> The > > >>>>>>>>>>>> sequence diagrams Simon added are useful, and I think > > >> some > > >>>> of > > >>>>>> what > > >>>>>>>>> was > > >>>>>>>>>>> less > > >>>>>>>>>>>> clear was what we currently vs what the new changes are > > >>>> doing to > > >>>>>>> the > > >>>>>>>>>>>> architecture. This is no fault of Simon's - there simply > > >>>> wasn't > > >>>>>>> any > > >>>>>>>>>>>> architecture diagrams/documents around this before. > > >> Here's > > >>>> my > > >>>>>>>>>> impression > > >>>>>>>>>>> of > > >>>>>>>>>>>> the very very basic current state - someone more > > >> familiar > > >>>> with > > >>>>>>> this > > >>>>>>>>>>>> architecture please advise if I'm incorrect about > > >> anything > > >>>>>>> (probably > > >>>>>>>>>>> Ryan). > > >>>>>>>>>>>> > > >>>>>>>>>>>> https://imgur.com/f8GtSmh > > >>>>>>>>>>>> > > >>>>>>>>>>>> Zuul would be replacing the bit about expressjs in the > > >>>> diagram, > > >>>>>>> and > > >>>>>>>>>>> instead > > >>>>>>>>>>>> of node we have spring boot. This covers 1. 2 and 3 are > > >>>> other > > >>>>>>>>> issues. > > >>>>>>>>>> I'd > > >>>>>>>>>>>> like to see similar exposition of those server processes > > >>>> with > > >>>>>> knox > > >>>>>>>>>>>> involved. I imagine in that case we bump up from 3 to 4 > > >>>> server > > >>>>>>>>>> instances > > >>>>>>>>>>>> for the additional knox endpoint. > > >>>>>>>>>>>> > > >>>>>>>>>>>> Mike > > >>>>>>>>>>>> > > >>>>>>>>>>>> > > >>>>>>>>>>>> > > >>>>>>>>>>>> > > >>>>>>>>>>>> > > >>>>>>>>>>>> On Wed, Sep 19, 2018 at 11:28 AM James Sirota < > > >>>>>> jsir...@apache.org > > >>>>>>>> > > >>>>>>>>>>> wrote: > > >>>>>>>>>>>> > > >>>>>>>>>>>>> Thank you, Simon. The diagrams help a lot > > >>>>>>>>>>>>> > > >>>>>>>>>>>>> 19.09.2018, 21:27, "Simon Elliston Ball" < > > >>>>>>>>>> si...@simonellistonball.com > > >>>>>>>>>>>> : > > >>>>>>>>>>>>>> To clarify some of this I've put some documentation > > >>> into > > >>>>>>>>>>>>>> https://github.com/apache/metron/pull/1203 under > > >>>>>> METRON-1755 > > >>>>>>> ( > > >>>>>>>>>>>>>> https://issues.apache.org/jira/browse/METRON-1755). > > >>>>>> Hopefully > > >>>>>>>>> the > > >>>>>>>>>>>>> diagrams > > >>>>>>>>>>>>>> there should make it clearer. > > >>>>>>>>>>>>>> > > >>>>>>>>>>>>>> Simon > > >>>>>>>>>>>>>> > > >>>>>>>>>>>>>> On Tue, 18 Sep 2018 at 14:17, Simon Elliston Ball < > > >>>>>>>>>>>>>> si...@simonellistonball.com> wrote: > > >>>>>>>>>>>>>> > > >>>>>>>>>>>>>>> Hi Mike, > > >>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>> Some good points here which could do with some > > >>>>>>> clarification. > > >>>>>>>>> I > > >>>>>>>>>>>> suspect > > >>>>>>>>>>>>>>> the architecture documentation could be clearer and > > >>>> fill > > >>>>>> in > > >>>>>>>>> some > > >>>>>>>>>> of > > >>>>>>>>>>>>> these > > >>>>>>>>>>>>>>> gaps, and I'll have a look at working on that and > > >>>>>> providing > > >>>>>>>>> some > > >>>>>>>>>>>>> diagrams. > > >>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>> The short version is that the Zuul proxy gateway > > >> has > > >>>> been > > >>>>>>>>> added > > >>>>>>>>>> to > > >>>>>>>>>>>>> replace > > >>>>>>>>>>>>>>> the Nodejs express proxy used to gateway the REST > > >> api > > >>>>>> calls > > >>>>>>> in > > >>>>>>>>>> the > > >>>>>>>>>>>>> current > > >>>>>>>>>>>>>>> hosts. This is done in both cases to avoid CORS > > >>>>>> restrictions > > >>>>>>>>> by > > >>>>>>>>>>>>> allowing > > >>>>>>>>>>>>>>> the same host that serves the UI files to proxy > > >> call > > >>> to > > >>>>>> the > > >>>>>>>>> API. > > >>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>> The choice of Zuul was partly a pragmatic one (it's > > >>> the > > >>>>>> one > > >>>>>>>>>> that's > > >>>>>>>>>>>>> there > > >>>>>>>>>>>>>>> in the box as it were with Spring Boot, which we > > >> use > > >>>> for > > >>>>>> the > > >>>>>>>>> REST > > >>>>>>>>>>>> API, > > >>>>>>>>>>>>> via > > >>>>>>>>>>>>>>> the Spring Cloud Netflix project which wraps a > > >> bunch > > >>> of > > >>>>>>>>> related > > >>>>>>>>>>>> pieces > > >>>>>>>>>>>>> into > > >>>>>>>>>>>>>>> Spring). The choice of Spring Boot to host the UIs > > >>>>>>> themselves > > >>>>>>>>> was > > >>>>>>>>>>>>> similarly > > >>>>>>>>>>>>>>> for parity with the REST host, to simplify the > > >> stack > > >>>> (we > > >>>>>>>>> remove > > >>>>>>>>>> the > > >>>>>>>>>>>>>>> occasionally problematic need to install nodejs on > > >>>> target > > >>>>>>>>>> servers, > > >>>>>>>>>>>>> which is > > >>>>>>>>>>>>>>> outside of the regular OS and HDP stacks we > > >> support). > > >>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>> Arguably, the Zuul proxy is not necessary if we > > >> force > > >>>>>>>>> everything > > >>>>>>>>>>>>> through a > > >>>>>>>>>>>>>>> Knox instance, since Knox would provide a single > > >>>> endpoint. > > >>>>>>> We > > >>>>>>>>>>>> probably > > >>>>>>>>>>>>>>> however don't want to force Knox and SSL, hence > > >> using > > >>>> Zuul > > >>>>>>> to > > >>>>>>>>>> keep > > >>>>>>>>>>> it > > >>>>>>>>>>>>>>> closer to our current architecture. Zuul does some > > >>>> other > > >>>>>>> nice > > >>>>>>>>>>> things, > > >>>>>>>>>>>>> which > > >>>>>>>>>>>>>>> might help us in future, so it's really about > > >> laying > > >>>> down > > >>>>>>> some > > >>>>>>>>>>>> options > > >>>>>>>>>>>>> for > > >>>>>>>>>>>>>>> potentially doing micro-services style things at a > > >>>> later > > >>>>>>> date. > > >>>>>>>>>> I'm > > >>>>>>>>>>>> not > > >>>>>>>>>>>>>>> saying we have to, or even should go that way, it > > >>> will > > >>>>>> just > > >>>>>>>>> make > > >>>>>>>>>>> life > > >>>>>>>>>>>>>>> easier later if we decide to. It will also help us > > >> if > > >>>> we > > >>>>>>> want > > >>>>>>>>> to > > >>>>>>>>>>> add > > >>>>>>>>>>>>> HA, > > >>>>>>>>>>>>>>> circuit breaking etc to the architecture at a later > > >>>> date. > > >>>>>>> That > > >>>>>>>>>>> said, > > >>>>>>>>>>>> I > > >>>>>>>>>>>>>>> regret that I ever said the word micro-services, > > >>> since > > >>>>>> it's > > >>>>>>>>>> caused > > >>>>>>>>>>>>>>> confusion. Just think of it as a proxy to deal with > > >>> the > > >>>>>> CORS > > >>>>>>>>>>> problem. > > >>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>> Zuul is implemented as a set of filters, but we are > > >>> not > > >>>>>>> using > > >>>>>>>>> it > > >>>>>>>>>>> for > > >>>>>>>>>>>>> its > > >>>>>>>>>>>>>>> authentication filtering. We're using it as a > > >> proxy. > > >>>> Shiro > > >>>>>>> is > > >>>>>>>>> an > > >>>>>>>>>>>>>>> authentication framework, and could arguably used > > >> to > > >>>>>> provide > > >>>>>>>>> the > > >>>>>>>>>>>>> security > > >>>>>>>>>>>>>>> piece, but frankly wrapping shiro as a replacement > > >>> for > > >>>>>>> Spring > > >>>>>>>>>>>> Security > > >>>>>>>>>>>>> in a > > >>>>>>>>>>>>>>> Spring application seemed like it will make life a > > >>> lot > > >>>>>>> harder. > > >>>>>>>>>> This > > >>>>>>>>>>>>> could > > >>>>>>>>>>>>>>> be done, but it's not the native happy path, and > > >>> would > > >>>>>> pull > > >>>>>>> in > > >>>>>>>>>>>>> additional > > >>>>>>>>>>>>>>> dependencies that duplicate functionality that's > > >>>> already > > >>>>>>>>> embedded > > >>>>>>>>>>> in > > >>>>>>>>>>>>> Spring > > >>>>>>>>>>>>>>> Security. > > >>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>> The version of Knox used is the default from HDP. > > >> The > > >>>> link > > >>>>>>>>>> version > > >>>>>>>>>>>> you > > >>>>>>>>>>>>>>> mention is a docs link. I'll update it to be the > > >>> older > > >>>>>>>>> version, > > >>>>>>>>>>> which > > >>>>>>>>>>>>> is > > >>>>>>>>>>>>>>> the same and we can decide if we want to maintain > > >> the > > >>>>>>>>> freshness > > >>>>>>>>>> of > > >>>>>>>>>>> it > > >>>>>>>>>>>>> when > > >>>>>>>>>>>>>>> we look to upgrade underlying patterns. Either way, > > >>> the > > >>>>>>>>> content > > >>>>>>>>>> is > > >>>>>>>>>>>> the > > >>>>>>>>>>>>>>> same. > > >>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>> I did consider other hosting mechanisms, including > > >>>>>> Undertow > > >>>>>>> a > > >>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>> If you have a different suggestion to using the > > >>> Spring > > >>>>>>> default > > >>>>>>>>>> ways > > >>>>>>>>>>>> of > > >>>>>>>>>>>>>>> doing things, or we want to use a framework other > > >>> than > > >>>>>>> Spring > > >>>>>>>>> for > > >>>>>>>>>>>> this, > > >>>>>>>>>>>>>>> then maybe we could change to that, but the route > > >>>> chosen > > >>>>>>> here > > >>>>>>>>> is > > >>>>>>>>>>>>> definitely > > >>>>>>>>>>>>>>> the easy path in the context of the decision made > > >> to > > >>>> use > > >>>>>>>>> Spring > > >>>>>>>>>> in > > >>>>>>>>>>>>> metron > > >>>>>>>>>>>>>>> rest, and if anything opens up our choices while > > >>>>>> minimising, > > >>>>>>>>> in > > >>>>>>>>>>> fact > > >>>>>>>>>>>>>>> reducing, our dependency management overhead. > > >>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>> I hope that explains some of the thinking behind > > >> the > > >>>>>> choices > > >>>>>>>>>> made, > > >>>>>>>>>>>> but > > >>>>>>>>>>>>> the > > >>>>>>>>>>>>>>> guiding principals I followed were: > > >>>>>>>>>>>>>>> * Don't fight the framework if you don't have to > > >>>>>>>>>>>>>>> * Reduce the need for additional installation > > >> pieces > > >>>> and > > >>>>>>> third > > >>>>>>>>>>> party > > >>>>>>>>>>>>> repos > > >>>>>>>>>>>>>>> * Minimize dependencies we would have to manage > > >>>>>>>>>>>>>>> * Avoid excessive change of the architecture, or > > >>>> forcing > > >>>>>>>>> users to > > >>>>>>>>>>>> adopt > > >>>>>>>>>>>>>>> Knox if they didn't want the SSL overhead. > > >>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>> Simon > > >>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>> On Tue, 18 Sep 2018 at 02:46, Michael Miklavcic < > > >>>>>>>>>>>>>>> michael.miklav...@gmail.com> wrote: > > >>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>> Thanks for the write-up Ryan, this is a great > > >>> start. I > > >>>>>> have > > >>>>>>>>> some > > >>>>>>>>>>>>> further > > >>>>>>>>>>>>>>>> questions based on your feedback and in addition > > >> to > > >>> my > > >>>>>>>>> initial > > >>>>>>>>>>>> thread. > > >>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>> Just for clarification, what version of Knox are > > >> we > > >>>>>> using? > > >>>>>>>>> HDP > > >>>>>>>>>>>> 2.6.5, > > >>>>>>>>>>>>>>>> which > > >>>>>>>>>>>>>>>> is what we currently run full dev against, > > >> supports > > >>>>>> 0.12.0. > > >>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>> > > >>>>>>>>>>>>> > > >>>>>>>>>>>> > > >>>>>>>>>>> > > >>>>>>>>>> > > >>>>>>>>> > > >>>>>>> > > >>>>>> > > >>>> > > >>> > > >> > > > > > > https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.6.5/bk_release-notes/content/comp_versions.html > > >>>>>>>>>>>>>>>> . > > >>>>>>>>>>>>>>>> I see references to Knox 1.1.0 (latest) in this > > >>>> committed > > >>>>>>> PR > > >>>>>>>>> - > > >>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>> > > >>>>>>>>>>>>> > > >>>>>>>>>>>> > > >>>>>>>>>>> > > >>>>>>>>>> > > >>>>>>>>> > > >>>>>>> > > >>>>>> > > >>>> > > >>> > > >> > > > > > > https://github.com/apache/metron/pull/1111/files#diff-70b412194819f3cb829566f05d77c1a6R122 > > >>>>>>>>>>>>>>>> . > > >>>>>>>>>>>>>>>> This is probably just a super small mismatch, and > > >> it > > >>>>>>> probably > > >>>>>>>>>> goes > > >>>>>>>>>>>>> without > > >>>>>>>>>>>>>>>> saying, but I just want to be doubly sure that > > >> we're > > >>>>>>>>> installing > > >>>>>>>>>>> the > > >>>>>>>>>>>>>>>> default > > >>>>>>>>>>>>>>>> via the standard install mechanism as opposed to > > >>>>>> something > > >>>>>>>>>>> separate > > >>>>>>>>>>>>> and > > >>>>>>>>>>>>>>>> manual. > > >>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>> On the subject of Zuul wrt Nodejs filters. I'd > > >> like > > >>> to > > >>>>>> hear > > >>>>>>>>> some > > >>>>>>>>>>>> more > > >>>>>>>>>>>>>>>> detail on: > > >>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>> 1. Why do we need filtering via Zuul? For > > >> instance, > > >>> is > > >>>>>>>>>>> filtering > > >>>>>>>>>>>>>>>> routing > > >>>>>>>>>>>>>>>> not handled by Knox? From the beginner docs: "The > > >>>>>>> gateway > > >>>>>>>>>>> itself > > >>>>>>>>>>>>> is a > > >>>>>>>>>>>>>>>> layer > > >>>>>>>>>>>>>>>> over an embedded Jetty JEE server. At the very > > >>> highest > > >>>>>>>>> level > > >>>>>>>>>>> the > > >>>>>>>>>>>>>>>> gateway > > >>>>>>>>>>>>>>>> processes requests by using request URLs to lookup > > >>>>>>>>> specific > > >>>>>>>>>> JEE > > >>>>>>>>>>>>> Servlet > > >>>>>>>>>>>>>>>> Filter chain that is used to process the request. > > >>> The > > >>>>>>>>> gateway > > >>>>>>>>>>>>> framework > > >>>>>>>>>>>>>>>> provides extensible mechanisms to assemble chains > > >> of > > >>>>>>>>> custom > > >>>>>>>>>>>> filters > > >>>>>>>>>>>>>>>> that > > >>>>>>>>>>>>>>>> support secured access to services." [1] > > >>>>>>>>>>>>>>>> 2. What other library options were considered for > > >>> this > > >>>>>>>>>> feature > > >>>>>>>>>>>> and > > >>>>>>>>>>>>> how > > >>>>>>>>>>>>>>>> was it chosen over the others? I search on "apache > > >>>>>>> spring > > >>>>>>>>> web > > >>>>>>>>>>>>> filters" > > >>>>>>>>>>>>>>>> and > > >>>>>>>>>>>>>>>> it's almost all about Shiro - > > >>>>>>>>>>>> https://shiro.apache.org/spring.html. > > >>>>>>>>>>>>> I > > >>>>>>>>>>>>>>>> also see quite a bit about filtering for Spring > > >> Boot > > >>>>>>>>>>> applications > > >>>>>>>>>>>>> along > > >>>>>>>>>>>>>>>> with a write-up of how to accomplish the same with > > >>> Web > > >>>>>>> MVC > > >>>>>>>>>>> here - > > >>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>> > > >>>>>>>>>>>>> > > >>>>>>>>>>>> > > >>>>>>>>>>> > > >>>>>>>>>> > > >>>>>>>>> > > >>>>>>> > > >>>>>> > > >>>> > > >>> > > >> > > > > > > https://stackoverflow.com/questions/19825946/how-to-add-a-filter-class-in-spring-boot > > >>>>>>>>>>>>>>>> . > > >>>>>>>>>>>>>>>> The Knox documentation boilerplate examples are > > >> also > > >>>>>>> using > > >>>>>>>>>>> Shiro. > > >>>>>>>>>>>>>>>> "shiro.ini - The configuration file for the Shiro > > >>>>>>>>>>> authentication > > >>>>>>>>>>>>>>>> provider’s > > >>>>>>>>>>>>>>>> filters. This information is derived from the > > >>>>>>> information > > >>>>>>>>> in > > >>>>>>>>>>> the > > >>>>>>>>>>>>>>>> provider > > >>>>>>>>>>>>>>>> section of the topology file." [1] > > >>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>> My assumption is that there are deliberate > > >> decisions > > >>>> in > > >>>>>>>>> favor of > > >>>>>>>>>>>> this > > >>>>>>>>>>>>> mix > > >>>>>>>>>>>>>>>> of technologies over others, and I think some > > >>>> additional > > >>>>>>>>>>> explanation > > >>>>>>>>>>>>> will > > >>>>>>>>>>>>>>>> make that clear. As it stands per the Knox > > >>>> documentation, > > >>>>>>> it > > >>>>>>>>>> looks > > >>>>>>>>>>>>> like > > >>>>>>>>>>>>>>>> we're going on a different route from the > > >>>>>>>>> preferred/recommended > > >>>>>>>>>>>>> idioms. > > >>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>> [1] > > >>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>> > > >>>>>>>>>>>>> > > >>>>>>>>>>>> > > >>>>>>>>>>> > > >>>>>>>>>> > > >>>>>>>>> > > >>>>>>> > > >>>>>> > > >>>> > > >>> > > >> > > > > > > http://knox.apache.org/books/knox-0-12-0/dev-guide.html#Architecture+Overview > > >>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>> Ryan, I agree about microservices. This should not > > >>>> derail > > >>>>>>> nor > > >>>>>>>>>> be a > > >>>>>>>>>>>>> major > > >>>>>>>>>>>>>>>> part of discussion around this feature, imho. I > > >>> think > > >>>>>>> there's > > >>>>>>>>>>> quite > > >>>>>>>>>>>> a > > >>>>>>>>>>>>> bit > > >>>>>>>>>>>>>>>> left to discuss on that subject. I want to make > > >> sure > > >>>> that > > >>>>>>>>> we're > > >>>>>>>>>>> not > > >>>>>>>>>>>>>>>> prematurely favoring architectural choices by > > >>> pulling > > >>>> in > > >>>>>>>>>> libraries > > >>>>>>>>>>>>> that > > >>>>>>>>>>>>>>>> are > > >>>>>>>>>>>>>>>> potentially opinionated about how to accomplish > > >>> those > > >>>>>>> goals. > > >>>>>>>>> If > > >>>>>>>>>>> they > > >>>>>>>>>>>>> are, > > >>>>>>>>>>>>>>>> I > > >>>>>>>>>>>>>>>> would expect we are comfortable ripping those > > >>>> libraries > > >>>>>> out > > >>>>>>>>> if > > >>>>>>>>>> the > > >>>>>>>>>>>>>>>> community favors a different direction. > > >>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>> On the subject of Spring Boot vs Nodejs. I can see > > >>>> some > > >>>>>>>>>> rationale > > >>>>>>>>>>>> for > > >>>>>>>>>>>>>>>> making things homogenous (though, in a > > >> microservices > > >>>>>>>>>> architecture, > > >>>>>>>>>>>> if > > >>>>>>>>>>>>> we > > >>>>>>>>>>>>>>>> go > > >>>>>>>>>>>>>>>> that route, that's not strictly necessary), but > > >> what > > >>>> is > > >>>>>> the > > >>>>>>>>>>>>> justification > > >>>>>>>>>>>>>>>> for Spring Boot over Nodejs? Why would want one > > >> over > > >>>> the > > >>>>>>>>> other? > > >>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>> On Mon, Sep 17, 2018 at 3:38 PM Ryan Merriman < > > >>>>>>>>>>> merrim...@gmail.com> > > >>>>>>>>>>>>>>>> wrote: > > >>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>> I have reviewed a couple different PRs so I'll > > >> add > > >>>> some > > >>>>>>>>>> context > > >>>>>>>>>>>>> where I > > >>>>>>>>>>>>>>>>> can. Obviously Simon would be the most qualified > > >>> to > > >>>>>>> answer > > >>>>>>>>> but > > >>>>>>>>>>>> I'll > > >>>>>>>>>>>>>>>> add my > > >>>>>>>>>>>>>>>>> thoughts. > > >>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>> For question 1, while they may not all be > > >>> necessary > > >>>> I > > >>>>>>>>> think it > > >>>>>>>>>>>> does > > >>>>>>>>>>>>> make > > >>>>>>>>>>>>>>>>> sense to include them in this feature branch if > > >>> our > > >>>>>>> primary > > >>>>>>>>>> goal > > >>>>>>>>>>>> is > > >>>>>>>>>>>>>>>>> integrating Knox SSO. We could push off removing > > >>>> JDBC > > >>>>>>>>>>>> authentication > > >>>>>>>>>>>>>>>> for > > >>>>>>>>>>>>>>>>> reasons I'll get to in my response to question > > >> 2. > > >>>> If we > > >>>>>>>>> want > > >>>>>>>>>> to > > >>>>>>>>>>> do > > >>>>>>>>>>>>> one > > >>>>>>>>>>>>>>>> at > > >>>>>>>>>>>>>>>>> a time (switch to spring boot, add Zuul as a > > >>>>>> dependency, > > >>>>>>>>> then > > >>>>>>>>>>> add > > >>>>>>>>>>>>> Knox > > >>>>>>>>>>>>>>>> SSO) > > >>>>>>>>>>>>>>>>> then that's ok but I do think there are > > >>> dependencies > > >>>>>> and > > >>>>>>>>>> should > > >>>>>>>>>>> be > > >>>>>>>>>>>>> done > > >>>>>>>>>>>>>>>> in > > >>>>>>>>>>>>>>>>> order. For example, adding Knox SSO requires > > >> some > > >>>> work > > >>>>>>>>> around > > >>>>>>>>>>>>> request > > >>>>>>>>>>>>>>>>> filtering. If we were to do this before moving > > >> to > > >>>>>> Spring > > >>>>>>>>> Boot > > >>>>>>>>>> we > > >>>>>>>>>>>>> would > > >>>>>>>>>>>>>>>>> need to implement the filters in Nodejs which > > >>> would > > >>>> be > > >>>>>>>>>> throwaway > > >>>>>>>>>>>>> once we > > >>>>>>>>>>>>>>>>> get around to migrating away from that. For > > >> Zuul, > > >>> I > > >>>>>>> believe > > >>>>>>>>>> it's > > >>>>>>>>>>>>>>>> purpose > > >>>>>>>>>>>>>>>>> is to facilitate the filtering (although it > > >> does a > > >>>> lot > > >>>>>>>>> more) > > >>>>>>>>>> so > > >>>>>>>>>>> it > > >>>>>>>>>>>>>>>> doesn't > > >>>>>>>>>>>>>>>>> make sense to add that separate from the Knox > > >> SSO > > >>>> work. > > >>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>> For question 2, I think you bring up a good > > >> point. > > >>>> We > > >>>>>>>>> probably > > >>>>>>>>>>>> don't > > >>>>>>>>>>>>>>>> want > > >>>>>>>>>>>>>>>>> to just rip our current authentication method > > >> out. > > >>>> We > > >>>>>>> might > > >>>>>>>>>> want > > >>>>>>>>>>>> to > > >>>>>>>>>>>>>>>>> consider deprecating it instead and making Knox > > >>> SSO > > >>>> and > > >>>>>>>>> LDAP > > >>>>>>>>>>>>>>>> authentication > > >>>>>>>>>>>>>>>>> optional. > > >>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>> For question 3, this is a bigger shift than > > >> just a > > >>>>>>>>> component > > >>>>>>>>>>>>> upgrade. > > >>>>>>>>>>>>>>>> It's > > >>>>>>>>>>>>>>>>> more like shifting platforms, from Elasticsearch > > >>> to > > >>>>>> Solr > > >>>>>>>>> for > > >>>>>>>>>>>>> example. > > >>>>>>>>>>>>>>>> Like > > >>>>>>>>>>>>>>>>> I alluded to in my response to question 1, I > > >> don't > > >>>>>> think > > >>>>>>> we > > >>>>>>>>>>> should > > >>>>>>>>>>>>>>>> require > > >>>>>>>>>>>>>>>>> throwaway work just because we want to review > > >>> these > > >>>>>> parts > > >>>>>>>>>>>>> separately. > > >>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>> For question 4, I will defer to Simon. I don't > > >>>> believe > > >>>>>> we > > >>>>>>>>>>>>> necessarily > > >>>>>>>>>>>>>>>>> require Zuul so I will let him elaborate on why > > >> he > > >>>>>> choose > > >>>>>>>>> that > > >>>>>>>>>>>>> library > > >>>>>>>>>>>>>>>> and > > >>>>>>>>>>>>>>>>> what the potential impact is of adding it to our > > >>>>>> project. > > >>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>> For question 5 and 6, I will also defer to Simon > > >>> on > > >>>>>> this. > > >>>>>>>>> The > > >>>>>>>>>>>> focus > > >>>>>>>>>>>>> of > > >>>>>>>>>>>>>>>>> this feature as I understand it is a consistent > > >>>>>>>>> authentication > > >>>>>>>>>>>>> mechanism > > >>>>>>>>>>>>>>>>> and support for SSO. I will let him lay out his > > >>>> vision > > >>>>>>> for > > >>>>>>>>>> micro > > >>>>>>>>>>>>>>>> services. > > >>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>> Knox SSO would be a great improvement and is > > >> what > > >>> I > > >>>>>> think > > >>>>>>>>> we > > >>>>>>>>>>>> should > > >>>>>>>>>>>>>>>> focus > > >>>>>>>>>>>>>>>>> on in this feature branch. Micro services is > > >>>> something > > >>>>>> we > > >>>>>>>>>> should > > >>>>>>>>>>>>>>>> certainly > > >>>>>>>>>>>>>>>>> discuss but it might be a bit of a distraction > > >>> and I > > >>>>>>>>> wouldn't > > >>>>>>>>>>> want > > >>>>>>>>>>>>> to > > >>>>>>>>>>>>>>>> hold > > >>>>>>>>>>>>>>>>> up the other useful parts. > > >>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>> On Fri, Sep 14, 2018 at 8:38 PM Michael > > >> Miklavcic > > >>> < > > >>>>>>>>>>>>>>>>> michael.miklav...@gmail.com> wrote: > > >>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>> Hey all, > > >>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>> I started looking through the Knox SSO feature > > >>>> branch > > >>>>>>>>> (see > > >>>>>>>>>>> here > > >>>>>>>>>>>>>>>>>> > > >>> https://issues.apache.org/jira/browse/METRON-1663 > > >>>> ). > > >>>>>>>>> This is > > >>>>>>>>>>>> some > > >>>>>>>>>>>>>>>> great > > >>>>>>>>>>>>>>>>> new > > >>>>>>>>>>>>>>>>>> security functionality work and it looks like > > >> it > > >>>> will > > >>>>>>>>> bring > > >>>>>>>>>>> some > > >>>>>>>>>>>>>>>>> important > > >>>>>>>>>>>>>>>>>> new features to the Metron platform. I'm > > >> coming > > >>> at > > >>>>>> this > > >>>>>>>>>> pretty > > >>>>>>>>>>>>> green, > > >>>>>>>>>>>>>>>> so > > >>>>>>>>>>>>>>>>> I > > >>>>>>>>>>>>>>>>>> do have some questions regarding the proposed > > >>>> changes > > >>>>>>>>> from a > > >>>>>>>>>>>> high > > >>>>>>>>>>>>>>>> level > > >>>>>>>>>>>>>>>>>> architectural perspective. There are a few > > >>> changes > > >>>>>>> within > > >>>>>>>>>> the > > >>>>>>>>>>>>> current > > >>>>>>>>>>>>>>>> FB > > >>>>>>>>>>>>>>>>>> PR's that I think could use some further > > >>>> explanation. > > >>>>>>> At > > >>>>>>>>>> first > > >>>>>>>>>>>>>>>> glance, it > > >>>>>>>>>>>>>>>>>> seems we could potentially simplify this > > >> branch > > >>> a > > >>>>>> great > > >>>>>>>>> deal > > >>>>>>>>>>> and > > >>>>>>>>>>>>> get > > >>>>>>>>>>>>>>>> it > > >>>>>>>>>>>>>>>>>> completed much sooner if we narrowed the > > >> focus a > > >>>> bit. > > >>>>>>>>> But I > > >>>>>>>>>>>> could > > >>>>>>>>>>>>>>>>> certainly > > >>>>>>>>>>>>>>>>>> be wrong here and happy for other opinions. I > > >>>>>> searched > > >>>>>>>>>> through > > >>>>>>>>>>>> the > > >>>>>>>>>>>>>>>>> mailing > > >>>>>>>>>>>>>>>>>> list history to see if there is any additional > > >>>>>>> background > > >>>>>>>>>> and > > >>>>>>>>>>>> the > > >>>>>>>>>>>>> main > > >>>>>>>>>>>>>>>>>> DISCUSS thread I could find was regarding > > >>>> initially > > >>>>>>>>> setting > > >>>>>>>>>> up > > >>>>>>>>>>>> the > > >>>>>>>>>>>>>>>>> feature > > >>>>>>>>>>>>>>>>>> branch, which talked about adding Knox and > > >> LDAP. > > >>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>> > > >>>>>>>>>>>>> > > >>>>>>>>>>>> > > >>>>>>>>>>> > > >>>>>>>>>> > > >>>>>>>>> > > >>>>>>> > > >>>>>> > > >>>> > > >>> > > >> > > > > > > https://lists.apache.org/thread.html/cac2e6314284015b487121e77abf730abbb7ebec4ace014b19093b4c@%3Cdev.metron.apache.org%3E > > >>>>>>>>>>>>>>>>>> . > > >>>>>>>>>>>>>>>>>> If I've missed any follow-up, please let me > > >>> know. > > >>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>> Looking at the broader set of Jiras associated > > >>>> with > > >>>>>>> 1663 > > >>>>>>>>> and > > >>>>>>>>>>> the > > >>>>>>>>>>>>>>>> first PR > > >>>>>>>>>>>>>>>>>> 1665, it looks like there are 4 main thrusts > > >> of > > >>>> this > > >>>>>>>>> branch > > >>>>>>>>>>>> right > > >>>>>>>>>>>>> now: > > >>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>> 1. Knox/SSO > > >>>>>>>>>>>>>>>>>> 2. Node migrated to Spring Boot > > >>>>>>>>>>>>>>>>>> 3. JDBC removed completely in favor of LDAP > > >>>>>>>>>>>>>>>>>> 4. Introduction of Zuul, also microservices? > > >>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>> I strongly urge for the purpose of reviewing > > >>> this > > >>>>>>> feature > > >>>>>>>>>>> branch > > >>>>>>>>>>>>> that > > >>>>>>>>>>>>>>>> we > > >>>>>>>>>>>>>>>>>> base much of the discussion off of > > >>>>>>>>>>>>>>>>>> > > >>> https://issues.apache.org/jira/browse/METRON-1755 > > >>>> , > > >>>>>> the > > >>>>>>>>>>>>> architecture > > >>>>>>>>>>>>>>>>>> diagram. Minimally, an explanation of the > > >>> current > > >>>>>>>>>> architecture > > >>>>>>>>>>>>> along > > >>>>>>>>>>>>>>>> with > > >>>>>>>>>>>>>>>>>> discussion around the additional proposed > > >>> changes > > >>>> and > > >>>>>>>>>>> rationale > > >>>>>>>>>>>>> would > > >>>>>>>>>>>>>>>> be > > >>>>>>>>>>>>>>>>>> useful for evaluation. I don't have a solid > > >>> enough > > >>>>>>>>>>> understanding > > >>>>>>>>>>>>> yet > > >>>>>>>>>>>>>>>> of > > >>>>>>>>>>>>>>>>> the > > >>>>>>>>>>>>>>>>>> full scope of changes and how they differ from > > >>> the > > >>>>>>>>> existing > > >>>>>>>>>>>>>>>> architecture > > >>>>>>>>>>>>>>>>>> just from looking at the PR's alone. > > >>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>> 1. The first question is a general one > > >> regarding > > >>>> the > > >>>>>>>>>> necessity > > >>>>>>>>>>>> of > > >>>>>>>>>>>>>>>> the > > >>>>>>>>>>>>>>>>> 3 > > >>>>>>>>>>>>>>>>>> additional features alongside Knox - migrating > > >>>> Node > > >>>>>> to > > >>>>>>>>>> Spring > > >>>>>>>>>>>>> Boot, > > >>>>>>>>>>>>>>>>>> removing JDBC altogether, adding dependencies > > >> on > > >>>>>>>>> Netflix's > > >>>>>>>>>>> Zuul > > >>>>>>>>>>>>>>>>>> framework. > > >>>>>>>>>>>>>>>>>> Are these necessary for adding Knox/SSO? They > > >>> seem > > >>>>>> like > > >>>>>>>>>>>>> potentially > > >>>>>>>>>>>>>>>>>> separate features, imo. > > >>>>>>>>>>>>>>>>>> 2. It looks like LDAP will be a required > > >>> component > > >>>>>> for > > >>>>>>>>>>>> interacting > > >>>>>>>>>>>>>>>>> with > > >>>>>>>>>>>>>>>>>> Metron via the UI's. I see this PR > > >>>>>>>>>>>>>>>>>> https://github.com/apache/metron/pull/1186 > > >>> which > > >>>>>>> removes > > >>>>>>>>>> JDBC > > >>>>>>>>>>>>>>>>>> authentication. Are we ready to remove it > > >>>> completely > > >>>>>> or > > >>>>>>>>>> would > > >>>>>>>>>>> it > > >>>>>>>>>>>>> be > > >>>>>>>>>>>>>>>>>> better > > >>>>>>>>>>>>>>>>>> to leave it as a minimal installation option? > > >>>> What is > > >>>>>>> the > > >>>>>>>>>>>> proposed > > >>>>>>>>>>>>>>>>>> migration path for existing users? Do we feel > > >>>>>>> comfortable > > >>>>>>>>>>>>> requiring > > >>>>>>>>>>>>>>>>> that > > >>>>>>>>>>>>>>>>>> all installations, including full dev, install > > >>> and > > >>>>>>>>> configure > > >>>>>>>>>>>> LDAP? > > >>>>>>>>>>>>>>>> For > > >>>>>>>>>>>>>>>>>> comparison, in the PCAP feature branch we > > >>>> discussed > > >>>>>>>>> removing > > >>>>>>>>>>> the > > >>>>>>>>>>>>>>>>>> existing > > >>>>>>>>>>>>>>>>>> PCAP REST application in the initial > > >> discussion, > > >>>> got > > >>>>>>>>>>> agreement, > > >>>>>>>>>>>>> and > > >>>>>>>>>>>>>>>>>> later > > >>>>>>>>>>>>>>>>>> removed it in the course of working on the > > >>> feature > > >>>>>>>>> branch. > > >>>>>>>>>> The > > >>>>>>>>>>>> PR > > >>>>>>>>>>>>>>>> is > > >>>>>>>>>>>>>>>>>> fairly > > >>>>>>>>>>>>>>>>>> clear, however I think we're just missing some > > >>>> basic > > >>>>>>>>>>> discussion > > >>>>>>>>>>>>>>>> around > > >>>>>>>>>>>>>>>>>> the > > >>>>>>>>>>>>>>>>>> implications, as I've outlined above. Some > > >>>> additional > > >>>>>>>>>> relevant > > >>>>>>>>>>>>>>>>>> discussion > > >>>>>>>>>>>>>>>>>> occurred on this PR > > >>>>>>>>>>> https://github.com/apache/metron/pull/1112 > > >>>>>>>>>>>>>>>> which > > >>>>>>>>>>>>>>>>>> would be good to summarize for purposes of > > >> this > > >>>>>>>>> overarching > > >>>>>>>>>>>>>>>>> architecture > > >>>>>>>>>>>>>>>>>> discussion. > > >>>>>>>>>>>>>>>>>> 3. Migration from Node to Spring Boot. I > > >> believe > > >>>> this > > >>>>>>> is > > >>>>>>>>>>> already > > >>>>>>>>>>>>>>>> used > > >>>>>>>>>>>>>>>>> by > > >>>>>>>>>>>>>>>>>> the REST application and if anything brings > > >> some > > >>>>>>>>> cohesion to > > >>>>>>>>>>> our > > >>>>>>>>>>>>>>>>> server > > >>>>>>>>>>>>>>>>>> strategy. Strictly speaking, is there a reason > > >>>> this > > >>>>>> is > > >>>>>>>>>>> required > > >>>>>>>>>>>>> for > > >>>>>>>>>>>>>>>>>> Knox? > > >>>>>>>>>>>>>>>>>> It seems comparable to a component upgrade, > > >> such > > >>>> as > > >>>>>>>>> moving > > >>>>>>>>>>> from > > >>>>>>>>>>>> ES > > >>>>>>>>>>>>>>>> 2.x > > >>>>>>>>>>>>>>>>>> to > > >>>>>>>>>>>>>>>>>> 5.6.x and upgrading Angular 6. > > >>>>>>>>>>>>>>>>>> 4. Introduction of Netflix's Zuul. > > >>>>>>>>>>>>>>>>>> > > >>> https://issues.apache.org/jira/browse/METRON-1665 > > >>>> . > > >>>>>>>>>>>>>>>>>> - > "The UIs currently proxy to the REST API > > >> to > > >>>> avoid > > >>>>>>>>> CORS > > >>>>>>>>>>>>>>>> issues, > > >>>>>>>>>>>>>>>>>> this will be achieved with Zuul." > > >>>>>>>>>>>>>>>>>> - Can we elaborate more on where or how CORS > > >> is > > >>> a > > >>>>>>> problem > > >>>>>>>>>> with > > >>>>>>>>>>>>>>>> our > > >>>>>>>>>>>>>>>>>> existing architecture, how Zuul will help > > >> solve > > >>>> that, > > >>>>>>> and > > >>>>>>>>>> how > > >>>>>>>>>>> it > > >>>>>>>>>>>>>>>>>> fits with > > >>>>>>>>>>>>>>>>>> Knox? Wouldn't this be handled by Knox? Since > > >>>> Larry > > >>>>>>> McCay > > >>>>>>>>>>>>>>>> chimed in > > >>>>>>>>>>>>>>>>>> with > > >>>>>>>>>>>>>>>>>> interest on the original SSO thread about the > > >>> FB, > > >>>> I'm > > >>>>>>>>> hoping > > >>>>>>>>>>> he > > >>>>>>>>>>>>>>>> is > > >>>>>>>>>>>>>>>>>> also > > >>>>>>>>>>>>>>>>>> willing to chime in on this as well. > > >>>>>>>>>>>>>>>>>> - This looks like it has the potential to be a > > >>>> rather > > >>>>>>>>> large > > >>>>>>>>>>>>>>>> piece > > >>>>>>>>>>>>>>>>> of > > >>>>>>>>>>>>>>>>>> fundamental infrastructure (as it's also > > >>>> pertinent to > > >>>>>>>>>>>>>>>>> microservices) > > >>>>>>>>>>>>>>>>>> to > > >>>>>>>>>>>>>>>>>> pull into the platform, and I'd like to be > > >> sure > > >>>> the > > >>>>>>>>>> community > > >>>>>>>>>>> is > > >>>>>>>>>>>>>>>>>> aware of > > >>>>>>>>>>>>>>>>>> and is OK with the implications. > > >>>>>>>>>>>>>>>>>> 5. > "The proposal is to use a spring boot > > >>>>>> application, > > >>>>>>>>>>> allowing > > >>>>>>>>>>>>>>>> us to > > >>>>>>>>>>>>>>>>>> harmonize the security implementation across > > >> the > > >>>> UI > > >>>>>>>>> static > > >>>>>>>>>>>> servers > > >>>>>>>>>>>>>>>> and > > >>>>>>>>>>>>>>>>>> the > > >>>>>>>>>>>>>>>>>> REST layer, and to provide a routing platform > > >>> for > > >>>>>> later > > >>>>>>>>>>>>>>>>> microservices." > > >>>>>>>>>>>>>>>>>> - > > >>>>>>>>>>>>>>>>>> > > >>> https://issues.apache.org/jira/browse/METRON-1665 > > >>>> . > > >>>>>>>>>>>>>>>>>> - Microservices is a pretty loaded term. I > > >> know > > >>>> there > > >>>>>>> had > > >>>>>>>>>> been > > >>>>>>>>>>>>>>>> some > > >>>>>>>>>>>>>>>>>> discussion a while back during the PCAP > > >> feature > > >>>>>> branch > > >>>>>>>>>> start, > > >>>>>>>>>>>>>>>> but I > > >>>>>>>>>>>>>>>>>> don't > > >>>>>>>>>>>>>>>>>> recall ever reaching a consensus on it. More > > >>>> detail > > >>>>>> in > > >>>>>>>>> this > > >>>>>>>>>>>>>>>> thread > > >>>>>>>>>>>>>>>>> - > > >>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>> > > >>>>>>>>>>>>> > > >>>>>>>>>>>> > > >>>>>>>>>>> > > >>>>>>>>>> > > >>>>>>>>> > > >>>>>>> > > >>>>>> > > >>>> > > >>> > > >> > > > > > > https://lists.apache.org/thread.html/1db7c6fa1b0f364f8c03520db9989b4f7a446de82eb4d9786055048c@%3Cdev.metron.apache.org%3E > > >>>>>>>>>>>>>>>>>> . > > >>>>>>>>>>>>>>>>>> Can we get some clarification on what is meant > > >>> by > > >>>>>>>>>>> microservices > > >>>>>>>>>>>>>>>>>> in the case > > >>>>>>>>>>>>>>>>>> of this FB and relevant PR's, what that > > >>>> architecture > > >>>>>>>>> looks > > >>>>>>>>>>> like, > > >>>>>>>>>>>>>>>>> and > > >>>>>>>>>>>>>>>>>> how > > >>>>>>>>>>>>>>>>>> it's achieved with the proposed changes in > > >> this > > >>>>>> PR/FB? > > >>>>>>> It > > >>>>>>>>>>> seems > > >>>>>>>>>>>>>>>>> Zuul > > >>>>>>>>>>>>>>>>>> is > > >>>>>>>>>>>>>>>>>> also pertinent to this discussion, but there > > >> are > > >>>> many > > >>>>>>>>> ways > > >>>>>>>>>> to > > >>>>>>>>>>>>>>>>>> skin this cat > > >>>>>>>>>>>>>>>>>> so I don't want to presume - > > >>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>> > > >>>>>>>>>>>>> > > >>>>>>>>>> > > >>>>>>> > > >>>> > > https://blog.heroku.com/using_netflix_zuul_to_proxy_your_microservices > > >>>>>>>>>>>>>>>>>> 6. Zuul, Spring Boot, and microservices - > > >>> Closely > > >>>>>>>>> related to > > >>>>>>>>>>>>>>>>> point 5 > > >>>>>>>>>>>>>>>>>> above. It seems that we weren't quite ready > > >> for > > >>>> this > > >>>>>>>>> when it > > >>>>>>>>>>> was > > >>>>>>>>>>>>>>>>>> brought up > > >>>>>>>>>>>>>>>>>> in May, or at the very least we had some > > >> concern > > >>>> of > > >>>>>>> what > > >>>>>>>>>>>> direction > > >>>>>>>>>>>>>>>> to > > >>>>>>>>>>>>>>>>>> go. > > >>>>>>>>>>>>>>>>>> What is the operational impact, mpack impact, > > >>> and > > >>>> how > > >>>>>>> we > > >>>>>>>>>>> propose > > >>>>>>>>>>>>> to > > >>>>>>>>>>>>>>>>>> manage > > >>>>>>>>>>>>>>>>>> it with Kerberos, etc.? > > >>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>> > > >>>>>>>>>>>>> > > >>>>>>>>>>>> > > >>>>>>>>>>> > > >>>>>>>>>> > > >>>>>>>>> > > >>>>>>> > > >>>>>> > > >>>> > > >>> > > >> > > > > > > https://lists.apache.org/thread.html/c19904681e6a6d9ea3131be3d1a65b24447dca31b4aff588b263fd87@%3Cdev.metron.apache.org%3E > > >>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>> There is a lot to like in this feature branch, > > >>>> imo. > > >>>>>>> Great > > >>>>>>>>>>>> feature > > >>>>>>>>>>>>>>>>> addition > > >>>>>>>>>>>>>>>>>> with Knox and SSO. Introduction of LDAP > > >> support > > >>>> for > > >>>>>>>>>>>>> authentication for > > >>>>>>>>>>>>>>>>>> Metron UI's. Simplification/unification of our > > >>>> server > > >>>>>>>>>> hosting > > >>>>>>>>>>>>>>>>>> infrastructure. I'm hoping we can flesh out > > >> some > > >>>> of > > >>>>>> the > > >>>>>>>>>>> details > > >>>>>>>>>>>>>>>> pointed > > >>>>>>>>>>>>>>>>> out > > >>>>>>>>>>>>>>>>>> above a bit more and get this feature through. > > >>>> Great > > >>>>>>>>> work so > > >>>>>>>>>>>> far! > > >>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>> Best, > > >>>>>>>>>>>>>>>>>> Mike Miklavcic > > >>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>> -- > > >>>>>>>>>>>>>>> -- > > >>>>>>>>>>>>>>> simon elliston ball > > >>>>>>>>>>>>>>> @sireb > > >>>>>>>>>>>>>> > > >>>>>>>>>>>>>> -- > > >>>>>>>>>>>>>> -- > > >>>>>>>>>>>>>> simon elliston ball > > >>>>>>>>>>>>>> @sireb > > >>>>>>>>>>>>> > > >>>>>>>>>>>>> ------------------- > > >>>>>>>>>>>>> Thank you, > > >>>>>>>>>>>>> > > >>>>>>>>>>>>> James Sirota > > >>>>>>>>>>>>> PMC- Apache Metron > > >>>>>>>>>>>>> jsirota AT apache DOT org > > >>>>>>>>>>>>> > > >>>>>>>>>>>>> > > >>>>>>>>>>>> > > >>>>>>>>>>> > > >>>>>>>>>> > > >>>>>>>>> > > >>>>>>>> > > >>>>>>> > > >>>>>> -- > > >>>>>> > > >>>>>> Jon Zeolla > > >>>> > > >>>> ------------------- > > >>>> Thank you, > > >>>> > > >>>> James Sirota > > >>>> PMC- Apache Metron > > >>>> jsirota AT apache DOT org > > >>>> > > >>> > > >> > > >
