Hey all, I started looking through the Knox SSO feature branch (see here https://issues.apache.org/jira/browse/METRON-1663). This is some great new security functionality work and it looks like it will bring some important new features to the Metron platform. I'm coming at this pretty green, so I do have some questions regarding the proposed changes from a high level architectural perspective. There are a few changes within the current FB PR's that I think could use some further explanation. At first glance, it seems we could potentially simplify this branch a great deal and get it completed much sooner if we narrowed the focus a bit. But I could certainly be wrong here and happy for other opinions. I searched through the mailing list history to see if there is any additional background and the main DISCUSS thread I could find was regarding initially setting up the feature branch, which talked about adding Knox and LDAP. https://lists.apache.org/thread.html/cac2e6314284015b487121e77abf730abbb7ebec4ace014b19093b4c@%3Cdev.metron.apache.org%3E. If I've missed any follow-up, please let me know.
Looking at the broader set of Jiras associated with 1663 and the first PR 1665, it looks like there are 4 main thrusts of this branch right now: 1. Knox/SSO 2. Node migrated to Spring Boot 3. JDBC removed completely in favor of LDAP 4. Introduction of Zuul, also microservices? I strongly urge for the purpose of reviewing this feature branch that we base much of the discussion off of https://issues.apache.org/jira/browse/METRON-1755, the architecture diagram. Minimally, an explanation of the current architecture along with discussion around the additional proposed changes and rationale would be useful for evaluation. I don't have a solid enough understanding yet of the full scope of changes and how they differ from the existing architecture just from looking at the PR's alone. 1. The first question is a general one regarding the necessity of the 3 additional features alongside Knox - migrating Node to Spring Boot, removing JDBC altogether, adding dependencies on Netflix's Zuul framework. Are these necessary for adding Knox/SSO? They seem like potentially separate features, imo. 2. It looks like LDAP will be a required component for interacting with Metron via the UI's. I see this PR https://github.com/apache/metron/pull/1186 which removes JDBC authentication. Are we ready to remove it completely or would it be better to leave it as a minimal installation option? What is the proposed migration path for existing users? Do we feel comfortable requiring that all installations, including full dev, install and configure LDAP? For comparison, in the PCAP feature branch we discussed removing the existing PCAP REST application in the initial discussion, got agreement, and later removed it in the course of working on the feature branch. The PR is fairly clear, however I think we're just missing some basic discussion around the implications, as I've outlined above. Some additional relevant discussion occurred on this PR https://github.com/apache/metron/pull/1112 which would be good to summarize for purposes of this overarching architecture discussion. 3. Migration from Node to Spring Boot. I believe this is already used by the REST application and if anything brings some cohesion to our server strategy. Strictly speaking, is there a reason this is required for Knox? It seems comparable to a component upgrade, such as moving from ES 2.x to 5.6.x and upgrading Angular 6. 4. Introduction of Netflix's Zuul. https://issues.apache.org/jira/browse/METRON-1665. - > "The UIs currently proxy to the REST API to avoid CORS issues, this will be achieved with Zuul." - Can we elaborate more on where or how CORS is a problem with our existing architecture, how Zuul will help solve that, and how it fits with Knox? Wouldn't this be handled by Knox? Since Larry McCay chimed in with interest on the original SSO thread about the FB, I'm hoping he is also willing to chime in on this as well. - This looks like it has the potential to be a rather large piece of fundamental infrastructure (as it's also pertinent to microservices) to pull into the platform, and I'd like to be sure the community is aware of and is OK with the implications. 5. > "The proposal is to use a spring boot application, allowing us to harmonize the security implementation across the UI static servers and the REST layer, and to provide a routing platform for later microservices." - https://issues.apache.org/jira/browse/METRON-1665. - Microservices is a pretty loaded term. I know there had been some discussion a while back during the PCAP feature branch start, but I don't recall ever reaching a consensus on it. More detail in this thread - https://lists.apache.org/thread.html/1db7c6fa1b0f364f8c03520db9989b4f7a446de82eb4d9786055048c@%3Cdev.metron.apache.org%3E. Can we get some clarification on what is meant by microservices in the case of this FB and relevant PR's, what that architecture looks like, and how it's achieved with the proposed changes in this PR/FB? It seems Zuul is also pertinent to this discussion, but there are many ways to skin this cat so I don't want to presume - https://blog.heroku.com/using_netflix_zuul_to_proxy_your_microservices 6. Zuul, Spring Boot, and microservices - Closely related to point 5 above. It seems that we weren't quite ready for this when it was brought up in May, or at the very least we had some concern of what direction to go. What is the operational impact, mpack impact, and how we propose to manage it with Kerberos, etc.? https://lists.apache.org/thread.html/c19904681e6a6d9ea3131be3d1a65b24447dca31b4aff588b263fd87@%3Cdev.metron.apache.org%3E There is a lot to like in this feature branch, imo. Great feature addition with Knox and SSO. Introduction of LDAP support for authentication for Metron UI's. Simplification/unification of our server hosting infrastructure. I'm hoping we can flesh out some of the details pointed out above a bit more and get this feature through. Great work so far! Best, Mike Miklavcic
