Hi Rob - Thanks for your thoughts and insights into the KnoxSSO/KnoxToken signing keys and how they should align with the TLS management improvements that you are working on. These details should be captured in any related KIPs/JIRAs.
In essence, we are moving actual PKI use of keystores away from the master secret and related assumptions but continue to support them for backward compatibility. This will provide easier integration with external management and provisioning tooling - so I think it makes sense. As for the scope of the 1.3.0 release, it will no doubt be a subset of the listed categories and issues that I outlined in this thread. I wanted to capture as much as I could for the creation of KIP one-pagers that can be used to align like work across releases. We will then select the subset of work from the KIPs that can be accomplished in the timeframe and/or adjust the timeframe. Make sense? thanks, --larry On Mon, Feb 18, 2019 at 12:41 PM Robert Levas <rle...@cloudera.com.invalid> wrote: > Larry... > > Being new to the project, I do not have much to contribute related to the > changes for 1.3.0; however, this seems like a pretty large list of items. > Mid-April seems optimistic to me, but then again I am new to the team and > not sure how quickly we work. That said, I am up for the challenge. > > One thing to note on the SSO items is that Knox may need some cleanup on > how the signing key is configured. The current set of configuration > properties used to declare where to find it is lacking and assumes that the > decryption keys for the keystore and key are the master key (with a little > hack in there to customize password for the key). I think this should be > more consistent with how the custom identity and trust store locations > facility will work.. and maybe even utilize some of the work related to > syncing the master key (*Management Improvements/ Master Secret > synchronization across instances*) to also sync the signing key when > multiple instances are involved. However, this could be work that pushes > us over the mid-April target. > > Thanks... > Rob > > > On Fri, Feb 15, 2019 at 10:01 PM Phil Zampino <pzamp...@apache.org> wrote: > > > Thanks for collecting this comprehensive list of improvements. Many of > > these things have been on the “wish list” for a while now, and it would > be > > great to get them done. > > > > I’ll see if I can write up some KIP content and/or one-pagers to propose > > some details for some of this work. Then we can discuss in more detail, > and > > define some specific tasks/Jira issues. > > > > I think shooting for a release mid-April is a good goal, even if we can’t > > complete the list exhaustively. > > > > Thanks again, > > Phil > > > > On Tue, Feb 12, 2019 at 3:51 PM larry mccay <lmc...@apache.org> wrote: > > > > > All - > > > > > > I'd like to officially start the planning for the 1.3.0 release of > Apache > > > Knox. > > > > > > After looking at the list of outstanding JIRAs with fixVersion of > 1.3.0, > > > existing KIPs and considering requirements for a more containerized and > > > cloud oriented world, I have a like of general categories: > > > > > > * TLS Improvements > > > - Configurable Keystore Location and Password > > > - Configurable Truststore Location and Password > > > - Mutual Auth SSL truststores and client certs keystores > > > - Dynamic keystore/truststore loading > > > * Must keep in mind the KnoxCLI for create-cert, export-cert > > > > > > * Management Improvements > > > There have been a number of people asking about the following and they > > are > > > all encountering similar pain - whether it be from a containerization > > > context, DevOps or management tool, perspective a number of these are > > > painful today for Knox admins. > > > - Eliminate needs to have access to the Knox machines > > > - Bootstrap config for log locations, pids, env variable overrides, > > etc > > > - Remote access to public certs (for SSL and for various signature > > > verification(knoxtoken, knoxsso, etc)) > > > - UI for Alias Management > > > - Surface logs in UI(maybe?) and API > > > - Master Secret synchronization across instances > > > - Service Discovery from new source/s > > > - new Remote Config Monitor source/s > > > - new Remote Alias Service for Vault > > > > > > * SSO > > > Easing the configuration required to enable all of the participating > > > applications for KnoxSSO across a deployment will be important in > general > > > but being able to more easily provision this for cloud deployments will > > be > > > key. > > > - IDP Initiated Flow with Landing Page (Okta portal page) > > > - challenge - particpating apps are configured for a single IDP > > > - Landing Page is like Okta - with links to UIs - how do we > deal > > > with multiple topologies > > > - KnoxSSOut - logout from landing page > > > - Keycloak? > > > - Add a standard integration pattern for SPs to register with > > KnoxSSO - > > > like SAML? > > > - Azure AD with OAuth/OpenID Connect > > > - Internal hostnames vs external for cookie domain > > > - IDP metadata file for SPs to pull KnoxSSO public cert and any > other > > > details from an endpoint > > > > > > * Performance > > > Performance is an ongoing item but deployment scenarios for cloud and > > > hybrid may introduce new strategies for things like load balancing in > > front > > > of and behind the Knox gateway. > > > - LoadBalancing from dispatch > > > - HA provider config? > > > - service specific > > > - Hive cookie - sticky sessions > > > - Knox HA story requires sticky sessions > > > - TLS Options > > > - Sizing Guidelines > > > - JVM tuning > > > - number of instances based on anticipated load (services need > to > > > be taken into account) > > > > > > * KnoxShell > > > As deployments move more and more to the cloud and/or hybrid > > combinations, > > > the need to SSH to a gateway machine becomes more and more awkward. If > we > > > can provide a powerful enough scripting experience for Data Worker/ETL > > type > > > roles, we will be able to limit the need to SSH. > > > - Remote CLI classes > > > - ETL Usecase Classes > > > - Livy classes > > > > > > * KnoxCLI > > > - return error codes rather than throwing exception and always > > > returning zero > > > > > > Many of our 1.3.0 JIRAs fall into one or more of the above categories. > > > There are also quite a few of dispatch related issues filed. > > > > > > I realize that is a rather large list and a mixed back of category and > > > detail in places. > > > > > > I think that we need to write up a couple KIPs to encompass the work > > above > > > that we would like to tackle in 1.3.0 and then identify/file JIRAs to > > > represent the work. > > > We will need to carefully manage scope in order to get this done but > > > articulating this roadmap with some new KIP pages will help 1.3.0 and > > > future releases. > > > Thoughts? > > > > > > As for timeline, I am thinking that mid-April may feel appropriate. > > > Thoughts? > > > > > > thanks, > > > > > > --larry > > > > > >
