Great, thanks Larry ! On Fri, Jul 20, 2018 at 10:53 AM larry mccay <lmc...@apache.org> wrote:
> I just pushed KNOX-1394 for changing the default whitelist to align with > the DEMO LDAP config in gateway-site.xml. > This will realign with the dev and demo environment assumptions that Knox > has always had OOTB. > > I will cut an RC shortly. > > > On Fri, Jul 20, 2018 at 12:40 AM, larry mccay <lmc...@apache.org> wrote: > > > Yes, along with ERROR level logging when falling back to specific host/ip > > or localhost variants. > > > > On Thu, Jul 19, 2018, 11:48 PM Philip Zampino <pzamp...@gmail.com> > wrote: > > > >> Sure, but I'm also going to implement the following for the default > >> behavior (when the DEFAULT value is specified for the whitelist > property): > >> > >> > >> 1. Attempt to determine the domain from the X-Forwarded-Host header > >> value > >> 2. If domain could not be determined, attempt to determine the domain > >> from the InetAddress.getLocalHost().getCanonicalHostName() value > >> 3. If domain could not be determined, attempt to determine the domain > >> from the requested host name > >> 4. If the domain could be determined from any of these sources, then > >> the > >> default whitelist will be based on that domain > >> 5. If the domain cannot be determined > >> a. If the requested host name is NOT a variant of localhost, then the > >> whitelist will be restricted to that specific host name > >> b. Otherwise, the localhost whitelist will be the default > >> > >> Does that sound like a good strategy? > >> > >> > >> On Thu, Jul 19, 2018 at 8:15 PM larry mccay <lmc...@apache.org> wrote: > >> > >> > Yes, does that sound appropriate to you? > >> > If the LDAP config in gateway-site.xml gets updated to product the > >> > whitelist would be in the same place. > >> > > >> > On Thu, Jul 19, 2018 at 6:26 PM, Philip Zampino <pzamp...@gmail.com> > >> > wrote: > >> > > >> > > I am working on a solution for the ip address being treated as a > >> hostname > >> > > issue. > >> > > > >> > > On Thu, Jul 19, 2018 at 6:24 PM larry mccay <lmc...@apache.org> > >> wrote: > >> > > > >> > > > Playing around a bit more, I noticed that there is > nondeterministic > >> > > > behavior of the default whitelist feature. > >> > > > Especially on macs - since the hostname ends up being any number > of > >> > > things. > >> > > > I have noticed the following things when there is no explicit > >> whitelist > >> > > > configured: > >> > > > > >> > > > * ip address based whitelist being derived which is treated like a > >> > domain > >> > > > * localhost is not supported out of the box unless the logic is > >> unable > >> > to > >> > > > determine a domain > >> > > > * sometimes my host is HW14155.home and sometimes it is > >> new-host-5.home > >> > > for > >> > > > some reason > >> > > > > >> > > > Given that all of our samples and docs assume localhost and OOTB > we > >> are > >> > > > setup for DEMO LDAP server, I propose that we at least add > localhost > >> > back > >> > > > for OOTB. > >> > > > Ip address handling may be worth tackling as well but only if we > >> can do > >> > > it > >> > > > in a day. > >> > > > > >> > > > Thoughts? > >> > > > > >> > > > > >> > > > On Thu, Jul 19, 2018 at 6:12 PM, larry mccay <lmc...@apache.org> > >> > wrote: > >> > > > > >> > > > > Awesome - just checked it out and I will kick off a new build > >> > shortly! > >> > > > > > >> > > > > On Thu, Jul 19, 2018 at 6:01 PM, Sandeep Moré < > >> moresand...@gmail.com > >> > > > >> > > > > wrote: > >> > > > > > >> > > > >> Hello Larry, > >> > > > >> > >> > > > >> I committed the fix to master and v1.1.0, it is under the JIRA > >> > > KNOX-1391 > >> > > > >> <https://issues.apache.org/jira/browse/KNOX-1391>. > >> > > > >> we should be good to to cut the RC, provided there are no more > >> > issues > >> > > ! > >> > > > >> > >> > > > >> Thanks ! > >> > > > >> Sandeep > >> > > > >> > >> > > > >> On Thu, Jul 19, 2018 at 4:25 PM larry mccay < > >> larry.mc...@gmail.com> > >> > > > >> wrote: > >> > > > >> > >> > > > >> > Awesome, @sandeep! > >> > > > >> > I'll keep an eye out. > >> > > > >> > > >> > > > >> > Once that lands, you can bump this thread and I'll cut the > RC. > >> > > > >> > Obviously, we will need it in both master and v1.1.0 > branches. > >> > > > >> > > >> > > > >> > On Thu, Jul 19, 2018 at 4:19 PM, Sandeep Moré < > >> > > moresand...@gmail.com> > >> > > > >> > wrote: > >> > > > >> > > >> > > > >> > > Hello Larry, > >> > > > >> > > > >> > > > >> > > Yes, I have seen those exceptions, they seem to be > happening > >> > > fairly > >> > > > >> > > consistently and only for KnoxSSO redirects when trying to > >> > access > >> > > > >> admin > >> > > > >> > UI, > >> > > > >> > > I am taking a look at them as we speak, will open up a JIRA > >> for > >> > it > >> > > > as > >> > > > >> > well. > >> > > > >> > > It would be good if we can get it in, I will try to get the > >> fix > >> > > out > >> > > > as > >> > > > >> > soon > >> > > > >> > > as I can. > >> > > > >> > > > >> > > > >> > > Best, > >> > > > >> > > Sandeep > >> > > > >> > > > >> > > > >> > > On Thu, Jul 19, 2018 at 4:15 PM larry mccay < > >> lmc...@apache.org> > >> > > > >> wrote: > >> > > > >> > > > >> > > > >> > > > @Phil, I see a couple commits land that seem to address > the > >> > NPE. > >> > > > >> > > > Is that correct? > >> > > > >> > > > > >> > > > >> > > > I have also seen an IllegalStateException during redirect > >> from > >> > > > >> Admin UI > >> > > > >> > > to > >> > > > >> > > > KnoxSSO. > >> > > > >> > > > Has anyone seen this and/or is working on it - is it > >> related > >> > to > >> > > > the > >> > > > >> > NPE? > >> > > > >> > > > I don't think it is since I see it more frequently and > not > >> > > always > >> > > > >> with > >> > > > >> > > the > >> > > > >> > > > NPEs. > >> > > > >> > > > > >> > > > >> > > > I'd like to get a new RC cut by end of the week, if > >> possible. > >> > > > >> > > > > >> > > > >> > > > On Fri, Jul 13, 2018 at 7:57 PM, larry mccay < > >> > lmc...@apache.org > >> > > > > >> > > > >> > wrote: > >> > > > >> > > > > >> > > > >> > > > > Agreed, Phil. > >> > > > >> > > > > I have cut an RC but we need to address this first. > I'll > >> > hold > >> > > > >> off on > >> > > > >> > > > > announcing it. > >> > > > >> > > > > > >> > > > >> > > > > On Fri, Jul 13, 2018, 11:36 AM Phil Zampino < > >> > > > pzamp...@apache.org> > >> > > > >> > > wrote: > >> > > > >> > > > > > >> > > > >> > > > >> During some testing of the proposed 1.1.0 code, I've > >> > > discovered > >> > > > >> some > >> > > > >> > > > NPEs > >> > > > >> > > > >> in filters (e.g., AclsAuthorizationFilter, > >> > > > >> > HadoopGroupProviderFilter), > >> > > > >> > > > >> which are concerning. > >> > > > >> > > > >> > >> > > > >> > > > >> I've committed a change to address the > >> > > AclsAuthorizationFilter, > >> > > > >> but > >> > > > >> > > > seeing > >> > > > >> > > > >> similar behavior for the HadoopGroupProviderFilter has > >> > > > increased > >> > > > >> my > >> > > > >> > > > >> concern > >> > > > >> > > > >> that there may be a more fundamental problem. > >> > > > >> > > > >> In both cases, it seems that the filters are being > >> invoked > >> > > > prior > >> > > > >> to > >> > > > >> > > (or > >> > > > >> > > > >> during) their respective init() methods have been > >> invoked. > >> > > > Thus, > >> > > > >> > > members > >> > > > >> > > > >> which should be initialized in the init() method are > not > >> > yet > >> > > > >> > > > initialized. > >> > > > >> > > > >> > >> > > > >> > > > >> This can be consistently reproduced, though it is a > bit > >> of > >> > a > >> > > > >> pain: > >> > > > >> > > > >> > >> > > > >> > > > >> - Install Knox (‘ant install-test-home’, or just > >> unzip > >> > > > >> > > > knox-1.1.0.zip) > >> > > > >> > > > >> - Start the gateway > >> > > > >> > > > >> - Access the Admin UI > >> > > > >> > > > >> > >> > > > >> > > > >> > >> > > > >> > > > >> Note that the latest 1.1.0 source has a *fix* for the > >> > > > >> > > > >> AclsAuthorizationFilter NPE, but master does not yet > >> have > >> > > this > >> > > > >> > change. > >> > > > >> > > > >> This > >> > > > >> > > > >> is important because that change effectively hides the > >> > issue. > >> > > > >> > > > >> > >> > > > >> > > > >> I think we should determine what's happening with this > >> > before > >> > > > >> > > > >> producing/testing a release candidate. > >> > > > >> > > > >> > >> > > > >> > > > >> > >> > > > >> > > > >> > >> > > > >> > > > >> > >> > > > >> > > > >> On Sat, Feb 24, 2018 at 12:57 PM larry mccay < > >> > > > lmc...@apache.org> > >> > > > >> > > wrote: > >> > > > >> > > > >> > >> > > > >> > > > >> > All - > >> > > > >> > > > >> > > >> > > > >> > > > >> > Sorry for the delay on this topic. > >> > > > >> > > > >> > > >> > > > >> > > > >> > We are going to start of this planning thread with > ~85 > >> > > > >> Unresolved > >> > > > >> > > > JIRAs > >> > > > >> > > > >> in > >> > > > >> > > > >> > either 1.1.0 or 0.15.0 fixVersion. > >> > > > >> > > > >> > > >> > > > >> > > > >> > project = KNOX AND resolution = Unresolved AND > >> fixVersion > >> > > in > >> > > > >> > (1.1.0, > >> > > > >> > > > >> > 0.15.0) ORDER BY priority DESC, updated DESC > >> > > > >> > > > >> > > >> > > > >> > > > >> > I will spend some time migrating all 0.15.0 to 1.1.0 > >> to > >> > > begin > >> > > > >> with > >> > > > >> > > and > >> > > > >> > > > >> then > >> > > > >> > > > >> > we will need to go through and see what is already > >> taken > >> > > care > >> > > > >> of > >> > > > >> > or > >> > > > >> > > > can > >> > > > >> > > > >> > wait for a 1.2.0 or later. > >> > > > >> > > > >> > > >> > > > >> > > > >> > I also have a couple KIPs in mind to target larger > >> > > > >> features/themes > >> > > > >> > > for > >> > > > >> > > > >> this > >> > > > >> > > > >> > release. > >> > > > >> > > > >> > > >> > > > >> > > > >> > Off the top of my head: > >> > > > >> > > > >> > > >> > > > >> > > > >> > * I think we need to address some cloud specific > >> usecases > >> > > and > >> > > > >> plan > >> > > > >> > > to > >> > > > >> > > > >> > provide a KIP for that. Hybrid cloud/federated knox > >> > > > instances, > >> > > > >> > Azure > >> > > > >> > > > AD > >> > > > >> > > > >> > integration, ID mapping from Hadoop user to IAM > >> > > users/roles, > >> > > > >> etc. > >> > > > >> > > > >> Perhaps > >> > > > >> > > > >> > some CASB-like features if they make sense. > >> > > > >> > > > >> > > >> > > > >> > > > >> > * I also think we need one for articulating a > >> reasonable > >> > > flow > >> > > > >> for > >> > > > >> > > > >> Logout in > >> > > > >> > > > >> > KnoxSSO. There are a lot of little nuances to logout > >> > across > >> > > > >> > multiple > >> > > > >> > > > >> apps > >> > > > >> > > > >> > and between different IDPs. This will require some > >> > > > discussion. > >> > > > >> > > > >> > > >> > > > >> > > > >> > * Another thing that has been tugging at my interest > >> has > >> > > been > >> > > > >> the > >> > > > >> > > fact > >> > > > >> > > > >> that > >> > > > >> > > > >> > we may be able provide some common libraries to help > >> > > > ecosystem > >> > > > >> > > > >> applications > >> > > > >> > > > >> > uptake the trusted proxy pattern and KnoxSSO. > >> > > > >> > > > >> > > >> > > > >> > > > >> > Anyway, these are my initial thoughts, please feel > >> free > >> > to > >> > > > >> raise > >> > > > >> > > > >> additional > >> > > > >> > > > >> > ideas/themes for KIPs, etc. > >> > > > >> > > > >> > > >> > > > >> > > > >> > I was thinking that we could try and target an end > of > >> > March > >> > > > or > >> > > > >> Mid > >> > > > >> > > > April > >> > > > >> > > > >> > 1.1.0 release. > >> > > > >> > > > >> > > >> > > > >> > > > >> > Thoughts? > >> > > > >> > > > >> > > >> > > > >> > > > >> > --larry > >> > > > >> > > > >> > > >> > > > >> > > > >> > >> > > > >> > > > > > >> > > > >> > > > > >> > > > >> > > > >> > > > >> > > >> > > > >> > >> > > > > > >> > > > > > >> > > > > >> > > > >> > > >> > > >
