I was checking against HDP 2.5.3 and Hadoop 2.7. I haven't checked against anything later yet. Good to know that Yarn UI v2 should have this fixed.
Yea security in the Hadoop/Spark/etc space is definitely pushing a boulder up a hill. The number of times I've heard "why can't I copy/paste the example from the internet" is getting to be old. Trying to make it seamless but there are a lot of insecure practices out there :( Kevin Risden On Wed, May 9, 2018 at 9:41 AM, larry mccay <[email protected]> wrote: > This should not be the case for the new Yarn UI v2. > But it is an ongoing issue that is starting to feel like pushing a boulder > up hill. > > I've considered adding a trusted proxy module to Knox that folks could add > as a dependency. > Problem is, every time that I start doing it - it ends up being what is > already available in hadoop. > > > On Wed, May 9, 2018 at 10:36 AM, Kevin Risden <[email protected]> wrote: > >> Thanks Larry. This also seems to affect YARN as well when looking at the >> YARN UI support in Apache Knox. Same sort of issue: "Logged in as: knox". >> >> Kevin Risden >> >> On Thu, May 3, 2018 at 10:10 AM, larry mccay <[email protected]> wrote: >> >>> This can only be addressed in Hadoop, AFAICT. >>> There are so many UIs and even APIs not supporting trusted proxies and >>> it is really becoming a problem. >>> >>> We need to file JIRAs where this support is missing and potentially >>> provide patches as it seems folks are reluctant to add proper support for >>> it anymore. >>> >>> On Wed, May 2, 2018 at 2:56 PM, Kevin Risden <[email protected]> wrote: >>> >>>> There has been quite a bit of work done around enabling Knox and the >>>> MapReduce Job History server UI. However, I think we are going to run into >>>> a fundamental problem with the MapReduce Job History server and >>>> impersonation/doAs. >>>> >>>> Based on our research, we found that the MR Job History server doesn't >>>> support the "doAs" pattern that is used by Knox and Hue. >>>> >>>> If I navigate to a topology /jobhistory/ in the top right hand corner >>>> is: "Logged in as: knox". Knox sends a doAs parameter (just like Hue) >>>> and it seems like MR Job History server is ignoring this. Ideally the >>>> "Logged in as:" would be the user who authenticated to Knox and not the >>>> user Knox is running as (according to Kerberos). >>>> >>>> Right now this causes problems for us because the Knox user doesn't >>>> have access to view the job logs. If I navigate to an attempt log it looks >>>> to proxy correctly but end up with this error message: >>>> >>>> User [knox] is not authorized to view the logs for >>>> attempt_1525122616004_0164_m_000000_0 in log file >>>> >>>> The same type of error happens in Hue when trying to look at job logs. >>>> >>>> >>>> User [hue] is not authorized to view the logs for >>>> job_1521053483563_0133 in log file >>>> >>>> >>>> It looks like this might have been previously reported as KNOX-747 but >>>> was closed as can't reproduce. >>>> >>>> I don't think this is a problem with Knox but bringing it up here in >>>> case there are any ideas on workarounds. This might have to be fixed >>>> upstream in the Hadoop project. >>>> >>>> Any ideas on how to move forward with Knox and MapReduce Job History >>>> server? >>>> >>>> Kevin Risden >>>> >>> >>> >> >
