[jira] [Commented] (MAPREDUCE-6741) add JHS support to hide job conf properties from Web UI
[ https://issues.apache.org/jira/browse/MAPREDUCE-6741?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15392159#comment-15392159 ] Jason Lowe commented on MAPREDUCE-6741: --- The job.xml is necessary to get the job going -- it's the base conf to be used by the AM and all tasks. The other conf file is only created for historical record. We could probably just copy job.xml, but that would be more expensive in practice than just having the AM write out its own conf. Note that the two files aren't lying around at the same time very long. The job.xml file is in the staging directory which is deleted shortly after the copy is made for the JHS. > add JHS support to hide job conf properties from Web UI > --- > > Key: MAPREDUCE-6741 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6741 > Project: Hadoop Map/Reduce > Issue Type: Improvement > Components: mrv2 >Affects Versions: 2.7.2 >Reporter: Haibo Chen >Assignee: Haibo Chen > > JHS today displays all Job conf properties in Web UI directly. Users may have > some credentials or any sensitive information they added to the job conf but > do not want to be shown in Web UI. It'd be nice if we can allow users to > specify a set of properties which JHS will filter out when Job conf is > displayed. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Commented] (MAPREDUCE-6741) add JHS support to hide job conf properties from Web UI
[
https://issues.apache.org/jira/browse/MAPREDUCE-6741?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15390092#comment-15390092
]
Haibo Chen commented on MAPREDUCE-6741:
---
Good point! I have not thought about updating the jobconf file in HDFS. A
question that may or may not be relevant to this. I notice that for every job
there are always two job conf files. One file, named job.xml, is submitted by
the client and localized as task container resource. The other, named
{jobid}_conf.xml, is created by JobHistoryEventHandler and loaded by JHS. Any
particular reason why we have two files lying around for one job?
> add JHS support to hide job conf properties from Web UI
> ---
>
> Key: MAPREDUCE-6741
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6741
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
>Affects Versions: 2.7.2
>Reporter: Haibo Chen
>Assignee: Haibo Chen
>
> JHS today displays all Job conf properties in Web UI directly. Users may have
> some credentials or any sensitive information they added to the job conf but
> do not want to be shown in Web UI. It'd be nice if we can allow users to
> specify a set of properties which JHS will filter out when Job conf is
> displayed.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Commented] (MAPREDUCE-6741) add JHS support to hide job conf properties from Web UI
[ https://issues.apache.org/jira/browse/MAPREDUCE-6741?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15389744#comment-15389744 ] Jason Lowe commented on MAPREDUCE-6741: --- If that's the case, would it make more sense to have the AM redact the values when it writes out the jobconf that will be picked up by the JHS? Then even admins can't see it once the job is completed because there should be no more copies lying around with the original values, even in HDFS. > add JHS support to hide job conf properties from Web UI > --- > > Key: MAPREDUCE-6741 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6741 > Project: Hadoop Map/Reduce > Issue Type: Improvement > Components: mrv2 >Affects Versions: 2.7.2 >Reporter: Haibo Chen >Assignee: Haibo Chen > > JHS today displays all Job conf properties in Web UI directly. Users may have > some credentials or any sensitive information they added to the job conf but > do not want to be shown in Web UI. It'd be nice if we can allow users to > specify a set of properties which JHS will filter out when Job conf is > displayed. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Commented] (MAPREDUCE-6741) add JHS support to hide job conf properties from Web UI
[ https://issues.apache.org/jira/browse/MAPREDUCE-6741?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15389732#comment-15389732 ] Haibo Chen commented on MAPREDUCE-6741: --- I am thinking of redacting the values in the UI for all users to make it simple. This essentially comes from requests from customers who have legacy MR jobs that used S3/Azure as input Instead of using CredentialProvider, they put their credentials directly in JobConf, which is then displayed directly JHS UI. Some of them are aware of job ACL, but also want to disallow queue admins or whoever are always given access to all jobs by JHS, to easily access their credentials. Redacting the credentials in UI doesn't really solve the security problem. The credentials are still stored in jobconf in HDFS after all. But it does make the credentials less exposed. > add JHS support to hide job conf properties from Web UI > --- > > Key: MAPREDUCE-6741 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6741 > Project: Hadoop Map/Reduce > Issue Type: Improvement > Components: mrv2 >Affects Versions: 2.7.2 >Reporter: Haibo Chen >Assignee: Haibo Chen > > JHS today displays all Job conf properties in Web UI directly. Users may have > some credentials or any sensitive information they added to the job conf but > do not want to be shown in Web UI. It'd be nice if we can allow users to > specify a set of properties which JHS will filter out when Job conf is > displayed. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Commented] (MAPREDUCE-6741) add JHS support to hide job conf properties from Web UI
[ https://issues.apache.org/jira/browse/MAPREDUCE-6741?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15389627#comment-15389627 ] Jason Lowe commented on MAPREDUCE-6741: --- Another requirements clarification: when the properties are to be hidden, will they be completely removed from the output or will the keys appear with redacted values? > add JHS support to hide job conf properties from Web UI > --- > > Key: MAPREDUCE-6741 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6741 > Project: Hadoop Map/Reduce > Issue Type: Improvement > Components: mrv2 >Affects Versions: 2.7.2 >Reporter: Haibo Chen >Assignee: Haibo Chen > > JHS today displays all Job conf properties in Web UI directly. Users may have > some credentials or any sensitive information they added to the job conf but > do not want to be shown in Web UI. It'd be nice if we can allow users to > specify a set of properties which JHS will filter out when Job conf is > displayed. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Commented] (MAPREDUCE-6741) add JHS support to hide job conf properties from Web UI
[ https://issues.apache.org/jira/browse/MAPREDUCE-6741?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15389626#comment-15389626 ] Jason Lowe commented on MAPREDUCE-6741: --- Will this filtering be for all users or will it still show the properties for the owner of the job? > add JHS support to hide job conf properties from Web UI > --- > > Key: MAPREDUCE-6741 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6741 > Project: Hadoop Map/Reduce > Issue Type: Improvement > Components: mrv2 >Affects Versions: 2.7.2 >Reporter: Haibo Chen >Assignee: Haibo Chen > > JHS today displays all Job conf properties in Web UI directly. Users may have > some credentials or any sensitive information they added to the job conf but > do not want to be shown in Web UI. It'd be nice if we can allow users to > specify a set of properties which JHS will filter out when Job conf is > displayed. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Commented] (MAPREDUCE-6741) add JHS support to hide job conf properties from Web UI
[ https://issues.apache.org/jira/browse/MAPREDUCE-6741?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15388543#comment-15388543 ] Haibo Chen commented on MAPREDUCE-6741: --- Thanks [~jlowe] for pointing out my false statement above. Will update the jira description. > add JHS support to hide job conf properties from Web UI > --- > > Key: MAPREDUCE-6741 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6741 > Project: Hadoop Map/Reduce > Issue Type: Improvement > Components: mrv2 >Affects Versions: 2.7.2 >Reporter: Haibo Chen >Assignee: Haibo Chen > > JHS today display all Job conf properties in Web UI directly. Anyone who has > access to JHS web UI essentially has access to Job Conf of any other users' > job. Users may have some credentials or any sensitive information they added > to the job conf. It'd be nice if we can allow users to specify a set of > properties which JHS will filter out when Job conf is displayed. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Commented] (MAPREDUCE-6741) add JHS support to hide job conf properties from Web UI
[ https://issues.apache.org/jira/browse/MAPREDUCE-6741?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15388293#comment-15388293 ] Jason Lowe commented on MAPREDUCE-6741: --- bq. Anyone who has access to JHS web UI essentially has access to Job Conf of any other users' job. This is not true. The job's setting for mapreduce.job.acl-view-job is respected by the history server, and users who do not have view access will not be able to see the job's configuration even though they have access to the JHS web UI. Having the ability to filter individual conf keys is one thing, but they should not be completely wide-open today unless the user didn't bother to restrict the visibility of their job. > add JHS support to hide job conf properties from Web UI > --- > > Key: MAPREDUCE-6741 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6741 > Project: Hadoop Map/Reduce > Issue Type: Improvement > Components: mrv2 >Affects Versions: 2.7.2 >Reporter: Haibo Chen >Assignee: Haibo Chen > > JHS today display all Job conf properties in Web UI directly. Anyone who has > access to JHS web UI essentially has access to Job Conf of any other users' > job. Users may have some credentials or any sensitive information they added > to the job conf. It'd be nice if we can allow users to specify a set of > properties which JHS will filter out when Job conf is displayed. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
