[ https://issues.apache.org/jira/browse/MAPREDUCE-6638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15498224#comment-15498224 ]
Haibo Chen commented on MAPREDUCE-6638: --------------------------------------- Thanks for your comments, Karthik! Uploading a new patch to address your suggestions, which does make the logging much more user-friendly. bq. One could store the encrypted key in KMS. Once stored, we could do one of the following: 1) Tasks have a way to fetch this key directly 2) Leave the tasks as is, but augment the AM to recover this key as part of recovery. I think option 2 is advantageous in that the change needed is minimal and only one query for each job attempt to the safe store is needed instead of one for each task. Will create a follow up jira once this is done. > Do not attempt to recover jobs if encrypted spill is enabled > ------------------------------------------------------------ > > Key: MAPREDUCE-6638 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6638 > Project: Hadoop Map/Reduce > Issue Type: Improvement > Components: applicationmaster > Affects Versions: 2.7.2 > Reporter: Karthik Kambatla > Assignee: Haibo Chen > Attachments: mapreduce6638.001.patch, mapreduce6638.002.patch, > mapreduce6638.003.patch > > > Post the fix to CVE-2015-1776, jobs with ecrypted spills enabled cannot be > recovered if the AM fails. We should store the key some place safe so they > can actually be recovered. If there is no "safe" place, at least we should > restart the job by re-running all mappers/reducers. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org