Igor Bernstein created CRUNCH-600:
-------------------------------------
Summary: Credentials are lost in local runner mode
Key: CRUNCH-600
URL: https://issues.apache.org/jira/browse/CRUNCH-600
Project: Crunch
Issue Type: Bug
Reporter: Igor Bernstein
Priority: Minor
The credentials attached to a job are lost when CrunchOutputs builds a new Job
in getJob.
For example, the following scenario fails:
{code:title=MyTarget.java}
class MyTarget implements MapReduceTarget {
void configureForMapReduce(Job job, PType<?> ptype, Path outputPath, String
name) {
job.getCredentials().addSecretKey("myPassword", "secret".getBytes());
CrunchOutputs.addNamedOutput(job, name,
FormatBundle.forOutput(MyOutputFormat.class), Object.class);
}
//..
}
{code}
{code:title=MyOutputFormat.java}
class MyOutputFormat extends OutputFormat<Object,Object> {
public RecordWriter<Object, Object> getRecordWriter(TaskAttemptContext
context) throws IOException, InterruptedException {
Assert.equals("secret".getBytes(),
context.getCredentials().getSecretKey("myPassword"));
}
}
{code}
This happens because CrunchOutputs.getJob copies the baseConf using the base
Configuration class and the base Configuration class doesn't have a notion of
Credentials, so the credentials get dropped.
This is issue only manifests itself when using the local runner. In an actual
hadoop cluster the credentials will be inherited from UserGroupInformation.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
