Hi,
I am running Flink-1.3.2 on yarn (Cloudera 2.6.0-cdh5.7.6). The application
stream data from kafka, groups by key, creates a session window and writes
to HDFS using a rich window function in the "window.apply" method.
The rich window function creates the sequence file thus
SequenceFile.createWriter(
conf,
new Option[] {
Writer.file(new Path("flink-output/" + filePath)),
Writer.compression(CompressionType.BLOCK,
new DefaultCodec()),
Writer.keyClass(BytesWritable.class),
Writer.valueClass(BytesWritable.class) })
The "conf" is created in the "open" method thus
conf = HadoopFileSystem.getHadoopConfiguration();
for (Map.Entry<String, String> entry :
parameters.toMap().entrySet()) {
conf.set(entry.getKey(), entry.getValue());
}
where parameters is the flink.configuration.Configuration object that is an
argument to the open method
The applications runs for about 10 hours before it fails with kerberos
error "Caused by: javax.security.sasl.SaslException: GSS initiate failed
[Caused by GSSException: No valid credentials provided (Mechanism level:
Failed to find any Kerberos tgt)]"
The flink-conf.yaml has the following properties set.
security.kerberos.login.keytab: <keytab location>
security.kerberos.login.principal:<principal>
security.kerberos.login.contexts: Client,KafkaClien
Any help would be appreciated.
Thanks,
Prabhu