Hi Flume team and experts, I am using Thrift Sources/Sinks in Kerberos environment, found the agents only work with principle "flume". If using other valid principle it will hit ERROR of "GSS initiate failed". For some reason I can not use Flume principle, can someone give me some guide on this? Thanks.
Here's my configuration file: g1.sources.source1.type = spooldir g1.sources.source1.spoolDir = /test g1.sources.source1.fileHeader = false g1.sinks.sink1.type = thrift g1.sinks.sink1.hostname = localhost g1.sinks.sink1.port = 55555 g1.channels.channel1.type = memory g1.channels.channel1.capacity = 1000 g1.channels.channel1.transactionCapacity = 100 g1.sources.source1.channels = channel1 g1.sinks.sink1.channel = channel1 g2.sources = source2 g2.sinks = sink2 g2.channels = channel2 g2.sources.source2.type = thrift g2.sources.source2.bind = localhost g2.sources.source2.port = 55555 g2.sinks.sink2.type = hdfs g2.sinks.sink2.hdfs.path = /tmp g2.sinks.sink2.hdfs.filePrefix = thriftData g2.sinks.sink2.hdfs.writeFormat = Text g2.sinks.sink2.hdfs.fileType = DataStream g2.channels.channel2.type = memory g2.channels.channel2.capacity = 1000 g2.channels.channel2.transactionCapacity = 100 g2.sources.source2.channels = channel2 g2.sinks.sink2.channel = channel2 g1.sinks.sink1.kerberos = true g1.sinks.sink1.client-principal = flume/[email protected] g1.sinks.sink1.client-keytab = /etc/security/keytabs/flume-1563.server.keytab g1.sinks.sink1.server-principal = flume/[email protected] g2.sources.source2.kerberos = true g2.sources.source2.agent-principal = flume/[email protected] g2.sources.source2.agent-keytab = /etc/security/keytabs/flume-1563.server.keytab If using other valid principle, will hit error: g1.sinks.sink1.kerberos = true g1.sinks.sink1.client-principal = [email protected] g1.sinks.sink1.client-keytab = /home/test/test.keytab g1.sinks.sink1.server-principal = [email protected] g2.sources.source2.kerberos = true g2.sources.source2.agent-principal = [email protected] g2.sources.source2.agent-keytab = /home/test/test.keytab Agent g1: ERROR server.TThreadPoolServer: Error occurred during processing of message. java.lang.RuntimeException: org.apache.thrift.transport.TTransportException: Peer indicated failure: GSS initiate failed at org.apache.thrift.transport.TSaslServerTransport $Factory.getTransport(TSaslServerTransport.java:219) at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run (TThreadPoolServer.java:189) at java.util.concurrent.ThreadPoolExecutor.runWorker (ThreadPoolExecutor.java:1142) Agent g2: ERROR transport.TSaslTransport: SASL negotiation failure javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Server not found in Kerberos database (7) - UNKNOWN_SERVER)] at com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge (GssKrb5Client.java:211)
