[
https://issues.apache.org/jira/browse/LIVY-595?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16837210#comment-16837210
]
yanchao commented on LIVY-595:
------------------------------
*livy log:*
19/05/10 17:15:48 RPC-Handler-4 INFO RpcServer: server challenge payload byte
is : [5, 4, 0, -1, 0, 12, 0, 0, 0, 0, 0, 0, 14, 12, 88, 110, 4, 1, 0, 0, 91,
-60, -82, 68, 104, 45, -96, -54, 76, -95, 0, 57].
Krb5Context.unwrap: token=[05 04 00 ff 00 0c 00 00 00 00 00 00 0e 0c 58 6e 04
01 00 00 5b c4 ae 44 68 2d a0 ca 4c a1 00 39 ]
19/05/10 17:15:48 Thread-123 INFO LineBufferedStream: stdout: 2019-05-10
17:15:48,059 | INFO | RPC-Handler-3 | yc add : SASL confidentiality enabled,
and class is org.apache.livy.rsc.rpc.Rpc$SaslClientHandler |
org.apache.livy.rsc.rpc.SaslHandler.channelRead0(SaslHandler.java:90)
19/05/10 17:15:48 Thread-123 INFO LineBufferedStream: stdout: 2019-05-10
17:15:48,059 | INFO | RPC-Handler-3 | yc add onComplete |
org.apache.livy.rsc.rpc.SaslHandler.channelRead0(SaslHandler.java:95)
Krb5Context.unwrap: data=[04 01 00 00 ]
19/05/10 17:15:48 RPC-Handler-4 INFO RpcServer: AuthorizeCallback set true
19/05/10 17:15:48 RPC-Handler-4 INFO{color:#FF0000} RpcServer: after server
evaluate response byte is : null.{color}
19/05/10 17:15:48 RPC-Handler-4 INFO RpcServer$SaslServerHandler: Sending SASL
challenge response clientId is null, payload is null.
19/05/10 17:15:48 RPC-Handler-4 INFO KryoMessageCodec: {color:#FF0000}Encoded
message of type org.apache.livy.rsc.rpc.Rpc$SaslMessage (4 bytes){color}
19/05/10 17:15:48 RPC-Handler-4 INFO KryoMessageCodec: Encoded ByteBuf class
io.netty.buffer.UnpooledUnsafeNoCleanerDirectByteBuf
19/05/10 17:15:48 RPC-Handler-4 DEBUG Rpc: [id: 0x36c6e919,
L:/192.168.100.25:10000 - R:/192.168.100.25:59218] WRITE: 8B
+-------------------------------------------------+
| 0 1 2 3 4 5 6 7 8 9 a b c d e f |
+--------+-------------------------------------------------+----------------+
|00000000| 00 00 00 04 14 01 00 00 |........ |
+--------+-------------------------------------------------+----------------+
19/05/10 17:15:48 RPC-Handler-4 DEBUG Rpc: [id: 0x36c6e919,
L:/192.168.100.25:10000 - R:/192.168.100.25:59218] FLUSH
19/05/10 17:15:48 RPC-Handler-4 INFO RpcServer$SaslServerHandler: ended
writeAndFlush!
19/05/10 17:15:48 RPC-Handler-4 INFO RpcServer: server isComplete true
19/05/10 17:15:48 RPC-Handler-4 INFO RpcServer$SaslServerHandler: yc add : SASL
confidentiality enabled, and class is
org.apache.livy.rsc.rpc.RpcServer$SaslServerHandler
19/05/10 17:15:48 RPC-Handler-4 INFO RpcServer$SaslServerHandler: yc add
onComplete
19/05/10 17:15:48 RPC-Handler-4 INFO RpcServer: onComplete.
*driver log:*
19/05/10 17:15:48 Thread-123 INFO LineBufferedStream: stdout: 2019-05-10
17:15:48,062 | DEBUG | RPC-Handler-3 | [id: 0xc76a1550, L:/192.168.100.25:59218
- R:/192.168.100.25:10000] RECEIVED: 8B
19/05/10 17:15:48 Thread-123 INFO LineBufferedStream: stdout:
+-------------------------------------------------+
19/05/10 17:15:48 Thread-122 INFO LineBufferedStream: stdout:
Krb5Context.unwrap: token=[14 01 00 00 ]
19/05/10 17:15:48 Thread-123 INFO LineBufferedStream: stdout: | 0 1 2 3 4 5 6 7
8 9 a b c d e f |
19/05/10 17:15:48 Thread-123 INFO LineBufferedStream: stdout:
+--------+-------------------------------------------------+----------------+
19/05/10 17:15:48 Thread-123 INFO LineBufferedStream: stdout: |00000000| 00 00
00 04 14 01 00 00 |........ |
19/05/10 17:15:48 Thread-123 INFO LineBufferedStream: stdout:
+--------+-------------------------------------------------+----------------+ |
io.netty.util.internal.logging.Slf4JLogger.debug(Slf4JLogger.java:71)
{color:#FF0000}19/05/10 17:15:48 Thread-123 INFO LineBufferedStream: stdout:
2019-05-10 17:15:48,062 | INFO | RPC-Handler-3 | unwrap data is [20, 1, 0, 0],
offset is 0, len is 4. |
org.apache.livy.rsc.rpc.Rpc$SaslClientHandler.unwrap(Rpc.java:480){color}
19/05/10 17:15:48 Thread-123 INFO LineBufferedStream: stdout: 2019-05-10
17:15:48,064 | INFO | RPC-Handler-3 | [ReplDriver] Caught exception in channel
pipeline. |
org.apache.livy.rsc.rpc.RpcDispatcher.exceptionCaught(RpcDispatcher.java:177)
19/05/10 17:15:48 Thread-123 INFO LineBufferedStream: stdout:
io.netty.handler.codec.DecoderException:
javax.security.sasl.SaslException:{color:#FF0000} Problems unwrapping SASL
buffer [Caused by GSSException: Defective token detected (Mechanism level: Wrap
Token (new format):Cannot read all 12 bytes needed to form this token!)]{color}
19/05/10 17:15:48 Thread-123 INFO LineBufferedStream: stdout: at
io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:442)
19/05/10 17:15:48 Thread-123 INFO LineBufferedStream: stdout: at
io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:248)
19/05/10 17:15:48 Thread-123 INFO LineBufferedStream: stdout: at
io.netty.handler.codec.ByteToMessageCodec.channelRead(ByteToMessageCodec.java:103)
19/05/10 17:15:48 Thread-123 INFO LineBufferedStream: stdout: at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:357)
19/05/10 17:15:48 Thread-123 INFO LineBufferedStream: stdout: at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:343)
19/05/10 17:15:48 Thread-123 INFO LineBufferedStream: stdout: at
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:336)
19/05/10 17:15:48 Thread-123 INFO LineBufferedStream: stdout: at
io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:240)
19/05/10 17:15:48 Thread-123 INFO LineBufferedStream: stdout: at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:357)
19/05/10 17:15:48 Thread-123 INFO LineBufferedStream: stdout: at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:343)
19/05/10 17:15:48 Thread-123 INFO LineBufferedStream: stdout: at
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:336)
19/05/10 17:15:48 Thread-123 INFO LineBufferedStream: stdout: at
io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:86)
19/05/10 17:15:48 Thread-123 INFO LineBufferedStream: stdout: at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:357)
19/05/10 17:15:48 Thread-123 INFO LineBufferedStream: stdout: at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:343)
19/05/10 17:15:48 Thread-123 INFO LineBufferedStream: stdout: at
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:336)
19/05/10 17:15:48 Thread-123 INFO LineBufferedStream: stdout: at
io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1294)
19/05/10 17:15:48 Thread-123 INFO LineBufferedStream: stdout: at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:357)
19/05/10 17:15:48 Thread-123 INFO LineBufferedStream: stdout: at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:343)
19/05/10 17:15:48 Thread-123 INFO LineBufferedStream: stdout: at
io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:911)
19/05/10 17:15:48 Thread-123 INFO LineBufferedStream: stdout: at
io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:131)
19/05/10 17:15:48 Thread-123 INFO LineBufferedStream: stdout: at
io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:643)
19/05/10 17:15:48 Thread-123 INFO LineBufferedStream: stdout: at
io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:566)
19/05/10 17:15:48 Thread-123 INFO LineBufferedStream: stdout: at
io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:480)
19/05/10 17:15:48 Thread-123 INFO LineBufferedStream: stdout: at
io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:442)
19/05/10 17:15:48 Thread-123 INFO LineBufferedStream: stdout: at
io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:131)
19/05/10 17:15:48 Thread-123 INFO LineBufferedStream: stdout: at
java.lang.Thread.run(Thread.java:748)
19/05/10 17:15:48 Thread-123 INFO LineBufferedStream: stdout: Caused by:
javax.security.sasl.SaslException: Problems unwrapping SASL buffer [Caused by
GSSException: Defective token detected (Mechanism level: Wrap Token (new
format):Cannot read all 12 bytes needed to form this token!)]
19/05/10 17:15:48 Thread-123 INFO LineBufferedStream: stdout: at
com.sun.security.sasl.gsskerb.GssKrb5Base.unwrap(GssKrb5Base.java:86)
19/05/10 17:15:48 Thread-123 INFO LineBufferedStream: stdout: at
org.apache.livy.rsc.rpc.Rpc$SaslClientHandler.unwrap(Rpc.java:481)
19/05/10 17:15:48 Thread-123 INFO LineBufferedStream: stdout: at
org.apache.livy.rsc.rpc.KryoMessageCodec.doWrapOrUnWrap(KryoMessageCodec.java:146)
19/05/10 17:15:48 Thread-123 INFO LineBufferedStream: stdout: at
org.apache.livy.rsc.rpc.KryoMessageCodec.maybeDecrypt(KryoMessageCodec.java:121)
19/05/10 17:15:48 Thread-123 INFO LineBufferedStream: stdout: at
org.apache.livy.rsc.rpc.KryoMessageCodec.decode(KryoMessageCodec.java:76)
19/05/10 17:15:48 Thread-123 INFO LineBufferedStream: stdout: at
io.netty.handler.codec.ByteToMessageCodec$1.decode(ByteToMessageCodec.java:42)
19/05/10 17:15:48 Thread-123 INFO LineBufferedStream: stdout: at
io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:411)
19/05/10 17:15:48 Thread-123 INFO LineBufferedStream: stdout: ... 24 more
19/05/10 17:15:48 Thread-123 INFO LineBufferedStream: stdout: Caused by:
GSSException: Defective token detected (Mechanism level: Wrap Token (new
format):Cannot read all 12 bytes needed to form this token!)
19/05/10 17:15:48 Thread-123 INFO LineBufferedStream: stdout: at
sun.security.jgss.krb5.MessageToken_v2.<init>(MessageToken_v2.java:258)
19/05/10 17:15:48 Thread-123 INFO LineBufferedStream: stdout: at
sun.security.jgss.krb5.MessageToken_v2.<init>(MessageToken_v2.java:165)
19/05/10 17:15:48 Thread-123 INFO LineBufferedStream: stdout: at
sun.security.jgss.krb5.WrapToken_v2.<init>(WrapToken_v2.java:71)
19/05/10 17:15:48 Thread-123 INFO LineBufferedStream: stdout: at
sun.security.jgss.krb5.Krb5Context.unwrap(Krb5Context.java:1056)
> Replace DEGEST-MED5 with GSSAPI(Kerberos) in the RPC sasl
> ---------------------------------------------------------
>
> Key: LIVY-595
> URL: https://issues.apache.org/jira/browse/LIVY-595
> Project: Livy
> Issue Type: Improvement
> Components: RSC, Server
> Affects Versions: 0.5.0
> Reporter: yanchao
> Priority: Blocker
> Original Estimate: 168h
> Remaining Estimate: 168h
>
> This is a English version
> DIGEST-MD5 has been considered as a non-secure encryption mechanism in the
> industry, so according to the company's security requirements, it is replaced
> by GSSAPI (kerberos authentication);
> Initially, I just changed the configuration value of livy. rsc. rpc. sasl.
> mechanisms to GSSAPI, but reported an error: Failed to find any Kerberos
> credentails; so I started my painful journey to modify the source code (thank
> you very much if you have a feasible configuration plan to inform). The
> specific steps are as follows:
>
> 1) In the Rpc and RpcServer classes, create LoginContext and login when
> creating client and server for sasl, and encapsulate Sasl. createSaslServer
> and Sasl. createSaslClient with Subject. doAs.
> 2) The parameters of Sasl. createSaslServer and Sasl. createSaslClient mainly
> change protocol to the user name of principal (i.e. the first paragraph of
> principal), and server Name to the qualified name of principal (i.e. the
> second paragraph of principal). Other parameters remain unchanged and login
> succeeds.
>
> Question: Client and server can communicate, the first sendHello can succeed,
> but the second time Livy returns token to driver, driver unwrap error:
> {color:#FF0000}Caused by GSSException: Defective token detection (Mechanism
> level: Wrap Token (new format): Cannot read all 12 bytes needed to form this
> token!){color}
>
> My analysis: I tracked livy's log. The byte array returned to driver is null
> and sent to driver by Chanel Rpc.SaslMessage object, when unwrap, \{data is
> [20, 1, 0, 0], offset is 0, len is 4}, driver unwrap will report an error.
>
>
> The problem is too difficult to solve, I need help now. thinks everyone.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
